Traditional Chinese medicine decoction piece recognition method and system based on spectral image and big data model

Abstract

The application discloses a traditional Chinese medicine decoction piece recognition method and system based on spectral images and big data models, and the method comprises the following steps: inputting a spectral vector into a pre-trained big data spectral classification model to obtain a candidate category and a confidence degree; mapping the spectral vector into an inferred trait set by using a spectral-trait correlation model, and performing semantic matching with a pre-constructed traditional Chinese medicine decoction piece trait knowledge graph to obtain a trait consistency score of each candidate category; synchronously collecting a visible light image of a to-be-tested decoction piece, verifying the texture and color features of a cross section, and obtaining a visual verification score; and performing multi-modal fusion decision-making. Through three-level progressive cooperation of hyperspectral big data coarse screening, knowledge graph semantic verification and visible light fine verification, the application realizes accurate and objective recognition of traditional Chinese medicine decoction pieces, especially easily confused products, the recognition process is traceable and interpretable, and a reliable scheme is provided for intelligent quality control of traditional Chinese medicine decoction pieces.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models. Background Technology

[0002] Quality control of prepared Chinese medicinal herbs is a crucial link in ensuring the clinical efficacy and safety of traditional Chinese medicine. Traditional identification of prepared Chinese medicinal herbs mainly relies on experienced appraisers who judge their characteristics through sensory means such as visual inspection, touch, smell, and taste. This method is highly subjective, difficult to quantify, and requires extremely high professional competence from the appraisers. With the development of spectroscopic analysis technology, non-destructive testing methods such as near-infrared spectroscopy and Raman spectroscopy have been explored for application in the identification of authenticity and quality evaluation of prepared Chinese medicinal herbs.

[0003] In recent years, hyperspectral imaging technology has received increasing attention in the field of traditional Chinese medicine (TCM) identification due to its ability to simultaneously acquire spatial and continuous spectral information of samples. For example, patent CN102288551A discloses a method for identifying TCM using hyperspectral data. This method establishes a standard hyperspectral database of TCM and compares the hyperspectral data of the TCM to be tested with the database for similarity to achieve identification. Furthermore, patent CN121577551A discloses a method for quality detection of TCM decoction pieces based on hyperspectral imaging. This method identifies abnormal bands by comparing the spectral curves of historical normal reflectance with those of moldy samples, thereby determining whether the TCM decoction pieces are moldy. Another patent, CN115979973B, discloses a hyperspectral TCM identification method based on a dual-channel compressed attention network. This method reduces hyperspectral data redundancy through a grouping and fusion module and utilizes a dual-channel compressed attention network to simultaneously learn spatial and spectral feature representations.

[0004] However, the existing technologies still have the following shortcomings: First, most methods rely solely on spectral data of a single modality for discrimination, failing to fully utilize the macroscopic morphological characteristics of Chinese herbal medicine slices (such as cross-sectional color, texture, powderiness, and fibrousness) for cross-validation, thus the accuracy and robustness of the identification results need to be improved; Second, most hyperspectral identification methods based on deep learning are "black box" models, lacking interpretability and making it difficult to provide discrimination criteria that conform to drug inspection standards; Third, existing methods do not effectively combine spectral information with the systematic morphological knowledge system in Chinese herbal medicine identification, resulting in limited ability to distinguish easily confused varieties such as Astragalus membranaceus and Astragalus sibiricum, and Dioscorea opposita and Pueraria lobata. Summary of the Invention

[0005] To overcome the problems mentioned in the background art, the present invention proposes a method and system for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models.

[0006] The technical solution of the present invention is as follows: On one hand, a method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models includes the following steps: S1: Acquire hyperspectral images of Chinese herbal medicine samples and generate a spectral reflectance data cube containing multiple bands based on the acquired data; S2: Based on a preset spectral preprocessing procedure, extract spectral data of the region of interest from the data cube to form the spectrum to be measured; S3: Use a pre-trained big data spectral classification model to classify the spectrum to be tested, and obtain a list of candidate types of Chinese herbal medicine slices corresponding to the sample and the classification confidence of each candidate type. S4: Use a pre-trained spectral-trait association model to infer the attributes of the spectrum to be tested, and generate an inferred trait set containing at least one spectral inferable trait attribute of the medicinal slices. S5: Input the inferred trait set as a query condition into the pre-constructed knowledge graph of Chinese herbal medicine trait to perform graph search, obtain the trait nodes matched by the inferred trait set and the authoritative standard trait information of each candidate trait, and calculate the trait consistency score of each candidate trait. S6: Determine at least one key verification trait based on the trait consistency score, acquire a visible light image of the sample, and extract directional visual features corresponding to the key verification trait from the visible light image to generate a visible light verification score for each key verification trait. S7: Combining the classification confidence score, the trait consistency score, and the visible light verification score, the final herbal slice identification result and overall confidence score of the sample are determined by a multimodal fusion decision model.

[0007] Preferably, the step of extracting spectral data of the region of interest from the data cube based on a preset spectral preprocessing procedure to form the spectrum to be measured specifically includes: The data cube is subjected to black-and-white correction and standard reference plate correction; Noise and optical path differences are eliminated using spectral smoothing and normalization algorithms; The cross-sectional area or specific texture area of ​​the sample is extracted as the region of interest, and the average spectrum of all pixels in the region is calculated as the spectrum to be measured.

[0008] Preferably, in the step of classifying the spectrum to be measured using a pre-trained big data spectral classification model, the big data spectral classification model is a deep learning model based on a one-dimensional convolutional neural network or a spectral Transformer, and its training sample library contains hyperspectral data collected from standard Chinese herbal medicine slices samples from different origins and batches, along with their corresponding slice type labels.

[0009] Preferably, in the step of using a pre-trained spectral-property association model to infer the attributes of the spectrum to be measured, the spectral-property association model is used to establish a mapping relationship between the spectral characteristics of a specific band and the macroscopic morphological attributes of the medicinal slices, and the morphological attributes include at least cross-sectional color and powdery characteristics.

[0010] Preferably, the knowledge graph of the properties of traditional Chinese medicine decoction pieces is pre-constructed in the following way: Extract descriptive statements about the properties of each medicinal slice from the texts of the Chinese Pharmacopoeia and the Identification of Traditional Chinese Medicine. Entity recognition and relation extraction are performed on the description statements of the characteristics to generate a graph structure with the name of the medicinal slice as the central node and texture, color, cross-sectional features, odor and taste as attribute nodes. The specific description values ​​corresponding to each attribute are stored as node attributes or text value nodes connected to the attribute nodes.

[0011] Preferably, the step of extracting directional visual features corresponding to the key verification traits from the visible light image and generating visible light verification scores for each key verification trait specifically includes: When the key verification trait involves the strength of fibrousness, texture analysis is performed on the visible light image to extract gray-level co-occurrence matrix features or local binary pattern features, and a pre-trained texture classifier is used to output the fibrousness matching probability. When the key verification trait involves color attributes, the color value of the corresponding region in the visible light image is calculated under the standard light source color space, and the color difference is compared with the color description in the authoritative standard trait information to generate a color matching score.

[0012] Preferably, the step of determining the final herbal slice identification result and overall confidence level of the sample by combining the classification confidence level, the trait consistency score, and the visible light verification score through a multimodal fusion decision model is as follows: Weighted voting or Bayesian networks are used to fuse multiple scores of each candidate herbal slice. The weight of each score is dynamically adjusted according to the accuracy of each modality in historical verification experiments. The candidate herbal slice with the highest weighted comprehensive score is selected as the final identification result.

[0013] Preferably, after determining the final herbal slice identification result, the method further includes: Generate an interpretability identification report, which includes the similarity between the spectrum to be tested and the standard spectrum, the comparison results between the inferred trait set and the authoritative standard trait information, and the texture or color evidence image on which the visible light verification score is based.

[0014] On the other hand, a traditional Chinese medicine decoction piece identification system based on spectral images and big data models includes: The hyperspectral image acquisition unit is used to acquire hyperspectral data cubes of traditional Chinese medicine decoction pieces samples; A spectral preprocessing and feature extraction unit is used to extract the spectrum to be measured from the data cube; The big data spectral classification unit has a built-in pre-trained big data spectral classification model, which is used to output a list of candidate categories and classification confidence. The spectral-trait association inference unit has a built-in pre-trained spectral-trait association model for generating inferred trait sets. The knowledge graph construction and search unit stores a pre-constructed knowledge graph of the characteristics of Chinese herbal medicine slices, and is used to perform graph search and calculate the characteristic consistency score based on the inferred characteristic set. The visible light image verification unit is used to acquire visible light images of the sample, extract directional visual features, and output a visible light verification score. The multimodal fusion decision unit is used to integrate the classification confidence, trait consistency score and visible light verification score to output the final recognition result.

[0015] Preferably, the visible light image verification unit includes a macro lens and a microscope lens, as well as a texture analysis module and a color analysis module. The texture analysis module is used to extract gray-level co-occurrence matrix or local binary mode features to evaluate fiber properties, and the color analysis module is used to calculate the cross-sectional color value in the CIELab color space and compare the color difference with the pharmacopoeia standard.

[0016] The beneficial effects of this invention are: 1. Traditional identification of Chinese herbal medicine slices often relies on single methods, such as simply taking pictures of the appearance or measuring only near-infrared spectra. This results in limited information dimensions and makes it difficult to comprehensively consider multiple characteristics such as shape, color, odor, taste, and texture. This invention innovatively constructs a three-level progressive identification chain: "hyperspectral big data model coarse screening—knowledge graph semantic matching—visible light image fine verification." Hyperspectral imaging captures the reflectivity of the slice cross-section in hundreds of continuous bands, sensitively reflecting subtle differences in internal chemical components (such as starch and fiber) and physical structures, performing probability ranking at the "microscopic spectral fingerprint" level. The pre-constructed knowledge graph of Chinese herbal medicine slice characteristics transforms descriptive experiences such as "firm texture, yellowish-white cross-section, and sufficient powderiness" from authoritative works like the Chinese Pharmacopoeia into computable structured rules, providing classic-level logical verification for hyperspectral physical inferences. The visible light image verification stage further performs targeted verification of macroscopic textures and precise colors that cannot be directly determined by spectroscopy, such as "fiber strength" and "whether the color truly matches yellowish-white." The three methods support and cross-validate each other, significantly reducing misjudgments caused by variations in origin, processing differences, or single-modal noise. This is particularly advantageous in identifying easily confused pharmacopoeia and non-pharmacopoeia products, such as Astragalus membranaceus and Astragalus sibiricum, Glycyrrhiza uralensis and Glycyrrhiza glabra, and Dioscorea opposita and Pueraria lobata.

[0017] 2. Many existing deep learning-based "black box" identification methods, while potentially accurate, fail to provide convincing reasons for drug inspectors and are difficult to pass regulatory review. This invention maps the qualitative description system in traditional Chinese medicine identification into triples and relationships in a knowledge graph, ensuring that every identification conclusion can be traced back to standard morphological clauses. When outputting results, the system simultaneously generates a detailed report including hyperspectral curve comparison charts, a list of item-by-item comparisons between inferred traits and pharmacopoeia standard traits, visible light texture and color verification measurements, and judgment thresholds. This serves as direct evidence for quality inspection and as a teaching tool for newcomers to understand the intrinsic relationship between the microscopic characteristics and macroscopic properties of medicinal slices, truly realizing the transformation of artificial intelligence identification from "unexplainable guessing" to "verifiable judgment."

[0018] 3. Traditional identification methods heavily rely on the complete shape, cross-sectional texture, tactile feel, and odor of medicinal slices. When the slices are powdery, fragmented, or their thickness does not meet tactile requirements, even experts struggle to make a judgment. The core of this invention's hyperspectral image acquisition lies in scanning local cross-sections, extracting chemical components and microscopic physical states from spectral signals at the tens of micrometer level. Combined with rules in a knowledge graph regarding attributes such as "cross-sectional features" and "texture," and micro-area texture analysis under a visible light microscope, it can still identify species and quality even when the macroscopic shape is lost. For example, for unknown powdery substances, as long as a smooth detection surface can be prepared, the system can quickly identify whether it contains abnormal starch as a counterfeit product through spectral analysis, and then confirm whether it meets the characteristic of "strong fibrousness" by combining microscopic fiber texture. This expands the application scenarios of automated identification, suitable for rapid sampling and anti-counterfeiting inspection of bulk goods before pharmaceutical raw materials are stored.

[0019] 4. This invention, through the knowledge graph construction step, precipitates the textual experience scattered in "Traditional Chinese Medicine Identification" and the processing standards of various provinces into a standardized rule base that can be queried and reasoned; through the spectral-property association model, it transforms sensory indicators such as "sufficient powderiness" and "yellowish-white color" into quantitative spectral indicators; through visible light texture analysis, it transforms tactile terms such as "strong fiberity" into texture feature vectors, which can replace manual 24 / 7 uninterrupted objective judgment. Attached Figure Description

[0020] Figure 1 The diagram shows the overall architecture of the Chinese herbal medicine slice identification system based on spectral images and big data models of the present invention. Figure 2 The diagram shows the overall flowchart of the method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models according to the present invention. Figure 3 The diagram shown is a schematic of a one-dimensional convolutional neural network structure for the big data spectral classification model in this invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0022] It should be understood that the technical solution of the present invention can be embodied in the form of methods and systems. In the following description, the present invention will be fully disclosed in the order of system architecture, detailed method flow, and typical application examples, so that those skilled in the art can reproduce it.

[0023] Example 1 like Figure 1 As shown, a traditional Chinese medicine (TCM) decoction piece recognition system based on spectral images and big data models is presented. This system includes a hyperspectral image acquisition unit, a spectral preprocessing and feature extraction unit, a big data spectral classification unit, a spectral-trait association inference unit, a knowledge graph construction and search unit, a visible light image verification unit, and a multimodal fusion decision unit. These units are connected via a data bus or network interface to collaboratively complete the recognition task.

[0024] A hyperspectral image acquisition unit is used to acquire hyperspectral data cubes of traditional Chinese medicine decoction pieces. In this embodiment, a pushbroom hyperspectral imager is used, with a spectral coverage range of 400nm to 2500nm, covering the visible and near-infrared bands, a spectral resolution better than 5nm, and a spatial resolution better than 0.1mm. During acquisition, the decoction piece to be tested is broken open along its cross-section or placed directly on the stage, and pushed at a constant speed by a precision displacement platform to obtain a three-dimensional data block containing two-dimensional spatial data and one-dimensional spectral data.

[0025] The spectral preprocessing and feature extraction unit performs operations such as correction, smoothing, and normalization on the data cube, and extracts the spectrum of the region of interest. This unit can be implemented using an embedded processor or computer program, and includes a black-and-white correction module, a spectral smoothing and normalization module, and a ROI selection module.

[0026] The big data spectral classification unit incorporates a pre-trained big data spectral classification model. This model receives pre-processed spectra of the target medicinal herbs and outputs a list of candidate medicinal herb types and the classification confidence score for each candidate. The classification model can utilize deep learning architectures such as one-dimensional convolutional neural networks (1D-CNN) or spectral transformers. The training database contains hyperspectral data of hundreds of standard samples of traditional Chinese medicinal herbs from multiple authentic producing areas and different harvesting years.

[0027] The spectral-trait association inference unit has a built-in pre-trained spectral-trait association model, which is used to map the spectrum to be measured to several macroscopic trait attributes of the medicinal slices, such as cross-sectional color and powderiness. This model can be implemented using partial least squares regression (PLSR), support vector regression (SVR), or shallow neural networks.

[0028] The knowledge graph construction and search unit stores a pre-constructed knowledge graph of the properties of Chinese medicinal herbs and provides graph search and reasoning functions. This knowledge graph consists of herb nodes, attribute nodes, and relation edges, with information extracted from authoritative sources such as the *Chinese Pharmacopoeia* and *Identification of Traditional Chinese Medicine*. The search unit receives the inferred trait set, performs a graph query, and returns matching herb nodes and corresponding standard trait information, while simultaneously calculating a trait consistency score.

[0029] The visible light image verification unit is used to acquire high-resolution visible light images of samples in specific scenarios and to perform directional visual feature extraction and analysis. This unit includes a macro lens, a ring LED light source (D65 standard light source), an industrial camera, and texture and color analysis modules. The texture analysis module extracts Gray-Level Co-occurrence Matrix (GLCM) or Local Binary Pattern (LBP) features to evaluate fiberity; the color analysis module calculates the color values ​​of the cross-sectional area in the CIELab color space and compares the color difference with standard colors in the knowledge graph.

[0030] The multimodal fusion decision unit receives scoring information from the big data spectral classification unit, the knowledge graph construction and search unit, and the visible light image verification unit. It makes a comprehensive decision using weighted voting or a Bayesian network and outputs the final herbal slice name, comprehensive confidence level, and interpretability report.

[0031] Those skilled in the art will understand that the above-described units can be implemented using computer software, dedicated hardware, or a combination of both. For example, deep learning models can be deployed on GPU servers, and knowledge graphs can be stored using the Neo4j graph database.

[0032] Example 2 A method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models. (See also: [link to relevant documentation]) Figure 2 The method specifically includes the following steps.

[0033] Step S1: Hyperspectral Image Acquisition and Initial Data Generation The sample of the Chinese herbal medicine to be tested is placed on the stage of the hyperspectral image acquisition unit 100. Based on the sample morphology, a method that exposes the cross-section is preferred to ensure the acquisition of spectral information reflecting its internal structure and chemical composition. A pushbroom imager continuously scans the sample, acquiring the spectral reflectance intensity values ​​of each pixel in hundreds of consecutive wavelength bands, ultimately forming a data cube of size M×N×B, where M and N are the number of rows and columns of spatial pixels, and B is the number of wavelength bands. In this embodiment, the number of wavelength bands B can be 256 or 512, covering the visible to near-infrared region.

[0034] To ensure data consistency and comparability, dark current data (with the light source turned off and the lens cap on) and standard white board data (high reflectivity standard diffuse reflector) were recorded simultaneously during the acquisition process.

[0035] Step S2: Spectral preprocessing and extraction of the target spectrum First, perform black-and-white correction on the original data cube using the formula: in, The original image signal, This is a dark current image. This is a standard whiteboard image. The corrected relative reflectance value eliminates the effects of uneven illumination and detector dark noise.

[0036] Subsequently, the Savitzky-Golay convolutional smoothing algorithm was used to smooth the spectral curve of each pixel, with a window width of 11 bands and a polynomial order of 2, to remove high-frequency random noise while preserving the main spectral features. After smoothing, the spectrum was vector normalized, i.e., the spectral vector of each pixel was divided by its L2 norm, to eliminate the optical path effect caused by uneven sample surface or small differences in illumination distance. Alternatively, standard normal variable (SNV) transformation or other alternative methods can be used.

[0037] Next, the region of interest (ROI) is extracted. For the cross-section of the medicinal slices, the user can interactively select a uniform region in the center of the cross-section that is free of bubbles and impurities through software, or the system can automatically identify the medicinal slice region using Otsu thresholding. The average spectrum of all pixels within the selected ROI is used as the representative spectral vector to be measured for that sample. This averaging operation further suppresses local heterogeneity, resulting in a higher signal-to-noise ratio and representativeness of the measured spectrum.

[0038] Step S3: Output candidate categories based on big data spectral classification model The big data spectral classification model is a classifier based on deep neural networks. This embodiment uses a one-dimensional convolutional neural network (1D-CNN) structure, such as... Figure 3 As shown, the network's input layer accepts a preprocessed B-dimensional spectral vector. This is followed by three convolutional modules, each containing a one-dimensional convolutional layer (kernel sizes of 7, 5, and 3, with increasing filter numbers of 64, 128, and 256), a batch normalization layer, and a ReLU activation function, with a max-pooling layer following each module. The convolutional layers automatically extract local spectral patterns, such as absorption features caused by specific chemical groups. The output of the last pooling layer is flattened and connected to a fully connected layer (128 neurons) and a Dropout layer (dropout rate of 0.5). Finally, a softmax layer outputs the posterior probability, or classification confidence, for each herbal slice category.

[0039] During the training phase, the model requires a pre-established standard spectral database. This database is constructed as follows: Samples of processed medicinal slices, verified by experts, are collected from multiple authentic producing regions (e.g., Astragalus membranaceus from Inner Mongolia and Gansu, Codonopsis pilosula from Gansu and Shanxi). For each batch, at least 50 typical individual slices are collected, and cross-sectional hyperspectral images are extracted according to step S2, then bound to the slice name. The database covers over 500 commonly used processed medicinal slices in the Chinese Pharmacopoeia, with at least 200 samples for each type to cover variations in origin, batch, and year. During training, the data is randomly divided into training, validation, and test sets (8:1:1 ratio), and training is performed using the cross-entropy loss function and the Adam optimizer until the validation accuracy converges. The final model achieves an overall Top-1 accuracy of over 95% on the test set.

[0040] When predicting the spectrum $x$ to be measured, the model outputs the probability vector of all L-type medicinal slices. The top N categories with the highest probabilities (N can be 5 or 10) are selected as the candidate categories, and their corresponding confidence levels are assigned. For example, the output for the test sample might be: Astragalus membranaceus (0.89), Astragalus sibiricum (0.07), Codonopsis pilosula (0.03), Glycyrrhiza uralensis (0.01), etc.

[0041] Step S4: Generate the inferred trait set by inferring the spectral-trait association. Although hyperspectral data directly reflects chemical composition and physical structure, this invention establishes a spectral-morphological correlation model to align with traditional morphological terms in knowledge graphs. For cross-sectional color attributes, the reflectance spectrum in the visible light band (400-780nm) can be converted into tristimulus values ​​XYZ using a color matching function, and then into CIELab color coordinates. Specifically, the spectral power distribution of the D65 standard light source is utilized. CIE1931 standard observer color matching function Calculate the tristimulus values: ； ； ; in, This invention refers to the visible light reflectance extracted from corrected hyperspectral data. These are the normalization coefficients. The CIELab value can be easily calculated from XYZ. Then, based on the traditional color description conventions of Chinese medicine, the Lab values ​​were mapped to qualitative descriptions such as "yellowish-white" and "brownish-yellow." For example, a discrimination rule was established based on a large number of labeled samples: when... , Between 15 and 30, and When the color is close to 0, it is "yellowish-white". The model can directly output qualitative color description values.

[0042] The "flouriness" characteristic is closely related to starch content, and starch exhibits characteristic absorption in the near-infrared region (e.g., moisture and starch-related absorption around 1400nm and 1900nm). This invention constructs a flouriness index prediction model based on partial least squares regression (PLSR). First, a PLSR model is established for the spectra and scores of a batch of samples with known flouriness grades (e.g., strong, medium, weak, or quantified into 1-5 points by expert sensory evaluation), and the optimal number of principal factors is selected. In use, the sample spectrum is input, and the model outputs a continuous flouriness index. A value greater than 0.7 is considered "sufficiently powdery" or "showing powderiness." Furthermore, this approach can also be used to infer other cross-sectional characteristics such as "keratinous" or "granular."

[0043] The spectral-trait association model can also encompass more attributes that can be directly calculated from spectral or hyperspectral images. For example, it can infer preliminary indicators of "fibrousness" based on hyperspectral image texture (such as gray-level co-occurrence matrix). However, this step is not the final determination, but only one of the inputs for knowledge graph search. After this step, an inferred trait set is generated, such as: {cross-section color = "yellowish-white", cross-section feature = "powdery"}.

[0044] Step S5: Knowledge Graph Search and Trait Consistency Score Calculation First, the construction process of the knowledge graph of the properties of traditional Chinese medicine decoction pieces is described. This graph uses decoction pieces as core nodes, associating various property attributes. During construction, the descriptive text under the "Properties" item of each traditional Chinese medicine decoction piece from textbooks such as the *Chinese Pharmacopoeia* (2020 edition) and *Traditional Chinese Medicine Identification* is used as the data source. Natural language processing techniques are used for entity and relation extraction. For example, for the text "Astragalus membranaceus: firm texture, yellowish-white cross-section, abundant powder, slightly fragrant odor, slightly sweet taste," the following triples are extracted using predefined domain regular expressions and the BERT-BiLSTM-CRF sequence labeling model: (Astragalus membranaceus, texture: firm) (Astragalus membranaceus, cross-section color: yellowish-white) (Astragalus membranaceus, cross-sectional characteristics, high powder content) (Astragalus, aroma, slightly fragrant) (Astragalus, taste, slightly sweet) Perform this operation for each medicinal slice, while simultaneously normalizing the attribute values, unifying "highly powdery," "showing obvious powderiness," and "strongly powdery" into different standard labels, but retaining the original text for reference. All triples are stored in the Neo4j graph database. To improve retrieval efficiency, indexes are created on attribute nodes and attribute values. In addition, reverse relationships are established, such as (yellowish-white, is → cross-sectional color, Astragalus), to facilitate reverse retrieval of medicinal slices based on characteristics.

[0045] Once the inferred trait set is obtained, the system automatically generates a Cypher query statement. For example: MATCH(p: slice)-[: cross-sectional color]->(c: color {name: 'yellowish-white'}),(p)-[: cross-sectional features]->(f: features {name: 'sufficiently powdery'}) RETURN p.name This query will return a list of slices that simultaneously satisfy these attributes. Simultaneously, the system extracts the complete standard trait information of these candidate slices in the spectral data from the Top-N of the hyperspectral classification candidate list. For example, the standard traits of candidate Astragalus membranaceus are obtained as follows: {Texture: firm, Cross-sectional color: yellowish-white, Cross-sectional features: highly fibrous and powdery, Odor: slightly fragrant, Taste: slightly sweet}.

[0046] The consistency score for traits is calculated as follows: For each candidate medicinal slice, the inferred trait set is compared one by one with its standard trait. Each matching attribute is assigned a score of 1, and a partial match is assigned 0.5. The total score can be set as a weighted sum. For example, cross-sectional color and cross-sectional features each have a weight of 1. If the inferred trait is "yellowish-white, with sufficient powderiness," while the standard is "yellowish-white, with strong fibrousness and powderiness," then the color matches (score 1). Although the cross-sectional features contain "powderyness," the inferred trait does not mention "strong fibrousness," but it can be considered a partial match (because the standard contains powderiness, and the inferred trait is sufficient powderiness, it can be considered a match and score 1 point, but it does not cover fibrousness, which can be verified through other means). In this embodiment, for greater refinement, the attributes are decomposed: if the inferred "sufficient powderiness" is semantically similar to "showing powderiness" in the standard, it is considered a match; however, for "strong fibrousness," which is not provided in the inferred set, it is additionally marked as a trait to be verified.

[0047] Ultimately, each candidate Obtain a trait consistency score The value ranges from 0 to 1. The higher the score, the more consistent the hyperspectral physical inferences are with the records in classical texts.

[0048] Step S6: Visible light image acquisition and orientation visual feature verification In some cases, spectral analysis alone cannot fully confirm certain key traits, especially the textural information implied in "strong fibrousness" and "firm texture," as well as colors requiring precise verification. Therefore, the system filters traits not directly covered by the spectrum or showing significant differences among candidate medicinal slices from trait consistency comparisons as key verification traits. For example, when distinguishing between Astragalus membranaceus and Astragalus rubra, both have yellowish-white cross-sections and are powdery, but Astragalus membranaceus has a more prominent "strong fibrousness," while the xylem of Astragalus rubra is paler and the fibrous description is slightly weaker. Therefore, the system automatically marks "strong fibrousness" as a key verification trait and triggers the visible light image verification unit.

[0049] Visible light image acquisition: Using an industrial camera equipped with a macro lens (magnification ratio of 1:1 to 5:1), high-resolution images are captured of the cross-section or specific area of ​​the medicinal slices under uniform illumination from a D65 standard ring light source to obtain clear images.

[0050] Directional visual feature extraction can be divided into two cases: (1) Validation for fibrous texture. The central region of the cross-section is selected from the visible light image as the sub-image for analysis and converted to grayscale. The gray-level co-occurrence matrix (GLCM) and distance parameters are calculated. Take one pixel and extract texture features such as contrast, energy, entropy, homogeneity, and correlation from four angles: 0°, 45°, 90°, and 135°, to form a feature vector. Simultaneously, Local Binary Pattern (LBP) features can also be extracted, such as using a uniform LBP pattern with a radius of 2 and 16 neighborhood points. These features are then input into a pre-trained classifier (such as a Support Vector Machine (SVM) or a lightweight neural network). The classifier training process involves collecting a large number of cross-sectional images of medicinal slices judged by experts as "strong fibrous," "medium fibrous," and "weak fibrous / predominantly powdery," extracting features in the same way, labeling them, and training the SVM classifier. For the test image, the classifier outputs the probability that it belongs to the "strong fibrous" category. .

[0051] (2) Precise verification of color attributes. Although hyperspectral imaging can calculate color, the visible light verification unit uses a high-resolution color camera that is more in line with human visual perception and standard evaluation, taking pictures under a standard D65 light source and performing precise color calibration. The image is converted from sRGB to CIELab space, and the cross-sectional area is extracted. The average value is then calculated. Next, the color difference is compared with the standard color value of the candidate medicinal slice stored in the knowledge graph (which can be obtained by consulting literature and averaging multiple batches of samples), using the CIEDE2000 color difference formula. If the color difference... If the color score is less than a preset threshold (e.g., 3.0), the colors are considered highly consistent, and the color verification score is [scoring value]. Set it to 1; if the color difference is between 3 and 6, set it to 0.8; if it is greater than 6, the score is low. This provides bimodal cross-validation for color attributes.

[0052] For each key validation trait, a corresponding image analysis was performed, generating a score between 0 and 1. These scores were denoted as... And calculate its weighted average or use it directly as input for the next stage.

[0053] Step S7: Multimodal fusion decision outputs the final recognition result. At this point, for each candidate medicinal slice We have the following scores: Confidence of big data spectral classification ; Trait Consistency Score ; The visible light verification score vector, averaged as denoted as .

[0054] The multimodal fusion decision unit employs a weighted voting strategy. The weight of each score is determined through offline experiments. For example, a batch of expert-approved test sets (containing common easily confused pairs) is selected, and each unit is run separately to calculate the accuracy of each modality's individual decision-making. Assuming the spectral classification accuracy is... The accuracy of trait consistency matching as a coarse screening method is [percentage missing]. The accuracy rate of visible light verification is The weights of each mode can then be set as follows: Alternatively, the normalized squared accuracy can be used. Bayesian networks can also be used to model the conditional dependencies between modes, but the weighted method is simpler and more effective.

[0055] The comprehensive score of candidate herbal medicine slice i is defined as follows: ,in, , , For adjustable weights, satisfying The emphasis on accuracy or interpretability can be adjusted according to the application scenario. A typical initial value can be set to... .

[0056] Calculate the score for all candidates and select the highest score as the final identification result. The overall confidence level can be defined as the highest score itself, or modulated by the difference in scores between candidates. If the highest score and the second-highest score are very close, a low confidence level is indicated, and manual review is recommended.

[0057] Step S8: Generate an interpretability identification report To meet the stringent requirements of the pharmaceutical industry for traceable and interpretable test results, the system automatically generates a visually appealing identification report after a decision is made. Example report content includes: Test summary: Sample number, test time, final identification result and overall confidence level.

[0058] Spectral comparison: Displays the superimposed curve of the spectrum to be tested and the standard spectrum of Astragalus membranaceus in the library, and marks the matching peaks of key wavelengths and gives the similarity value.

[0059] Trait inference and graph comparison: List the comparison results of the inferred trait set (yellowish-white cross-section, sufficient powderiness) with the standard trait entries of Astragalus membranaceus in the knowledge graph, and use "√" to indicate a match.

[0060] Visible light verification details: Displays cross-sectional macro images, overlaying texture analysis results (e.g., fiber orientation marked with lines) and Lab value measurement area bounding boxes on the images; provides a fiberity score of 0.87 and a color difference value. wait.

[0061] Conclusion: "Based on hyperspectral classification, semantic matching of knowledge graph, and verification of visible light image texture and color, the sample is identified as Astragalus membranaceus with a confidence level of 98.7%."

[0062] Such reports can provide quality inspectors with a direct chain of evidence and can also be used to train medicinal herb identification personnel, promoting the transmission of experience.

[0063] Specific Implementation: To make the technical solution and effects of the present invention clearer, the complete process is explained by taking the identification of a sample of medicinal slices (actually Astragalus membranaceus) as an example.

[0064] Sample collection and preprocessing: The whole medicinal slice was broken off, and a flat section was taken and placed under a hyperspectral imager. A data cube of 512×512×256 was obtained by scanning. After black and white correction, a 20×20 pixel region within the section was selected to calculate the average spectrum.

[0065] Hyperspectral classification: The spectrum to be measured is input into the 1D-CNN model, and the Top-5 candidates output are: Astragalus membranaceus (0.89), Astragalus rubra (0.07), Codonopsis pilosula (0.03), Astragalus membranaceus var. truncatum (0.006), etc.

[0066] Spectral-trait inference: The correlation model calculates the cross-sectional color from the visible light band. The cross-section was determined to be "yellowish-white"; the predicted value of the powder index was 0.91, which corresponds to "sufficient powderiness". The inferred trait set = {cross-section color: yellowish-white, cross-section characteristics: sufficient powderiness}.

[0067] Knowledge graph search: The query retrieves medicinal slices that simultaneously possess the characteristics of "yellowish-white" cross-section color and "high powderiness," returning eight types of medicinal slices including Astragalus membranaceus, Astragalus rubra, Paeonia lactiflora, and Dioscorea opposita. Combining hyperspectral candidate samples, Astragalus membranaceus and Astragalus rubra were given particular attention. The system retrieved the standard characteristics of Astragalus membranaceus: {Cross-section color: yellowish-white, cross-section characteristics: highly fibrous and powdery, texture: firm, ...}; Astragalus rubra: {Cross-section color: yellowish-white, cross-section characteristics: highly fibrous, powdery, bark yellowish-white, wood pale yellow, ...}. Characteristic consistency score: Astragalus membranaceus matched the inferred "high powderiness" with "powdery appearance," and the color matched, resulting in a consistency score of 0.85; Astragalus rubra scored similarly at 0.82. However, since the inferred set did not include "high fibrousness," it became a key verification trait.

[0068] Visible light verification: Macro-photographs of the cross-section image were taken, and the texture analysis module extracted GLCM features. The SVM classifier determined the probability of "strong fibrousness" to be 0.92. The color verification module measured the Lab value of the cross-section and compared it with the Astragalus standard color (…). )of The score is 1.65, far below the threshold of 3, which is acceptable. For Astragalus membranaceus, the woody part should be lighter in color, but the sample is uniform overall, so the color verification score for Astragalus membranaceus may be slightly lower. Final visible light verification scores: fiberity matching 0.92, color matching 0.98, overall $S_{cv}=0.95$.

[0069] Multimodal fusion: weights set to Astragalus score: Astragalus score: Clearly, Astragalus membranaceus scored significantly higher than Astragalus membranaceus, and the final determination was Astragalus membranaceus, with a proposed overall confidence level of 100%. That is, 96%.

[0070] Report output: The system generates a report containing all the above evidence, and the conclusion is that it is Astragalus membranaceus.

[0071] As can be seen from the above embodiments, this invention comprehensively utilizes the chemical / physical analysis capabilities of hyperspectral imaging, the prior rules of knowledge graphs, and the fine texture verification of visible light images to form a complete and interpretable automated identification scheme for traditional Chinese medicine (TCM) decoction pieces. Compared with single near-infrared spectroscopy identification or image identification, the method of this invention significantly improves accuracy, robustness, and the ability to identify counterfeit and substandard products. It can be used in the quality control process of TCM decoction pieces in pharmaceutical manufacturing enterprises and drug testing institutions, and has good application prospects.

[0072] It should be noted that those skilled in the art should understand that the network structure, threshold, weight, and other values ​​given in the above embodiments are all exemplary illustrations. In practical applications, they can be adjusted according to the characteristics and needs of the medicinal slices. These adjustments do not depart from the core idea of ​​the present invention and fall within the protection scope of the present invention.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models, characterized in that, Includes the following steps: S1: Acquire hyperspectral images of Chinese herbal medicine samples and generate a spectral reflectance data cube containing multiple bands based on the acquired data; S2: Based on a preset spectral preprocessing procedure, extract spectral data of the region of interest from the data cube to form the spectrum to be measured; S3: Use a pre-trained big data spectral classification model to classify the spectrum to be tested, and obtain a list of candidate types of Chinese herbal medicine slices corresponding to the sample and the classification confidence of each candidate type. S4: Use a pre-trained spectral-trait association model to infer the attributes of the spectrum to be tested, and generate an inferred trait set containing at least one spectral inferable trait attribute of the medicinal slices. S5: Input the inferred trait set as a query condition into the pre-constructed knowledge graph of Chinese herbal medicine trait to perform graph search, obtain the trait nodes matched by the inferred trait set and the authoritative standard trait information of each candidate trait, and calculate the trait consistency score of each candidate trait. S6: Determine at least one key verification trait based on the trait consistency score, acquire a visible light image of the sample, and extract directional visual features corresponding to the key verification trait from the visible light image to generate a visible light verification score for each key verification trait. S7: Combining the classification confidence score, the trait consistency score, and the visible light verification score, the final herbal slice identification result and overall confidence score of the sample are determined by a multimodal fusion decision model.

2. The method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models according to claim 1, characterized in that, The step of extracting spectral data of the region of interest from the data cube based on a preset spectral preprocessing procedure to form the spectrum to be measured specifically includes: The data cube is subjected to black-and-white correction and standard reference plate correction; Noise and optical path differences are eliminated using spectral smoothing and normalization algorithms; The cross-sectional area or specific texture area of ​​the sample is extracted as the region of interest, and the average spectrum of all pixels in the region is calculated as the spectrum to be measured.

3. The method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models according to claim 1, characterized in that, In the step of classifying the spectrum to be measured using a pre-trained big data spectral classification model, the big data spectral classification model is a deep learning model based on a one-dimensional convolutional neural network or a spectral Transformer. Its training sample library contains hyperspectral data collected from standard Chinese herbal medicine slices samples from different origins and batches, along with their corresponding slice type labels.

4. The method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models according to claim 1, characterized in that, In the step of using a pre-trained spectral-property association model to infer the attributes of the spectrum to be tested, the spectral-property association model is used to establish a mapping relationship between the spectral characteristics of a specific band and the macroscopic morphological attributes of the medicinal slices, and the morphological attributes include at least cross-sectional color and powdery characteristics.

5. The method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models according to claim 1, characterized in that, The knowledge graph of the properties of traditional Chinese medicine decoction pieces is pre-constructed in the following ways: Extract descriptive statements about the properties of each herbal slice from the text of standardized Chinese herbal medicine knowledge content; Entity recognition and relation extraction are performed on the description statements of the characteristics to generate a graph structure with the name of the medicinal slice as the central node and texture, color, cross-sectional features, odor and taste as attribute nodes. The specific description values ​​corresponding to each attribute are stored as node attributes or text value nodes connected to the attribute nodes.

6. The method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models according to claim 1, characterized in that, The step of extracting directional visual features corresponding to the key verification traits from the visible light image and generating visible light verification scores for each key verification trait specifically includes: When the key verification trait involves the strength of fibrousness, texture analysis is performed on the visible light image to extract gray-level co-occurrence matrix features or local binary pattern features, and a pre-trained texture classifier is used to output the fibrousness matching probability. When the key verification trait involves color attributes, the color value of the corresponding region in the visible light image is calculated under the standard light source color space, and the color difference is compared with the color description in the authoritative standard trait information to generate a color matching score.

7. The method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models according to claim 1, characterized in that, The step of determining the final herbal slice identification result and overall confidence level of the sample by combining the classification confidence level, the trait consistency score, and the visible light verification score through a multimodal fusion decision model is as follows: Weighted voting or Bayesian networks are used to fuse multiple scores of each candidate herbal slice. The weight of each score is dynamically adjusted according to the accuracy of each modality in historical verification experiments. The candidate herbal slice with the highest weighted comprehensive score is selected as the final identification result.

8. The method for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models according to claim 1, characterized in that, After determining the final herbal slice identification result, the method further includes: Generate an interpretability identification report, which includes the similarity between the spectrum to be tested and the standard spectrum, the comparison results between the inferred trait set and the authoritative standard trait information, and the texture or color evidence image on which the visible light verification score is based.

9. A system for identifying traditional Chinese medicine decoction pieces based on spectral images and big data models, used to implement the method as described in any one of claims 1 to 8, characterized in that, include: The hyperspectral image acquisition unit is used to acquire hyperspectral data cubes of traditional Chinese medicine decoction pieces samples; A spectral preprocessing and feature extraction unit is used to extract the spectrum to be measured from the data cube; The big data spectral classification unit has a built-in pre-trained big data spectral classification model, which is used to output a list of candidate categories and classification confidence. The spectral-trait association inference unit has a built-in pre-trained spectral-trait association model for generating inferred trait sets. The knowledge graph construction and search unit stores a pre-constructed knowledge graph of the characteristics of Chinese herbal medicine slices, and is used to perform graph search and calculate the characteristic consistency score based on the inferred characteristic set. The visible light image verification unit is used to acquire visible light images of the sample, extract directional visual features, and output a visible light verification score. The multimodal fusion decision unit is used to integrate the classification confidence, trait consistency score and visible light verification score to output the final recognition result.

10. The traditional Chinese medicine decoction piece identification system based on spectral images and big data models according to claim 9, characterized in that, The visible light image verification unit includes a macro lens and a microscope lens, as well as a texture analysis module and a color analysis module. The texture analysis module is used to extract gray-level co-occurrence matrix or local binary mode features to evaluate fiber properties, and the color analysis module is used to calculate the cross-sectional color value in the CIELab color space and compare the color difference with authoritative standard morphological information.

Images