An AI-based traditional Chinese medicine decoction piece automatic classification method and system

Abstract

The application relates to the technical field of tablet classification, and discloses an automatic Chinese medicinal decoction piece classification method and system based on AI, which comprises a multi-source data acquisition module, which acquires visible light images, multispectral images and microscopic images; a data processing registration module, which pre-processes and temporally and spatially registers the multi-source data and outputs standardized data; a feature extraction and fusion module, which extracts visible light, spectral and microscopic texture features and performs deep weighted fusion to obtain a standardized fusion feature vector; a multi-dimensional joint classification module, which outputs multi-dimensional classification results of base original varieties, authenticity, processing specifications and specification levels based on parallel multi-classification heads according to the fusion feature vector; and a sorting execution module, which performs sorting operation according to the multi-dimensional classification results. The application utilizes deep weighted fusion and multi-dimensional joint classification, improves the accuracy and fineness of Chinese medicinal decoction piece sorting, and realizes intelligent automatic grading of the comprehensive quality of the decoction pieces.

Description

Technical Field

[0001] This invention relates to the field of pharmaceutical tablet classification technology, and more specifically, to an AI-based automatic classification method and system for traditional Chinese medicine decoction pieces. Background Technology

[0002] Traditional Chinese medicine (TCM) decoction pieces are the core form of TCM clinical medication, and their quality directly affects clinical efficacy and medication safety. In the modern production process of TCM decoction pieces, sorting is a crucial step in quality control. Traditionally, the sorting of TCM decoction pieces relied mainly on manual experience, with professionals judging and classifying them by visually observing the shape and color of the pieces, combined with touch and smell. However, this manual sorting method has many inherent drawbacks: firstly, it is inefficient and cannot meet the needs of large-scale, continuous production; secondly, the classification results are highly subjective and easily affected by factors such as worker fatigue and experience differences, leading to poor consistency and a high false positive rate; finally, engaging in high-intensity, repetitive visual recognition work can easily damage workers' eyesight and physical and mental health.

[0003] In recent years, with the development of machine vision and artificial intelligence technologies, some automated sorting equipment based on image recognition has emerged. These existing technologies typically use only a single visible light camera to acquire images of the surface of medicinal herbs and use convolutional neural networks for simple category identification. However, such solutions have significant technical bottlenecks. The identification and classification of Chinese medicinal herbs is a highly complex task. Their authenticity, variety confirmation, processing specifications (such as raw and processed products), and grading depend not only on macroscopic shape, color, and texture, but also on essential attributes such as internal chemical composition and microscopic structure. A single visible light image cannot penetrate the surface to obtain the chemical fingerprint information and cellular-level microstructural characteristics of the herbs, resulting in a serious lack of accurate judgment on the authenticity, processing compliance, and internal quality grade of the herbs. For example, similar-looking adulterants and counterfeit products cannot be distinguished by optical appearance alone; and the degree of processing (such as the saturation of wine-processing and vinegar-processing) and internal quality grade of the herbs also require information support at the chemical and microscopic levels. In addition, existing solutions can usually only achieve a single-dimensional coarse classification (e.g., only distinguishing varieties), and cannot simultaneously output multi-dimensional fine classification results such as origin, authenticity, processing specifications, and grade, which makes it difficult to meet the requirements of modern Chinese medicine production for full-process and multi-dimensional quality control.

[0004] Therefore, there is an urgent need for an automatic classification system that can integrate multimodal and multidimensional information from Chinese herbal medicine slices to achieve accurate, efficient, and multidimensional classification. Summary of the Invention

[0005] In view of this, the present invention proposes an AI-based automatic classification method and system for traditional Chinese medicine decoction pieces, aiming to solve one or more of the above problems.

[0006] On the one hand, this invention proposes an AI-based automatic classification system for traditional Chinese medicine decoction pieces, the system comprising:

[0007] The multi-source data acquisition module is configured to acquire multi-source data of Chinese herbal medicine pieces to be sorted on the conveyor line. The multi-source data includes visible light images, multispectral images, and microscopic images.

[0008] The data processing and registration module is configured to preprocess and spatiotemporally register the multi-source data and output aligned standardized data.

[0009] The feature extraction and fusion module is configured to extract visible light features, spectral features, and microtexture features from the standardized data, and to perform depth-weighted fusion of the visible light features, spectral features, and microtexture features to obtain a standardized fusion feature vector.

[0010] The multidimensional joint classification module is configured to output multidimensional classification results based on the fused feature vector using a parallel multi-classification head. The multidimensional classification results include the original variety classification results, the authenticity classification results, the processing specification classification results, and the specification grade classification results.

[0011] The sorting execution module is configured to perform sorting operations on the Chinese herbal medicine pieces to be sorted based on the multidimensional classification results.

[0012] Preferably, the multi-source data acquisition module includes:

[0013] The visible light imaging unit is configured to use a high-resolution industrial line scan camera to acquire visible light images of the Chinese herbal medicine pieces to be sorted, the visible light images including the shape, color, texture, and size of the Chinese herbal medicine pieces to be sorted;

[0014] The multispectral imaging unit is configured to acquire multispectral images of the Chinese herbal medicine slices to be sorted in the 400nm–1700nm band using a multispectral imager.

[0015] The microscopic imaging unit is configured to acquire microscopic images of the Chinese herbal medicine slices to be sorted using a variable magnification stereomicroscopic imaging system. The microscopic images include the cell structure, tissue arrangement, and oil cell distribution on the surface of the slices.

[0016] The multispectral imaging unit and the microscopic camera unit are integrated into the same optical path through a coaxial common-path beam-splitting optical system, and the field of view overlap of the multispectral imaging unit and the microscopic camera unit is ≥95%.

[0017] Preferably, the data processing and registration module performs preprocessing and spatiotemporal registration on the multi-source data, including:

[0018] The visible light image and the microscopic image are subjected to denoising, contrast enhancement, sharpening and target region segmentation to obtain an image data stream;

[0019] The multispectral image is subjected to background subtraction, scattering correction, smoothing and noise reduction, and normalization to obtain a spectral data stream;

[0020] A hardware-triggered synchronization unit generates a synchronization pulse signal to align the image data stream and the spectral data stream along the time axis. A Kalman filter algorithm is then used to perform real-time spatial registration correction on the spectral data stream to compensate for spatial position deviations caused by the operation of the conveyor line.

[0021] Preferably, the feature extraction and fusion module includes:

[0022] The visible light feature extraction branch is configured to use a ResNet-50 variant network pre-trained on ImageNet, taking the normalized data corresponding to the visible light image as input and outputting a 2048-dimensional color-texture-shape feature vector;

[0023] The spectral feature extraction branch is configured to use a one-dimensional convolutional neural network combined with a multi-head self-attention mechanism, inputting the standardized data corresponding to the multispectral image and outputting a 1024-dimensional chemical fingerprint feature vector.

[0024] The microtexture feature extraction branch is configured to use an improved Transformer variant architecture, taking into input normalized data corresponding to the microscopic image and outputting a 2048-dimensional microtexture feature vector.

[0025] Preferably, the feature extraction and fusion module performs depth-weighted fusion of the visible light features, spectral features, and microtexture features to obtain a standardized fusion feature vector, including:

[0026] Visible light features, spectral features, and micro-texture features are linearly projected and mapped to a unified dimension, then stacked into a feature sequence.

[0027] The feature sequences are subjected to deep cross-modal interaction through a multi-head self-attention mechanism, so that the features of each modality are integrated with information from other modalities; after processing by residual connections and feedforward networks, an attention pooling mechanism is used to dynamically calculate the contribution weight for each modality.

[0028] The interaction features of each modality are weighted and summed according to the contribution weights, and then subjected to linear projection and layer normalization to output a standardized fusion feature vector.

[0029] Preferably, the parallel multi-classification head includes:

[0030] The original variety classifier is configured to receive the fused feature vector, process it sequentially through a set of fully connected layers for dimensionality reduction and hidden layers, and then connect to the original variety output layer to output the probability distribution of the original variety.

[0031] The authenticity classifier is configured to receive the fused feature vector, process it through two fully connected layers, output a binary classification probability value, and output a determination result of whether the product is genuine or counterfeit based on the comparison result of the binary classification probability value and a preset threshold.

[0032] A processing specification classifier is configured to receive the fused feature vector, first amplify the feature dimensions related to processing through an attention module, and then output the probability distribution of processing specifications through a fully connected layer, wherein the categories of processing specifications include raw products and roasted products.

[0033] A specification grade classifier is configured to receive the fused feature vector and, in conjunction with the size or thickness information of the medicinal slices extracted from the segmentation branch, output a probability distribution of the specification grades, which include general goods, first-level selected goods, and second-level selected goods.

[0034] Preferably, when the multidimensional joint classification module outputs the multidimensional classification result based on the fused feature vector using the parallel multi-classification head, it further includes:

[0035] The parallel multi-classification head is jointly optimized during training using a weighted multi-task loss function. The weighted multi-task loss function includes the cross-entropy loss of the original variety classifier, the binary classification loss of the true and false classifier, the cross-entropy loss of the processing specification classifier, and the cross-entropy loss of the specification grade classifier multiplied by a preset weighting coefficient α. The weighting coefficient α is used to adjust the training priority of the grade classification task relative to other tasks.

[0036] Preferably, the sorting execution module encodes the original variety classification results, authenticity classification results, processing specification classification results, and specification grade classification results into eight-bit graded labels, constructs a sorting decision tree, and assigns a corresponding sorting channel and robotic arm grasping strategy to each coding combination.

[0037] Preferably, it further includes: a knowledge base update module, connected to the multidimensional joint classification module, configured to include the corresponding sample into the incremental training set when the confidence level output by the multidimensional joint classification module is lower than a preset threshold, and to trigger incremental learning during idle periods and update the network parameters of the feature extraction fusion module and the multidimensional joint classification module.

[0038] Compared with existing technologies, the advantages of this invention are as follows: By setting up a multi-source data acquisition module, this invention simultaneously acquires visible light images, multispectral images, and microscopic images of the Chinese herbal medicine slices to be sorted. This technical approach enables the system to simultaneously acquire essential attribute information of the slices in three dimensions: macroscopic (shape, color), chemical (spectral information), and microscopic (cellular structure). Compared with existing technologies that rely solely on visible light images, this invention fundamentally solves the problem of a single information source. Furthermore, through a feature extraction and fusion module, the above three features are deeply weighted and fused, resulting in a standard fused feature vector that encompasses comprehensive attribute information of the slices. Therefore, this system can more accurately identify varieties, genuine and counterfeit products, and products with different processing methods that are similar in appearance but differ in chemical composition or microstructure, thereby greatly improving the accuracy and robustness of origin identification, authenticity judgment, and processing specification determination.

[0039] The data processing registration module in this invention, particularly its hardware-triggered synchronization unit generating synchronization pulse signals and combining them with a Kalman filter algorithm for real-time spatial registration correction, effectively solves the problem of multi-source image mismatch in dynamic conveyor line scenarios in existing technologies. This ensures a high degree of temporal and spatial alignment (field of view overlap ≥ 95%) between visible light, multispectral, and microscopic images from different imaging units. This allows subsequent feature extraction and fusion to be performed based on precise pixel-level or feature-level correspondences, avoiding feature confusion and classification errors caused by data misalignment. This provides a crucial guarantee for the stable and reliable operation of the entire system in industrial production line environments.

[0040] The feature extraction and fusion module of this invention does not simply concatenate three features, but rather uses a multi-head self-attention mechanism to perform cross-modal deep interaction on the projected feature sequence, enabling features from each modality to be fused with information from other modalities. An attention pooling mechanism is employed to dynamically calculate the contribution weight for each modality. The beneficial effect of this technique is that the system can intelligently adjust the fusion ratio of different feature sources based on the specific circumstances of each Chinese herbal medicine slice (for example, for slices with significant color differences, the weight of visible light features may be higher; for slices with similar chemical compositions, the weight of micro-texture features will adaptively increase), achieving optimal fused feature representation and thus obtaining classification performance that surpasses that of a single optimal modality.

[0041] This invention employs a multi-dimensional joint classification module, where four classifiers—origin variety, authenticity, processing specifications, and grade—work in parallel, all based on the same standardized fusion feature vector. This architecture enables the system to output detailed classification results across four dimensions in a single, end-to-end manner, significantly improving sorting efficiency and information richness. Furthermore, during the training phase, a weighted multi-task loss function is used for joint optimization, particularly by setting an independent weighting coefficient α for the grade classifier's loss. This effectively addresses the challenge of learning grade classification tasks due to their small granular differences. This allows the system to not only know "what" (variety), "whether" (authenticity), and "how" (processing) of the medicinal slices, but also accurately assess their "quality" (grade), achieving a leap from extensive to intensive processing and meeting the urgent need for intelligent and multi-dimensional quality control in modern Chinese medicine production.

[0042] On the other hand, this application also provides an AI-based automatic classification method for Chinese herbal medicine decoction pieces, applied to the aforementioned AI-based automatic classification system for Chinese herbal medicine decoction pieces, the method comprising:

[0043] Collect multi-source data of Chinese herbal medicine slices to be sorted on the conveyor line, including visible light images, multispectral images and microscopic images;

[0044] The multi-source data is preprocessed and spatiotemporally registered to output aligned and standardized data.

[0045] Visible light features, spectral features, and microtexture features are extracted from the standardized data, and the visible light features, spectral features, and microtexture features are then fused using a depth-weighted method to obtain a standardized fused feature vector.

[0046] Based on the parallel multi-classification head, multi-dimensional classification results are output according to the fused feature vector. The multi-dimensional classification results include the original variety classification results, the authenticity classification results, the processing specification classification results, and the specification grade classification results.

[0047] Based on the multidimensional classification results, the Chinese herbal medicine slices to be sorted are sorted.

[0048] It is understood that the system provided in this application has the same beneficial effects, which will not be elaborated here. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0050] Figure 1 A functional block diagram of an AI-based automatic classification system for traditional Chinese medicine decoction pieces provided in an embodiment of the present invention;

[0051] Figure 2 A flowchart illustrating an AI-based automatic classification method for traditional Chinese medicine decoction pieces, provided as an embodiment of the present invention. Detailed Implementation

[0052] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey its scope to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0053] In some embodiments of this application, see Figure 1 As shown, this embodiment provides an AI-based automatic classification system for traditional Chinese medicine decoction pieces. The system includes:

[0054] The multi-source data acquisition module is configured to acquire multi-source data of Chinese herbal medicine slices to be sorted on the conveyor line. The multi-source data includes visible light images, multispectral images, and microscopic images.

[0055] The data processing and registration module is configured to preprocess and spatiotemporally register the multi-source data and output aligned standardized data.

[0056] The feature extraction and fusion module is configured to extract visible light features, spectral features, and microtexture features from the standardized data, and to perform deep weighted fusion of the visible light features, spectral features, and microtexture features to obtain a standardized fusion feature vector.

[0057] The multidimensional joint classification module is configured to output multidimensional classification results based on the fused feature vector using a parallel multi-classification head. The multidimensional classification results include the original variety classification results, the authenticity classification results, the processing specification classification results, and the specification grade classification results.

[0058] The sorting execution module is configured to perform sorting operations on the Chinese herbal medicine pieces to be sorted based on the multidimensional classification results.

[0059] By acquiring three types of multi-source data—visible light images, multispectral images, and microscopic images—the classification and identification characteristics of Chinese herbal medicine slices can be comprehensively covered from multiple dimensions, including appearance, chemical composition attributes, and microstructure, avoiding the problems of missed detections and misjudgments caused by single-modal data. After preprocessing and spatiotemporal registration of the multi-source data, the three types of features are extracted and deeply weighted fusion is performed, which can effectively integrate feature information from different dimensions, reduce interference caused by feature redundancy and heterogeneity, and obtain more recognizable fused features. Based on parallel multi-classification heads, the classification results of four dimensions—origin variety, authenticity, processing specifications, and grade—are output simultaneously, eliminating the need to complete the classification task in stages and multiple times, which greatly improves the efficiency and accuracy of Chinese herbal medicine slice classification. It can adapt to the automation requirements of modern Chinese herbal medicine processing and sorting lines, solve the problems of manual sorting relying on experience, low efficiency, and inconsistent standards, and also make up for the shortcomings of insufficient accuracy of single machine vision classification methods.

[0060] In some embodiments of this application, the multi-source data acquisition module includes: a visible light imaging unit configured to acquire visible light images of the Chinese herbal medicine slices to be sorted using a high-resolution industrial line scan camera, the visible light images including the shape, color, texture, and size of the Chinese herbal medicine slices to be sorted; a multispectral imaging unit configured to acquire multispectral images of the Chinese herbal medicine slices to be sorted in the 400nm–1700nm wavelength range using a multispectral imager; and a microscopic imaging unit configured to acquire microscopic images of the Chinese herbal medicine slices to be sorted using a variable magnification stereomicroscopic imaging system, the microscopic images including the cell structure, tissue arrangement, and oil cell distribution on the surface of the slices; wherein the multispectral imaging unit and the microscopic imaging unit are integrated in the same optical path through a coaxial common-path beam-splitting optical system, and the field of view overlap of the multispectral imaging unit and the microscopic imaging unit is ≥95%.

[0061] Specifically, the visible light imaging unit acquires the overall morphological characteristics of the processed medicinal slices for preliminary variety and specification identification; the multispectral imaging unit acquires spectral information reflecting the internal chemical composition of the slices for authenticity identification and processing analysis; and the microscopic imaging unit acquires cellular-level structural information of the surface and near-surface of the slices for origin identification and quality rating. Simultaneously, a coaxial common-path beam-splitting optical system integrates the multispectral imaging unit and the microscopic imaging unit into the same optical path, ensuring a field-of-view overlap of no less than 95%. This aims to eliminate feature misalignment caused by field-of-view differences, ensuring that the acquired multispectral data and microscopic images strictly correspond to the same small region of the slice, thereby improving the accuracy of subsequent data registration and feature fusion.

[0062] Specifically, the visible light imaging unit uses a high-resolution industrial line scan camera as its core imaging device. As the herbal slices to be sorted move at a constant speed along the conveyor line, the line scan camera scans the surface of the slices line by line, ultimately stitching together a complete two-dimensional visible light image. This image clearly presents the shape, surface color, macroscopic texture, and geometric dimensions of the slices. This information is mainly used to extract macroscopic features such as shape, color, texture, and size of the slices, providing visual basis for preliminary identification of the original variety and classification of specifications. The multispectral imaging unit uses a multispectral imager, whose operating wavelength covers 400nm to 1700nm, encompassing the visible to near-infrared spectral region. Different chemical components (such as alkaloids, flavonoids, and glycosides) have characteristic absorption or reflection peaks at specific wavelengths; therefore, the acquired multispectral images can form a spectral "fingerprint" reflecting the internal chemical composition of the slices. In practice, the multispectral imager acquires images band by band using built-in spectroscopic devices (such as liquid crystal tunable filters or acousto-optic tunable filters), thereby obtaining the spectral response curve of each pixel in different bands. The microscopic imaging unit employs a variable magnification stereomicroscopic imaging system with adjustable magnification to adapt to the observation needs of medicinal slices of different particle sizes. The microscopic imaging unit is used to acquire images of the microstructure of the surface and near-surface of the medicinal slices, such as the morphology and arrangement of epidermal cells, the distribution characteristics of stomata and trichomes, the morphology and density of secretory tissues (such as oil cells and oil chambers), and the type and arrangement of cross-sectional vessels. These microscopic texture features play a crucial role in identifying adulterated products with similar origins, determining the degree of cell structure damage during processing, and assessing the fineness grade of the medicinal slices.

[0063] To achieve precise spatial correspondence between multispectral images and microscopic images, this invention integrates the multispectral imaging unit and the microscopic imaging unit into a single optical path using a coaxial, common-path beam-splitting optical system. Specifically, light reflected or transmitted from the surface of the slice first passes through a shared front objective lens and then enters a beam-splitting prism (or beam-splitting fiber bundle). The beam-splitting prism splits the incident light into two paths according to a predetermined ratio (e.g., 50:50): one path guides the photoelectric sensor of the multispectral imager to generate the multispectral image; the other path guides the photoelectric sensor of the variable-magnification stereomicroscopy imaging system to generate the microscopic image. Through precise optical design and mechanical adjustment, the focal planes of the two imaging units are conjugate, and their field of view centers coincide, ensuring that the final acquired multispectral image and microscopic image have a field of view overlap of 95% or higher.

[0064] In some embodiments of this application, the data processing registration module preprocesses and performs spatiotemporal registration on the multi-source data, including: performing denoising, contrast enhancement, sharpening, and target region segmentation on the visible light image and the microscopic image to obtain an image data stream; performing background subtraction, scattering correction, smoothing and noise reduction, and normalization on the multispectral image to obtain a spectral data stream; generating a synchronization pulse signal using a hardware-triggered synchronization unit to align the image data stream and the spectral data stream on the time axis, and using a Kalman filter algorithm to perform real-time spatial registration correction on the spectral data stream to compensate for spatial position deviations caused by the operation of the conveyor line.

[0065] Specifically, by enhancing and segmenting visible light and microscopic images, the effective target area of ​​the medicinal slices can be highlighted and background clutter can be suppressed; by performing scattering correction and normalization on multispectral images, the influence of light intensity variations and sample thickness differences on spectral reflectance can be eliminated; by achieving strict alignment of the time axis through hardware-triggered synchronization and by compensating for spatial position deviations in real time through the Kalman filter algorithm, it can be ensured that each frame of image data and the corresponding spectral data come from the same part of the same medicinal slice in both time and space, laying the foundation for deep interaction and fusion of cross-modal features.

[0066] Specifically, for the raw images output by the visible light imaging unit and the microscopic imaging unit, median filtering or Gaussian filtering algorithms are first used for noise reduction to eliminate sensor noise and ambient light interference generated during image acquisition. Then, contrast enhancement is performed using adaptive histogram equalization to make the texture and edges of the sliced ​​food area clearer. Based on this, the Laplacian operator or unsharpening mask technique is used to sharpen the image, enhancing the surface feature details of the sliced ​​food. Finally, threshold segmentation or a deep learning-based semantic segmentation model (such as U-Net) is used to separate the target area of ​​the sliced ​​food from the conveyor belt background, retaining only the pixel area of ​​the sliced ​​food itself, forming a clean image data stream. For the raw spectral data output by the multispectral imaging unit, background subtraction is first performed. This involves acquiring a reference background image when no samples are passing through and subtracting this background signal from each subsequent sample image to eliminate the influence of dark current and stray light from the environment. Next, standard normal variable transformation or multivariate scattering correction methods are used to correct the scattering of the spectral data, reducing interference from optical path changes caused by uneven surface texture and varying particle sizes of the sliced ​​food. Then, a Savitzky-Golay smoothing filter is used to smooth and reduce noise in the spectral curves, removing high-frequency random noise while preserving effective spectral features. Finally, normalization is performed, typically using max-min normalization or vector normalization, to ensure that different spectral data streams have uniform dimensions and numerical ranges, facilitating subsequent feature extraction. To achieve temporal synchronization between the image data stream and the spectral data stream, the system is equipped with a hardware trigger synchronization unit. The core of this unit is a high-precision pulse generator, which generates synchronization trigger signals based on the pulse signals from the conveyor encoder and sends them to the visible light camera, microscope camera, and multispectral imager, ensuring that all three trigger acquisition at the same time. For the small phase difference caused by sensor response delay, the system further employs a timestamp-based interpolation alignment algorithm to match each frame of data in the multispectral data stream with the visible light and microscope image frames at the nearest time point, achieving precise alignment on the time axis.

[0067] Due to vibrations, speed fluctuations, and slight rolling or slippage of the medicinal slices during the operation of the conveyor line, the spatial position of the same medicinal slice in multi-source images may shift. To address this issue, this invention employs a Kalman filter algorithm for real-time spatial registration and correction of the spectral data stream. Specifically, firstly, the centroid coordinates of the medicinal slice are extracted from the visible light image as a reference position; then, the imaging position of the corresponding medicinal slice in the multispectral image is used as the observation value; the Kalman filter dynamically estimates the optimal spatial offset in real time based on the predicted position value from the previous moment and the observed value at the current moment, and feeds this offset back to the image resampling module of the multispectral imaging unit for inverse translation and affine transformation correction of the spectral image. Through frame-by-frame iterative updates, the system can effectively compensate for sub-pixel-level spatial position deviations caused by the operation of the conveyor line, ensuring that the spatial overlap error between the visible light image, the microscopic image, and the multispectral image in the final output standardized data is less than one pixel, thus providing strictly aligned input data for the subsequent feature extraction and fusion module.

[0068] Specifically, the deviation compensation steps include: Setting the reference time axis: The trigger pulse of the visible light imaging unit (fastest frame rate) is used as the global time reference. The hardware trigger synchronization unit ensures that all sensors start acquisition under the same global pulse sequence. Estimating position offset: When a slice of food is detected by the visible light sensor (e.g., through grating or image segmentation), its position xvis(t0) is recorded; based on the conveyor belt speed v, the time t1 = t0 + Δd / v (Δd is the optical axis distance between sensors) is predicted to arrive in front of the multispectral sensor; when the multispectral sensor is actually triggered, the image within this time window is extracted, and the actual observation position zMS of the slice is found through a target matching algorithm; the difference between the observed value and the predicted value is calculated: δ = zMS − x^MS − (predicted position). Kalman filter update: The difference δ is used as the observation input to the Kalman filter; the filter outputs the optimal estimated state x^MS (most likely true position); simultaneously, the instantaneous speed correction of the conveyor belt is estimated online to reduce system error. Spatial coordinate remapping: Using the estimated optimal position x^MS, calculate the required translation vector (Δx, Δy) between two image frames; resample the multispectral image (or microscopic image) (e.g., translation transformation in affine transformation) to align the coordinates of the herbal slice in the image to the coordinates of the registered RGB image. Image fusion generation: After spatial registration, the same herbal slice occupies the same pixel area in the three modalities of the image, allowing for direct pixel-level or feature-level fusion.

[0069] In some embodiments of this application, the feature extraction and fusion module includes: a visible light feature extraction branch, configured to use a ResNet-50 variant network pre-trained on ImageNet, inputting standardized data corresponding to the visible light image and outputting a 2048-dimensional color-texture-shape feature vector; a spectral feature extraction branch, configured to use a one-dimensional convolutional neural network combined with a multi-head self-attention mechanism, inputting standardized data corresponding to the multispectral image and outputting a 1024-dimensional chemical fingerprint feature vector; and a microscopic texture feature extraction branch, configured to use an improved Transformer variant architecture, inputting standardized data corresponding to the microscopic image and outputting a 2048-dimensional microscopic texture feature vector.

[0070] Specifically, dedicated deep learning network architectures were designed for three different modalities of standardized data: visible light images, multispectral images, and microscopic images, to extract the most discriminative deep features for each modality. The visible light feature extraction branch aims to capture the macroscopic visual attributes of the medicinal slices (such as color distribution, surface texture contours, and geometry); the spectral feature extraction branch extracts continuous spectral fingerprints characterizing the internal chemical composition of the slices; and the microscopic texture feature extraction branch extracts cellular-level tissue structure features of the surface and near-surface of the slices. Through the collaborative work of these three branches, the original multi-source heterogeneous data can be mapped to a unified multimodal feature space, providing information-rich and highly representative feature vectors for subsequent cross-modal deep weighted fusion.

[0071] Specifically, the visible light feature extraction branch uses a ResNet-50 variant network pre-trained on the ImageNet large image dataset as its backbone. The input is the standardized data corresponding to the visible light images received by this branch (visible light images after preprocessing and spatiotemporal registration). The ResNet-50 variant network has undergone certain structural adjustments based on the original ResNet-50, such as removing the original classification layer and adjusting the output dimension of the global average pooling layer to 2048. The pre-trained weights enable the network to have good general image feature extraction capabilities, and then fine-tuning is performed in this system to adapt it to the visible light image features of traditional Chinese medicine decoction pieces. After the input image is processed by convolutional layers, residual blocks, and global average pooling, a 2048-dimensional feature vector is finally output. This vector comprehensively encodes visual information such as the color distribution, macroscopic texture direction, and outer contour shape of the traditional Chinese medicine decoction pieces to be sorted, denoted as the color-texture-shape feature vector. The spectral feature extraction branch adopts a hybrid architecture of a one-dimensional convolutional neural network combined with a multi-head self-attention mechanism. The input is the standardized data corresponding to the multispectral images received by this branch, specifically the spectral curve corresponding to each target region of the medicinal slice, i.e., the numerical sequence of reflectance or absorbance at each wavelength in the 400nm to 1700nm band for each sampling point. A one-dimensional convolutional neural network (CNN) is composed of multiple stacked one-dimensional convolutional and pooling layers, used to extract spectral feature peaks and trends within a local band. Based on the feature map output by the one-dimensional CNN, a multi-head self-attention module is superimposed. This module effectively captures global spectral correlation information and enhances the characterization of weak chemical signals by modeling the dependencies between different band positions. After global average pooling, this branch outputs a 1024-dimensional feature vector, representing the unique chemical composition combination information of the medicinal slice, denoted as the chemical fingerprint feature vector. The microscopic texture feature extraction branch adopts an improved Transformer variant architecture. The input is the standardized data corresponding to the microscopic images received by this branch (enhanced and registered microscopic images). Because microscopic images contain rich cellular-level texture details, traditional convolutional networks have limitations in receptive field and long-range dependency modeling. Therefore, this branch adopts a Transformer variant based on a self-attention mechanism. Specifically, the microscopic image is first divided into multiple fixed-size image patches, and each image patch is linearly projected into a sequence embedding vector, while adding positional encoding to preserve spatial layout information. Subsequently, this sequence is fed into multiple stacked Transformer encoder layers, each containing a multi-head self-attention module and a feedforward network. After multiple layers of encoding, the average of all sequence vectors or the output corresponding to the classification token is taken, and mapped to a 2048-dimensional feature vector through a fully connected layer. This vector contains microscopic structural information such as cell morphology, tissue arrangement, and oil cell distribution density on the surface of the medicinal slices, and is denoted as the microtexture feature vector.

[0072] The microtexture feature extraction network adopts the SwinTransformer based on the shift window attention mechanism as its basic architecture, and makes the following improvements on this basis: (1) adopts a pyramid-style multi-scale patch embedding strategy to extract multi-scale features with patch sizes of 8×8, 16×16 and 32×32 pixels respectively; (2) inserts a depthwise separable convolutional layer after the self-attention layer to enhance the local texture modeling capability; (3) adopts a learnable two-dimensional relative position bias to replace the absolute position encoding to adapt to microscopic image inputs with different aspect ratios; (4) adds an auxiliary extraction branch of the second-order statistics of the gray-level co-occurrence matrix in the middle layer of the network to explicitly capture the arrangement pattern features of the organization.

[0073] In some embodiments of this application, the feature extraction and fusion module performs deep weighted fusion of the visible light features, spectral features, and microtexture features to obtain a standardized fusion feature vector. This includes: mapping the visible light features, spectral features, and microtexture features to a unified dimension via linear projection and then stacking them into a feature sequence; performing cross-modal deep interaction on the feature sequence using a multi-head self-attention mechanism, so that the features of each modality are fused with information from other modalities; after processing with residual connections and a feedforward network, dynamically calculating the contribution weight for each modality using an attention pooling mechanism; weighting and summing the interactive features of each modality according to the contribution weight, and then performing linear projection and layer normalization processing to output a standardized fusion feature vector.

[0074] Specifically, the main purpose of the deep weighted fusion process in the feature extraction and fusion module is to organically integrate the visible light features, spectral features, and micro-texture features extracted by the three parallel branches. This allows features from different modalities to fully interact and complement each other, and dynamically determines the contribution of each modal feature based on the actual situation of each specific sample. The final output is a fused feature vector that is complete in information, has low redundancy, and possesses unified representational capabilities. This process solves the problems of traditional feature concatenation methods, such as the inability to achieve cross-modal information interaction and the inability of fixed feature weights to adapt to sample differences. It provides more discriminative and robust feature inputs for the subsequent multi-dimensional joint classification module.

[0075] Specifically, firstly, the 2048-dimensional color-texture-shape feature vector obtained from the visible light branch, the 1024-dimensional chemical fingerprint feature vector obtained from the spectral branch, and the 2048-dimensional microtexture feature vector obtained from the microtexture branch are input into three independent linear projection layers. Each linear projection layer is a fully connected layer, and its output dimension is uniformly set to the same dimension (e.g., 512 dimensions) to map the three feature vectors, which originally had different dimensions, to a common feature space. After projection, the three projected feature vectors are stacked in sequence dimension to form a feature sequence of length 3, with each element being a 512-dimensional vector. Each position in this sequence represents a compressed feature representation of a modality. The stacked feature sequence is then input into a multi-head self-attention module. In this module, each modal feature in the sequence interacts with the features of the other two modalities through query, key, and value calculation. Specifically, for the visible light modality, its output features not only rely on its original features but also incorporate relevant information from the spectral and micro-texture modalities; similarly, the features of the spectral and micro-texture modalities also incorporate information from the other two modalities. Through a multi-head mechanism, the module can capture complex correlations between modalities from multiple different subspaces. This process ensures that the output features of each modality ultimately become the result of cross-modal fusion. The output of the multi-head self-attention module passes through a residual connection, i.e., it is added to the original stacked feature sequence to alleviate the gradient vanishing problem in deep networks and retain some original modal information. Subsequently, the result after the residual connection is input into a feedforward network (usually composed of two fully connected layers and an activation function) to further perform a nonlinear transformation on the fused features. Another residual connection is added after this feedforward network, finally obtaining a feature sequence that has undergone sufficient interaction and transformation. To adaptively determine the contribution of different modalities to the current classification task, this module adopts an attention pooling mechanism. Specifically, each modal feature vector in the feature sequence after cross-modal interaction and feedforward processing is input into a small attention scoring network. The network outputs a scalar score representing the importance of the modality feature in the current sample. Then, softmax normalization is applied to all modality scores to obtain a dynamic contribution weight for each modality. Since this weight is calculated in real-time based on the input samples, the modality weights can differ for different herbal sample samples. For example, for samples with significant color differences, the weight of the visible light modality will automatically increase; for samples with similar chemical compositions but significant differences in cell structure, the weight of the microtexture modality will increase accordingly. The interaction feature vectors of each modality are multiplied by their corresponding contribution weights, and then a weighted sum is performed across all modalities to obtain a single fused feature vector. This weighted summation process is essentially an adaptive aggregation of modal information based on the characteristics of the samples themselves.Finally, the fused feature vector is passed sequentially through a linear projection layer (fully connected layer) and a layer normalization module to adjust the numerical distribution of the features and accelerate the convergence of the subsequent classifier. After the above processing, a standardized fused feature vector is finally output, which simultaneously contains complementary information from macroscopic, chemical, and microscopic modalities, and the contribution of each modality is optimized according to the specific sample.

[0076] In some embodiments of this application, the parallel multi-classification head includes: a primary variety classifier, configured to receive the fused feature vector, process it sequentially through a fully connected layer set for dimensionality reduction and a hidden layer, and then connect to the primary variety output layer to output the probability distribution of the primary variety; a genuine / counterfeit classifier, configured to receive the fused feature vector, process it through two fully connected layers to output a binary classification probability value, and output a determination result of genuine or counterfeit products based on the comparison result of the binary classification probability value with a preset threshold; a processing specification classifier, configured to receive the fused feature vector, first amplify the feature dimensions related to processing through an attention module, and then output the probability distribution of processing specifications through a fully connected layer, wherein the categories of processing specifications include raw products and roasted products; and a specification grade classifier, configured to receive the fused feature vector and, in combination with the size or thickness information of the processed slices extracted by the segmentation branch, output the probability distribution of the specification grade, wherein the specification grade includes general goods, first-level selected goods, and second-level selected goods.

[0077] Specifically, the main purpose of setting up the parallel multi-classification heads is to simultaneously output the classification results of Chinese herbal medicine slices in four dimensions: origin variety, authenticity, processing specifications, and grade, based on the same standardized fusion feature vector. Each classifier has a different network structure designed for its specific task: the origin variety classifier focuses on multi-level feature dimensionality reduction and discrimination; the authenticity classifier uses a simple binary classification structure and introduces a threshold to control the false alarm rate; the processing specifications classifier introduces an attention module to enhance the ability to select processing-related features; and the grade classifier additionally integrates slice size or thickness information to improve the accuracy of fine-grained grade discrimination. Through the above differentiated classification head design, the system can efficiently and accurately complete multi-dimensional joint classification tasks, meeting the needs of comprehensive quality control of Chinese herbal medicine slices.

[0078] Specifically, the origin variety classifier is used to identify the biological origin and variety category of Chinese medicinal herbs (e.g., different sources of Astragalus membranaceus, different varieties of Bupleurum chinense, etc.). Its network structure is as follows: First, the input fused feature vector is fed into a set of fully connected layers, which can consist of multiple cascaded fully connected layers. This gradually reduces the feature vector from its original dimension (e.g., 512 dimensions) to an intermediate dimension (e.g., 256 dimensions) to extract the high-order features most relevant to variety identification. Subsequently, a non-linear transformation is performed through one or more hidden layers (usually combined with ReLU activation function and Dropout layer). Finally, an origin variety output layer is connected. This output layer is a fully connected layer with the number of neurons equal to the total number of origin variety categories in the training set, and uses the Softmax activation function to output the probability distribution of each variety category. During classification, the category corresponding to the maximum probability is taken as the origin variety classification result for that medicinal herb.

[0079] The authenticity classifier is used to determine whether the medicinal slices to be sorted are genuine or counterfeit. Considering that authenticity determination is essentially a binary classification task, its network structure is designed to be relatively simple: the input standardized fusion feature vector passes through two fully connected layers sequentially. The first fully connected layer can be set with an appropriate output dimension (e.g., 128 dimensions) and is equipped with an activation function. The second fully connected layer outputs a two-dimensional vector (or a one-dimensional scalar, which is transformed by the Sigmoid function to obtain the probability value of being genuine). The final output is a binary probability value, representing the probability that the medicinal slice belongs to the genuine product. To facilitate subsequent sorting, the classifier compares this probability value with a preset threshold (e.g., 0.5, which can also be adjusted according to the actual application scenario): if the probability value is greater than or equal to the threshold, the output is "genuine"; if the probability value is less than the threshold, the output is "counterfeit".

[0080] The processing specification classifier is used to distinguish the processing specifications of Chinese medicinal herbs, mainly including raw herbs (unprocessed) and processed herbs (processed by wine, vinegar, honey, etc.). Since the processing method has specific effects on the chemical composition and microstructure of the herbs, and these effects may be reflected in certain dimensions of the fused feature vector, this classifier first inputs the fused feature vector into an attention module. The attention module can specifically employ channel attention (such as the SENet module) or spatial attention structure to adaptively amplify feature dimensions relevant to processing status discrimination and suppress irrelevant or interfering features. The weighted feature vector after passing through the attention module is then fed into one or more fully connected layers, and finally connected to a binary classification output layer (raw / processed), using a Softmax function to output the probability distribution of the two processing specifications.

[0081] A specification grading classifier is used to assess the commodity specification grades of Chinese herbal medicine slices, including but not limited to general grade, first-grade selected grade, and second-grade selected grade. Grading assessment relies not only on macroscopic and microscopic texture features but also on the geometric dimensions (e.g., length, width) or thickness of the slices. Therefore, the classifier's network structure is specifically designed: in addition to receiving the standardized fused feature vector, it incorporates slice size or thickness information extracted by a segmentation branch. Specifically, the segmentation branch can be the bounding rectangle size, area, or perimeter of the slice calculated after instance segmentation of a visible light image, or it can be the slice thickness estimate obtained after microscopic image analysis. This size / thickness information is mapped into a low-dimensional vector through a separate encoding layer (e.g., a fully connected layer) and then concatenated with the fused feature vector. The concatenated joint feature vector is then fed into multiple fully connected layers for depth transformation, and finally connected to a multi-classification output layer to output the probability distribution of specification grades such as general grade, first-grade selected grade, and second-grade selected grade. During classification, the grade corresponding to the maximum probability is taken as the output result.

[0082] In some embodiments of this application, when the multidimensional joint classification module outputs multidimensional classification results based on the fused feature vector using the parallel multi-classification head, it further includes: the parallel multi-classification head performs joint optimization using a weighted multi-task loss function during training; the weighted multi-task loss function includes the sum of the cross-entropy loss of the original variety classifier, the binary classification loss of the true / false classifier, the cross-entropy loss of the processing specification classifier, and the cross-entropy loss of the specification grade classifier multiplied by a preset weighting coefficient α; the weighting coefficient α is used to adjust the training priority of the grade classification task relative to other tasks.

[0083] Specifically, a weighted multi-task loss function is introduced for joint optimization during the training of the parallel multi-classification head. This allows the four tasks—origin variety classification, authenticity classification, processing specification classification, and specification grade classification—to learn synchronously and mutually reinforce each other within the same deep learning model. Since the four tasks share the underlying feature extraction and fusion module, joint optimization forces the fused feature vectors to simultaneously possess the discriminative power required for all classification tasks, avoiding feature bias caused by single-task training. Furthermore, considering that specification grade classification is a fine-grained classification task with small differences between categories and high classification difficulty, making it susceptible to suppression by other tasks during joint training, a weighting coefficient α is specifically set to increase the training priority of the grade classification task. This ensures that this task receives sufficient gradient update contributions, thereby balancing the learning effects of each task and improving the overall accuracy and robustness of multi-dimensional classification.

[0084] In some embodiments of this application, the sorting execution module encodes the original variety classification results, authenticity classification results, processing specification classification results, and specification grade classification results into eight-bit graded labels, constructs a sorting decision tree, and assigns a corresponding sorting channel and robotic arm grasping strategy to each coding combination.

[0085] Specifically, the sorting execution module encodes the multidimensional classification results into graded labels and constructs a sorting decision tree. This involves comprehensively encoding the classification results across four dimensions—origin variety, authenticity, processing specifications, and grade—to form a unified sorting instruction system. Since the total number of combinations of multidimensional classification results is large (e.g., several varieties multiplied by two authenticity states, two processing specifications, and three grades), processing them one by one using simple conditional judgments would lead to complex and difficult-to-maintain sorting logic. By constructing eight-bit graded labels and establishing a sorting decision tree, the multidimensional classification results can be mapped to a limited number of standard coded combinations. Each combination can be pre-specified with corresponding sorting channels and robotic arm grasping strategies, thereby achieving refined and automated sorting of different categories of medicinal slices, avoiding mixing and missorting, and improving the efficiency and reliability of the sorting execution process.

[0086] Specifically, the system first receives four classification results from the multidimensional joint classification module: original variety classification result (assuming K possible varieties), authenticity classification result (genuine or counterfeit), processing specification classification result (raw or roasted), and grade classification result (general goods, first-level selected goods, or second-level selected goods). To facilitate computer processing and decision tree traversal, the sorting execution module encodes these four results into an eight-bit binary hierarchical label. The specific encoding rules are as follows: Bits 1-3: used to represent the original variety category, up to 8 different varieties can be encoded. Bit 4: used to represent the authenticity status, for example, 0 represents counterfeit goods, 1 represents genuine goods. Bits 5-6: used to represent the processing specification, for example, 00 represents raw goods, 01 represents wine-roasted goods, 10 represents vinegar-roasted goods, and 11 represents honey-roasted goods. Bits 7-8: used to represent the grade, for example, 00 represents general goods, 01 represents first-level selected goods, and 10 represents second-level selected goods. Through the above encoding method, each combination of multidimensional classification results corresponds to a unique eight-bit binary string, i.e., an eight-bit hierarchical label. The sorting execution module pre-constructs a sorting decision tree based on the actual sorting channel layout and robotic arm capabilities of the production workshop. This decision tree takes the eight-bit graded label as input and outputs the sorting channel number and the robotic arm grasping strategy. The root node of the decision tree receives the eight-bit graded label and branches layer by layer according to the priority order of each bit in the label (usually from high to low bit, e.g., first determine authenticity, then variety, then processing specifications, and finally grade). Each internal node corresponds to a value judgment on a bit (e.g., is the 4th bit 1?), and each leaf node corresponds to a unique sorting channel number and a set of robotic arm grasping strategies. The construction of the decision tree can be completed automatically by the system: traversing all possible eight-bit label combinations, and assigning sorting channels and grasping strategies to each leaf node according to preset business rules (e.g., all counterfeit products, regardless of variety, are sent to the waste channel; genuine products are sent to different packaging channels according to variety and grade). For each coding combination (i.e., each leaf node), the system specifies a specific sorting channel and robotic arm grasping strategy. The sorting channels guide medicinal slices into different downstream paths, such as: a primary sorting channel for genuine products, a secondary sorting channel for genuine products, a discard channel for counterfeit products, and a channel requiring manual re-inspection. The robotic arm's gripping strategy includes, but is not limited to, parameters such as the activation / deactivation of suction cups or grippers, the movement speed during gripping, the coordinates of the target placement location, and whether secondary re-inspection is required. By customizing different gripping strategies for each code combination, the sorting execution module can adapt to the physical characteristics (such as brittleness, weight, and surface smoothness) and quality requirements of different categories of medicinal slices.In actual operation, the sorting execution module receives the multidimensional classification results output by the multidimensional joint classification module in real time, encodes them into eight-bit grading labels, and then quickly traverses the pre-built sorting decision tree, locating the corresponding leaf node within milliseconds. It then reads the sorting channel and robotic arm grasping strategy specified by that node and sends the corresponding control commands to the robotic arm controller and sorting channel switching mechanism to complete the automatic sorting operation for the current herbal slices. This encoding-decision tree mechanism makes the sorting logic clear, the execution efficiency high, and it is easy to expand: when a new variety or grade needs to be added, only the allocation of the encoding bits needs to be adjusted and the relevant nodes of the decision tree updated.

[0087] In some embodiments of this application, the system further includes: a knowledge base update module connected to the multidimensional joint classification module, configured to include the corresponding sample into the incremental training set when the confidence level output by the multidimensional joint classification module is lower than a preset threshold, and to trigger incremental learning during idle periods and update the network parameters of the feature extraction fusion module and the multidimensional joint classification module.

[0088] Specifically, the main purpose of setting up the knowledge base update module is to enable the system to continuously learn and self-optimize, allowing it to identify low-confidence samples that the current model struggles to accurately classify online, and collect these samples as valuable high-value training data. When the system is idle, the module automatically triggers incremental learning, using newly collected samples to fine-tune and update the network parameters of the feature extraction and fusion module and the multi-dimensional joint classification module. This mechanism solves the problems of traditional deep learning models being unable to adapt to changes in data distribution after deployment and lacking the ability to handle newly emerging medicinal herb varieties or variant samples. It can continuously improve the system's classification accuracy and generalization ability without interrupting normal production and sorting, effectively addressing the concept drift phenomenon that may occur during long-term operation.

[0089] On the other hand, see Figure 2 As shown, this application also provides an AI-based automatic classification method for Chinese herbal medicine decoction pieces, applied to the aforementioned AI-based automatic classification system for Chinese herbal medicine decoction pieces. The method includes:

[0090] Collect multi-source data of Chinese herbal medicine slices to be sorted on the conveyor line. The multi-source data includes visible light images, multispectral images and microscopic images.

[0091] The multi-source data is preprocessed and spatiotemporally registered to output aligned and standardized data.

[0092] Visible light features, spectral features, and microtexture features are extracted from the standardized data, and the visible light features, spectral features, and microtexture features are then deeply weighted and fused to obtain a standardized fused feature vector.

[0093] Based on the parallel multi-classification head, multi-dimensional classification results are output according to the fused feature vector. The multi-dimensional classification results include the original variety classification results, the authenticity classification results, the processing specification classification results, and the specification grade classification results.

[0094] Based on the multidimensional classification results, the Chinese herbal medicine slices to be sorted are sorted.

[0095] By collecting and fusing multi-source data to extract features, it can combine multi-dimensional information such as visible light, spectrum, and micro-texture to avoid classification errors caused by insufficient information from a single image, thereby improving the accuracy of traditional Chinese medicine decoction piece classification. At the same time, the parallel multi-classification head can output classification results from multiple dimensions at once, eliminating the need for multiple classification tests in steps, which greatly improves the efficiency of automatic classification and can meet the actual needs of large-scale sorting on production lines. Compared with manual sorting, it reduces labor costs and avoids subjective errors in human judgment, providing stable and reliable automated technical support for the quality control of traditional Chinese medicine decoction pieces.

[0096] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0098] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An AI-based automatic classification system for traditional Chinese medicine decoction pieces, characterized in that, The system includes: The multi-source data acquisition module is configured to acquire multi-source data of Chinese herbal medicine pieces to be sorted on the conveyor line. The multi-source data includes visible light images, multispectral images, and microscopic images. The data processing and registration module is configured to preprocess and spatiotemporally register the multi-source data and output aligned standardized data. The feature extraction and fusion module is configured to extract visible light features, spectral features, and microtexture features from the standardized data, and to perform depth-weighted fusion of the visible light features, spectral features, and microtexture features to obtain a standardized fusion feature vector. The multidimensional joint classification module is configured to output multidimensional classification results based on the fused feature vector using a parallel multi-classification head. The multidimensional classification results include the original variety classification results, the authenticity classification results, the processing specification classification results, and the specification grade classification results. The sorting execution module is configured to perform sorting operations on the Chinese herbal medicine pieces to be sorted based on the multidimensional classification results.

2. The AI-based automatic classification system for traditional Chinese medicine decoction pieces according to claim 1, characterized in that, The multi-source data acquisition module includes: The visible light imaging unit is configured to use a high-resolution industrial line scan camera to acquire visible light images of the Chinese herbal medicine pieces to be sorted, the visible light images including the shape, color, texture, and size of the Chinese herbal medicine pieces to be sorted; The multispectral imaging unit is configured to acquire multispectral images of the Chinese herbal medicine slices to be sorted in the 400nm–1700nm band using a multispectral imager. The microscopic imaging unit is configured to acquire microscopic images of the Chinese herbal medicine slices to be sorted using a variable magnification stereomicroscopic imaging system. The microscopic images include the cell structure, tissue arrangement, and oil cell distribution on the surface of the slices. The multispectral imaging unit and the microscopic camera unit are integrated into the same optical path through a coaxial common-path beam-splitting optical system, and the field of view overlap of the multispectral imaging unit and the microscopic camera unit is ≥95%.

3. The AI-based automatic classification system for traditional Chinese medicine decoction pieces according to claim 1, characterized in that, The data processing and registration module performs preprocessing and spatiotemporal registration on the multi-source data, including: The visible light image and the microscopic image are subjected to denoising, contrast enhancement, sharpening and target region segmentation to obtain an image data stream; The multispectral image is subjected to background subtraction, scattering correction, smoothing and noise reduction, and normalization to obtain a spectral data stream; A hardware-triggered synchronization unit generates a synchronization pulse signal to align the image data stream and the spectral data stream along the time axis. A Kalman filter algorithm is then used to perform real-time spatial registration correction on the spectral data stream to compensate for spatial position deviations caused by the operation of the conveyor line.

4. The AI-based automatic classification system for traditional Chinese medicine decoction pieces according to claim 1, characterized in that, The feature extraction and fusion module includes: The visible light feature extraction branch is configured to use a ResNet-50 variant network pre-trained on ImageNet, taking the normalized data corresponding to the visible light image as input and outputting a 2048-dimensional color-texture-shape feature vector; The spectral feature extraction branch is configured to use a one-dimensional convolutional neural network combined with a multi-head self-attention mechanism, inputting the standardized data corresponding to the multispectral image and outputting a 1024-dimensional chemical fingerprint feature vector. The microtexture feature extraction branch is configured to use an improved Transformer variant architecture, taking into input normalized data corresponding to the microscopic image and outputting a 2048-dimensional microtexture feature vector.

5. The AI-based automatic classification system for traditional Chinese medicine decoction pieces according to claim 4, characterized in that, The feature extraction and fusion module performs a depth-weighted fusion of the visible light features, spectral features, and microtexture features to obtain a standardized fusion feature vector, including: Visible light features, spectral features, and micro-texture features are linearly projected and mapped to a unified dimension, then stacked into a feature sequence. The feature sequences are subjected to deep cross-modal interaction through a multi-head self-attention mechanism, so that the features of each modality are integrated with information from other modalities; after processing by residual connections and feedforward networks, an attention pooling mechanism is used to dynamically calculate the contribution weight for each modality. The interaction features of each modality are weighted and summed according to the contribution weights, and then subjected to linear projection and layer normalization to output a standardized fusion feature vector.

6. The AI-based automatic classification system for traditional Chinese medicine decoction pieces according to claim 1, characterized in that, The parallel multi-classification head includes: The original variety classifier is configured to receive the fused feature vector, process it sequentially through a set of fully connected layers for dimensionality reduction and hidden layers, and then connect to the original variety output layer to output the probability distribution of the original variety. The authenticity classifier is configured to receive the fused feature vector, process it through two fully connected layers, output a binary classification probability value, and output a determination result of whether the product is genuine or counterfeit based on the comparison result of the binary classification probability value and a preset threshold. A processing specification classifier is configured to receive the fused feature vector, first amplify the feature dimensions related to processing through an attention module, and then output the probability distribution of processing specifications through a fully connected layer, wherein the categories of processing specifications include raw products and roasted products. A specification grade classifier is configured to receive the fused feature vector and, in conjunction with the size or thickness information of the medicinal slices extracted from the segmentation branch, output a probability distribution of the specification grades, which include general goods, first-level selected goods, and second-level selected goods.

7. The AI-based automatic classification system for traditional Chinese medicine decoction pieces according to claim 6, characterized in that, When the multidimensional joint classification module outputs the multidimensional classification result based on the fused feature vector using the parallel multi-classification head, it also includes: The parallel multi-classification head is jointly optimized during training using a weighted multi-task loss function. The weighted multi-task loss function includes the cross-entropy loss of the original variety classifier, the binary classification loss of the true and false classifier, the cross-entropy loss of the processing specification classifier, and the cross-entropy loss of the specification grade classifier multiplied by a preset weighting coefficient α. The weighting coefficient α is used to adjust the training priority of the grade classification task relative to other tasks.

8. The AI-based automatic classification system for traditional Chinese medicine decoction pieces according to claim 7, characterized in that, The sorting execution module encodes the original variety classification results, authenticity classification results, processing specification classification results, and specification grade classification results into eight-bit graded labels, constructs a sorting decision tree, and assigns a corresponding sorting channel and robotic arm grasping strategy to each coding combination.

9. The AI-based automatic classification system for traditional Chinese medicine decoction pieces according to claim 1, characterized in that, Also includes: The knowledge base update module, connected to the multidimensional joint classification module, is configured to include the corresponding sample in the incremental training set when the confidence level output by the multidimensional joint classification module is lower than a preset threshold, and to trigger incremental learning during idle periods and update the network parameters of the feature extraction fusion module and the multidimensional joint classification module.

10. An AI-based automatic classification method for traditional Chinese medicine decoction pieces, applied to the AI-based automatic classification system for traditional Chinese medicine decoction pieces as described in any one of claims 1-9, characterized in that, The method includes: Collect multi-source data of Chinese herbal medicine slices to be sorted on the conveyor line, including visible light images, multispectral images and microscopic images; The multi-source data is preprocessed and spatiotemporally registered to output aligned and standardized data. Visible light features, spectral features, and microtexture features are extracted from the standardized data, and the visible light features, spectral features, and microtexture features are then fused using a depth-weighted method to obtain a standardized fused feature vector. Based on the parallel multi-classification head, multi-dimensional classification results are output according to the fused feature vector. The multi-dimensional classification results include the original variety classification results, the authenticity classification results, the processing specification classification results, and the specification grade classification results. Based on the multidimensional classification results, the Chinese herbal medicine slices to be sorted are sorted.

Images