Traditional Chinese medicine material variety analysis and recognition system based on machine vision

Abstract

This invention belongs to the field of artificial intelligence and discloses a machine vision-based system for analyzing and identifying varieties of traditional Chinese medicinal materials. The system includes: a data acquisition module; a data preprocessing module; a feature extraction module; an identification and classification module; a quality assessment module; and a result output and management module. This invention, through the result output and management module, stores and displays the variety identification results and the quality grade assessment results. By employing multi-dimensional visual data acquisition, refined preprocessing, deep learning-based multimodal feature extraction and fusion, and intelligent variety identification and quality grade assessment, it constructs a comprehensive, efficient, and standardized system for identifying traditional Chinese medicinal materials. This system can improve the accuracy and efficiency of traditional Chinese medicinal material identification, reduce the subjectivity of manual identification, and provide solid technical support for the modern management and quality control of traditional Chinese medicinal materials.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, specifically relating to a machine vision-based system for analyzing and identifying varieties of Chinese medicinal herbs. Background Technology

[0002] As traditional Chinese medicinal herbs gain increasing importance in the global pharmaceutical and healthcare field, their quality and safety have become crucial to the industry's development. Accurate and efficient identification of the varieties and quality of traditional Chinese medicinal herbs is the cornerstone of ensuring the efficacy of traditional Chinese medicine, safeguarding public medication safety, and regulating market order.

[0003] Traditional identification of Chinese medicinal materials mainly relies on experienced experts to identify them through sensory and physical characteristics such as appearance, odor, and microstructure, or in conjunction with physicochemical analysis techniques such as thin-layer chromatography and high-performance liquid chromatography. However, these methods generally suffer from inherent drawbacks such as low efficiency, strong subjectivity, and accuracy limited by expert experience, making them ill-suited to the complex reality of the vast variety of Chinese medicinal materials, significant morphological variations, and the proliferation of easily confused and counterfeit products. Especially when facing large-scale, high-throughput identification needs, traditional manual identification methods are far from sufficient.

[0004] With the development of modern technology, although machine vision technology has been introduced into the field of Chinese medicinal herb identification, attempting to extract visual features through image processing and pattern recognition, existing machine vision solutions mostly focus on a single visual modality, failing to effectively capture the deep, multi-dimensional identification features of Chinese medicinal herbs, such as internal texture, fine structure, and their association with origin and growing environment. More importantly, current technology lacks in-depth mining and integration of knowledge information such as descriptive textual information, historical documents, and pharmacopoeia standards related to Chinese medicinal herbs, making it difficult to form a comprehensive identification capability that mutually corroborates vision and knowledge. This results in identification results lacking sufficient semantic support and knowledge reasoning, making it difficult to achieve accurate analysis and identification of Chinese medicinal herb varieties, especially when identification features are not obvious or there are subtle differences, significantly reducing accuracy and reliability.

[0005] Furthermore, existing identification systems generally lack unified quantitative indicators and standardized feature libraries, making it difficult to objectively evaluate the identification process and establish a unique digital fingerprint for each medicinal material. This severely hinders the standardization, intelligentization, and traceability of the identification process for Chinese medicinal materials. Therefore, there is an urgent need for a Chinese medicinal material variety analysis and identification system that can overcome the above shortcomings and achieve high efficiency, accuracy, and a standardized system. Summary of the Invention

[0006] To address the technical problems of low efficiency, poor accuracy, and lack of standardized identification systems in existing traditional Chinese medicine identification methods, this invention provides a machine vision-based Chinese medicine variety analysis and identification system. By integrating high-resolution image acquisition, deep learning feature extraction, multimodal recognition and classification, and intelligent quality assessment technologies, it constructs an automated, standardized, and high-precision solution for Chinese medicine identification and quality control, thereby improving the efficiency, accuracy, and standardization of Chinese medicine identification.

[0007] This invention provides a machine vision-based method for analyzing and identifying varieties of traditional Chinese medicinal materials, comprising:

[0008] The data acquisition module acquires multi-dimensional visual data of the Chinese medicinal material samples to be analyzed.

[0009] The data preprocessing module performs image correction, noise removal, and region of interest segmentation on the multi-dimensional visual data.

[0010] The feature extraction module extracts multi-scale, multi-modal discriminative feature vectors from the preprocessed visual data.

[0011] The identification and classification module identifies and classifies the Chinese medicinal materials samples based on the identification feature vector.

[0012] The quality assessment module evaluates the quality grade of the Chinese medicinal material sample based on the variety identification results and the identification feature vector.

[0013] The results output and management module stores and displays the variety identification results and the quality grade assessment results.

[0014] Furthermore, the acquisition of multi-dimensional visual data of the medicinal herb sample to be analyzed through the data acquisition module specifically includes:

[0015] Visible light images of the Chinese medicinal material samples were acquired using a high-resolution color area array camera;

[0016] Near-infrared images of the Chinese medicinal herb samples were acquired using a near-infrared camera;

[0017] The surface three-dimensional morphology data of the Chinese medicinal material sample were obtained by a three-dimensional structured light scanner.

[0018] In a preferred embodiment of the present invention, the high-resolution color area array camera is equipped with a photosensitive element with more than five million pixels, achieving an optical resolution of capturing spatial details of less than 20 μm per pixel, and has an adjustable focal length optical lens to adapt to the imaging needs of medicinal materials of different sizes. The near-infrared camera operates in the wavelength range of 780 nm to 1100 nm, and is used to capture the absorption characteristics of the internal structure and chemical components of traditional Chinese medicine materials.

[0019] The 3D structured light scanner employs sinusoidal fringe projection and phase-shifting algorithms, achieving a point cloud density greater than 100 points per square millimeter and a depth measurement accuracy of ±50μm, to accurately reconstruct the surface contours and uneven features of the medicinal materials. During data acquisition, the camera and scanner are positioned above the sample, and a high-precision motion control platform precisely positions and moves the sample along the X, Y, and Z axes, ensuring comprehensive and seamless scanning and image acquisition from different focal planes and multiple viewing angles. Simultaneously, it is equipped with a uniform ring light source, coaxial light source, and transmissive light source with adjustable color temperature and brightness to adapt to the optical characteristics of different medicinal materials, eliminating shadows and enhancing texture details.

[0020] Furthermore, the step of performing image correction, noise removal, and region of interest segmentation on the multi-dimensional visual data through the data preprocessing module specifically includes:

[0021] The visible light image is subjected to distortion correction, brightness equalization, and color calibration.

[0022] The near-infrared image is subjected to speckle noise removal and contrast enhancement.

[0023] The three-dimensional topography data is subjected to point cloud denoising, surface smoothing, and three-dimensional registration.

[0024] A deep learning semantic segmentation model is used to separate the foreground and background of the visible light image, near-infrared image and three-dimensional morphology data, extract the main body area of ​​the Chinese medicinal material as the region of interest, and perform size normalization processing.

[0025] In a preferred embodiment of the present invention, the image correction uses a pre-calibrated camera parameter matrix for geometric distortion correction; the brightness equalization is achieved using an adaptive histogram equalization algorithm; and the color calibration is based on reference data from a standard color chart for color space conversion and adjustment. The speckle noise removal employs a non-local mean filtering algorithm. The point cloud denoising uses a statistical filtering method to remove outliers.

[0026] The deep learning semantic segmentation model employs either U-Net or Mask R-CNN architecture, trained on a large-scale labeled dataset, achieving a segmentation accuracy greater than 95%, and is used to accurately identify and extract the complete morphology of Chinese medicinal materials. The size normalization process scales all regions of interest images to a uniform pixel size, such as 256×256 pixels, to eliminate the impact of size differences on subsequent feature extraction.

[0027] Furthermore, the extraction of multi-scale, multi-modal discriminative feature vectors from the preprocessed visual data via the feature extraction module specifically includes:

[0028] The convolutional neural network is used to extract depth texture features, shape features, and color features from the visible light image;

[0029] Internal structural features and chemical composition-related features are extracted from the near-infrared image using an independent convolutional neural network.

[0030] Surface geometric features and macroscopic structural features are extracted from the three-dimensional topography data using a three-dimensional point cloud processing network.

[0031] The depth texture features, shape features, color features, internal structure features, chemical composition-related features, surface geometric features, and macroscopic structure features are fused to form the discrimination feature vector.

[0032] In a preferred embodiment of the present invention, the convolutional neural network for extracting depth texture features, shape features, and color features adopts a residual network architecture with a depth of fifty layers, and performs transfer learning based on an ImageNet pre-trained model. The convolutional neural network for extracting internal structural features and chemical composition-related features adopts the EfficientNet architecture to adapt to the specific spectral characteristics of near-infrared images.

[0033] The 3D point cloud processing network for extracting surface geometric and macroscopic structural features adopts the PointNet or PointNet++ architecture to directly process unstructured 3D point cloud data and extract local and global spatial geometric information, including curvature, normal vector, roughness, and volume. The feature fusion employs an attention mechanism, assigning weights based on the importance of different modal features for identifying Chinese medicinal materials, and then performing weighted concatenation to form a single identification feature vector with dimension 2048.

[0034] Furthermore, the step of identifying and classifying the medicinal herb samples based on the identification feature vector by the identification and classification module specifically includes:

[0035] The discriminative feature vector is input into a multi-label classifier;

[0036] The multi-label classifier outputs the variety category of the Chinese medicinal material sample and the corresponding recognition confidence level.

[0037] In a preferred embodiment of the present invention, the multi-label classifier employs a support vector machine or gradient boosting decision tree model, trained and optimized using a large-scale, multi-variety dataset of Chinese medicinal herbs. The variety categories cover common Chinese medicinal herbs included in the National Pharmacopoeia, including but not limited to over three hundred varieties such as ginseng, angelica, wolfberry, and licorice.

[0038] The recognition confidence level is expressed as a percentage, indicating the reliability of the recognition result. When the recognition confidence level is lower than a preset threshold, the system prompts for manual review.

[0039] Furthermore, the step of evaluating the quality grade of the Chinese medicinal material sample through the quality assessment module based on the variety identification result and the identification feature vector specifically includes:

[0040] Extract specific characteristics related to the quality grade of the Chinese medicinal materials;

[0041] Input the specific features into the quality assessment model;

[0042] The quality assessment model outputs the quality grade of the Chinese medicinal material sample.

[0043] As a preferred embodiment of the present invention, the specific characteristics related to the quality grade of the Chinese medicinal materials include, but are not limited to: color uniformity, number of surface defects, and proportion of insect-infested or moldy areas in visible light images; internal structure density and foreign matter doping signal intensity in near-infrared images; and shape regularity, crack length and depth, and uniformity of size distribution in three-dimensional morphology data.

[0044] The quality assessment model adopts a regression-based convolutional neural network model. By learning the characteristic distribution of Chinese medicinal materials of different quality grades, it classifies them into multiple preset grades, such as first-grade, second-grade, qualified, or unqualified products, and provides quantitative values ​​of specific defect indicators, such as the number of surface spots and the proportion of foreign matter area. The classification of quality grades is based on the provisions of the National Pharmacopoeia on the quality standards of Chinese medicinal materials.

[0045] Furthermore, the storage and display of the variety identification results and the quality grade assessment results through the result output and management module specifically includes:

[0046] The variety identification results, the quality grade assessment results, the original visual data, and the identification feature vectors are stored in the database;

[0047] The user interface displays the variety identification results, the quality grade assessment results, key feature maps, and historical batch data.

[0048] In a preferred embodiment of the present invention, the database is a relational or non-relational database with a storage capacity of more than 100,000 medicinal material sample data, supporting fast query and data traceability functions. The user interface includes a high-resolution display that shows in real time the image of the medicinal material to be identified, the identified variety name, confidence level, quality grade, specific values ​​of various quality indicators, and information such as identification time and operator. The user interface also provides a historical query function, allowing users to view and export identification reports for historical batches.

[0049] This invention provides a machine vision-based system for analyzing and identifying varieties of traditional Chinese medicinal materials, comprising:

[0050] The data acquisition module is used to acquire multi-dimensional visual data of the Chinese medicinal materials samples to be analyzed;

[0051] The data preprocessing module is used to perform image correction, noise removal, and region of interest segmentation on the multi-dimensional visual data;

[0052] The feature extraction module is used to extract multi-scale, multi-modal discriminative feature vectors from the preprocessed visual data;

[0053] The identification and classification module is used to identify and classify the Chinese medicinal material samples based on the identification feature vector.

[0054] The quality assessment module is used to assess the quality grade of the Chinese medicinal material sample based on the variety identification results and the identification feature vector.

[0055] The results output and management module is used to store and display the variety identification results and the quality grade assessment results.

[0056] Furthermore, the data acquisition module includes:

[0057] A high-resolution color area array camera is used to acquire visible light images of the Chinese medicinal herb samples;

[0058] A near-infrared camera is used to acquire near-infrared images of the Chinese medicinal herb samples;

[0059] A three-dimensional structured light scanner is used to acquire the surface three-dimensional morphology data of the Chinese medicinal material sample;

[0060] A high-precision motion control platform is used to carry samples of Chinese medicinal materials and perform precise positioning and movement along the X, Y, and Z axes.

[0061] Multispectral light sources are used to provide uniform illumination with adjustable color temperature and brightness.

[0062] In a preferred embodiment of the present invention, the high-resolution color area array camera is equipped with a photosensitive element with more than five million pixels, achieving an optical resolution of less than 20 μm of spatial detail per pixel. The near-infrared camera operates in the wavelength range of 780 nm to 1100 nm. The three-dimensional structured light scanner employs a sinusoidal fringe projection and phase-shifting algorithm, achieving a point cloud density greater than one hundred points per square millimeter and a depth measurement accuracy of ±50 μm.

[0063] The high-precision motion control platform is driven by a closed-loop servo motor, with an XYZ axis travel of 200mm×200mm×100mm and a repeatability of ±5μm. The multispectral light source integrates multiple light-emitting diode arrays, providing various spectral illumination modes such as white light, ultraviolet light, and near-infrared light, and its light intensity can be precisely controlled.

[0064] Furthermore, the data preprocessing module is implemented through a processing unit, which is equipped with a memory and a processor. The processing unit executes program instructions stored in the memory to perform the following functions:

[0065] The visible light image is subjected to distortion correction, brightness equalization, and color calibration.

[0066] The near-infrared image is subjected to speckle noise removal and contrast enhancement.

[0067] The three-dimensional topography data is subjected to point cloud denoising, surface smoothing, and three-dimensional registration.

[0068] A deep learning semantic segmentation model is used to separate the foreground and background of the visible light image, near-infrared image and three-dimensional morphology data, extract the main body area of ​​the Chinese medicinal material as the region of interest, and perform size normalization processing.

[0069] In a preferred embodiment of the present invention, the deep learning semantic segmentation model adopts the U-Net or Mask R-CNN architecture.

[0070] Furthermore, the feature extraction module is implemented through a processing unit, which is equipped with a memory and a processor. The processing unit executes program instructions stored in the memory to achieve the following functions:

[0071] The convolutional neural network is used to extract depth texture features, shape features, and color features from the visible light image;

[0072] Internal structural features and chemical composition-related features are extracted from the near-infrared image using an independent convolutional neural network.

[0073] Surface geometric features and macroscopic structural features are extracted from the three-dimensional topography data using a three-dimensional point cloud processing network.

[0074] The depth texture features, shape features, color features, internal structure features, chemical composition-related features, surface geometric features, and macroscopic structure features are fused to form the discrimination feature vector.

[0075] In a preferred embodiment of the present invention, the convolutional neural network for extracting depth texture features, shape features, and color features employs a residual network architecture. The convolutional neural network for extracting internal structural features and chemical composition-related features employs an EfficientNet architecture. The 3D point cloud processing network for extracting surface geometric features and macroscopic structural features employs a PointNet or PointNet++ architecture. The feature fusion employs an attention mechanism.

[0076] Furthermore, the identification and classification module is implemented through a processing unit, which is equipped with a memory and a processor. The processing unit executes program instructions stored in the memory to achieve the following functions:

[0077] The discriminative feature vector is input into a multi-label classifier;

[0078] The multi-label classifier outputs the variety category of the Chinese medicinal material sample and the corresponding recognition confidence level.

[0079] In a preferred embodiment of the present invention, the multi-label classifier employs a support vector machine or a gradient boosting decision tree model.

[0080] Furthermore, the quality assessment module is implemented through a processing unit, which is equipped with a memory and a processor. The processing unit executes program instructions stored in the memory to perform the following functions:

[0081] Extract specific characteristics related to the quality grade of the Chinese medicinal materials;

[0082] Input the specific features into the quality assessment model;

[0083] The quality assessment model outputs the quality grade of the Chinese medicinal material sample.

[0084] In a preferred embodiment of the present invention, the quality assessment model adopts a regression-based convolutional neural network model.

[0085] Furthermore, the result output and management module is implemented through a processing unit, which is equipped with a memory and a processor. The processing unit executes program instructions stored in the memory to achieve the following functions:

[0086] The variety identification results, the quality grade assessment results, the original visual data, and the identification feature vectors are stored in the database;

[0087] The user interface displays the variety identification results, the quality grade assessment results, key feature maps, and historical batch data.

[0088] In a preferred embodiment of the present invention, the database is a relational database or a non-relational database.

[0089] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0090] 1. This invention provides a machine vision-based system for analyzing and identifying Chinese medicinal herbs, improving the efficiency of herb identification. By integrating a high-resolution color area array camera, a near-infrared camera, and a 3D structured light scanner, it can automatically and rapidly acquire multi-dimensional, high-precision visual data of Chinese medicinal herbs, replacing the time-consuming and labor-intensive process of traditional manual identification. The data acquisition speed can reach more than 20 samples per minute, which is dozens of times more efficient than manual identification.

[0091] 2. This invention improves the accuracy of identifying Chinese medicinal materials. The data preprocessing module denoises, corrects, and precisely segments the raw data, ensuring the quality of the input data. The feature extraction module employs a multi-path deep learning network to extract multi-scale, multi-modal deep identification features from visible light, near-infrared, and three-dimensional morphology data. These features are more discriminative than macroscopic characteristics observed manually.

[0092] 3. This invention establishes a standardized identification system for Chinese medicinal materials. Through standardized data acquisition procedures, unified image processing algorithms, and rigorously trained recognition models, the subjectivity and inconsistencies inherent in manual identification are eliminated. The quality assessment module can quantitatively analyze various indicators such as color, shape, size, integrity, and defects of medicinal materials according to national pharmacopoeia standards, and provide objective quality grades, achieving standardization of the identification process and traceability of results. This provides a unified and reliable quality control basis for the production, circulation, and use of Chinese medicinal materials, helping to ensure their quality and safety.

[0093] 4. This invention achieves comprehensive quality assessment capabilities. It not only accurately identifies the varieties of Chinese medicinal materials but also intelligently assesses their quality grades. By analyzing defect characteristics, internal structure, and macroscopic morphology, it quantitatively detects common quality problems such as insect infestation, mold, cracks, and foreign matter adulteration, providing specific quantitative indicators. This provides data support for the quality grading and optimization of Chinese medicinal materials, effectively preventing substandard medicinal materials from entering the market.

[0094] 5. This invention possesses excellent scalability and adaptability. Through the model training and optimization module, the system can continuously learn and update the model to adapt to new medicinal material varieties, variations in medicinal materials from different producing areas, and constantly improving quality standards. Its modular design makes the system easy to maintain and upgrade, and its functions can be expanded according to actual needs. Attached Figure Description

[0095] Figure 1 This is a schematic diagram of the overall technical architecture of the machine vision-based Chinese medicinal herb variety analysis and identification system proposed in this invention.

[0096] Figure 2 This is a schematic diagram of the core principle framework of multimodal discrimination feature extraction and fusion in this invention;

[0097] Figure 3 This is a logical flowchart of the multi-dimensional visual data acquisition process in this invention;

[0098] Figure 4 This is a flowchart illustrating the logical flow of multi-dimensional visual data preprocessing in this invention.

[0099] Figure 5 This is a logical flowchart of the identification and classification of Chinese medicinal materials in this invention;

[0100] Figure 6 This is a logical flowchart of the quality grade assessment of Chinese medicinal materials in this invention; Detailed Implementation

[0101] The exemplary embodiments, features, and technical details of this application will be described in detail below. In view of the technical problems of traditional Chinese medicinal material identification methods, such as low efficiency, insufficient accuracy, and lack of standardized identification systems, this application proposes a machine vision-based system and method for analyzing and identifying Chinese medicinal material varieties. This system aims to achieve automated, high-precision identification and quality grade assessment of Chinese medicinal material varieties by deeply integrating multi-dimensional visual data with advanced machine vision algorithms, thereby overcoming the shortcomings of existing technologies in terms of efficiency and standardization.

[0102] Please refer to Figure 1-6 In one embodiment of this application, the system architecture of the machine vision-based Chinese medicinal herb variety analysis and identification system includes, but is not limited to: a data acquisition unit, a data preprocessing unit, a feature extraction and fusion unit, a variety identification and classification unit, a quality grade evaluation unit, and a result output and management unit.

[0103] The system comprises several modules: a data acquisition unit to obtain multi-dimensional visual information of Chinese medicinal materials; a data preprocessing unit to standardize the acquired raw data to eliminate noise and improve data quality; a feature extraction and fusion unit to extract identification features from the preprocessed data and effectively fuse them; a variety identification and classification unit to identify and classify Chinese medicinal material varieties based on the fused features; a quality grade assessment unit to intelligently assess the quality grade of the identified Chinese medicinal materials; and a results output and management unit to display, store, and manage the identification and assessment results. These units work together to form a complete automated identification process for Chinese medicinal materials.

[0104] In the technical solution of this application, the method for analyzing and identifying Chinese medicinal materials based on machine vision includes the following steps: S1, collecting multi-dimensional visual data; S2, preprocessing multi-dimensional visual data; S3, extracting and fusing multi-modal identification features; S4, identifying and classifying Chinese medicinal materials; S5, evaluating the quality grade of Chinese medicinal materials.

[0105] In one embodiment of the present invention, step S1 involves multi-dimensional visual data acquisition. It should be understood that traditional identification of Chinese medicinal materials mainly relies on human experience, and the identification process is easily affected by subjective factors, resulting in low efficiency and difficulty in standardization. To overcome these limitations, this step aims to construct a high-precision, multi-spectral visual data acquisition system to obtain rich surface characteristics of Chinese medicinal materials under different lighting conditions and observation angles. This includes, but is not limited to, acquiring key information such as macroscopic morphology, microstructure, color spectrum, and surface texture, providing comprehensive data support for subsequent machine vision analysis. The multi-dimensional visual data acquisition process is further subdivided into several sub-steps.

[0106] In one sub-step, the method acquires a two-dimensional color image of the medicinal herbs under standard white light illumination using a high-resolution visible light image acquisition module. This module typically includes a high-resolution industrial camera, such as one equipped with a 12-megapixel to 50-megapixel complementary metal-oxide-semiconductor sensor, supplemented by a fixed-focus or zoom lens with a focal length range of 25mm to 150mm to accommodate the acquisition needs of medicinal herbs of different sizes. During image acquisition, the system uses an adjustable integrating sphere or diffuse light source to provide uniform non-directional illumination, ensuring that the surface of the medicinal herbs has no obvious shadows or highlights, thus accurately reflecting their natural color and texture.

[0107] Acquisition parameters such as exposure time, white balance, and gamma correction are automatically adjusted or finely manually according to a preset calibration process to ensure stable image quality and high color fidelity. After each acquisition, image data, along with metadata such as timestamps and shooting angles, is stored on a local or remote data storage server in a lossless compressed format, such as tag image file format or portable network graphics format. Data transmission is conducted via Gigabit Ethernet or Universal Serial Bus 3.0 interface to ensure that the data transmission rate meets real-time requirements.

[0108] In another sub-step, the method captures the spectral information of the medicinal materials in the wavelength range of 400 to 1000 nanometers using a hyperspectral imaging module. At the heart of this module is a pushbroom or staring hyperspectral camera with a spectral resolution of 5 to 10 nanometers. During acquisition, the medicinal material sample is placed on a linear scanning stage and passes through the field of view of the hyperspectral camera at a constant speed. A broadband halogen lamp or LED array provides ample and stable illumination.

[0109] Hyperspectral image data is stored in a cubic format, with two dimensions representing spatial information and one dimension representing spectral information. Each pixel contains a spectral curve reflecting the absorption, reflection, or scattering characteristics of the medicinal herb at that point to different wavelengths of light. These spectral features are crucial for distinguishing medicinal herbs with similar appearances but different chemical compositions. Data transmission interfaces typically employ camera link interfaces or fiber optic channels, supporting high-bandwidth data streams. During the data verification phase, radiometric calibration is performed periodically using a standard reference board to eliminate errors caused by light source fluctuations and uneven sensor responses, ensuring the accuracy of the spectral data.

[0110] In another sub-step, the method constructs 3D point cloud data or depth images of the medicinal materials using a 3D depth sensing module. This module can be implemented using techniques such as structured light projection, lidar scanning, or stereo vision matching. For example, a known coded pattern can be projected onto the surface of the medicinal material, and a high-resolution camera can capture the deformed pattern. The 3D coordinates of each point can then be calculated using triangulation principles. Another approach is to use a time-of-flight sensor to directly measure the time required for light to travel from the sensor to the surface of the object and back to obtain depth information.

[0111] 3D data provides precise geometric shapes, dimensions, and surface undulations of Chinese medicinal herbs, which is crucial for identifying herbs with specific morphological characteristics. The acquired point cloud data is typically stored in a point cloud data exchange format, containing the 3D coordinates and color information of each point. Error control during data acquisition includes calibrating baseline distance, lens distortion correction, and multi-view fusion algorithms to improve the accuracy of 3D reconstruction.

[0112] In another sub-step, the method may also integrate other auxiliary sensors, such as an infrared thermal imaging module, to acquire the surface temperature distribution of the medicinal materials. Different states of some medicinal materials, such as freshness or degree of mold, can cause slight changes in surface temperature due to differences in metabolic activity or moisture content, which can be captured by a high-sensitivity infrared thermal imager. Infrared thermal imaging data can serve as supplementary information to help determine the activity of the medicinal materials or the presence of internal defects.

[0113] When the data acquisition unit executes these sub-steps, the data flow and processing between its various modules are as follows: First, all sensors are started synchronously, and visible light images, hyperspectral data, three-dimensional depth data, and infrared thermal imaging data are acquired in parallel. The acquired raw data are transmitted to a central data processing unit through their respective dedicated data interfaces.

[0114] The central processing unit includes a high-speed cache for temporary storage of incoming data and performs preliminary format conversion and timestamp alignment of the data using a multi-threaded or parallel computing architecture. The stability of the data stream is ensured by both transmission bandwidth and processing power. Anomaly handling mechanisms include packet loss detection and retransmission, sensor fault alarms, and automatic recovery from data acquisition interruptions, ensuring data integrity and reliability. All acquisition parameters, such as sampling frequency, resolution, and gain, are uniformly managed in the control software and configured through a user interface or automated scripts.

[0115] In one embodiment of the present invention, step S2 involves multi-dimensional visual data preprocessing. It should be understood that the raw multi-dimensional visual data acquired from the data acquisition unit often contains noise, inconsistencies, and redundant information, which can severely affect the accuracy and efficiency of subsequent feature extraction and recognition. Therefore, this step aims to denoise, enhance, standardize, and align the raw data through a series of refined preprocessing operations, thereby generating high-quality, consistent data representations and laying a solid foundation for subsequent in-depth analysis. The multi-dimensional visual data preprocessing process is broken down into several sub-steps.

[0116] In one sub-step, the method performs image denoising and enhancement on the visible light image. Image denoising typically employs algorithms such as nonlocal mean filtering, 3D block matching filtering, or Gaussian mixture models to eliminate random noise introduced by sensor noise and uneven ambient lighting, smooth image regions, and preserve edge details.

[0117] Image enhancement includes contrast stretching, adaptive histogram equalization (AHE), and color space transformation, such as from red, green, and blue to brightness, hue, and saturation, to improve image visibility and distinguishability. For example, by limiting contrast in adaptive histogram equalization, artifacts in local areas caused by over-enhancement can be effectively avoided. The parameter sources and verification mechanisms are as follows: parameters such as the kernel size, standard deviation, and number of iterations of the denoising filter are adaptively adjusted based on the image signal-to-noise ratio and texture complexity, or optimal values ​​are obtained through cross-validation. Parameters of the enhancement algorithm, such as the contrast limiting factor and brightness offset, are dynamically set according to the brightness distribution characteristics of the Chinese medicinal herb image. After processing, the image data is converted into a uniform grayscale or color image format, such as an 8-bit or 16-bit per channel image array.

[0118] In another sub-step, the method performs radiance correction and spectral denoising on the hyperspectral data. Radiance correction aims to eliminate the effects of factors such as light source intensity fluctuations, inconsistent sensor responses, and dark current noise on spectral reflectance. This is typically achieved by measuring a standard white plate and dark current, and applying the following formula for correction:

[0119] ;

[0120] in, This represents the corrected spectral reflectance. This indicates the original spectral intensity of the Chinese medicinal material sample. Indicates the spectral intensity of dark current. Indicates the spectral intensity of a standard whiteboard. Indicates wavelength.

[0121] Spectral denoising employs spectral smoothing algorithms, such as moving average, Savitzky-Golay filtering, or wavelet denoising, to reduce random fluctuations in the spectral curve. Additionally, baseline correction is performed to eliminate background drift or scattering effects. Parameter selection, such as the window size and polynomial order of the Savitzky-Golay filter, is optimized based on the noise level of the spectral data and the required smoothing of features. Verification mechanisms include comparison with a standard spectral library to ensure that the corrected spectral data conforms to physical reality.

[0122] In another sub-step, the method performs point cloud denoising and registration on the 3D depth data. Point cloud denoising can employ statistical filtering or out-of-radius point removal algorithms to eliminate outliers and sensor noise. Registration aligns point cloud data acquired from multiple scans or from different viewpoints into a unified coordinate system, for example, through iterative nearest-neighbor algorithms or normal distribution transformation algorithms. Registration accuracy is crucial for constructing a complete and seamless 3D model of Chinese medicinal materials. Parameters include the number of iterations for point cloud matching, the convergence threshold, and the corresponding point search radius. The verification mechanism includes calculating the root mean square error of the overlapping area of ​​the registered point clouds.

[0123] In another sub-step, the method performs image segmentation and region of interest extraction. This step aims to separate the main body of the medicinal herb from the background and locate its key identification parts. For visible light images, a deep learning-based semantic segmentation model, such as YouNet or a fully convolutional network, can be used to achieve pixel-level foreground-background segmentation of the medicinal herb. For hyperspectral data, spectral features can be used for classification, such as identifying the medicinal herb region through support vector machines or random forests.

[0124] For 3D depth data, foreground segmentation can be performed using geometric feature extraction methods such as curvature and normals, or clustering-based methods. The extracted medicinal herb regions or key parts serve as input for subsequent feature extraction. In this step, the data structure is transformed from the original image or point cloud into region-of-interest data with masks or bounding boxes. Anomaly handling includes marking segmentation failures or inaccurate region extraction, triggering manual review or the use of alternative segmentation algorithms.

[0125] The data preprocessing unit executes these sub-steps with the following internal data flow and processing: raw visible light images, hyperspectral cube data, and 3D point cloud data from the data acquisition unit enter the preprocessing unit in parallel. The image processing pipeline, spectral processing pipeline, and 3D data processing pipeline operate independently, each performing its own function. Denoising, correction, and enhancement operations are performed sequentially on their respective data types. After basic processing, all modal data enters a multimodal alignment module. This module is responsible for aligning data from different modalities to the same reference frame based on timestamps and spatial coordinate information, ensuring that all visual information of each sample corresponds consistently in time and space. For example, registering a hyperspectral image onto a visible light image, or projecting a 3D point cloud onto a 2D image plane.

[0126] The accuracy of data alignment is evaluated through feature point matching or geometric transformation. Finally, the preprocessed data is passed to the feature extraction and fusion unit in a unified structured format, such as a multidimensional tensor or a list of feature vectors. Anomalies such as data corruption or calibration failures are logged and trigger warnings, potentially leading to the discarding or re-acquisition of that batch of data.

[0127] In one embodiment of the present invention, step S3 involves multimodal discriminative feature extraction and fusion. It should be understood that the preprocessed multidimensional visual data contains rich information about Chinese medicinal materials, but this raw data is still high-dimensional and redundant. Directly using this data for recognition is inefficient and easily affected by non-discriminative features. The core objective of this step is to utilize a deep learning model to extract highly discriminative low-dimensional feature representations from visible light images, hyperspectral data, and three-dimensional depth data, and to effectively combine features from different modalities through a sophisticated fusion mechanism, forming a comprehensive and robust characterization of the varieties and quality of Chinese medicinal materials. The multimodal discriminative feature extraction and fusion process is further refined into several sub-steps.

[0128] In one sub-step, the method utilizes a visual feature extractor based on a deep convolutional neural network to extract high-level semantic features from visible light images. This feature extractor typically employs a deep convolutional neural network model pre-trained on a large image dataset, such as a residual network, a densely connected network, or a visual transducer. These networks, through multi-layer convolution, pooling, and activation function operations, can automatically learn hierarchical features in the image, such as texture, color, and shape. For example, shallow layers of the network may learn basic texture features such as edges and corners, while deeper layers can capture high-level semantic features such as the overall morphology and spot distribution of medicinal materials. For different medicinal materials, such as rhizomes, leaves, or flowers and fruits, different pre-trained models or transfer learning fine-tuning can be used to adapt to their specific morphological characteristics. The dimension of the feature vector is typically set between 512 and 2048 dimensions.

[0129] In another sub-step, the method utilizes a spectral feature extractor based on a hybrid architecture of convolutional neural networks and recurrent neural networks to extract spectral features from hyperspectral data. Hyperspectral data contains both spatial and spectral information. Therefore, the extractor first processes the spectral curve of each pixel using a one-dimensional convolutional layer to capture local correlations between different bands, thereby extracting features in the spectral dimension. Subsequently, a long short-term memory network or a gated recurrent unit can be combined to model the sequential spectral features to capture long-distance dependencies between spectral bands and reduce the dimensionality to obtain a compact spectral feature vector. Alternatively, a three-dimensional convolutional neural network can be used to directly extract spatiotemporal integrated features from hyperspectral cubic data, considering both spatial texture and spectral information. The extracted features can reflect the differences in the internal chemical composition of Chinese medicinal materials, such as the absorption peaks of active ingredients and moisture content.

[0130] In another sub-step, the method utilizes a geometric feature extractor based on a point cloud processing network to extract geometric features from 3D depth data. For 3D point cloud data, models such as point networks, point network-plus-point networks, or graph convolutional networks can be used to directly learn local and global geometric features from the unordered point cloud. These features include the overall shape, surface curvature, unevenness, size ratio, and geometric structure of specific parts of the medicinal materials. For example, for some medicinal materials with unique growth shapes, their 3D geometric features are extremely important identification criteria. The extracted feature vector dimension is kept consistent with the visual feature vector dimension for subsequent fusion.

[0131] In another sub-step, the method fuses features from different modalities. The goal of feature fusion is to effectively integrate discriminative information from different visual modalities to form a more comprehensive and robust single feature representation. Fusion strategies can employ early fusion, late fusion, or intermediate fusion. In this embodiment, the method employs a deep learning-based intermediate fusion strategy, implemented through a multimodal feature fusion network. This network receives output feature vectors from a visible light feature extractor, a hyperspectral feature extractor, and a 3D geometric feature extractor as input. The fusion network may contain multiple fully connected layers, attention mechanisms, or gating mechanisms.

[0132] Through attention mechanisms, such as channel attention or spatial attention, the system can adaptively allocate weights for different modal features based on the characteristics of the current sample.

[0133] ;

[0134] in, This represents the fused feature vector. Indicates the first Feature vectors of each modality Indicates the first Adaptive weights for each modality This represents the total number of modes.

[0135] When the visible light image quality is poor, such as being blurry, the system automatically reduces its weight and instead focuses on hyperspectral and 3D geometric features. The fusion network learns through training how to optimally combine these heterogeneous features to maximize discrimination performance. For example, if the color features of the medicinal material in the sample are very prominent, then color-related features from the visible light image may be given higher weight; if the key identification point of the sample lies in its unique microstructure or chemical composition, then the weight of hyperspectral or 3D geometric features will be increased accordingly. The fused feature vector is typically set to 1024 or 2048 dimensions, and it will serve as the final discrimination feature input to the variety identification and classification unit. During the fusion process, regularization techniques such as dropout and batch normalization are also introduced to prevent overfitting and improve the model's generalization ability. Interaction and anomaly handling include monitoring the output quality of each modality feature extractor, such as the norm of the feature vector or the distribution of activation values. Once anomalies are detected, such as feature values ​​that are too small or too large, warnings are triggered, and the fusion weights may be adjusted or a re-collection of data may be requested.

[0136] In one embodiment of the present invention, step S4 involves identifying and classifying Chinese medicinal materials. It should be understood that after the multimodal identification feature extraction and fusion are completed, a fused feature vector that is highly representative of the Chinese medicinal material sample has been obtained. The goal of this step is to use this fused feature vector, through a trained classification model, to accurately determine which preset variety category the Chinese medicinal material sample belongs to. This requires the classification model to have strong pattern recognition and generalization capabilities, capable of handling differences in the appearance of Chinese medicinal materials caused by different batches, different growth environments, and even different processing methods. The process of identifying and classifying Chinese medicinal materials is further broken down into several sub-steps.

[0137] In one sub-step, the method classifies the fused feature vectors into different varieties using a deep neural network-based classifier. This classifier typically consists of one or more fully connected layers. Its input layer receives the fused feature vectors, and its output layer uses a soft maximum activation function to output a probability value for each predefined category of Chinese medicinal herbs. The classifier is trained using a cross-entropy loss function and its network weights are adjusted via backpropagation. The training dataset contains fused feature vectors of a large number of Chinese medicinal herb samples with known variety labels.

[0138] To improve the generalization ability of the classifier, regularization techniques such as data augmentation, batch normalization, and dropout are employed during training. In practical applications, when the fused feature vector of a new Chinese medicinal herb sample is input into the classifier, the output layer generates a probability distribution containing all possible variety categories. The category with the highest probability value is identified as the variety of the Chinese medicinal herb.

[0139] In another sub-step, the method introduces a confidence assessment mechanism. The probability value output by the classifier directly reflects the model's confidence level in the recognition result. Typically, a confidence threshold is set, such as 95%. If the highest probability value is lower than this threshold, the recognition result is considered to have insufficient confidence, and the system will mark it as pending review. This may trigger a second assessment by a human expert, or prompt the user to adjust the acquisition angle, lighting conditions, etc., to obtain clearer data for re-identification. This mechanism effectively avoids misjudgments under low confidence levels, ensuring the reliability of the recognition results.

[0140] In another sub-step, the method also considers multi-classifier voting or ensemble learning strategies. For certain varieties that are difficult to distinguish, multiple independent classifiers can be trained, for example, based on different deep learning architectures or different feature subsets. In the final decision-making stage, the outputs of multiple classifiers are comprehensively judged through ensemble learning methods such as weighted voting or stacked generalization to improve the overall recognition accuracy and robustness. For example, if two classifiers identify ginseng as human ginseng, while a third classifier identifies it as codonopsis, the system can perform weighted voting based on the classifiers' historical performance and confidence levels to ultimately determine the result.

[0141] In another sub-step, the method verifies and provides feedback on the classification results. The identified variety name is compared with a standard Chinese medicinal materials database to obtain detailed information about the variety, such as its scientific name, synonyms, main producing areas, and key identification points. If the identification result conflicts with known information in the database, such as discrepancies between shape features and category descriptions, the system will issue a warning and may initiate an exception handling process. The feedback mechanism also allows human experts to correct the system's identification results. This corrected data can be used to iteratively optimize the model's performance, forming a closed loop of continuous learning.

[0142] The data flow and processing of the variety identification and classification unit during these sub-steps are as follows: The fused feature vector output by the fusion feature extraction and fusion unit is transmitted to the input interface of the variety identification and classification unit. First, this vector is fed into a pre-loaded deep neural network classification model for inference. The model's output is a high-dimensional vector, where each element corresponds to the log odds of a variety category. These log odds are converted into a probability distribution using a soft maximum function. Subsequently, the system extracts the highest probability value and its corresponding variety label. This highest probability value is compared with a preset confidence threshold.

[0143] If the confidence level meets the requirements, the variety label directly enters the quality grade assessment unit. If the confidence level is insufficient, the result is marked and enters the manual review queue. Anomaly handling includes situations such as classification model loading failure, inference time exceeding the threshold, or abnormally flat output probability distribution. The system will log and backtrack to the previous step to check the input features or prompt the model to reload. The classifier's training parameters, such as learning rate, batch size, and training period, are determined through hyperparameter optimization algorithms such as grid search or Bayesian optimization.

[0144] In one embodiment of the present invention, step S5 involves evaluating the quality grade of Chinese medicinal materials. It should be understood that the value of Chinese medicinal materials depends not only on their variety but, more importantly, on their quality grade. Traditional quality assessments heavily rely on expert experience, sensory judgment, and time-consuming and labor-intensive physicochemical testing, resulting in high subjectivity, low efficiency, and inconsistent standards. This step aims to automate and objectively evaluate the quality grade of Chinese medicinal materials based on variety information identified by machine vision and previously extracted fusion features, combined with preset quality evaluation standards. The quality grade evaluation process for Chinese medicinal materials is broken down into several sub-steps.

[0145] In one sub-step, the method retrieves the corresponding quality assessment standards based on the identified medicinal herb varieties. Different medicinal herb varieties have different quality evaluation indicators and grading rules. For example, for root and rhizome medicinal herbs such as ginseng, quality may focus on the thickness of the root, the rhizome, the fibrous roots, the cross-sectional texture, and the presence of insect infestation and mold; for flower and fruit medicinal herbs such as wolfberry, the focus is on their size, color, plumpness, moisture content, and breakage rate. The system maintains a database containing quality assessment standards for various medicinal herbs. Whenever a medicinal herb variety is identified, the system automatically queries and loads the specific assessment dimensions and indicator weights corresponding to that variety. These standards are usually stored in the form of structured data, such as key-value pairs or rule lists, for easy machine parsing.

[0146] In another sub-step, the method extracts or computes features relevant to quality assessment from the fused feature vector. While the fused feature vector has been used for variety identification, it also contains a wealth of quality-related information. For example, by analyzing visible light image features, the color uniformity, surface gloss, presence of spots or mold in medicinal materials can be assessed; by analyzing three-dimensional geometric features, their dimensions such as length, diameter, perimeter, volume, density, and shape regularity can be accurately measured; by analyzing hyperspectral features, the content of active ingredients, moisture content, and the presence of abnormal chemical substances can be inferred. These quality-related sub-features will be further decomposed from the fused feature vector or computed through a regression model. For example, a pre-trained regression sub-network can be used to map the fused feature vector to specific quality index values, such as the predicted content of a certain active ingredient.

[0147] In another sub-step, the method utilizes a quality assessment model based on multi-output regression or multi-class classification to quantitatively evaluate the quality grade of Chinese medicinal materials. This model can be a multi-output regression network that directly predicts the values ​​of various quality indicators, such as predicting the length of the medicinal material as 8.5 cm, the diameter as 1.2 cm, and the color uniformity as 0.9; or it can be a multi-class classifier that directly outputs the quality grade of the medicinal material, such as premium, first-grade, second-grade, or unqualified. The model's input is the extracted quality-related features. The training dataset contains feature data of a large number of medicinal material samples with known quality grades or quality indicator values. The training objective of the model is to minimize the error between the predicted and true values.

[0148] In another sub-step, the method generates a quality assessment report and visualizes it. The assessment results not only include the final quality grade but also detail the values ​​of each assessment indicator, their corresponding standard ranges, and the specific judgment criteria provided by the system, such as insufficient length or minor mold spots on the surface. This information is presented to the user in clear and easy-to-understand charts and text, for example, through a graphical user interface, and can be exported as a printable report file. Visualizations may include highlighting abnormal areas in a 3D model, comparative analysis of spectral curves, and color distribution histograms, enabling users to intuitively understand the assessment results.

[0149] The data flow and processing of the quality grade assessment unit during these sub-steps are as follows: The variety label and fused feature vector output by the variety identification and classification unit are transmitted to the quality grade assessment unit. First, the variety label is used to query the internal quality standard database and load specific assessment parameters for that variety. Subsequently, the fused feature vector is fed into a series of pre-trained sub-models, which are specifically designed to extract or regress specific quality-related indicator values. These indicator values, such as size data, color index, and spectral characteristics, are compared and calculated with the loaded quality standards. Finally, through a set of decision rules or a multi-output regression model, the comprehensive quality grade of the medicinal material is derived.

[0150] The evaluation results, along with detailed indicator data, are formatted and displayed and stored through the results output and management unit. Anomaly handling includes situations where the quality assessment model's predicted values ​​exceed reasonable ranges or key quality indicator data is missing; the system will mark these as anomalies and prompt the user for manual intervention. The parameters of the evaluation model, such as weights and thresholds, are regularly updated and validated using expert knowledge and actual sampling data.

[0151] The system also includes a results output and management unit. This unit is responsible for presenting the identified varieties, confidence levels, quality grades, and detailed evaluation reports of Chinese medicinal materials to the operators. Output methods may include real-time updates on a high-resolution display screen, generation of printed reports, and data transmission to the Chinese medicinal material management database via a network interface. Management functions involve querying historical identification records, statistical analysis, and user access control. All identification data, such as original images, hyperspectral data, 3D models, extracted features, identification results, and evaluation reports, are structured and stored in a central database, with version control and secure backups to ensure data integrity, traceability, and security. The data query interface supports various query conditions, such as by variety, by date, and by quality grade, facilitating data analysis and traceability for users.

[0152] In summary, this application constructs a comprehensive, efficient, and standardized identification system for Chinese medicinal materials through multi-dimensional visual data acquisition, refined preprocessing, deep learning-based multimodal feature extraction and fusion, and intelligent variety identification and quality grade assessment. This system can improve the accuracy and efficiency of Chinese medicinal material identification, reduce the subjectivity of manual identification, and provide solid technical support for the modern management and quality control of Chinese medicinal materials.

[0153] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "includes" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A machine vision-based system for analyzing and identifying varieties of traditional Chinese medicinal materials, characterized in that, include: The data acquisition module is used to acquire multi-dimensional visual data of the Chinese medicinal material sample to be analyzed, including visible light images, near-infrared images, and three-dimensional surface morphology data. The data preprocessing module is used to perform image correction, noise removal, and region of interest segmentation on the multi-dimensional visual data. The preprocessing includes noise elimination, image quality enhancement, and localization of the main area of ​​the Chinese medicinal materials. The feature extraction module is used to extract multi-scale, multi-modal discriminative feature vectors from the preprocessed visual data. The feature vectors integrate features of depth texture, shape, color, internal structure, chemical composition, surface geometry, and macroscopic structure. The identification and classification module is used to input the identification feature vector into a multi-label classifier to identify and classify the Chinese medicinal material sample based on the identification feature vector, and output the variety category and corresponding identification confidence of the Chinese medicinal material sample. The quality assessment module is used to extract specific features related to the quality grade of the Chinese medicinal material variety based on the variety identification results and the identification feature vector, input the specific features into the quality assessment model to assess the quality grade of the Chinese medicinal material sample, and output the quality grade of the Chinese medicinal material sample. The results output and management module is used to store and display the variety identification results and the quality grade assessment results. The storage and display provide report information for the identification of Chinese medicinal materials.

2. The machine vision-based Chinese medicinal herb variety analysis and identification system according to claim 1, characterized in that, The data acquisition module obtains multi-dimensional visual data of the Chinese medicinal material samples to be analyzed, including: Visible light images of the Chinese medicinal material samples were acquired using a high-resolution color area array camera; Near-infrared images of the Chinese medicinal herb samples were acquired using a near-infrared camera; The surface three-dimensional morphology data of the Chinese medicinal material sample were obtained by a three-dimensional structured light scanner.

3. The machine vision-based Chinese medicinal herb variety analysis and identification system according to claim 2, characterized in that, The data preprocessing module performs image correction, noise removal, and region of interest segmentation on the multi-dimensional visual data, including: The visible light image is subjected to distortion correction, brightness equalization, and color calibration. The near-infrared image is subjected to speckle noise removal and contrast enhancement. The three-dimensional topography data is subjected to point cloud denoising, surface smoothing, and three-dimensional registration. A deep learning semantic segmentation model is used to separate the foreground and background of the visible light image, near-infrared image and three-dimensional morphology data, extract the main body area of ​​the Chinese medicinal material as the region of interest, and perform size normalization processing.

4. The machine vision-based Chinese medicinal herb variety analysis and identification system according to claim 3, characterized in that, The feature extraction module extracts multi-scale, multi-modal discriminative feature vectors from the preprocessed visual data, including: The convolutional neural network is used to extract depth texture features, shape features, and color features from the visible light image; Internal structural features and chemical composition-related features are extracted from the near-infrared image using an independent convolutional neural network. Surface geometric features and macroscopic structural features are extracted from the three-dimensional topography data using a three-dimensional point cloud processing network. The depth texture features, shape features, color features, internal structure features, chemical composition-related features, surface geometric features, and macroscopic structure features are fused to form the discrimination feature vector.

5. The machine vision-based Chinese medicinal herb variety analysis and identification system according to claim 4, characterized in that, Based on the identification feature vector, the identification and classification module performs variety identification and classification on the Chinese medicinal material samples, including: The discriminative feature vector is input into a multi-label classifier; The multi-label classifier outputs the variety category of the Chinese medicinal material sample and the corresponding recognition confidence level.

6. The machine vision-based Chinese medicinal herb variety analysis and identification system according to claim 5, characterized in that, The quality assessment module evaluates the quality grade of the Chinese medicinal material samples based on the variety identification results and the identification feature vectors, including: Extract specific characteristics related to the quality grade of the Chinese medicinal materials; Input the specific features into the quality assessment model; The quality assessment model outputs the quality grade of the Chinese medicinal material sample.

7. The machine vision-based Chinese medicinal herb variety analysis and identification system according to claim 6, characterized in that, The result output and management module stores and displays the variety identification results and the quality grade assessment results, including: The variety identification results, the quality grade assessment results, the original visual data, and the identification feature vectors are stored in the database; The user interface displays the variety identification results, the quality grade assessment results, key feature maps, and historical batch data.

8. The machine vision-based Chinese medicinal herb variety analysis and identification system according to claim 2, characterized in that: The high-resolution color area array camera is equipped with a sensor with more than five million pixels, and its optical resolution can capture spatial details of less than 20μm per pixel.

9. The machine vision-based Chinese medicinal herb variety analysis and identification system according to claim 2, characterized in that: The near-infrared camera operates in the wavelength range of 780nm to 1100nm and is used to capture the absorption characteristics of the internal structure and chemical components of Chinese medicinal materials.

10. The machine vision-based Chinese medicinal herb variety analysis and identification system according to claim 2, characterized in that: The three-dimensional structured light scanner uses sinusoidal fringe projection and phase shift algorithm, with a point cloud density of more than 100 points per square millimeter and a depth measurement accuracy of ±50μm. During the data acquisition process, the Chinese medicinal material samples are precisely positioned and moved using a high-precision motion control platform, and uniform illumination with adjustable color temperature and brightness is provided by a multispectral light source.

Images