A method for quality evaluation of traditional Chinese medicine decoction pieces by fusing multi-modal intelligent sensory data

Abstract

The application discloses a traditional Chinese medicine decoction piece quality evaluation method fusing multi-modal intelligent sensory data, and comprises the following steps: step one, acquiring multi-modal sensory data and preprocessing; step two, performing feature extraction to obtain sensory feature vectors of each kind of sensory data; step three, performing feature-level fusion on visual feature vectors, olfactory feature vectors and gustatory feature vectors to generate a comprehensive sensory feature vector set; step four, performing feature analysis on the comprehensive sensory feature vector by using an improved t-SNE algorithm, and generating a quality evaluation result by using LightGBM; step five, analyzing the quality fluctuation trend of the traditional Chinese medicine decoction piece by using a time series analysis method, and predicting potential quality problems and future quality change trends; and step six, outputting a quality evaluation report and performing data visualization display. The application improves the accuracy of traditional Chinese medicine decoction piece quality evaluation by means of multi-modal data fusion and the improved t-SNE algorithm.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensing and data processing technology, and in particular to a method for evaluating the quality of traditional Chinese medicine decoction pieces that integrates multimodal intelligent sensory data. Background Technology

[0002] As an important component of traditional Chinese medicine (TCM), the quality of prepared herbal medicines directly affects efficacy and safety. Currently, traditional methods for evaluating the quality of prepared herbal medicines largely rely on manual sensory assessment, such as judging quality through visual inspection, smell, and taste. While these methods allow for judgment based on human experience, they have significant limitations. First, the manual assessment process heavily depends on the assessor's personal experience and subjective judgment, making it susceptible to interference from environmental factors, emotions, and other non-objective factors, resulting in insufficient accuracy and consistency in quality assessment. Second, traditional sensory evaluation methods are typically very time-consuming, especially in large-scale production processes, where the efficiency of manual assessment is particularly low, failing to meet the demands for efficient and precise quality monitoring.

[0003] With the development of science and technology, especially the advancement of sensor technology and big data analytics, quality assessment methods based on sensor data and machine learning have gradually gained attention. Many studies have begun to explore multimodal data fusion techniques to improve the automation and accuracy of quality assessment. Sensory data such as appearance, odor, and taste of traditional Chinese medicine (TCM) decoction pieces are collected using devices like visual, olfactory, and gustatory sensors, and quality assessment is achieved through data analysis and processing. However, existing sensor-based quality assessment methods still have some problems, mainly in the application of data fusion techniques and algorithm models. While some existing technologies use multimodal sensor data for quality assessment, the differences between different sensory data often cannot be fully considered during feature extraction and data fusion, resulting in fused data that may not accurately reflect the quality characteristics of TCM decoction pieces.

[0004] Furthermore, existing technologies often employ traditional machine learning models, such as decision trees and support vector machines, for quality classification and evaluation after multimodal data fusion. However, these methods suffer from drawbacks when dealing with large-scale, complex data, including high computational costs and insufficient feature selection. Their effectiveness is particularly limited when dealing with complex, non-linear data such as traditional Chinese medicine (TCM) decoction pieces. Especially in practical applications, because the quality of TCM decoction pieces is influenced by multiple factors, traditional models often struggle to effectively capture the patterns of quality variation.

[0005] Therefore, how to provide a method for evaluating the quality of traditional Chinese medicine decoction pieces that integrates multimodal intelligent sensory data is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a method for quality evaluation of traditional Chinese medicine (TCM) decoction pieces that integrates multimodal intelligent sensory data. This invention improves the accuracy and stability of TCM decoction piece quality assessment by fusing visual, olfactory, and gustatory multimodal intelligent sensory data and combining an improved t-SNE algorithm and LightGBM. Through feature-level fusion, data dimensionality reduction, and time series analysis, it can automatically identify quality fluctuations, predict potential problems, and optimize production processes, ensuring the real-time nature and accuracy of quality control. This overcomes the subjectivity and inefficiency of traditional methods, providing a more intelligent and efficient quality assessment solution.

[0007] A method for quality evaluation of traditional Chinese medicine decoction pieces that integrates multimodal intelligent sensory data according to an embodiment of the present invention includes the following steps:

[0008] Step 1: Acquire the appearance image data, odor data, and taste data of Chinese herbal medicine slices through visual sensors, olfactory sensors, and taste sensors respectively, and preprocess the acquired sensory data;

[0009] Step 2: Extract features from the preprocessed sensory data to obtain sensory feature vectors for each sensory data type; Step 3: Perform feature-level fusion of the obtained visual, olfactory, and gustatory feature vectors to generate a comprehensive sensory feature vector set; Step 4: Utilize an improved t-SNE algorithm to perform feature analysis on the comprehensive sensory feature vectors, generating low-dimensional embedding results, and use LightGBM to classify and evaluate the quality characteristics of Chinese herbal medicine slices, generating quality assessment results; Step 5: Compare the quality assessment results of the current batch with historical quality data, use time series analysis to identify and analyze the trend of quality fluctuations in Chinese herbal medicine slices, predict potential quality problems and future quality change trends, identify periodic fluctuations and adjust the production process in a timely manner, and detect abnormal fluctuations by combining residual analysis; Step 6: Output a quality assessment report and visualize the data.

[0010] Optionally, the preprocessing of the acquired sensory data specifically includes:

[0011] Gaussian filtering is used to remove noise from the appearance image data, and size normalization is performed to ensure that each image is the same size. Histogram equalization is used to adjust the brightness and contrast of the appearance image to enhance the visual effect of the appearance image and ensure that the feature extraction is not affected by the image quality.

[0012] The odor data is standardized by odor component concentration to make the odor data comparable on the same scale. Principal component analysis is used to reduce the dimensionality of the multiple odor component data, remove redundant information and retain the main components.

[0013] The sliding window method was used to smooth and denoise the taste data, reducing sudden abnormal fluctuations. The taste data was also normalized to ensure that taste data from different time periods were analyzed under the same standard.

[0014] Optionally, the step of extracting features from the preprocessed sensory data to obtain a sensory feature vector for each type of sensory data is specifically as follows:

[0015] Deep features of an appearance image are extracted using a convolutional neural network. These deep features include color, texture, edges, and shape. Local texture features in the appearance image are extracted using the histogram of oriented gradients method. The extracted visual features are then converted into visual feature vectors through a vectorization process. These visual features include both deep features and local texture features of the appearance image.

[0016] For odor data, the statistical characteristics of odor component concentrations are calculated, including mean, variance, and peak concentration, to capture the distribution characteristics of odor data. Fourier transform is then used to perform frequency domain analysis on the odor data to extract frequency features. The statistical characteristics and frequency features are then concatenated to form an odor feature vector, which represents the comprehensive characteristics of the odor data.

[0017] Statistical features of taste data are extracted, including mean, standard deviation and peak concentration. Wavelet transform is used to perform frequency domain analysis on the taste data to extract frequency domain features in different frequency ranges. The statistical features and frequency domain features of the taste data are concatenated to generate a taste feature vector.

[0018] Optionally, the feature-level fusion specifically includes:

[0019] Z-score standardization was applied to the visual feature vector, olfactory feature vector, and gustatory feature vector, adjusting the mean of each feature vector to 0 and the variance to 1. The standardized feature vector can eliminate the scale difference between different sensory data and ensure the fairness of the subsequent fusion process.

[0020] The feature vectors of vision, smell, and taste are sequentially concatenated into a fused feature vector. The fused feature vector contains data from the three senses, with the data dimensions of each sense being directly combined side by side. This method can preserve the detailed information of each sense data and ensure that the original features are not lost during the fusion of sensory data.

[0021] The fused feature vectors are subjected to dimensionality reduction using linear discriminant analysis to remove noise and redundant information, resulting in a comprehensive sensory feature vector set.

[0022] Optionally, the improved t-SNE algorithm is specifically as follows:

[0023] Based on a comprehensive sensory feature vector set, the improved t-SNE algorithm constructs a distance matrix for high-dimensional data by calculating the Manhattan distance between each pair of high-dimensional sample points, where each high-dimensional sample point is a comprehensive sensory feature vector.

[0024] Based on the distance matrix, the improved t-SNE algorithm uses a Gaussian distribution to calculate the similarity between each pair of high-dimensional sample points and converts it into conditional probabilities, which represent the similarity between two samples given one sample in high-dimensional space.

[0025] The distance between sample points in the low-dimensional space is matched with the corresponding conditional probability in the high-dimensional space, thereby maintaining the similarity and difference in the original data. Sample points with high conditional probability in the high-dimensional space are closer in the low-dimensional space, and sample points with low conditional probability are farther away in the low-dimensional space.

[0026] By calculating the Kullback-Leibler divergence, the difference in conditional probabilities between the high-dimensional space and the low-dimensional space is minimized, and the gradient of the Kullback-Leibler divergence with respect to the position of each sample point in the low-dimensional space is calculated. The gradient represents the direction and magnitude in which each sample point should move in the low-dimensional space, so that the distance between sample points in the low-dimensional space better conforms to the similarity relationship in the high-dimensional space.

[0027] Based on the calculated gradient, the improved t-SNE algorithm adjusts the position of each sample point in the low-dimensional space. The adjustment magnitude is controlled by the learning rate, which represents the step size of each gradient update. In the initial stage, a learning rate greater than a set threshold is used to accelerate the convergence process. When convergence is near, a learning rate less than a set threshold is used to adjust the position of the sample points. The set threshold learning rate is 0.01 to avoid oscillation problems caused by excessively large step sizes.

[0028] Through gradient updates, low-dimensional embedding results are generated. Based on the low-dimensional embedding results, LightGBM is used to classify and evaluate the quality characteristics of Chinese herbal medicine slices, generating quality assessment results.

[0029] Optionally, the step of using LightGBM to classify and evaluate the quality characteristics of traditional Chinese medicine decoction pieces based on the low-dimensional embedding results, and generating quality evaluation results, specifically involves:

[0030] In a low-dimensional space, the low-dimensional embedding results are used as input features to construct a quality assessment dataset for traditional Chinese medicine (TCM) decoction pieces. This dataset includes the low-dimensional spatial coordinates, additional features, and corresponding preliminary predicted quality labels for each TCM decoction piece. The additional features include historical quality data and production environment data, and the quality labels include excellent, good, average, and poor. The quality assessment dataset is then preprocessed. This preprocessing includes standardizing the low-dimensional spatial coordinates to ensure that features across different TCM decoction pieces have the same scale, and handling missing values ​​and outlier detection for the additional features. Finally, the preprocessed quality assessment dataset is input into the LightGBM classification model.

[0031] The hyperparameters of the LightGBM model were optimized using K-fold cross-validation on both the training and validation sets. These hyperparameters included the learning rate, tree depth, number of leaf nodes, subsample ratio, and maximum number of features.

[0032] The LightGBM model is trained using gradient boosting decision trees to learn the relationship between low-dimensional embedding results and the quality labels of Chinese herbal medicine (TCM) decoction pieces. The LightGBM model improves classification performance by integrating multiple decision trees and optimizes nodes using a leaf node splitting strategy, enabling the model to better capture complex relationships between features. The trained LightGBM model is then used to assess the quality of new TCM decoction pieces. Through LightGBM feature importance assessment, the features with the greatest impact on the quality assessment are identified, and key quality features are accurately classified and evaluated. Based on the quality classification labels output by the LightGBM model, quality assessment results for the TCM decoction pieces are generated, including excellent, good, average, and poor.

[0033] Optionally, the step of comparing the quality assessment results of the current batch with historical quality data, using time series analysis to identify and analyze the trend of quality fluctuations in Chinese herbal medicine slices, predicting potential quality problems and future quality change trends, identifying periodic fluctuations and adjusting the production process in a timely manner, and combining residual analysis to detect abnormal fluctuations, specifically includes:

[0034] Based on the quality assessment results of the current batch and combined with the quality data of historical batches, a complete quality time series is formed, which includes the quality assessment results at different time points.

[0035] The STL time series decomposition method was used to decompose the quality time series, identify long-term trends, seasonal fluctuations and random fluctuations, and identify quality fluctuation cycles in the production process through periodic analysis.

[0036] LSTM neural networks are used to predict potential quality problems and future quality fluctuation trends, identify periodic fluctuations, and adjust the production process in a timely manner.

[0037] Calculate the residual of the quality time series, which is the difference between the predicted value and the actual value, and use the standard deviation method to detect outliers in the residual and mark the time points of abnormal quality fluctuations.

[0038] Based on the predicted quality fluctuation trends and the analysis results of abnormal fluctuations, adjust the key process parameters that may affect quality during production, including temperature, humidity, and raw material ratio.

[0039] Adjust production schedules to avoid producing large quantities of Chinese herbal medicine slices during periods of high risk of quality fluctuations.

[0040] Optionally, the output of the quality assessment report and the data visualization display specifically include:

[0041] A quality assessment report is generated based on the quality assessment results, the analysis results of quality fluctuation trends and abnormal fluctuations.

[0042] Heatmaps are used to show the distribution of different Chinese herbal medicine pieces in low-dimensional space, and quality classification is distinguished by color.

[0043] Line graphs are used to show the trends and abnormal fluctuations in the quality of Chinese herbal medicine slices, helping decision-makers to quickly identify patterns in quality fluctuations.

[0044] By comparing the quality assessment results of the current batch with those of historical batches using bar charts, production personnel can identify potential quality differences.

[0045] The beneficial effects of this invention are:

[0046] 1. This invention significantly improves the objectivity and accuracy of quality evaluation of traditional Chinese medicine (TCM) decoction pieces by integrating multimodal intelligent sensory data, including visual, olfactory, and gustatory data. Traditional TCM decoction piece quality evaluation methods rely on manual sensory evaluation, which is significantly subjective and inconsistent. In contrast, this invention automatically collects sensory data through sensors, avoiding the biases of manual evaluation and providing more stable and reliable quality evaluation results.

[0047] 2. This invention employs advanced techniques in sensory data preprocessing and feature extraction, such as Gaussian filtering, principal component analysis, and Fourier transform, effectively removing noise and redundant information, thereby ensuring that the quality features of each type of sensory data are accurately extracted and preserved. The application of these techniques makes the data clearer, facilitating subsequent fusion and analysis.

[0048] 3. This invention employs a feature-level fusion method, combining feature vectors from vision, smell, and taste into a comprehensive sensory feature vector set. This fusion method preserves detailed information for each sensory data point and avoids data loss, enabling the fused data to comprehensively reflect the quality characteristics of traditional Chinese medicine decoction pieces. Simultaneously, an improved t-SNE algorithm is used for data dimensionality reduction, effectively mapping complex high-dimensional data to a low-dimensional space for easier intuitive understanding and analysis.

[0049] 4. This invention effectively improves the accuracy and stability of quality assessment by combining the LightGBM algorithm for the classification and evaluation of quality features. The LightGBM algorithm demonstrates good performance in handling large-scale data and complex nonlinear problems, and can identify the quality characteristics of Chinese herbal medicine slices and generate accurate quality assessment results. In particular, it can effectively uncover potential patterns when processing multimodal sensory data.

[0050] 5. This invention also introduces a time series analysis method. By comparing the quality assessment results with historical data, it is possible to identify and analyze the trends in the quality fluctuations of traditional Chinese medicine decoction pieces. Using time series analysis, potential quality problems and future quality change trends can be predicted in advance, and abnormal fluctuations can be detected in a timely manner through residual analysis. This provides a reliable basis for quality control and optimization in the production process, enabling dynamic adjustments to the production process and avoiding the production of large quantities of substandard products during periods of high risk of quality fluctuations.

[0051] 6. This invention uses data visualization technology to transform complex quality assessment results into easily understandable and analyzable graphics, helping production personnel and decision-makers to quickly identify quality problems and trends and take corresponding control measures, thereby ensuring the quality stability and continuous pass rate of traditional Chinese medicine decoction pieces in the production process. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 This is an overall flowchart of a method for evaluating the quality of traditional Chinese medicine decoction pieces that integrates multimodal intelligent sensory data, as proposed in this invention.

[0054] Figure 2 This is a flowchart illustrating the data dimensionality reduction process using the improved t-SNE algorithm for a method of evaluating the quality of traditional Chinese medicine decoction pieces that integrates multimodal intelligent sensory data, as proposed in this invention.

[0055] Figure 3This is a schematic diagram of the LightGBM quality assessment model structure for a method of evaluating the quality of traditional Chinese medicine decoction pieces that integrates multimodal intelligent sensory data, as proposed in this invention. Detailed Implementation

[0056] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0057] refer to Figure 1-3 A method for quality evaluation of traditional Chinese medicine decoction pieces that integrates multimodal intelligent sensory data includes the following steps:

[0058] This embodiment includes the following steps:

[0059] Step 1: Acquire the appearance image data, odor data, and taste data of Chinese herbal medicine slices through visual sensors, olfactory sensors, and taste sensors respectively, and preprocess the acquired sensory data;

[0060] Step 2: Extract features from the preprocessed sensory data to obtain sensory feature vectors for each sensory data type; Step 3: Perform feature-level fusion of the obtained visual, olfactory, and gustatory feature vectors to generate a comprehensive sensory feature vector set; Step 4: Utilize an improved t-SNE algorithm to perform feature analysis on the comprehensive sensory feature vectors, generating low-dimensional embedding results, and use LightGBM to classify and evaluate the quality characteristics of Chinese herbal medicine slices, generating quality assessment results; Step 5: Compare the quality assessment results of the current batch with historical quality data, use time series analysis to identify and analyze the trend of quality fluctuations in Chinese herbal medicine slices, predict potential quality problems and future quality change trends, identify periodic fluctuations and adjust the production process in a timely manner, and detect abnormal fluctuations by combining residual analysis; Step 6: Output a quality assessment report and visualize the data.

[0061] This invention provides an automated and precise method for evaluating the quality of traditional Chinese medicine (TCM) decoction pieces by integrating multimodal intelligent sensory data from vision, smell, and taste. Traditional quality assessment relies on manual sensory evaluation, which is inherently subjective and inefficient. This invention eliminates human interference through automated data collection and analysis, ensuring the objectivity and consistency of the assessment. The improved t-SNE algorithm effectively reduces the dimensionality of high-dimensional sensory data, generating low-dimensional embedding results that intuitively display the quality characteristics of TCM decoction pieces. Further, a LightGBM model is used for precise quality assessment and classification. This method can monitor quality fluctuations in the production process in real time, promptly identify potential quality problems, optimize production processes, and significantly improve the efficiency and stability of quality control.

[0062] In this embodiment, the preprocessing of the acquired sensory data specifically includes:

[0063] Gaussian filtering is used to remove noise from the appearance image data, and size normalization is performed to ensure that each image is the same size. Histogram equalization is used to adjust the brightness and contrast of the appearance image to enhance the visual effect of the appearance image and ensure that the feature extraction is not affected by the image quality.

[0064] The odor data is standardized by odor component concentration to make the odor data comparable on the same scale. Principal component analysis is used to reduce the dimensionality of the multiple odor component data, remove redundant information and retain the main components.

[0065] The sliding window method was used to smooth and denoise the taste data, reducing sudden abnormal fluctuations. The taste data was also normalized to ensure that taste data from different time periods were analyzed under the same standard.

[0066] This invention employs advanced preprocessing methods such as Gaussian filtering, principal component analysis, and sliding window method to denoise, normalize, and reduce the dimensionality of appearance images, odor data, and taste data, ensuring high-quality sensory data input. These preprocessing steps effectively reduce data noise and redundant information, eliminate scale differences between different data sources, and provide a stable and reliable foundation for subsequent feature extraction and fusion. The preprocessed data is cleaner and more standardized, which helps improve the ability to extract quality features in subsequent analysis and ensures the efficiency and accuracy of quality assessment.

[0067] In this embodiment, the step of extracting features from the preprocessed sensory data to obtain a sensory feature vector for each type of sensory data is specifically as follows:

[0068] Deep features of an appearance image are extracted using a convolutional neural network. These deep features include color, texture, edges, and shape. Local texture features in the appearance image are extracted using the histogram of oriented gradients method. The extracted visual features are then converted into visual feature vectors through a vectorization process. These visual features include both deep features and local texture features of the appearance image.

[0069] For odor data, the statistical characteristics of odor component concentrations are calculated, including mean, variance, and peak concentration, to capture the distribution characteristics of odor data. Fourier transform is then used to perform frequency domain analysis on the odor data to extract frequency features. The statistical characteristics and frequency features are then concatenated to form an odor feature vector, which represents the comprehensive characteristics of the odor data.

[0070] Statistical features of taste data are extracted, including mean, standard deviation and peak concentration. Wavelet transform is used to perform frequency domain analysis on the taste data to extract frequency domain features in different frequency ranges. The statistical features and frequency domain features of the taste data are concatenated to generate a taste feature vector.

[0071] This invention utilizes convolutional neural networks to extract deep features from appearance images, such as color, texture, edges, and shape, while simultaneously extracting local texture features through histograms of oriented gradients. This refined feature extraction method effectively captures detailed information in appearance images, providing a solid foundation for subsequent quality assessment. For odor data, this invention further enriches the quality representation of sensory data by calculating the statistical characteristics of odor components and extracting frequency features through frequency domain analysis. For taste data, this invention employs wavelet transform to extract frequency domain features, enabling in-depth analysis of quality variations across different frequency ranges. These techniques ensure that the feature vectors of each type of sensory data accurately represent its quality characteristics, significantly improving the accuracy and effectiveness of data processing.

[0072] In this embodiment, the feature-level fusion specifically refers to:

[0073] Z-score normalization was applied to the visual feature vector, olfactory feature vector, and gustatory feature vector:

[0074] ;

[0075] in, The feature vectors are Z-score standardized. This represents the original value of the i-th sample on the j-th sensory feature. Let be the weight of the j-th sensory feature, reflecting the importance of each sensory data point. Let be the mean of the j-th sensory feature. Let be the standard deviation of the j-th sensory characteristic.

[0076] The mean of each feature vector is adjusted to 0 and the variance is adjusted to 1. The standardized feature vectors can eliminate the scale difference between different sensory data and ensure the fairness of the subsequent fusion process.

[0077] The feature vectors of vision, smell, and taste are sequentially concatenated into a fused feature vector. The fused feature vector contains data from the three senses, with the data dimensions of each sense being directly combined side by side. This method can preserve the detailed information of each sense data and ensure that the original features are not lost during the fusion of sensory data.

[0078] The fused feature vectors are subjected to dimensionality reduction using linear discriminant analysis to remove noise and redundant information, resulting in a comprehensive sensory feature vector set.

[0079] By employing feature-level fusion, this invention unifies the processing of visual, olfactory, and gustatory feature vectors, resolving the scale difference issue between multimodal data. Z-score normalization is used to standardize the data from each sensory sensory level, ensuring fair comparison of data from different sensory sensory systems at the same scale. The feature vectors from vision, smell, and taste are concatenated sequentially, preserving detailed information from each sensory data point and ensuring no loss of original features during data fusion. Linear discriminant analysis is used to reduce the dimensionality of the fused feature vectors, effectively removing noise and redundant information. The resulting comprehensive sensory feature vector set accurately reflects the overall quality characteristics of traditional Chinese medicine decoction pieces, providing more precise input data for subsequent quality assessment.

[0080] In this embodiment, the improved t-SNE algorithm is specifically as follows:

[0081] Based on a comprehensive sensory feature vector set, the improved t-SNE algorithm constructs a distance matrix for high-dimensional data by calculating the Manhattan distance between each pair of high-dimensional sample points, where each high-dimensional sample point is a comprehensive sensory feature vector.

[0082] Based on the distance matrix, the improved t-SNE algorithm uses a Gaussian distribution to calculate the similarity between each pair of high-dimensional sample points and converts it into conditional probabilities, which represent the similarity between two samples given one sample in high-dimensional space.

[0083] The distance between sample points in the low-dimensional space is matched with the corresponding conditional probability in the high-dimensional space, thereby maintaining the similarity and difference in the original data. Sample points with high conditional probability in the high-dimensional space are closer in the low-dimensional space, and sample points with low conditional probability are farther away in the low-dimensional space.

[0084] By calculating the Kullback-Leibler divergence, the difference in conditional probabilities between the high-dimensional and low-dimensional spaces is minimized:

[0085] ;

[0086] in, The Kullback-Leibler divergence between the high-dimensional space and the low-dimensional space. Let be the conditional probability between the i-th and j-th sample points in the high-dimensional space. Let be the conditional probability between the i-th and j-th sample points in the low-dimensional space.

[0087] Calculate the gradient of the Kullback-Leibler divergence with respect to the position of each sample point in the low-dimensional space. The gradient represents the direction and magnitude in which each sample point should move in the low-dimensional space so that the distance between sample points in the low-dimensional space better matches the similarity relationship in the high-dimensional space.

[0088] Based on the calculated gradient, the improved t-SNE algorithm adjusts the position of each sample point in the low-dimensional space. The adjustment magnitude is controlled by the learning rate, which represents the step size of each gradient update. In the initial stage, a learning rate greater than a set threshold is used to accelerate the convergence process. When convergence is near, a learning rate less than a set threshold is used to adjust the position of the sample points. The set threshold learning rate is 0.01 to avoid oscillation problems caused by excessively large step sizes.

[0089] Through gradient updates, low-dimensional embedding results are generated. Based on the low-dimensional embedding results, LightGBM is used to classify and evaluate the quality characteristics of Chinese herbal medicine slices, generating quality assessment results.

[0090] The improved t-SNE algorithm effectively preserves the similarity between samples during dimensionality reduction of high-dimensional sensory data by calculating the Manhattan distance between each pair of sample points and converting it into conditional probabilities. Compared with traditional dimensionality reduction algorithms, the improved t-SNE better preserves local structure and reduces distortion caused by dimensionality reduction when optimizing the distribution of points in the low-dimensional space. By minimizing the Kullback-Leibler divergence, the algorithm can accurately optimize the distribution of points in the low-dimensional space, making similar sample points more clustered, thereby improving the data dimensionality reduction and visualization effects. In addition, the gradient update method allows the sample points to be gradually adjusted in the low-dimensional space, and the final low-dimensional embedding results are more consistent with the inherent structure of the high-dimensional data, improving the accuracy of quality assessment and visualization effects.

[0091] In this embodiment, the process of classifying and evaluating the quality characteristics of traditional Chinese medicine decoction pieces using LightGBM based on low-dimensional embedding results to generate quality evaluation results specifically includes:

[0092] In a low-dimensional space, the low-dimensional embedding results are used as input features to construct a quality assessment dataset for traditional Chinese medicine (TCM) decoction pieces. This dataset includes the low-dimensional spatial coordinates, additional features, and corresponding preliminary predicted quality labels for each TCM decoction piece. The additional features include historical quality data and production environment data, and the quality labels include excellent, good, average, and poor. The quality assessment dataset is then preprocessed. This preprocessing includes standardizing the low-dimensional spatial coordinates to ensure that features across different TCM decoction pieces have the same scale, and handling missing values ​​and outlier detection for the additional features. Finally, the preprocessed quality assessment dataset is input into the LightGBM classification model.

[0093] The hyperparameters of the LightGBM model were optimized using K-fold cross-validation on both the training and validation sets. These hyperparameters included the learning rate, tree depth, number of leaf nodes, subsample ratio, and maximum number of features.

[0094] The LightGBM model is trained using gradient boosting decision trees to learn the relationship between low-dimensional embedding results and the quality labels of Chinese herbal medicine (TCM) decoction pieces. The LightGBM model improves classification performance by integrating multiple decision trees and optimizes nodes using a leaf node splitting strategy, enabling the model to better capture complex relationships between features. The trained LightGBM model is then used to assess the quality of new TCM decoction pieces. Through LightGBM feature importance assessment, the features with the greatest impact on the quality assessment are identified, and key quality features are accurately classified and evaluated. Based on the quality classification labels output by the LightGBM model, quality assessment results for the TCM decoction pieces are generated, including excellent, good, average, and poor.

[0095] By using LightGBM based on low-dimensional embedding results for quality assessment, this invention fully utilizes the feature information obtained in the low-dimensional space and combines it with the powerful classification capabilities of LightGBM to achieve accurate quality assessment of traditional Chinese medicine (TCM) decoction pieces. Compared with traditional classification algorithms, LightGBM is more efficient in processing large-scale data and can effectively capture complex nonlinear relationships in the data. By training the LightGBM model, not only can accurate quality labels be assigned to each TCM decoction piece, but also key features with the greatest impact on quality assessment can be extracted through feature importance evaluation, thereby further optimizing quality control. This method improves the reliability of quality assessment, reduces the need for human intervention, and can quickly respond to quality fluctuations during the production process.

[0096] In this embodiment, the process of comparing the quality assessment results of the current batch with historical quality data, using time series analysis to identify and analyze the trend of quality fluctuations in Chinese herbal medicine slices, predicting potential quality problems and future quality change trends, identifying periodic fluctuations and adjusting the production process in a timely manner, and detecting abnormal fluctuations in conjunction with residual analysis, specifically includes:

[0097] Based on the quality assessment results of the current batch and combined with the quality data of historical batches, a complete quality time series is formed, which includes the quality assessment results at different time points.

[0098] The STL time series decomposition method was used to decompose the quality time series, identify long-term trends, seasonal fluctuations and random fluctuations, and identify quality fluctuation cycles in the production process through periodic analysis.

[0099] LSTM neural networks are used to predict potential quality problems and future quality fluctuation trends, identify periodic fluctuations, and adjust the production process in a timely manner.

[0100] Calculate the residual of the quality time series, which is the difference between the predicted value and the actual value, and use the standard deviation method to detect outliers in the residual and mark the time points of abnormal quality fluctuations.

[0101] Based on the predicted quality fluctuation trends and the analysis results of abnormal fluctuations, adjust the key process parameters that may affect quality during production, including temperature, humidity, and raw material ratio.

[0102] Adjust production schedules to avoid producing large quantities of Chinese herbal medicine slices during periods of high risk of quality fluctuations.

[0103] By combining time series analysis methods, this invention can monitor and predict the quality fluctuation trend of Chinese herbal medicine slices in real time, effectively identifying potential quality problems. Through STL time series decomposition, this invention can extract long-term trends, seasonal fluctuations, and random fluctuation components, helping to analyze potential periodic quality fluctuations during production. Using LSTM neural networks, this invention can accurately predict future quality change trends and promptly detect the risk of quality decline. Combined with residual analysis, it can effectively detect and mark anomalies in quality fluctuations, thereby identifying potential quality problems in advance. Through real-time adjustments to the production process, this invention can avoid producing large quantities of substandard Chinese herbal medicine slices during high-risk periods, improving the stability of the production process and the quality pass rate.

[0104] In this embodiment, the step of outputting a quality assessment report and visualizing the data specifically includes:

[0105] A quality assessment report is generated based on the quality assessment results, the analysis results of quality fluctuation trends and abnormal fluctuations.

[0106] Heatmaps are used to show the distribution of different Chinese herbal medicine pieces in low-dimensional space, and quality classification is distinguished by color.

[0107] Line graphs are used to show the trends and abnormal fluctuations in the quality of Chinese herbal medicine slices, helping decision-makers to quickly identify patterns in quality fluctuations.

[0108] By comparing the quality assessment results of the current batch with those of historical batches using bar charts, production personnel can identify potential quality differences.

[0109] This invention utilizes data visualization technology to transform quality assessment results and quality fluctuation trends into intuitive graphics, helping decision-makers quickly understand the quality status of traditional Chinese medicine (TCM) decoction pieces. Using visualization methods such as heatmaps, line graphs, and bar charts, the distribution and quality change trends of TCM decoction pieces of different quality grades can be clearly displayed, providing strong decision support for production personnel. Through the output of quality reports and real-time monitoring, quality fluctuations occurring during the production process can be quickly responded to and adjusted, effectively ensuring the quality stability of the production process. This visualization method makes quality control more transparent, improving decision-making efficiency and product qualification rate during the production process.

[0110] Example 1:

[0111] To verify the feasibility of this invention in practice, it was applied to the quality assessment and control tasks of a traditional Chinese medicine (TCM) decoction piece manufacturing enterprise. This enterprise produces various TCM decoction pieces, among which the quality of wolfberry decoction pieces fluctuates significantly, mainly manifested in uneven appearance and color, deviations in odor concentration, and inconsistencies in taste intensity. Due to these quality issues, the enterprise faces considerable quality control pressure during production, and traditional manual sensory evaluation methods are insufficient to meet the quality monitoring needs of large-scale production. Manual evaluation is not only inefficient but also prone to missing potential quality problems, leading to the production of substandard products and impacting market competitiveness.

[0112] To overcome these problems, this invention proposes a method for quality assessment of traditional Chinese medicine (TCM) decoction pieces that integrates multimodal intelligent sensory data. By introducing intelligent sensory sensors, this method can collect real-time images of the appearance, aroma, and taste of wolfberry decoction pieces. Intelligent algorithms are then used for quality assessment, achieving automation and precision in the process, and enabling real-time monitoring of quality fluctuations during production. Through data processing and analysis, the system effectively improves the accuracy and efficiency of quality assessment, helping production personnel to promptly identify and adjust problems.

[0113] In this embodiment, two batches of wolfberry slices from June and July 2023 were selected for the experiment. The goal of the experiment was to verify the effectiveness of the method of the present invention in improving the accuracy of quality assessment, production efficiency, and the accuracy of quality fluctuation prediction. During the experiment, the company installed visual, olfactory, and gustatory sensors in the production line to collect sensory data of each batch of wolfberry slices in real time during the production process. The collected raw data underwent preprocessing using techniques such as Gaussian filtering, principal component analysis, and sliding window method to ensure data cleanliness and standardization. After feature extraction, deep features such as color, texture, edge, and shape were extracted from the appearance image data using a convolutional neural network; frequency domain features were extracted from the odor data using Fourier transform and statistical feature analysis; and frequency domain analysis was performed on the gustatory data using wavelet transform. Subsequently, all feature data were merged into a comprehensive sensory feature vector set using a feature-level fusion method, providing accurate input data for subsequent quality assessment.

[0114] To further process these feature data, an improved t-SNE algorithm was used for dimensionality reduction, mapping high-dimensional data to a low-dimensional space, making the data easier to understand and analyze in the lower-dimensional space. Using the dimensionality-reduced low-dimensional embedding results, the LightGBM algorithm was employed for quality assessment, generating quality assessment results (e.g., excellent, good, medium, poor) for each batch of medicinal slices. This method successfully replaced traditional manual sensory evaluation, not only improving the accuracy of the assessment but also accelerating quality control during the production process.

[0115] Table 1 Comparative Data Table of Quality Assessment of Traditional Chinese Medicine Decoction Pieces

[0116]

[0117] As shown in Table 1, the quality assessment results using the method of this invention are significantly superior to those of traditional manual assessment methods. In experiments conducted in June and July 2023, the quality results assessed using manual methods were rated as "Good" and "Average," while the results assessed using the method of this invention were rated as "Excellent" and "Good," with misclassification rates of 4% and 5%, respectively, significantly lower than the 19% and 22% of manual assessment. Furthermore, the accuracy rates for predicting quality fluctuation trends using the method of this invention were 91% and 93%, respectively, while the accuracy rates for manual assessment methods were only 70% and 68%. This difference indicates that the method of this invention can not only conduct more accurate quality assessments but also effectively predict quality fluctuations during the production process, providing production personnel with accurate early warning information and helping to adjust production processes in a timely manner.

[0118] Regarding production process adjustments, traditional methods require multiple adjustments, specifically 12 and 15 times, while the method of this invention requires only 4 and 5 adjustments, significantly reducing the number of manual interventions during production and improving the level of automation. Furthermore, experiments have verified the improvement in production efficiency; the production efficiency using the method of this invention is approximately 30%-40% higher than that of manual evaluation methods, and the time required for quality assessment during production is greatly shortened, reducing production costs.

[0119] Table 1 shows that the method of the present invention demonstrates significant advantages in terms of accuracy of quality assessment, fluctuation prediction, and production efficiency, successfully solving the problems of subjectivity and inefficiency of traditional methods. Through the method of the present invention, the quality stability of the production process of traditional Chinese medicine decoction pieces is significantly improved, not only increasing the product qualification rate but also reducing quality fluctuations during production, providing enterprises with a more efficient and intelligent quality control solution.

[0120] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for evaluating the quality of traditional Chinese medicine decoction pieces by integrating multimodal intelligent sensory data, characterized in that, Includes the following steps: Step 1: Acquire the appearance image data, odor data, and taste data of Chinese herbal medicine slices through visual sensors, olfactory sensors, and taste sensors respectively, and preprocess the acquired sensory data; Step 2: Extract features from the preprocessed sensory data to obtain sensory feature vectors for each type of sensory data; Step 3: Perform feature-level fusion of the obtained visual, olfactory, and gustatory feature vectors to generate a comprehensive sensory feature vector set; Step 4: Analyze the comprehensive sensory feature vectors using the improved t-SNE algorithm to generate low-dimensional embedding results, and use LightGBM to classify and evaluate the quality characteristics of Chinese herbal medicine slices to generate quality assessment results; The improved t-SNE algorithm is specifically as follows: Based on a comprehensive sensory feature vector set, the improved t-SNE algorithm constructs a distance matrix for high-dimensional data by calculating the Manhattan distance between each pair of high-dimensional sample points, where each high-dimensional sample point is a comprehensive sensory feature vector. Based on the distance matrix, the improved t-SNE algorithm uses a Gaussian distribution to calculate the similarity between each pair of high-dimensional sample points and converts it into conditional probabilities, which represent the similarity between two samples given one sample in high-dimensional space. The distance between sample points in the low-dimensional space is matched with the corresponding conditional probability in the high-dimensional space. Sample points with high conditional probability in the high-dimensional space are closer in the low-dimensional space, and sample points with low conditional probability are farther away in the low-dimensional space. By calculating the Kullback-Leibler divergence, the difference in conditional probabilities between the high-dimensional and low-dimensional spaces is minimized, and the gradient of the Kullback-Leibler divergence with respect to the position of each sample point in the low-dimensional space is calculated, whereby the gradient represents the direction and magnitude in which each sample point should move in the low-dimensional space. Based on the calculated gradient, the improved t-SNE algorithm adjusts the position of each sample point in the low-dimensional space. The adjustment magnitude is controlled by the learning rate, which represents the step size of each gradient update. In the initial stage, a learning rate greater than a set threshold is used to accelerate the convergence process. When convergence is near, a learning rate less than a set threshold is used to adjust the position of the sample points. The set threshold learning rate is 0.

01. Through gradient updates, low-dimensional embedding results are generated, and LightGBM is used to classify and evaluate the quality characteristics of Chinese herbal medicine slices based on the low-dimensional embedding results, generating quality evaluation results. The method of using LightGBM to classify and evaluate the quality characteristics of traditional Chinese medicine decoction pieces based on low-dimensional embedding results, and generating quality evaluation results, specifically includes: In a low-dimensional space, the low-dimensional embedding results are used as input features to construct a quality assessment dataset for traditional Chinese medicine (TCM) decoction pieces. This dataset includes the low-dimensional spatial coordinates, additional features, and corresponding preliminary predicted quality labels for each TCM decoction piece. The additional features include historical quality data and production environment data, and the quality labels include excellent, good, average, and poor. The quality assessment dataset is preprocessed, including standardizing the low-dimensional spatial coordinates and handling missing values ​​and outlier detection for the additional features. The preprocessed quality assessment dataset is then input into the LightGBM classification model. The hyperparameters of the LightGBM model were optimized using K-fold cross-validation on both the training and validation sets. These hyperparameters included the learning rate, tree depth, number of leaf nodes, subsample ratio, and maximum number of features. The LightGBM model is trained using gradient boosting decision trees to learn the relationship between low-dimensional embedding results and the quality labels of Chinese herbal medicine (TCM) decoction pieces. The LightGBM model improves classification performance by integrating multiple decision trees and optimizes nodes using a leaf node splitting strategy. The trained LightGBM model is then used to assess the quality of new TCM decoction pieces. Through LightGBM feature importance assessment, the features with the greatest impact on TCM decoction piece quality assessment are identified, and key quality features are accurately classified and assessed. Based on the quality classification labels output by the LightGBM model, quality assessment results for TCM decoction pieces are generated, including excellent, good, average, and poor. Step five: The quality assessment results of the current batch are compared with historical quality data. Time series analysis is used to identify and analyze the trend of quality fluctuations in TCM decoction pieces, predict potential quality problems and future quality change trends, identify periodic fluctuations, and adjust the production process in a timely manner. Residual analysis is combined to detect abnormal fluctuations. Step six: A quality assessment report is output and the data is visualized.

2. The method for evaluating the quality of traditional Chinese medicine decoction pieces by integrating multimodal intelligent sensory data according to claim 1, characterized in that, The preprocessing of the acquired sensory data specifically includes: The Gaussian filtering method is used to remove noise from the appearance image data, the size is normalized, and the brightness and contrast of the appearance image are adjusted by histogram equalization technology to enhance the visual effect of the appearance image. The odor data is standardized by odor component concentration, and the dimensionality of the multiple odor component data is reduced by principal component analysis to remove redundant information and retain the main components. The sliding window method was used to smooth and denoise the taste data, and the taste data was normalized.

3. The method for evaluating the quality of traditional Chinese medicine decoction pieces by integrating multimodal intelligent sensory data according to claim 1, characterized in that, The process of extracting features from the preprocessed sensory data to obtain a sensory feature vector for each type of sensory data is as follows: Deep features of an appearance image are extracted using a convolutional neural network. These deep features include color, texture, edges, and shape. Local texture features in the appearance image are extracted using the histogram of oriented gradients method. The extracted visual features are then converted into visual feature vectors through a vectorization process. These visual features include both deep features and local texture features of the appearance image. For odor data, the statistical characteristics of odor component concentrations are calculated, including mean, variance and peak concentration, to capture the distribution characteristics of odor data. Fourier transform is used to perform frequency domain analysis on odor data to extract frequency features from the odor data. The statistical characteristics and frequency features are then concatenated to form an odor feature vector. Statistical features of taste data are extracted, including mean, standard deviation and peak concentration. Wavelet transform is used to perform frequency domain analysis on the taste data to extract frequency domain features in different frequency ranges. The statistical features and frequency domain features of the taste data are concatenated to generate a taste feature vector.

4. The method for evaluating the quality of traditional Chinese medicine decoction pieces by integrating multimodal intelligent sensory data according to claim 1, characterized in that, The feature-level fusion specifically refers to: Z-score standardization was applied to the visual feature vector, olfactory feature vector, and gustatory feature vector, adjusting the mean of each feature vector to 0 and the variance to 1. The feature vectors of vision, smell and taste are connected in sequence to form a fused feature vector, which contains data from the three senses, with the data dimensions of each sense being directly combined side by side. The fused feature vectors are subjected to dimensionality reduction using linear discriminant analysis to remove noise and redundant information, resulting in a comprehensive sensory feature vector set.

5. The method for evaluating the quality of traditional Chinese medicine decoction pieces by integrating multimodal intelligent sensory data according to claim 1, characterized in that, The process involves comparing the current batch's quality assessment results with historical quality data, using time series analysis to identify and analyze trends in the quality fluctuations of traditional Chinese medicine decoction pieces, predicting potential quality problems and future quality change trends, identifying periodic fluctuations and adjusting the production process in a timely manner, and detecting abnormal fluctuations using residual analysis. Specifically: Based on the quality assessment results of the current batch and combined with the quality data of historical batches, a complete quality time series is formed, which includes the quality assessment results at different points in time. The STL time series decomposition method was used to decompose the quality time series, identify long-term trends, seasonal fluctuations and random fluctuations, and identify quality fluctuation cycles in the production process through periodic analysis. LSTM neural networks are used to predict potential quality problems and future quality fluctuation trends, identify periodic fluctuations, and adjust the production process in a timely manner. Calculate the residual of the quality time series, which is the difference between the predicted value and the actual value, and use the standard deviation method to detect outliers in the residual and mark the time points of abnormal quality fluctuations. Based on the predicted quality fluctuation trends and the analysis results of abnormal fluctuations, adjust the key process parameters that may affect quality during production, including temperature, humidity, and raw material ratio.

6. The method for evaluating the quality of traditional Chinese medicine decoction pieces by integrating multimodal intelligent sensory data according to claim 1, characterized in that, The output quality assessment report and data visualization display are as follows: A quality assessment report is generated based on the quality assessment results, the analysis results of quality fluctuation trends and abnormal fluctuations. Heatmaps are used to show the distribution of different Chinese herbal medicine pieces in low-dimensional space, and quality classification is distinguished by color. Line graphs are used to illustrate the trends and abnormal fluctuations in the quality of Chinese herbal medicine slices. The quality assessment results of the current batch are compared with those of historical batches using a bar chart.

Images