An intelligent screening method for traditional Chinese medicine decoction pieces based on multi-modal data fusion

Abstract

The present application relates to the technical field of traditional Chinese medicine decoction pieces screening, and discloses a kind of intelligent screening method of traditional Chinese medicine decoction pieces of multi-modal data fusion.The method synchronously collects the visible light image data, near infrared spectrum data and odor volatile data of target traditional Chinese medicine decoction pieces by multi-source sensor array;Three kinds of data are respectively executed morphological cleaning, baseline drift correction, environmental interference rejection operation;From the preprocessed data, extract decoction piece morphology feature set, chemical component feature set, volatile substance feature set;Calculate the morphology stability factor, component stability factor and volatile stability factor corresponding to the three kinds of feature sets;Construct a three-dimensional feature fusion space, and map the three kinds of feature sets into a fusion feature vector;In the space, perform centralized search with the fusion feature vector as the starting point, and determine the optimal screening decision vector;Thus, more accurate grade classification and quality screening can be realized, and the efficiency of traditional Chinese medicine decoction piece screening can be significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of traditional Chinese medicine decoction pieces screening, in particular to a multi-modal data fusion intelligent screening method for traditional Chinese medicine decoction pieces. BACKGROUND

[0002] As a key form of clinical application of traditional Chinese medicine, the quality of traditional Chinese medicine decoction pieces is directly related to the clinical efficacy and safety of drug use. In the current production and circulation links of traditional Chinese medicine decoction pieces, quality screening mainly relies on traditional manual identification methods, which are based on the experience of identification personnel to judge the appearance, odor and other characteristics of decoction pieces. This method has the obvious problems of strong subjectivity and poor stability. The judgment results of different identification personnel or the same personnel in different states are prone to deviation, which makes it difficult to ensure the consistency and reliability of the screening results.

[0003] With the development of technology, some fields have begun to try to use single detection technology for quality screening of traditional Chinese medicine decoction pieces, such as using visible light image technology to analyze the shape of decoction pieces, or using near-infrared spectroscopy technology to detect the chemical components of decoction pieces. However, single detection technology has obvious limitations. Only relying on visible light image technology cannot obtain the chemical component information inside the decoction pieces, and it is difficult to accurately judge the content of effective components and the internal quality of the decoction pieces; although near-infrared spectroscopy technology can reflect the chemical component situation, it has weak recognition ability for the appearance defects of decoction pieces, such as insect damage and mold. In addition, the smell of traditional Chinese medicine decoction pieces is one of the important characteristics of its quality, and at present there is no effective technical means to quantify it and integrate it into the screening process, which leads to the fact that the existing screening methods cannot fully cover the key quality indicators of decoction pieces.

[0004] In the existing attempts of using multiple technical means, there are obvious deficiencies in data processing and fusion. The data types obtained by different detection technologies are quite different, and the data formats and feature dimensions are different. Traditional data processing methods cannot effectively integrate different types of data, and often can only analyze each type of data separately and simply stack the results, which cannot fully tap the internal correlation between different data, resulting in limited accuracy and robustness of the screening model. These problems together lead to the fact that the current traditional Chinese medicine decoction piece screening technology cannot meet the demand of modern traditional Chinese medicine industry for high-quality and high-efficiency screening, and an integrated multi-dimensional data fusion technology is urgently needed to achieve comprehensive and accurate screening. SUMMARY

[0005] The purpose of the present application is to provide a multi-modal data fusion intelligent screening method for traditional Chinese medicine decoction pieces to solve the problems raised in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides a multi-modal data fusion intelligent screening method for traditional Chinese medicine decoction pieces, which comprises:

[0007] Synchronously collecting visible light image data, near-infrared spectrum data and odor volatile data of the target traditional Chinese medicine decoction piece through a multi-source sensor array;

[0008] Performing a morphological cleaning operation on the visible light image data, a baseline drift correction operation on the near-infrared spectrum data, and an environmental interference elimination operation on the odor volatile data;

[0009] Extracting a decoction piece morphology feature set from the cleaned visible light image data, a chemical component feature set from the corrected near-infrared spectrum data, and a volatile substance feature set from the interference-eliminated odor volatile data;

[0010] Calculating a morphology stability factor of the decoction piece morphology feature set, a component stability factor of the chemical component feature set, and a volatility stability factor of the volatile substance feature set;

[0011] Constructing a three-dimensional feature fusion space according to the morphology stability factor, the component stability factor and the volatility stability factor, and mapping the decoction piece morphology feature set, the chemical component feature set and the volatile substance feature set into a fusion feature vector;

[0012] In the three-dimensional feature fusion space, performing a centralized search operation with the fusion feature vector as a starting point to determine an optimal screening decision vector;

[0013] According to the optimal screening decision vector, performing grade classification and quality screening on the target traditional Chinese medicine decoction piece.

[0014] Preferably, the method for performing a morphological cleaning operation on the visible light image data is as follows:

[0015] Identifying a background pixel region in the visible light image data, and separating adhered decoction piece contours by using a morphological opening operation;

[0016] Calculating a minimum circumscribed rectangle of each decoction piece contour to generate a decoction piece spatial distribution map;

[0017] Eliminating edge incomplete regions according to the decoction piece spatial distribution map to output a complete decoction piece image data set.

[0018] Preferably, the method for extracting the decoction piece morphology feature set from the cleaned visible light image data is as follows:

[0019] Based on the complete decoction piece image data set, calculating a surface texture complexity, an edge curvature distribution histogram and an aspect ratio feature of each decoction piece;

[0020] Aggregating the surface texture complexity, the edge curvature distribution histogram and the aspect ratio feature of all decoction pieces to form the decoction piece morphology feature set.

[0021] Preferably, the method for calculating the morphological stability factor of the morphological feature set, the component stability factor of the chemical component feature set, and the volatile stability factor of the volatile substance feature set is as follows:

[0022] The first quartile and the third quartile of all surface texture complexities in the morphological feature set of the medicinal decoction pieces are calculated.

[0023] The lower bound of the texture aggregation is extended downward by a preset texture offset from the first quartile, and the upper bound of the texture aggregation is extended upward by a preset texture offset from the third quartile.

[0024] The variance value of the surface texture complexity between the lower bound of the texture aggregation and the upper bound of the texture aggregation is calculated as the morphological stability factor.

[0025] Preferably, the method for constructing a three-dimensional feature fusion space according to the morphological stability factor, the component stability factor, and the volatile stability factor is as follows:

[0026] The three-dimensional space coordinate axes are predefined, wherein the X-axis corresponds to the morphological feature dimension of the medicinal decoction pieces, the Y-axis corresponds to the chemical component feature dimension, and the Z-axis corresponds to the volatile substance feature dimension.

[0027] The morphological stability factor is mapped to the X-axis coordinate value, the component stability factor is mapped to the Y-axis coordinate value, and the volatile stability factor is mapped to the Z-axis coordinate value.

[0028] A three-dimensional coordinate point composed of the X-axis coordinate value, the Y-axis coordinate value, and the Z-axis coordinate value is taken as a center point to generate a three-dimensional feature fusion space containing historical screening decision vectors.

[0029] Preferably, the method for performing a centralized search operation with the fusion feature vector as a starting point is as follows:

[0030] In the three-dimensional feature fusion space, a target subspace containing the fusion feature vector is selected.

[0031] A spherical decision neighborhood is constructed with the fusion feature vector as the center and a preset decision step length as the radius.

[0032] The neighborhood decision density of the historical screening decision vectors in the spherical decision neighborhood is calculated.

[0033] Preferably, the method for determining the optimal screening decision vector is as follows:

[0034] A first candidate decision vector is randomly selected at the edge of the spherical decision neighborhood.

[0035] The first candidate neighborhood decision density of the first candidate decision vector is calculated.

[0036] When the first candidate neighborhood decision density is greater than the neighborhood decision density, the first candidate decision vector is updated as the current ball center and the spherical decision neighborhood is reconstructed, and the iteration is performed until a preset iteration number is met.

[0037] Preferably, the method further comprises:

[0038] When the first candidate neighborhood decision density is less than the neighborhood decision density, the number of accumulated decision update failures is accumulated;

[0039] If the number of decision update failures exceeds a preset failure threshold, the decision vector corresponding to the current ball center is set as an optimal screening decision vector.

[0040] Preferably, the method for grading and quality screening of the target traditional Chinese medicine decoction piece according to the optimal screening decision vector is:

[0041] The optimal screening decision vector is subjected to normalization processing to generate a standard screening decision value;

[0042] According to the distribution of the standard screening decision value in a preset grading threshold interval, a quality grade label of the target traditional Chinese medicine decoction piece is output.

[0043] Preferably, the method for performing normalization processing on the optimal screening decision vector is:

[0044] The length of the optimal screening decision vector in the three-dimensional feature fusion space is calculated;

[0045] The length is divided by a preset maximum radius of the decision space to obtain the standard screening decision value.

[0046] Compared with the prior art, the present application has the following advantages:

[0047] The multi-modal data fusion intelligent screening method of traditional Chinese medicine decoction pieces can comprehensively obtain the appearance, internal chemical components and volatile odor characteristics of the traditional Chinese medicine decoction pieces by synchronously collecting visible light image data, near-infrared spectrum data and odor volatile data through a multi-source sensor array, breaks the limitation of traditional single detection technology that can only obtain part of the quality information, and realizes full coverage of the decoction piece quality indicators. Compared with traditional manual identification, the method avoids subjective bias caused by manual experience through objective sensor data collection, makes the screening process more objective and consistent, and can effectively reduce the fluctuation of screening results caused by differences in personnel experience.

[0048] In the data preprocessing link, morphological cleaning, baseline drift correction and environmental interference removal operations are performed according to the characteristics of different types of data, which can effectively remove noise and interference information in the data. The morphological cleaning of visible light images can eliminate impurity pixels and background interference in the image, highlight the true morphological characteristics of the decoction pieces, and improve the accuracy of subsequent morphological feature extraction; the baseline drift correction of near-infrared spectral data can eliminate spectral baseline shift caused by factors such as instrument error and environmental temperature change, ensuring the authenticity and reliability of chemical component information; the environmental interference removal of odor volatile substance data can exclude the influence of other odor molecules in the external environment, accurately capture the volatile substance characteristics of the decoction pieces themselves, and provide a high-quality data basis for subsequent feature extraction.

[0049] Extracting morphological feature sets, chemical component feature sets and volatile substance feature sets from different types of preprocessed data respectively, and calculating corresponding stability factors, can convert each dimension of data into feature indicators with clear physical meaning. The morphological stability factor can quantitatively reflect the integrity and consistency of the decoction piece morphology, the component stability factor can reflect the uniformity and effective component distribution of the chemical components, and the volatile stability factor can represent the stability and typicality of the odor characteristics. The introduction of these stability factors provides clear and representative feature basis for subsequent data fusion, making the features of different types of data comparable and fusible.

[0050] By constructing a three-dimensional feature fusion space, the three types of feature sets are mapped into a fusion feature vector, realizing the deep integration of different dimensions of data instead of simple result superposition. This fusion method can fully exploit the internal relationship between morphology, chemical components and odor characteristics, such as the potential link between decoction piece morphology defects and chemical component changes, odor abnormalities, so that the fused feature vector can more comprehensively and accurately reflect the overall quality status of the decoction pieces. Compared with the traditional data superposition method, this fusion strategy greatly improves the richness and effectiveness of feature information, providing a more reliable basis for subsequent screening decisions.

[0051] Performing centralized search operations in the three-dimensional feature fusion space to determine the optimal screening decision vector can make accurate screening judgments based on multi-dimensional fusion features, avoiding misjudgments that may be caused by single feature analysis. This search process can fully utilize the comprehensive information in the fusion feature vector, comprehensively consider morphological, chemical component and odor factors, and make overall evaluation of the quality of decoction pieces, so as to realize more accurate grade classification and quality screening. In addition, the whole process of this method is based on standardized technical operations, which is easy to realize automation and large-scale application, can significantly improve the efficiency of Chinese herbal decoction piece screening, reduce the dependence on manual operation, meet the development needs of modern Chinese medicine industry, help to promote the upgrading of Chinese herbal decoction piece quality control technology, and promote the standardization and modernization development of Chinese medicine industry. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The working principle diagram of the intelligent screening method of traditional Chinese medicine decoction pieces based on multi-modal data fusion;

[0053] Figure 2 The flowchart of the morphological cleaning operation of visible light image data;

[0054] Figure 3 The flowchart of calculating three stability factors;

[0055] Figure 4 The flowchart of performing a centralized search operation. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0057] Please refer to Figure 1 The present application provides an intelligent screening method of traditional Chinese medicine decoction pieces based on multi-modal data fusion, which comprises:

[0058] Visible light image data, near-infrared spectral data, and odor volatile matter data of target Chinese herbal medicine slices were simultaneously acquired using a multi-source sensor array. The multi-source sensor array included a high-resolution camera, a near-infrared spectrometer, and a gas sensor to ensure synchronous data acquisition. After acquisition, morphological cleaning was performed on the visible light image data, involving image preprocessing to remove noise and separate adhering objects; baseline drift correction was performed on the near-infrared spectral data to eliminate errors caused by instrument drift; and environmental interference removal was performed on the odor volatile matter data to filter out the influence of environmental factors such as temperature and humidity. Next, a morphological feature set, including surface texture and geometric shape information, was extracted from the cleaned visible light image data; a chemical component feature set reflecting the chemical composition of the slices was extracted from the corrected near-infrared spectral data; and a volatile substance feature set characterizing odor properties was extracted from the interference-removed odor volatile matter data. Subsequently, morphological stability factors for the morphological feature set, component stability factors for the chemical component feature set, and volatility stability factors for the volatile substance feature set were calculated; these factors were used to quantify the stability of the features. A three-dimensional feature fusion space is constructed based on morphological stability factors, component stability factors, and volatility stability factors. The morphological feature sets, chemical component feature sets, and volatile substance feature sets of medicinal slices are mapped to fused feature vectors. Within the three-dimensional feature fusion space, a centralized search operation is performed starting from the fused feature vectors to determine the optimal screening decision vector. Based on the optimal screening decision vector, the target medicinal slices are graded and screened for quality, and the quality grade labels of the slices are output.

[0059] Example 1: See Figure 2 During the morphological cleaning operation, visible light image data was acquired using a high-resolution industrial camera with an image resolution of 4096×2160 pixels and a bit depth of 8 bits. The acquired images were first converted to grayscale, and a weighted average method was used to calculate the grayscale value of each pixel. Background pixel region identification was based on grayscale threshold segmentation, with a threshold set at 30 grayscale values; pixels below this value were identified as background. The morphological opening operation used a circular structuring element with a radius of 5 pixels. The opening operation first performed an erosion operation on the image to eliminate small noise points and weak connections, followed by a dilation operation to restore the main shape of the medicinal slices. The separation of the contours of adhered medicinal slices was achieved by calculating connected components, with each connected component representing a potential medicinal slice object.

[0060] The calculation of the minimum bounding rectangle is based on the separated herb contour, using a rotating bounding box algorithm, traversing all points on the contour, calculating the bounding box in each direction, and selecting the smallest rectangle with the smallest area as the minimum bounding rectangle. The coordinates, width and height of each rectangle are recorded. The spatial distribution map of the herb is generated in the form of a two-dimensional array, with the same size as the original image, and each element stores the herb identifier or background marker at the corresponding position. The removal of the edge missing area is based on the position of the minimum bounding rectangle. If any side of the rectangle is less than 10 pixels away from the image boundary, the herb is considered to be edge missing and removed from the data set. The complete herb image data set is output as a series of cropped sub-images, each containing a complete herb, with a uniform size of 256x256 pixels.

[0061] The extraction of the herb morphology feature set is based on the complete herb image data set. The calculation of the surface texture complexity uses a gray level co-occurrence matrix, with the direction set to 0 degrees and the distance set to 1 pixel. The contrast and entropy features are extracted from the matrix. The contrast reflects the clarity of the texture, and the entropy represents the randomness of the texture. The surface texture complexity of each herb is represented by the weighted sum of the contrast and the entropy, with weights of 0.6 and 0.4 respectively. The edge curvature distribution histogram is obtained by first extracting the herb contour using the Canny edge detection algorithm, and then smoothing the contour point set using a B-spline curve. The curvature calculation is based on the fitted curve, and the reciprocal of the curvature radius is calculated at each contour point. The curvature value is normalized to the range of 0 to 1. The histogram divides the curvature value into 10 intervals, and the proportion of the number of points in each interval is calculated. The aspect ratio feature is directly calculated from the minimum bounding rectangle. The length is the long side of the rectangle, the width is the short side, and the ratio is the length divided by the width. All feature values are normalized to eliminate the dimension effect.

[0062] The composition of the herb morphology feature set is obtained by aggregating the feature values of all herbs. Each herb's features are represented as a vector, containing the surface texture complexity, the 10 interval values of the edge curvature distribution histogram, and the aspect ratio. The feature set is stored as a matrix, with rows corresponding to herb samples and columns corresponding to feature dimensions. During feature extraction, parallel computing is used to process each herb sub-image to improve processing efficiency. The final output of the herb morphology feature set is used for subsequent stability factor calculation and feature fusion. In specific applications, the collection environment of visible light image data needs to maintain uniform illumination to avoid shadow and glare interference. The effectiveness of the morphological cleaning operation depends on the size and shape selection of the structural elements, which need to be adjusted according to the type of herb. The accuracy of feature extraction is affected by image resolution and preprocessing effect, and high-resolution images can provide more detailed texture and edge information.

[0063] Example 2: see Figure 3In the process of calculating the morphological stability factor, the set of morphological characteristics of the decoction piece includes the surface texture complexity values of multiple decoction pieces. Assuming that a batch of 50 samples of Huangqi decoction pieces have surface texture complexity values distributed between 0.15 and 0.85 after normalization processing. First, sort all the values in ascending order, and determine the value at the 25% position as 0.32 (the first quartile) and the value at the 75% position as 0.68 (the third quartile). The preset texture offset is set to 0.05, which is determined based on historical data statistical analysis. The lower bound of texture aggregation is calculated as 0.32 minus 0.05, which is 0.27, and the upper bound of texture aggregation is calculated as 0.68 plus 0.05, which is 0.73. Select all surface texture complexity values within the range of 0.27 to 0.73, a total of 38 samples. Calculate the variance of these sample values: first, calculate the mean value 0.49, then calculate the sum of the squared differences between each value and the mean value, and finally divide by the number of samples to get the variance 0.021. This variance value is used as the morphological stability factor.

[0064] For the chemical component feature set, taking Danshen decoction pieces as an example, the feature set includes the concentration value of the main active ingredient Danshensan B. The concentration values of 50 samples are distributed between 8.2 mg / g and 22.7 mg / g. After sorting, the first quartile is determined to be 12.3 mg / g and the third quartile is determined to be 18.9 mg / g. The preset component offset is set to 1.2 mg / g, which is based on the concentration fluctuation range of similar decoction pieces. The lower bound of component aggregation is 12.3 minus 1.2, which is 11.1 mg / g, and the upper bound of component aggregation is 18.9 plus 1.2, which is 20.1 mg / g. Select 42 samples with concentration values between 11.1 mg / g and 20.1 mg / g, calculate the variance: after calculating the mean value 15.8 mg / g, calculate the sum of the squared deviations of each sample value from the mean value, and divide by the number of samples to get the variance 4.37. This value is the component stability factor.

[0065] In the processing of the volatile substance feature set, the feature set includes the volatile concentration value of menthol, and the 50 sample data is distributed between 105 ppm and 287 ppm. After sorting, the first quartile is determined to be 142 ppm and the third quartile is determined to be 218 ppm. The preset volatile offset is set to 25 ppm, which is determined according to the detection accuracy of the sensor. The lower bound of volatile aggregation is 142 minus 25, which is 117 ppm, and the upper bound of volatile aggregation is 218 plus 25, which is 243 ppm. Select 39 samples with concentration values between 117 ppm and 243 ppm, calculate the mean value 183 ppm, then calculate the sum of the squared differences between each sample value and the mean value, and divide by the number of samples to get the variance 1026. This variance value is used as the volatile stability factor.

[0066] The above calculation process needs to pay attention to three key links: first, the calculation of quartiles adopts linear interpolation method, and when the sample quantity cannot be divided, the position is determined by proportion. Second, the setting of offset needs to consider the distribution characteristics of characteristic values, the texture offset is usually 0.3 to 0.7 times of the standard deviation of characteristic values, the chemical component offset refers to the standard error of the detection method, and the volatile offset combines the sensor accuracy parameters. Third, the variance calculation adopts the population variance formula to ensure the comparability of the stability factor. When implemented, a data processing pipeline is established: after the characteristic value is input, it is automatically sorted, the quartile calculation module is called, the aggregation interval is generated according to the preset offset, and finally the variance operation is performed. All parameter settings are recorded in the configuration file, including offset coefficient, variance calculation method, etc. For different medicinal piece varieties, the parameter combination stored in the configuration file, such as the texture offset coefficient of Astragalus root medicinal piece is 0.5, the component offset of Salvia miltiorrhiza medicinal piece is a fixed value of 1.2, and the volatile offset of Mentha haplocalyx medicinal piece is a fixed value of 25. The system automatically loads the corresponding parameters according to the medicinal piece type.

[0067] The characteristic value abnormality processing mechanism contains two levels: when the characteristic value exceeds the historical data range, the data review process is started; when the sample proportion in the aggregation interval is less than 60%, the offset is automatically expanded by 10% to recalculate. All stability factor calculation results are attached with confidence labels, and the confidence is determined according to the sample proportion in the aggregation interval. The proportion higher than 80% is marked as high confidence, the proportion between 60% and 80% is marked as medium confidence, and the proportion less than 60% is marked as low confidence.

[0068] The implementation mode shows parameter sensitivity in multiple batch tests. When the texture offset coefficient is in the interval of 0.4 to 0.6, the morphological stability factor fluctuates less than 5%; the component offset fixed value changes ±0.2 mg / g, causing the factor to fluctuate about 3%; and the volatile offset fixed value changes ±5 ppm, causing the factor to fluctuate about 7%. Therefore, the parameter setting needs to be calibrated by historical data, and at least 5 batches of data need to be accumulated for new medicinal piece varieties to determine the optimal parameters.

[0069] Example 3: see Figure 4 In the process of constructing the three-dimensional feature fusion space, a space coordinate system needs to be established first. Three orthogonal coordinate axes are defined: the X-axis represents the medicinal piece morphological feature dimension, the value range of which is determined by historical morphological stability factor data, and is usually normalized to the interval [0, 1]; the Y-axis corresponds to the chemical component feature dimension, which is calibrated based on the statistical distribution of the component stability factor; and the Z-axis represents the volatile substance feature dimension, whose scale is adjusted according to the maximum and minimum values of the volatile stability factor. The unit length of each axis represents the change amplitude of the stability factor, and the three dimensions together constitute a standardized cubic space.

[0070] The calculated morphological stability factor, component stability factor and volatility stability factor are mapped to spatial coordinate values. The mapping process employs a linear transformation:

[0071] ;

[0072] where: is the morphological stability factor of the current batch, and are the minimum and maximum values of the morphological stability factor in the historical data, respectively; is the component stability factor of the current batch, and are its historical minimum and maximum values; is the volatility stability factor of the current batch, and are its historical minimum and maximum values. This mapping ensures that all coordinate values fall within the range [0, 1].

[0073] The historical screening decision vectors are loaded from the database, each vector containing three-dimensional coordinate values of past successful screening cases. These vectors are stored in the form of a point cloud in the feature fusion space, forming a decision sample distribution. The space center point is determined as the coordinate value obtained by the current mapping , and a cubic subspace with a side length of 1 is generated around it, which contains all historical decision points.

[0074] In the centralized search operation, the coordinate point corresponding to the fusion feature vector is constructed as the center of the spherical decision neighborhood. The spherical radius is set to 0.1, which is adjusted according to the distribution density of historical decision points. The point set within the spherical neighborhood is defined as all points satisfying , where is the Euclidean distance.

[0075] The calculation of neighborhood decision density uses the kernel density estimation method. For the historical decision points within the spherical neighborhood, the density calculation formula is:

[0076] ;

[0077] where: the kernel function uses the Epanechnikov kernel: when , otherwise 0. This kernel function gives greater weight to points close to the center of the sphere, making the density estimation better reflect the local clustering characteristics.

[0078] ​The historical decision database needs to be maintained and updated with new successful screening cases regularly. The database records the values of the three stability factors and their corresponding quality level labels for each case. The latest data is automatically synchronized during the construction of the space to ensure that the feature fusion space always reflects the latest screening experience.

[0079] An outlier processing mechanism is set for the coordinate mapping step. When the stability factors of the current batch exceed the historical range, an extreme value compression strategy is adopted: If , then take If , then take The same processing is applied to the coordinate mapping of the Y and Z axes. This processing avoids the coordinate values from exceeding the range [0, 1] and ensures the integrity of the space structure. The radius of the spherical neighborhood is set with adaptive characteristics. When the overall distribution of historical decision points is sparse, the radius is automatically increased to 0.15; when the distribution is dense, the radius is reduced to 0.08. Adaptive adjustment is based on the average nearest neighbor distance of all points in the feature space to ensure that a sufficient number of sample points are included in the neighborhood for density estimation. The implementation of kernel density estimation uses a hierarchical calculation strategy. First, calculate the distance between the center of the sphere and each historical decision point, and select the point set that meets the distance condition; then calculate the normalized distance of each point; finally, calculate the weight according to the kernel function and sum it up. Parallel optimization is used in the calculation process to improve processing efficiency.

[0080] The visualization auxiliary function of the feature fusion space provides a three-dimensional scatter plot display, where decision points of different quality levels are marked with different colors. The coordinate point corresponding to the current batch is displayed in a prominent color, and the spherical decision neighborhood is presented in the form of a transparent sphere. This visualization helps to understand the spatial geometric relationship of the decision-making process.

[0081] A quality control log is established during implementation to record the parameter settings, coordinate mapping results, and neighborhood density values of each space construction. When the log detects that the density value is abnormally low for several consecutive times, it triggers a data review process to check the reliability of the sensor data acquisition and feature extraction steps.

[0082] In the implementation of the centralized search operation to determine the optimal screening decision vector, the search parameters need to be initialized first. The coordinate point of the current fusion feature vector in the three-dimensional feature fusion space is taken as the initial center of the sphere, the radius of the spherical decision neighborhood is set to 0.1 units of length, the preset number of iterations is 100 times, and the preset failure threshold is set to 10 times. The search process records the current center coordinates, neighborhood decision density, candidate decision vector state, and the number of decision update failures.

[0083] When the search process is started for the first time, a point is randomly selected on the boundary of the current spherical decision neighborhood as the first candidate decision vector. The random selection uses a uniform distribution sampling algorithm to ensure that any location on the sphere has an equal probability of being selected. The neighborhood decision density of this candidate vector is calculated, which is the distribution density of historical decision points within a spherical region of the same radius centered at the point. The density calculation uses a kernel density estimation method, using an Epanechnikov kernel function for weighted calculation. The candidate neighborhood decision density is compared with the current neighborhood decision density of the sphere center. If the candidate density is greater than the current density, the sphere center is moved to the candidate point location, and the decision update failure count is reset. The spherical decision neighborhood is then reconstructed with the new sphere center, and the next iteration continues. If the candidate density is less than or equal to the current density, the decision update failure count is accumulated, and another candidate point is selected on the current neighborhood boundary for testing.

[0084] When the decision update failure count exceeds the preset failure threshold, the search process is terminated, and the decision vector corresponding to the current sphere center is determined as the optimal screening decision vector. The entire search process records detailed data for each iteration step, including the coordinates of each selected candidate point, the density calculation results, and the decision state. Referring to Table 1, the iteration search process record for a specific batch is shown.

[0085] Table 1: Optimal Screening Decision Vector Search Process Record Table

[0086] Number of iterations Current sphere center coordinates (x, y, z) Current density Candidate point coordinates (x, y, z) Candidate density Density comparison result Failure count Decision state 1 (0.42,0.38,0.51) 0.72 (0.47,0.33,0.56) 0.68 Less than 1 Continue 2 (0.42,0.38,0.51) 0.72 (0.39,0.45,0.49) 0.75 Greater than 0 Move 3 (0.39,0.45,0.49) 0.75 (0.36,0.48,0.52) 0.78 Greater than 0 Move 4 (0.36,0.48,0.52) 0.78 (0.33,0.51,0.55) 0.81 Greater than 0 Move 5 (0.33,0.51,0.55) 0.81 (0.31,0.53,0.58) 0.79 Less than 1 Continue ... ... ... ... ... ... ... ... 23 (0.28,0.59,0.62) 0.92 (0.25,0.61,0.65) 0.89 Less than 8 Continue 24 (0.28,0.59,0.62) 0.92 (0.26,0.58,0.64) 0.90 Less than 9 Continue 25 (0.28,0.59,0.62) 0.92 (0.27,0.60,0.63) 0.88 Less than 10 Terminate

[0087] As can be seen from the table record, the search process terminates after 25 iterations. The initial sphere center coordinates are (0.42, 0.38, 0.51), and the corresponding neighborhood decision density is 0.72. During the iteration process, the sphere center position gradually moves to areas with higher density. In the second iteration, the density of the candidate point (0.39, 0.45, 0.49) is 0.75, which is greater than the current density, and the sphere center moves to this location. In the third and fourth iterations, higher-density candidate points are successfully found, and the sphere center moves to (0.36, 0.48, 0.52) and (0.33, 0.51, 0.55), with densities increasing to 0.78 and 0.81, respectively.

[0088] From the fifth iteration, there are multiple cases where the density of the candidate point is lower than the current density, and the failure count gradually accumulates. In the 23rd-25th iterations, the densities of the three consecutive candidate points are all lower than the current density of 0.92, and the failure count reaches the threshold of 10 times, terminating the search process. The final optimal screening decision vector corresponds to the coordinates (0.28, 0.59, 0.62), which has a higher neighborhood decision density of 0.92, indicating that this region has a large number of historical high-quality screening decisions.

[0089] The random sampling algorithm uses the Mason rotation algorithm to generate uniformly distributed random numbers, ensuring the uniform distribution of candidate points on the spherical surface. The coordinate calculation of each candidate point is realized through spherical coordinate conversion, generating azimuth and zenith angles randomly, and then converting them into rectangular coordinate system coordinates. In density calculation, the bandwidth parameter of the kernel function is adaptively adjusted according to the overall distribution density of historical decision points, ensuring the accuracy of density estimation. The iteration control mechanism includes timeout protection, which forcibly terminates when the number of iterations reaches the preset 100 times, avoiding infinite loops. At the same time, a density growth threshold is set, and if the density growth amplitude is less than 0.01 for 10 consecutive iterations, the search is terminated early, considering that the local optimal solution has been converged. All parameter settings are recorded in the system configuration file and can be adjusted according to different medicinal material varieties.

[0090] The historical decision database is regularly maintained, removing outdated decision records and adding new successful screening cases. The database records the three-dimensional coordinates, density values, quality grade labels, and collection timestamps of each decision point. The system automatically performs cluster analysis on historical data to identify spatial distribution areas corresponding to different quality grades, providing a reference for the search process. The visual monitoring interface displays the search process in real time, showing the distribution of historical decision points in the form of a three-dimensional scatter plot, with the current sphere center marked as a red sphere, candidate points marked as yellow dots, and search paths displayed as connected lines. The operator can visually observe the progress of the search process.

[0091] The abnormal handling mechanism includes multiple levels: when 20 consecutive candidate points cannot find higher density, the search radius is automatically expanded by 50%; when the search process falls into a local optimum, the restart mechanism is enabled to start searching from the original starting point; when there is insufficient historical data, the operator is prompted to supplement the data or adjust the parameters. All abnormal situations are recorded in the system log for subsequent analysis and optimization.

[0092] After obtaining the optimal screening decision vector, it needs to be normalized to generate the standard screening decision value. The vector is represented as a coordinate point in the three-dimensional feature fusion space, and its module length is obtained by calculating the Euclidean distance. Specifically, given the coordinate values of the optimal screening decision vector as (x, y, z), the module length is calculated as the square root of the sum of the squares of the coordinates. The maximum radius of the decision space is determined according to the maximum module length of all decision vectors in the historical data, usually taking 1.05 to 1.2 times the historical maximum module length to accommodate possible out-of-range values. Divide the calculated module length by the maximum radius to get the standard screening decision value, which is normalized to the [0, 1] interval. If the calculation result is greater than 1, take the value as 1; less than 0, take the value as 0.

[0093] The grading threshold interval of the standard screening decision value is set to three consecutive intervals: [0, 0.3] corresponds to a low quality grade, [0.3, 0.7] corresponds to a medium quality grade, and [0.7, 1.0] corresponds to a high quality grade. Each interval includes the lower bound but not the upper bound, for example, 0.3 belongs to the low quality grade interval, and 0.7 belongs to the medium quality grade interval. The quality grade label uses a textual description, with the low quality grade labeled as "Level 3", the medium quality grade labeled as "Level 2", and the high quality grade labeled as "Level 1". A threshold adjustment mechanism is established during implementation, and the system regularly statistics the screening result distribution of the last 100 batches. When the distribution changes significantly, the grading threshold interval is automatically adjusted. The adjustment principle is to keep the proportion of samples in each grade relatively stable, and to avoid the grade distribution from shifting due to environmental changes or raw material differences. The adjustment amplitude is not more than 0.05 each time, to ensure the stability of the threshold change.

[0094] When outputting the quality grade label, a confidence index is also generated. The confidence is calculated according to the distance of the standard screening decision value from the nearest threshold boundary. The greater the distance, the higher the confidence. The confidence is divided into high, medium, and low levels, corresponding to distances greater than 0.1, between 0.05 and 0.1, and less than 0.05, respectively. High confidence results are directly output, medium confidence results are prompted for review, and low confidence results start the manual review process. When the standard screening decision value falls between 0.29 and 0.31 or between 0.69 and 0.71, the system automatically performs a secondary verification. The secondary verification is performed by analyzing the original feature data distribution of the batch of decoction pieces. If the feature distribution shows a clear bias, the grade division is adjusted. For example, if the standard screening decision value is 0.305 but the morphological features are significantly better than those of similar second-level decoction pieces, it may be upgraded to first level.

[0095] Detailed logs are recorded during implementation, including the optimal screening decision vector coordinates of each batch, the calculated modulus, the used maximum radius value, the standard screening decision value, the finally determined quality grade, and the confidence level. Log data is used for subsequent system optimization and parameter adjustment. A feedback mechanism is also established to compare actual use effects with system prediction results, gradually optimizing the grading threshold interval setting. The system provides manual adjustment functions, allowing experienced operators to fine-tune the grading threshold according to actual conditions. Manual adjustment records the reasons for the change and the adjustment amplitude, which are used to improve the system's adaptive learning ability. All adjustment operations require double confirmation to ensure the rationality and traceability of the changes. The output of the quality grade label uses a standardized format, including batch number, decoction piece variety, detection time, standard screening decision value, quality grade, and confidence level. The output information is also presented in a visual manner, with different colors representing different grades: green for first level, blue for second level, and yellow for third level. This intuitive display method allows operators to quickly grasp the screening results.

[0096] The system periodically performs retrospective analysis on historical grading results to check for systematic bias. When a consistent deviation in grade distribution is found for a certain variety, a special calibration procedure is initiated. The calibration procedure optimizes feature weights and threshold settings by reanalyzing historical data for that variety, ensuring accuracy and consistency of grading standards. The entire implementation process focuses on stability and repeatability. All computational parameters and threshold settings are saved in configuration files, which are automatically loaded every time the system starts. Configuration file versions are strictly managed, and any modification requires a change note and effective time. This management approach ensures consistency of screening results between different times and different operators.

[0097] It should be noted that the relational terms herein, such as first and second, and the like, are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0098] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary and that changes in form and detail can be made without departing from the spirit and scope of the application, which is defined by the following claims and their equivalents.

Claims

1. A method for intelligent screening of traditional Chinese medicine decoction pieces using multimodal data fusion, characterized in that, Includes the following steps: Visible light image data, near-infrared spectral data, and odor volatile data of the target Chinese herbal medicine pieces are collected synchronously using a multi-source sensor array. Morphological cleaning is performed on the visible light image data, baseline drift correction is performed on the near-infrared spectral data, and environmental interference removal is performed on the odor volatiles data. The morphological feature set of medicinal slices was extracted from the cleaned visible light image data, the chemical component feature set was extracted from the corrected near-infrared spectral data, and the volatile substance feature set was extracted from the odor volatile data after removing interference. Calculate the morphological stability factor of the morphological feature set of the medicinal slices, the component stability factor of the chemical component feature set, and the volatility stability factor of the volatile substance feature set; A three-dimensional feature fusion space is constructed based on the morphological stability factor, component stability factor, and volatility stability factor, and the morphological feature set, chemical component feature set, and volatile substance feature set of the medicinal slices are mapped into fused feature vectors; In the three-dimensional feature fusion space, a centralized search operation is performed with the fused feature vector as the starting point to determine the optimal screening decision vector; The target Chinese medicinal herbs are classified and screened for quality based on the optimal screening decision vector. The method for calculating the morphological stability factor of the morphological feature set of the medicinal slices, the component stability factor of the chemical component feature set, and the volatility stability factor of the volatile substance feature set is as follows: The first and third quartiles of the surface texture complexity of all surface textures in the set of morphological features of the medicinal slices were statistically analyzed. The first quartile is extended downward by a preset texture offset as the lower bound of texture aggregation, and the third quartile is extended upward by the preset texture offset as the upper bound of texture aggregation. Calculate the variance of the surface texture complexity between the lower bound and the upper bound of the texture aggregation, and set it as the morphological stability factor; The first and third quartiles of the concentration values ​​of all key active ingredients in the set of chemical component characteristics were statistically analyzed. The first quartile is extended downward by a preset component offset as the lower bound of component aggregation, and the third quartile is extended upward by the preset component offset as the upper bound of component aggregation. Calculate the variance of the concentration values ​​of the key active ingredient between the lower boundary and the upper boundary of the component aggregation, and set it as the component stability factor; The first quartile and the third quartile of the concentration values ​​of all characteristic volatile substances in the volatile substance characteristic set are statistically analyzed; the first quartile is extended downward by a preset volatile offset as the lower boundary of volatile aggregation, and the third quartile is extended upward by the preset volatile offset as the upper boundary of volatile aggregation. Calculate the variance of the characteristic volatile concentration values ​​between the lower boundary and the upper boundary of the volatile aggregation, and set it as the volatile stability factor; The method for constructing a three-dimensional feature fusion space based on the morphological stability factor, component stability factor, and volatility stability factor is as follows: A predefined three-dimensional spatial coordinate axis is used, where the X-axis corresponds to the morphological characteristics of the medicinal slices, the Y-axis corresponds to the chemical composition characteristics, and the Z-axis corresponds to the volatile substance characteristics. The morphological stability factor is mapped to an X-axis coordinate value, the component stability factor is mapped to a Y-axis coordinate value, and the volatility stability factor is mapped to a Z-axis coordinate value; Using the three-dimensional coordinate point formed by the X-axis coordinate value, the Y-axis coordinate value, and the Z-axis coordinate value as the center point, a three-dimensional feature fusion space containing historical screening decision vectors is generated.

2. The intelligent screening method for traditional Chinese medicine decoction pieces based on multimodal data fusion according to claim 1, characterized in that, The method for performing morphological cleaning on the visible light image data is as follows: Background pixel regions in visible light image data are identified, and morphological opening operations are used to separate the contours of adhered medicinal slices. Calculate the minimum bounding rectangle of each medicinal slice outline to generate a spatial distribution map of the medicinal slices; Based on the spatial distribution map of the medicinal slices, the edge-damaged areas are removed, and a complete medicinal slice image dataset is output.

3. The intelligent screening method for traditional Chinese medicine decoction pieces based on multimodal data fusion according to claim 2, characterized in that, The method for extracting the morphological feature set of medicinal slices from cleaned visible light image data is as follows: Based on the complete medicinal slice image dataset, the surface texture complexity, edge curvature distribution histogram, and aspect ratio features of each medicinal slice are calculated. The surface texture complexity, edge curvature distribution histogram, and aspect ratio features of all medicinal slices are aggregated to form the morphological feature set of the medicinal slices.

4. The intelligent screening method for traditional Chinese medicine decoction pieces based on multimodal data fusion according to claim 1, characterized in that, The method for performing a centralized search operation starting from the fused feature vector is as follows: In the three-dimensional feature fusion space, a target subspace containing the fused feature vector is selected; A spherical decision neighborhood is constructed with the fused feature vector as the center and a preset decision step size as the radius; Calculate the neighborhood decision density of the historical screening decision vector within the spherical decision neighborhood.

5. The intelligent screening method for traditional Chinese medicine decoction pieces based on multimodal data fusion according to claim 4, characterized in that, The method for determining the optimal screening decision vector is as follows: A first candidate decision vector is randomly selected at the edge of the spherical decision neighborhood. Calculate the decision density of the first candidate neighborhood of the first candidate decision vector; When the decision density of the first candidate neighborhood is greater than the neighborhood decision density, the first candidate decision vector is updated to the current sphere center and the spherical decision neighborhood is reconstructed. This process is repeated iteratively until the preset number of iterations is met.

6. The intelligent screening method for traditional Chinese medicine decoction pieces based on multimodal data fusion according to claim 5, characterized in that, The method further includes: When the decision density of the first candidate neighborhood is less than the decision density of the neighborhood, the cumulative number of decision update failures is counted. If the number of decision update failures exceeds a preset failure threshold, the decision vector corresponding to the current ball center is set as the optimal screening decision vector.

7. The intelligent screening method for traditional Chinese medicine decoction pieces based on multimodal data fusion according to claim 1, characterized in that, The method for classifying and screening target Chinese medicinal herbs based on the optimal screening decision vector is as follows: The optimal screening decision vector is normalized to generate a standard screening decision value; Based on the distribution of the standard screening decision values ​​within the preset grading threshold range, the quality grade label of the target Chinese herbal medicine slices is output.

8. The intelligent screening method for traditional Chinese medicine decoction pieces based on multimodal data fusion according to claim 7, characterized in that, The method for normalizing the optimal screening decision vector is as follows: Calculate the magnitude of the optimal screening decision vector in the three-dimensional feature fusion space; Divide the modulus by the preset maximum radius of the decision space to obtain the standard screening decision value.

Images