A clustering-based identification method and system for traditional Chinese medicine
Through the clustering method based on the effective radius of weight, the problem of parameter sensitivity and low calculation efficiency in Chinese medicine identification is solved, and the problem of different components and structure of Chinese medicine is quickly discovered, which improves the accuracy and efficiency of Chinese medicine identification.
Patent Information
- Application Number
- CN202111276838.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-29
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-10-29
AI Technical Summary
The existing clustering methods have problems such as parameter sensitivity, low calculation efficiency, and inability to process irregular distribution data in Chinese medicine identification, resulting in low efficiency and accuracy of Chinese medicine identification.
Using a clustering method based on the effective radius of weight, the weight value of each Chinese medicine to be identified is calculated, and the suspected problem Chinese medicine is selected and divided into multiple clusters, reducing the impact of empirical parameters on the traditional spatial density clustering method and improving Accuracy of identification of traditional Chinese medicine.
It has achieved rapid discovery of problems with different components and structures in a large number of traditional Chinese medicines to be identified, reducing the impact of empirical parameters on clustering, improving the accuracy and efficiency of traditional Chinese medicine identification, and reducing the cost of trial and error.
Smart Images

Figure CN113963808B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of traditional Chinese medicine quality control, and in particular to a traditional Chinese medicine identification method and system based on clustering. Background Art
[0002] The statements in this section merely mention background art related to the present invention and do not necessarily constitute prior art.
[0003] At present, clustering algorithms are generally required when using terahertz spectroscopy to study the authenticity of traditional Chinese medicine. The reason is that traditional Chinese medicine contains multiple chemical components, and their spectral characteristics in the terahertz band are superimposed together and lose the iconic absorption peak. There is a complex relationship between the authenticity of the medicine and the components themselves. Therefore, the directly detected terahertz spectral information of traditional Chinese medicine can neither directly reflect the differences and similarities of the origin and batches, nor directly explain the relationship between any two samples. Since the clustering algorithm realizes deep data mining of terahertz spectra, the samples are clustered into several "clusters" according to the spectral information after dimension reduction, and researchers can discover special samples with different components and structures. Since terahertz spectroscopy is a non-destructive measurement, researchers can specifically supplement the detailed chemical composition of these exotic samples with chromatography and mass spectrometry experiments, thereby reducing the overall research cost and improving research efficiency.
[0004] Clustering of two-dimensional data is widely used in various types of spectral analysis. Since the spectrum itself is high-dimensional data, the differences between samples have very complex correlations with chemical components. Spectral analysis requires a data analysis method that can project high-dimensional samples into low-dimensional space for visualization. Two-dimensional data can be depicted in the xy plane, and the concentration, divisibility, and interference of the data are easy to intuitively judge, so it has become the preferred data form for related algorithm research.
[0005] However, the existing clustering methods have certain defects when used for identification of traditional Chinese medicine. Clustering based on partitioning (such as Kmeans, K-modes and Kmedians) is simple and efficient, but it needs to set the K value and is sensitive to the initial point and outliers; the hierarchical clustering algorithm has good interpretability, but the time complexity is high, and the greedy algorithm is prone to make mistakes at every step; density-based clustering (such as DBSCAN) is not sensitive to noise, but the clustering results are highly correlated with parameter selection; grid-based clustering is fast, but the parameters are sensitive and cannot handle irregularly distributed data, and there is a certain contradiction between clustering efficiency and accuracy; model-based clustering (such as Gaussian mixture GMM and self-organizing neural network SOM) expresses the division of "classes" in the form of probability, but the execution efficiency is not high; fuzzy clustering algorithm has a good clustering effect on data that meets the normal distribution, and the algorithm is sensitive to isolated points, but because it cannot ensure convergence to an optimal solution, the performance of the algorithm depends on the initial cluster center, which often needs to be obtained through other fast methods. Summary of the invention
[0006] In order to address the deficiencies of the prior art, the present invention provides a clustering-based Chinese medicine identification method and system, which realizes pattern recognition of Chinese medicine spectra and quickly discovers problematic Chinese medicines with different ingredients and structures among a large number of Chinese medicines to be identified.
[0007] In a first aspect, the present invention provides a method for identifying traditional Chinese medicine based on clustering;
[0008] A clustering-based traditional Chinese medicine identification method, comprising:
[0009] Obtain the spectral characteristics of all Chinese medicines to be identified;
[0010] Based on the spectral features, the distance matrix and weighted effective radius are calculated;
[0011] Based on the weighted effective radius and the distance matrix, the weight value of each Chinese medicine to be identified is calculated;
[0012] Based on the weight value, suspected problematic Chinese medicines are extracted from all the Chinese medicines to be identified, and the remaining Chinese medicines to be identified are obtained;
[0013] The remaining Chinese medicines to be identified are divided into multiple clusters;
[0014] Determine the cluster affiliation of suspected problematic Chinese medicines and identify the problematic Chinese medicines.
[0015] Furthermore, the weighted effective radius is calculated as follows:
[0016] Select the minimum value greater than 0 in each column of the distance matrix;
[0017] The maximum value among all minimum values is used as the effective radius of the weight.
[0018] Furthermore, the specific calculation steps of the weight value are:
[0019] All the Chinese medicines to be identified within the effective radius of the weight of the Chinese medicines to be identified whose weight values need to be calculated are taken as the center point;
[0020] A central point is randomly selected, and all the Chinese medicines to be identified within the weighted effective radius of the central point are selected;
[0021] According to the distance from the center point, all selected Chinese medicines to be identified are sorted in descending order, and the weight contribution of the Chinese medicine to be identified that needs to calculate the weight value under the center point is the sorting sequence number;
[0022] Until all the centers are selected, the weight contributions of the Chinese medicine to be identified at all center points for which the weight values need to be calculated are obtained, and the weight contributions are added up to obtain the weight values.
[0023] Furthermore, the suspected problematic Chinese medicine is a Chinese medicine to be identified whose weight value is less than a set threshold.
[0024] Furthermore, the specific steps of dividing the remaining Chinese medicines to be identified into multiple clusters are:
[0025] Step 601: taking the remaining Chinese medicines to be identified as a sample set to be scanned;
[0026] Step 602: Select a point from the sample set to be scanned and put it into the queue of the kth cluster;
[0027] Step 603: Search all points within the effective weight radius of the ith sample in the queue of the kth cluster to obtain multiple search points, then extract the ith sample point from the set of sample points to be scanned, and compare each search point with the existing sample points in the queue of the kth cluster: if it already exists in the queue of the kth cluster, do not process it; if it does not exist in the queue of the kth cluster, put the search point at the end of the queue of the kth cluster;
[0028] Step 604: Check whether the length of the queue of the kth cluster is equal to the search index value i. If not, increase the value of i by 1 and repeat step 603. If equal, the sample division of the kth cluster is completed, and check whether the number of elements in the sample set to be scanned is 0. If not, increase the value of k by 1 and return to step 602 for execution. If it is 0, the algorithm stops, and finally obtains the preliminary clustering results of multiple clusters, and there is no intersection element between any two clusters.
[0029] Furthermore, the specific steps of determining the cluster affiliation of the suspected problematic Chinese medicine are as follows:
[0030] Calculate the minimum distance from each suspected problematic Chinese medicine to each cluster;
[0031] Each suspected problematic Chinese medicine is assigned to the cluster with the smallest distance to it.
[0032] Furthermore, the specific steps of identifying problematic Chinese medicines are:
[0033] Based on the minimum distance, all suspected problematic Chinese medicines are arranged in ascending order;
[0034] Check each suspected problematic Chinese medicine in reverse order according to the arrangement order. If the minimum distance from a suspected problematic Chinese medicine to the suspected problematic Chinese medicines in other clusters is smaller than the minimum distance from the suspected problematic Chinese medicine to the remaining suspected problematic Chinese medicines in the cluster, then the suspected problematic Chinese medicine is a problematic Chinese medicine.
[0035] In a second aspect, the present invention provides a Chinese medicine identification system based on clustering;
[0036] A clustering-based Chinese medicine identification system, comprising:
[0037] A data acquisition module is configured to: acquire spectral characteristics of all Chinese medicines to be identified;
[0038] A weighted effective radius calculation module is configured to: calculate a distance matrix and a weighted effective radius based on spectral features;
[0039] A weight value calculation module is configured to: calculate the weight value of each Chinese medicine to be identified based on the weight effective radius and the distance matrix;
[0040] A suspected problematic Chinese medicine extraction module is configured to: extract suspected problematic Chinese medicines from all the Chinese medicines to be identified based on weight values to obtain the remaining Chinese medicines to be identified;
[0041] A cluster division module is configured to: divide the remaining Chinese medicines to be identified into a plurality of clusters;
[0042] The problematic Chinese medicine identification module is configured to: determine the cluster affiliation of the suspected problematic Chinese medicine and identify the problematic Chinese medicine.
[0043] In a third aspect, the present invention further provides an electronic device, comprising:
[0044] a memory for non-transitory storage of computer-readable instructions; and
[0045] a processor for executing the computer readable instructions,
[0046] When the computer-readable instructions are executed by the processor, the method described in the first aspect is executed.
[0047] In a fourth aspect, the present invention further provides a storage medium that non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the instructions of the method described in the first aspect are executed.
[0048] Compared with the prior art, the present invention has the following beneficial effects:
[0049] The present invention discloses a clustering-based Chinese medicine identification method, which can quickly find problematic Chinese medicines with different components and structures among a large number of Chinese medicines to be identified.
[0050] The clustering-based Chinese medicine identification method of the present invention reduces the influence of empirical parameters (radius and minimum number of points in the neighborhood of core objects) on the traditional spatial density clustering method, achieves better spatial density clustering effect, and improves the accuracy of problem Chinese medicine identification.
[0051] The present invention discloses a clustering-based traditional Chinese medicine identification method, which amplifies the difference between cluster boundary points and center points by weighting, screens outliers according to the total weight to reduce interference with subsequent processing, thereby achieving a better clustering effect.
[0052] The invention discloses a clustering-based Chinese medicine identification method, in which a weighted effective radius R is proposed, which is calculated based on sample distribution rather than estimated based on sample distribution experience, thereby effectively avoiding interference to clustering caused by insufficient experience of researchers and unreasonable parameter settings, and reducing trial and error costs.
[0053] The present invention discloses a clustering-based Chinese medicine identification method, which first removes some suspected problematic Chinese medicines for scanning and then determines the attribution of the remaining Chinese medicines. This method can more intuitively determine the distinguishability of the Chinese medicines and is also helpful in determining the degree of interference of boundary points on clustering, thereby solving the instability problem of the calculation results of traditional clustering algorithms being affected by the scanning starting point.
[0054] Advantages of additional aspects of the present invention will be partially given in the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0056] Figure 1 This is a schematic diagram of calculating the weighted effective radius according to the first embodiment of the present invention;
[0057] Figure 2 Schematic diagram of a weight value calculation method according to Embodiment 1 of the present invention;
[0058] Figure 3 This is a schematic diagram of a sample screening method according to Embodiment 1 of the present invention;
[0059] Figure 4 This is a schematic diagram of a scanning algorithm flow chart of the first embodiment of the present invention;
[0060] Figure 5 This is a schematic diagram of two-dimensional samples to be clustered according to the first embodiment of the present invention;
[0061] Figure 6 This is a schematic diagram of sample clustering results after removing suspected boundary points according to the first embodiment of the present invention;
[0062] Figure 7 This is a schematic diagram of sample clustering results for marking suspected boundary points according to the first embodiment of the present invention;
[0063] Figure 8 This is a schematic diagram of the clustering results of all sample points in Example 1 of the present invention. DETAILED DESCRIPTION
[0064] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0065] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0066] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.
[0067] In this embodiment, all data is obtained in compliance with laws and regulations and based on the user's consent, and is used legally.
[0068] Terminology explanation:
[0069] The density-based clustering algorithm DBSCAN is sensitive to parameters because it requires empirical settings for the search radius r and the minimum number of included points minPts. The algorithm uses these two parameters to determine the local density. The density definition actually uses the radius r to measure the minimum difference between the boundaries of two clusters and the limit of the scanning step of two iterations, πr 2 / minPts measures the average density of samples within a cluster. r is too small or minPts is too large, which results in the expected cluster being split into several small clusters with no substantial differences. On the contrary, it is easy to classify points with significant differences in spatial distribution into the same category. In addition, for samples at the boundary between clusters, their belonging may fluctuate depending on which cluster is detected first, so this processing method has a certain instability.
[0070] The OPTICS algorithm is based on DBSCAN and has been improved to make up for the limitation that the results are sensitive to empirical parameters. OPTICS does not display the resulting clusters, but generates an augmented cluster ranking for cluster analysis, which represents the density-based clustering structure of each sample point. The information it contains is equivalent to the density-based clustering obtained from a general parameter setting, that is, the clustering results of the DBSCAN algorithm based on any parameters r and minPts can be obtained from this ranking. Slight changes in r and MinPts do not affect the relative output order of the sample points and have no effect on the analysis of the clustering results.
[0071] Embodiment 1
[0072] This embodiment provides a traditional Chinese medicine identification method based on clustering;
[0073] A clustering-based traditional Chinese medicine identification method comprises the following steps:
[0074] Step 1: Obtain the spectral characteristics of all Chinese medicines to be identified.
[0075] Step 101: Use an instrument to collect spectra of all Chinese medicines to be identified.
[0076] As an implementation method, the spectrum is a terahertz time-domain spectrometer that collects spectral information of all Chinese medicine samples to be identified in the terahertz band; or, the spectrum is a near-infrared spectrum, ultraviolet-visible spectrum or Raman spectrum collected by instruments such as a Fourier infrared spectrometer, an ultraviolet-visible spectrophotometer or a Raman spectrometer.
[0077] Step 102: Perform dimension reduction processing on the spectral data and finally map it into two-dimensional data to obtain a spectral feature data set D after dimension transformation.
[0078] As an implementation method, after all spectral feature data of the Chinese medicine to be identified are collected, the distribution of sample points in the two-dimensional feature space is obtained by principal component analysis (PCA), or the two-dimensional feature data is obtained by dimensionality reduction in other ways.
[0079] Assuming that there are m Chinese medicines to be identified, the data set D after dimension transformation contains m samples, each sample has 2 dimensions, that is, D is an m*2 array, and the first and second columns are the values of the samples in the first and second dimensions respectively.
[0080] Step 2: Calculate the distance matrix Dist based on the spectral data of the Chinese medicine to be identified.
[0081] Specifically, the distances between all the Chinese medicine spectra to be identified in the data set D are calculated pairwise to obtain a matrix Dist of size m*m, where Dist(a,b) represents the distance between the a-th Chinese medicine to be identified and the b-th Chinese medicine to be identified, where a and b are both positive integers and 1≤a≤m, 1≤b≤m.
[0082] As an implementation method, each sample in the data set D is a two-dimensional vector, and the distance calculation is performed using the definition of Euclidean distance, that is,
[0083]
[0084] Step 3: Calculate the weighted effective radius R based on the distance matrix.
[0085] The present invention determines the effective radius of weight based on the two-dimensional spatial distribution of all sample points. The specific calculation method of the effective radius of weight R is to select the minimum value greater than 0 in each column of the distance matrix Dist, and then use the maximum value of all minimum values as the effective radius of weight R. The reason for selecting this value as the effective radius of weight is that no matter how the sample points are distributed, all samples are not the only points in their R neighborhood, and the radius R is as small as possible to prevent scattered noise points from making a large contribution to the total weight value.
[0086] Specifically, Figure 1 As shown, assume that there are 6 data points. First, calculate the distance matrix Dist (diagram matrix, ①) for all data points in the two-dimensional distribution. xy Represents the distance from the sample marked as x to the sample marked as y, and then determines the minimum radius of each point to its neighboring point based on each column of the matrix (find the minimum positive value of each column in the table shown in the figure, as shown by the bold elements in ② and ①), and finally selects the largest one as the neighborhood radius R by comparing the minimum values of these radii (find the maximum value of the row vector, as shown in ③).
[0087] It should be noted that the weighted effective radius R and the radius in the DBSCAN method have the following important differences: the weighted effective radius described in the present invention is obtained by calculation, is only related to the data distribution of the sample set, and does not rely on any experience. The radius r in the DBSCAN method can be selected manually based on experience in combination with the sample distribution conditions; the role of calculating R in the present invention is to determine the range of weighted neighboring points, and the role of r in the DBSCAN method is to determine whether each point belongs to a core point in combination with the parameter minPts. In the present invention, all valid data points are treated equally, there is no concept and annotation of "core points" in the scanned objects, and there is no parameter equivalent to minPts.
[0088] Step 4: Based on the weighted effective radius and the distance matrix, the weight value of each Chinese medicine to be identified is calculated, that is, the weight of each sample in the data set D is calculated.
[0089] Among them, the specific calculation steps of the weight value are: first, all the Chinese medicines to be identified within the weight effective radius R of the Chinese medicine to be identified for which the weight value needs to be calculated are taken as the center point; then, a center point is randomly selected, and all the Chinese medicines to be identified within the weight effective radius R of the center point are selected; then, all the selected Chinese medicines to be identified are sorted in descending order according to the distance from the center point, and the weight contribution of the Chinese medicine to be identified for which the weight value needs to be calculated at the center point is the sorting sequence number; finally, until all the centers are selected, the weight contributions of the Chinese medicine to be identified for which the weight value needs to be calculated at all the center points are obtained, and the weight contributions are added up to obtain the weight value.
[0090] In other words, the total weight value W of the i-th samplei It is the sum of the weights contributed by each sample point in its R neighborhood (R neighborhood: a circle with the sample point as the center and R as the radius, including the interior and the boundary) when it is the center point. Specifically, suppose the neighborhood of a certain sample (center point) contains N points (excluding itself, a total of N-1 points), and the distances from the N-1 points to the center point are sorted in descending order, then the sequence number represents the weight contribution of the center point to the corresponding point. In particular, the weight contribution of any sample to itself is equal to the total number of points N in its R neighborhood.
[0091] Specifically, Figure 2 As shown in the figure, the sample distribution of each area is consistent, indicating the weights contributed by a, b, c, d, e, and f to other points. A circle is drawn with a as the center and R as the radius. The obtained black solid circle contains 4 points. They are sorted in descending order according to the distance from each point to a. The serial number indicates the contribution of point a to other points. The contribution of a to its own weight is the total number of points in and on the black solid circle (i.e., the boundary indicated by the neighborhood radius R), which is 4. Therefore, the contribution of d from a is 2. Since points e and f are outside the effective radius of the weight, a contributes 0 to their weights. Similarly, b contributes 2 to the weight of d, c contributes 2 to the weight of d, d contributes 6 to the weight of d, e contributes 1 to the weight of d, and f contributes 2 to the weight of d. As mentioned above, the total weight value obtained by point d is the sum of the weight values given to it by all points in its neighborhood R. Therefore, its total weight value is 2+2+2+6+1+2=15. In other examples, although the sample distribution and the calculated R are different, the calculation method is the same as this example.
[0092] The advantage of the weight calculation rule of the present invention is that within the effective weight range, samples at close distances can provide larger weights, and the more samples there are in the R neighborhood, the larger the maximum weight that the central point can give to others, so that the total weight value obtained by the sample points takes into account both distance and density. In addition, the distance matrix has been calculated, and the weight is directly obtained by comparing the distance, which has a faster calculation speed, thereby accelerating the speed of identification of traditional Chinese medicine.
[0093] Step 5: Identify suspected boundary points based on the weight values, extract suspected boundary points (i.e. suspected problematic Chinese medicines) from all Chinese medicines to be identified, and obtain the remaining Chinese medicines to be identified and their distance matrix. Among them, the suspected boundary points are Chinese medicines to be identified whose weight values are less than the set threshold.
[0094] The first column of the data set D is denoted as D1, which is a column vector; all sample points are arranged in ascending order of weight, and some points with too small weights are removed (extracted from D1, that is, let D1(x)| Wx<W = []). All sample points extracted do not participate in subsequent processing, and the remaining relative order remains unchanged. The data set after extracting the singular points is still recorded as D1, and its distance matrix is Dist1.
[0095] like Figure 3 As shown in , the samples of a specific distribution are weighted and then statistics are performed based on the weights. Figure 3 In the above figure, it is observed that the sample points are relatively concentrated, and the points with a total weight value less than 100 account for a small proportion of the total. It can be empirically determined that W = 100. This value affects the subsequent calculation amount, but as long as the several areas surrounded by the suspected boundary points can be clearly separated, it will not affect the final result.
[0096] Step 6: Divide the remaining Chinese medicines to be identified into multiple clusters, that is, take the remaining Chinese medicines to be identified as the sample set to be scanned, and scan the sample set to be scanned to obtain multiple clusters, such as Figure 4 As shown, the specific steps are as follows:
[0097] Step 601: Assuming that all points in the sample set to be scanned are divided into k clusters, the cluster number k is initialized to 1, an empty queue "k" (i.e., queue "1") is established, the sample set to be scanned S is initialized to {1, 2, 3...m}, and the search index i is initialized to 1;
[0098] Step 602: Select a point from the sample set S to be scanned (i.e., a feature point of the remaining Chinese medicine to be identified) and put it into the queue "k";
[0099] Step 603: Search all points in the neighborhood of the i-th sample R in queue "k" (i.e., within the effective radius of the weight), obtain multiple search points, and then extract the sample points from the set of sample points to be scanned. Compare each search point with the existing sample points in queue "k". Points that already exist in queue "k" are not processed, and all search points that do not exist in queue "k" are placed at the end of queue "k";
[0100] Step 604, check whether the length of queue "k" is equal to the search index value i, if the result is "no", then i=i+1 (replace the original variable i with the value i+1), and repeat step 603; if the result is "yes", the sample division of the kth cluster is completed, check whether the number of elements in the sample set to be scanned is 0, if the result is "no", select any unprocessed sample point from it, execute k=k+1 (replace the original variable k with the value k+1, and start generating the "k+1"th cluster), and return to step 602 for execution; if the result is "yes", the algorithm stops, and finally obtains the preliminary clustering results of multiple clusters, and there is no intersection element between any two clusters.
[0101] Note: Step 6 describes a data scanning method, which is essentially still a density-based scan, but is different from the DBSCAN method and the OPTICS method. The technical solution of the present invention does not have the concept of minPts (the minimum number of points in the R neighborhood required to become a core point), so it is impossible to generate core points and boundary points in the DBSCAN and OPTICS methods. Possible "abnormal points" or "noise points" have been processed in step 5. In this step, all points with a step length within R from any point in the cluster are attributed to the cluster.
[0102] Step 7. Determine the cluster affiliation of suspected boundary points (i.e., suspected problematic Chinese medicines) according to the nearest neighbor principle. These clusters represent different types of Chinese medicines, and identify the Chinese medicine samples to be tested and re-tested. Specifically, calculate the minimum distance from each suspected problematic Chinese medicine to each cluster, and assign each suspected problematic Chinese medicine to the cluster where the sample with the smallest minimum distance is located; based on the minimum distance, arrange all suspected problematic Chinese medicines in ascending order; check each suspected problematic Chinese medicine in reverse order according to the arrangement order. If the minimum distance from a suspected problematic Chinese medicine to suspected problematic Chinese medicines in other clusters is less than the minimum distance from the suspected problematic Chinese medicine to the remaining suspected problematic Chinese medicines in the cluster, then the suspected problematic Chinese medicine is an abnormal Chinese medicine to be tested and re-tested. The specific steps are as follows:
[0103] Step 701, determine the determination order: calculate the k minimum distances from each suspected boundary point to the k cluster points respectively and take the minimum value, then sort all the suspected boundary points in ascending order according to the minimum value, and determine the cluster affiliation of each point in turn according to the sorting order.
[0104] Step 702, determine the final cluster belonging of each suspected boundary point: if the minimum distance from any suspected boundary point to the i-th cluster is less than the minimum distance from it to all other clusters, then the suspected boundary point is determined to belong to the i-th cluster. When calculating the minimum distance from a suspected boundary point to a cluster, the suspected boundary point newly added to the cluster is never considered, that is, the minimum distance must exist between a scanned point (the point processed in step 6) and the suspected boundary point.
[0105] Step 703: After all suspected boundary points are processed, the minimum distance from each suspected boundary point to the corresponding suspected boundary points in other clusters is checked in reverse order. If the distance (points a and b) is less than the minimum distance from it to the remaining suspected boundary points of this cluster, a and b are marked as boundary points.
[0106] After step 703, all algorithms are finished and cluster labels of all samples are obtained. Each sample point has a unique cluster to which it belongs. Some samples may be marked as boundary points, and the boundary points are the Chinese medicine samples to be tested and re-tested.
[0107] In the experiment, samples with compact distribution and less interference were clustered. The samples were obtained by principal component analysis and dimensionality reduction of spectra obtained by spectral measurement of three similar types of traditional Chinese medicine. In the experiment, the samples were finally divided into three clusters, with fewer interfering sample points and relatively clear boundaries between clusters. After weighted processing, the "abnormal points" found were mostly sample points far away from the center of the cluster, and no point was finally marked as a boundary point.
[0108] Figure 5 It is a schematic diagram of unclassified samples, and the data distribution is typical to a certain extent. Samples with similar properties are distributed close to each other, and samples with obvious differences in properties are distributed farther apart. After being processed by the technical solution involved in the present invention, some samples are marked as suspected boundary samples, which are surrounded by several areas. The sample points in the same area are concentrated, corresponding to the points with high weight values; the sample points in different areas can be more clearly separated (the minimum distance between samples from two different clusters is greater than the minimum distance between the two sample points to other points in their respective clusters). The schematic diagram of the sample clustering results after removing the boundary points is shown in the figure. Figure 6 As shown in the figure, the sample clustering result diagram with boundary information is as follows Figure 7 The clustering results of all sample points are shown in Figure 8 Compare Figure 7 and Figure 8 It can be seen that due to the long distance between the boundaries of samples in different clusters, the cluster categories of boundary samples and the center samples of the surrounding area are the same.
[0109] Figure 8 It also shows that the final cluster affiliation of all suspected boundary points has been confirmed, and no suspected boundary point has been confirmed as a "boundary point", indicating that the clustering effect is relatively ideal. Figure 8 The clustering results given are consistent with the existing labels, indicating that terahertz spectroscopy can effectively distinguish the three types of samples, and individual differences in the same drug have little effect on the classification; if the clustering results of some samples do not match the known labels, that is, boundary points, these samples will be sent for chromatographic experiments to specifically analyze the abnormal components.
[0110] Due to the differences in types of Chinese medicine, there are differences in the spectral information of Chinese medicine, and the content of some individual components deviates significantly during the introduction, cultivation and processing stages of the medicinal materials. Although some medicines have different functions, their spectra are highly similar. These special medicines need in-depth research and analysis. The purpose of this application is to divide the Chinese medicine into different clusters based on the spectral characteristics of the Chinese medicine to be identified, and to identify heterogeneous Chinese medicine samples that deviate from general characteristics or samples with highly similar spectral characteristics for subsequent analysis to reduce the overall research and measurement costs. There is interference in the spectrum of Chinese medicine, and the problem of high differences in data dimensions is difficult to visualize, and the presence of a small amount of abnormal Chinese medicine will interfere with the clustering of other samples. Moreover, in the analysis of various electromagnetic spectra including terahertz spectra, the traditional density-based clustering method has the problem of being sensitive to the definition of radius and core objects. The method of the present invention takes into account the influence of the distribution of points around each sample when determining the neighborhood radius of the sample point, determines the importance of the sample in the clustering process by the total weight of each sample point, discards the suspected boundary points that are easy to interfere with before the scanning starts, and effectively makes up for the limitation of the high reliance on experience judgment in the classic density clustering method DBSCAN, so it has a better processing effect and improves the accuracy of Chinese medicine identification; in addition, the method of this embodiment, in the process of processing suspected boundary points, reasonably limits the discrimination order, eliminates the uncertainty of processing boundaries to a certain extent, therefore, the method of this embodiment realizes data mining and in-depth analysis of spectra, which is helpful to carry out various indirect studies including the detection of the authenticity of Chinese medicine.
[0111] Embodiment 2
[0112] This embodiment provides a traditional Chinese medicine identification system based on clustering;
[0113] A clustering-based Chinese medicine identification system, comprising:
[0114] A data acquisition module is configured to: acquire the spectral features of all Chinese medicines to be identified; it includes a spectral acquisition module and a preprocessing module; wherein the spectral acquisition module is a terahertz time-domain spectrometer, a Fourier transform infrared spectrometer, an ultraviolet visible spectrophotometer or a Raman spectrometer, and is used to acquire spectra of all Chinese medicine samples to be identified; the preprocessing module is connected to the spectral acquisition module, and is used to perform dimensionality reduction processing on the spectral features, convert the spectral data into two-dimensional data, and obtain the spectral features after the dimension transformation.
[0115] A weighted effective radius calculation module is configured to: calculate a distance matrix and a weighted effective radius based on spectral features;
[0116] A weight value calculation module is configured to: calculate the weight value of each Chinese medicine to be identified based on the weight effective radius and the distance matrix;
[0117] A suspected problematic Chinese medicine extraction module is configured to: extract suspected problematic Chinese medicines from all the Chinese medicines to be identified based on weight values to obtain the remaining Chinese medicines to be identified;
[0118] A cluster division module is configured to: divide the remaining Chinese medicines to be identified into a plurality of clusters;
[0119] The problematic Chinese medicine identification module is configured to: determine the cluster affiliation of the suspected problematic Chinese medicine and identify the problematic Chinese medicine.
[0120] It should be noted that the above modules correspond to the steps in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Embodiment 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.
[0121] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0122] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.
[0123] Embodiment 3
[0124] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory so that the electronic device executes the method described in the above embodiment one.
[0125] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0126] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.
[0127] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.
[0128] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.
[0129] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0130] Embodiment 4
[0131] This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in the first embodiment is completed.
[0132] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A clustering-based identification method for traditional Chinese medicine. Its characteristics are: include: Collect spectra of all Chinese medicines to be identified; Perform dimension reduction processing on the spectral data and map it into two-dimensional data to obtain a spectral feature data set after dimension transformation; The distances between all the Chinese medicine spectra to be identified in the spectral feature data set are calculated pairwise to obtain a distance matrix; the minimum value greater than 0 in each column of the distance matrix is selected; The maximum value among all minimum values is taken as the effective radius of weight; Based on the weighted effective radius and the distance matrix, the weight value of each Chinese medicine to be identified is calculated; The specific calculation steps of the weight value are as follows: all the Chinese medicines to be identified within the effective weight radius of the Chinese medicines to be identified whose weight values need to be calculated are taken as the center point; a center point is randomly selected, and all the Chinese medicines to be identified within the effective weight radius of the center point are selected; According to the distance from the center point, all selected Chinese medicines to be identified are sorted in descending order, and the weight contribution of the Chinese medicine to be identified that needs to calculate the weight value under the center point is the sorting sequence number; Until all centers are selected, the weight contributions of the Chinese medicine to be identified under all center points for which weight values need to be calculated are obtained, and the weight contributions are summed to obtain the weight value; Based on the weight value, suspected problematic Chinese medicines are extracted from all the Chinese medicines to be identified, and the remaining Chinese medicines to be identified are obtained; The remaining Chinese medicines to be identified are divided into multiple clusters; Determine the cluster affiliation of suspected problematic Chinese medicines and identify the problematic Chinese medicines.
2. A clustering-based Chinese medicine identification method as claimed in claim 1, Its characteristics are: The suspected problematic Chinese medicine is a Chinese medicine to be identified whose weight value is less than a set threshold.
3. A clustering-based Chinese medicine identification method as claimed in claim 1, Its characteristics are: The specific steps of dividing the remaining Chinese medicines to be identified into multiple clusters are: Step 601: taking the remaining Chinese medicines to be identified as a sample set to be scanned; Step 602: Select a point from the sample set to be scanned and put it into the queue of the kth cluster; Step 603: Search all points within the effective weight radius of the ith sample in the queue of the kth cluster to obtain multiple search points, then extract the ith sample point from the set of sample points to be scanned, and compare each search point with the sample points already in the queue of the kth cluster; if it already exists in the queue of the kth cluster, do not process it; if it does not exist in the queue of the kth cluster, put the search point at the end of the queue of the kth cluster; Step 604: Check whether the length of the queue of the kth cluster is equal to the search index value i. If not, increase the value of i by 1 and repeat step 603. If equal, the sample division of the kth cluster is completed, and check whether the number of elements in the sample set to be scanned is 0. If not, increase the value of k by 1 and return to step 602 for execution. If it is 0, the algorithm stops, and finally obtains the preliminary clustering results of multiple clusters, and there is no intersection element between any two clusters.
4. A clustering-based Chinese medicine identification method as claimed in claim 1, Its characteristics are: The specific steps of determining the cluster affiliation of the suspected problematic Chinese medicine are as follows: Calculate the minimum distance from each suspected problematic Chinese medicine to each cluster; Each suspected problematic Chinese medicine is assigned to the cluster with the smallest distance to it.
5. A clustering-based Chinese medicine identification method as claimed in claim 4, Its characteristics are: The specific steps of identifying problematic Chinese medicines are: Based on the minimum distance, all suspected problematic Chinese medicines are arranged in ascending order; Check each suspected problematic Chinese medicine in reverse order according to the arrangement order. If the minimum distance from a suspected problematic Chinese medicine to the suspected problematic Chinese medicines in other clusters is smaller than the minimum distance from the suspected problematic Chinese medicine to the remaining suspected problematic Chinese medicines in the cluster, then the suspected problematic Chinese medicine is a problematic Chinese medicine.
6. A clustering-based Chinese medicine identification system, Its characteristics are: include: A data acquisition module is configured to: collect spectra of all Chinese medicines to be identified; Perform dimension reduction processing on the spectral data and map it into two-dimensional data to obtain a spectral feature data set after dimension transformation; The weighted effective radius calculation module is configured to: calculate the distance between all the Chinese medicine spectra to be identified in the spectral feature data set, and obtain a distance matrix; select the minimum value greater than 0 in each column of the distance matrix; The maximum value among all minimum values is taken as the effective radius of weight; The weight value calculation module is configured to: calculate the weight value of each Chinese medicine to be identified based on the weight effective radius and the distance matrix; the specific calculation steps of the weight value are: all the Chinese medicines to be identified within the weight effective radius of the Chinese medicine to be identified for which the weight value needs to be calculated are taken as the center point; randomly select a center point, and select all the Chinese medicines to be identified within the weight effective radius of the center point; According to the distance from the center point, all selected Chinese medicines to be identified are sorted in descending order, and the weight contribution of the Chinese medicine to be identified that needs to calculate the weight value under the center point is the sorting sequence number; Until all centers are selected, the weight contributions of the Chinese medicine to be identified under all center points for which weight values need to be calculated are obtained, and the weight contributions are summed to obtain the weight value; A suspected problematic Chinese medicine extraction module is configured to: extract suspected problematic Chinese medicines from all the Chinese medicines to be identified based on weight values to obtain the remaining Chinese medicines to be identified; A cluster division module is configured to: divide the remaining Chinese medicines to be identified into a plurality of clusters; The problematic Chinese medicine identification module is configured to: determine the cluster affiliation of the suspected problematic Chinese medicine and identify the problematic Chinese medicine.
7. An electronic device, Its characteristics include: a memory for non-transitory storage of computer readable instructions; as well as a processor for executing the computer readable instructions, When the computer-readable instructions are executed by the processor, the method described in any one of claims 1 to 5 is executed.
8. A storage medium, Its characteristics are: The computer-readable instructions are non-transitory stored, wherein when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method according to any one of claims 1 to 5 are executed.
