Multi-label traditional Chinese medicine syndrome differentiation method based on tongue vein data and fuzzy neural network
Through the multi-label TCM dialectical method based on tongue pulse data and fuzzy neural network, a smart TCM dialectical network model was constructed, which solved the problem of TCM dialectical dependence on experience and information complexity, and achieved more accurate syndrome identification and multi-label prediction, improving The application effect of intelligent traditional Chinese medicine.
Patent Information
- Application Number
- CN202410818982.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-24
- Publication Date
- 2024-11-19
AI Technical Summary
Traditional Chinese medicine syndrome differentiation relies on doctor experience and complex four-diagnosis information, lacks unified standards, and it is difficult to achieve accurate multi-label syndrome identification. Moreover, the intelligent application of traditional Chinese medicine is limited by the number of advanced experts and workload.
The multi-label Chinese medicine dialectical method based on tongue pulse data and fuzzy neural network is adopted to pre-process the tongue pulse signal through wavelet transformation and cubic spline interpolation method, combining residual network, one-dimensional convolution neural network and multimodal fuzzy neural network. Build a Chinese medicine intelligent dialectical network model, integrate tongue pulse and tongue image characteristics for multi-label prediction.
It realizes more accurate identification of Chinese medicine syndromes, improves the accuracy of multi-label prediction and model generalization capabilities, and is better than the performance of commonly used machine learning methods.
Smart Images

Figure CN118983073A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of medicine, deep learning, and multimodal fusion technology, and specifically to a multi-label Chinese medicine syndrome differentiation method based on tongue and pulse data and fuzzy neural network. Background Art
[0003] TCM theory emphasizes the unity and integrity of the human body, and believes that the human body is an organic whole with the five internal organs (heart, liver, spleen, lungs, and kidneys) as the core. The various components of the human body coordinate and restrict each other in function and influence each other in pathology. Therefore, the external organs of the human body are connected to the internal organs through the meridian system, and the information of internal organ diseases can be reflected in the external organs and tissues, which doctors can detect through TCM diagnostic methods. In TCM diagnosis, doctors collect and analyze the patient's physiological state from different angles through the four diagnostic methods of inspection, auscultation, questioning, and palpation, and obtain the final diagnosis result after comprehensively considering the patient information collected through the four examinations.
[0004] As a key component of the TCM diagnosis and treatment system, TCM syndrome differentiation faces many challenges in practical application. First, TCM syndrome differentiation depends largely on the doctor's personal experience and subjective judgment, which requires doctors to have rich practical experience and theoretical knowledge to make accurate analysis and diagnosis. Secondly, the information obtained from the four examinations of TCM is complex and lacks unified standards, and it usually takes years of study and practice to master. In addition, due to the practicality of TCM theory and operation skills, the training of TCM practitioners depends more on the inheritance of teachers and clinical practice, which limits the widespread application of TCM. In addition, the number of senior TCM experts is limited, and the workload of the TCM industry is heavy. Therefore, it is necessary to use artificial intelligence and fuzzy logic technology to promote the intelligent syndrome differentiation of TCM. Summary of the invention
[0005] The technical problem to be solved by this application is: how to further apply deep learning and fuzzy logic technology to TCM intelligent syndrome differentiation to achieve accurate multi-label syndrome recognition. This application provides a multi-label TCM syndrome differentiation method based on tongue and pulse data and fuzzy neural network to solve the above problem, and evaluates its performance.
[0006] The present application is implemented through the following technical solutions: The present application provides a multi-label TCM syndrome differentiation method based on tongue and pulse data and fuzzy neural network, the method comprising:
[0007] Step 1: Select the subjects and use the corresponding data acquisition equipment to collect the clinical tongue and pulse data and tongue pictures of the subjects;
[0008] Step 2: TCM experts label the tongue and pulse data, tongue pictures, and syndromes of the subjects;
[0009] Step 3: Use wavelet transform method to remove high-frequency noise of tongue and pulse signals and remove the baseline of tongue and pulse signals based on cubic spline interpolation method;
[0010] Step 4: Remove the background information in the original tongue image and keep only the tongue body;
[0011] Step 5: Construct a TCM intelligent syndrome differentiation network model, which consists of three parts: residual network, one-dimensional convolutional neural network and multimodal fuzzy neural network;
[0012] Step 6: Input the preprocessed tongue image to the residual network for multi-label prediction of tongue image, and input the preprocessed tongue pulse data to the one-dimensional convolutional neural network for multi-label prediction of tongue pulse signals;
[0013] Step 7: Use multimodal fuzzy neural network to fuse tongue pulse and tongue image features to perform multi-label prediction of TCM syndromes;
[0014] Step 8: Use the data obtained in steps 3 and 4 to train the TCM intelligent syndrome differentiation network model according to the methods in steps 5 to 7; use the trained model to perform multi-label prediction on the newly obtained tongue and pulse images.
[0015] Furthermore, the one-dimensional convolutional neural network input signal in step 5 is the pulse data X′ RP , denote X′ RP The three tongue pulse signals in are [α 1,1 ,α 1,2 ,α 1,3 ], α 1,1 ,α 1,2 ,α 1,3 The three pulse signals are obtained after cubic spline interpolation, and then through N P A one-dimensional convolutional layer extracts high-level features of the tongue and pulse X P ; The operation method of the one-dimensional convolution layer is:
[0016]
[0017] in, For nth P The output feature vector obtained after the jth convolution calculation of the convolutional layer, n P =1,…,N P , j = 1, ..., J, J represents the total number of convolution kernels, For nth P The mth input feature vector of a one-dimensional convolutional layer, m = 1, ..., M, M is the number of input feature vectors, * represents the convolution operation, For nth P The jth convolution kernel in the one-dimensional convolution layer is convolved with the mth input feature vector. For nth P The bias term of the jth convolution kernel in the one-dimensional convolution layer, σ is the Relu activation function; the feature vector of the last layer of the one-dimensional convolution Combining features into a feature map
[0018]
[0019] Get high-level features of tongue and pulse X P After that, the first stage of pulse multi-label classification will be carried out; first, the feature map X P Flatten into a one-dimensional vector x P ;
[0020]
[0021] in represents vector transposition, flatten(.) represents the flattening operation, and then passes through the fully connected layer;
[0022] h P =σ(W P1 x P +b P1 )
[0023] z P =W P2 h P +b P2
[0024] Among them, W P1 , W P2 is the weight parameter of the fully connected layer, b P1 、b P2 is the bias term; h P is the intermediate parameter of the calculation process;
[0025] After full connection, we get vector z P , z P The dimension of corresponds to the number of pulse label classes; finally, in the output layer, the vector z P Each value in uses the Sigmoid function to obtain the final predicted probability value of each type of pulse label;
[0026]
[0027] in It is the mth P The probability value of pulse prediction, M P is the total number of pulse categories, Indicates the mth P Pulse input value;
[0028] make It is the pulse multi-label prediction vector, so the overall forward propagation process of the one-dimensional convolutional neural network is expressed as follows;
[0029]
[0030] where θ P is the parameter to be optimized for the entire one-dimensional convolutional neural network, X′ RP represents the preprocessed pulse data, and 1DCNN() represents a one-dimensional convolutional neural network.
[0031] Furthermore, the residual network loss function in step 5 is:
[0032]
[0033] represents the predicted probability value set of tongue image data, Represents the predicted probability value of tongue image data;
[0034] Furthermore, the multimodal fuzzy neural network in step 7 includes: a pre-network and a post-network; the number of input nodes of the pre-network and the post-network is M TP ; and the input data range is from 0 to 1, recorded as Indicates the probability value of the corresponding tongue and pulse prediction;
[0035] The antecedent network includes: a fuzzy set partitioning layer, a fuzzification layer, a fuzzy rule layer, and a normalization layer. The number of nodes in the fuzzy set partitioning layer is M. TP , the label corresponding to each node is divided into three fuzzy sets with different credibility; the number of nodes in the fuzzification layer is 3M TP , each node is equipped with a fuzzy membership function, and the fuzzy membership function is used to generate a membership value for the three fuzzy sets corresponding to each label;
[0036] The fuzzy rule layer includes K nodes π1,...,π K , corresponding to K pieces of TCM knowledge, each π k According to the premise of TCM knowledge k Select the membership value of the corresponding fuzzy set as input, and perform a permutation combination on the input value; then use the multiplication operator on the membership of each fuzzy set in the fuzzy rule to obtain the applicability of the fuzzy rule. The calculation expression is:
[0037]
[0038] in, represents the rth item under the kth piece of TCM knowledge k The applicability of fuzzy rules is fuzzy rules, Lk represents the number of tongue and pulse labels in the kth piece of TCM knowledge, Ψ represents the set formed by permuting and combining the corresponding membership degrees according to TCM knowledge, represents the membership values in Ψ; the total number of fuzzy rules finally obtained is:
[0039]
[0040] It can be seen that the number of fuzzy rules for each TCM knowledge transformation is the same as L k Therefore, in order to more uniformly express the applicability of fuzzy rules, the following conversion is used The subscript is ρ r , let R0=0,
[0041]
[0042] Among them, ρ r is the applicability of the rth fuzzy rule, R n-1 represents the total number of fuzzy rules converted from the n-1th piece of TCM knowledge, and R represents the total number of fuzzy rules obtained after all TCM knowledge is converted;
[0043] The normalization layer is used to perform normalization calculation on the applicability of the R fuzzy rules;
[0044]
[0045] in It is called the normalized applicability of the rth fuzzy rule;
[0046] Furthermore, the specific method of the post-processing network is:
[0047] Step s1: where d nr represents the output value of the nth label predicted under the rth fuzzy rule, f nr (·) represents a linear function, b nr is the bias term, is the weighting coefficient;
[0048] Step s2: Get the fuzzy rule output value d nr After that, it will undergo a defuzzification process, which consists of N parallel sub-networks with the same structure, each of which will produce a label prediction quantity;
[0049]
[0050] Among them, h n is the weighted sum of the consequents of each fuzzy rule, and these weighted coefficients are the normalized applicability of the antecedents of each fuzzy rule. represents the normalized fitness, dnr represents the output value of the consequent network, and N represents the number of syndromes to be predicted;
[0051] Step s3: The output layer uses the sigmoid function to obtain the predicted probability values of various TCM labels
[0052]
[0053] Through the above steps, the present application realizes intelligent prediction of TCM syndromes through the TSDN model, and compared with other commonly used machine learning methods, the present invention has higher evaluation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 The following is a flowchart of the TSDN model and its two-stage training.
[0055] Figure 2 This is the flowchart of multi-label prediction of tongue and pulse signals based on the 1DCNN model.
[0056] Figure 3 This is the flow chart of tongue image multi-label prediction based on ResNet.
[0057] Figure 4 It is the MFNN structure based on traditional Chinese medicine knowledge. DETAILED DESCRIPTION
[0058] Pulse and tongue images each provide rich health information about the human body, but analyzing them separately often makes it difficult to fully grasp the complex physiological phenomena of the human body. This is because TCM theory emphasizes the principle of holistic view and syndrome differentiation and treatment, and believes that the human body is a unified whole, with close connections between the internal organs and between the internal organs and the external environment. Therefore, only by combining the data of pulse and tongue images and comprehensively analyzing the information reflected by the two can we more accurately reveal the physiological state of the human body and the existing health problems, and at the same time help TCM practitioners identify the syndrome types of diseases.
[0059] In TCM theory, syndrome is a very important concept. It is a disease classification based on the changes in the patient's condition and the specific manifestations of the disease development. Each syndrome represents a specific pathological state. TCM treatment is based on syndromes. Only by accurately judging the patient's syndrome type can we provide a basis for subsequent treatment options, thereby achieving the right medicine for the right condition and avoiding unnecessary side effects. This is what TCM calls "treatment based on syndrome differentiation."
[0060] This application will construct a TCM Intelligent Syndrome Differentiation Network (TSDN), which can not only use the data of both tongue and pulse, but also use the theoretical knowledge of TCM as a guide to better integrate tongue and pulse information. In addition, in order to make full use of the pulse and tongue labels in the tongue and pulse data, this paper also designs a two-stage training scheme. The overall process and each module are as follows Figure 1 As shown, through a phased training process, the scenario of a TCM expert from perceiving the basic tongue, pulse and tongue, to summarizing the pulse and tongue patterns, and then to diagnosing TCM syndromes is simulated, fully tapping and utilizing the potential of these TCM labels and prior knowledge.
[0061] The intelligent syndrome differentiation network consists of three core modules: pulse feature extraction module, tongue feature extraction module, and multimodal fuzzy neural network module based on TCM knowledge. Among them, the pulse feature extraction module adopts the 1DCNN model, and the tongue feature extraction module adopts the ResNet-50 model. Tongue pulse and tongue image are two heterogeneous data, which reflect the state of the human body from different angles and belong to a multimodal problem. This application constructs a multimodal fuzzy neural network to realize the fusion of tongue and pulse data. It refers to the theory of TCM syndrome differentiation, combines the prior knowledge of traditional Chinese medicine with fuzzy logic, and constructs a set of special fuzzy rules. The specific steps are as follows:
[0062] Step 1:
[0063] The one-dimensional convolutional neural network model established in this paper is as follows Figure 2 The input of the model is the tongue and pulse data X′ after preprocessing. RP , denote X′ RP The three tongue pulse signals in are [α 1,1 ,α 1,2 ,α 1,3 ], then through N P A one-dimensional convolutional layer extracts high-level features of the tongue and pulse X P The processing of each one-dimensional convolution layer is similar, and they all contain a set of learnable one-dimensional convolution kernels. Each convolution kernel slides on the time dimension of the tongue and pulse data, calculates the linear weighted sum of the local area, and introduces nonlinearity through the activation function to enhance the expressiveness of the features. The output of each one-dimensional convolution layer is called a feature map. The feature map reflects the abstraction and representation of different features in the tongue and pulse data at different levels, and the obtained feature map will serve as the input of the next one-dimensional convolution layer.
[0064] The one-dimensional convolutional layer operation can be described by the following formula
[0065]
[0066] in, For nth P (n P =1,…,N P ) is the output feature vector obtained after the jth (j=1,…,J) convolution calculation of the convolutional layer, n P The mth (m=1,...,M) input feature vector of the layer, M is the number of input feature vectors, * represents the convolution operation, is the jth convolution kernel convolved with the mth input feature vector, is the bias term, σ is the Relu activation function. The feature vector of the last layer of the one-dimensional convolution Can be combined into feature maps
[0067] Get high-level features of tongue and pulse X P After that, the first stage of pulse multi-label classification will be carried out. First, the feature map X P Flatten into a one-dimensional vector
[0068]
[0069] in represents the vector transpose. Then through the fully connected layer
[0070] h P =σ(W P1 x P +b P1 )
[0071] z P =W P2 h P +b P2
[0072] Among them, W P1 , W P2 is the weight parameter of the fully connected layer, b P1 、b P2 is the bias term. In addition, the fully connected layer can capture complex data patterns due to its high number of parameters and complex feature combination capabilities, but it may also capture the noise in the training data, which is prone to overfitting when making decisions. Therefore, this paper applies the Dropout mechanism in the fully connected layer, which randomly discards some neurons to reduce the dependence on specific training samples, thereby enhancing the generalization ability of the model and reducing the risk of overfitting.
[0073] Vector z P is a 17-dimensional vector, corresponding to the 17 common pulse labels in the collected tongue and pulse dataset. Finally, in the output layer, the vector z PEach value in uses the Sigmoid function to obtain the final predicted probability value of each type of pulse label
[0074]
[0075] in It is the mth P The probability value of pulse prediction, M P It is the total number of pulse categories.
[0076] make It is the pulse multi-label prediction vector, so the overall forward propagation process of the one-dimensional convolutional neural network is expressed as follows
[0077]
[0078] where θ P It is the parameter to be optimized for the entire one-dimensional convolutional neural network.
[0079] Step 2:
[0080] Each deep learning network in tongue image multi-label classification and recognition goes through a similar process, such as Figure 3 As shown. Vector z T The dimension of is 17, which corresponds to the 17 common tongue image labels in the collected tongue and pulse dataset. Finally, in the output layer, the vector z T Each value in uses the Sigmoid function to obtain the final predicted probability value of each type of tongue image label
[0081]
[0082] in It is the mth T The probability value of tongue image prediction, M T It is the total number of tongue image categories.
[0083] make It is a multi-label prediction vector of tongue image, and constructs the target vector y according to the tongue image labels of TCM experts T , the target vector is also a 17-dimensional vector, each dimension represents a tongue feature, such as pale red tongue, yellow fur, cracks, etc. The loss function uses binary cross entropy loss, treating each tongue label as an independent binary classification problem:
[0084]
[0085] in It is the mth T The true value of the tongue image label (0 means no, 1 means yes).
[0086] Step 3:
[0087] The multimodal fuzzy neural network MFNN based on traditional Chinese medicine knowledge designed in this paper can be divided into two key parts: the antecedent network and the consequent network. The two each undertake different tasks and work together. The main functions of the antecedent network include fuzzification of input variables, matching of fuzzy rules, and weight combination. It will output the applicability of the activated fuzzy rules, and this applicability will be used to modulate the output of the consequent part of the corresponding fuzzy rules. The main functions of the consequent network include processing of input variables, application and synthesis of fuzzy rules, and defuzzification. It will eventually output specific values that can be actually used, such as the probability value of a certain syndrome. The overall model structure is as follows Figure 4 shown.
[0088] (1) Precondition Network
[0089] First, the fuzzification layer of the antecedent network will transform the input variables Each node in this layer is a square node equipped with a fuzzy membership function, which is specifically used to calculate the membership of the input variable in a specific tongue or pulse fuzzy set. In order to reflect different input features more carefully, the language label of each tongue pulse has been converted into three fuzzy sets representing different degrees. Therefore, each input variable will generate three corresponding membership values, which reflect the corresponding membership degree of the variable in each degree fuzzy set.
[0090] Next is the fuzzy rule layer, which has a total of K square nodes. Figure 4 In the K It corresponds to the K pieces of TCM knowledge summarized in Table 1. Each π k According to the premise of TCM knowledge k Select the membership value of the corresponding fuzzy set as input, and perform a permutation combination on the input value. Then use the multiplication operator on the membership of each fuzzy set in the fuzzy rule to obtain the applicability of the fuzzy rule. The generalized calculation expression is as follows:
[0091]
[0092] in represents the rth item under the kth piece of TCM knowledge k The applicability of fuzzy rules is fuzzy rules, Ψ represents the set of corresponding memberships arranged and combined according to TCM knowledge. Therefore, the total number of fuzzy rules finally obtained is
[0093]
[0094] It can be seen that the number of fuzzy rules for each TCM knowledge transformation is the same as L kTherefore, in order to more uniformly express the applicability of fuzzy rules, the following conversion is used The subscript is ρ r , let R0=0,
[0095]
[0096] where ρ r is the applicability of the rth fuzzy rule. The subscript transformation process is Figure 4 It is represented by a circular equal sign node.
[0097] Afterwards, the normalization layer of the antecedent network normalizes the applicability of the R fuzzy rules.
[0098]
[0099] in It is called the normalized applicability of the rth fuzzy rule. It balances the influence of each rule, ensures that a single rule does not have an excessive impact on the network output, and enhances the synergy between different rules, which helps to maintain the balance and rationality of network decision-making. At this point, K pieces of traditional Chinese medicine knowledge have been transformed into R fuzzy rules, and the output of the antecedent network is obtained. It measures the matching degree of the antecedent of each fuzzy rule and also provides the necessary information for the consequent network to correctly process the consequent part of the rule.
[0100] (2) Post-processing network
[0101] The consequent part of the fuzzy rule constructed in this paper is not a fuzzy set in the traditional sense, but a mathematical expression. Here, the first-order rule of Takagi-Sugeno type fuzzy reasoning will be used, that is, the output of each fuzzy rule is the linear weighted sum of all input variables, and the expression is as follows:
[0102]
[0103] where d nr represents the output value of predicting the nth syndrome under the rth fuzzy rule, b nr is the bias term, is the weighting coefficient.
[0104] After obtaining the fuzzy rule output value, it will be defuzzified. Figure 4 It shows that the post-processing network is composed of N parallel sub-networks with the same structure, and each sub-network will generate a syndrome prediction indicator.
[0105]
[0106] It can be seen that h nis the weighted sum of the consequents of each fuzzy rule, and these weighted coefficients are the normalized applicability of the antecedents of each fuzzy rule. This processing method makes the contribution of each rule in the final output proportional to the applicability of its antecedent, thereby ensuring that those rules that better match the current input situation have a greater influence on the final decision result. At the same time, the system can ensure that the weighted sum of the applicability of all rules is 1, which not only makes the output result reasonably scaled, but also maintains the relative importance of different rules.
[0107] Finally, the output layer uses the sigmoid function to obtain the predicted probability values of various TCM syndromes.
[0108]
[0109] Step 4:
[0110] This section will verify the specific performance of the proposed TCM intelligent syndrome differentiation network model in actual diagnosis and compare it with a variety of commonly used machine learning methods. In the experiment, the pulse feature extraction module and the tongue feature extraction module were not changed, and only the multimodal fusion module was adjusted. In order to comprehensively evaluate the performance of the model, this study used multiple evaluation indicators in the multi-label classification scenario, including F1 score (F1_m), accuracy (Accuracy_m), precision (Precision_m), recall (Recall_m) and Hamming loss (Hamming_m). The calculation method of these five indicators is as follows:
[0111]
[0112] where |·| represents the number of positive labels in the set, z i represents the true label, Represents the prediction result of the model. The multi-label accuracy reflects the ratio of the number of correctly predicted positive labels to the sum of all predicted positive labels and actual positive labels. It needs to be calculated separately for each sample and then averaged.
[0113]
[0114] The multi-label recall rate represents the ratio of the number of successfully predicted positive labels to the number of true positive labels, and is averaged over all samples. A high recall rate indicates a lower false negative rate, which measures the "completeness" of the result.
[0115]
[0116] Multi-label precision reflects the ratio of the number of successfully predicted positive labels to the number of all positive labels predicted by the model, and is also the average of all samples. A high precision means fewer false positives and measures the "quality" of the results.
[0117]
[0118] The multi-label F1 value is a performance indicator that comprehensively considers precision and recall, and can provide a single metric that balances the two. Among these five indicators, the larger the values of Accuracy_m, Precision_m, Recall_m and F1_m, the better the multi-label classification performance of the model, and the smaller the Hamming_m, the better. The experimental results are shown in Table 2. As can be seen from Table 2, the TSDN model proposed in the present invention is superior to other machine learning algorithms in many indicators.
[0119] Table 1 Theoretical knowledge on tongue and pulse syndrome differentiation in traditional Chinese medicine
[0120]
[0121] Table 2 Performance comparison of different models in TCM syndrome differentiation experiments.
[0122]
Claims
1. A multi-label TCM syndrome differentiation method based on tongue and pulse data and fuzzy neural network, the method comprising: Step 1: Select the subjects and use the corresponding data acquisition equipment to collect the clinical tongue and pulse data and tongue pictures of the subjects; Step 2: TCM experts label the tongue and pulse data, tongue pictures, and syndromes of the subjects; Step 3: Use wavelet transform method to remove high-frequency noise of tongue and pulse signals and remove the baseline of tongue and pulse signals based on cubic spline interpolation method; Step 4: Remove the background information in the original tongue image and keep only the tongue body; Step 5: Construct a TCM intelligent syndrome differentiation network model, which consists of three parts: residual network, one-dimensional convolutional neural network and multimodal fuzzy neural network; Step 6: Input the preprocessed tongue image to the residual network for multi-label prediction of tongue image, and input the preprocessed tongue pulse data to the one-dimensional convolutional neural network for multi-label prediction of tongue pulse signals; Step 7: Use multimodal fuzzy neural network to fuse tongue pulse and tongue image features to perform multi-label prediction of TCM syndromes; Step 8: Use the data obtained in steps 3 and 4 to train the TCM intelligent syndrome differentiation network model according to the methods in steps 5 to 7; use the trained model to perform multi-label prediction on the newly obtained tongue and pulse images.
2. A multi-label TCM syndrome differentiation method based on tongue and pulse data and fuzzy neural network as claimed in claim 1, characterized in that: The one-dimensional convolutional neural network input signal in step 5 is the pulse data X′ RP , denote X′ RP The three tongue pulse signals in are [α 1,1 ,α 1,2 ,α 1,3 ], α 1,1 ,α 1,2 ,α 1,3 The three pulse signals are obtained after cubic spline interpolation, and then through N P A one-dimensional convolutional layer extracts high-level features of the tongue and pulse X P ; The operation method of the one-dimensional convolution layer is: in, For nth P The output feature vector obtained after the jth convolution calculation of the convolutional layer, n P =1,…,N P , j = 1, ..., J, J represents the total number of convolution kernels, For nth P The mth input feature vector of a one-dimensional convolutional layer, m = 1, ..., M, M is the number of input feature vectors, * represents the convolution operation, For nth P The jth convolution kernel in the one-dimensional convolution layer is convolved with the mth input feature vector. For nth P The bias term of the jth convolution kernel in the one-dimensional convolution layer, σ is the Relu activation function; the feature vector of the last layer of the one-dimensional convolution Combining features into a feature map Get high-level features of tongue and pulse X P After that, the first stage of pulse multi-label classification will be carried out; first, the feature map X P Flatten into a one-dimensional vector x P ; in represents vector transposition, flatten(.) represents the flattening operation, and then passes through the fully connected layer; h P =σ(W P1 x P +b P1 ) z P =W P2 h P +b P2 Among them, W P1 , W P2 is the weight parameter of the fully connected layer, b P1 , b P2 is the bias term; h P is the intermediate parameter of the calculation process; After full connection, we get vector z P , z P The dimension of corresponds to the number of pulse label classes; finally, in the output layer, the vector z P Each value in uses the Sigmoid function to obtain the final predicted probability value of each type of pulse label; in It is the mth P The probability value of pulse prediction, M P is the total number of pulse categories, Indicates the mth P Pulse input value; make It is the pulse multi-label prediction vector, so the overall forward propagation process of the one-dimensional convolutional neural network is expressed as follows; where θ P is the parameter to be optimized for the entire one-dimensional convolutional neural network, X′ RP represents the preprocessed pulse data, and 1DCNN() represents a one-dimensional convolutional neural network.
3. A multi-label TCM syndrome differentiation method based on tongue and pulse data and fuzzy neural network as claimed in claim 1, characterized in that: The residual network loss function in step 5 is: represents the predicted probability value set of tongue image data, Indicates the predicted probability value of tongue image data.
4. A multi-label TCM syndrome differentiation method based on tongue and pulse data and fuzzy neural network as claimed in claim 1, characterized in that: The multimodal fuzzy neural network in step 7 includes: a pre-network and a post-network; the number of input nodes of the pre-network and the post-network is M TP ; and the input data range is from 0 to 1, recorded as i=1,2,…,M TP , Indicates the probability value of the corresponding tongue and pulse prediction; The antecedent network includes: a fuzzy set partitioning layer, a fuzzification layer, a fuzzy rule layer, and a normalization layer. The number of nodes in the fuzzy set partitioning layer is M. TP , the label corresponding to each node is divided into three fuzzy sets with different credibility; the number of nodes in the fuzzification layer is 3M TP , each node is equipped with a fuzzy membership function, and the fuzzy membership function is used to generate a membership value for the three fuzzy sets corresponding to each label; The fuzzy rule layer includes K nodes π1,…,π K , corresponding to K pieces of TCM knowledge, each π k According to the premise of TCM knowledge k Select the membership value of the corresponding fuzzy set as input, and perform a permutation combination on the input value; then use the multiplication operator on the membership of each fuzzy set in the fuzzy rule to obtain the applicability of the fuzzy rule. The calculation expression is: in, represents the rth item under the kth item of TCM knowledge k The applicability of fuzzy rules is fuzzy rules, L k represents the number of tongue and pulse labels in the kth piece of TCM knowledge, Ψ represents the set formed by permuting and combining the corresponding membership degrees according to TCM knowledge, represents the membership values in Ψ; the total number of fuzzy rules finally obtained is: It can be seen that the number of fuzzy rules for each TCM knowledge transformation is the same as L k Therefore, in order to more uniformly express the applicability of fuzzy rules, the following conversion is used The subscript is ρ r , let R0=0, Among them, ρ r is the applicability of the rth fuzzy rule, R n-1 represents the total number of fuzzy rules converted from the n-1th piece of TCM knowledge, and R represents the total number of fuzzy rules obtained after all TCM knowledge is converted; The normalization layer is used to perform normalization calculation on the applicability of the R fuzzy rules; in It is called the normalized applicability of the rth fuzzy rule.
5. The multi-label TCM syndrome differentiation method based on tongue and pulse data and fuzzy neural network as claimed in claim 1, characterized in that: The specific method of the post-processing network is: Step s1: where d nr represents the output value of the nth label predicted under the rth fuzzy rule, f nr (·) represents a linear function, b nr is the bias term, is the weighting coefficient; Step s2: Get the fuzzy rule output value d nr After that, it will undergo a defuzzification process, which consists of N parallel sub-networks with the same structure, each of which will produce a label prediction quantity; Among them, h n is the weighted sum of the consequents of each fuzzy rule, and these weighted coefficients are the normalized applicability of the antecedents of each fuzzy rule. represents the normalized fitness, d nr represents the output value of the consequent network, and N represents the number of syndromes to be predicted; Step s3: The output layer uses the sigmoid function to obtain the predicted probability values of various TCM labels
