Traditional Chinese medicine intelligent classification model construction method and system based on Transform neural network

Abstract

The invention discloses a traditional Chinese medicine intelligent classification model construction method and system based on a Transform neural network, and the method comprises the following steps: S1, collecting the multi-source data of a patient and a control group, and screening out the index data incorporated into a model; s2, performing dynamic feature extraction and weight distribution on the index data, and constructing a traditional Chinese medicine intelligent classification model by adopting a Transform network; and S3, training and verifying the traditional Chinese medicine intelligent classification model. A multi-scale automatic segmentation network is developed by combining the convolutional neural network and a Transform architecture, a traditional Chinese medicine intelligent dialectical diagnosis and treatment platform is constructed, objectization and integration of disease diagnosis are realized, and the clinical curative effect is improved. The method is beneficial to intelligent auxiliary diagnosis and treatment decision of skin diseases.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular, to a method and system for constructing a traditional Chinese medicine intelligent classification model based on a Transformer neural network. Background Art

[0002] Existing traditional Chinese medicine diagnosis and treatment systems mainly rely on traditional empirical medicine models, and have the following problems:

[0003] 1. Strong subjectivity and lack of standardization: Traditional Chinese medicine syndrome differentiation relies on the experience of physicians and is easily affected by subjective factors. The syndrome differentiation results for the same patient may vary among different physicians.

[0004] 2. Low efficiency: Traditional Chinese medicine diagnosis and treatment requires combining the four diagnostic methods to identify diseases and syndromes, and it is difficult to quickly process a large amount of patient information simultaneously.

[0005] Lack of intelligent support.

[0006] Existing traditional Chinese medicine diagnosis and treatment systems are mostly based on simple rule bases or shallow machine learning models, and cannot make full use of massive medical data and complex clinical experience. When there are contradictions in system signs such as local skin lesions and tongue image scores of patients, it is difficult to directly judge.

[0007] In addition, the application of existing artificial intelligence technologies in the field of traditional Chinese medicine still has the following deficiencies:

[0008] 1. Limitations of single indicators: Existing traditional Chinese medicine intelligent devices (such as tongue diagnosis instruments and pulse diagnosis instruments) can only collect physiological indicators in a single dimension (such as tongue images and pulse conditions), and cannot comprehensively analyze multi-source data (symptoms, laboratory indicators).

[0009] 2. Insufficient algorithm adaptability: Traditional machine learning methods (such as support vector machine SVM, etc.) have limited effects in processing multi-dimensional and non-linear data of traditional Chinese medicine, and it is difficult to capture the deep-level associations and dynamic weight distributions among symptoms.

[0010] 3. Insufficient clinical verification: Existing models are mostly constructed based on theories, and lack large-scale clinical randomized controlled trials to verify their actual efficacy and generalization ability.

[0011] Therefore, there is an urgent need for a traditional Chinese medicine intelligent diagnosis and treatment system based on advanced artificial intelligence technology to solve the above problems and improve the scientificity and practicality of traditional Chinese medicine diagnosis and treatment. Summary of the Invention

[0012] The purpose of the present invention is to provide a method and system for constructing a traditional Chinese medicine intelligent classification model based on a Transformer neural network to solve the problems raised in the above background art.

[0013] To achieve the above-mentioned invention object, an aspect of the present invention provides a method for constructing a traditional Chinese medicine intelligent classification model based on a Transformer neural network, including the following steps:

[0014] Step S1, collect multi-source data of patients and the control group, and screen out the index data included in the model;

[0015] Step S2, perform dynamic feature extraction and weight assignment on the index data, and construct a traditional Chinese medicine intelligent classification model using a Transformer network;

[0016] Step S3, train and validate the traditional Chinese medicine intelligent classification model.

[0017] Further, it is characterized in that step S1 is divided into the following steps:

[0018] Step S101, classify the text data of patients and the control group based on a Transformer network;

[0019] Step S102, obtain the importance ranking of the index data through correlation research;

[0020] Step S103, determine the included indexes, including clinical indexes, laboratory indexes, and subjective indexes.

[0021] Further, the encoder of the traditional Chinese medicine intelligent classification model adopts a Transformer network, which consists of 8 stacks to form a feature extractor, uses 6 attention layers to enhance the representation of the input layer, and calculates the final classification result of the network through a linear layer activated by Max.

[0022] Further, in step S3, the Loss curve of the model and the confusion matrix of the classification model are used to verify the stability and authenticity of the model.

[0023] Further, in step S3, the ten-fold cross-validation method is used to evaluate the performance of the model. The data set is divided into 10 groups, and cross-validated 10 times. Each time, 9 groups are taken as training data, and the remaining 1 group is taken as test data. The evaluation scores of different indexes are calculated, and then the average value of the evaluation scores of the 10 cross-validations is taken to calculate the evaluation effect of the model.

[0024] Further, in step S3, a forward feature selection process is used for correlation research, and the index with the best performance is added in each round of the process.

[0025] Further, the method is applicable to the adjuvant treatment of skin diseases.

[0026] Another aspect of the present invention provides a system for constructing a traditional Chinese medicine intelligent classification model based on a Transformer neural network, including a collection module, a modeling module, and a verification module, wherein:

[0027] The acquisition module is used to acquire multi-source data of patients and the control group, and screen out the biomarkers included in the model;

[0028] The modeling module is used to extract dynamic features and assign weights to the data, and construct a traditional Chinese medicine intelligent classification model using the Transformer network;

[0029] The verification module is used to train and verify the traditional Chinese medicine intelligent classification model.

[0030] Since the present system and method are adopted, compared with the prior art, the following advantages are achieved:

[0031] 1. Aiming at the difficulties in the clinical diagnosis of psoriasis with blood stasis syndrome, the present invention proposes a binary classification method based on artificial intelligence, and uses deep learning technology to assist doctors in accurate syndrome differentiation, so as to improve the consistency and reliability of diagnosis. This method can effectively make up for the subjectivity problem of traditional syndrome differentiation methods, and provide a more objective and accurate judgment basis for clinical diagnosis and treatment.

[0032] 2. The present invention introduces the Transformer architecture to improve the accuracy of diagnosis and decision-making optimization ability. Compared with traditional syndrome differentiation methods, the Transformer network model can more accurately identify patients with blood stasis syndrome, showing excellent syndrome differentiation ability, thereby improving the objectivity and accuracy of traditional Chinese medicine diagnosis. In addition, by combining the convolutional neural network (CNN) and the Transformer structure, the present invention develops a multi-scale automatic segmentation network, and constructs an intelligent traditional Chinese medicine syndrome differentiation diagnosis and treatment platform based on this, realizing the objectification, standardization and generalization of disease diagnosis. The application of this technology not only improves the intelligent auxiliary diagnosis ability of skin diseases such as psoriasis, but also can provide more accurate decision-making support for personalized treatment, thereby further optimizing the clinical efficacy. Description of the Drawings

[0033] Figure 1 It is a flowchart of a method for constructing a traditional Chinese medicine intelligent classification model based on the Transformer neural network.

[0034] Figure 2 It is a schematic structural diagram of the traditional Chinese medicine intelligent classification model.

[0035] Figure 3 It is a graph of the model Loss curve and the confusion matrix.

[0036] Figure 4 It is a t-SNE visualization result graph.

[0037] Figure 5 It is a comparison result graph of different stack numbers and attention layers.

[0038] Figure 6 To evaluate the discrimination ability of the Transformer syndrome differentiation model and the model diagram using the ROC curve. Specific implementation manners

[0039] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0040] As Figure 1 shown in the method flow chart of the present invention, the core of the present invention lies in constructing a traditional Chinese medicine intelligent syndrome differentiation diagnosis and treatment system based on the Transformer neural network. The specific steps are as follows:

[0041] Step S1, multi-source data screening and collection.

[0042] The present invention screens the syndrome differentiation factors of psoriasis blood stasis syndrome by retrospectively observing the clinical indicators of 300 patients with psoriasis blood stasis syndrome and 300 healthy volunteer controls in the control group. The steps include:

[0043] Step S101, classifying the two groups of text data based on the Transformer network;

[0044] Step S102, obtaining the importance ranking of these clinical indicators through correlation research. The results of the systematic evaluation show that the levels of IL-6, TNF-α, IFN-γ, and IL-17 in the serum of patients with psoriasis blood stasis syndrome are significantly higher than those in the healthy control group, while the level of IL-10 is significantly lower.

[0045] Step S103, determine the included data as follows: Clinical indicators include Psoriasis Area and Severity Index (PASI) score and Body Surface Area (BSA) of skin lesions. Laboratory indicators are Squamous Cell Carcinoma Antigen (SCCA), Tumor Necrosis Factor - Alpha (TNF - α), Interleukin - 23 (IL - 23), and Interleukin - 17 (IL - 17). Subjective indicators include Visual Analog Scale (VAS) score reflecting the severity of itching, Dermatology Life Quality Index (DLQI), Quality Of Life (QOL), Self - rating Anxiety Scale (SAS), Self - rating Depression Scale (SDS), Xerostomia Questionnaire (XQ), Constipation Clinical Score (CCS), and Chinese Medicine Syndrome Scoring Scale (CMSSS).

[0046] Step S2, perform dynamic feature extraction and weight assignment on the indicator data, and construct a traditional Chinese medicine intelligent classification model using the Transformer network.

[0047] Utilize the attention mechanism of Transformer to automatically learn the importance of different features and solve the limitations of fixed weight assignment in traditional methods. As Figure 2 shown is the structure and role of Transformer in medical feature classification tasks, demonstrating how to automatically learn the importance of different features through the attention mechanism and optimize the classification performance. The main components of Transformer are the multi - head attention layer and the feed - forward layer. Since the input of Transformer is the baseline features of patients, we only use Transformer in the encoder stage, as Figure 2 shown in A, where represents the feature. As Figure 2 As shown in B is the structure diagram of multi-head attention. Specifically, the encoder stage is created by 8 stacks, which are composed of multi-head attention layers, and the feed-forward layer extracts features from the input layer. To balance the classification performance and model complexity, we use 8 stacks to form a feature extractor for training. Among them , , , where , , and , , is , and 's learning weight parameters and biases. and 's attention weight matrix can be expressed as: , where is the dimension of K and Q, is the scaling coefficient, and the Softmax activation normalizes the values to [0,1]. The attentive vector is calculated as: ; The multi-head attention output calculation is expressed as: , where H1, H2,..., H t represents multiple attention heads. In the multi-head attention layer, different numbers of scaled dot-product attention layers can learn various context representations. As Figure 2 shown in C is the structure diagram of scaled dot-product attention. , is the predicted probability, represents the parameters of the linear layer, is the feature output of the previous linear layer. The cross-entropy loss calculation is expressed as: , where is the true label of each category. Therefore, we empirically use 6 attention layers to enhance the representation of the input layer and calculate the final classification result of the network through the linear layer with Max activation.

[0048] Step S3, intelligent dialectical model training and verification.

[0049] To verify the stability and authenticity of the model, first, the Loss curve and the classification model confusion matrix of the model are given, asFigure 3 As shown. Here, we separately compared the losses of the Transformer and the Convolutional Neural Network (CNN). As Figure 3 Figure A shows the Loss curve of the model. The x-axis represents the number of model iterations, and the y-axis represents the model loss. It can be clearly seen from the Loss curve that after approximately 1000 epochs, the loss of the Transformer is smaller than that of the CNN, proving that the classification model proposed in this paper can achieve better classification performance. In addition, as Figure 3 Figure B shows that the x-axis and y-axis of the confusion matrix represent the predicted class labels and the true class labels respectively. In the first row of the confusion matrix, from left to right, they represent True Positive (TP) and False Negative (FN); in the second row of the confusion matrix, from left to right, they represent FP and TN. Judging from the results, TP and TN have higher values than FP and FN, which further indicates that the proposed network can obtain better classification results.

[0050] Subsequently, to evaluate the performance of the model, we divided the dataset into 10 groups and conducted 10 validations. Each time, we took 9 groups as the training data and 1 group as the test data, and calculated the evaluation scores of different metrics. Finally, the evaluation scores of the ten-fold cross-validation were averaged to calculate its effectiveness. The validation results are shown in Table 1. The final average precision, recall, accuracy, F1-score, and AUC are 0.91, 0.88, 0.90, 0.89, and 0.89 respectively, further proving the effectiveness of the classification model.

[0051] Table 1: Ten-fold cross-validation

[0052] Items Precision Recall Accuracy F1-score AUC Fold-1 0.93 0.90 0.92 0.92 0.92 Fold-2 0.89 0.86 0.88 0.88 0.88 Fold-3 0.88 0.94 0.90 0.91 0.90 Fold-4 0.87 0.82 0.83 0.84 0.84 Fold-5 0.93 0.86 0.90 0.89 0.90 Fold-6 0.99 0.98 0.99 0.99 0.99 Fold-7 0.89 0.89 0.9 0.89 0.90 Fold-8 0.90 0.84 0.87 0.87 0.87 Fold-9 0.85 0.82 0.85 0.83 0.85 Fold-10 0.93 0.86 0.90 0.89 0.90 Average 0.91 0.88 0.90 0.89 0.89

[0053] To obtain a more comprehensive evaluation, we compared the proposed model with other traditional classification methods, including Support Vector Machine (SVM), Decision Tree (DT), Linear, Long Short-term Memory (LSTM), and CNN. The comparison results are shown in Table 2. The model has the best classification performance on different evaluation metrics, indicating that this model is expected to become an effective method for automatic psoriasis diagnosis.

[0054] Table 2: Comparison with other methods

[0055] Methods Precision Recall Accuracy F1-score AUC SVM 0.81 0.76 0.86 0.81 0.85 DT 0.83 0.71 0.83 0.80 0.82 Linear 0.80 0.70 0.85 0.76 0.83 LSTM 0.83 0.77 0.83 0.83 0.87 CNN 0.82 0.76 0.85 0.83 0.83 Transformer 0.91 0.88 0.90 0.89 0.89

[0056] To further verify the effectiveness of the model, we used t-SNE to visualize the results in the feature space, as Figure 4 shown. Among them, Figure 4 the left figure is the original data, which shows overlapping clusters between healthy people and psoriasis patients; the right figure is the t-SNE visualization result of the classification model, showing a good separation effect for clusters of different classes. This visualization result also proves the effectiveness of the classification model.

[0057] To test the depth of the model, networks were constructed using different numbers of stacks. In this study, the stack numbers {4, 6, 8, 10, 12} were used to evaluate its performance. When the number of stacked layers increased to 8, the experimental results improved, but the increase in the number of stacked layers had little effect on the final classification performance. Therefore, 8 stacks were finally used to construct the classification network. In addition, in the multi-head attention layer, which consists of multiple scaled dot-product attention layers and is capable of learning attention features and context features from the input, we conducted a large number of experiments to explore the impact of different numbers of attention layers on the final classification performance. The results showed that the best performance could be obtained with the number 6, as Figure 5 shown. This indicates that reducing the number of attention layers may not effectively learn the representation of the input data, and increasing the number of attention layers may also lead to overfitting problems.

[0058] To determine the importance ranking of each indicator, we conducted a correlation study. As shown in Table 3, we used a forward feature selection process, adding the indicator with the best performance in each round. A total of 14 rounds of tests were conducted. The experimental results showed that CMSSS achieved the best performance on a single indicator. The importance ranking results were CMSSS, DLQI, BSA, PASI, SCC, VAS, QOL, SAS, SDS, XQ, TNF-α, IL-23, IL-17, CCS in sequence. In addition, adding all the indicators improved the performance of the classification model, with an accuracy of 0.90.

[0059] Table 3: Importance of each indicator through correlation study

[0060] Round Indicators Accuracy 1 CMSSS 0.53 2 CMSSS + DLQI 0.58 3 CMSSS + DLQI + BSA 0.62 4 CMSSS + DLQI + BSA + PASI 0.66 5 CMSSS + DLQI + BSA + PASI + SCC 0.70 6 CMSSS + DLQI + BSA + PASI + SCC + VAS 0.73 7 CMSSS + DLQI + BSA + PASI + SCC + VAS + QOL 0.76 8 CMSSS + DLQI + BSA + PASI + SCC + VAS + QOL + SAS 0.79 9 CMSSS + DLQI + BSA + PASI + SCC + VAS + QOL + SAS + SDS 0.82 10 CMSSS + DLQI + BSA + PASI + SCC + VAS + QOL + SAS + SDS + XQ 0.84 11 CMSSS + DLQI + BSA + PASI + SCC + VAS + QOL + SAS + SDS + XQ + TNF-α 0.86 12 CMSSS + DLQI + BSA + PASI + SCC + VAS + QOL + SAS + SDS + XQ + TNF-α + IL-23 0.88 13 CMSSS + DLQI + BSA + PASI + SCC + VAS + QOL + SAS + SDS + XQ + TNF-α + IL-23 + IL-17 0.89 14 CMSSS + DLQI + BSA + PASI + SCC + VAS + QOL + SAS + SDS + XQ + TNF-α + IL-23 + IL-17 + CCS 0.90

[0061] Compared with traditional dialectical methods, the verification model has advantages in improving curative effects and reducing recurrence rates. In the clinical verification part, there were 64 patients with blood stasis syndrome of psoriasis vulgaris. The treatment period for both groups of patients was 8 weeks. A clinical randomized concurrent control trial was adopted to analyze the differences in the clinical curative effects of the Transformer artificial intelligence decision-making system in syndrome differentiation and treatment and the traditional syndrome differentiation and treatment by physicians. The total effective rate of the Transformer artificial intelligence syndrome differentiation treatment was 67.74% after 4 weeks and 93.54% after 8 weeks, both of which were better than those of the traditional syndrome differentiation group. It was proved that the curative effect of the Transformer artificial intelligence syndrome differentiation treatment for psoriasis vulgaris with blood stasis syndrome was better than that of the traditional syndrome differentiation by physicians. The advantages were to improve the clinical curative effect and reduce the recurrence rate. Especially in the improvement of clinical curative effect indexes such as PASI, BSA, VAS, PGA, DLQI, SAS, SDS, and PSQI, it showed advantages over traditional syndrome differentiation, and the syndrome differentiation efficiency was more accurate.

[0062] To evaluate the clinical practicability of the model, the ROC curve was further used to evaluate the discrimination ability and model efficiency of the Transformer syndrome differentiation model. As Figure 6 shown, the results showed that the area under the curve AUC value was: 0.734 (0.608, 0.86), and the critical value (threshold) was: 4.55 (0.71, 0.677). AUC = 0.734 was greater than 0.7, indicating that the two groups of patients could be significantly distinguished by the PASI score. That is, the PASI score was significantly reduced in the Transformer model group after treatment. By inferring the syndrome differentiation efficiency from the treatment results, that is, there were more actual patients with blood stasis syndrome in the Transformer model group (patients with effective treatment were regarded as actual patients with blood stasis syndrome), indicating that the Transformer model could well identify patients with blood stasis syndrome and had good accurate syndrome differentiation ability.

[0063] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a traditional Chinese medicine intelligent classification model based on the Transformer neural network, characterized in that It includes the following steps: Step S1: Collect multi-source data of patients and the control group, and screen out the index data included in the model; Step S2: Extract dynamic features and assign weights to the index data, and use the Transformer network to construct a traditional Chinese medicine intelligent classification model; Step S3: Train and validate the traditional Chinese medicine intelligent classification model.

2. The method for constructing a traditional Chinese medicine intelligent classification model based on the Transformer neural network according to claim 1, wherein, Step S1 is divided into the following steps: Step S101: Classify the text data of patients and the control group based on the Transformer network; Step S102: Obtain the importance ranking of the index data through correlation research; Step S103: Determine the included indexes, including clinical indexes, laboratory indexes and subjective indexes.

3. A method for constructing a traditional Chinese medicine intelligent classification model based on the Transformer neural network according to claim 1, characterized in that, The encoder of the traditional Chinese medicine intelligent classification model adopts the Transformer network, which consists of 8 stacks to form a feature extractor, uses 6 attention layers to enhance the representation of the input layer, and calculates the final classification result of the network through the linear layer activated by Max.

4. A method for constructing a traditional Chinese medicine intelligent classification model based on a Transformer neural network according to claim 1, characterized in that, In Step S3, the Loss curve of the model and the confusion matrix of the classification model are used to verify the stability and authenticity of the model.

5. A method for constructing a traditional Chinese medicine intelligent classification model based on the Transformer neural network according to claim 1, characterized in that, In Step S3, the ten-fold cross-validation method is used to evaluate the performance of the model. The data set is divided into 10 groups and cross-validated 10 times. Each time, 9 groups are taken as training data, and the remaining 1 group is taken as test data. The evaluation scores of different indexes are calculated, and then the average value of the evaluation scores of the 10 cross-validations is taken to calculate the model evaluation effect.

6. The method for constructing a traditional Chinese medicine intelligent classification model based on the Transformer neural network according to claim 1, wherein In Step S3, a forward feature selection process is used for correlation research, and the index with the best performance is added in each round of the process.

7. A method for constructing a traditional Chinese medicine intelligent classification model based on a Transformer neural network according to claim 1, characterized in that The method is applicable to the adjuvant treatment of skin diseases.

8. A system for constructing a traditional Chinese medicine intelligent classification model based on a Transformer neural network, characterized in that, It includes a collection module, a modeling module, and a verification module, where: The collection module is used to collect multi-source data of patients and the control group, and screen out the biomarkers included in the model; The modeling module is used to extract dynamic features and assign weights to the data, and use the Transformer network to construct a traditional Chinese medicine intelligent classification model; The verification module is used to train and validate the traditional Chinese medicine intelligent classification model.

Images