A question sentence classification method and application thereof

Abstract

The application discloses a kind of interrogative sentence classification methods, the present application is trained by pre-training BERT model, utilize the word vector representation of interrogative sentence under large-scale sentence, each word will have different vector representation under different interrogative sentence;Interrogative sentence is classified by using two-layer bidirectional GRU network structure to process text sequence, and the idea of residual network is used to combine the original word vector and the output of the first layer of GRU network as the input of the second layer of GRU, which can speed up the convergence speed of the model, and the attention mechanism is used to pay attention to the important features of interrogative sentence, to improve the classification accuracy of interrogative sentence.The present application can further capture the dependency problem of interrogative sentence text by using GRU model, and the features that have greater influence on interrogative sentence classification are given higher weight by using attention mechanism, so that the classification accuracy of interrogative sentence in medical question and answer system is further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a question sentence classification method and application thereof, and especially to a Chinese question sentence classification method, and particularly to a Chinese medical question and answer system question sentence classification method. BACKGROUND

[0002] With the development and application popularization of network technology, the application prospect of medical question and answer system is very extensive. Users can understand relevant medical knowledge or consult the treatment method of some diseases through the medical question and answer system. At present, the mainstream construction method of the medical question and answer system based on knowledge base is the method based on text matching, that is, the related entities in the question sentence and the category corresponding to the question sentence are first recognized, and then the answer is found from the knowledge base according to the entities and the question category, so that accurate classification of the question sentence is an important basis for the medical question and answer system.

[0003] The question sentence classification method of the existing medical question and answer system can be divided into two categories in general, one is the method based on feature engineering, that is, the features such as the part of speech and syntax of the question sentence are obtained by using tools for classification, this kind of method first uses the word segmentation tool to perform word segmentation and part of speech tagging, finds out the key words and dependency syntax relationship in the question sentence and other features, and then uses the extracted features for classification to obtain the classification result; the other is the method based on deep learning, that is, the features are automatically extracted by using neural network for classification, such as the medical question classification method proposed in Chinese patent "CN110176315A medical question and answer method and system, electronic equipment and computer readable medium", which splices the vector matrix containing word information and topic information according to the document topic information and vector data corresponding to the medical question, and inputs the vector into the bidirectional gate recurrent unit to obtain the score of each category corresponding to the medical consultation question sentence, and the category with the highest score is the category corresponding to the question sentence.

[0004] The length of the question sentence of the Chinese medical question and answer system is not uniform, and many field vocabularies are contained in the sentence. The existing method based on feature engineering needs to extract features manually, which may lead to insufficient feature extraction and affect the accuracy of question sentence classification; and the existing deep learning method based on bidirectional gate recurrent unit cannot solve the ambiguity problem of the word when constructing the word vector of the question sentence, because the vector representation of each word is fixed, for example, the sentences: "I recently have too much psychological burden" and "Yesterday, the burden I carried was too heavy, and I have a sore back today", the meaning of "burden" in the two sentences is different in different contexts, and the final question sentence classification result should also be different; in addition, the gate recurrent unit is prone to gradient disappearance and gradient explosion problems, and the training is more difficult. SUMMARY

[0005] In view of the above defects, the application provides a question sentence classification method, which can be used in a Chinese medical question and answer system, the method uses a BERT (Bidirectional Encoder Representations from Transformers) model trained by a multi-layer attention mechanism on a large-scale corpus to obtain word vector representation of a question sentence, solves the ambiguity problem of an existing word vector, and proposes a residual network idea to solve the training difficulty of a recurrent neural network, accelerates the training process, makes the model converge faster, and improves the question sentence classification accuracy.

[0006] The application is realized by the following technical solutions:

[0007] A question sentence classification method comprises the following steps:

[0008] S1. Question sentence preprocessing: pre-process the collected question sentences in a related field, and each pre-processed question sentence is represented as q={q1,q2,...,q k ,...,q n}, wherein q k represents the kth word in the question sentence, and n is the total number of words in the question sentence;

[0009] S2. Vector representation of the question sentence: use a pre-trained BERT model to map and represent the question sentence q as a vector v q ={x1,x2,...,x k ,...,x n}, wherein x k represents a word vector corresponding to the word q k .

[0010] S3. Establishing a question sentence classification model: a two-layer bidirectional GRU model with an attention mechanism is used to extract high-level features of the question sentence, a forward GRU processes the sequential input of the question sentence, and a backward GRU processes the reverse sequential input of the question sentence;

[0011] S31. High-level feature extraction of the question sentence: a two-layer bidirectional GRU network structure is used to process the text sequence, a residual network idea is used, the original word vector is combined with the output of the first layer of the GRU network, and then the combined output is used as the input of the second layer of the GRU; an attention mechanism is used to assign a weight to the output hidden state of each time of the second layer of the GRU, and finally the hidden states of each time are added up to obtain the final high-level feature representation of the sentence;

[0012] S32. Question category probability calculation: the high-level feature representation of the sentence obtained in step S31 is input into a linear layer for classification, and the probability corresponding to each category is output, and the category with the highest probability is taken as the question category predicted by the model;

[0013] S33. Cost function and iterative training: divide the question dataset into a training set and a test set, train the model using the training samples, and finally obtain a question classification model;

[0014] S4. New question classification: for a user input question, after preprocessing in step S1, then vectorizing the question in step S2, input into the trained question classification model, and get the classification result.

[0015] Further preferably, step S31 specifically comprises the following steps:

[0016] S311. Bidirectional GRU model:

[0017] The calculation process of the first layer of the GRU model at time t is as follows:

[0018]

[0019]

[0020]

[0021] wherein, x t represents the word vector input at time t of the first layer of the GRU model, i.e. the word vector corresponding to the tth word in the question, represents the output of the update gate at time t of the first layer of the GRU model, r t 1 represents the output of the reset gate at time t of the first layer of the GRU model, represents the hidden state at time t of the first layer of the GRU model, is the corresponding parameter matrix of x t in the update gate of the first layer of the GRU model, is the corresponding parameter matrix of the hidden state at the previous time of the first layer of the GRU model , W t 1 is the corresponding parameter matrix of x t in the reset gate of the first layer of the GRU, is the corresponding parameter matrix of the hidden state at the previous time of the first layer of the GRU model , W 1 is the corresponding parameter matrix of x t in the output gate of the first layer of the GRU model, 1 is the corresponding parameter matrix of U t 1 is the reset gate output r at the previous time of the hidden state of the first layer of the GRU model, represents a dot product operation of two vectors;

[0022] The question vector v q is input to the forward GRU network in order, and the hidden state of each word at time t is obtained k ... n The question vector v is input to the backward GRU, and the hidden state of each word at time t is obtained The forward and are concatenated as the output of the first layer of the GRU at time t

[0023] The word vector representation x t of the original question is connected to the output of the first layer of the GRU to form the input of the second layer of the GRU at time t, which is calculated as follows:

[0024]

[0025]

[0026]

[0027] wherein, represents the output of the update gate of the second layer of the GRU model at time t, r t 2 represents the output of the reset gate of the second layer of the GRU model at time t, represents the hidden state of the second layer of the GRU model at time t, W z 2 is the vector formed by concatenating x t and the hidden state of the first layer of the GRU model at time t corresponding to the parameter matrix of the update gate of the second layer of the GRU model, is the vector formed by concatenating the hidden state of the previous time of the update gate of the second layer of the GRU model corresponding to the parameter matrix of the update gate of the second layer of the GRU model, W t 2 is the vector formed by concatenating x t and the hidden state of the first layer of the GRU model at time t corresponding to the parameter matrix of the reset gate of the second layer of the GRU model, is the vector formed by concatenating the hidden state of the previous time of the reset gate of the second layer of the GRU model corresponding to the parameter matrix of the reset gate of the second layer of the GRU model, W 2 is the vector formed by concatenating x t and the hidden state of the first layer of the GRU model at time t​ The parameter matrix corresponding to the vector formed by linking them, U 2 is the reset gate output r t 2 and the hidden state of the previous time The parameter matrix corresponding to the dot product operation result, σ(.) represents the sigmoid activation function, tanh(.) represents the hyperbolic tangent function, represents the dot product operation of two vectors. Similarly, the outputs of the forward and backward GRU at time t are linked together as the output of the second layer of the GRU at time t

[0028] S312. Attention mechanism:

[0029] The output hidden state of each time of the second layer of the GRU model is assigned a weight a i , and finally the hidden states of each time are accumulated to obtain the final sentence high-level feature representation h q , which is calculated as follows:

[0030]

[0031] a t = softmax(e t ) (8);

[0032]

[0033] wherein, represents the hidden vector of the second layer of the GRU at time t, W Attention is the hidden state of the second layer of the GRU at time t , b is a bias vector.

[0034] Further preferably, in step S32, the training process selects cross-entropy as the loss function, which is calculated as follows:

[0035]

[0036] wherein, p i is the output of the linear layer, representing the probability that the question is the i-th category, N represents the number of categories of the question, y i represents whether the classification output of the question is equal to the true category label of the question, y i ∈{0,1}, if the category label of the question output by the model is equal to the true label, y i is 1, otherwise y i is 0.

[0037] Further preferably, the pre-trained BERT model adopts a 768-dimensional Chinese word vector open sourced by Google.

[0038] Further preferably, the pre-trained BERT model is set to 16 for a training batch size, 0.001 for a learning rate, 0.00005 for a BERT fine-tuning learning rate, and Adam as an optimizer.

[0039] Further preferably, the method further comprises collecting question data before step S1, collecting a certain number of questions in a related field, labeling and classifying, and each question corresponds to a category.

[0040] Further preferably, the medical field questions are collected and classified into 16 categories, which are: commonly used drugs for diseases, food suitable for diseases, required examination items for diseases, food to be avoided for diseases, recommended drugs for diseases, disease symptoms, disease complications, disease treatment departments, disease treatment methods, disease treatment time, disease cure probability, disease treatment cost, disease prevention measures, disease causes, disease description and disease infectivity.

[0041] The application of the above question classification method is used in a medical field question and answer system for Chinese question classification.

[0042] The application has the following advantages and beneficial effects:

[0043] The application uses the pre-trained BERT model to obtain the word vector representation of the question under large-scale corpus training, each word has different vector representation in different questions; the two-layer bidirectional GRU network structure is used to process the text sequence, the original word vector is combined with the first layer output of the GRU network to form the second layer input of the GRU, which can accelerate the convergence speed of the model, and the attention mechanism is used to pay attention to the important features of the question to improve the question classification accuracy.

[0044] Compared with the previous word2vec, Glove and other word vector generation methods, the BERT model is used to pre-train the word vector representation of the question, which can solve the ambiguity problem of the word vector in different contexts; compared with the original BERT model, the GRU model is used to further capture the dependency problem of the question text, and the attention mechanism is used to assign a higher weight to the features that have a greater impact on the question classification, so that the question classification accuracy of the medical question and answer system is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the embodiments of the application, constitute a part of the application, and do not constitute a limitation of the embodiments of the application. In the drawings:

[0046] Figure 1This is the overall flow of a question classification method for a Chinese medical question-and-answer system according to the present invention;

[0047] Figure 2 This is a model architecture diagram of the overall framework of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0049] Example 1

[0050] This embodiment provides a method for classifying questions in a Chinese medical question-and-answer system, specifically employing the following steps:

[0051] S1. Medical Question Data Collection:

[0052] We extracted 1250 medical-related questions from the publicly available doctor-patient dialogue data on the website and manually labeled them, assigning each question to a category. The questions were divided into 16 categories: commonly used medications for the disease, recommended foods for the disease, required examinations for the disease, foods to avoid for the disease, recommended medications for the disease, disease symptoms, complications of the disease, treatment departments for the disease, treatment methods for the disease, treatment time for the disease, probability of cure for the disease, treatment costs for the disease, prevention measures for the disease, causes of the disease, description of the disease, and contagiousness of the disease.

[0053] S2. Question Preprocessing: Traditional Chinese characters in the questions are converted to simplified Chinese characters using a word list mapping method. A common stop word list is used to match the questions and remove stop words. Each preprocessed question can be represented as q = {q1, q2, ..., q...} k ,...,q n}, where q k This represents the k-th word in the question, where n is the total number of words in the question.

[0054] S3. Vector Representation of Questions: Using a pre-trained BERT model, the question q is mapped and represented as a vector v. q ={x1,x2,...,x k ,...,x n}, where x k The word q k The corresponding word vectors. In this embodiment, the BERT model uses 768-dimensional Chinese word vectors from Google's open source, the training batch size is set to 16, the learning rate is set to 0.001, the BERT fine-tuning learning rate is 0.00005, and Adam is used as the optimizer.

[0055] S4. Establish a question classification model: a two-layer bidirectional GRU model with attention mechanism is used to extract high-level features of the question, the forward GRU processes the sequential input of the question, and the backward GRU processes the reverse sequential input of the question;

[0056] S41. High-level feature extraction of the question: a two-layer bidirectional GRU network structure is used to process the text sequence, and the idea of residual network is combined to combine the original word vector and the first layer output of the GRU network as the input of the second layer of the GRU; the attention mechanism is used to assign a weight to the output hidden state of each time of the second layer of the GRU, and finally the hidden states of each time are added up to obtain the final sentence high-level feature representation;

[0057] The scheme adopts a two-layer bidirectional GRU (Gate Recurrent Unit) model with attention mechanism to extract high-level features of the question, the forward GRU processes the sequential input of the question, and the backward GRU processes the reverse sequential input of the question, and the model structure is as follows Figure 2 .

[0058] S411. Bidirectional GRU model:

[0059] The calculation process of the first layer of the GRU model at time t is as follows:

[0060]

[0061]

[0062]

[0063] wherein, x t represents the word vector input at time t of the first layer of the GRU model, i.e. the word vector corresponding to the tth word in the question, represents the output of the update gate of the first layer of the GRU model at time t, r t 1 represents the output of the reset gate of the first layer of the GRU model at time t, represents the hidden state of the first layer of the GRU model at time t, is the parameter matrix corresponding to x t of the update gate of the first layer of the GRU model, is the parameter matrix corresponding to the hidden state of the last time of the update gate of the first layer of the GRU model , W t 1 is the parameter matrix corresponding to x t of the reset gate of the first layer of the GRU, is the parameter matrix corresponding to the hidden state of the last time of the reset gate of the first layer of the GRU model , W 1 is the parameter matrix corresponding to xt corresponding parameter matrix, U 1 is the reset gate output r t 1 and the hidden state at the previous time step corresponding parameter matrix, σ(.) represents the sigmoid activation function, tanh(.) represents the hyperbolic tangent function, represents the dot product operation of two vectors;

[0064] The question vector v q = {x1, x2,..., x k ,...,x n} is sequentially input into the forward GRU network to obtain the hidden state at time t corresponding to each word The question vector is represented in reverse order as input into the backward GRU to obtain the hidden state at time t corresponding to each word The forward and are concatenated as the output of the first layer of GRU at time t

[0065] The idea of residual network is used to connect the original question word vector representation x t and the output of the first layer of GRU after concatenation as the input of the second layer of GRU at time t, which is calculated as follows:

[0066]

[0067]

[0068]

[0069] wherein, represents the output of the update gate of the second layer of GRU at time t, r t 2 represents the output of the reset gate of the second layer of GRU at time t, represents the hidden state of the second layer of GRU at time t, W z 2 is the vector formed by concatenating x t and the hidden state of the first layer of GRU at time t corresponding parameter matrix, is the hidden state of the previous time step in the update gate of the second layer of GRU corresponding parameter matrix, W t 2 is the vector formed by concatenating xt the hidden state of the first layer of the GRU model at time t the parameter matrix corresponding to the concatenated vector, the hidden state of the previous time of the reset gate of the second layer of the GRU model the parameter matrix corresponding to the concatenated vector, W 2 the output of the second layer of the GRU model t the hidden state of the first layer of the GRU model at time t the parameter matrix corresponding to the concatenated vector, U 2 the output of the reset gate of the second layer of the GRU model t 2 the hidden state of the previous time the parameter matrix corresponding to the dot product operation result, σ(.) represents the sigmoid activation function, tanh(.) represents the hyperbolic tangent function, represents the dot product operation of two vectors. Similarly, the outputs of the forward and backward GRU at time t are concatenated as the output of the second layer of the GRU at time t

[0070] S412. Attention mechanism:

[0071] an attention mechanism is used to assign a weight α to the output hidden state of each time of the second layer of the GRU model i , and finally the hidden states of each time are accumulated to obtain the final sentence high-level feature representation h q , which is calculated as follows:

[0072]

[0073] α t = softmax(e t ) (8)

[0074]

[0075] wherein, the hidden vector of the second layer of the GRU at time t, W Attention is the hidden state of the second layer of the GRU at time t the parameter matrix corresponding to the concatenated vector, b is the bias vector.

[0076] S42. Question category probability calculation: input the sentence high-level feature representation obtained in step S31 into a linear layer for classification, output the probability corresponding to each category, and the category with the highest probability is taken as the question category predicted by the model.

[0077] S43. Cost Function and Iterative Training: The question dataset is divided into training and test sets. The model is trained using the training samples to obtain the question classification model. The training process selects cross-entropy as the loss function, calculated as follows:

[0078]

[0079] Where, p i This is the output of the linear layer, representing the probability that the question belongs to the i-th category, where N represents the number of question categories. i This indicates whether the classification output of the question is equal to the true category label of the question. i If the model outputs a category label for the question that is equal to the true label, then y ∈{0,1}. i If y is 1, otherwise y i It is 0.

[0080] S5. New Question Classification: For a question input by a user, after preprocessing in step S1, the question is vectorized in step S2 and input into the trained question classification model to obtain the classification result.

[0081] Example 2

[0082] Based on the solution given in Embodiment 1, the following specific examples, combined with the appendix, will illustrate further. Figure 2 The present invention will be described in further detail below. In this specific embodiment, taking the question "Can leukemia be cured?" as an example, we first preprocess the text and obtain the word vector x corresponding to each word in the sentence through the BERT model. i The sentence can be represented as v q Then, it is fed into a two-layer bidirectional GRU network. The input of the first layer of the GRU network is the word vector obtained by the BERT model. Through the first layer, we can extract the hidden state of each word. The original word vector x i and the hidden state vector of the first layer of the GRU network The connection is made, and the input is fed into the second layer of the GRU to extract the high-level hidden states of each word. Then, the attention layer is used to feed the hidden state vector of the second layer of the GRU network. Learning weight α i The learned weights are used to sum the hidden state vectors of the second layer of the GRU network to obtain the final high-level feature representation h. q Finally, the data is input into a linear layer for classification, and the output question category is "probability of disease cure".

[0083] With the same data set and parameters, the bidirectional gate recurrent unit BiGRU, the BERT model and the model (BERT+BiGRU+Attention) proposed in the present application are respectively used for experimental test, and the results are shown in Table 1. The results show that the question classification accuracy and F1 score of the present application are better than those of other models.

[0084] Table 1 Comparison of experimental results

[0085]

[0086] The above specific embodiments further explain the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for classifying questions, characterized in that, Includes the following steps: S1. Question Preprocessing: The collected questions from relevant fields are preprocessed. Each preprocessed question is represented as q = {q1, q2, ..., q}. k ,...,q n }, where q k This represents the k-th word in the question, where n is the total number of words in the question. S2. Vector Representation of Questions: Using a pre-trained BERT model, the question q is mapped and represented as a vector v. q ={x1,x2,...,x k ,...,x n }, where x k The word q k The corresponding word vectors; S3. Establish a question classification model: Use a two-layer bidirectional GRU model with attention mechanism to extract high-level features of questions. The forward GRU processes the sequential input of questions, and the backward GRU processes the reverse input of questions. S31. Advanced Feature Extraction of Questions: The text sequence is processed using a two-layer bidirectional GRU network structure. Combining the idea of ​​residual networks, the original word vectors are combined with the output of the first layer of the GRU network and used as the input of the second layer of GRU. An attention mechanism is used to assign a weight to the hidden state of the output at each time step of the second layer of GRU. Finally, the hidden states at each time step are accumulated to obtain the final advanced feature representation of the sentence. S311. Bidirectional GRU model: The calculation process of the first layer of the GRU model at time t is as follows: Where, x t This represents the word vector input to the first layer of the GRU model at time t. This represents the output of the gate that updates at time t in the first layer of the GRU model, r. t 1 This indicates the output of the gate in the first layer of the GRU model at time t when it is reset. This represents the hidden state of the first layer of the GRU model at time t. It is the first-level update gate of the GRU model, x t The corresponding parameter matrix, It is the hidden state of the previous time step in the first-level update gate of the GRU model. The corresponding parameter matrix, W t 1 It is the first level of the GRU reset gate x t The corresponding parameter matrix, It is the hidden state of the previous time step in the reset gate of the first layer of the GRU model. The corresponding parameter matrix, W 1 x is the output gate of the first layer of the GRU model t The corresponding parameter matrix, U 1 It is the output r of the reset gate in the first layer output gate of the GRU model. t 1 Hidden state from the previous moment The parameter matrix corresponding to the dot product result, where σ(.) represents the sigmoid activation function and tanh(.) represents the hyperbolic tangent function. This represents the dot product operation of two vectors; The question vector v q ={x1,x2,...,x k ,...,x n The words are sequentially fed into a feedforward GRU network to obtain the hidden state at time t for each word. Reverse the question vector as follows: The input is fed into a backward GRU to obtain the hidden state at time t for each word. Before and Link them together as the output of the first layer of the GRU at time t. Using the concept of residual networks, the word vector representation x of the original question is transformed. t Output of GRU layer 1 The concatenated vector is used as the input to the second layer of the GRU at time t, and is calculated as follows: in, This represents the output of the gate that updates at time t in the second layer of the GRU model, r. t 2 This indicates the output of the reset gate in the second layer of the GRU model at time t. This represents the hidden state of the second layer of the GRU model at time t. It is x in the second-level update gate of the GRU model t Hidden state at time t in the first layer of the GRU model The parameter matrix corresponding to the vector formed by linking them together. It is the hidden state of the previous time step in the second-level update gate of the GRU model. The corresponding parameter matrix, W t 2 It is x in the second-level reset gate of the GRU model t Hidden state at time t in the first layer of the GRU model The parameter matrix corresponding to the vector formed by linking them together. It is the hidden state of the previous time step in the reset gate of the second layer of the GRU model. The corresponding parameter matrix, W 2 It is x in the output gate of the second layer of the GRU model t Hidden state at time t in the first layer of the GRU model The parameter matrix corresponding to the vector formed by linking them together, U 2 It is the output r of the reset gate in the second layer output gate of the GRU model. t 2 Hidden state from the previous moment The parameter matrix corresponding to the dot product result, where σ(.) represents the sigmoid activation function and tanh(.) represents the hyperbolic tangent function. This represents the dot product operation of two vectors; similarly, it represents the output of the forward and backward GRU at time t. Link them together as the output of the second layer of the GRU at time t. S32. Question category probability calculation: Input the high-level feature representation of the sentence obtained in step S31 into a linear layer for classification, output the probability corresponding to each category, and take the category with the highest probability as the question category predicted by the model; S33. Cost Function and Iterative Training: Divide the question dataset into a training set and a test set, use the training samples to train the model, and finally obtain the question classification model; S4. New Question Classification: For a question input by a user, after preprocessing in step S1, the question is vectorized in step S2 and input into the trained question classification model to obtain the classification result.

2. The question classification method according to claim 1, characterized in that, Step S31 also includes the following steps: S312. Attention Mechanisms: The attention mechanism is used to assign a weight α to the output and hidden states at each time step of the second layer of the GRU model. i Finally, the hidden states at each time step are summed to obtain the final high-level feature representation h of the sentence. q The calculation is as follows: a t =softmax(e t ) (8) in, W represents the hidden vector at time t in layer 2 of the GRU. Attention It is the hidden state of the second layer of the GRU at time t. The corresponding parameter matrix, b is the bias vector.

3. The question classification method according to claim 1, characterized in that, In step S32, the training process selects cross-entropy as the loss function, and calculates it as follows: Where, p i This is the output of the linear layer, representing the probability that the question belongs to the i-th category, where N represents the number of question categories. i This indicates whether the classification output of the question is equal to the true category label of the question. i If the model outputs a category label for the question that is equal to the true label, then y ∈{0,1}. i If y is 1, otherwise y i It is 0.

4. The question classification method according to claim 1, characterized in that, The pre-trained BERT model uses 768-dimensional Chinese word vectors from Google's open-source database.

5. The question classification method according to claim 4, characterized in that, The pre-trained BERT model has a training batch size of 16, a learning rate of 0.001, a BERT fine-tuning learning rate of 0.00005, and uses Adam as the optimizer.

6. The question classification method according to claim 5, characterized in that, Before step S1, the process also includes question data collection, which involves collecting a certain number of questions in the relevant field, labeling and classifying them, with each question corresponding to a category.

7. The question classification method according to claim 6, characterized in that, The questions collected are categorized into 16 types: commonly used drugs for diseases, recommended foods for diseases, required examinations for diseases, foods to avoid for diseases, recommended drugs for diseases, symptoms of diseases, complications of diseases, departments treating diseases, treatment methods for diseases, treatment time for diseases, probability of cure for diseases, cost of treatment for diseases, preventive measures for diseases, causes of diseases, descriptions of diseases, and transmissibility of diseases.

8. The application method of the question classification method according to any one of claims 1 to 7, characterized in that, This system is used in the medical field to classify Chinese questions.

Images