Intelligent conversation method and device for improving efficiency of traditional Chinese medicine consultation

Abstract

The present invention discloses an intelligent conversation method and device for improving the efficiency of traditional Chinese medicine consultation, which belongs to the field of natural language processing and artificial intelligence. The technical problem to be solved by the present invention is how to better understand the context of conversation records, highlight the key information in conversation records and replies to be pushed, so as to determine the conversation records in the conversation records that are highly relevant to the replies to be pushed and filter out irrelevant information, thereby improving the prediction accuracy of the intelligent conversation system. The technical solution adopted is: S1, obtaining an intelligent conversation data set: crawling public traditional Chinese medicine consultation intelligent conversation data from the Internet and constructing a traditional Chinese medicine consultation intelligent conversation data set by itself; S2, constructing an intelligent conversation model: constructing an intelligent conversation model based on a conversation semantic adapter module; S3, training the intelligent conversation model: training the intelligent conversation model constructed in step S2 on the intelligent conversation training data set obtained in step S1.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and natural language processing technology, and in particular to an intelligent conversation method and device for improving the efficiency of traditional Chinese medicine consultation. Background Art

[0002] Traditional Chinese Medicine (TCM) is a medical system rooted in traditional Chinese culture. Its long history makes it one of the oldest and most complex medical disciplines in the world. However, learning TCM is not easy. Its theoretical framework is vast and complex, requiring mastery of numerous classic texts and diagnostic and treatment methods. Learners must devote considerable time and effort to understanding and applying TCM theory and practice. Furthermore, the learning process requires extensive clinical practice to enhance diagnostic and treatment skills. The high cost of TCM training also contributes to the relatively small number of TCM practitioners. Compared to Western medicine, training TCM practitioners requires longer time and greater resources, resulting in a shortage of TCM practitioners to meet social demand. Consequently, TCM practitioners are relatively scarce in the market, and limited resources are unable to meet the public's demand for TCM services. The introduction of intelligent conversation systems can alleviate this shortage of TCM practitioners to some extent. Leveraging artificial intelligence and big data analytics, these systems provide fast and accurate TCM health consultations and diagnostic recommendations, helping people address common health issues.

[0003] Conversation coherence is a crucial metric in intelligent conversation systems, impacting the fluency and naturalness of system responses. However, existing intelligent conversation methods have shortcomings in this regard. Existing intelligent conversation systems often fail to highlight key information in conversation transcripts and pending replies, resulting in an inability to filter out conversation transcripts with high relevance to pending replies. This results in replies pushed to users lacking support from previous context, limiting the logic and coherence of the conversation. Furthermore, when the user's expression changes slightly or the question's intent is unclear, the system is prone to misjudgment, and the returned answer may not meet the user's expectations, further reducing coherence. Existing methods have not effectively addressed this issue, and intelligent conversation tasks remain a challenging task.

[0004] In response to the above problems, the present invention proposes an intelligent conversation method and device for improving the efficiency of traditional Chinese medicine consultation, which can better understand the context of conversation records, highlight the key information in conversation records and replies to be pushed, so as to determine the conversation records in the conversation records that are highly relevant to the replies to be pushed and filter out irrelevant information, thereby improving the prediction accuracy of the intelligent conversation model. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an intelligent conversation method and device for improving the efficiency of traditional Chinese medicine consultation, so as to better understand the context of conversation records, highlight the key information in conversation records and replies to be pushed, so as to determine the conversation records in the conversation records that are highly relevant to the replies to be pushed and filter out irrelevant information, thereby improving the prediction accuracy of the intelligent conversation model.

[0006] The present invention adopts the following technical means to achieve the purpose of the invention:

[0007] An intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation, characterized in that the method comprises the following steps:

[0008] S1. Obtaining an intelligent conversation dataset: crawling publicly available TCM consultation intelligent conversation data from the Internet and constructing an intelligent conversation dataset on your own;

[0009] S2. Building an intelligent conversation model: Building an intelligent conversation model based on the conversation semantic adapter module;

[0010] S3. Training the intelligent conversation model: The intelligent conversation model constructed in step S2 is trained on the intelligent conversation training data set obtained in step S1.

[0011] As a further limitation of the present technical solution, step S2 is used to construct an input module, a pre-trained embedding module, a conversation semantic adapter module and a label prediction module, thereby constructing an intelligent conversation model.

[0012] As a further limitation of this technical solution, the construction input module extracts a piece of data from the constructed TCM consultation intelligent conversation data set, which contains n conversation records, a reply to be pushed and a label, and formalizes them as h1, h2, ..., h n , r and the corresponding label value to obtain an input data.

[0013] As a further limitation of the present technical solution, the pre-trained embedding module uses the pre-trained language model BERT to train the input data h1, h2, ..., h n and r are respectively extracted to obtain the feature representation of each conversation record in the conversation record and the feature representation of the reply to be pushed, which are recorded as and The specific implementation is shown in the following formula:

[0014]

[0015]

[0016] …,

[0017]

[0018]

[0019]

[0020] Among them, h1, h2, ..., h n-1 , h n Represents conversation record 1, conversation record 2, ..., conversation record n-1, and conversation record n ​​in the conversation record, r represents the reply to be pushed, and BERT() indicates the use of the pre-trained language model BERT to encode the content in the brackets.

[0021] As a further limitation of the present technical solution, the conversation semantic adapter module receives the feature representation of each conversation record output by the pre-trained embedding module, namely And the feature of the reply to be pushed is The feature representation of each conversation record and the feature representation of the reply to be pushed are connected separately, and then each connection result is encoded by an encoder to obtain multiple joint encodings; each joint encoding is calculated to be relevant to each corresponding conversation record, and the features of each conversation record are filtered based on each correlation representation to obtain the encoding representation of each conversation record, and the context feature aggregation operation is completed on the encoding representation of all conversation records through the connection operation and the self-attention mechanism; at the same time, the semantic features of all joint encodings are fused through the gating mechanism, and the semantic feature fusion representation is filtered through the self-attention mechanism; finally, the context fusion embedding and the semantic interaction embedding are semantically adapted and fused through the connection operation and the dense network based on the ReLU activation function to obtain the conversation semantic adaptation embedding, which is passed to the label prediction module;

[0022] Specifically, the implementation process of this module is as follows:

[0023] (1) Connection conversation record 1 shows And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code 1, which is recorded as Then the joint code 1 is And conversation record 1 shows Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding 1, which is recorded as Then embed the correlation into 1, And conversation record 1 shows Perform the MatMul matrix multiplication operation to obtain the conversation record 1 code, which is recorded as The specific implementation is shown in the following formula:

[0024]

[0025]

[0026]

[0027] in, The conversation record 1 indicates that Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets;

[0028] (2) Connection conversation record 2 shows And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code 2, which is recorded as Then the joint code 2 is And conversation record 2 shows Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding 2, which is recorded as Then embed the correlation into 2, And conversation record 2 shows Perform the MatMul matrix multiplication operation to obtain the conversation record 2 code, which is recorded as The specific implementation is shown in the following formula:

[0029]

[0030]

[0031]

[0032] in, The conversation record 2 indicates that, Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets;

[0033] Then the joint code 1 is and joint coding 2 Perform selective fusion to obtain fusion embedding 1, which is recorded as The specific implementation is shown in the following formula:

[0034]

[0035]

[0036] in, represents joint coding 1, represents joint coding 2, and Represents the weight to be trained, σ represents the sigmoid function, gate1 represents the first layer gate weight, and ⊙ represents element-wise multiplication;

[0037] (3) Connection conversation record 3 shows And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code 3, which is recorded as Then the joint code 3 is And conversation record 3 shows Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding 3, which is recorded as Then embed the correlation into 3 And conversation record 3 shows Perform the MatMul matrix multiplication operation to obtain the conversation record 3 encoding, which is recorded as The specific implementation is shown in the following formula:

[0038]

[0039]

[0040]

[0041] in, Conversation record 3 indicates that, Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets;

[0042] Then the fusion embedding 1 is and joint coding 3 Perform selective fusion to obtain fusion embedding 2, which is recorded as The specific implementation is shown in the following formula:

[0043]

[0044]

[0045] in, represents fusion embedding 1, represents joint coding 3, and Represents the weight to be trained, σ represents the sigmoid function, gate2 represents the second-layer gating weight, and ⊙ represents element-wise multiplication;

[0046] (4) Connection conversation record 4 shows And the reply to be pushed Subsequent operation process and obtaining conversation record 3 coding and fusion embedding 2 The operation is similar, thus obtaining the conversation record 4 encoding and fusion embedding 3, which is recorded as and And so on, until the connection conversation record n ​​is represented And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code n, which is recorded as Then the joint code n is And the conversation record n ​​indicates Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding n, which is recorded as Then embed the correlation n And the conversation record n ​​indicates Perform the MatMul matrix multiplication operation to obtain the conversation record n ​​code, which is recorded as The specific implementation is shown in the following formula:

[0047]

[0048]

[0049]

[0050] in, Indicates conversation record n ​​indicates, Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets;

[0051] Then the fusion is embedded into n-2 through the gating mechanism. and joint coding n Perform selective fusion to obtain fusion embedding n-1, recorded as The specific implementation is shown in the following formula:

[0052]

[0053]

[0054] in, represents fusion embedding n-2, represents the joint code n, and Represents the weight to be trained, σ represents the sigmoid function, gate n-1 represents the gate weight of the n-1th layer, ⊙ represents element-wise multiplication;

[0055] (5) All conversation records are encoded through the join operation and self-attention mechanism to complete the context feature aggregation operation, thereby obtaining the context fusion embedding, which is recorded as At the same time, the fusion is embedded into n-1 through the self-attention mechanism. Perform feature filtering to obtain semantic interaction embedding, which is recorded as Finally, the context fusion embedding is performed through the connection operation and the dense network based on the ReLU activation function and semantic interaction embedding Perform semantic adaptation fusion operation to obtain conversation semantic adaptation embedding, which is recorded as The specific implementation is shown in the following formula:

[0056]

[0057]

[0058]

[0059] in, Indicates the code of conversation record 1, the code of conversation record 2, the code of conversation record 3, ..., the code of conversation record n, Represents fusion embedding n-1, Self-Attention(Concatenate(;)) indicates the use of the self-attention mechanism to encode the connection results of the contents in the brackets, Self-Attention(;) indicates the use of the self-attention mechanism to filter the semantic features of the contents in the brackets, ReLU(Dense(Concatenate(;))) indicates the use of the dense network Dense based on the ReLU activation function to perform semantic adaptation and fusion operations on the connection results in the brackets.

[0060] As a further limitation of the present technical solution, the label prediction module uses the conversation semantic adaptation embedding output by the conversation semantic adapter module as the input of this module, and processes it using a fully connected network with a layer dimension of 1 and an activation function of Sigmod, so as to obtain the probability that the current reply to be pushed is the correct reply to be pushed; compare the probabilities of different replies to be pushed, and push the reply to be pushed with the highest probability as the correct reply to be pushed to the user.

[0061] An intelligent conversation device for improving the efficiency of traditional Chinese medicine consultation using claim 1, characterized in that the device comprises:

[0062] An intelligent conversation data set acquisition unit is used to crawl public TCM consultation intelligent conversation data from the Internet and construct an intelligent conversation data set by itself;

[0063] An intelligent conversation model construction unit, which is used to construct an input module, a pre-trained embedding module, a conversation semantic adapter module, and a label prediction module, thereby constructing an intelligent conversation model;

[0064] The intelligent conversation model training unit is used to build loss functions and optimization functions to complete the prediction of replies to be pushed.

[0065] An electronic device comprising: a memory and at least one processor;

[0066] Wherein, the memory stores a computer program;

[0067] The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the above-mentioned intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation.

[0068] A computer-readable storage medium stores a computer program, which can be executed by a processor to implement the above-mentioned intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] (1) The present invention can effectively extract deep semantic features of conversation records and replies to be pushed by pre-training the embedding module, thereby improving the semantic extraction capability of the intelligent conversation system;

[0071] (2) The present invention can better understand the conversation record context through the conversation semantic adapter module, thereby obtaining a higher quality context representation;

[0072] (3) The present invention can highlight key information in the conversation record and the reply to be pushed through the conversation semantic adapter module, so as to determine the conversation record with high relevance to the reply to be pushed in the conversation record and filter out irrelevant information, thereby obtaining a more accurate conversation representation;

[0073] (4) The method and device proposed in the present invention, combined with the conversation semantic adapter module, can effectively improve the accuracy of the intelligent conversation system in pushing correct replies to be pushed. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The present invention is a flowchart of an intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation.

[0075] Figure 2 Flowchart for building an intelligent conversation model.

[0076] Figure 3 Flowchart for training an intelligent conversation model.

[0077] Figure 4 The present invention is a flow chart of an intelligent conversation device for improving the efficiency of traditional Chinese medicine consultation.

[0078] Figure 5 This is a structural diagram of the conversation semantics adapter module.

[0079] Figure 6 Schematic diagram of the framework of the intelligent conversation model used to improve the efficiency of traditional Chinese medicine consultation. DETAILED DESCRIPTION

[0080] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0081] The following is a detailed description of an intelligent conversation method and device for improving the efficiency of traditional Chinese medicine consultation according to the present invention with reference to the accompanying drawings and specific embodiments.

[0082] Example 1:

[0083] The overall model framework structure of the present invention is as follows Figure 6 As shown. Figure 6 As can be seen, the main framework of the present invention includes a pre-trained embedding module, a conversation semantics adapter module, and a label prediction module. The pre-trained embedding module extracts features from each conversation record and reply to be pushed in the conversation log, thereby obtaining a feature representation of each conversation record and a feature representation of the reply to be pushed, which are then passed to the conversation semantics adapter module. The conversation semantics adapter module first concatenates the feature representation of each conversation record and the feature representation of the reply to be pushed. It then encodes each concatenation result using an encoder, resulting in multiple joint encodings. Each joint encoding is then correlated with its corresponding conversation record. Based on each correlation representation, features are filtered for each conversation record to obtain an encoded representation for each conversation record. Contextual features are aggregated across all encoded representations of conversation records using a concatenation operation and a self-attention mechanism. Semantic features are fused across all joint encodings using a gating mechanism, and feature filtering is performed on the fused representations using a self-attention mechanism. Finally, semantic adaptation is performed on the context-fused embedding and feature filtering using a concatenation operation and a dense network based on the ReLU activation function, resulting in a conversation semantics-adapted embedding, which is then passed to the label prediction module. The tag prediction module maps the conversation semantics embedding to a floating-point value in a specified interval, which is used as the match between the reply to be pushed and the conversation record. It then compares the matching scores of different replies and selects the one with the highest matching score as the correct reply. The details are as follows:

[0084] (1) The pre-trained embedding module uses the pre-trained language model BERT to extract features from the input data, thereby obtaining the feature representation of each conversation record in the conversation record and the embedding of the reply to be pushed;

[0085] (2) The conversation semantics adapter module extracts and filters the feature representation of each conversation record and the feature representation of the reply to be pushed through joint encoding and interactive operations, thereby obtaining the conversation semantic adaptation embedding;

[0086] (3) The tag prediction module takes the conversation semantic adaptation embedding as the input of this module and processes it using a fully connected network with a layer dimension of 1 and an activation function of Sigmod, so as to obtain the probability that the current reply to be pushed is the correct reply to be pushed; compare the probabilities of different replies to be pushed, and push the reply to be pushed with the highest probability as the correct reply to be pushed to the user.

[0087] Example 2:

[0088] As attached Figure 1 As shown, the present invention is an intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation, which includes the following steps:

[0089] S1. Obtaining an intelligent conversation dataset: crawling publicly available TCM consultation intelligent conversation data from the Internet and constructing an intelligent conversation dataset on your own;

[0090] S2. Building an intelligent conversation model: Building an intelligent conversation model using the conversation semantic adapter module;

[0091] S3. Training the intelligent conversation model: The intelligent conversation model constructed in step S2 is trained on the intelligent conversation training data set obtained in step S1.

[0092] S1. Obtaining the intelligent conversation dataset

[0093] S101. Use crawlers to obtain data: Write a crawler program to crawl the public doctor-patient conversation data on the traditional Chinese medicine consultation platform that opens medical records to the public, and obtain the original data of intelligent conversation.

[0094] For example, a crawler program is written for a TCM consultation platform that opens its medical records to the public to crawl the doctor-patient conversation records, thereby obtaining the original data of the intelligent conversation. An example of a doctor-patient conversation record on a TCM consultation platform is shown in the following table:

[0095]

[0096]

[0097] S102. Construct a dataset of intelligent conversations for traditional Chinese medicine consultations: construct positive samples and negative samples based on each conversation record, and divide all sample data into training sets and test sets according to a preset ratio.

[0098] For example, taking the data shown in step S101 as an example, consider S1 to S5 as the conversation record sequence, and S6 as the correct reply to be pushed, which is marked as 1; and randomly select 3 conversation records from the other data and use them as the incorrect replies for the data, and mark them as 0. The data format in this dataset is as follows:

[0099]

[0100] The dataset is divided into training set and test set in a ratio of 4:1.

[0101] S2. Build an intelligent conversation model

[0102] The process of building an intelligent conversation model is as follows: Figure 2 As shown in the figure, the main operations are to build an input module, build a pre-trained embedding module, build a conversation semantic adapter module, and build a label prediction module.

[0103] S201. Build input module

[0104] A piece of data is extracted from the TCM consultation intelligent conversation dataset constructed in step S102. The data contains n conversation records, a reply to be pushed, and a label. The data is formalized into h1, h2, ..., hn, r and the corresponding label values, thereby obtaining an input data.

[0105] For example, using the data shown in S102 as an example, we will show an input data. The result is as follows:

[0106] (h1: Hello doctor, I have pain from my chest to my throat, a foreign body sensation, burping, and sour food coming back up from my stomach into my throat, especially in the afternoon and evening. What's going on? h2: This is a typical symptom of gastroesophageal reflux. How long has it been going on? h3: I've had acid reflux for several months, and sternum pain and sore throat for more than 10 days. h4: Have you done any related tests before? For example, gastroscopy. h5: I had a gastroscopy in June this year, and the result was chronic superficial gastritis with erosion. I prescribed some medicine, but there was no improvement. Doctor, what should I do next? r: Please send me your medication list first. Also, don't eat sweets or spicy foods in the near future. Your condition is very sensitive to diet, and you should pay attention to regulating your mood. 1).

[0107] S202. Build a pre-trained embedding module

[0108] The pre-trained embedding module uses the pre-trained language model BERT to extract features from the input data h1, h2, ..., hn and r, respectively, to obtain the feature representation of each conversation record in the conversation record and the feature representation of the reply to be pushed, which is recorded as and The specific implementation is shown in the following formula:

[0109]

[0110]

[0111] …,

[0112]

[0113]

[0114]

[0115] Among them, h1, h2, …, hn-1, hn represent conversation record 1, conversation record 2, …, conversation record n-1, and conversation record n ​​in the session record, r represents the reply to be pushed, and BERT() indicates using the pre-trained language model BERT to encode the content in the brackets.

[0116] For example, when implementing this invention, the pre-trained language model BERT is called to complete the encoding operation of each conversation record and the reply to be pushed. All parameter settings are based on the default settings of BERT in PyTorch. In PyTorch, the code described above is implemented as follows:

[0117] #Use bert's encoding layer to encode the input data

[0118] h_encoder_list = []

[0119] for iin h_embed_list:

[0120] h_encoder_list.append(BERT(i)[1])

[0121] r_embed=BERT(r)[1]

[0122] Among them, h_embed_list stores each conversation record in the session record, r is the reply to be pushed, h_encoder_list stores the feature representation of each conversation record in the session record, and r_embed is the embedding of the reply to be pushed.

[0123] S203. Constructing a conversation semantic adapter module

[0124] The conversation semantic adapter module receives the feature representation of each conversation record output by the pre-trained embedding module, namely And the feature of the reply to be pushed is The feature representation of each conversation record and the feature representation of the reply to be pushed are connected, and each connection result is encoded by an encoder to obtain multiple joint encodings. The correlation between each joint encoding and its corresponding conversation record is calculated, and the features of each conversation record are filtered based on each correlation representation to obtain the encoding representation of each conversation record. The context feature aggregation operation is completed on the encoding representation of all conversation records through the connection operation and the self-attention mechanism. At the same time, the semantic features of all joint encodings are fused through the gating mechanism, and the semantic feature fusion representation is filtered through the self-attention mechanism. Finally, the context fusion embedding and the semantic interaction embedding are semantically adapted and fused through the connection operation and the dense network based on the ReLU activation function to obtain the conversation semantic adaptation embedding, which is passed to the label prediction module.

[0125] Specifically, the implementation process of this module is as follows:

[0126] S20301 Connection conversation record 1 shows And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code 1, which is recorded as Then the joint code 1 is And conversation record 1 shows Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding 1, which is recorded as Then embed the correlation into 1, And conversation record 1 shows Perform the MatMul matrix multiplication operation to obtain the conversation record 1 code, which is recorded as The specific implementation is shown in the following formula:

[0127]

[0128]

[0129]

[0130] in, The conversation record 1 indicates that Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets;

[0131] S20302 Connection conversation record 2 shows And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code 2, which is recorded as Then the joint code 2 is And conversation record 2 shows Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding 2, which is recorded as Then embed the correlation into 2, And conversation record 2 shows Perform the MatMul matrix multiplication operation to obtain the conversation record 2 code, which is recorded as The specific implementation is shown in the following formula:

[0132]

[0133]

[0134]

[0135] in, The conversation record 2 indicates that, Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets;

[0136] Then the joint code 1 is and joint coding 2 Perform selective fusion to obtain fusion embedding 1, which is recorded as The specific implementation is shown in the following formula:

[0137]

[0138]

[0139] in, represents joint coding 1, represents joint coding 2, and Represents the weight to be trained, σ represents the sigmoid function, gate1 represents the first layer gate weight, and ⊙ represents element-wise multiplication;

[0140] S20303 Connection conversation record 3 shows And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code 3, which is recorded as Then the joint code 3 is And conversation record 3 shows Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding 3, which is recorded as Then embed the correlation into 3 And conversation record 3 shows Perform the MatMul matrix multiplication operation to obtain the conversation record 3 encoding, which is recorded as The specific implementation is shown in the following formula:

[0141]

[0142]

[0143]

[0144] in, Conversation record 3 indicates that, Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets;

[0145] Then the fusion embedding 1 is and joint coding 3 Perform selective fusion to obtain fusion embedding 2, which is recorded as The specific implementation is shown in the following formula:

[0146]

[0147]

[0148] in, represents fusion embedding 1, represents joint coding 3, and Represents the weight to be trained, σ represents the sigmoid function, gate2 represents the second-layer gating weight, and ⊙ represents element-wise multiplication;

[0149] S20304 Connection conversation record 4 shows And the reply to be pushed Subsequent operation process and obtaining conversation record 3 coding and fusion embedding 2 The operation is similar, thus obtaining the conversation record 4 encoding and fusion embedding 3, which is recorded as and And so on, until the connection conversation record n ​​is represented And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code n, which is recorded as Then the joint code n is And the conversation record n ​​indicates Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding n, which is recorded as Then embed the correlation n And the conversation record n ​​indicates Perform the MatMul matrix multiplication operation to obtain the conversation record n ​​code, which is recorded as The specific implementation is shown in the following formula:

[0150]

[0151]

[0152]

[0153] in, Indicates conversation record n ​​indicates, Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets;

[0154] Then the fusion is embedded into n-2 through the gating mechanism. and joint coding n Perform selective fusion to obtain fusion embedding n-1, recorded as The specific implementation is shown in the following formula:

[0155]

[0156]

[0157] in, represents fusion embedding n-2, represents the joint code n, and Represents the weight to be trained, σ represents the sigmoid function, gate n-1 represents the gate weight of the n-1th layer, ⊙ represents element-wise multiplication;

[0158] S20305 encodes all conversation records through the join operation and self-attention mechanism to complete the context feature aggregation operation, thereby obtaining the context fusion embedding, which is recorded as At the same time, the fusion is embedded into n-1 through the self-attention mechanism. Perform feature filtering to obtain semantic interaction embedding, which is recorded as Finally, the context fusion embedding is performed through the connection operation and the dense network based on the ReLU activation function and semantic interaction embedding Perform semantic adaptation fusion operation to obtain conversation semantic adaptation embedding, which is recorded as The specific implementation is shown in the following formula:

[0159]

[0160]

[0161]

[0162] in, Indicates the code of conversation record 1, the code of conversation record 2, the code of conversation record 3, ..., the code of conversation record n, Represents fusion embedding n-1, Self-Attention(Concatenate(;)) indicates the use of the self-attention mechanism to encode the connection results of the contents in the brackets, Self-Attention(;) indicates the use of the self-attention mechanism to filter the semantic features of the contents in the brackets, ReLU(Dense(Concatenate(;))) indicates the use of the dense network Dense based on the ReLU activation function to perform semantic adaptation and fusion operations on the connection results in the brackets.

[0163] For example, the encoding network encoder in this invention uses the Transformer Encoder, the encoding dimension is set to 512, the number of layers is set to 1, and the Self-Attention uses the Add Self-Attention calculation method. Taking the calculation of semantic interaction embedding as an example, the calculation process is as follows:

[0164]

[0165] This formula represents the interactive computation of semantic interaction embedding itself through element-by-element addition operation. represents fusion embedding n-1, and Represents the parameters to be trained;

[0166]

[0167] This formula represents the attention weight α obtained by normalization operation, i and i' represent the element subscripts in the corresponding input tensor, and l represents the input tensor The number of elements in , and the meanings of other symbols are the same as above;

[0168]

[0169] This formula indicates that the obtained attention weight α is used to complete the feature screening of the fusion embedding n-1, thereby obtaining the semantic interaction embedding, and l represents and the number of elements in α;

[0170] In PyTorch, the code described above is implemented as follows:

[0171]

[0172]

[0173] Among them, history_embed_list represents a list of all conversation record representations in the session record; response_embed represents the reply representation to be pushed; and yisp represents conversation semantic adaptation embedding.

[0174] S204, build label prediction module

[0175] The tag prediction module uses the conversation semantic adaptation embedding output from step S203 as the input of this module, and processes it using a fully connected network with a layer dimension of 1 and an activation function of Sigmod, so as to obtain the probability that the current reply to be pushed is the correct reply to be pushed; compare the probabilities of different replies to be pushed, and push the reply to be pushed with the highest probability as the correct reply to be pushed to the user.

[0176] When the model has not been trained, step S3 needs to be further executed for training to optimize the parameters of the model; when the model training is completed, step S204 can be executed to predict which of the replies to be pushed is the correct reply to be pushed.

[0177] S3. Training the intelligent conversation model

[0178] The intelligent conversation model constructed in step S2 is trained on the intelligent conversation training data set obtained in step S1. The process is as follows: Figure 3 shown.

[0179] S301. Construct a loss function.

[0180] The present invention adopts cross entropy as the loss function, and the formula is as follows.

[0181]

[0182] Among them, y true is the true label, y pred is the correct probability output by the model.

[0183] For example, in PyTorch, the code described above is implemented as follows:

[0184] #Calculate the error between the predicted value and the label through the cross entropy loss function

[0185] loss_fct = CrossEntropyLoss()

[0186] loss=loss_fct(logits.view(-1,self.num_labels),labels.view(-1))

[0187] Among them, labels is the real label, and logits is the correct probability of the model output.

[0188] S302. Build optimization function

[0189] After testing various optimization functions, the model finally chose to use the AdamW optimization function as the optimization function. Except for its learning rate set to 2e-5, the other hyperparameters of AdamW were set to the default values ​​in pytorch.

[0190] For example, in PyTorch, the code described above is implemented as follows:

[0191] #Optimize the model parameters through AdamW optimizer

[0192] optimizer=AdamW(optimizer_grouped_parameters,lr=2e-5)

[0193] optimizer_grouped_parameters are the parameters to be optimized, which default to all parameters in the intelligent conversation model.

[0194] When the model has not been trained, step S3 needs to be further executed for training to optimize the parameters of the model; when the model training is completed, step S204 can be executed to predict which of the replies to be pushed is the correct reply to be pushed.

[0195] Example 3:

[0196] This device mainly consists of three units, namely the intelligent conversation data set acquisition unit, the intelligent conversation model construction unit and the intelligent conversation model training unit. Figure 4 The specific functions of each unit are described as follows:

[0197] The intelligent conversation data set acquisition unit is used to crawl the public Chinese medicine consultation intelligent conversation data from the Internet and construct the intelligent conversation data set by itself.

[0198] The intelligent conversation model construction unit is used to construct an input module, a pre-trained embedding module, a conversation semantic adapter module, and a label prediction module, thereby constructing an intelligent conversation model.

[0199] The intelligent conversation model training unit is used to build loss functions and optimization functions to complete the prediction of replies to be pushed.

[0200] Furthermore, the intelligent conversation model construction unit also includes:

[0201] The input module unit is responsible for preprocessing the original data set to construct the input data.

[0202] Construct a pre-trained embedding module unit, which is responsible for encoding the input data using the pre-trained language model BERT, so as to obtain the feature representation of each conversation record in the historical conversation and the feature representation of the reply to be pushed.

[0203] Construct a conversation semantic adapter module unit, which is responsible for receiving the feature representation of each conversation record and the feature representation of the reply to be pushed output by the pre-trained embedding module, and then respectively connect the feature representation of each conversation record and the feature representation of the reply to be pushed, and then perform encoding operations on each connection result through an encoder to obtain multiple joint encodings, and then calculate the correlation between each joint encoding and each corresponding conversation record, and perform feature screening on each conversation record based on each correlation representation to obtain the encoding representation of each conversation record, and complete the context feature aggregation operation on the encoding representation of all conversation records through the connection operation and the self-attention mechanism, and at the same time perform semantic feature fusion on all joint encodings through the gating mechanism, and perform feature filtering on the semantic feature fusion representation through the self-attention mechanism, and finally perform semantic adaptation fusion operation on the context fusion embedding and the semantic interaction embedding through the connection operation and the dense network based on the ReLU activation function to obtain the conversation semantic adaptation embedding.

[0204] Build a tag prediction module unit, which is responsible for judging whether the current reply to be pushed is the correct one based on the conversation semantic adaptation embedding.

[0205] The intelligent conversation model training unit also includes:

[0206] Construct a loss function unit, which is responsible for calculating the error between the predicted result and the true label using the cross entropy loss function.

[0207] Construct an optimization function unit, which is responsible for training and adjusting the parameters in model training to reduce prediction errors.

[0208] Example 4:

[0209] This embodiment also provides an electronic device, including: a memory and a processor;

[0210] wherein the memory stores computer-executable instructions;

[0211] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation in any embodiment of the present invention.

[0212] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0213] The memory can be used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, the memory can also include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state memory devices.

[0214] Example 5:

[0215] This embodiment further provides a computer-readable storage medium storing a plurality of instructions, which are loaded by a processor to cause the processor to execute the intelligent conversation method for improving the efficiency of Traditional Chinese Medicine consultations according to any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, wherein the storage medium stores software program code that implements the functions of any of the above-described embodiments, and a computer (or CPU or MPU) of the system or device can read and execute the program code stored in the storage medium.

[0216] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0217] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-Rs, CD-RWs, DVD-ROMs, DVD-RYMs, DVD-RWs, DVD+RWs), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer via a communications network.

[0218] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0219] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation, characterized by: The method comprises the following steps: S1. Obtaining an intelligent conversation dataset: crawling publicly available TCM consultation intelligent conversation data from the Internet and constructing an intelligent conversation dataset on your own; S2. Building an intelligent conversation model: Building an intelligent conversation model based on the conversation semantic adapter module; S3. Training the intelligent conversation model: training the intelligent conversation model constructed in step S2 on the intelligent conversation training dataset obtained in step S1; The conversation semantic adapter module receives the feature representation of each conversation record output by the pre-trained embedding module, namely And the feature representation of the reply to be pushed The feature representation of each conversation record and the feature representation of the reply to be pushed are connected separately, and then each connection result is encoded by an encoder to obtain multiple joint codes; each joint code is calculated to be relevant to each corresponding conversation record, and each conversation record is feature filtered based on each correlation representation to obtain the encoded representation of each conversation record, and the context feature aggregation operation is completed on the encoded representation of all conversation records through the connection operation and the self-attention mechanism; at the same time, the semantic features of all joint codes are fused through the gating mechanism, and the semantic feature fusion representation is feature filtered through the self-attention mechanism; finally, the context fusion embedding and the semantic interaction embedding are semantically adapted and fused through the connection operation and the dense network based on the ReLU activation function to obtain the conversation semantic adaptation embedding, and pass it to the label prediction module.

2. The intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation according to claim 1 is characterized by: The step S2 is used to construct an input module, a pre-trained embedding module, a conversation semantic adapter module, and a label prediction module, thereby constructing an intelligent conversation model.

3. The intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation according to claim 2 is characterized by: The input module extracts a piece of data from the constructed TCM consultation intelligent conversation dataset, which contains n conversation records, a reply to be pushed and a label, and formalizes them into h1, h2, ..., h n , r and the corresponding label value to obtain an input data.

4. The intelligent conversation method for improving the efficiency of TCM consultation according to claim 3 is characterized by: The pre-trained embedding module uses the pre-trained language model BERT to train the input data h1, h2, ..., h n and r are respectively extracted to obtain the feature representation of each conversation record in the conversation record and the feature representation of the reply to be pushed, which are recorded as and The specific implementation is shown in the following formula: Among them, h1, h2, ..., h n-1 , h n Represents conversation record 1, conversation record 2, ..., conversation record n-1, and conversation record n ​​in the conversation record, r represents the reply to be pushed, and BERT() indicates the use of the pre-trained language model BERT to encode the content in the brackets.

5. The intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation according to claim 1 is characterized by: The implementation process of the conversation semantics adapter module is as follows: S20301 Connection conversation record 1 shows And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code 1, which is recorded as Then the joint code 1 is And conversation record 1 shows Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding 1, which is recorded as Then embed the correlation into 1, And conversation record 1 shows Perform the MatMul matrix multiplication operation to obtain the conversation record 1 code, which is recorded as The specific implementation is shown in the following formula: in, The conversation record 1 indicates that Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets; S20302 Connection conversation record 2 shows And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code 2, which is recorded as Then the joint code 2 is And conversation record 2 shows Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding 2, which is recorded as Then embed the correlation into 2, And conversation record 2 shows Perform the MatMul matrix multiplication operation to obtain the conversation record 2 code, which is recorded as The specific implementation is shown in the following formula: in, The conversation record 2 indicates that, Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets; Then the joint code 1 is and joint coding 2 Perform selective fusion to obtain fusion embedding 1, which is recorded as The specific implementation is shown in the following formula: in, represents joint coding 1, represents joint coding 2, and Represents the weight to be trained, σ represents the sigmoid function, gate1 represents the first layer gate weight, and ⊙ represents element-wise multiplication; S20303 Connection conversation record 3 shows And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code 3, which is recorded as Then the joint code 3 is And conversation record 3 shows Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding 3, which is recorded as Then embed the correlation into 3 And conversation record 3 shows Perform the MatMul matrix multiplication operation to obtain the conversation record 3 encoding, which is recorded as The specific implementation is shown in the following formula: in, Conversation record 3 indicates that, Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets; Then the fusion embedding 1 is and joint coding 3 Perform selective fusion to obtain fusion embedding 2, which is recorded as The specific implementation is shown in the following formula: in, represents fusion embedding 1, represents joint coding 3, and Represents the weight to be trained, σ represents the sigmoid function, gate2 represents the second-layer gating weight, and ⊙ represents element-wise multiplication; S20304 Connection conversation record 4 shows And the reply to be pushed Subsequent operation process and obtaining conversation record 3 coding and fusion embedding 2 The operation is similar, thus obtaining the conversation record 4 encoding and fusion embedding 3, which is recorded as and And so on, until the connection conversation record n ​​is represented And the reply to be pushed And use an encoder to encode the connection result to obtain the joint code n, which is recorded as Then the joint code n is And the conversation record n ​​indicates Perform the Dot multiplication operation and use the Softmax activation function to normalize the operation result to obtain the correlation embedding n, which is recorded as Then embed the correlation n And the conversation record n ​​indicates Perform the MatMul matrix multiplication operation to obtain the conversation record n ​​code, which is recorded as The specific implementation is shown in the following formula: in, Indicates conversation record n ​​indicates, Indicates a reply to be pushed, · indicates Dot multiplication operation, Indicates the MatMul matrix multiplication operation, Encoder(Concatenate(;)) indicates the use of the encoder Encoder to encode the connection result in the brackets, and Softmax() indicates the use of the Softmax function to normalize the content in the brackets; Then the fusion is embedded into n-2 through the gating mechanism. and joint coding n Perform selective fusion to obtain fusion embedding n-1, recorded as The specific implementation is shown in the following formula: in, represents fusion embedding n-2, represents the joint code n, and Represents the weight to be trained, σ represents the sigmoid function, gate n-1 represents the gate weight of the n-1th layer, ⊙ represents element-wise multiplication; S20305 encodes all conversation records through the join operation and self-attention mechanism to complete the context feature aggregation operation, thereby obtaining the context fusion embedding, which is recorded as At the same time, the fusion is embedded into n-1 through the self-attention mechanism. Perform feature filtering to obtain semantic interaction embedding, which is recorded as Finally, the context fusion embedding is performed through the connection operation and the dense network based on the ReLU activation function and semantic interaction embedding Perform semantic adaptation fusion operation to obtain conversation semantic adaptation embedding, which is recorded as The specific implementation is shown in the following formula: in, Indicates the code of conversation record 1, the code of conversation record 2, the code of conversation record 3, ..., the code of conversation record n, Represents fusion embedding n-1, Self-Attention(Concatenate(;)) indicates the use of the self-attention mechanism to encode the connection results of the contents in the brackets, Self-Attention(;) indicates the use of the self-attention mechanism to filter the semantic features of the contents in the brackets, ReLU(Dense(Concatenate(;))) indicates the use of the dense network Dense based on the ReLU activation function to perform semantic adaptation and fusion operations on the connection results in the brackets.

6. The intelligent conversation method for improving the efficiency of TCM consultation according to claim 5 is characterized by: The tag prediction module uses the conversation semantic adaptation embedding output by the conversation semantic adapter module as the input of this module, and processes it using a fully connected network with a layer dimension of 1 and an activation function of Sigmod, so as to obtain the probability that the current reply to be pushed is the correct reply to be pushed; compare the probabilities of different replies to be pushed, and push the reply to be pushed with the highest probability as the correct reply to be pushed to the user.

7. A device utilizing the intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation according to claim 1, characterized in that: The device includes: An intelligent conversation data set acquisition unit is used to crawl public TCM consultation intelligent conversation data from the Internet and construct an intelligent conversation data set by itself; An intelligent conversation model construction unit, which is used to construct an input module, a pre-trained embedding module, a conversation semantic adapter module, and a label prediction module, thereby constructing an intelligent conversation model; The intelligent conversation model training unit is used to build loss functions and optimization functions to complete the prediction of replies to be pushed.

8. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores a computer program; The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the intelligent conversation method for improving the efficiency of traditional Chinese medicine consultation as described in any one of claims 1 to 6.

Images