A health question-answering model training, health question-answering method and related device

Abstract

The present invention discloses a training method for a health question-answering model, a health question-answering method, and related devices. The method comprises the following steps: collecting multiple samples; inputting health question information and health answer information into a bidirectional encoder based on a twin sentence vector transformer for preliminary encoding, obtaining a question intermediate vector and an answer intermediate vector output by an intermediate layer; converting the question intermediate vector into a question key vector controlled by question keywords; converting the answer intermediate vector into an answer key vector controlled by answer keywords; inputting the question key vector and the answer key vector into a bidirectional encoder based on a twin sentence vector transformer for further encoding, obtaining a question target vector and an answer target vector; and training the bidirectional encoder based on the twin sentence vector transformer into a health question-answering model based on the question target vector and the answer target vector. The bidirectional encoder based on the twin sentence vector transformer focuses on keywords containing health entity words, thereby improving adaptability in the health field and the accuracy of answer recall.

Description

Technical Field

[0001] The present invention relates to the technical field of health, and in particular to a health question-answering model training, a health question-answering method and related devices. Background Art

[0002] In the intelligent customer service scenario in the health field, intelligent questions and answers will often be provided. The user's intention will be identified and the closest standard answer will be given, which can quickly help users solve some repetitive or common problems independently, such as the efficacy of nutrients, instructions for use of nutrients, etc.

[0003] The bidirectional encoder based on the twin sentence vector transformer (Sentence-BERT) is currently one of the mainstream recall models for intelligent question answering. It calculates vectors for the knowledge base in advance, thereby quickly responding to recall tasks and saving a lot of inference time.

[0004] On the one hand, the amount of question-answering data in the health field is scarce, so Sentence-BERT has poor adaptability to the health field, resulting in low accuracy of recalled answers.

[0005] On the other hand, since entity words such as products, diseases, and nutrients in the health field are iterating rapidly, in order to understand new entity words, developers retrain Sentence-BERT at regular intervals. Due to the large amount of annotations, the cost of training Sentence-BERT is high and the efficiency is low. Summary of the Invention

[0006] The present invention provides a health question-answering model training, a health question-answering method, and related devices to solve the problem of how to improve the accuracy of Sentence-BERT recall answers and improve the efficiency of Sentence-BERT in adapting to new entity words.

[0007] According to one aspect of the present invention, a method for training a health question-answering model is provided, the method comprising:

[0008] S101: Collect multiple samples, each sample including health question information and health answer information, wherein the health question information contains question keywords of health entity words, and the health answer information contains answer keywords of health entity words;

[0009] S102: Input the health question information and the health answer information into a bidirectional encoder based on a twin sentence vector transformer for preliminary encoding, and obtain the question intermediate vector and the answer intermediate vector output by the intermediate layer;

[0010] S103: Converting the question intermediate vector into a question key vector controlled by the question keywords;

[0011] S104: Converting the answer intermediate vector into an answer key vector controlled by the answer keyword;

[0012] S105: Input the question key vector and the answer key vector into the bidirectional encoder based on the twin sentence vector transformer to continue encoding, and obtain a question target vector and an answer target vector;

[0013] S106: Train the bidirectional encoder based on the twin sentence vector transformer into a health question-answering model according to the question target vector and the answer target vector.

[0014] According to another aspect of the present invention, a health question-answering method is provided, the method comprising:

[0015] S201: Load the health question answering model belonging to the bidirectional encoder based on the twin sentence vector transformer;

[0016] S202: If health problem information input by the client is received, extracting problem keywords including health entity words from the health problem information;

[0017] S203: Inputting the health question information into the health question-answering model for preliminary encoding to obtain an intermediate vector of the question output by the intermediate layer;

[0018] S204: Converting the question intermediate vector into a question key vector controlled by the question keywords;

[0019] S205: Inputting the question key vector into the health question-answering model to continue encoding and obtain the question target vector;

[0020] S206: Recalling the health answer information used to answer the health question information based on the question key vector and a preset answer key vector, wherein the answer key vector is encoded from the health question and answer information and answer keywords containing health entity words in the health answer information;

[0021] S207: Pushing the health answer information for answering the health question information to the client for display.

[0022] According to another aspect of the present invention, an electronic device is provided, comprising:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program and a health question and answer model that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the training method of the health question and answer model or the health question and answer method described in any embodiment of the present invention.

[0026] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores a computer program and a health question and answer model. The computer program is used to enable a processor to implement the training method of the health question and answer model or the health question and answer method described in any embodiment of the present invention when executed.

[0027] In this embodiment, multiple samples are collected, each sample including health question information, health answer information, question keywords containing health entity words in the health question information, and answer keywords containing health entity words in the health answer information; the health question information and the health answer information are input into a bidirectional encoder based on a twin sentence vector converter for preliminary encoding to obtain a question intermediate vector and an answer intermediate vector output by the intermediate layer; the question intermediate vector is converted into a question key vector controlled by the question keywords; the answer intermediate vector is converted into an answer key vector controlled by the answer keywords; the question key vector and the answer key vector are input into a bidirectional encoder based on a twin sentence vector converter to continue encoding to obtain a question target vector and an answer target vector; the bidirectional encoder based on the twin sentence vector converter is trained as a health question-answering model based on the question target vector and the answer target vector. This embodiment introduces question keywords and answer keywords containing health entity words to train the bidirectional encoder based on the twin sentence vector transformer, so that the bidirectional encoder based on the twin sentence vector transformer can focus more on question keywords and answer keywords containing health entity words, thereby improving the adaptability of the bidirectional encoder based on the twin sentence vector transformer in the health field, thereby improving the accuracy of answer recall. Moreover, by customizing the structure of encoding question key vectors and answer key vectors, when updating and iterating health entity words and retraining the bidirectional encoder based on the twin sentence vector transformer, the amount of annotation can be effectively reduced, thereby reducing the cost of training the bidirectional encoder based on the twin sentence vector transformer and improving the efficiency of training the bidirectional encoder based on the twin sentence vector transformer.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 This is a flowchart of a method for training a health question-answering model according to the first embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of the structure of a Sentence-BERT provided according to the first embodiment of the present invention;

[0032] Figure 3 2. This is a schematic diagram of the structure of a Sentence-BERT with keyword introduction provided in Example 1 of the present invention;

[0033] Figure 4 1 is a schematic diagram of a process for generating a problem target vector according to the first embodiment of the present invention;

[0034] Figure 5 1 is a schematic diagram of the structure of an output layer provided according to the first embodiment of the present invention;

[0035] Figure 6 1 is a schematic diagram of a process for generating an answer target vector according to the first embodiment of the present invention;

[0036] Figure 7 1 is a schematic diagram of the structure of an output layer provided according to the first embodiment of the present invention;

[0037] Figure 8 This is a flow chart of a health question-and-answer method provided according to the second embodiment of the present invention;

[0038] Figure 9 2 is a schematic structural diagram of a health question-answering model training device according to a third embodiment of the present invention;

[0039] Figure 10 This is a schematic diagram of the structure of a health question-and-answer device provided according to a fourth embodiment of the present invention;

[0040] Figure 11 It is a structural diagram of an electronic device provided by Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0042] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] Example 1

[0044] Figure 1 This is a flowchart of a method for training a health question-answering model provided in the first embodiment of the present invention. This embodiment is applicable to the case where Sentence-BERT is trained as a health question-answering model by incorporating keywords containing health entity words. This method can be executed by a training device for a health question-answering model. The training device for the health question-answering model can be implemented in the form of hardware and / or software. The training device for the health question-answering model can be configured in an electronic device. Figure 1 As shown, the method includes:

[0045] Step S101: Collect multiple samples.

[0046] In this embodiment, multiple samples can be collected through data sets disclosed by a third party, website log file accumulation, etc., and each sample is a quintuple, which includes health question information, health answer information, question keywords containing health entity words in the health question information, answer keywords containing health entity words in the health answer information, and the true similarity between the health question information and the health answer information.

[0047] Among them, health problem information refers to health-related questions, such as, "Can different vitamins be taken together?"

[0048] Health answer information is health-related answers used to answer health questions, for example, "Different types of nutrients can be supplemented together if necessary, and the same type of nutrients should not be consumed in excess of the recommended amount at the same time. For example, vitamin C tablets are used to supplement vitamin C, and B vitamin tablets are used to supplement B vitamins. They belong to different types of vitamins, and these two products can be supplemented together."

[0049] Health entity words are health-related entity words, including the names of nutrients (such as vitamin D), nutritional supplements (such as protein powder), and diseases (such as osteoporosis).

[0050] Question keywords are keywords in health question information, and answer keywords are keywords in health answer information. Generally, question keywords and answer keywords contain not only one or more health entity words, but also words that express intentions, words that express emotions, etc. In some cases, the health entity words contained in question keywords and answer keywords are empty, and this embodiment does not limit this.

[0051] In a specific implementation, the sample types may include positive samples and negative samples. In positive samples, the health answer information can accurately answer the questions raised by the health problem information. In negative samples, the health answer information cannot accurately answer the questions raised by the health problem information.

[0052] When making the original copy, mutually matching health question information and health answer information can be obtained through third-party public data sets, website log file accumulation, etc. The so-called mutual matching may mean that the health answer information accurately answers the questions raised by the health question information.

[0053] For health question information and health answer information, we can use statistical methods to summarize the vocabulary of keywords, the vocabulary of standard entity words and their synonyms, and the vocabulary of stop words from the health question information and health answer information, and dynamically expand these vocabulary lists through continuous iterative optimization.

[0054] Among them, stop words are some words and symbols that are ineffective for understanding the content, such as modal particles, punctuation marks, etc.

[0055] At this time, at least one of the following pre-processing can be performed on the health question information and health answer information:

[0056] 1. Remove stop words

[0057] A regular expression is constructed using a word list of stop words, and the regular expression is matched with the health question information and the health answer information, and the stop words appearing in the health question information and the health answer information are deleted.

[0058] 2. Extract keywords

[0059] A dictionary tree is established using a vocabulary of keywords and a vocabulary of entity words. The dictionary tree is used to extract keywords containing health entity words from health question information as question keywords. The dictionary tree is used to extract keywords containing health entity words from health answer information as answer keywords.

[0060] 3. Normalization

[0061] Match the synonym vocabulary with the entity words in the keywords (question keywords, answer keywords), and map the entity words belonging to synonyms to standard entity words.

[0062] A numerical value (such as 1) that tends to the upper limit of the similarity (ie, 1) is configured between the health question information and the health answer information as the actual similarity.

[0063] At this time, the health question information and health answer information that match each other, the question keywords, the answer keywords, and the actual similarity can be constructed as positive samples.

[0064] When making negative samples, in addition to directly constructing mutually mismatched health question information and health answer information, and constructing mutually mismatched health question information and health answer information, question keywords, answer keywords, and actual similarities as negative samples, negative sampling can also be performed on positive samples to improve the efficiency of making negative samples and improve the robustness of the health question and answer model.

[0065] In one method of making negative samples, health question information and question keywords are sampled from the current positive sample, and k (k is a positive integer) health answer information and answer keywords are randomly negatively sampled from other positive samples except the current positive sample. Then, the health question information in the current positive sample does not match the health answer information in other positive samples.

[0066] A value (such as 0.03) that tends to be the lower limit of the similarity (ie, 0) between the health question information and the health answer information is configured as the actual similarity.

[0067] The health question information and question keywords from the current positive sample, the health answer information and answer keywords from other positive samples, and the actual similarity are constructed as negative samples.

[0068] In another way of making negative samples, health problem information and question keywords are sampled from the current positive sample, and a first target sample is selected for the current positive sample. The first target sample is other positive samples with the same question keywords as the current positive sample. K (k is a positive integer) health answer information and answer keywords are randomly negatively sampled from the first target sample. Then, the health problem information in the current positive sample and the health answer information in the first target sample do not match each other, and the health problem information in the current positive sample and the health problem information in the first target sample have certain similarities.

[0069] The smallest value between the first and second values ​​is taken as the actual similarity between the health question information and the health answer information, wherein the first value tends to the upper limit of similarity (i.e. 1), such as 0.95, and the second value is the specified first ratio (the first ratio is less than 1, such as 0.25) of the number of overlaps in question keywords and / or answer keywords between the current positive sample and the first target sample.

[0070] The health question information and question keywords from the current positive sample, the health answer information and answer keywords from the first target sample, and the actual similarity are constructed as negative samples.

[0071] In another way of making negative samples, health answer information and answer keywords are sampled from the current positive sample, and a second target sample is selected for the current positive sample. The second target sample is other positive samples with the same answer keywords as the current positive sample. K (k is a positive integer) health question information and question keywords are randomly negatively sampled from the second target sample. Then, the health answer information in the current positive sample and the health question information in the second target sample do not match each other, and the health answer information in the current positive sample and the health answer information in the second target sample have certain similarities.

[0072] The smallest value between the third and fourth values ​​is taken as the actual similarity between the health question information and the health answer information, where the third value tends to the upper limit value (i.e. 1), such as 0.95, and the fourth value is a specified second ratio (the second ratio is less than 1, such as 0.25) of the number of overlaps in question keywords and / or answer keywords between the current positive sample and the second target sample.

[0073] The health question information and question keywords from the second target sample, the health answer information and answer keywords from the current positive sample, and the actual similarity are constructed as negative samples.

[0074] Of course, the above-mentioned method for preparing the sample is merely an example. When implementing this embodiment, other methods for preparing the sample can be set according to actual circumstances, and this embodiment does not limit this. In addition, in addition to the above-mentioned method for preparing the sample, those skilled in the art can also adopt other methods for preparing the sample according to actual needs, and this embodiment does not limit this either.

[0075] Step S102: Input the health question information and health answer information into the bidirectional encoder based on the twin sentence vector transformer for preliminary encoding to obtain the question intermediate vector and answer intermediate vector output by the intermediate layer.

[0076] like Figure 2 As shown in the figure, Sentence-BERT is a twin network based on the pre-trained BERT (Bidirectional Encoder Representation from Transformers, a bidirectional encoder from an encoding-decoding architecture). BERT is built based on a multi-layer bidirectional Transformer (encoding-decoding architecture).

[0077] In Sentence-BERT, a sentence pair (SentenceA, SentenceB) is input into two BERTs with shared parameters. All word vectors of the BERT output sentence are passed to the Pooling layer for pooling operations (such as average pooling, averaging all word vectors in the dimension of sentence length). The sentence vector (u, v) representation of each sentence is obtained, and the similarity (Sim(u, v)) is calculated using functions such as cosine value.

[0078] In this embodiment, if Figure 3 As shown, the health question information and health answer information are input into the two channels of Sentence-BERT for preliminary encoding, and the hidden layer output of the middle layer of Sentence-BERT is obtained, that is, the vector encoding the health question information is used as the question middle vector H_question_middle∈R b×max_seq×hidden , and the vector encoding the health answer information is 1368 as the answer intermediate vector H_answer_middle∈R b×max_seq×hidden , where b is the batch size, max_seq is the maximum sequence length, and hidden is the hidden layer length.

[0079] The middle layer is any layer of the two BERTs whose parameters are shared by Sentence-BERT. For example, if the two BERTs whose parameters are shared by Sentence-BERT have twelve Transformer layers, the middle layer can be the sixth Transformer layer, the middle layer can also be the twelfth Transformer layer, and so on.

[0080] In the specific implementation, Figure 4 As shown, the health problem information is input into a preset first tokenizer (such as BERT-tokenlizer) for processing to obtain a first digital sequence input_ids, a first segmentation sequence segment_ids and a first mask sequence mask_ids to meet the input specifications of BERT.

[0081] like Figure 6 As shown, the health answer information is input into the preset first tokenizer (such as BERT-tokenlizer) for processing to obtain the second digital sequence input_ids, the second segmentation sequence segment_ids and the second mask sequence mask_ids to meet the input specifications of BERT.

[0082] like Figure 4 and Figure 6 As shown, the first digital sequence input_ids, the first segmentation sequence segment_ids and the first mask sequence mask_ids, as well as the second digital sequence input_ids, the second segmentation sequence segment_ids and the second mask sequence mask_ids are input into two channels of the bidirectional encoder (Sentence-BERT) based on the twin sentence vector transformer for preliminary encoding to obtain the question intermediate vector H_question_middle and the answer intermediate vector H_answer_middle output by the intermediate layer.

[0083] Step S103: Convert the question intermediate vector into a question key vector controlled by question keywords.

[0084] In this embodiment, if Figure 3 As shown, the question middle vector H_question_middle can be encoded with reference to the question keywords, increasing the importance of the question keywords to the question middle vector H_question_middle, thereby converting the question middle vector H_question_middle into a question key vector H_q controlled by the question keywords.

[0085] In one embodiment of the present invention, step S103 may include the following steps:

[0086] Step S1031: Convert the question intermediate vector into a question query vector, a question index vector and a question content vector.

[0087] In this embodiment, if Figure 4 As shown, the question middle vector H_question_middle is encoded and converted into a question query vector Q_question, a question index vector K_question and a question content vector V_question to meet the input specifications of the first self-attention layer.

[0088] In the specific implementation, the question intermediate vector H_question_middle is linearly transformed to generate the first query matrix (Q matrix, that is, a matrix composed of a group of Qurey sets), the first index matrix (K matrix, that is, a matrix composed of a group of Key sets), and the first content matrix (V matrix, a matrix composed of a group of Value sets).

[0089] Multiply the question middle vector H_question_middle by the first query matrix (Q matrix) to obtain the question query vector Q_question.

[0090] Multiply the question middle vector H_question_middle by the first index matrix (K matrix) to obtain the question index vector K_question.

[0091] Multiply the question middle vector H_question_middle by the first content matrix (V matrix) to obtain the question content vector V_question.

[0092] Step S1032: Encode the question keywords to obtain a first word index vector and a first word content vector.

[0093] In this embodiment, if Figure 4 As shown, the question keywords are encoded to obtain the first word index vector K_qe and the first word content vector V_qe, so that the first word index vector K_qe and the first word content vector V_qe are in the same vector space as the question index vector K_question and the question content vector V_question for fusion.

[0094] In the specific implementation, Figure 4 As shown, the question keyword is input into the preset second word segmenter for processing to obtain a third digital sequence keyword_input_ids, a third segmentation sequence keyword_segment_ids and a third mask sequence keyword_mask_ids.

[0095] Among them, the second word segmenter is a word segmenter trained for keywords containing health entity words, for example, a word segmenter trained by adding a vocabulary of entity words to BERT-tokenlizer.

[0096] Input the third digital sequence keyword_input_ids, the third fragment sequence keyword_segment_ids and the third mask sequence keyword_mask_ids into the preset word vector matrix for processing to obtain the hidden layer output question key vector H_q_keyword'∈R b×max_seq×hidden .

[0097] Input the question key vector H_q_keyword' into the preset question index matrix W_K_qe∈R hidden×hidden , and obtain the first word index vector K_qe.

[0098] Input the first word index vector K_qe into the preset question content matrix V_K_qe∈R hidden×hidden Processed in , we get the first word content vector V_qe∈R b×max_seq×hidden .

[0099] Step S1033: Merge the question index vector and the first word index vector into a first target index vector.

[0100] In this embodiment, if Figure 4 As shown, the question index vector K_question and the first word index vector K_qe can be fused by linear or nonlinear addition, multiplication or splicing to obtain the first target index vector Kq.

[0101] In one example, the first word index vector K_qe can be multiplied by a preset first hyperparameter to obtain a first weighted index vector, and the first weighted index vector and the question index vector K_question can be concatenated to form a first target index vector Kq∈R b×2*max_seq×hidden , then, the fusion process is expressed as Kq = [λqe·K_qe, K_question], where λqe is the first hyperparameter, λqe∈[0,1].

[0102] Step S1034: Merge the question content vector and the first word content vector into a first target content vector.

[0103] In this embodiment, if Figure 4 As shown, the question content vector V_question and the first word content vector V_qe can be fused by linear or nonlinear addition, multiplication or splicing to obtain the first target content vector Vq.

[0104] In one example, the first word content vector V_qe can be multiplied by a preset first hyperparameter to obtain a first weighted content vector, and the first weighted content vector and the question content vector V_question can be concatenated to form a first target content vector Vq∈R b×2*max_seq×hidden , then, the fusion process is expressed as Vq = [λqe·V_qe, V_question], where λqe is the first hyperparameter, λqe∈[0,1].

[0105] Step S1035: Input the question query vector, the first target index vector, and the first target content vector into a preset first self-attention layer for processing to obtain a question candidate vector.

[0106] In this embodiment, if Figure 4 As shown, the question query vector Q_question, the first target index vector Kq and the first target content vector Vq are input into the preset first self-attention layer for processing to obtain the question candidate vector A_q.

[0107] The first self-attention layer belongs to the self-attention mechanism, also known as the internal attention mechanism, which is a type of attention mechanism that associates different positions of a single sequence to calculate the representation of the same sequence.

[0108] Use the KQV model to explain the Attention mechanism:

[0109] Assuming the input is Q (Query), the context is stored in the memory in the form of key-value pairs (Key / Value, K / V, also known as index / content), then the attention mechanism is a mapping function from Query to a series of key-value pairs.

[0110] The attention mechanism essentially assigns a weight coefficient to each element in a sequence, known as soft addressing. If each element in the sequence is stored as a key-value pair, the attention mechanism performs addressing by calculating the similarity between Q and K. The calculated similarity between Q and K reflects the importance of the retrieved V value, i.e., the weight. The weighted sum of these values ​​yields the attention value.

[0111] The special point of SelfAttention in the KQV model is that Q = K = V, which is calculated by multiplying the similarity between texts by the text itself.

[0112] More specifically, the first self-attention layer processes as follows:

[0113]

[0114] Among them, A_q is the question candidate vector, Q is the question query vector Q_question, K is the first target index vector Kq, V is the first target content vector Vq, and d is the dimension of the feature.

[0115] Step S1036: extract features from the question candidate vector to obtain the question key vector.

[0116] In this embodiment, if Figure 4 As shown in Figure 1, the question candidate vector A_q is input into the output layer to extract high-dimensional features and obtain the question key vector H_q.

[0117] In an example of an output layer, such as Figure 5 As shown in the figure, the output layer includes three fully connected layers (FC), namely the first problem fully connected layer Dense1′, the second problem fully connected layer Dense2′, and the third problem fully connected layer Dense3′.

[0118] In this example, the question candidate vector A_q is input into the preset first question fully connected layer Dense1′ and mapped to the first question feature vector, and the first question feature vector is activated using activation functions such as Sigmoid (S-type growth function, converting any numerical value into probability (0-1)).

[0119] If the activation of the first problem feature vector is completed, the first problem feature vector is input into the preset second problem fully connected layer Dense2′ and mapped into the second problem feature vector, and the second problem feature vector is activated using activation functions such as ReLU (Rectified Linear Unit).

[0120] If the activation of the second question feature vector is completed, the second question feature vector is input into the preset third question fully connected layer Dense3′ and mapped into the third question feature vector. The third question feature vector is activated using an activation function such as Sigmoid to obtain the question key vector H_q.

[0121] In this example, the process of extracting the question key vector H_q is expressed as follows:

[0122] H_q=Sigmoid(Dense3′(ReLU(Dense2′(Sigmoid(Dense1′(A_q))))))

[0123] Among them, Dense1′ is the fully connected layer for the first problem, Dense2′ is the fully connected layer for the second problem, and Dense3′ is the fully connected layer for the third problem. Sigmoid and ReLu are both activation functions.

[0124] Step S104: Convert the intermediate answer vector into a key answer vector controlled by the answer keyword.

[0125] In this embodiment, if Figure 3 As shown, the answer middle vector H_answer_middle can be encoded with reference to the answer keywords, increasing the importance of the answer keywords to the answer middle vector H_answer_middle, thereby converting the answer middle vector H_answer_middle into an answer key vector H_a controlled by the answer keywords.

[0126] In one embodiment of the present invention, step S104 may include the following steps:

[0127] Step S1041: Convert the answer intermediate vector into an answer query vector, an answer index vector, and an answer content vector.

[0128] In this embodiment, if Figure 6 As shown, the answer intermediate vector H_answer_middle is encoded and converted into the answer query vector Q_answer, the answer index vector K_answer, and the answer content vector V_answer to meet the input specifications of the second self-attention layer.

[0129] In the specific implementation, the answer intermediate vector H_answer_middle is linearly transformed to generate the second query matrix (Q matrix, that is, a matrix composed of a group of Qurey sets), the second index matrix (K matrix, that is, a matrix composed of a group of Key sets), and the second content matrix (V matrix, a matrix composed of a group of Value sets).

[0130] Multiply the answer middle vector H_answer_middle by the second query matrix (Q matrix) to obtain the answer query vector Q_answer.

[0131] Multiply the answer middle vector H_answer_middle by the second index matrix (K matrix) to obtain the answer index vector K_answer.

[0132] Multiply the answer middle vector H_answer_middle by the second content matrix (V matrix) to obtain the answer content vector V_answer.

[0133] Step S1042: Encode the answer keywords to obtain a second word index vector and a second word content vector.

[0134] In this embodiment, if Figure 6As shown, the answer keywords are encoded to obtain the second word index vector K_ae and the second word content vector V_ae, so that the second word index vector K_ae and the second word content vector V_ae are in the same vector space as the answer query vector Q_answer, the answer index vector K_answer, and the answer content vector V_answer for fusion.

[0135] In the specific implementation, Figure 6 As shown, the answer keyword is input into the preset second word segmenter for processing to obtain a fourth digital sequence keyword_input_ids, a fourth segmentation sequence keyword_segment_ids and a fourth mask sequence keyword_mask_ids.

[0136] Input the fourth digital sequence keyword_input_ids, the fourth fragment sequence keyword_segment_ids and the fourth mask sequence keyword_mask_ids into the preset word vector matrix for processing to obtain the answer key vector H_a_keyword'∈R b×max_seq×hidden .

[0137] Input the answer key vector H_a_keyword' into the preset answer index matrix W_K_ae∈R hidden×hidden , and obtain the second word index vector K_ae.

[0138] Input the second word index vector K_ae into the preset answer content matrix V_K_ae∈R hidden×hidden Processed in , the second word content vector V_ae∈R is obtained b×max_seq×hidden .

[0139] Step S1043: Merge the answer index vector and the second word index vector into a second target index vector.

[0140] In this embodiment, if Figure 6 As shown, the answer index vector K_answer and the second word index vector K_ae can be fused by linear or nonlinear addition, multiplication or splicing to obtain the second target index vector Ka.

[0141] In one example, the second word index vector K_ae is multiplied by a preset second hyperparameter to obtain a second weighted index vector, and the second weighted index vector and the answer index vector K_answer are concatenated to form a second target index vector Ka∈

[0142] R b×2*max_seq×hidden, then, the fusion process is expressed as Ka = [λae·K_ae, K_answer], where λae is the second hyperparameter, λae∈[0, 1].

[0143] Step S1044: Merge the answer content vector and the second word content vector into a second target content vector.

[0144] In this embodiment, if Figure 6 As shown, the answer content vector V_answer and the second word content vector V_ae can be fused by linear or nonlinear addition, multiplication or splicing to obtain the second target content vector Va.

[0145] In one example, the second word content vector V_ae can be multiplied by a preset second hyperparameter to obtain a second weighted content vector, and the second weighted content vector and the answer content vector V_answer can be concatenated to form a second target content vector Va∈R b×2*max_seq×hidden , then, the fusion process is expressed as Va = [λae·V_ae, V_answer], where λae is the second hyperparameter, λae∈[0,1].

[0146] Step S1045: Input the answer query vector, the second target index vector, and the second target content vector into a preset second self-attention layer for processing to obtain an answer candidate vector.

[0147] In this embodiment, if Figure 6 As shown, the answer query vector Q_answer, the second target index vector Ka and the second target content vector Va are input into the preset second self-attention layer for processing to obtain the answer candidate vector A_a.

[0148] The second self-attention layer belongs to the self-attention mechanism. The processing process of the second self-attention layer is as follows:

[0149]

[0150] Among them, A_a is the answer candidate vector, Q is the answer query vector Q_answer, K is the second target index vector Ka, V is the second target content vector Va, and d is the dimension of the feature.

[0151] Step S1046: extract features from the answer candidate vector to obtain the answer key vector.

[0152] In this embodiment, if Figure 6 As shown, the answer candidate vector A_a is input into the output layer to extract high-dimensional features and obtain the answer key vector H_a.

[0153] In an example of an output layer, such as Figure 7 As shown in the figure, the output layer includes three fully connected layers, namely the first answer fully connected layer Dense 1, the second answer fully connected layer Dense 2, and the third answer fully connected layer Dense 3.

[0154] In this example, the answer candidate vector A_a is input into the preset first answer fully connected layer Dense1 and mapped to the first answer feature vector, and the first answer feature vector is activated using an activation function such as Sigmoid.

[0155] If the activation of the first answer feature vector is completed, the first answer feature vector is input into the preset second answer fully connected layer Dense2 and mapped into the second answer feature vector, and the second answer feature vector is activated using activation functions such as ReLU.

[0156] If the activation of the second answer feature vector is completed, the second answer feature vector is input into the preset third answer fully connected layer Dense3 and mapped to the third answer feature vector. The third answer feature vector is activated using activation functions such as Sigmoid to obtain the answer key vector H_a.

[0157] In this example, the process of extracting the answer key vector H_a is expressed as follows:

[0158] H_a=Sigmoid(Dense3(ReLU(Dense2(Sigmoid(Dense1(A_a))))))

[0159] Among them, Dense 1 is the first answer fully connected layer, Dense 2 is the second answer fully connected layer, Dense 3 is the third answer fully connected layer, Sigmoid and ReLu are both activation functions.

[0160] Step S105: Input the question key vector and the answer key vector into the bidirectional encoder based on the twin sentence vector transformer to continue encoding and obtain the question target vector and the answer target vector.

[0161] like Figure 3 As shown, on the one hand, the question key vector H_q is re-input into Sentence-BERT to continue to complete the remaining encoding, thereby outputting the question target vector H1q. On the other hand, the answer key vector H_a is re-input into Sentence-BERT to continue to complete the remaining encoding, thereby outputting the answer target vector Hla.

[0162] Step S106: Train the bidirectional encoder based on the twin sentence vector transformer into a healthy question-answering model according to the question target vector and the answer target vector.

[0163] In this embodiment, Sentence-BERT can be trained based on the question target vector H1q and the answer target vector H1a. When the training is completed, Sentence-BERT is recorded as a health question and answer model, which is used to infer the similarity between health question information and health answer information.

[0164] In one embodiment of the present invention, step S106 may include the following steps:

[0165] Step S1061: Calculate the predicted similarity between the health question information and the health answer information based on the question target vector and the answer target vector.

[0166] In this embodiment, if Figure 3 As shown, the question target vector H1q is compared with the answer target vector H1a to infer the degree of matching between the health question information and the health answer information, which is recorded as the predicted similarity Sim(H1q, H1a).

[0167] For example, the cosine value between the question target vector H1q and the answer target vector H1a can be calculated as follows:

[0168]

[0169] Among them, Scos is the cosine value, and |||| is the modulo operation.

[0170] The cosine value is activated using activation functions such as Sigmoid to obtain the predicted similarity between the health question information and the health answer information.

[0171] When using Sigmoid activation, the predicted similarity is expressed as follows:

[0172]

[0173] Among them, Scos′ is the predicted similarity and Scos is the cosine value.

[0174] Of course, the above-described method for calculating predicted similarity is merely an example. When implementing this embodiment, other methods for calculating predicted similarity may be used according to actual circumstances, such as cosine value, Euclidean distance, etc., and this embodiment does not limit this. Furthermore, in addition to the above-described method for calculating predicted similarity, those skilled in the art may also use other methods for calculating predicted similarity according to actual needs, and this embodiment does not limit this.

[0175] Step S1062: Calculate the loss value based on the predicted similarity and the true similarity.

[0176] For the same pair of health question information and health answer information, there is both a standardized similarity (i.e., true similarity) and an inferred similarity (i.e., predicted similarity). At this time, the true similarity and the predicted similarity can be substituted into the preset loss function to calculate the difference between the predicted similarity and the true similarity at a certain point, which is recorded as the loss value LOSS.

[0177] Exemplarily, the mean square error (MSE) between the predicted similarity and the true similarity is calculated as the loss value, expressed as Lmse=||Scos'-S||2, where Lmse is the loss value, Scos' is the predicted similarity, S is the true similarity, and ||||2 is the mean square error.

[0178] Furthermore, the mean square error is also called quadratic loss, L2 loss (MeanSquareError, QuadraticLoss, L2Loss), and MSE is the sum of the squares of the distances between the true similarity and the predicted similarity.

[0179] Of course, the above loss function is only an example. When implementing this embodiment, other loss functions may be set according to actual conditions, such as the mean absolute error (MAE), etc., and this embodiment does not limit this. In addition, in addition to the above loss function, those skilled in the art may also use other loss functions according to actual needs, and this embodiment does not limit this.

[0180] Step S1063: Update the bidirectional encoder based on the twin sentence vector transformer according to the loss value.

[0181] After completing forward propagation in Sentence-BERT, you can perform backpropagation on Sentence-BERT and substitute the loss value into optimization algorithms such as SGD (stochastic gradient descent) and Adam (adaptive momentum). The gradients of the parameters in Sentence-BERT are calculated and the parameters in Sentence-BERT are updated according to the gradients.

[0182] Step S1064: determine whether the preset training conditions are met; if so, execute step S1065; if not, return to execute steps S102 to S105.

[0183] Step S1065: Output the bidirectional encoder based on the twin sentence vector transformer as a health question-answering model.

[0184] In this embodiment, training conditions can be set in advance for training Sentence-BERT as conditions for stopping Sentence-BERT training. For example, the number of iterations reaches a threshold, the amplitude of the loss value changes for multiple consecutive times is less than a certain threshold, etc. In each round of iterative training, it is determined whether the training conditions are met.

[0185] If the training conditions are met, the training is considered complete and Sentence-BERT (including parameters) is output as Sentence-BERT.

[0186] If the training conditions are not met, the next round of iterative training can be entered, and steps S102 to S105 can be executed again, and the iterative training can be repeated until the training conditions are met.

[0187] In this embodiment, multiple samples are collected, each sample including health question information, health answer information, question keywords containing health entity words in the health question information, and answer keywords containing health entity words in the health answer information; the health question information and the health answer information are input into a bidirectional encoder based on a twin sentence vector converter for preliminary encoding to obtain a question intermediate vector and an answer intermediate vector output by the intermediate layer; the question intermediate vector is converted into a question key vector controlled by the question keywords; the answer intermediate vector is converted into an answer key vector controlled by the answer keywords; the question key vector and the answer key vector are input into a bidirectional encoder based on a twin sentence vector converter to continue encoding to obtain a question target vector and an answer target vector; the bidirectional encoder based on the twin sentence vector converter is trained as a health question-answering model based on the question target vector and the answer target vector. This embodiment introduces question keywords and answer keywords containing health entity words to train the bidirectional encoder based on the twin sentence vector transformer, so that the bidirectional encoder based on the twin sentence vector transformer can focus more on question keywords and answer keywords containing health entity words, thereby improving the adaptability of the bidirectional encoder based on the twin sentence vector transformer in the health field, thereby improving the accuracy of answer recall. Moreover, by customizing the structure of encoding question key vectors and answer key vectors, when updating and iterating health entity words and retraining the bidirectional encoder based on the twin sentence vector transformer, the amount of annotation can be effectively reduced, thereby reducing the cost of training the bidirectional encoder based on the twin sentence vector transformer and improving the efficiency of training the bidirectional encoder based on the twin sentence vector transformer.

[0188] Example 2

[0189] Figure 8This is a flowchart of a health question-answering method provided in the second embodiment of the present invention. This embodiment can be applied to the situation where keywords containing health entity words and recall answers are incorporated into the health question-answering model belonging to Sentence-BERT. The method can be executed by a health question-answering device. The training device of the health question-answering model can be implemented in the form of hardware and / or software. The health question-answering device can be configured in an electronic device. Figure 8 As shown, the method includes:

[0190] Step S201: Load the health question-answering model belonging to the bidirectional encoder based on the twin sentence vector transformer.

[0191] In practical applications, the training method of the health question-answering model described in any embodiment of the present invention can be applied to train Sentence-BERT into a health question-answering model. When the performance of the health question-answering model passes the evaluation, it is deployed online for operation. At this time, the health question-answering model can be loaded into the memory for operation.

[0192] Step S202: If health problem information input by the client is received, problem keywords including health entity words are extracted from the health problem information.

[0193] In this embodiment, the user can log in to the client using an account number, password, etc., or can log in to the client using temporary identity information. The client can be a browser, a shopping application, an instant messaging tool, etc.

[0194] Users can enter the manual customer service system through the client and input health problem information. When receiving the health problem information, they can use the dictionary tree established by the keyword vocabulary and entity word vocabulary to extract keywords containing health entity words from the health problem information as problem keywords.

[0195] Step S203: Input the health question information into the health question-answering model for preliminary encoding to obtain the intermediate vector of the question output by the intermediate layer.

[0196] In this embodiment, the health question information is input into one of the channels of the health question-answering model for preliminary encoding to obtain the hidden layer output of the middle layer of the health question-answering model, that is, the vector encoding the health question information, as the question intermediate vector.

[0197] In a specific implementation, the health problem information is input into a preset first tokenizer (such as BERT-tokenlizer) for processing to obtain a first digital sequence, a first fragmentation sequence, and a first mask sequence to meet the BERT input specification.

[0198] The first digital sequence, the first fragment sequence, and the first mask sequence are input into one of the channels of the health question-answering model for preliminary encoding to obtain the intermediate vector of the question output by the intermediate layer.

[0199] Step S204: Convert the question intermediate vector into a question key vector controlled by the question keywords.

[0200] In this embodiment, the question intermediate vector may be encoded with reference to the question keywords, and the importance of the question keywords to the question intermediate vector may be increased, thereby converting the question intermediate vector into a question key vector controlled by the question keywords.

[0201] In one embodiment of the present invention, step S204 may include the following steps:

[0202] Step S2041: Convert the question intermediate vector into a question query vector, a question index vector and a question content vector.

[0203] In a specific implementation, a first query matrix, a first index matrix, and a first content matrix are generated for the question intermediate vector; the question intermediate vector is multiplied with the first query matrix, the first index matrix, and the first content matrix respectively to obtain a question query vector, a question index vector, and a question content vector.

[0204] Step S2042: Encode the question keywords to obtain a first word index vector and a first word content vector.

[0205] In the specific implementation, the question keywords are input into the preset second word segmenter for processing to obtain a third digital sequence, a third fragmentation sequence and a third mask sequence; the third digital sequence, the third fragmentation sequence and the third mask sequence are input into the preset word vector matrix for processing to obtain the question key vector; the question key vector is input into the preset question index matrix for processing to obtain the first word index vector; the first word index vector is input into the preset question content matrix for processing to obtain the first word content vector.

[0206] Step S2043: Merge the question index vector and the first word index vector into a first target index vector.

[0207] In a specific implementation, the first word index vector is multiplied by a preset first hyperparameter to obtain a first weighted index vector; and the first weighted index vector and the question index vector are concatenated into a first target index vector.

[0208] Step S2044: Merge the question content vector and the first word content vector into a first target content vector.

[0209] In a specific implementation, the first word content vector is multiplied by a preset first hyperparameter to obtain a first weighted content vector; and the first weighted content vector and the question content vector are concatenated into a first target content vector.

[0210] Step S2045: Input the question query vector, the first target index vector, and the first target content vector into a preset first self-attention layer for processing to obtain a question candidate vector.

[0211] Step S2046: extract features from the question candidate vector to obtain the question key vector.

[0212] In a specific implementation, the question candidate vector is input into the preset first question fully connected layer and mapped into the first question feature vector; the first question feature vector is activated; if the activation of the first question feature vector is completed, the first question feature vector is input into the preset second question fully connected layer and mapped into the second question feature vector; the second question feature vector is activated; if the activation of the second question feature vector is completed, the second question feature vector is input into the preset third question fully connected layer and mapped into the third question feature vector; the third question feature vector is activated to obtain the question key vector.

[0213] Step S205: Input the question key vector into the health question-answering model to continue encoding and obtain the question target vector.

[0214] The question key vector is re-input into the health question answering model to continue the remaining encoding, thereby outputting the question target vector.

[0215] Step S206: Recall health answer information for answering the health question information based on the question key vector and the preset answer key vector.

[0216] In practical applications, the training method of the health question and answer model of any embodiment of the present invention can be applied to train Sentence-BERT into a health question and answer model. When the training is completed, the answer keywords containing health entity words in the health question and answer information and the health answer information have been encoded into answer key vectors, that is, the answer key vectors are encoded from the answer keywords containing health entity words in the health question and answer information and the health answer information.

[0217] Then, the health answer information used to answer the current health question information is recalled from all the health answer information according to the question target vector and the answer target vector.

[0218] In a specific implementation, the similarity between the health question information and the health answer information is calculated based on the question key vector and the preset answer key vector.

[0219] For example, the cosine value between the question target vector and the answer target vector can be calculated, and the cosine value can be activated using an activation function such as Sigmoid to obtain the similarity between the health question information and the health answer information.

[0220] Recall multiple pieces of health answer information with the highest similarity as health answer information for answering health question information.

[0221] Step S207: Push the health answer information used to answer the health question information to the client for display.

[0222] The health answer information used to answer health question information can be sorted according to similarity, and the health answer information used to answer the current health question information can be pushed to the client. The client displays the health answer information used to answer the current health question information for the user to browse.

[0223] In the embodiment of the present invention, since the processes of training the health question and answer model, generating question key vectors, generating answer key vectors, and calculating the similarity between health question information and health answer information are basically similar to the application of Example 1, the description is relatively simple. For relevant details, please refer to the partial description of Example 1. This embodiment will not be described in detail here.

[0224] In this embodiment, a health question-answering model belonging to a bidirectional encoder based on a twin sentence vector transformer is loaded; if health question information input by a client is received, question keywords including health entity words are extracted from the health question information; the health question information is input into the health question-answering model for preliminary encoding to obtain a question intermediate vector output by the intermediate layer; the question intermediate vector is converted into a question key vector controlled by the question keywords; the question key vector is input into the health question-answering model to continue encoding to obtain a question target vector; based on the question key vector and the preset answer key vector, health answer information used to answer the health question information is recalled, the answer key vector being encoded from the health question-answering information and the answer keywords containing health entity words in the health answer information; the health answer information used to answer the health question information is pushed to the client for display. This embodiment introduces question keywords and answer keywords containing health entity words to focus the attention of the health question-answering model more on question keywords and answer keywords containing health entity words, thereby improving the adaptability of the bidirectional encoder based on the twin sentence vector transformer in the health field, thereby improving the accuracy of answer recall.

[0225] In an experiment, without incorporating keywords containing health entity words (i.e., question keywords, answer keywords), the ten health answer information with the highest ranking were obtained, with a recall rate of about 75%. Among them, 40% of the health question information that was not recalled was because the keywords containing health entity words could not be identified. After incorporating keywords containing health entity words, the ten health answer information with the highest ranking were obtained, and the recall rate increased by 5.5%.

[0226] Example 3

[0227] Figure 9 This is a structural diagram of a training device for a health question-answering model provided in Example 3 of the present invention. Figure 9 As shown, the device includes:

[0228] A sample collection module 901 is configured to collect a plurality of samples, each of which includes health question information and health answer information, wherein the health question information includes question keywords containing health entity words and the health answer information includes answer keywords containing health entity words;

[0229] A preliminary encoding module 902 is configured to input the health question information and the health answer information into a bidirectional encoder based on a twin sentence vector transformer for preliminary encoding, thereby obtaining an intermediate vector for the question and an intermediate vector for the answer outputted by the intermediate layer;

[0230] A question key vector conversion module 903 is used to convert the question intermediate vector into a question key vector controlled by the question keywords;

[0231] An answer key vector conversion module 904 is used to convert the answer intermediate vector into an answer key vector controlled by the answer keyword;

[0232] A further encoding module 905 is configured to input the question key vector and the answer key vector into the bidirectional encoder based on the twin sentence vector transformer to continue encoding, thereby obtaining a question target vector and an answer target vector.

[0233] The health question-answering model training module 906 is used to train the bidirectional encoder based on the twin sentence vector transformer into a health question-answering model according to the question target vector and the answer target vector.

[0234] The training device for the health question and answer model provided in an embodiment of the present invention can execute the training method for the health question and answer model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the training method for the health question and answer model.

[0235] Example 4

[0236] Figure 10This is a structural diagram of a health question-answering device provided by the fourth embodiment of the present invention. Figure 10 As shown, the device includes:

[0237] A health question-answering model loading module 1001 is used to load a health question-answering model belonging to a bidirectional encoder based on a twin sentence vector transformer;

[0238] The question keyword extraction module 1002 is configured to extract question keywords including health entity words from the health question information upon receiving health question information input by the client;

[0239] A preliminary encoding module 1003 is used to input the health question information into the health question-answering model for preliminary encoding to obtain an intermediate vector of the question output by the intermediate layer;

[0240] A question key vector conversion module 1004 is used to convert the question intermediate vector into a question key vector controlled by the question keywords;

[0241] A further encoding module 1005 is configured to input the question key vector into the health question-answering model to continue encoding and obtain a question target vector;

[0242] A health answer information recall module 1006 is configured to recall the health answer information used to answer the health question information based on the question key vector and a preset answer key vector, wherein the answer key vector is encoded from the health question and answer information and answer keywords containing health entity words in the health answer information;

[0243] The health answer information push module 1007 is used to push the health answer information used to answer the health question information to the client for display.

[0244] The health question-and-answer device provided in the embodiment of the present invention can execute the health question-and-answer method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the health question-and-answer method.

[0245] Example 5

[0246] Figure 11A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0247] like Figure 11 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor and the health question-and-answer model described in any embodiment of the present invention. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13 and the health question-and-answer model described in any embodiment of the present invention. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12 and RAM 13 are connected to each other via a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0248] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0249] The processor 11 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the training method of the health question-answering model or the health question-answering method.

[0250] In some embodiments, the training method of the health question and answer model or the health question and answer method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the training method of the health question and answer model or the health question and answer method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the training method of the health question and answer model or the health question and answer method in any other appropriate manner (for example, by means of firmware).

[0251] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0252] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0253] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0254] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0255] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0256] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0257] Example 6

[0258] An embodiment of the present invention also provides a computer program product, which includes a computer program and the health question and answer model described in any embodiment of the present invention. When the computer program is executed by a processor, it implements the training method or health question and answer method of the health question and answer model provided in any embodiment of the present invention.

[0259] The computer program product may be implemented by writing computer program code for performing the operations of the present invention in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0260] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0261] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A training method for a health question-answering model, characterized in that: The method comprises: S101: Collect multiple samples, each of which includes health question information and health answer information, wherein the health question information contains question keywords of health entity words, and the health answer information contains answer keywords of health entity words; S102: Input the health question information and the health answer information into a bidirectional encoder based on a twin sentence vector transformer for preliminary encoding, and obtain the question intermediate vector and the answer intermediate vector output by the intermediate layer; S103: Converting the question intermediate vector into a question key vector controlled by the question keywords; S104: Converting the answer intermediate vector into an answer key vector controlled by the answer keyword; S105: Input the question key vector and the answer key vector into the bidirectional encoder based on the twin sentence vector transformer to continue encoding, and obtain a question target vector and an answer target vector; S106: Training the bidirectional encoder based on the twin sentence vector transformer into a health question answering model according to the question target vector and the answer target vector; Wherein, the step S103 includes: Converting the question intermediate vector into a question query vector, a question index vector and a question content vector; Encode the question keywords to obtain a first word index vector and a first word content vector; Merging the question index vector and the first word index vector into a first target index vector; Merging the question content vector and the first word content vector into a first target content vector; Inputting the question query vector, the first target index vector, and the first target content vector into a preset first self-attention layer for processing to obtain a question candidate vector; Extract features from the question candidate vector to obtain a question key vector; The step S104 includes: Converting the answer intermediate vector into an answer query vector, an answer index vector, and an answer content vector; Encode the answer keywords to obtain a second word index vector and a second word content vector; Merging the answer index vector and the second word index vector into a second target index vector; Merging the answer content vector and the second word content vector into a second target content vector; Inputting the answer query vector, the second target index vector, and the second target content vector into a preset second self-attention layer for processing to obtain an answer candidate vector; Features are extracted from the answer candidate vector to obtain the answer key vector.

2. The method according to claim 1, characterized in that The step S102 includes: Inputting the health problem information into a preset first word segmenter for processing to obtain a first digital sequence, a first fragment sequence, and a first mask sequence; Inputting the health answer information into a preset first word segmenter for processing to obtain a second digital sequence, a second fragment sequence, and a second mask sequence; The first digital sequence, the first segmentation sequence and the first mask sequence, as well as the second digital sequence, the second segmentation sequence and the second mask sequence are input into a bidirectional encoder based on a twin sentence vector transformer for preliminary encoding to obtain the question intermediate vector and the answer intermediate vector output by the intermediate layer.

3. The method according to claim 1, characterized in that Convert the intermediate question vector into a question query vector, a question index vector, and a question content vector, including: Generating a first query matrix, a first index matrix, and a first content matrix for the question intermediate vector; The question intermediate vector is multiplied by the first query matrix, the first index matrix, and the first content matrix respectively to obtain a question query vector, a question index vector, and a question content vector.

4. The method according to claim 1, wherein Encode the question keywords to obtain the first word index vector and the first word content vector, including: Input the question keyword into a preset second word segmenter for processing to obtain a third digital sequence, a third fragment sequence and a third mask sequence; Inputting the third digital sequence, the third fragment sequence, and the third mask sequence into a preset word vector matrix for processing to obtain a question key vector; Input the question key vector into a preset question index matrix for processing to obtain a first word index vector; The first word index vector is input into a preset question content matrix for processing to obtain a first word content vector.

5. The method according to claim 1, wherein The question index vector and the first word index vector are merged into the first target index vector, including: Multiplying the first word index vector by a preset first hyperparameter to obtain a first weighted index vector; splicing the first weight adjustment index vector and the problem index vector into a first target index vector; The question content vector and the first word content vector are fused into a first target content vector, including: Multiplying the first word content vector by a preset first hyperparameter to obtain a first weighted content vector; The first weighted content vector and the question content vector are concatenated into a first target content vector.

6. The method according to claim 1, wherein Extract features from the question candidate vector to obtain the question key vector, including: Inputting the question candidate vector into a preset first question fully connected layer and mapping it into a first question feature vector; activating the first problem feature vector; If the activation of the first question feature vector is completed, the first question feature vector is input into a preset second question fully connected layer and mapped into a second question feature vector; activating the second question feature vector; If the activation of the second question feature vector is completed, the second question feature vector is input into a preset third question fully connected layer and mapped into a third question feature vector; The third question feature vector is activated to obtain a question key vector.

7. The method according to claim 1, characterized in that Convert the answer intermediate vector into the answer query vector, answer index vector, and answer content vector, including: generating a second query matrix, a second index matrix, and a second content matrix for the intermediate answer vector; Multiply the answer intermediate vector with the second query matrix, the second index matrix, and the second content matrix respectively to obtain an answer query vector, an answer index vector, and an answer content vector.

8. The method according to claim 1, characterized in that Encode the answer keywords to obtain the second word index vector and the second word content vector, including: Inputting the answer keyword into a preset second word segmenter for processing to obtain a fourth digital sequence, a fourth fragment sequence, and a fourth mask sequence; Inputting the fourth digital sequence, the fourth fragmentation sequence, and the fourth mask sequence into a preset word vector matrix for processing to obtain an answer key vector; Inputting the answer key vector into a preset answer index matrix for processing to obtain a second word index vector; The second word index vector is input into a preset answer content matrix for processing to obtain a second word content vector.

9. The method according to claim 1, characterized in that Merging the answer index vector and the second word index vector into a second target index vector includes: Multiplying the second word index vector by a preset second hyperparameter to obtain a second weighted index vector; Concatenate the second weighted index vector and the answer index vector into a second target index vector; The step of fusing the answer content vector and the second word content vector into a second target content vector includes: Multiplying the second word content vector by a preset second hyperparameter to obtain a second weighted content vector; The second weighted content vector and the answer content vector are concatenated into a second target content vector.

10. The method according to claim 1, characterized in that Extract features from the answer candidate vector to obtain the answer key vector, including: Input the answer candidate vector into a preset first answer fully connected layer and map it into a first answer feature vector; Activating the first answer feature vector; If the activation of the first answer feature vector is completed, the first answer feature vector is input into a preset second answer fully connected layer and mapped into a second answer feature vector; activating the second answer feature vector; If the activation of the second answer feature vector is completed, the second answer feature vector is input into the preset third answer fully connected layer and mapped into the third answer feature vector; The third answer feature vector is activated to obtain an answer key vector.

11. The method according to any one of claims 1 to 10, characterized in that Each of said samples also includes the true similarity between the health question information and the health answer information; The step S106 includes: Calculating the predicted similarity between the health question information and the health answer information based on the question target vector and the answer target vector; Calculating a loss value based on the predicted similarity and the true similarity; Updating the bidirectional encoder based on the twin sentence vector transformer according to the loss value; Determine whether the preset training conditions are met; if so, output the bidirectional encoder based on the twin sentence vector converter as a health question and answer model; if not, return to execute the preliminary encoding of the health question information and the health answer information into the bidirectional encoder based on the twin sentence vector converter to obtain the question intermediate vector and the answer intermediate vector output by the intermediate layer.

12. The method according to claim 11, characterized in that Calculating the predicted similarity between the health question information and the health answer information based on the question target vector and the answer target vector includes: Calculating the cosine value between the question target vector and the answer target vector; activating the cosine value to obtain a predicted similarity between the health question information and the health answer information; The calculating of the loss value based on the predicted similarity and the true similarity includes: The mean square error between the predicted similarity and the true similarity is calculated as a loss value.

13. The method according to any one of claims 1 to 10, characterized in that The samples include positive samples and negative samples; S101 includes: Obtaining matching health question information and health answer information; Extracting question keywords containing health entity words from the health question information and extracting answer keywords containing health entity words from the health answer information; Configuring an actual similarity between the health question information and the health answer information that tends to an upper limit value; Constructing the health question information and the health answer information that match each other, the question keywords, the answer keywords, and the actual similarity into a positive sample; Sampling the health problem information and the problem keywords from the current positive sample; Randomly negatively sampling the health answer information and the answer keywords from the other positive samples; Configuring an actual similarity between the health question information and the health answer information that approaches a lower limit value; Constructing the health question information and the question keywords from the current positive sample, the health answer information and the answer keywords from other positive samples, and the actual similarity into a negative sample; Sampling the health problem information and the problem keywords from the current positive sample; Randomly negatively sampling the health answer information and the answer keywords from a first target sample, where the first target sample is other positive samples having the same question keywords as the positive sample; The smallest value between the first value and the second value is taken as the actual similarity between the health question information and the health answer information, the first value tends to the upper limit value, and the second value is a specified first ratio of the number of overlaps in the question keywords and / or the answer keywords between the current positive sample and the first target sample; Constructing the health question information and the question keywords from the current positive sample, the health answer information and the answer keywords from the first target sample, and the actual similarity into a negative sample; Sampling the health answer information and the answer keywords from the current positive sample; Randomly negatively sampling the health question information and the question keywords from a second target sample, where the second target sample is another positive sample having the same answer keywords as the current positive sample; The smallest value between the third value and the fourth value is taken as the actual similarity between the health question information and the health answer information, the third value tends to the upper limit value, and the fourth value is a specified second ratio of the number of overlaps in the question keywords and / or the answer keywords between the current positive sample and the second target sample; The health question information and the question keywords from the second target sample, the health answer information and the answer keywords from the current positive sample, and the actual similarity are constructed as a negative sample.

14. A health question-answering method, characterized in that: The method comprises: S201: Loading a health question-answering model trained according to the method of any one of claims 1-13, which is a bidirectional encoder based on a twin sentence vector transformer; S202: If health problem information input by the client is received, extracting problem keywords including health entity words from the health problem information; S203: Inputting the health question information into the health question-answering model for preliminary encoding to obtain an intermediate vector of the question output by the intermediate layer; S204: Converting the question intermediate vector into a question key vector controlled by the question keywords; S205: Inputting the question key vector into the health question-answering model to continue encoding and obtain the question target vector; S206: Recalling health answer information for answering the health question information based on the question key vector and a preset answer key vector, wherein the answer key vector is encoded from the health question and answer information and answer keywords containing health entity words in the health answer information; S207: Pushing the health answer information for answering the health question information to the client for display.

15. The method according to claim 14, characterized in that The S206 includes: Calculating the similarity between the health question information and the health answer information based on the question key vector and the preset answer key vector; Recall a plurality of the health answer information with the highest similarity as the health answer information for answering the health question information.

16. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program and a health question and answer model that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the training method of the health question and answer model described in any one of claims 1-13 or the health question and answer method described in any one of claims 14-15.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program and a health question-and-answer model, and the computer program is used to enable the processor to implement the training method of the health question-and-answer model described in any one of claims 1-13 or the health question-and-answer method described in any one of claims 14-15 when executed.

Images