Answer abstract extraction method and training method and device of answer abstract extraction model

Abstract

The application discloses an answer abstract extraction method and a training method and device of an answer abstract extraction model, and belongs to the technical field of artificial intelligence. The method comprises the following steps: obtaining an input question; determining a solution text corresponding to the input question based on the input question; inputting the input question and the solution text into an answer abstract extraction model to perform answer abstract extraction processing, so as to obtain answer abstract information corresponding to the input question; wherein the answer abstract extraction model is obtained by training sample questions, sample solution texts and probability distribution information corresponding to each text unit in the sample solution texts as label information, the probability distribution information is determined according to similarity information between a candidate answer and a labeled answer in the sample solution text, and is used for representing probabilities that the text units in the solution text are respectively starting positions and ending positions of the answer abstract. The application trains and optimizes the answer abstract extraction model based on the above method, and improves the accuracy and flexibility of the model in extracting the answer abstract.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an answer abstract extraction method and a training method and device of an answer abstract extraction model. BACKGROUND

[0002] With the rapid development of information technology, it has become an important demand of users to quickly and conveniently obtain information, and intelligent question answering services have received widespread attention. How to quickly and accurately provide users with the required information has become an important problem to be solved.

[0003] In order to facilitate users to quickly and accurately find the information they want, some application programs generally provide intelligent question answering services, such as intelligent customer service, intelligent consultation and other services. In related technologies, the intelligent question answering service generally inputs the question input by the user into a machine learning model, and then outputs text information related to the question input by the user and feeds back to the user. The machine learning model is generally trained with artificially annotated answers as label information, and the label value is generally a discrete numerical value, such as 0 or 1.

[0004] In related technologies, the label information used by the machine learning model during training is single, and the dependence on artificial annotation quality is strong, and the model is prone to overfitting. SUMMARY

[0005] The embodiments of the present application provide an answer abstract extraction method, a training method and device of an answer abstract extraction model, which can enrich the label information of the sample, reduce the dependence of the model on the artificial annotation quality, effectively prevent the model from overfitting, and improve the accuracy and flexibility of the model in extracting answer abstracts.

[0006] According to an aspect of an embodiment of the present application, an answer abstract extraction method is provided, which comprises:

[0007] obtaining the input question;

[0008] determining a solution text corresponding to the input question based on the input question;

[0009] inputting the input question and the solution text into an answer abstract extraction model for answer abstract extraction processing to obtain answer abstract information corresponding to the input question;

[0010] The answer abstract extraction model is a machine learning model trained by taking a sample question, a sample answer text, and probability distribution information corresponding to each text unit in the sample answer text as label information, wherein the probability distribution information is determined by similarity information between at least one candidate answer in the sample answer text and a labeled answer corresponding to the sample question, and is used to represent probabilities that the text units in the answer text are respectively start positions and end positions of predicted answer abstract information.

[0011] According to an aspect of an embodiment of the present application, a training method of an answer abstract extraction model is provided, and the method comprises:

[0012] obtaining a sample question, a sample answer text, and a labeled answer corresponding to the sample question, wherein the sample answer text comprises at least one candidate answer;

[0013] determining similarity information between the at least one candidate answer and the labeled answer;

[0014] based on the similarity information, determining probability distribution information corresponding to each text unit in the sample answer text, wherein the probability distribution information is used to represent probabilities that the text units are respectively start positions and end positions of answer abstract information;

[0015] performing machine learning training by taking the probability distribution information as label information to obtain an answer abstract extraction model, so that the answer abstract extraction model determines answer abstract information corresponding to an input question based on the input question.

[0016] According to an aspect of an embodiment of the present application, an answer abstract extraction device is provided, and the device comprises:

[0017] a question obtaining module configured to obtain the input question;

[0018] an answer text determining module configured to determine an answer text corresponding to the input question based on the input question;

[0019] an answer abstract extraction module configured to input the input question and the answer text to an answer abstract extraction model to perform answer abstract extraction processing, and obtain answer abstract information corresponding to the input question;

[0020] The answer abstract extraction model is a machine learning model trained by taking a sample question, a sample answer text, and probability distribution information corresponding to each text unit in the sample answer text as label information, wherein the probability distribution information is determined by similarity information between at least one candidate answer in the sample answer text and a labeled answer corresponding to the sample question, and is used to represent probabilities that the text units in the answer text are start positions and end positions of predicted answer abstract information.

[0021] According to an aspect of an embodiment of the present application, a device for training an answer abstract extraction model is provided, and the device comprises:

[0022] A sample acquisition module is configured to acquire a sample question, a sample answer text, and a labeled answer corresponding to the sample question, wherein the sample answer text comprises at least one candidate answer.

[0023] A similarity determination module is configured to determine similarity information between the at least one candidate answer and the labeled answer.

[0024] A probability distribution determination module is configured to determine, based on the similarity information, probability distribution information corresponding to each text unit in the sample answer text, wherein the probability distribution information is used to represent probabilities that the text units are start positions and end positions of answer abstract information.

[0025] A model training module is configured to perform machine learning training by taking the probability distribution information as label information, to obtain an answer abstract extraction model, so that the answer abstract extraction model determines answer abstract information corresponding to an input question based on the input question.

[0026] According to an aspect of an embodiment of the present application, a computer device is provided, which comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above-mentioned training method of the answer abstract extraction model or the above-mentioned answer abstract extraction method.

[0027] According to an aspect of an embodiment of the present application, a computer readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-mentioned training method of the answer abstract extraction model or the above-mentioned answer abstract extraction method.

[0028] According to an aspect of an embodiment of the present application, a computer program product or computer program is provided, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the training method of the answer abstract extraction model or the answer abstract extraction method.

[0029] The technical solutions provided by the embodiments of the present application can bring the following beneficial effects:

[0030] By calculating the similarity between each candidate answer and the labeled answer in the answer text, the probability distribution of each text unit being the start position of the answer and the probability distribution of each text unit being the end position of the answer in the answer text are determined, and then the two probability distributions are used as label information, i.e., each text unit corresponds to the two probabilities as the soft label of the text unit, which greatly enriches the label information of the sample, reduces the dependence of the model on the quality of manual labeling, and avoids the influence of labeling errors or labeling noise. Finally, based on the soft label of each text unit, the answer abstract extraction model is trained and optimized, which can enable the model to better learn the advantages and disadvantages between other answers and the labeled answer, effectively prevent overfitting of the model, improve the accuracy and flexibility of the model in extracting answer abstracts, and make the answer abstract extraction result more reasonable, ensuring that the required key information is quickly and accurately provided to the user. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0032] Figure 1 is a schematic diagram of an application running environment provided by an embodiment of the present application;

[0033] Figure 2 is a flowchart of a training method of an answer abstract extraction model provided by an embodiment of the present application;

[0034] Figure 3 is a flowchart of a training method of an answer abstract extraction model provided by an embodiment of the present application;

[0035] Figure 4 is a flowchart of a training method of an answer abstract extraction model provided by another embodiment of the present application;

[0036] Figure 5is a flowchart of an answer abstract extraction method provided by an embodiment of the present application;

[0037] Figure 6 An exemplary schematic diagram of an answer abstract extraction process is shown;

[0038] Figure 7 is a flowchart of an answer abstract extraction method provided by another embodiment of the present application;

[0039] Figure 8 An exemplary schematic diagram of an answer abstract extraction model is shown;

[0040] Figure 9 is a block diagram of a training device of an answer abstract extraction model provided by an embodiment of the present application;

[0041] Figure 10 is a block diagram of an answer abstract extraction device provided by an embodiment of the present application;

[0042] Figure 11 is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0043] The training method of the answer abstract extraction model and the answer abstract extraction method provided by the embodiments of the present application relate to artificial intelligence technology, which is briefly described below to facilitate understanding by those skilled in the art.

[0044] Artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0045] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0046] Natural Language Processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural Language Processing is a science that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language used in daily life, so it is closely related to the study of linguistics. Natural Language Processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and other technologies.

[0047] Machine Leaming (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a specialized study of how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine Leaming is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine Leaming and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0048] Deep learning: The concept of deep learning comes from the study of artificial neural networks. Multi-layer perceptron with multiple hidden layers is a deep learning structure. Deep learning forms more abstract high-level representation attribute classes or features by combining low-level features to discover distributed feature representation of data.

[0049] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0050] Reference is made to Figure 1 which shows a schematic diagram of an application running environment provided by an embodiment of the present application. The application running environment can include a terminal 10 and a server 20.

[0051] The terminal 10 can be an electronic device such as a mobile phone, a tablet computer, a game console, an e-book reader, a multimedia playback device, a wearable device, a PC (Personal Computer), etc. The client of the application can be installed in the terminal 10.

[0052] In the embodiments of the present application, the application program can be any application program capable of providing intelligent question and answer service. Typically, the application program is a search engine type application program. Of course, intelligent question and answer service can also be provided in other types of application programs in addition to the search engine type application program. For example, shopping type application programs, social type application programs, interactive entertainment type application programs, browser application programs, operation assistance type application programs, intelligent customer service type application programs, personal assistant type application programs, virtual reality (VR) type application programs, augmented reality (AR) type application programs, etc. The embodiments of the present application do not limit this. The intelligent question and answer service described above can be intelligent voice question and answer service, or intelligent text question and answer service. The embodiments of the present application do not limit this.

[0053] The server 20 is configured to provide background service for the client of the application program in the terminal 10. For example, the server 20 can be a background server of the application program described above. The server 20 can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platform. Optionally, the server 20 simultaneously provides background service for the application programs in multiple terminals 10.

[0054] In one embodiment, multiple servers can be composed into a blockchain, and the servers are nodes on the blockchain.

[0055] In one embodiment, the data involved in the answer abstract extraction method or the training method of the answer abstract extraction model can be saved on the blockchain, for example, the question sentence, the answer text matched with the question sentence, or the related model, etc. The data can be saved on the blockchain.

[0056] Optionally, the terminal 10 and the server 20 can communicate with each other through the network 30. The terminal 10 and the server 20 can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0057] Before introducing the method embodiments provided by the present application, the application scenarios, related terms or names that can be involved in the method embodiments of the present application are briefly introduced, so as to facilitate the understanding of the technical personnel in the field of the present application.

[0058] BERT: (Bidirectional Encoder Representations from Transformers), a pre-trained language model.

[0059] Sentence: a unit of text separated by symbols such as commas, periods, question marks, exclamation marks, etc.

[0060] P (Precision), indicates how many words in the model's output belong to the annotated answer.

[0061] R: (Recall), indicates how many words in the annotated answer appear in the model's output.

[0062] F1: the harmonic mean of P and R, is an evaluation index of the answer extraction model. Optionally, F1 = precision x recall x 2 / (precision + recall).

[0063] The answer extraction task refers to extracting the answer related to the given question from the given document, while requiring the extracted answer to be complete and as concise as possible.

[0064] Please refer to Figure 2 , which shows a flowchart of the training method of the answer extraction model provided by an embodiment of the present application. The method can be applied in a computer device, which refers to an electronic device with data calculation and processing capabilities, such as the server 20 or terminal 10 in the application program running environment shown in Figure 1 . The method can include the following steps (210-240).

[0065] Step 210, obtaining a sample question, a sample answer text, and an annotated answer corresponding to the sample question.

[0066] The above sample answer text includes at least one candidate answer.

[0067] The above sample answer text is a text used to answer the sample question. The expression form of the content used to answer the sample question in the sample answer text has diversity, for example, the sample answer text includes both the content answering the question and the related expansion content of the question, which can help understand the answer to the question, but also affects the intuitiveness of the information obtained by the user. For example, in the answer text, a long analysis is first made for the corresponding question, and the answer to the question is given at the end of the answer text, but the user needs to browse the entire answer text to clearly understand the answer corresponding to the input question.

[0068] Optionally, a sample question and a sample answer text corresponding to the sample question are determined from a historical corpus. Optionally, a labeled answer corresponding to the sample question is obtained through manual labeling, i.e., a manually labeled answer. Optionally, the labeled answer corresponding to the sample question is determined based on a natural language understanding model.

[0069] Optionally, the candidate answers herein are all possible answers in the sample answer text. Assuming that the sample question is Q and the sample answer text is a document P, and there are N sentences in the document P, the size of the candidate answers is approximately N*N, or approximately 1+2+…+N-1+N, i.e., the number of combinations of all possible start positions s and all possible end positions e.

[0070] In step 220, similarity information between at least one candidate answer and the labeled answer is determined.

[0071] The similarity information includes similarity between each candidate answer and the labeled answer, and the similarity is used to represent the degree of similarity between the candidate answer and the labeled answer. Optionally, the similarity includes semantic similarity, text similarity, structural similarity, etc., which are not limited by the embodiments of the present application.

[0072] In one possible implementation, given a sample question, a sample answer text related to the sample question, and a manually labeled answer corresponding to the sample question, the similarity relationship between all possible candidate answers in the sample answer text and the labeled answer can be calculated, so as to measure the quality of each candidate answer.

[0073] Optionally, the similarity relationship can be represented as r(A, A * ), where A is a candidate answer, and A * is a labeled answer.

[0074] In an exemplary embodiment, as shown in FIG. 2, a flowchart of a training method of an answer abstract extraction model is shown, and the step 220 includes the following sub-steps (221-222). Figure 3

[0075] In step 221, for a target candidate answer in the at least one candidate answer, word overlap between the target candidate answer and the labeled answer is determined.

[0076] The target candidate answer is any candidate answer in the at least one candidate answer. Optionally, the word overlap between the target candidate answer and the labeled answer is determined by calculating the number of repeated words therebetween. Further, the word overlap can be used as the similarity therebetween. Optionally, the word overlap is an F1 value.

[0077] ​At step 222, similarity information is obtained based on the word overlap between each of the at least one candidate answer and the annotated answer.

[0078] The similarity information includes the word overlap between each of the candidate answers and the annotated answer.

[0079] In a possible implementation, the similarity information includes semantic similarity between each of the at least one candidate answer and the annotated answer.

[0080] Correspondingly, the step 222 can be implemented in the following manner.

[0081] The semantic similarity is obtained based on the word overlap between each of the at least one candidate answer and the annotated answer, and the similarity matrix is generated based on the semantic similarity.

[0082] The value of an element of the similarity matrix represents the semantic similarity between the candidate answer and the annotated answer, where the start position of the text unit is the text unit corresponding to the row of the element, and the end position of the text unit is the text unit corresponding to the column of the element.

[0083] In a possible implementation, the similarity between the candidate answer and the annotated answer is determined by a quality evaluation function. When designing the quality evaluation function of the candidate answer, the following characteristics need to be met: the more similar the candidate answer and the annotated answer are, the higher the quality score obtained by the quality evaluation function is. Alternatively, the quality evaluation function can be implemented by scoring the semantic similarity between the candidate answer and the annotated answer. Alternatively, the semantic similarity between two texts can be accurately measured by a machine learning algorithm, which is difficult but accurate. Alternatively, the semantic similarity between the candidate answer and the annotated answer can also be approximated by measuring the word overlap (i.e., F1 value) between the candidate answer and the annotated answer, that is, the word overlap between the candidate answer and the annotated answer is taken as the semantic similarity between the candidate answer and the annotated answer.

[0084] For example, for the question "What are the causes of a cold?" and the paragraph "There are many causes of a cold, the common cause is bacterial infection, and it may also be caused by a virus. Specific treatment needs to be discussed, if it is a cold, xx medicine can be used, if it is caused by a virus, xx medicine can be used", if the standard answer annotated by human is "the common cause is bacterial infection, and it may also be caused by a virus.", the F1 value between the candidate answer "There are many causes of a cold, the common cause is bacterial infection, and it may also be caused by a virus." and the standard answer "the common cause is bacterial infection, and it may also be caused by a virus." is 0.78. The F1 value between the candidate answer "Specific treatment needs to be discussed, if it is a cold, xx medicine can be used, if it is caused by a virus, xx medicine can be used" and the standard answer "the common cause is bacterial infection, and it may also be caused by a virus." is 0.24. It can be seen that the word coincidence degree can reflect the advantages and disadvantages of each candidate answer to a certain extent.

[0085] Optionally, for each candidate answer, the F1 value between it and the standard answer is calculated, thereby obtaining an N*N size similarity matrix. The element in the i-th row and the j-th column of the matrix represents the word coincidence degree between the candidate answer and the standard answer when the candidate answer starts from the i-th short sentence and ends at the j-th short sentence. Wherein, i and j are positive integers greater than or equal to 1 and less than or equal to N.

[0086] Step 230, based on the similarity information, determine the probability distribution information corresponding to each text unit in the sample answer text.

[0087] The probability distribution information is used to represent the probability of each text unit being the start position and the end position of the answer abstract information.

[0088] The text unit refers to the combination unit of words in the text, which corresponds to an entity in the text. The text unit can be a word, a letter, a word, a phrase, a sentence, a paragraph, etc.

[0089] In an exemplary embodiment, the probability distribution information includes a first probability distribution, and the first probability distribution is used to represent the probability of each text unit being the start position. The first probability distribution includes the first probability corresponding to each text unit. The first probability is used to represent the probability of the text unit being the start position.

[0090] Correspondingly, as shown in Figure 3 The above step 230 includes the following sub-steps (231-233).

[0091] Step 231, for each target text unit in the text unit, obtain the first semantic similarity information between each candidate answer starting from the target text unit and the annotated answer.

[0092] The target text unit refers to any one of the text units. Each candidate answer with the target text unit as the starting position refers to a candidate answer with the target text unit as the same starting position. The starting positions of the candidate answers are all the same, and are all the target text units.

[0093] The first semantic similarity information includes the similarity between each candidate answer with the target text unit as the starting position and the labeled answer, or a value positively correlated with the similarity, i.e., a similarity representation value. Alternatively, the value is a similarity representation value obtained by exponential operation on the similarity as an independent variable. The independent variable can also be a ratio of the similarity and a parameter. The parameter is used to balance the correlation between the similarity representation value and the similarity, so as to ensure that the change rates of the two are consistent or stable in a certain interval, or less than a change rate threshold.

[0094] Alternatively, the first semantic similarity information is the sum of the similarities between each candidate answer with the target text unit as the starting position and the labeled answer. Alternatively, the first semantic similarity information is the sum of the similarity representation values between each candidate answer with the target text unit as the starting position and the labeled answer.

[0095] In step 232, the first probability of the target text unit is determined based on the first semantic similarity information.

[0096] According to the similarity between all candidate answers with the target text unit as the starting position and the labeled answer, the first probability of the target text unit as the starting position can be determined.

[0097] In a possible implementation, the similarity matrix is used to record the similarity information, and the step 231 can be implemented in the following manner.

[0098] Alternatively, each element on the row corresponding to the target text unit in the similarity matrix is determined as the first semantic similarity information.

[0099] The row corresponding to the target text unit in the similarity matrix includes the similarities corresponding to each candidate answer with the target text unit as the starting position. Therefore, the values of each element on the row corresponding to the target text unit can be used to represent the first semantic similarity information.

[0100] In an example, assuming that the target text unit is the i th text unit in the sample answer text, the i th row in the similarity matrix includes the similarities corresponding to each candidate answer with the i th text unit as the starting position.

[0101] Correspondingly, the step 232 can be implemented in the following manner.

[0102] Optionally, a ratio of a sum of elements on the row corresponding to the target text unit to a sum of elements in the similarity matrix is determined as the first probability of the target text unit.

[0103] Optionally, the first probability of the target text unit is obtained based on elements on the row corresponding to the target text unit and elements in the similarity matrix.

[0104] Optionally, a ratio of a sum of similarity representation values corresponding to elements on the row corresponding to the target text unit to a sum of similarity representation values corresponding to elements in the similarity matrix is determined as the first probability of the target text unit.

[0105] At step 233, a first probability distribution is determined based on the first probability of each text unit.

[0106] The first probability distribution includes the first probability of each text unit being a start position.

[0107] In an exemplary embodiment, the probability distribution information further includes a second probability distribution, and the second probability distribution is used to represent a probability of each text unit being an end position. The second probability distribution includes a second probability corresponding to each text unit. The second probability is used to represent a probability of the text unit being an end position.

[0108] Correspondingly, as shown in Figure 3 the above step 230 includes the following sub-steps (234-236).

[0109] At step 234, for the target text unit, second semantic similarity information between each candidate answer taking the target text unit as an end position and the labeled answer is obtained.

[0110] Each candidate answer taking the target text unit as an end position refers to a candidate answer taking the target text unit as the same end position. The end positions of the above-mentioned each candidate answer are all the same, and are all the target text unit.

[0111] The above-mentioned second semantic similarity information includes a similarity between each candidate answer taking the target text unit as an end position and the labeled answer, or a value positively correlated with the similarity, i.e., a similarity representation value. Optionally, the value is a similarity representation value obtained by exponential operation on the similarity as an independent variable. The above-mentioned independent variable can also be a ratio between the similarity and a parameter. The above-mentioned parameter is used to balance a correlation between the similarity representation value and the similarity, which is conducive to ensuring that a change rate between them remains consistent or stable in a certain interval, or is less than a change rate threshold.

[0112] Optionally, the second semantic similarity information is a sum of similarities between each candidate answer ending at the target text unit and the annotated answer.

[0113] At step 235, a second probability of the target text unit is determined based on the second semantic similarity information.

[0114] The first probability of the target text unit ending at the target text unit is determined based on similarities between all candidate answers ending at the target text unit and the annotated answer.

[0115] In the above embodiment of recording the similarity information by using the similarity matrix, the step 234 can be implemented in the following way.

[0116] Optionally, each element on a column corresponding to the target text unit in the similarity matrix is determined as the second semantic similarity information.

[0117] The column corresponding to the target text unit in the similarity matrix includes similarities corresponding to each candidate answer ending at the target text unit. Therefore, the value of each element on the column corresponding to the target text unit can be used to represent the second semantic similarity information.

[0118] In one example, assuming that the target text unit is the jth text unit in the sample answer text, the jth column in the similarity matrix includes similarities corresponding to each candidate answer ending at the jth text unit.

[0119] Correspondingly, the step 235 can be implemented in the following way.

[0120] Optionally, a ratio of a sum of each element on the column corresponding to the target text unit to a sum of each element in the similarity matrix is determined as the second probability of the target text unit.

[0121] Optionally, the second probability of the target text unit is determined based on each element on the column corresponding to the target text unit and each element in the similarity matrix.

[0122] Optionally, a ratio of a sum of similarity representation values corresponding to each element on the column corresponding to the target text unit to a sum of similarity representation values corresponding to each element in the similarity matrix is determined as the second probability of the target text unit.

[0123] At step 236, a second probability distribution is determined based on the second probability of each text unit.

[0124] The second probability distribution includes the second probability of each text unit ending at the target text unit.

[0125] In a possible implementation, since the BERT-based answer abstract extraction model needs to predict the start position and the end position of the answer when performing extraction, the quality evaluation score obtained in the foregoing step for representing the similarity between the candidate answer and the labeled answer needs to be converted into the form of a probability distribution of the start position (first probability distribution) and a probability distribution of the end position (second probability distribution). Optionally, the quality evaluation score is the semantic similarity between the candidate answer and the labeled answer,

[0126] Optionally, the first probability distribution p s (s|A * , τ) of each text unit being the start position in the answer text is determined by using the following formula (1) e (e|A * , τ).

[0127]

[0128] wherein A represents the candidate answer set, τ is a hyperparameter between 0 and 1, r(s, e, A * ) represents the similarity between the candidate answer with the start sentence position being the text unit s and the text unit e and the labeled answer A * . Optionally, the similarity is the F1 value. exp(*) represents the exponential function. exp(r(s, e, A * ) / τ) is a representation of the similarity between the candidate answer with the start sentence position being the text unit s and the text unit e and the labeled answer A*. ∑ e exp(r(s, e, A * ) / τ) represents the sum of the similarity representation values between each candidate answer with the start position being the text unit s and the labeled answer A * . ∑ s exp(r(s, e, A * ) / τ) represents the sum of the similarity representation values between each candidate answer with the start position being the text unit e and the labeled answer A * . ∑ s ∑ e exp(r(s, e, A * ) / τ) represents the sum of the similarity representation values between each candidate answer and the labeled answer A * .

[0129] In step 240, the probability distribution information is taken as label information to perform machine learning training, to obtain an answer abstract extraction model, so that the answer abstract extraction model determines the answer abstract information corresponding to the input question based on the input question.

[0130] In an example embodiment, as shown in FIG. 1, the step 240 comprises the following sub-steps (241-243). Figure 3

[0131] The step 241, for each target text unit, takes the first probability and the second probability of the target text unit as the label information of the target text unit.

[0132] The label of the target text unit is the first probability and the second probability of the target text unit. Both the first probability and the second probability are continuous probabilities between 0 and 1. Optionally, the first probability and the second probability of the target text unit are soft labels of the target text unit.

[0133] The step 242, in the case that the label information of each text unit is supervised information, inputs the sample question and the sample answer text into the target machine learning model to obtain the first output probability and the second output probability corresponding to each text unit.

[0134] The first output probability is the probability of predicting that each text unit is a start position, and the second output probability is the probability of predicting that each text unit is an end position.

[0135] The supervised information is the soft label of each text unit, i.e., the first probability and the second probability of each text unit.

[0136] The process of outputting the first output probability and the second output probability corresponding to each text unit by the target machine learning model can refer to the following answer extraction method embodiment.

[0137] The step 243, under the condition that the first output probability, the second output probability, and the label information of each text unit satisfy the target loss constraint condition, obtains the answer summary extraction model.

[0138] The target loss constraint condition means that the first output probability and the second output probability of each text unit and the first probability and the second probability in the label information of each text unit satisfy the mathematical relationship indicated by the target loss function. Optionally, the target loss function is a cross-entropy loss function, and the specific setting of the target loss function is not limited in the embodiments of the present application, which can be determined according to actual conditions.

[0139] The condition that the first output probability, the second output probability, and the label information of each text unit satisfy the target loss constraint condition means that the target machine learning model has met the requirements of the answer summary extraction task, has relatively stable parameters, and has good model effect, which can be used as the answer summary extraction model.

[0140] Optionally, the target machine learning model is a BERT model, but the type of the target machine learning model is not limited in the embodiments of the present application, which can select a suitable machine learning model according to actual conditions. ​

[0141] In a possible implementation, the first probability and the second probability of each text unit in the sample answer text being a start position and an end position are taken as soft labels of the respective text unit. The soft label gives a probability value between 0 and 1 to each possible answer start position and a probability value between 0 and 1 to each possible answer end position. Compared with the technical solution in which only the manually annotated answer start position is given a probability value of 1 and the start positions of other non-standard answers are given a probability value of 0, the soft label has more rich label information, can effectively prevent model overfitting, and reduce the dependence of the model on the annotation quality.

[0142] Optionally, the soft label is determined based on the two probability distributions, and a loss function corresponding to the answer summary extraction model can be defined as formula (2) below.

[0143]

[0144] wherein p s (s|A * , t) and p e (e|A * , t) are the first probability of the answer start position and the second probability distribution of the end position obtained in equation (1), respectively, and the soft label of each sentence can be determined on this basis, p(i=s) is the probability of the model predicting that the ith text unit is a start position, p(i=e) is the probability of the model predicting that the ith text unit is an end position, log p(i=s) and log p(i=e) are the log likelihood of the answer start position probability p(i=s) and the end position probability p(i=e) predicted by the model, respectively.

[0145] Optionally, p(i=e) and p(i=e) can be obtained by formula (3) below.

[0146]

[0147] wherein W s , W e , b s , b e represent the weight and bias parameters of the answer summary prediction module, respectively, v i is a text unit feature, and s and e represent the start sentence position and the end sentence position of the answer summary predicted by the model, respectively.

[0148] Optionally, another loss function corresponding to the answer summary extraction model can be defined as formula (4) below.

[0149]

[0150] where p(i=s) is the probability of the ith text unit being the start position, p(i=e) is the probability of the ith text unit being the end position, s and e respectively represent the start position and the end position of the annotated answer, and II(condition) represents an indicator function, which is 1 only when the condition in the parentheses is true, and 0 otherwise, i.e., the probability of each text unit in the sample answer text being the start position is 0 except for the text unit s, and the probability of each text unit in the sample answer text being the end position is 0 except for the text unit e. * and e * respectively represent the start position and the end position of the annotated answer, and II(condition) represents an indicator function, which is 1 only when the condition in the parentheses is true, and 0 otherwise, i.e., the probability of each text unit in the sample answer text being the start position is 0 except for the text unit s, and the probability of each text unit in the sample answer text being the end position is 0 except for the text unit e. * * respectively represent the start position and the end position of the annotated answer, and II(condition) represents an indicator function, which is 1 only when the condition in the parentheses is true, and 0 otherwise, i.e., the probability of each text unit in the sample answer text being the start position is 0 except for the text unit s, and the probability of each text unit in the sample answer text being the end position is 0 except for the text unit e.

[0151] In one example, referring to Table 1 below, Table 1 shows an example in which the annotated answer has noise but the extraction result based on the soft label is more complete. From the content of Table 1, it is not difficult to see that the annotator made some errors in the answer annotation process. Therefore, when a model trained using labels of 0 or 1 (referred to as a hard label model in Table 1) is used, the model will overfit the annotation errors of the annotator. However, a model trained based on the probability distribution of the soft label (referred to as a soft label model in Table 1) considers that other candidate answers can also be a reasonable answer, so the extraction result of the model will be more complete than the annotated result, effectively avoiding the influence of model overfitting.

[0152] Table 1

[0153]

[0154] In some cases, the answer summary extraction model may extract an answer that is not exactly the same as the annotated answer, but the answer summary extracted by the model does not affect the reading experience. In another example, referring to Table 2 below, Table 2 shows an example in which the extraction result of a model trained based on the soft label is different from the annotated answer but is still reasonable. From the content of Table 2, it is not difficult to see that the extraction result of the soft label model is different from the annotated answer, but it is still a reasonable answer and is more complete.

[0155] Table 2

[0156]

[0157]

[0158] ​In summary, the technical scheme provided by the embodiments of the present application calculates the similarity between each candidate answer and the labeled answer in the answer text, and then determines the probability distribution of each text unit being the start position of the answer and the probability distribution of each text unit being the end position of the answer, and then the above two probability distributions can be used as label information, that is, each text unit corresponds to the above two probabilities as the soft label of the text unit, which greatly enriches the label information of the sample, reduces the dependence of the model on the quality of artificial labeling, and avoids the influence of labeling errors or labeling noise. Finally, based on the soft label of each text unit, the answer abstract extraction model is trained and optimized, which can enable the model to better learn the advantages and disadvantages between other answers and the labeled answer, effectively prevent the model from overfitting, improve the accuracy and flexibility of the model in extracting the answer abstract, and make the answer abstract extraction result more reasonable, thereby ensuring that the required key information is provided to the user quickly and accurately.

[0159] In some embodiments, the reading comprehension model can be used to predict the positions of the start sentence and the end sentence of the answer abstract. During training, the model maximizes the probability of the start sentence and the end sentence of the answer abstract. However, this approach ignores other reasonable answers that are different from the standard answer. In addition, the labeling personnel inevitably make some errors during the labeling process, and this training method of maximizing the start position of the standard answer will force the model to fit the noise in the labeling, thereby causing the model to have poor performance.

[0160] The answer abstract extraction model training method based on the soft label provided by the embodiments of the present application can alleviate the above problems to improve the performance of the answer abstract extraction model. Specifically, for each candidate answer, the quality evaluation function is used to calculate the similarity score between it and the labeled answer. Then, the probability distribution of the start sentence of the answer abstract and the probability distribution of the end sentence can be obtained based on the similarity score of the candidate answer, and finally the answer abstract extraction model is trained based on the two probability distributions as the soft label of each sentence. Related experiments show that after the answer abstract extraction model is optimized using the soft label, the F1 value of the model is improved, and the model is more robust to the noise in artificial labeling.

[0161] The beneficial effects of the embodiments of the present application will be described in detail below in combination with specific cases.

[0162] Case 1: Ignore the relationship between non-standard answers. For example, for the question "What causes a cold?" and the paragraph "There are many reasons for a cold, the common cause is bacterial infection, and it may also be caused by a virus. Specific treatment needs to be discussed, if it is cold, xx medicine can be used, if it is caused by a virus, xx medicine can be used." If the artificially annotated answer for this question paragraph is "The common cause is bacterial infection, and it may also be caused by a virus." Then in the model training process based on hard labels, the model will maximize the probability of the short sentence "The common cause is bacterial infection" and "It may also be caused by a virus" as the beginning and end of the answer sentence respectively. If the model's prediction result is "There are many reasons for a cold, the common cause is bacterial infection" or "If it is cold, xx medicine can be used, if it is caused by a virus, xx medicine can be used", it will be punished. Obviously, although the model's prediction result is "There are many reasons for a cold, the common cause is bacterial infection", it is obviously better than predicting "If it is cold, xx medicine can be used, if it is caused by a virus, xx medicine can be used". Therefore, the optimization method based on maximizing the probability of the starting sentence ignores the relationship between non-standard answers. The technical scheme provided in the embodiment of the application uses two probability distributions as label information of each text unit, which can avoid the model ignoring the relationship between non-standard answers, so that the model output result is more accurate and reasonable.

[0163] Case 2: Inconsistency or noise exists in the annotation process, leading to overfitting of the model. Because there is a certain subjectivity in the annotation process, different people or models may give different or even wrong annotation results for the same sample. If the prediction result of the answer summary extraction model is required to be completely consistent with the annotation, it is too strict, which leads the model to ignore other answers different from the standard answer but still reasonable. For example, for the question "What causes a cold?" and the paragraph "There are many reasons for a cold, the common cause is bacterial infection, and it may also be caused by a virus. Specific treatment needs to be discussed, if it is cold, xx medicine can be used, if it is caused by a virus, xx medicine can be used", the model's prediction result "There are many reasons for a cold, the common cause is bacterial infection, and it may also be caused by a virus." Although it is different from the annotation "The common cause is bacterial infection, and it may also be caused by a virus.", it is still a reasonable answer. The technical scheme provided in the embodiment of the application uses probability distribution as label information, which is more rich, can avoid the influence brought by the annotation process, and prevent the model from overfitting.

[0164] Please refer to Figure 4Fig. 4 shows a flowchart of a method for training an answer extraction model according to another embodiment of the present application. The method can be applied in a computer device, which is an electronic device with data computing and processing capability. For example, the execution subject of each step can be a server 20 or a terminal 10 in the application environment as shown in Fig. 1. Figure 1 The method can include the following steps (401-413).

[0165] In step 401, a sample question, a sample answer text and a labeled answer corresponding to the sample question are obtained.

[0166] In step 402, for a target candidate answer in the at least one candidate answer, word overlap between the target candidate answer and the labeled answer is determined.

[0167] In step 403, semantic similarity is obtained based on the word overlap between each of the at least one candidate answer and the labeled answer.

[0168] In step 404, a similarity matrix is generated based on the semantic similarity.

[0169] Specifically, the value of an element of the similarity matrix represents the semantic similarity between the candidate answer and the labeled answer, which starts at the text unit corresponding to the row of the element and ends at the text unit corresponding to the column of the element.

[0170] In step 405, each element on the row corresponding to the target text unit in the similarity matrix is determined as first semantic similarity information.

[0171] In step 406, the ratio of the sum of the elements on the row corresponding to the target text unit to the sum of all elements in the similarity matrix is determined as the first probability of the target text unit.

[0172] In step 407, a first probability distribution is determined based on the first probability of each text unit.

[0173] In step 408, each element on the column corresponding to the target text unit in the similarity matrix is determined as second semantic similarity information.

[0174] In step 409, the ratio of the sum of the elements on the column corresponding to the target text unit to the sum of all elements in the similarity matrix is determined as the second probability of the target text unit.

[0175] In step 410, a second probability distribution is determined based on the second probability of each text unit.

[0176] In step 411, for the target text unit, the first probability and the second probability of the target text unit are taken as label information of the target text unit.

[0177] Step 412, in the case that the label information of each text unit is supervised information, input the sample question and the sample answer text into the target machine learning model to obtain the first output probability and the second output probability corresponding to each text unit.

[0178] Step 413, in the case that the first output probability, the second output probability and the label information of each text unit satisfy the target loss constraint condition, obtain the answer abstract extraction model.

[0179] The description of each step of the embodiment can refer to the above embodiment, which will not be repeated here.

[0180] To sum up, the technical scheme provided by the embodiment of the application calculates the similarity between each candidate answer in the answer text and the labeled answer, generates a similarity matrix, and then determines the first probability distribution of each text unit in the answer text being the start position of the answer and the second probability distribution of each text unit being the end position of the answer based on the value of each row or column element in the similarity matrix. Then the above two probability distributions can be used as the label information of each text unit, i.e. each text unit corresponds to a continuous first probability and a second probability in the range of 0 to 1, and as a soft label of the text unit. This greatly enriches the label information of the sample, reduces the dependence of the model on the quality of manual labeling, and avoids the influence of labeling errors or labeling noise. Finally, the answer abstract extraction model is iteratively trained and optimized based on the soft label of each text unit and the set loss constraint to ensure that the model can better learn the superiority-inferiority relationship between other answers and the labeled answer, effectively prevent the model from overfitting, and improve the accuracy, flexibility and rationality of the model in extracting the answer abstract, thereby ensuring that the required key information is quickly and accurately provided to the user.

[0181] Please refer to Figure 5 which shows a flowchart of an answer abstract extraction method provided by an embodiment of the application. The method can be applied in a computer device, which refers to an electronic device with data calculation and processing capability, such as the server 20 in the application program running environment shown in Figure 1 The method can include the following steps (510-530).

[0182] Step 510, obtain an input question.

[0183] The input question can be a query text input by a user in a search engine, a message input in a dialogue system, or an input question obtained based on voice information of the user. The embodiment of the application does not limit the acquisition method of the input question.

[0184] Step 520, determine an answer text corresponding to the input question based on the input question.

[0185] The above answer text is a text for answering the input question. The expression form of the content for answering the input question in the answer text has diversity, for example, the answer text includes the content of answering the question and the related extension content of the question. The extension content can help understand the answer of the question, but also affects the intuitiveness of the information obtained by the user. For example, in the answer text, a long analysis is first made for the corresponding question, and the answer to the question is given at the end of the answer text. However, the user needs to browse the entire answer text to clearly understand the answer corresponding to the input question.

[0186] In a possible implementation, a network search result corresponding to the input question is determined. The search result can be a web page, a document, a picture, and the like.

[0187] The answer text corresponding to the input question in the search result is determined. For example, the text content in the web page, the document, and the like is crawled, and the text content in the picture is recognized.

[0188] In step 530, the input question and the answer text are input into the answer abstract extraction model to perform answer abstract extraction processing, and answer abstract information corresponding to the input question is obtained.

[0189] The answer abstract extraction model is the answer abstract extraction model provided in the above embodiments.

[0190] The answer abstract extraction model is a machine learning model trained by taking a sample question, a sample answer text, and probability distribution information corresponding to each text unit in the sample answer text as label information. The probability distribution information is determined by similarity information between at least one candidate answer in the sample answer text and a labeled answer corresponding to the sample question, and is used to represent the probability that each text unit in the answer text is the start position and the end position of the predicted answer abstract information.

[0191] For the training process of the answer abstract extraction model, refer to the embodiments of the training method of the answer abstract extraction model, which will not be repeated here.

[0192] In an example, as Figure 6As shown, this example illustrates a schematic diagram of an answer summary extraction process. This example uses a question-and-answer service applied to a search engine as an example. The answer summary extraction method provided in this application embodiment can extract sentence-level answer summaries and display them to the user at the top of the search engine. When a user enters a factual question, such as searching for "furazolidone instructions and dosage," the search engine can perform a retrieval based on "furazolidone instructions and dosage" to determine the relevant documents corresponding to "furazolidone instructions and dosage" as the answer text. For example, the content of the relevant document is: "Furazoline tablets are a Western medicine antibacterial drug, mainly used to treat Helicobacter pylori infection. Adults take 0.1g orally once, 3-4 times a day; children take 5-10mg per kilogram of body weight per day, divided into four doses. The course of treatment for intestinal infections is 5-7 days." Next, the search engine can call the answer summary extraction interface, input the text content of "furazolidone instructions and dosage" and related documents into the answer summary extraction model. The answer summary extraction model outputs the answer summary from the related documents: "Adults take 0.1g orally once, 3-4 times a day; children take 5-10mg per kilogram of body weight per day, divided into four doses; the course of treatment for intestinal infection is 5-7 days." The output results are then fed back to the user, saving the user the time needed to find the answer.

[0193] In an exemplary embodiment, such as Figure 7 As shown, a flowchart of an answer summary extraction method provided in another embodiment of this application is illustrated, wherein step 530 includes the following sub-steps (531-535).

[0194] Step 531: Generate a concatenated character sequence based on the input question and answer text.

[0195] In one possible implementation, the input question and the words in the answer text can be concatenated to obtain a word concatenation sequence.

[0196] Optionally, the input question and answer text are encoded to obtain a concatenated character sequence corresponding to the input question and answer text.

[0197] In one example, such as Figure 8 As shown, this example illustrates a schematic diagram of an answer summary extraction model. Assume the user input question is Q, the given document is P, and Q is composed of (x1, x2, ..., x...). M ) represents that P consists of (y1, y2, ..., y P ) represents. Where x i Let y represent the i-th character in question Q. i Let M represent the i-th text unit in document P, and M be the number of characters in the character sequence corresponding to question Q. Assume that the number of characters in the character sequence corresponding to each text unit in document P is N.

[0198] The character sequence corresponding to question Q is (x1, ..., x...). M Where x1 is the first character in the character sequence corresponding to question Q, x M Let M be the Mth character in the character sequence corresponding to question Q.

[0199] The character sequence corresponding to the first text unit y1 in document P is (y 1，1 , ..., y 1，N ), where y 1，1 y1 is the first character in the character sequence corresponding to the first text unit y1. 1，N It is the Nth character in the character sequence corresponding to the first text unit y1.

[0200] The character sequence corresponding to the second text unit y2 in document P is (y 2，1 , ..., y 2，N ), where y 2，1 y is the first character in the character sequence corresponding to the second text unit y2. 2，N The Nth character in the character sequence corresponding to the second text unit y2.

[0201] The p-th text unit y in document P P The corresponding character sequence is (y P，1 , ..., y P，N ), where y P，1 For the p-th text unit y P The first character in the corresponding character sequence, y P，N For the p-th text unit y P The Nth character in the corresponding character sequence.

[0202] In one possible implementation, the words in the question and document, along with delimiters, are concatenated to obtain a concatenated character sequence. This concatenated character sequence can be represented as: [CLS]x1, ...,x M [SEP]y1,...,y P [SEP].

[0203] Optionally, the separators include a start position separator [CLS] and a type separator [SEP]. For the concatenated character sequence corresponding to question Q and document P, this includes one [CLS] and two [SEPs]. [CLS] is the first character in the concatenated character sequence, used to indicate the start position of the sequence. Following [CLS] is the character sequence x1, ..., x1, formed by the sequential arrangement of the characters from question Q. MThe characters in question Q are obtained by segmenting them according to a preset method. They can be single characters or words composed of multiple characters. The first [SEP] separator is located at the last character 'x' in the character sequence corresponding to question Q. M The second [SEP] separator is located between the first character y1 in the character sequence corresponding to document P, and is located at the last character y in the character sequence corresponding to document P. N After.

[0204] Step 532: Determine the character characteristics of each character in the concatenated character sequence.

[0205] Optionally, feature extraction processing is performed on each character in the concatenated character sequence to obtain the character features of each character. These character features are obtained by word representation processing or word embedding processing, representing the mathematical representation of the character in the feature space. These character features can characterize the semantic information of the character and its preceding and following context.

[0206] exist Figure 8 In the example shown, each character in the concatenated character sequence learns its corresponding latent representation H through a pre-trained bidirectional encoding representation BERT model. The latent features are multidimensional feature representations obtained by combining the contextual features of each character in the concatenated character sequence through feature extraction.

[0207] Alternatively, the implicit representation H can be determined by the following equation (5).

[0208] H = BERT([CLS], x1, ..., x2) M [SEP], y1, ..., y P (5)

[0209] Among them, matrix This represents the hidden layer output of the BERT model, where d represents the dimension of the BERT hidden layer, M and N represent the number of characters segmented in question Q and document P, respectively, and [CLS] and [SEP] represent special delimiters. Using the above method, the features of all characters in the concatenated character sequence can be obtained.

[0210] Optionally, the output of each character in the hidden layer of the BERT model can be used as the character feature of each character.

[0211] exist Figure 8 In the example shown, e cls The character characteristics of the special character [CLS] are defined here, and e1 is the character characteristic of character x1. M For the character x M Character features, e sep For the character characteristics of the special character [SEP], e 1，1 For the character y1，1 Character features, e 1，N For the character y 1，N Character features, e 2，1 For the character y 2，1 Character features, e 2，N For the character y 2，N Character features, e P，1 For the character y P，1 Character features, e P，N For the character y P，N Character characteristics.

[0212] Step 533: Based on character features, obtain the text unit features of each text unit in the input question and answer text.

[0213] For each target text unit, the character features of each character corresponding to the target text unit are extracted to obtain the text unit features of the target text unit.

[0214] The above feature extraction process is repeated for each text unit to obtain the text unit features of each text unit. Optionally, the above feature extraction process can be based on a self-attention mechanism, average pooling, or max pooling operation.

[0215] exist Figure 8 In the example shown, after obtaining the character features, text unit features (which can be sentence features) are obtained for each text unit (which can be a sentence) based on the character features. Optionally, the character feature sequence corresponding to the text unit is input into the text unit representation layer to obtain the corresponding text unit features. There are various methods to obtain text unit features from character features, such as self-attention mechanisms, average pooling, and max pooling.

[0216] The following example illustrates how to determine text unit features based on a self-attention mechanism. For the p-th text unit in a paragraph of document P, assuming that after character feature extraction, the implicit representation of each character in the character sequence corresponding to text unit p is (e p，1 e p，2 , ..., e p，N ).

[0217] Alternatively, the character weight can be determined by the following scoring function formula (6), whereby the character weight is used to characterize the importance of the character.

[0218]

[0219] Here, MLP stands for Multilayer Perceptron Model. Character features are input into the Multilayer Perceptron Model to determine the character feature value 'a'. p，i The above character feature value a p，iThis represents the importance of the i-th word in text unit p, softmax(*) is the probability normalization function, and score is the value representing the importance of the i-th word. p，i This represents the weight of the i-th word in text unit p.

[0220] The text unit features are determined based on the character weights and features of each character in the text unit. Optionally, the text unit feature v of the p-th text unit... p It is obtained through the following formula (7).

[0221] v p =∑ 1≤i≤N score p，i *e p，i (7)

[0222] exist Figure 8 In the example shown, v1 is the text unit feature of the first text unit, v2 is the text unit feature of the second text unit, and v p Let be the text unit feature of the p-th text unit.

[0223] Step 534: Based on the characteristics of the text units, determine the first output probability and the second output probability corresponding to each text unit in the solution text.

[0224] The first output probability is the probability that each text unit is the starting position of the answer summary information, and the second output probability is the probability that each text unit is the ending position of the answer summary information.

[0225] Optionally, the first output probability P(i=s) and the second output probability P(i=e) can be obtained by the following equation (8).

[0226]

[0227] Among them, W s W e b s b e These represent the weights and bias parameters of the answer summary prediction module, respectively. i is the text unit feature of the i-th text unit, and s and e represent the start and end positions of the predicted answer summary by the model, respectively. Optionally, the above W s W e b s b e Determined through model training.

[0228] exist Figure 8 In the example shown, the text unit features of each text unit are input into the answer prediction layer to determine the first output probability and the second output probability corresponding to each text unit.

[0229] At step 535, the answer abstract information is determined based on the first output probability and the second output probability corresponding to each text unit.

[0230] Optionally, the start position of the answer abstract is determined as the text unit corresponding to the maximum first output probability, the end position of the answer abstract is determined as the text unit corresponding to the maximum second output probability, and the answer abstract corresponding to the start position and the end position is determined.

[0231] To sum up, the technical scheme provided by the embodiments of the present application calculates the similarity between each candidate answer in the answer text and the labeled answer, and then determines the probability distribution of each text unit in the answer text being the start position of the answer and the probability distribution of each text unit being the end position of the answer. Then, the above two probability distributions are used as label information, i.e., each text unit corresponds to the above two probabilities as the soft label of the text unit, which greatly enriches the label information of the sample, reduces the dependence of the model on the quality of manual labeling, and avoids the influence of labeling errors or labeling noise. Finally, based on the soft label of each text unit, the answer abstract extraction model is trained and optimized, which enables the model to better learn the advantages and disadvantages between other answers and the labeled answer, effectively prevents overfitting of the model, and improves the accuracy and flexibility of the model in extracting the answer abstract. The answer abstract extraction model can output a more reasonable answer abstract according to the input question and the corresponding answer text, ensuring that the key information required by the user is provided quickly and accurately.

[0232] In addition, by performing feature extraction processing on the spliced character sequence composed of the input question and the answer text, the context features of each character can be determined, and then the semantic features of each text unit can be determined. Then, based on the semantic features of the input question and the answer text, the probability of each text unit in the answer text being the start position of the answer abstract and the probability of each text unit being the end position of the answer abstract can be outputted. The start position and the end position of the answer abstract are determined through the above two probabilities, and the answer abstract is fed back to the user, ensuring the accuracy of the output answer abstract and improving the efficiency of intelligent question answering services.

[0233] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0234] Please refer to Figure 9FIG. 9 is a block diagram illustrating an apparatus for training an answer abstract extraction model according to an embodiment of the present application. The apparatus has the functions of implementing the method for training an answer abstract extraction model described above, which can be implemented by hardware, or by execution of corresponding software by hardware. The apparatus can be a computer device, or can be arranged in a computer device. The apparatus 900 can include a sample obtaining module 910, a similarity determining module 920, a probability distribution determining module 930, and a model training module 940.

[0235] The sample obtaining module 910 is configured to obtain a sample question, a sample answer text, and a labeled answer corresponding to the sample question, the sample answer text including at least one candidate answer.

[0236] The similarity determining module 920 is configured to determine similarity information between the at least one candidate answer and the labeled answer.

[0237] The probability distribution determining module 930 is configured to determine, based on the similarity information, probability distribution information corresponding to each text unit in the sample answer text, the probability distribution information being used to represent probabilities of the each text unit being a start position and an end position of answer abstract information.

[0238] The model training module 940 is configured to perform machine learning training on the probability distribution information as label information, to obtain an answer abstract extraction model, so that the answer abstract extraction model determines answer abstract information corresponding to an input question based on the input question.

[0239] In an example embodiment, the probability distribution information includes a first probability distribution, the first probability distribution being used to represent a probability of the each text unit being the start position, and the probability distribution determining module 930 includes a similarity obtaining unit, a probability determining unit, and a probability distribution determining unit.

[0240] The similarity obtaining unit is configured to, for a target text unit of the each text unit, obtain first semantic similarity information between each candidate answer taking the target text unit as the start position and the labeled answer.

[0241] The probability determining unit is configured to determine a first probability of the target text unit based on the first semantic similarity information.

[0242] The probability distribution determining unit is configured to determine the first probability distribution based on the first probability of the each text unit, the first probability distribution including the first probability of the each text unit being the start position.

[0243] In an example embodiment, the probability distribution information further comprises a second probability distribution, the second probability distribution being used to represent probabilities of the text units being the ending position.

[0244] The similarity obtaining unit is further configured to obtain, for the target text unit, second semantic similarity information between each candidate answer taking the target text unit as the ending position and the labeled answer.

[0245] The probability determining unit is further configured to determine a second probability of the target text unit based on the second semantic similarity information.

[0246] The probability distribution determining unit is configured to determine the second probability distribution based on the second probabilities of the text units, the second probability distribution comprising second probabilities of the text units being the ending position.

[0247] In an example embodiment, the similarity determining module 920 comprises a word overlap degree determining unit and a similarity determining unit.

[0248] The word overlap degree determining unit is configured to determine, for a target candidate answer in the at least one candidate answer, a word overlap degree between the target candidate answer and the labeled answer.

[0249] The similarity determining unit is configured to obtain the similarity information based on the word overlap degrees between the at least one candidate answer and the labeled answer.

[0250] In an example embodiment, the similarity information comprises semantic similarities between the at least one candidate answer and the labeled answer, and the similarity determining unit comprises a semantic similarity determining subunit and a similarity matrix generating subunit.

[0251] The semantic similarity determining subunit is configured to obtain the semantic similarities based on the word overlap degrees between the at least one candidate answer and the labeled answer.

[0252] The similarity matrix generating subunit is configured to generate a similarity matrix based on the semantic similarities.

[0253] An element of the similarity matrix is used to represent a semantic similarity between a candidate answer and a labeled answer, the candidate answer taking a text unit corresponding to a row of the element as the starting position and taking a text unit corresponding to a column of the element as the ending position.

[0254] In an example embodiment, the similarity obtaining unit is specifically configured to:

[0255] Determine each element on the row corresponding to the target text unit in the similarity matrix as the first semantic similarity information.

[0256] The probability determination unit is specifically configured to:

[0257] Determine the ratio of the sum of each element on the row corresponding to the target text unit in the similarity matrix to the sum of each element in the similarity matrix as the first probability of the target text unit.

[0258] In an example embodiment, the similarity obtaining unit is further specifically configured to:

[0259] Determine each element on the column corresponding to the target text unit in the similarity matrix as the second semantic similarity information.

[0260] The probability determination unit is further specifically configured to:

[0261] Determine the ratio of the sum of each element on the column corresponding to the target text unit in the similarity matrix to the sum of each element in the similarity matrix as the second probability of the target text unit.

[0262] In an example embodiment, the model training module 940 includes a label information determination unit, a probability prediction unit, and a model determination unit.

[0263] The label information determination unit is configured to, for the target text unit, determine the first probability and the second probability of the target text unit as label information of the target text unit.

[0264] The probability prediction unit is configured to, in a case where the label information of each text unit is supervised information, input the sample question and the sample answer text into a target machine learning model to obtain a first output probability and a second output probability corresponding to each text unit, the first output probability being a probability of predicting that each text unit is the start position, and the second output probability being a probability of predicting that each text unit is the end position.

[0265] The model determination unit is configured to, in a case where the first output probability, the second output probability, and the label information of each text unit satisfy a target loss constraint condition, obtain the answer abstract extraction model.

[0266] In summary, the technical scheme provided by the embodiments of the present application calculates the similarity between each candidate answer in the answer text and the labeled answer, and then determines the probability distribution of each text unit in the answer text being the starting position of the answer and the probability distribution of each text unit being the ending position of the answer. Then, the above two probability distributions can be used as label information, that is, each text unit corresponds to the above two probabilities as the soft label of the text unit, which greatly enriches the label information of the sample, reduces the dependence of the model on the quality of manual labeling, and avoids the influence of labeling errors or labeling noise. Finally, based on the soft label of each text unit, the answer abstract extraction model is trained and optimized, which can enable the model to better learn the advantages and disadvantages between other answers and the labeled answer, effectively prevent overfitting of the model, improve the accuracy and flexibility of the model in extracting answer abstracts, and make the answer abstract extraction result more reasonable, thereby ensuring that the required key information is quickly and accurately provided to the user.

[0267] Reference is made to Figure 10 which shows a block diagram of an answer abstract extraction device provided by an embodiment of the present application. The device has the functions of implementing the above answer abstract extraction method, which can be implemented by hardware or corresponding software executed by hardware. The device can be a computer device or can be arranged in a computer device. The device 1000 can include a question acquisition module 1010, an answer text determination module 1020, and an answer abstract extraction module 1030.

[0268] The question acquisition module 1010 is configured to acquire the input question.

[0269] The answer text determination module 1020 is configured to determine an answer text corresponding to the input question based on the input question.

[0270] The answer abstract extraction module 1030 is configured to input the input question and the answer text into an answer abstract extraction model to perform answer abstract extraction processing, and obtain answer abstract information corresponding to the input question.

[0271] The answer abstract extraction model is a machine learning model trained by taking a sample question, a sample answer text, and probability distribution information corresponding to each text unit in the sample answer text as label information. The probability distribution information is determined by the similarity information between at least one candidate answer in the sample answer text and the labeled answer corresponding to the sample question, and is used to represent the probability of each text unit in the answer text being the starting position and the ending position of the predicted answer abstract information.

[0272] In an example embodiment, the answer summary extraction module 1030 comprises a sequence generation unit, a character feature extraction unit, a text unit feature extraction unit, a probability output unit, and an answer summary determination unit.

[0273] The sequence generation unit is configured to generate a concatenation character sequence based on the input question and the answer text.

[0274] The character feature extraction unit is configured to determine character features of each character in the concatenation character sequence.

[0275] The text unit feature extraction unit is configured to obtain text unit features of each text unit in the input question and the answer text based on the character features.

[0276] The probability output unit is configured to determine a first output probability and a second output probability corresponding to each text unit in the answer text according to the text unit features, the first output probability being a probability that the text unit is a start position of the answer summary information, and the second output probability being a probability that the text unit is predicted to be an end position of the answer summary information.

[0277] The answer summary determination unit is configured to determine the answer summary information based on the first output probability and the second output probability corresponding to each text unit.

[0278] In summary, the technical scheme provided by the embodiments of the present application calculates the similarity between each candidate answer and the labeled answer in the answer text, and then determines the probability distribution of each text unit being an answer start position and the probability distribution of each text unit being an answer end position in the answer text. Then, the above two probability distributions can be used as label information, i.e., each text unit corresponds to the above two probabilities as the soft label of the text unit, which greatly enriches the label information of the sample, reduces the dependence of the model on the quality of manual labeling, and avoids the influence of labeling errors or labeling noise. Finally, based on the soft labels of each text unit, the answer summary extraction model is trained and optimized, which enables the model to better learn the advantages and disadvantages between other answers and the labeled answer, effectively prevents model overfitting, and improves the accuracy and flexibility of the model in extracting answer summaries. The answer summary extraction model can output a more reasonable answer summary according to the input question and the corresponding answer text, ensuring that the required key information is provided to the user quickly and accurately.

[0279] In addition, by performing feature extraction processing on the spliced character sequence composed of the input question and the answer text, the context feature of each character can be determined, and then the semantic feature of each text unit can be determined. Then, according to the semantic features of the input question and the answer text, the probability of each text unit being the start position of the answer abstract and the probability of each text unit being the end position of the answer abstract can be output. The start position and the end position of the answer abstract are determined by the above two probabilities, and the answer abstract is fed back to the user to ensure the accuracy of the output answer abstract and improve the efficiency of the intelligent question and answer service.

[0280] It should be noted that the device provided in the above embodiment is only exemplified by the division of the above functional modules in realizing its functions. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0281] Please refer to Figure 11 which shows the structure block diagram of the computer device provided in an embodiment of the present application. The computer device can be a terminal or a server, which is used to execute the training method or the answer abstract extraction method of the answer abstract extraction model. Specifically:

[0282] The computer device 1100 includes a central processing unit (CPU) 1101, a system memory 1104 including a random access memory (RAM) 1102 and a read-only memory (ROM) 1103, and a system bus 1105 connecting the system memory 1104 and the central processing unit 1101. The computer device 1100 also includes a basic input / output system (I / O) 1106 that helps transfer information between various devices in the computer, and a mass storage device 1107 for storing an operating system 1113, application programs 1114 and other program modules 1115.

[0283] The basic input / output system 1106 includes the various components needed to display information and input information from a user, such as a display 1108 and input devices 1109, such as a mouse, keyboard, or electronic stylus. The display 1108 and input devices 1109 are connected to the central processing unit 1101 through an input / output controller 1110 that is connected to the system bus 1105. The basic input / output system 1106 can also include the input / output controller 1110 for receiving and processing input from a number of other devices, including a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1110 provides output to the display screen, a printer, or other type of output device.

[0284] The mass storage device 1107 is connected to the central processing unit 1101 through a mass storage controller (not shown) that is connected to the system bus 1105. The mass storage device 1107 and its associated computer readable media provide nonvolatile storage for the computer device 1100. That is, the mass storage device 1107 can include a computer readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0285] Without loss of generality, computer readable media can include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid state memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Computer storage media would not, however, include communication media including wired or wireless signaling media that communicate program code in a modulated data signal. The system memory 1104 and mass storage device 1107 described above can be embodied as a memory.

[0286] According to various embodiments of the present application, the computer device 1100 can also operate in a networked environment using a network connection to one or more remote computers. The computer device 1100 can connect to the network 1112 through a network interface unit 1111 attached to the system bus 1105, or can connect to other types of networks or remote computer systems (not shown) using other types of network interface units 1111.

[0287] The memory further includes a computer program stored in the memory and configured to be executed by one or more processors to implement the training method of the answer abstract extraction model or the answer abstract extraction method.

[0288] In an exemplary embodiment, a computer readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is executed by a processor to implement the training method of the answer abstract extraction model or the answer abstract extraction method.

[0289] Optionally, the computer readable storage medium can include a ROM (Read Only Memory), a RAM (Random Access Memory), a SSD (Solid State Drives), an optical disc or the like. Among them, the random access memory can include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0290] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device execute the training method of the answer abstract extraction model or the answer abstract extraction method.

[0291] It should be understood that "multiple" mentioned herein refers to two or more than two. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. In addition, the step numbers described herein only exemplarily show a possible execution order between steps, and in some other embodiments, the above steps can also be executed in a non-numbered order, such as two different numbered steps being executed at the same time, or two different numbered steps being executed in an order opposite to that shown in the figure, and the embodiments of the present application are not limited in this regard.

[0292] The above only describes exemplary embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An answer abstract extraction method, characterized by, The method comprises: obtaining an input question; determining a solution text corresponding to the input question based on the input question; inputting the input question and the solution text into an answer summary extraction model to perform answer summary extraction processing, and obtaining answer summary information corresponding to the input question; wherein the answer summary extraction model is a machine learning model trained using sample questions, sample solution texts, and probability distribution information corresponding to each text unit in the sample solution texts as label information, the probability distribution information is determined based on similarity information between at least one candidate answer in the sample solution text and the labeled answer corresponding to the sample question, and is used to represent the probability that each text unit in the solution text is the starting position and the ending position of the predicted answer summary information.

2. The method of claim 1, wherein, The input question and the solution text are input into the answer summary extraction model to perform answer summary extraction processing, and the answer summary information corresponding to the input question is obtained, comprising: generating a spliced character sequence based on the input question and the solution text; determining character features of each character in the spliced character sequence; based on the character features, obtaining text unit features of each text unit in the input question and the solution text; determining first output probability and second output probability corresponding to each text unit in the solution text according to the text unit features, the first output probability being the probability that the text unit is the starting position of the answer summary information, and the second output probability being the probability that the text unit is the ending position of the answer summary information; determining the answer summary information based on the first output probability and the second output probability corresponding to each text unit.

3. A training method of an answer abstract extraction model, characterized by, The method comprises: obtaining sample questions, sample solution texts, and labeled answers corresponding to the sample questions, the sample solution texts including at least one candidate answer; determining similarity information between the at least one candidate answer and the labeled answer; based on the similarity information, determining probability distribution information corresponding to each text unit in the sample solution text, the probability distribution information being used to represent the probability that each text unit is the starting position and the ending position of the answer summary information; performing machine learning training using the probability distribution information as label information to obtain an answer summary extraction model, so that the answer summary extraction model determines answer summary information corresponding to an input question based on the input question.

4. The method of claim 3, wherein, The probability distribution information includes a first probability distribution, which is used to represent the probability that each text unit is the starting position, and the determination of the probability distribution information corresponding to each text unit in the sample solution text based on the similarity information comprises: for a target text unit in the text units, obtaining first semantic similarity information between each candidate answer taking the target text unit as the starting position and the labeled answer; determining a first probability of the target text unit based on the first semantic similarity information; The first probability distribution is determined based on the first probability of each text unit, and the first probability distribution includes a first probability that each text unit is the start position.

5. The method of claim 4, wherein, The probability distribution information further includes a second probability distribution, and the second probability distribution is used to represent a probability that each text unit is the end position. The probability distribution information further includes a second probability distribution, and the second probability distribution is used to represent a probability that each text unit is the end position. For the target text unit, second semantic similarity information between each candidate answer taking the target text unit as the end position and the labeled answer is obtained. A second probability of the target text unit is determined based on the second semantic similarity information.

6. The method according to claim 4 or 5, characterized in that, The second probability distribution is determined based on the second probability of each text unit, and the second probability distribution includes a second probability that each text unit is the end position. The determination of the similarity information between the at least one candidate answer and the labeled answer includes: For a target candidate answer in the at least one candidate answer, word overlap degrees between the target candidate answer and the labeled answer are determined.

7. The method of claim 6, wherein, The similarity information is obtained based on the word overlap degrees between the at least one candidate answer and the labeled answer. The similarity information includes semantic similarities between the at least one candidate answer and the labeled answer, and the similarity information is obtained based on the word overlap degrees between the at least one candidate answer and the labeled answer, including: The semantic similarities are obtained based on the word overlap degrees between the at least one candidate answer and the labeled answer. A similarity matrix is generated based on the semantic similarities.

8. The method of claim 5, wherein, The value of an element of the similarity matrix is used to represent a semantic similarity between a candidate answer and a labeled answer, where a text unit corresponding to a row in which the element is located is taken as the start position, and a text unit corresponding to a column in which the element is located is taken as the end position. The probability distribution information is used as label information for machine learning training to obtain an answer abstract extraction model, including: For the target text unit, the first probability and the second probability of the target text unit are used as label information of the target text unit. In a case where the label information of each text unit is supervision information, the sample question and the sample answer text are input into a target machine learning model to obtain first output probabilities and second output probabilities corresponding to each text unit, the first output probability being a probability predicted for each text unit being the start position, and the second output probability being a probability predicted for each text unit being the end position.

9. An answer abstract extraction apparatus characterized by comprising: The answer abstract extraction model is obtained in a case where the first output probability, the second output probability, and the label information of each text unit satisfy a target loss constraint condition. The apparatus includes: A question acquisition module configured to acquire an input question; An answer text determination module configured to determine an answer text corresponding to the input question based on the input question; An answer abstract extraction module is configured to input the input question and the answer text into an answer abstract extraction model to perform answer abstract extraction processing, and obtain answer abstract information corresponding to the input question. The answer abstract extraction model is a machine learning model trained by using sample questions, sample answer texts, and probability distribution information corresponding to each text unit in the sample answer texts as label information. The probability distribution information is determined based on similarity information between at least one candidate answer in the sample answer text and the labeled answer corresponding to the sample question, and is used to represent probabilities of each text unit being a start position and an end position of predicted answer abstract information.

10. The apparatus of claim 9, wherein, The answer abstract extraction module includes: A sequence generation unit is configured to generate a concatenated character sequence based on the input question and the answer text. A character feature extraction unit is configured to determine character features of each character in the concatenated character sequence. A text unit feature extraction unit is configured to obtain text unit features of each text unit in the input question and the answer text based on the character features. A probability output unit is configured to determine first output probabilities and second output probabilities of each text unit in the answer text according to the text unit features. The first output probability is a probability of the text unit being the start position of the answer abstract information, and the second output probability is a probability of predicting the text unit being the end position of the answer abstract information. An answer abstract determination unit is configured to determine the answer abstract information based on the first output probabilities and the second output probabilities of each text unit. 11.A training apparatus of an answer abstract extraction model, characterized in that, The device includes: A sample acquisition module is configured to acquire sample questions, sample answer texts, and labeled answers corresponding to the sample questions. The sample answer texts include at least one candidate answer. A similarity determination module is configured to determine similarity information between the at least one candidate answer and the labeled answer. A probability distribution determination module is configured to determine probability distribution information corresponding to each text unit in the sample answer text based on the similarity information. The probability distribution information is used to represent probabilities of each text unit being a start position and an end position of answer abstract information. A model training module is configured to perform machine learning training by using the probability distribution information as label information, to obtain an answer abstract extraction model. The answer abstract extraction model is configured to determine answer abstract information corresponding to an input question based on the input question.

12. The apparatus of claim 11, wherein, The probability distribution information includes first probability distribution. The probability distribution determination module includes: A similarity acquisition unit is configured to acquire first semantic similarity information between each candidate answer with the start position and the labeled answer for a target text unit in the each text unit. A probability determination unit is configured to determine a first probability of the target text unit based on the first semantic similarity information. The probability distribution determining unit is configured to determine the first probability distribution based on the first probabilities of the text units, the first probability distribution comprising a first probability of each text unit being the start position.

13. The apparatus of claim 12, wherein, The probability distribution information further comprises a second probability distribution, the second probability distribution being used to represent a probability of each text unit being the end position. The similarity obtaining unit is further configured to, for the target text unit, obtain second semantic similarity information between each candidate answer taking the target text unit as the end position and the labeled answer. The probability determining unit is further configured to determine a second probability of the target text unit based on the second semantic similarity information. The probability distribution determining unit is further configured to determine the second probability distribution based on the second probabilities of the text units, the second probability distribution comprising a second probability of each text unit being the end position.

14. The apparatus of claim 12 or 13, wherein, The similarity determining module comprises: The word overlap degree determining unit is configured to, for a target candidate answer in the at least one candidate answer, determine a word overlap degree between the target candidate answer and the labeled answer. The similarity determining unit is configured to obtain the similarity information based on the word overlap degrees between the at least one candidate answer and the labeled answer.

15. The apparatus of claim 14, wherein, The similarity information comprises semantic similarity between the at least one candidate answer and the labeled answer, and the similarity determining unit comprises: The semantic similarity determining subunit is configured to obtain the semantic similarity based on the word overlap degrees between the at least one candidate answer and the labeled answer. The similarity matrix generating subunit is configured to generate a similarity matrix based on the semantic similarity. The value of an element of the similarity matrix is used to represent semantic similarity between a candidate answer and a labeled answer, the candidate answer taking a text unit corresponding to a row where the element is located as the start position, and the labeled answer taking a text unit corresponding to a column where the element is located as the end position.

16. The apparatus of claim 13, wherein, The model training module comprises: The label information determining unit is configured to, for the target text unit, take the first probability and the second probability of the target text unit as label information of the target text unit. The probability predicting unit is configured to, in a case where the label information of the text units is supervision information, input the sample question and the sample answer text into a target machine learning model to obtain first output probabilities and second output probabilities corresponding to the text units, the first output probability being a probability of predicting that each text unit is the start position, and the second output probability being a probability of predicting that each text unit is the end position. The model determining unit is configured to, in a case where the first output probability, the second output probability, and the label information of the text units satisfy a target loss constraint condition, obtain the answer abstract extraction model.

17. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the answer abstract extraction method according to any one of claims 1 to 2 or the training method of the answer abstract extraction model according to any one of claims 3 to 8. 18.A computer readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by a processor to implement the answer abstract extraction method according to any one of claims 1 to 2 or the training method of the answer abstract extraction model according to any one of claims 3 to 8.

19. A computer program product, characterised in that, The computer program product comprises computer instructions, which are loaded and executed by a processor to implement the answer abstract extraction method according to any one of claims 1 to 2 or the training method of the answer abstract extraction model according to any one of claims 3 to 8.

Images