Method, device, computer equipment and storage medium for determining the answer to the question

Abstract

This application provides a method, apparatus, computer device, and storage medium for determining the answer to a question, belonging to the field of natural language processing technology. The method includes: acquiring a sample question statement and multiple candidate answers corresponding to the sample question statement, each candidate answer having a label indicating whether it is the answer to the sample question statement; concatenating the sample question statement with the document to which each candidate answer belongs, obtaining multiple sample data, with the document marked with the position of the candidate answer; using the label of each candidate answer as supervision, training an answer determination model based on the multiple sample data, the answer determination model being used to determine the answer to the input question statement. Because documents are more complete and detailed in semantic information than candidate answers, the accuracy of the trained answer determination model is higher, and consequently, the accuracy of the answer obtained based on this answer determination model is also higher.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method, apparatus, computer device, and storage medium for determining the answer to a question. Background Technology

[0002] With the development of internet technology, users can search for answers to questions online. In related technologies, the user inputs a question, and the computer retrieves multiple documents related to the question from a large dataset. From each document, a fragment is extracted as a candidate answer, resulting in multiple candidate answers. Each candidate answer is then validated to determine if it answers the question, and the correct answer is returned to the user. Therefore, how to determine the correct answer to improve its accuracy has become a pressing problem. Summary of the Invention

[0003] This application provides a method, apparatus, computer device, and storage medium for determining the answer to a question, which can improve the accuracy of the answer. The technical solution is as follows:

[0004] On the one hand, a method for determining the answer to a question is provided, the method comprising:

[0005] Obtain a sample question statement and multiple candidate answers corresponding to the sample question statement. Each candidate answer has a label, which indicates whether the candidate answer is an answer to the sample question statement.

[0006] The sample question statement is concatenated with the document to which each candidate answer belongs to obtain multiple sample data, and the document is marked with the position of the candidate answer;

[0007] Using the label of each candidate answer as supervision, an answer determination model is trained based on the multiple sample data. The answer determination model is used to determine the answer to the input question statement.

[0008] Optionally, obtaining the probability corresponding to the candidate answer in the plurality of sample data through the answer determination model includes:

[0009] For each of the sample data, obtain the first word feature of the word sequence corresponding to the sample data;

[0010] The first word feature of the word sequence is input into the answer determination model to obtain the answer feature corresponding to the sample data;

[0011] Based on the answer features, the probability of candidate answers in the sample data is determined.

[0012] Optionally, the word sequence includes multiple words, the multiple words including multiple first words in the sample question statement and multiple second words in the document, and the first word feature of the word sequence includes the first word feature of each word;

[0013] The step of inputting the first word feature of the word sequence into the answer determination model to obtain the answer feature corresponding to the sample data includes:

[0014] The model is determined by the answer, and global attention processing is performed on the first word features of multiple first words in the sample question statement, and local attention processing is performed on the first word features of multiple second words in the document to obtain the second word features of each word.

[0015] From the second word features of each of the words, determine the answer features corresponding to the sample data.

[0016] Optionally, the plurality of words further includes a first delimiter and a second delimiter, and the first word feature of the word sequence further includes a second word feature of the first delimiter and the second delimiter, wherein the first delimiter is used to separate the sample question statement and the document, and the second delimiter is used to mark the position of the candidate answer in the document;

[0017] Before determining the answer features corresponding to the sample data from the second word features of each of the words, the method further includes:

[0018] The model is determined by the answer. Global attention processing is applied to the first word feature of the first separator to obtain the second word feature of the first separator. Global attention processing is applied to the first word feature of the second separator to obtain the second word feature of the second separator.

[0019] Optionally, determining the probability of candidate answers in the sample data based on the answer features includes:

[0020] The answer features are normalized to obtain the probability of candidate answers in the sample data.

[0021] On the other hand, a method for determining the answer to a question is provided, the method comprising:

[0022] Receive a question statement and obtain multiple candidate answers corresponding to the question statement;

[0023] The question statement is concatenated with the document to which each candidate answer belongs to obtain multiple target data, wherein the document contains the candidate answer and is marked with the position of the candidate answer;

[0024] Based on the multiple target data, the answer to the question statement is determined from the multiple candidate answers;

[0025] Output the answer to the question.

[0026] Optionally, determining the answer to the question statement from the multiple candidate answers based on the multiple target data includes:

[0027] Based on the multiple target data, the answer to the question statement is determined from the multiple candidate answers using an answer determination model;

[0028] The answer determination model is trained on multiple sample data with the labels of multiple candidate answers corresponding to the sample question statement as supervision. Each sample data is obtained by concatenating the sample question statement with the document to which the corresponding candidate answer belongs, and the document is marked with the position of the candidate answer.

[0029] Optionally, determining the answer to the question statement from the multiple candidate answers using an answer determination model based on the multiple target data includes:

[0030] The multiple target data are input into the answer determination model, and the probability of each candidate answer is output, whereby the probability represents the likelihood that the candidate answer is the answer to the question statement.

[0031] The candidate answer with the highest probability among the multiple candidate answers is determined as the answer to the question statement.

[0032] On the other hand, a device for determining the answer to a question is provided, the device comprising:

[0033] The acquisition module is used to acquire a sample question statement and multiple candidate answers corresponding to the sample question statement. Each candidate answer has a label, which is used to indicate whether the candidate answer is an answer to the sample question statement.

[0034] The concatenation module is used to concatenate the sample question statement with the document to which each candidate answer belongs, to obtain multiple sample data, wherein the document is marked with the position of the candidate answer;

[0035] A training module is used to train an answer determination model based on the multiple sample data, with the label of each candidate answer as supervision, the answer determination model being used to determine the answer to the input question statement.

[0036] Optionally, the splicing module is used to sequentially splice the sample question statement, the first separator, and the document to which each candidate answer belongs, to obtain the plurality of sample data, wherein the first separator is used to separate the sample question statement from the document.

[0037] Optionally, the splicing module is configured to, for each document to which the candidate answer belongs, obtain a second document segment from the document containing the candidate answer and a first document segment, with the segment length being a target length, wherein the target length is less than the length of the document; and splice the sample question statement with the second document segment to obtain the plurality of sample data; wherein the first document segment is a document segment in the document connected to the candidate answer and located before or after the candidate answer, and in the second document segment, a second separator mark is set between the first document segment and the candidate answer.

[0038] Optionally, the training module includes:

[0039] The acquisition unit is used to determine the model through the answer during the i-th iteration of training, and to acquire the probability corresponding to the candidate answer in the multiple sample data, wherein the probability represents the possibility that the candidate answer is the answer to the sample question statement, and i is a positive integer not less than 1;

[0040] A determining unit is configured to determine a loss value for the answer determining model based on the probability of each sample data and the label of the candidate answer, wherein the loss value is negatively correlated with the probability;

[0041] The training unit is used to perform the (i+1)th iteration process on the answer determination model based on the loss value when the loss value does not meet the loss value condition, and repeat the above training iteration process until the loss value meets the loss value condition.

[0042] Optionally, the acquisition unit is configured to, for each sample data, acquire the first word feature of the word sequence corresponding to the sample data; input the first word feature of the word sequence into the answer determination model to obtain the answer feature corresponding to the sample data; and determine the probability of the candidate answer in the sample data based on the answer feature.

[0043] Optionally, the word sequence includes multiple words, the multiple words including multiple first words in the sample question statement and multiple second words in the document, and the first word feature of the word sequence includes the first word feature of each word;

[0044] The acquisition unit is used to perform global attention processing on the first word features of multiple first words in the sample question statement and local attention processing on the first word features of multiple second words in the document through the answer determination model to obtain the second word features of each word; and to determine the answer features corresponding to the sample data from the second word features of each word.

[0045] Optionally, the plurality of words further includes a first delimiter and a second delimiter, and the first word feature of the word sequence further includes a second word feature of the first delimiter and the second delimiter, wherein the first delimiter is used to separate the sample question statement and the document, and the second delimiter is used to mark the position of the candidate answer in the document;

[0046] The acquisition unit is further configured to perform global attention processing on the first word feature of the first separator marker through the answer determination model to obtain the second word feature of the first separator marker, and perform global attention processing on the first word feature of the second separator marker to obtain the second word feature of the second separator marker.

[0047] Optionally, the determining unit is used to normalize the answer features to obtain the probability of candidate answers in the sample data.

[0048] On the other hand, a device for determining the answer to a question is provided, the device comprising:

[0049] The acquisition module is used to receive a question statement and acquire multiple candidate answers corresponding to the question statement;

[0050] The concatenation module is used to concatenate the question statement with the document to which each candidate answer belongs, to obtain multiple target data, wherein the document contains the candidate answer and marks the position of the candidate answer;

[0051] The determining module is used to determine the answer to the question statement from the multiple candidate answers based on the multiple target data;

[0052] The output module is used to output the answer to the question statement.

[0053] Optionally, the determining module is configured to determine the answer to the question statement from the multiple candidate answers based on the multiple target data and through an answer determination model;

[0054] The answer determination model is trained on multiple sample data with the labels of multiple candidate answers corresponding to the sample question statement as supervision. Each sample data is obtained by concatenating the sample question statement with the document to which the corresponding candidate answer belongs, and the document is marked with the position of the candidate answer.

[0055] Optionally, the determining module is configured to input the plurality of target data into the answer determining model, output the probability of each candidate answer, wherein the probability represents the likelihood that the candidate answer is the answer to the question statement; and determine the candidate answer with the highest probability among the plurality of candidate answers as the answer to the question statement.

[0056] On the other hand, a computer device is provided, the computer device including one or more processors and one or more memories, the one or more memories storing at least one computer program, the computer program being loaded and executed by the one or more processors to implement the method for determining the answer to the problem described in the foregoing aspects.

[0057] On the other hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer storage medium, the computer program being loaded and executed by a processor to implement the method for determining the answer to the problem described above.

[0058] On the other hand, a computer program product is provided, the computer program product including a computer program stored in a computer-readable storage medium, a processor of a computer device reading the computer program from the computer-readable storage medium, the processor executing the computer program, causing the computer device to perform a method for determining the answer to the problem described above.

[0059] In this embodiment, by concatenating the sample question statement with the document to which the candidate answer belongs, and the document also marks the position of the candidate answer, the answer determination model refers to the semantic information of the document to which the candidate answer belongs when determining whether the candidate answer is the answer to the question statement based on the sample data. Since the document is more complete and detailed in semantic information than the candidate answer, the accuracy of the trained answer determination model is higher, and thus the accuracy of the answer obtained based on the answer determination model is also higher. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of the implementation environment of a method for determining the answer to a question provided in an embodiment of this application;

[0062] Figure 2This is a flowchart of a method for determining the answer to a question according to an embodiment of this application;

[0063] Figure 3 This is a flowchart of another method for determining the answer to a question according to an embodiment of this application;

[0064] Figure 4 This is a schematic diagram of the attention of an encoder according to an embodiment of this application;

[0065] Figure 5 This is a schematic diagram illustrating the training process of an answer determination model according to an embodiment of this application;

[0066] Figure 6 This is a flowchart of another method for determining the answer to a question according to an embodiment of this application;

[0067] Figure 7 This is a flowchart of another method for determining the answer to a question according to an embodiment of this application;

[0068] Figure 8 This is a schematic diagram illustrating a process for determining the answer to a question according to an embodiment of this application;

[0069] Figure 9 This is a schematic diagram of a device for determining the answer to a problem according to an embodiment of this application;

[0070] Figure 10 This is a schematic diagram of the structure of a device for determining the answer to another question provided in the embodiments of this application;

[0071] Figure 11 This is a schematic diagram of the structure of a terminal according to an embodiment of this application;

[0072] Figure 12 This is a schematic diagram of the structure of a server according to an embodiment of this application. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0074] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "n," nor are they limited in quantity or execution order.

[0075] In this application, the term "at least one" means one or more, and "multiple" means two or more.

[0076] The following is an explanation of the terms used in this application.

[0077] Question answering (QA): For a user-provided question in free text format, QA returns a concise and accurate matching answer by searching a corpus, knowledge graph, or question-and-answer knowledge base. QA systems can be categorized into knowledge-based question answering (KBQA), text-based question answering (DocQA), such as machine reading comprehension (MRC), and question-and-answer pair-based systems, such as frequently asked questions (FAQ).

[0078] Machine reading comprehension: This refers to the technology that uses algorithms to enable computers to understand the semantics of articles and answer related questions. Common question types include fill-in-the-blank, multiple-choice, and open-ended questions. In DocQA, a fragment extraction-based reading comprehension technique is often used, extracting consecutive fragments from the relevant document as candidate answers.

[0079] Answer selection is an important task in QA. The process is defined as determining whether candidate answers extracted from a document are a true answer to the question, i.e., whether the candidate answers can answer the question.

[0080] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0081] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0082] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0083] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0084] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, smart customer service, vehicle networking, and intelligent transportation. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0085] The solutions provided in this application involve technologies such as natural language processing in artificial intelligence, which are specifically illustrated in the following embodiments.

[0086] The computer device provided in this application embodiment can be provided as a terminal or a server. The implementation environment consisting of a terminal and a server will be described below.

[0087] Figure 1 This is a schematic diagram illustrating the implementation environment of a method for determining the answer to a question according to an embodiment of this application. See also... Figure 1 The implementation environment includes a terminal 110 and a server 120. The terminal 110 is connected to the server 120 via a wireless network or a wired network.

[0088] Optionally, terminal 110 may be a smartphone, tablet, laptop, desktop computer, or smartwatch, but is not limited to these. Optionally, terminal 110 may have an application installed and running that supports determining the answer to a question. This application may be an information search application.

[0089] Optionally, server 120 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0090] Server 120 is used to provide background services for the application that determines the answer to the question. Optionally, server 120 may undertake the main computing work and terminal 110 may undertake the secondary computing work; or, server 120 may undertake the secondary computing work and terminal 110 may undertake the main computing work; or, server 120 and terminal 110 may collaborate on computing using a distributed computing architecture.

[0091] Optionally, terminal 110 generally refers to one of a plurality of terminals, and this application embodiment uses terminal 110 as an example for illustration. Those skilled in the art will know that the number of terminals 110 may be more or less. For example, there may be one terminal 110, or there may be dozens or hundreds of terminals 110, or even more, in which case other terminals may also be included in the above implementation environment. This application embodiment does not limit the number of terminals or the type of device.

[0092] After introducing the implementation environment of the question answer determination method provided in the embodiments of this application, the application scenarios of the question answer determination method provided in the embodiments of this application will be described below. It should be noted that the terminal in the following description is terminal 110 in the above implementation environment, and the server is server 120 in the above implementation environment. The question answer determination method provided in the embodiments of this application can be applied to various question answer determination scenarios, such as the application to the question answer determination scenario of medical questions, and this application embodiment does not limit it.

[0093] Taking the medical field as an example, the server provides a vertical search engine for the medical field. Based on the user's question, the server filters, mines, understands, and sorts relevant content within the medical field to determine the answer, thus providing the user with a reliable and convenient information search service. Similar to general search engines, the server's knowledge sources are diverse, including but not limited to medical journals, papers, various forums and websites on the internet, and direct documents from doctors. Although these data are all in text format, their formatting and length vary. After determining the answer, the server returns the webpage link corresponding to the document containing that answer to the terminal. The user needs to click on the webpage link displayed on the terminal to find the document segment they need from text of varying lengths. Correspondingly, the server can extract document segments directly related to the question from the document, i.e., the long text, as a webpage summary for the terminal to display to the user, thereby significantly improving the user's efficiency in obtaining information.

[0094] The server employs question-answering technology, modeling the aforementioned information search process as a question-answering system. Accordingly, the server uses a pipelined approach to implement the question-answering system, which consists of three steps: retrieval, extraction, and answer verification. Retrieval involves searching for documents relevant to the question, a fundamental function of the search system. Extraction uses machine reading comprehension technology to extract a short document fragment related to the question from each webpage as a summary. Answer verification validates the extracted summary to determine if it is the answer to the question, thus eliminating irrelevant document fragments and preventing the terminal from displaying invalid information.

[0095] For example, when a user wants to search for information through a terminal, they enter a question in the information search application on the terminal. The terminal obtains the question and sends it to the server. The server uses the question answer determination method provided in the embodiments of this application to determine the answer to the question and sends the answer to the terminal. The terminal displays the answer for the user to view, thereby completing the information search process.

[0096] After introducing the implementation environment and application scenarios of the embodiments of this application, the method for determining the answer to the question provided in the embodiments of this application will be described below. In the embodiments of this application, the method for determining the answer to the question provided in the embodiments of this application can be implemented by a server or a terminal as the execution subject, or it can be implemented through the interaction between the terminal and the server. The terminal is the terminal 110 in the above implementation environment, and the server is the server 120 in the above implementation environment. The embodiments of this application do not limit the execution subject.

[0097] In some embodiments, the server trains and stores the trained answer determination model; when an answer to a question needs to be determined, the server retrieves the question sent by the terminal, determines the answer using the trained answer determination model, and returns the answer to the terminal, thereby saving the terminal's workload. In other embodiments, the server sends the trained answer determination model to the terminal, and the terminal stores the answer determination model so that it can directly call the answer determination model to determine the answer when an answer to a question needs to be determined.

[0098] Figure 2 This is a flowchart of a method for determining the answer to a question according to an embodiment of this application. See also... Figure 2 In this embodiment of the application, the method is executed by the server and includes:

[0099] 201. The server retrieves the sample question statement and multiple candidate answers corresponding to the sample question statement. Each candidate answer has a label, which is used to indicate whether the candidate answer is the answer to the sample question statement.

[0100] Sample question statements are used to train the answer determination model. Examples of sample question statements include "Which department should I see for a liver checkup?", "How can I relieve a headache?", or "Will eating fried chicken make me fat?". The multiple candidate answers to a sample question statement are all related to that sample question statement, but may not be the actual answer.

[0101] The way candidate answer labels are represented can be set as needed, and this application embodiment does not limit this. For example, candidate answer labels can be represented by 1 or 0, where 1 represents that the candidate answer is the answer to the sample question statement, and 0 represents that the candidate answer is not the answer to the sample question statement. Taking the sample question statement "Which department should I see for a liver checkup?" as an example, if multiple candidate answers include candidate answer A "See a gastroenterologist for a liver checkup" and candidate answer B "See a surgeon", then candidate answer A is labeled as 1, and candidate answer B is labeled as 1.

[0102] 202. The server concatenates the sample question statement with the document to which each candidate answer belongs, resulting in multiple sample data, with the document marked with the position of the candidate answer.

[0103] The candidate answers are extracted from documents related to the sample question statement; these documents are also the documents to which the candidate answers belong. Through the above concatenation process, the sample data can be made to reflect the true semantics of the candidate answers.

[0104] 203. The server uses the label of each candidate answer as supervision and trains the answer determination model based on multiple sample data. The answer determination model is used to determine the answer to the input question statement.

[0105] It should be noted that, since the questions belong to a variety of fields, such as the medical field, the life field, the smart home field, or the mechanical field, the server can train a corresponding answer determination model for each field to improve the accuracy of the answer determination model. Thus, the server trains the answer determination model for that field based on sample question statements belonging to the same field.

[0106] In this embodiment, by concatenating the sample question statement with the document to which the candidate answer belongs, and the document also marks the position of the candidate answer, the answer determination model refers to the semantic information of the document to which the candidate answer belongs when determining whether the candidate answer is the answer to the question statement based on the sample data. Since the document is more complete and detailed in semantic information than the candidate answer, the accuracy of the trained answer determination model is higher, and thus the accuracy of the answer obtained based on the answer determination model is also higher.

[0107] In this embodiment, the training of the answer determination model includes multiple iterations. In the first iteration, the server inputs sample data into the answer determination model to obtain the prediction result of the first iteration. Based on the prediction result of the first iteration, a loss value is determined, and the model parameters are adjusted based on the loss value. The adjusted model parameters from the first iteration are used as the model parameters for the second iteration, and the second iteration is performed. This iterative process is repeated multiple times until the training meets the target conditions, resulting in a well-trained answer determination model.

[0108] In some embodiments, the target condition for training is that the number of training iterations of the model reaches a target number, which is a preset number of training iterations, such as 1000 times; or, the target condition for training is that the loss value meets a loss value condition, such as the loss value being less than 0.0001. This application does not limit the setting of the target condition. In this application embodiment, the example of the target condition being that the loss value meets the loss value condition is used for illustration.

[0109] Figure 3 This is a flowchart of a method for determining the answer to a question according to an embodiment of this application, such as... Figure 3 As shown, this embodiment of the application uses the i-th iteration of the server training the answer determination model based on sample data as an example for illustration, where i is a positive integer not less than 1. The method includes:

[0110] 301. The server retrieves the sample question statement and multiple candidate answers corresponding to the sample question statement. Each candidate answer has a label, which is used to indicate whether the candidate answer is the answer to the sample question statement.

[0111] Optionally, the sample question statements are retrieved by the server from a database. The database stores a large number of question statements, and the server determines the sample question statements for the answer determination model from these databases. The question statements in the database can be obtained by the server beforehand from historical information search processes. During these historical information search processes, users input question statements through their terminals, which then send the question statements to the server. The server receives the question statements and stores them in the database.

[0112] After obtaining the sample question statement, the server retrieves multiple candidate answers to the sample question statement. Optionally, the multiple candidate answers to the sample question statement come from the server's retrieval and extraction process. Accordingly, the server searches based on the sample question statement to obtain multiple documents related to the sample question statement; it then extracts from each document to obtain multiple candidate answers corresponding to the sample question statement.

[0113] The embodiments of this application do not limit the implementation method of the server for searching and extracting. For example, during the search, the server can perform understanding operations such as paraphrasing the query to recall as many relevant documents as possible. Alternatively, for each document found, the server can use the MRC extraction method to extract a document fragment from the document as a candidate answer.

[0114] In some embodiments, the server directly uses the extracted candidate answers as the answers to the sample question statement. Alternatively, the server filters the extracted candidate answers to obtain multiple candidate answers corresponding to the sample question statement. The server can filter based on a relevance parameter between the sample question statement and the document. This relevance parameter represents the degree of semantic association between the sample question statement and the document.

[0115] Optionally, for the extracted candidate answers, the server determines a relevance parameter between the sample question statement and the document to which the candidate answer belongs; from the extracted candidate answers, it determines the candidate answers corresponding to documents whose relevance parameter is greater than a relevance threshold, thereby obtaining multiple candidate answers corresponding to the sample question statement. Optionally, the server determines the relevance parameter between the sample question statement and the document title as the relevance parameter between the sample question statement and the document.

[0116] The relevance threshold can be set as needed, and this embodiment does not limit it. A relevance parameter greater than the relevance threshold indicates that the sample question statement is relatively relevant to the document, while a relevance parameter less than the relevance threshold indicates that the sample question statement is relatively unrelated to the document. Therefore, by setting a relevance threshold, multiple candidate answers corresponding to documents with a high relevance parameter to the sample question statement can be selected from the extracted candidate answers, thereby improving the accuracy of the candidate answers.

[0117] This application does not limit the number of candidate answers in the embodiments. For example, the server selects 5 candidate answers from the extracted candidate answers.

[0118] It should be noted that the server can also determine multiple candidate answers to a sample question statement in other ways. For example, during a historical information search, after receiving a question statement from a terminal, the server obtains multiple candidate answers to the question statement and stores the question statement and multiple candidate answers in the database so that the server can directly retrieve the sample question statement and its corresponding multiple candidate answers from the database in the future.

[0119] Optionally, for each candidate answer, the server manually labels the candidate answer to obtain the label of the candidate answer.

[0120] 302. The server concatenates the sample question statement with the document to which each candidate answer belongs, resulting in multiple sample data, with the document marked with the position of the candidate answer.

[0121] In some embodiments, for each document to which a candidate answer belongs, the server directly concatenates the sample question statement with the document to obtain multiple sample data. Alternatively, the server can also add a preset string to the sample data. Accordingly, for each document to which a candidate answer belongs, the server concatenates the preset string, the sample question statement, and the document to obtain multiple sample data. The sample data is represented as a text sequence. The preset string is a special pre-defined marker; during the training of the answer determination model, the features corresponding to the preset string aggregate the overall semantic information of the sample question statement and the document in the sample data.

[0122] In this embodiment, by concatenating a preset string, sample data, and a document, the concatenated sample data incorporates the entire document information on top of the candidate answer. As a result, the answer determination model can learn the contextual information near the candidate answer during training, thereby improving the accuracy of the trained model.

[0123] In some embodiments, the server does not restrict the concatenation order of the preset string, sample question statements, and document. When the concatenation order is not restricted, the concatenation order of the multiple sample question statements used for training the answer determination model is the same, so that the answer determination model can distinguish them. Optionally, the server concatenates the preset string, sample question statements, and document to obtain multiple sample data in the following ways: the server concatenates the preset string, sample question statements, and document sequentially to obtain multiple sample data. For example, if the preset string is [CLS], the concatenated sample data is represented as "[CLS]sample question statement / document". For ease of representation, a " / " is added between the sample question statement and the document; it should be noted that the " / " is not included in the concatenated sample data obtained by the server.

[0124] In this embodiment of the application, by concatenating the preset string, sample question statement and document in sequence, the answer determination model can distinguish each piece of data in the sample data, thereby improving the accuracy of the sample data.

[0125] In some embodiments, the server may separate the sample question statement and the document. Optionally, the server concatenates the sample question statement, the first delimiter, and the document sequentially to obtain multiple sample data, where the first delimiter is used to separate the sample question statement and the document. For example, if the first delimiter is [SEP], then the sample data is represented as "sample question statement [SEP] document".

[0126] The server can also concatenate a preset string, a sample question statement, a first delimiter, and a document sequentially to obtain sample data. For example, if the preset string is [CLS] and the first delimiter is [SEP], then the sample data is represented as "[CLS] Sample Question Statement [SEP] Document".

[0127] The server can also set a first separator mark after the document, then the sample data is represented as "[CLS]Sample Question Statement [SEP]Document [SEP]". In this embodiment of the application, the document that marks the position of the candidate answer is used as the "context", then the sample data is represented as "[CLS]Sample Question Statement [SEP]Context [SEP]".

[0128] In this embodiment, a first delimiter is used to separate the sample question statement and the document, enabling the answer determination model to distinguish between the sample question statement and the document, thereby improving the accuracy of the sample data.

[0129] In this embodiment, the server can set a first separator between the sample question statement and the document in each sample data, or the server can determine whether to set the first separator based on the document length. Accordingly, for the same sample question statement, if the documents to which each candidate answer belongs have different lengths, the server sets a first separator between the sample question statement and the document in the sample data to facilitate differentiation by the answer determination model. If the documents to which each candidate answer belongs have the same length, the server does not need to set a first separator between the sample question statement and the document in the sample data.

[0130] Optionally, after obtaining multiple candidate answers to the sample question statement, the server manually marks the position of each candidate answer in the document to which it belongs, so that the answer determination model can distinguish the candidate answer from other content in the document during training. Accordingly, the server sets a second separator in the document to mark the position of the candidate answer. The second separator is set at the beginning or end of the candidate answer. For example, if the second separator is [A], the document is represented as "document fragment [A]candidate answer [\A]document fragment", where a document fragment refers to the part of the document located near and connected to the candidate answer. Taking the preset string [CLS] and the first separator as [SEP] as an example, the sample data is represented as "[CLS]sample question statement [SEP]document fragment [A]candidate answer [\A]document fragment [SEP]".

[0131] The server employs a simple yet effective method to incorporate document information into candidate answers: it uses a second delimiter to mark the position of candidate answers within the document, thereby preserving the context of candidate answers in the sample data.

[0132] In some embodiments, the document containing the candidate answer is relatively long, and its length may vary for different documents. Since the input data length of the answer determination model is limited, the server can extract a document fragment containing the candidate answer from the document and concatenate the sample question statement with this document fragment to obtain sample data, thereby reducing the amount of sample data. Accordingly, the server concatenates the sample question statement with the document containing each candidate answer to obtain multiple sample data sets. This is achieved by: for each document containing a candidate answer, the server extracts a second document fragment containing the candidate answer and a first document fragment, with a fragment length equal to a target length (less than the document length); the sample question statement is then concatenated with the second document fragment to obtain multiple sample data sets.

[0133] In this embodiment, the first document fragment is a segment within a document that is connected to the candidate answer and located before or after it. In the second document fragment, a second separator is placed between the first document fragment and the candidate answer. This second separator marks the position of the candidate answer within the document. The target length can be set as needed, and this application does not limit it. In this embodiment, by obtaining the second document fragment from the document, the sample data includes both the candidate answer and the context of the first document fragment (i.e., the candidate answer), thus reducing the amount of sample data while ensuring its accuracy.

[0134] In this embodiment, the server obtains multiple sample data by executing steps 301-302. Steps 301-302 can be regarded as the process of acquiring training data for the answer determination model, i.e., the data preprocessing process. Subsequently, the server can perform multiple iterations to train the answer determination model based on multiple sample data. This embodiment takes the example of sample data being obtained by sequentially concatenating a preset string, a sample question statement, a first separator, and a document with the first separator. The document is obtained by sequentially concatenating a document fragment, a second separator, a candidate answer, and the second separator with the document fragment. For example, the sample data is "[CLS]Sample Question Statement[SEP]Document Fragment[A]Candidate Answer[\A]Document Fragment[SEP]".

[0135] 303. During the i-th iteration of training, the server obtains the first word feature of the word sequence corresponding to each sample data for each sample data.

[0136] The server performs word segmentation on the sample data to obtain a word sequence, which includes multiple words. Optionally, when the sample data contains sample question statements and documents, the multiple words include multiple first words from the sample question statements and multiple second words from the documents. If the sample data also includes at least one of a preset string, a first delimiter, or a second delimiter, then the multiple words in the word sequence also include at least one of the preset string, the first delimiter, or the second delimiter.

[0137] The answer determination model is configured with a vocabulary, which stores words and their features. After obtaining the word sequence of the sample data, the server uses the vocabulary to obtain the feature of each word among multiple words, and uses this feature as the first feature of the word. Therefore, the first feature of the word sequence includes the first feature of each word.

[0138] It should be noted that, since the sample data may include a preset string, a first delimiter, or a second delimiter, the server pre-adds these to the vocabulary and encodes them to obtain word features. This prevents the server from treating the sample data as a regular string during word segmentation, thus losing its tagging function. For example, if the second delimiter is segmented as a regular string, it will lose its function of indicating the answer's location.

[0139] In this embodiment, the answer determination model includes an encoder. Since Chinese characters cannot be directly used as input data for the encoder, the server first encodes multiple words corresponding to the word sequence according to a vocabulary to obtain the first word feature, and then inputs the first word feature into the encoder. Optionally, the first word feature is represented in the form of an embedding vector.

[0140] 304. The server inputs the first word feature of the word sequence into the answer determination model to obtain the answer features corresponding to the sample data.

[0141] The answer determination model processes the first-word features based on an attention mechanism. Optionally, when multiple words include multiple first words in the sample question statement and multiple second words in the document, the server inputs the first-word features of the word sequence into the answer determination model to obtain the answer features corresponding to the sample data. The implementation method includes: the server uses the answer determination model to perform global attention processing on the first-word features of multiple first words in the sample question statement and local attention processing on the first-word features of multiple second words in the document to obtain the second-word features of each word; from the second-word features of each word, the answer features corresponding to the sample data are determined.

[0142] In this embodiment, the encoder of the answer determination model is an attention-based encoder, and the server processes the first word features through the encoder. Because the documents introduced in the sample data are long, meaning the documents contain a large number of second words, the answer determination model needs to process a large number of first word features during the answer feature determination process. Therefore, by performing global attention processing on the first word features of the first word in the sample question statement and local attention processing on the first word features of the second words in the document, global attention processing is not required for multiple words. This significantly reduces data processing complexity and improves data processing efficiency.

[0143] Optionally, if multiple words also include a preset string, then the first word feature of the word sequence also includes the first word feature of the preset string. The server can then use the answer determination model to perform global attention processing on the first word feature of the preset string to obtain the second word feature of the preset string. In this embodiment, by performing global attention processing on the first word feature of the preset string, the preset string can also be aware of global semantic information. Since the preset string is unrelated to the sample question statement and document, the second word feature of the preset string can represent global semantic information more unbiasedly, thereby improving the accuracy of the second word feature of the preset string.

[0144] Optionally, if multiple words also include a first delimiter and a second delimiter, then the first word feature of the word sequence also includes the second word feature of the first delimiter and the second delimiter; then the server can also use the answer determination model to perform global attention processing on the first word feature of the first delimiter to obtain the second word feature of the first delimiter, and perform global attention processing on the first word feature of the second delimiter to obtain the second word feature of the second delimiter.

[0145] In this embodiment, by performing global attention processing on the first word features of both the first and second delimiters, these two delimiters can also notice global semantic information. Since the delimiters are unrelated to the sample question statements and documents, the second word features of the delimiters can represent global semantic information in a relatively unbiased manner, thereby improving the accuracy of the second word features of the delimiters.

[0146] In some embodiments, the encoder of the answer determination model is a Longformer (an encoder for processing long text sequences). The server encodes the first-word features of multiple words using the Longformer to obtain the second-word features of each word. To improve model processing speed and enhance the ability to model long text sequences, the Longformer modifies the self-attention mechanism in the Transformer (an encoder), changing it from global attention to local attention, while retaining global attention for some specific tokens (words) to collect global semantic information. See also... Figure 4 The diagram shows the attention of the encoder, which is a Longformer. For ordinary tokens, it only pays attention to some tokens near itself, such as multiple second words in a document; while for special tokens, it needs to pay attention to global semantic information, such as multiple first words in the sample question statement, the preset string, the first delimiter, and the second delimiter.

[0147] In some embodiments, the second word features of multiple words in a word sequence are obtained through a global attention mechanism. The server determines the answer features corresponding to the sample data from the second word features of each word by using the second word features of a preset string as the answer features corresponding to the sample data. In this embodiment, since the preset string can capture the semantic information of each word, the second word features of the preset string can be determined as the representation of the entire semantic vector of the sample data, i.e., the answer features. This saves data processing power for subsequent answer determination models while ensuring sufficient feature accuracy.

[0148] Alternatively, the server can determine the answer features corresponding to the sample data from the second word features of each word by fusing the second word features of the preset string, the first word in the sample question statement, and the second word features of each second word in the document to obtain the answer features. The fusion method can be set as needed, such as taking the average value. In this embodiment, the obtained answer features not only incorporate the entire semantic vector of the sample data represented by the preset string but also the semantic information of each word, thus determining the answer features with reference to more semantic information and greater accuracy.

[0149] It should be noted that the encoder in the answer determination model can be a pre-trained encoder, which can improve the training accuracy when training the answer determination model.

[0150] 305. Based on answer features, the server determines the probability of candidate answers in the sample data. The probability represents the likelihood that a candidate answer is the answer to the sample question statement.

[0151] In some embodiments, the server determines the probability of candidate answers in sample data based on answer features by performing normalization processing on the answer features to obtain the probability of candidate answers in the sample data. The answer determination model further includes a fully connected layer and a softmax (normalization) layer. The answer determination model first performs fully connected processing on the answer features, and then normalizes the answer features obtained from the fully connected layer. In this embodiment, by normalizing the answer features, the probability of a candidate answer can represent the likelihood that the candidate answer is the answer to the sample question statement, thereby enabling binary classification of candidate answers based on the probability.

[0152] In this embodiment of the application, the training process of the answer determination model is the process of verifying multiple candidate answers to a sample question statement. The server models the answer verification process as a binary classification task through the answer determination model, using the sample question statement and the document to which the candidate answer belongs as a training sample to determine the probability that the candidate answer is the true answer to the sample question statement.

[0153] In some embodiments, steps 303-305 represent a method by which the server, during the i-th iteration of training, determines the probability of candidate answers in the sample data through the answer determination model. The server may use other methods to obtain the probability of candidate answers in the sample data through the answer determination model, and this application embodiment does not limit this method.

[0154] In this embodiment of the application, the answer determination model encodes multiple words in the word sequence corresponding to the sample data to obtain the answer features of the candidate answer. Since the answer features integrate the semantic information of the sample data, and the document in the sample data not only contains the candidate answer, but also the context around the candidate answer, the answer features can be integrated with more semantic information in the document. Therefore, the accuracy of the probability of the candidate answer determined by the answer determination model based on the answer features is high.

[0155] 306. The server determines the loss value of the answer determination model based on the probability of each sample data and the label of the candidate answer. The loss value is negatively correlated with the probability.

[0156] The server uses a loss function based on the probability and label of each candidate answer to determine the corresponding loss value. Optionally, the server uses binary cross-entropy as the loss function for the answer determination model, and the loss function formula is as follows:

[0157] L=-(y·log p+(1-y)·log(1-p))

[0158] Where L represents the loss value, y represents the label of the candidate answer, y takes the value 0 or 1, y = 1 means that the candidate answer is the answer to the sample question statement, y = 0 means that the candidate answer is not the answer to the sample question statement, and p represents the probability, p ∈ (0, 1).

[0159] It should be noted that since one sample question statement corresponds to multiple sample data, and each iteration of the answer determination model can input one sample data or multiple corresponding sample data, this embodiment does not limit this. When one sample data is input in the i-th iteration, the loss value of the answer determination model is the loss value of the candidate answers in that sample data. When multiple sample data are input in the i-th iteration, the loss value of the answer determination model can be determined based on the loss value of each candidate answer in the multiple sample data. For example, the sum of the loss values ​​of multiple candidate answers can be used as the loss value of the answer determination model; another example is to use the average of the loss values ​​of multiple candidate answers as the loss value of the answer determination model.

[0160] It should be noted that the training process of the answer determination model requires a large amount of training data. Therefore, the number of sample question statements used in the training is relatively large. This application embodiment takes one sample question statement as an example for illustration. When the training data includes multiple sample question statements and each sample question statement corresponds to multiple sample data, the training process of the answer determination model includes multiple similar iterative processes, which will not be elaborated in this application embodiment.

[0161] It should be noted that the server can also determine the loss value through other loss functions, and this application does not limit this.

[0162] 307. If the loss value does not meet the loss value condition, the server performs the (i+1)th iteration process on the answer determination model based on the loss value, and repeats the above training iteration process until the loss value meets the loss value condition.

[0163] Optionally, the server adjusts the model parameters based on the loss value; based on the adjusted model parameters, the i+1th iteration of the answer determination model is performed.

[0164] 308. When the loss value meets the loss value condition, the server stops iterating and obtains the trained answer to determine the model.

[0165] In some embodiments, steps 303-308 are an implementation of the server training an answer determination model based on multiple sample data, with the label of each candidate answer as supervision.

[0166] In this embodiment, the candidate answer extracted from the document is a short document fragment, which loses some context that plays a crucial role in determining the answer to the question. For example, for the question "What department should I see to check my liver?", the candidate answer extracted from the document is "gastroenterology," which is a correct answer to the question. However, without context, the server has difficulty determining whether this candidate answer is a correct answer.

[0167] This application embodiment introduces document information into the sample data to supplement the context of the candidate answer, that is, the document to which the candidate answer belongs. Furthermore, this application embodiment can also encode the sample data through an encoder used to process long sequences in the answer determination model. Therefore, although this application embodiment introduces long document information, it can also enhance the long sequence modeling capability of the answer determination model through an encoder such as Longformer, thereby improving the accuracy of answer determination.

[0168] This invention improves the effectiveness of answer verification by introducing document information to complete the context. This is because the answers extracted by the model may have missing context or be incomplete, which requires combining the overall information of the document to better judge the correctness of the extracted answers.

[0169] For example, see Figure 5 The training process of the answer determination model is shown in the diagram. The server inputs the sample data "[CLS] Sample Question Statement [SEP] Document Fragment [A] Candidate Answer [\A] Document Fragment [SEP]" into the encoder Longformer, and inputs the encoded answer features into the classification layer.

[0170] In some embodiments, referring to Table 1, the baseline method (Method 1) is compared with the question answer determination method (Method 2) provided in this application embodiment under the conditions of having an answer, having no answer, having a correct answer, and having an incorrect answer. The evaluation metrics include precision, recall, and F1. F1 refers to the fuzzy matching result; F1 is the harmonic mean of precision and recall, i.e., F1 = precision × recall × 2 / (precision + recall).

[0171] The method provided in this application takes into account the context near the candidate answer, while the baseline method only considers the candidate answer. That is, the baseline method directly concatenates the sample question statement and the candidate answer to obtain the sample data.

[0172] As can be seen, the method provided in this application aims to improve the accuracy of identifying the correct answer (+3.5%) while ensuring the recall of correct answers. In particular, the method provided in this application can effectively correct misjudgments when the extracted candidate answers lack context or when different candidate answers contain a lot of common information.

[0173] Table 1

[0174]

[0175]

[0176] For example, referring to Table 2, the question is "Will eating French fries occasionally make me fat?" The extracted candidate answer is "Eating them occasionally will not make you fat. Fat gain is caused by eating unhealthy food for a long time and lacking exercise." On its own, this candidate answer may answer the question, but in the context, we know that the content of the document is "Will eating chicken cutlets occasionally make me fat?". Therefore, the answer determination model trained by the embodiments of this application determines this candidate answer as an answer that cannot answer the question.

[0177] Table 2

[0178]

[0179] Here, 1 indicates that the candidate answer is the answer to the question, and 0 indicates that the candidate answer is not the answer to the question.

[0180] In this embodiment, by concatenating the sample question statement with the document to which the candidate answer belongs, and marking the position of the candidate answer in the document, the answer determination model refers to the semantic information of the document to which the candidate answer belongs when determining whether the candidate answer is the answer to the question statement based on the sample data. Since the semantic information of the document is more complete and detailed than the semantic information of the candidate answer itself, the answer determination model trained with the sample data has higher accuracy, and thus the answer obtained by determining the answer based on the answer determination model has higher accuracy.

[0181] Figure 6 This is a flowchart of a method for determining the answer to a question according to an embodiment of this application, such as... Figure 6 As shown, this embodiment of the application is executed by a server. The method includes:

[0182] 601. The server receives the question statement and obtains multiple candidate answers corresponding to the question statement.

[0183] Optionally, the user enters a question statement in the terminal, the terminal obtains the question statement and sends it to the server, and the server receives the question statement.

[0184] 602. The server concatenates the question statement with the document to which each candidate answer belongs, resulting in multiple target data. The documents contain candidate answers and are marked with the positions of the candidate answers.

[0185] 603. The server determines the answer to the question statement from multiple candidate answers based on multiple target data.

[0186] 604. The server outputs the answer to the question.

[0187] In this embodiment of the application, the target data is obtained by concatenating the question statement with the document to which the candidate answer belongs. In the process of determining the answer based on the target data, not only the semantic information of the candidate answer is referenced, but also the semantic information of the surrounding context of the candidate answer. Since the referenced semantic information is relatively rich, the accuracy of the determined answer is high.

[0188] In this embodiment of the application, an answer determination model trained using sample data is used to determine the answer to a question. Figure 7 This is a flowchart of a method for determining the answer to a question according to an embodiment of this application, such as... Figure 7As shown, this embodiment of the application illustrates the method of a server determining the answer to a question using an answer determination model. The method includes:

[0189] 701. The server receives the question statement and obtains multiple candidate answers corresponding to the question statement.

[0190] The number of candidate answers can be set as needed, and this application embodiment does not limit this. In some embodiments, the implementation method of the server obtaining multiple candidate answers to the question statement is the same as the implementation method of the server obtaining multiple candidate answers to the sample question statement in step 301. For example, the multiple candidate answers to the question statement come from the server's retrieval and extraction process, which will not be described in detail here.

[0191] 702. The server concatenates the question statement with the document to which each candidate answer belongs, resulting in multiple target data. The documents contain candidate answers and are marked with the positions of the candidate answers.

[0192] Optionally, the implementation of step 702 is similar to that of step 302, and will not be described again in this embodiment.

[0193] 703. The server uses multiple target data and an answer determination model to determine the answer to the question statement from multiple candidate answers.

[0194] The answer determination model is trained on multiple sample data, with the labels of multiple candidate answers corresponding to the sample question statement as supervision. Each sample data is obtained by concatenating the sample question statement with the document to which the corresponding candidate answer belongs, and the document is marked with the position of the candidate answer.

[0195] In some embodiments, the answer determination model is achieved through... Figure 3 The training obtained from the illustrated embodiment will not be repeated here.

[0196] Optionally, the server determines the answer to the question statement from multiple candidate answers based on multiple target data through an answer determination model. The implementation method includes: the server inputs multiple target data into the answer determination model and outputs the probability of each candidate answer, where the probability represents the likelihood that the candidate answer is the answer to the question statement; and the candidate answer with the highest probability among the multiple candidate answers is determined as the answer to the question statement.

[0197] The higher the probability of a candidate answer, the greater the likelihood that it is the answer to the question. Therefore, the candidate answer with the highest probability is the most likely candidate answer to the question. By determining the candidate answer with the highest probability as the answer to the question, the accuracy of the answer is higher.

[0198] In some embodiments, step 703 is an implementation of the server determining the answer to the question statement from multiple candidate answers based on multiple target data.

[0199] See Figure 8 The process of determining the answer to the question shown involves the server performing a preliminary process of retrieval and extraction based on the question statement, obtaining multiple candidate answers. Then, in the data processing stage, special labels are used to preserve the context around the candidate answers while also clearly indicating the position of the candidate answers in the document, i.e., concatenating the question statement with the document. The server then uses Longformer to perform semantic encoding on the processed sample data to obtain features, which are then used for subsequent multi-answer comparison, i.e., comparing the probabilities of multiple candidate answers corresponding to the same sample question statement, and finally obtaining the answer to the question statement.

[0200] 704. The server outputs the answer to the question.

[0201] In some embodiments, the question is sent by the terminal to the server. After determining the answer, the server returns the answer to the terminal so that the terminal can display the answer, allowing the user to view the answer through the terminal.

[0202] The answer determination model provided in this application can be used as part of the offline data collection module in information search. The answers determined by the answer determination model can be used for the summary display of online web page content on the terminal. Because the determined answers are relatively accurate, the user experience can be greatly improved. This application embodiment enriches the context of candidate answers by introducing document information, thereby improving the accuracy of answer determination when the extracted candidate answers lack context or are incomplete, thus ensuring that the displayed summary is accurate.

[0203] In this embodiment of the application, the target data is obtained by concatenating the question statement with the document to which the candidate answer belongs. In the process of determining the answer based on the target data, not only the semantic information of the candidate answer is referenced, but also the semantic information of the surrounding context of the candidate answer. Since the referenced semantic information is relatively rich, the accuracy of the determined answer is high.

[0204] Figure 9 This is a schematic diagram of a problem answer determining device according to an embodiment of this application. See also... Figure 9 The device includes:

[0205] The acquisition module 901 is used to acquire the sample question statement and multiple candidate answers corresponding to the sample question statement. Each candidate answer has a label, which is used to indicate whether the candidate answer is the answer to the sample question statement.

[0206] The concatenation module 902 is used to concatenate the sample question statement with the document to which each candidate answer belongs, to obtain multiple sample data, with the document marked with the position of the candidate answer;

[0207] Training module 903 is used to train the answer determination model based on multiple sample data, with the label of each candidate answer as supervision. The answer determination model is used to determine the answer to the input question statement.

[0208] Optionally, the splicing module 902 is used to sequentially splice the sample question statement, the first delimiter, and the document to the document to which each candidate answer belongs, so as to obtain multiple sample data. The first delimiter is used to separate the sample question statement and the document.

[0209] Optionally, the splicing module 902 is used to, for each document to which a candidate answer belongs, obtain a second document fragment from the document containing the candidate answer and a first document fragment, with the fragment length being a target length less than the document length; and splice the sample question statement with the second document fragment to obtain multiple sample data; wherein, the first document fragment is a document fragment in the document that is connected to the candidate answer and is located before or after the candidate answer, and in the second document fragment, a second separator mark is set between the first document fragment and the candidate answer.

[0210] Optionally, training module 903 includes:

[0211] The acquisition unit is used to determine the model through the answer during the i-th iteration of training, and to acquire the probability corresponding to the candidate answer in multiple sample data. The probability represents the likelihood that the candidate answer is the answer to the sample question statement, and i is a positive integer not less than 1.

[0212] The determination unit is used to determine the loss value of the answer determination model based on the probability and the label of the candidate answer for each sample data. The loss value is negatively correlated with the probability.

[0213] The training unit is used to perform the (i+1)th iteration of the answer determination model based on the loss value when the loss value does not meet the loss value condition. The above training iteration process is repeated until the loss value meets the loss value condition.

[0214] Optionally, the acquisition unit is used to acquire the first word feature of the word sequence corresponding to each sample data for each sample data; input the first word feature of the word sequence into the answer determination model to obtain the answer feature corresponding to the sample data; and determine the probability of the candidate answer in the sample data based on the answer feature.

[0215] Optionally, the word sequence includes multiple words, which include multiple first words in the sample question statement and multiple second words in the document, and the first word feature of the word sequence includes the first word feature of each word;

[0216] The acquisition unit is used to determine the answer features of the model by performing global attention processing on the first word features of multiple first words in the sample question statement and local attention processing on the first word features of multiple second words in the document to obtain the second word features of each word; and from the second word features of each word, the answer features corresponding to the sample data are determined.

[0217] Optionally, the multiple words also include a first delimiter and a second delimiter, and the first word feature of the word sequence also includes a second word feature with the first delimiter and the second delimiter. The first delimiter is used to separate the sample question statement and the document, and the second delimiter is used to mark the position of the candidate answer in the document.

[0218] The acquisition unit is also used to determine the model through the answer, perform global attention processing on the first word feature of the first separator to obtain the second word feature of the first separator, and perform global attention processing on the first word feature of the second separator to obtain the second word feature of the second separator.

[0219] Optionally, a unit is defined for normalizing the answer features to obtain the probability of candidate answers in the sample data.

[0220] In this embodiment, by concatenating the sample question statement with the document to which the candidate answer belongs, and marking the position of the candidate answer in the document, the answer determination model refers to the semantic information of the document to which the candidate answer belongs when determining whether the candidate answer is the answer to the question statement based on the sample data. Since the semantic information of the document is more complete and detailed than the semantic information of the candidate answer itself, the answer determination model trained with the sample data has higher accuracy, and thus the answer obtained by determining the answer based on the answer determination model has higher accuracy.

[0221] Figure 10 This is a schematic diagram of a problem answer determining device according to an embodiment of this application. See also... Figure 10 The device includes:

[0222] The acquisition module 1001 is used to receive the question statement and obtain multiple candidate answers corresponding to the question statement;

[0223] The concatenation module 1002 is used to concatenate the question statement with the document to which each candidate answer belongs, to obtain multiple target data, where the document contains candidate answers and is marked with the position of the candidate answers;

[0224] The determination module 1003 is used to determine the answer to the question statement from multiple candidate answers based on multiple target data.

[0225] Output module 1004 is used to output the answer to the question statement.

[0226] Optionally, the determining module 1003 is used to determine the answer to the question statement from multiple candidate answers based on multiple target data and through the answer determination model;

[0227] The answer determination model is trained on multiple sample data, with the labels of multiple candidate answers corresponding to the sample question statement as supervision. Each sample data is obtained by concatenating the sample question statement with the document to which the corresponding candidate answer belongs, and the document is marked with the position of the candidate answer.

[0228] Optionally, the determining module 1003 is used to input multiple target data into the answer determining model, output the probability of each candidate answer, where the probability represents the likelihood that the candidate answer is the answer to the question statement; and determine the candidate answer with the highest probability among the multiple candidate answers as the answer to the question statement.

[0229] In this embodiment of the application, the target data is obtained by concatenating the question statement with the document to which the candidate answer belongs. In the process of determining the answer based on the target data, not only the semantic information of the candidate answer is referenced, but also the semantic information of the surrounding context of the candidate answer. Since the referenced semantic information is relatively rich, the accuracy of the determined answer is high.

[0230] It should be noted that the answer determination device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the answer determination device can be divided into different functional modules to complete all or part of the functions described above. In addition, the answer determination device and the answer determination method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0231] In the embodiments of this application, the computer device can be configured as a terminal or a server. When the computer device is configured as a terminal, the terminal can act as the execution subject to implement the technical solutions provided in the embodiments of this application. When the computer device is configured as a server, the server can act as the execution subject to implement the technical solutions provided in the embodiments of this application. Alternatively, the technical solutions provided in this application can be implemented through the interaction between the terminal and the server. The embodiments of this application do not limit this.

[0232] This application provides a computer device including one or more processors and one or more memories. The one or more memories store at least one computer program, which is loaded and executed by the one or more processors to implement the method for determining the answer to a problem in the above embodiments. In some embodiments, the computer device is provided as a terminal or a server. The structure of a terminal is described below:

[0233] Figure 11 This is a structural block diagram of a terminal 1100 provided according to an embodiment of this application. The terminal 1100 can be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 1100 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0234] Typically, terminal 1100 includes a processor 1101 and a memory 1102.

[0235] Processor 1101 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0236] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 are used to store at least one computer program, which is executed by the processor 1101 to implement the answer determination method for the problem provided in the method embodiments of this application.

[0237] In some embodiments, the terminal 1100 may also optionally include a peripheral device interface 1103 and at least one peripheral device. The processor 1101, memory 1102, and peripheral device interface 1103 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1103 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107, and a power supply 1108.

[0238] Peripheral device interface 1103 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1101 and memory 1102. In some embodiments, processor 1101, memory 1102 and peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1101, memory 1102 and peripheral device interface 1103 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0239] The radio frequency (RF) circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1104 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1104 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1104 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0240] Display screen 1105 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1105 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1101 for processing. In this case, display screen 1105 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1105, disposed on the front panel of terminal 1100; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal 1100 or in a folded design; in still other embodiments, display screen 1105 may be a flexible display screen, disposed on a curved or folded surface of terminal 1100. Furthermore, display screen 1105 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1105 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0241] The camera assembly 1106 is used to acquire images or videos. Optionally, the camera assembly 1106 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0242] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1101 for processing, or input to the radio frequency circuit 1104 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 1100. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0243] Power supply 1108 is used to power the various components in terminal 1100. Power supply 1108 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1108 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0244] In some embodiments, the terminal 1100 further includes one or more sensors 1109. The one or more sensors 1109 include, but are not limited to: an acceleration sensor 1110, a gyroscope sensor 1111, a pressure sensor 1112, an optical sensor 1113, and a proximity sensor 1114.

[0245] Accelerometer 1110 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established with terminal 1100. For example, accelerometer 1110 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1101 can control display screen 1105 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1110. Accelerometer 1110 can also be used for games or for acquiring user motion data.

[0246] The gyroscope sensor 1111 can detect the orientation and rotation angle of the terminal 1100. The gyroscope sensor 1111 can work in conjunction with the accelerometer sensor 1110 to collect the user's 3D movements on the terminal 1100. Based on the data collected by the gyroscope sensor 1111, the processor 1101 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0247] The pressure sensor 1112 can be disposed on the side bezel of the terminal 1100 and / or on the lower layer of the display screen 1105. When the pressure sensor 1112 is disposed on the side bezel of the terminal 1100, it can detect the user's grip signal on the terminal 1100, and the processor 1101 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1112. When the pressure sensor 1112 is disposed on the lower layer of the display screen 1105, the processor 1101 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1105. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0248] An optical sensor 1113 is used to collect ambient light intensity. In one embodiment, the processor 1101 can control the display brightness of the display screen 1105 based on the ambient light intensity collected by the optical sensor 1113. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1105 is increased; when the ambient light intensity is low, the display brightness of the display screen 1105 is decreased. In another embodiment, the processor 1101 can also dynamically adjust the shooting parameters of the camera assembly 1106 based on the ambient light intensity collected by the optical sensor 1113.

[0249] The proximity sensor 1114, also known as a distance sensor, is typically located on the front panel of the terminal 1100. The proximity sensor 1114 is used to detect the distance between the user and the front of the terminal 1100. In one embodiment, when the proximity sensor 1114 detects that the distance between the user and the front of the terminal 1100 is gradually decreasing, the processor 1101 controls the display screen 1105 to switch from a screen-on state to a screen-off state; when the proximity sensor 1114 detects that the distance between the user and the front of the terminal 1100 is gradually increasing, the processor 1101 controls the display screen 1105 to switch from a screen-off state to a screen-on state.

[0250] Those skilled in the art will understand that Figure 11 The structure shown does not constitute a limitation on terminal 1100 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0251] Figure 12 This is a schematic diagram of a server structure according to an embodiment of this application. The server 1200 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 1201 and one or more memories 1202. The memories 1202 store at least one computer program, which is loaded and executed by the processor 1201 to implement the answer determination method for the problems provided in the above-described method embodiments. Of course, the server 1200 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 1200 may also include other components for implementing device functions, which will not be elaborated here.

[0252] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor of a computer device to implement the answer determination method for the problem provided in the above-described method embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0253] In an exemplary embodiment, a computer program product is also provided, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the answer determination method for the problem provided in the above-described method embodiments.

[0254] In some embodiments, the computer program involved in the present application embodiments may be deployed and executed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network may constitute a blockchain system.

[0255] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0256] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining the answer to a question, characterized in that, The method includes: Obtain a sample question statement and multiple candidate answers corresponding to the sample question statement. Each candidate answer has a label, which indicates whether the candidate answer is an answer to the sample question statement. The sample question statement is concatenated with the document to which each candidate answer belongs to obtain multiple sample data, and the document is marked with the position of the candidate answer; Using the label of each candidate answer as supervision, and based on the answer features corresponding to the multiple sample data, an answer determination model is trained. The answer determination model is used to determine the answer to the input question statement. The training process of the model, as determined by the answer, includes: During the i-th iteration of training, for each sample data, the first word feature of the word sequence corresponding to the sample data is obtained. The word sequence includes multiple words, including multiple first words in the sample question statement, multiple second words in the document, a first separator and a second separator. The first word feature of the word sequence includes the first word feature of the first separator and the first word feature of the second separator. The first separator is used to separate the sample question statement and the document, and the second separator is used to mark the position of the candidate answer in the document. i is a positive integer not less than 1. The answer determination model performs global attention processing on the first word features of the first separator to obtain the second word features of the first separator, and performs global attention processing on the first word features of the second separator to obtain the second word features of the second separator; from the second word features of each word, the answer features corresponding to the sample data are determined.

2. The method according to claim 1, characterized in that, The process involves concatenating the sample question statements with the documents to which each candidate answer belongs, resulting in multiple sample data sets, including: For each document to which a candidate answer belongs, the sample question statement, the first delimiter, and the document are concatenated sequentially to obtain the multiple sample data.

3. The method according to claim 1, characterized in that, The process involves concatenating the sample question statements with the documents to which each candidate answer belongs, resulting in multiple sample data sets, including: For each document to which the candidate answer belongs, a second document segment containing the candidate answer and a first document segment with a segment length equal to a target length is obtained from the document, wherein the target length is less than the length of the document; The sample question statement is concatenated with the second document fragment to obtain the multiple sample data; Wherein, the first document fragment is a document fragment in the document that is connected to the candidate answer and is located before or after the candidate answer, and in the second document fragment, the first document fragment and the candidate answer are provided with the second separator mark.

4. The method according to claim 1, characterized in that, The step of training an answer determination model based on the answer features corresponding to the multiple sample data, using the label of each candidate answer as supervision, includes: During the i-th iteration of training, the model is determined by the answer, and the probability corresponding to the candidate answer in the multiple sample data is obtained. The probability represents the likelihood that the candidate answer is the answer to the sample question statement. Based on the probability of each sample data and the label of the candidate answer, a loss value for the answer determination model is determined, wherein the loss value is negatively correlated with the probability; If the loss value does not meet the loss value condition, the answer determination model is subjected to the (i+1)th iteration process based on the loss value, and the above training iteration process is repeated until the loss value meets the loss value condition.

5. The method according to claim 4, characterized in that, The step of obtaining the probability of the candidate answer from the multiple sample data through the answer determination model includes: The first word feature of the word sequence is input into the answer determination model to obtain the answer feature corresponding to the sample data; Based on the answer features, the probability of candidate answers in the sample data is determined.

6. The method according to claim 5, characterized in that, Before determining the answer features corresponding to the sample data from the second word features of each word, the method further includes: The model is determined by the answer, and global attention processing is performed on the first word features of multiple first words in the sample question statement, and local attention processing is performed on the first word features of multiple second words in the document to obtain the second word features of each word.

7. The method according to claim 5, characterized in that, Determining the probability of candidate answers in the sample data based on the answer features includes: The answer features are normalized to obtain the probability of candidate answers in the sample data.

8. A method for determining the answer to a question, characterized in that, The method includes: Receive a question statement and obtain multiple candidate answers corresponding to the question statement; The question statement is concatenated with the document to which each candidate answer belongs to obtain multiple target data, wherein the document contains the candidate answer and is marked with the position of the candidate answer; Based on the multiple target data, the answer to the question statement is determined from the multiple candidate answers using an answer determination model; Output the answer to the question statement; The answer determination model is trained based on multiple sample data, with the labels of multiple candidate answers corresponding to the sample question statement as supervision. Each sample data is obtained by concatenating the sample question statement with the document to which the corresponding candidate answer belongs. The answer determines the training process of the model, including: During the i-th iteration of training, for each sample data, the first word feature of the word sequence corresponding to the sample data is obtained. The word sequence includes multiple words, including multiple first words in the sample question statement, multiple second words in the document, a first separator, and a second separator. The first word feature of the word sequence includes the first word feature of the first separator and the first word feature of the second separator. The first separator is used to separate the sample question statement and the document, and the second separator is used to mark the position of the candidate answer corresponding to the sample question statement in the document. i is a positive integer not less than 1. The answer determination model performs global attention processing on the first word features of the first separator to obtain the second word features of the first separator, and performs global attention processing on the first word features of the second separator to obtain the second word features of the second separator; from the second word features of each word, the answer features corresponding to the sample data are determined. The answer determination model is trained based on the answer features and the labels of the candidate answers.

9. The method according to claim 8, characterized in that, The step of determining the answer to the question statement from the multiple candidate answers using an answer determination model based on the multiple target data includes: The multiple target data are input into the answer determination model, and the probability of each candidate answer is output, whereby the probability represents the likelihood that the candidate answer is the answer to the question statement. The candidate answer with the highest probability among the multiple candidate answers is determined as the answer to the question statement.

10. A device for determining the answer to a question, characterized in that, The device includes: The acquisition module is used to acquire a sample question statement and multiple candidate answers corresponding to the sample question statement. Each candidate answer has a label, which is used to indicate whether the candidate answer is an answer to the sample question statement. The concatenation module is used to concatenate the sample question statement with the document to which each candidate answer belongs, to obtain multiple sample data, wherein the document is marked with the position of the candidate answer; The training module is used to train an answer determination model based on the answer features corresponding to the multiple sample data, with the label of each candidate answer as supervision. The answer determination model is used to determine the answer to the input question statement. The training module includes an acquisition unit, used for: During the i-th iteration of training, for each sample data, the first word feature of the word sequence corresponding to the sample data is obtained. The word sequence includes multiple words, including multiple first words in the sample question statement, multiple second words in the document, a first separator and a second separator. The first word feature of the word sequence includes the first word feature of the first separator and the first word feature of the second separator. The first separator is used to separate the sample question statement and the document, and the second separator is used to mark the position of the candidate answer in the document. i is a positive integer not less than 1. The answer determination model performs global attention processing on the first word features of the first separator to obtain the second word features of the first separator, and performs global attention processing on the first word features of the second separator to obtain the second word features of the second separator; from the second word features of each word, the answer features corresponding to the sample data are determined.

11. The apparatus according to claim 10, characterized in that, The splicing module is used for: For each document to which a candidate answer belongs, the sample question statement, the first delimiter, and the document are concatenated sequentially to obtain the multiple sample data.

12. The apparatus according to claim 10, characterized in that, The splicing module is used for: For each document to which the candidate answer belongs, a second document segment containing the candidate answer and a first document segment with a segment length equal to a target length is obtained from the document, wherein the target length is less than the length of the document; The sample question statement is concatenated with the second document fragment to obtain the multiple sample data; Wherein, the first document fragment is a document fragment in the document that is connected to the candidate answer and is located before or after the candidate answer, and in the second document fragment, the first document fragment and the candidate answer are provided with the second separator mark.

13. The apparatus according to claim 10, characterized in that, The training module includes: The acquisition unit is used to obtain the probability corresponding to the candidate answer in the multiple sample data through the answer determination model during the i-th iteration of training, wherein the probability represents the likelihood that the candidate answer is the answer to the sample question statement; A determining unit is configured to determine a loss value for the answer determining model based on the probability of each sample data and the label of the candidate answer, wherein the loss value is negatively correlated with the probability; The training unit is used to perform the (i+1)th iteration process on the answer determination model based on the loss value when the loss value does not meet the loss value condition, and repeat the above training iteration process until the loss value meets the loss value condition.

14. The apparatus according to claim 13, characterized in that, The acquisition unit is used for: The first word feature of the word sequence is input into the answer determination model to obtain the answer feature corresponding to the sample data; Based on the answer features, the probability of candidate answers in the sample data is determined.

15. The apparatus according to claim 14, characterized in that, The acquisition unit is further configured to: The model is determined by the answer, and global attention processing is performed on the first word features of multiple first words in the sample question statement, and local attention processing is performed on the first word features of multiple second words in the document to obtain the second word features of each word.

16. The apparatus according to claim 14, characterized in that, The determining unit is used for: The answer features are normalized to obtain the probability of candidate answers in the sample data.

17. A device for determining the answer to a question, characterized in that, The device includes: The acquisition module is used to receive a question statement and acquire multiple candidate answers corresponding to the question statement; The concatenation module is used to concatenate the question statement with the document to which each candidate answer belongs, to obtain multiple target data, wherein the document contains the candidate answer and marks the position of the candidate answer; The determination module is used to determine the answer to the question statement from the multiple candidate answers based on the multiple target data and through the answer determination model; The output module is used to output the answer to the question statement; The answer determination model is trained based on multiple sample data, with the labels of multiple candidate answers corresponding to the sample question statement as supervision. Each sample data is obtained by concatenating the sample question statement with the document to which the corresponding candidate answer belongs. The answer determines the training process of the model, including: During the i-th iteration of training, for each sample data, the first word feature of the word sequence corresponding to the sample data is obtained. The word sequence includes multiple words, including multiple first words in the sample question statement, multiple second words in the document, a first separator, and a second separator. The first word feature of the word sequence includes the first word feature of the first separator and the first word feature of the second separator. The first separator is used to separate the sample question statement and the document, and the second separator is used to mark the position of the candidate answer corresponding to the sample question statement in the document. i is a positive integer not less than 1. The answer determination model performs global attention processing on the first word features of the first separator to obtain the second word features of the first separator, and performs global attention processing on the first word features of the second separator to obtain the second word features of the second separator; from the second word features of each word, the answer features corresponding to the sample data are determined. The answer determination model is trained based on the answer features and the labels of the candidate answers.

18. The apparatus according to claim 17, characterized in that, The determining module is used for: The multiple target data are input into the answer determination model, and the probability of each candidate answer is output, whereby the probability represents the likelihood that the candidate answer is the answer to the question statement. The candidate answer with the highest probability among the multiple candidate answers is determined as the answer to the question statement.

19. A computer device, characterized in that, The computer device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, the computer program being loaded and executed by the one or more processors to implement the method for determining the answer to the question as described in any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the method for determining the answer to the question as described in any one of claims 1 to 9.

21. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, a processor of a computer device reading the computer program from the computer-readable storage medium, and the processor executing the computer program to cause the computer device to perform a method for determining the answer to a question as described in any one of claims 1 to 9.

Images