Answer determination method and device, electronic equipment and computer readable medium

Abstract

The invention discloses an answer determination method and device, electronic equipment and a computer readable medium, and the method comprises the steps: carrying out the field recognition processing of a user question after the user question is received, obtaining a field recognition result, and enabling the field recognition result to represent which field the user question belongs to; according to the field identification result, determining a reference knowledge field of the user question, so that the reference knowledge field can represent which field needs to refer to knowledge information when the user question is answered; and then, according to the reference knowledge field, determining the answer to the user question, so that the answer can be automatically determined.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an answer determination method, device, electronic device, and computer-readable medium. Background Art

[0002] For some application scenarios, such as a certain intelligent question-answering scenario, after receiving the question (Query) input by the user, the answer to the question is first determined; then the answer is fed back to the user, thereby automatically answering the user's question.

[0003] However, how to determine the answer is a technical problem that needs to be solved. Summary of the Invention

[0004] In order to solve the above technical problems, the present application provides an answer determination method, device, electronic device, and computer-readable medium, which are conducive to improving the quality of answers.

[0005] In order to achieve the above objectives, the technical solutions provided by this application are as follows:

[0006] The present application provides a method for determining an answer, the method comprising:

[0007] After receiving the user question, performing domain identification processing on the user question to obtain a domain identification result;

[0008] Determine a reference knowledge domain for the user question based on the domain identification result; the reference knowledge domain includes at least one of a public network knowledge domain and an internal knowledge domain of an organization;

[0009] An answer to the user's question is determined based on the reference knowledge domain.

[0010] In one possible implementation, the domain identification result includes a confidence level corresponding to the public network knowledge domain and a confidence level corresponding to the internal organizational knowledge domain;

[0011] The process of determining the reference knowledge domain includes at least one of the following:

[0012] In response to a confidence level corresponding to the public network knowledge domain being higher than a first threshold, determining the public network knowledge domain as a reference knowledge domain for the user question;

[0013] In response to the confidence level corresponding to the intra-organizational knowledge domain being higher than a second threshold, determining the intra-organizational knowledge domain as a reference knowledge domain for the user question;

[0014] In response to the confidence level corresponding to the public network knowledge domain being not higher than the first threshold and the confidence level corresponding to the knowledge domain within the organization being not higher than the second threshold, determine the reference knowledge domain of the user question based on the public network knowledge domain and the knowledge domain within the organization.

[0015] In a possible implementation manner, if the reference knowledge domain includes the public network knowledge domain, the process of determining the answer to the user question includes:

[0016] Determine the answer corresponding to the public network knowledge domain according to the user question and a search engine; the search engine is used to obtain knowledge in the public network knowledge domain;

[0017] Determine the answer to the user question based on the answer corresponding to the public network knowledge domain.

[0018] In a possible implementation manner, the process of determining the answer corresponding to the public network knowledge domain includes:

[0019] Generate a search question according to the user question;

[0020] Use the search engine to perform a search process on the search question to obtain at least one search result;

[0021] Determine the answer corresponding to the public network knowledge domain according to the at least one search result.

[0022] In a possible implementation manner, the determining the answer corresponding to the public network knowledge domain according to the at least one search result includes:

[0023] Determine a target result from the at least one search result according to the correlation characterization data between each search result and the user question, and the correlation characterization data between the target result and the user question is greater than the correlation characterization data between any other search result in the at least one search result and the user question except the target result;

[0024] Perform an answer generation process based on the target result and the user question to obtain the answer corresponding to the public network knowledge domain.

[0025] In a possible implementation manner, if the reference knowledge domain includes the knowledge domain within the organization, the process of determining the answer to the user question includes:

[0026] Determine the answer corresponding to the knowledge domain within the organization according to the user question and the knowledge base of the knowledge domain within the organization;

[0027] Determine the answer to the user question based on the answer corresponding to the knowledge domain within the organization.

[0028] In a possible implementation, the answer corresponding to the knowledge area within the organization is obtained by processing the user's question using a pre-constructed knowledge model within the organization; the knowledge model within the organization is constructed based on the knowledge base of the knowledge area within the organization.

[0029] In a possible implementation, if the reference knowledge area includes the public network knowledge area and the knowledge area within the organization, then the answer to the user's question is determined based on the weight corresponding to the public network knowledge area, the answer corresponding to the public network knowledge area, the weight corresponding to the knowledge area within the organization, and the answer corresponding to the knowledge area within the organization; the answer corresponding to the public network knowledge area and the answer corresponding to the knowledge area within the organization are both determined based on the user's question.

[0030] In a possible implementation, the domain recognition result includes the confidence level corresponding to the public network knowledge area and the confidence level corresponding to the knowledge area within the organization;

[0031] The process of determining the answer to the user's question includes:

[0032] Determine the probability of using the answer corresponding to the public network knowledge area based on the product of the confidence level corresponding to the public network knowledge area and the weight corresponding to the public network knowledge area;

[0033] Determine the probability of using the answer corresponding to the knowledge area within the organization based on the product of the confidence level corresponding to the knowledge area within the organization and the weight corresponding to the knowledge area within the organization;

[0034] Select the answer to the user's question from the answer corresponding to the public network knowledge area and the answer corresponding to the knowledge area within the organization based on the probability of using the answer corresponding to the public network knowledge area and the probability of using the answer corresponding to the knowledge area within the organization.

[0035] In a possible implementation, the answer to the user's question is determined based on the weight corresponding to the public network knowledge area and the weight corresponding to the knowledge area within the organization;

[0036] The method further includes:

[0037] Display the answer to the user's question;

[0038] After obtaining the feedback information on the answer to the user's question, if the feedback information indicates that the answer to the user's question is inaccurate, then update the weight corresponding to the public network knowledge area and the weight corresponding to the knowledge area within the organization.

[0039] In a possible implementation, if the answer to the user question is the answer corresponding to the public network knowledge domain, then updating the weight corresponding to the public network knowledge domain and the weight corresponding to the knowledge domain within the organization includes:

[0040] Reducing the weight corresponding to the public network knowledge domain and increasing the weight corresponding to the knowledge domain within the organization.

[0041] In a possible implementation, if the answer to the user question is the answer corresponding to the knowledge domain within the organization, then updating the weight corresponding to the public network knowledge domain and the weight corresponding to the knowledge domain within the organization includes:

[0042] Reducing the weight corresponding to the knowledge domain within the organization and increasing the weight corresponding to the public network knowledge domain.

[0043] In a possible implementation, the answer to the user question is determined based on the knowledge base of the knowledge domain within the organization;

[0044] The method further includes:

[0045] Displaying the answer to the user question;

[0046] After obtaining feedback information on the answer to the user question, if the feedback information indicates that the answer to the user question is inaccurate, then a negative sample is constructed based on the user question and the answer to the user question; the negative sample is used to update the knowledge base of the knowledge domain within the organization.

[0047] In a possible implementation, the answer to the user question is determined using a pre-constructed answer generation model, and the negative sample is also used to update the answer generation model;

[0048] And / or,

[0049] The answer to the user question is determined based on a search question, the search question is obtained by processing the user question using a pre-constructed question generation model, and the negative sample is also used to update the question generation model.

[0050] In a possible implementation, the knowledge domain within the organization includes at least one professional knowledge domain.

[0051] This application provides an answer determination device, including:

[0052] A domain identification unit, configured to perform domain identification processing on the user question after receiving the user question, and obtain a domain identification result;

[0053] A first determination unit, configured to determine a reference knowledge domain of the user question according to the domain recognition result; the reference knowledge domain includes at least one of a public network knowledge domain and an intra-organization knowledge domain;

[0054] A second determination unit, configured to determine an answer to the user question according to the reference knowledge domain.

[0055] This application provides an electronic device, and the device includes: a processor and a memory;

[0056] The memory is configured to store instructions or computer programs;

[0057] The processor is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the answer determination method provided by this application.

[0058] This application provides a computer-readable medium, in which instructions or computer programs are stored. When the instructions or computer programs run on a device, the device is enabled to execute the answer determination method provided by this application.

[0059] This application provides a computer program product, which includes a computer program carried on a non-transitory computer-readable medium. The computer program includes program codes for executing the answer determination method provided by this application.

[0060] Compared with the related art, this application has at least the following advantages:

[0061] In the technical solution provided by this application, after receiving a user's question, the user's question is first processed for domain identification to obtain a domain identification result, so that the domain identification result can indicate which domain the user's question belongs to; then, based on the domain identification result, a reference knowledge domain for the user's question is determined, so that the reference knowledge domain can indicate which domain's knowledge information needs to be referred to when answering the user's question; then, based on the reference knowledge domain, an answer to the user's question is determined, so that automatic answer determination can be achieved. Among them, because the reference knowledge domain can indicate which domain's knowledge information needs to be referred to when answering the user's question, so that the answer determined based on the reference knowledge domain can indicate the corresponding answer result of the user's question in the reference knowledge domain, thereby enabling the answer to better meet the domain requirements, so that the accuracy impact caused by different answers to the same question in different domains can be effectively avoided, which is conducive to improving the answer quality. In addition, because the reference knowledge domain includes at least one of the public network knowledge domain and the intra-organization knowledge domain, so that the reference knowledge domain can better indicate which domain's knowledge information needs to be referred to when answering the user's question, thereby making the answer determined based on the reference knowledge domain more accurate, and further conducive to improving the answer quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments recorded in this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0063] Figure 1 It is a flowchart of a method for determining an answer provided by an embodiment of this application;

[0064] Figure 2 It is a schematic diagram of a question-and-answer process provided by an embodiment of this application;

[0065] Figure 3 It is a schematic diagram of a process for determining public network information provided by an embodiment of this application;

[0066] Figure 4 It is a schematic diagram of a process for determining an answer provided by an embodiment of this application;

[0067] Figure 5 It is a schematic diagram of the structure of an answer determination device provided by an embodiment of this application;

[0068] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of this application. Detailed implementation manners

[0069] In order to enable those skilled in the art to better understand the solution of this application, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0070] To better understand the technical solution provided by this application, the following first explains the answer determination method provided by this application with reference to some drawings. As Figure 1 shown, the answer determination method provided by the embodiments of this application includes S1-S3 below. Among them, this Figure 1 is a flowchart of an answer determination method provided by the embodiments of this application.

[0071] S1: After receiving the user's question, perform domain recognition processing on the user's question to obtain a domain recognition result.

[0072] Among them, the user's question refers to the question input by the user so that the user's question can represent the user's information query requirement. For example, the user's question can be implemented using the user's question shown in Figure 2 , Figure 3 or Figure 4 shown.

[0073] In addition, this application does not limit the acquisition method of the above user's question. For example, it can be implemented using any existing or future method that can obtain the Query input by the user.

[0074] The domain recognition result refers to the result obtained by performing domain recognition processing on the above user's question so that the domain recognition result can represent which domain the user's question belongs to, such as the public network knowledge domain or the intra-organization knowledge domain, etc. For the sake of easy understanding, the following introduces these two domains respectively.

[0075] For the above public network knowledge domain, the public network knowledge domain is used to provide a large amount of public knowledge, such as knowledge that can be retrieved from the Internet, etc.; and because the knowledge scope of the public network knowledge domain is relatively wide, it is difficult to build a complete public network knowledge base, so that it is impossible to build a model for providing public network knowledge for the public network knowledge domain, and thus the knowledge in the public network knowledge domain can only be obtained through a search engine. In addition, this application does not limit the public network knowledge domain. For example, the public network knowledge domain has the characteristics of publicity and / or real-time, so that the questions belonging to the public network knowledge domain have the characteristics shown in the following (1)-(3).

[0076] (1) There is real-time content, and this application does not limit this real-time content. For example, this real-time content can be content related to recent dates. Another example is that this real-time content can also be content related to the dynamic changes of information, such as content of press conferences, information, etc.

[0077] (2) There is public knowledge, and this application does not limit this public knowledge. For example, this public knowledge can refer to knowledge that does not belong to the knowledge field within the above-mentioned organization, such as historical and geographical knowledge, knowledge of biographical summaries, etc.

[0078] (3) There is a public network scope prompt word, and this application does not limit this public network scope prompt word. For example, this public network scope prompt word can be words such as public network, Internet, or a certain encyclopedia, etc.

[0079] Based on the relevant content of (1)-(3) above, it can be known that for the above user question, if the features shown in (1)-(3) appear in the user question, it can be determined that the user question belongs to the public network knowledge field. Thus, it can be determined that knowledge in the public network knowledge field needs to be referred to when answering and processing the user question, and it can be determined that the knowledge in the above-mentioned organization's internal knowledge field has little impact on answering the user question.

[0080] For the above-mentioned organization's internal knowledge field, this organization's internal knowledge field is used to describe the knowledge involved within a certain organization, such as some professional knowledge, etc.; and because the knowledge in this organization's internal knowledge field is limited, the knowledge in this organization's internal knowledge field can be represented by a pre-constructed knowledge base, so that the knowledge in this organization's internal knowledge field can be obtained from this knowledge base, and it also enables the knowledge in this organization's internal knowledge field to be obtained by a model pre-constructed based on this knowledge base. In addition, this application does not limit this organization's internal knowledge field. For example, this organization's internal knowledge field has the characteristic of professionalism, so that questions belonging to this organization's internal knowledge field have the characteristics shown in ①-③ below.

[0081] ① There are professional terms, and this application does not limit these professional terms. For example, if this organization's internal knowledge field includes the chemical field, then these professional terms can include some chemical nouns. Another example is that if this organization's internal knowledge field includes the computer field, then these professional terms can include some computer nouns.

[0082] ② There are organization-internal prompt words, and this application does not limit these organization-internal prompt words. For example, these organization-internal prompt words can include this organization, within the organization, my organization, etc. It should be noted that this application does not limit the implementation method of this organization. For example, this organization can be implemented using a company, a research institute, a school, an institution, etc.

[0083] ③There are related expressions within the organization, and this application does not limit such expressions. For example, some or all of the content in such expressions has a relatively high relevance to the organization.

[0084] Based on the relevant content in ①-③ above, for the user question above, if the features shown in ①-③ appear in the user question, it can be determined that the user question belongs to the knowledge domain within the organization. Thus, it can be determined that when answering and processing the user question, the knowledge in the knowledge domain within the organization needs to be referred to, and it can be determined that the knowledge in the above-mentioned public network knowledge domain cannot be used to answer the user question.

[0085] In fact, in some application scenarios, to better improve the quality of answers, the above-mentioned knowledge domain within the organization can be further refined into multiple domains. Based on this, this application also provides a possible implementation manner of the knowledge domain within the organization. In this implementation manner, the knowledge domain within the organization can include at least one professional knowledge domain, so that these professional knowledge domains can represent the knowledge within the organization in a more fine-grained manner, thereby facilitating better determination of the answer to the user question from these professional knowledge domains, and further facilitating the improvement of the answer quality. Among them, the professional knowledge domain is used to provide the knowledge involved in a certain profession within the organization; and this application does not limit the implementation manner of the at least one professional knowledge domain. For example, the at least one professional knowledge domain can include N professional knowledge domains, and the nth professional knowledge domain is used to provide the knowledge involved in the nth profession within the organization, where n is a positive integer, n ≤ N, and N is a positive integer. Another example is that the at least one professional knowledge domain can include the computer knowledge domain, the chemical knowledge domain,....

[0086] In addition, the present application does not limit the implementation manner of the above-mentioned domain recognition result. For example, in order to better improve the domain recognition effect, the domain recognition result may include the confidence corresponding to the public network knowledge domain and the confidence corresponding to the knowledge domain within the organization. Among them, the confidence corresponding to the public network knowledge domain is used to represent the possibility that the above user question belongs to the public network knowledge domain, so that subsequently, based on the confidence corresponding to the public network knowledge domain, it can be determined whether to use the knowledge in the public network knowledge domain when processing the reply to the user question. The confidence corresponding to the knowledge domain within the organization is used to represent the possibility that the user question belongs to the knowledge domain within the organization, so that subsequently, based on the confidence corresponding to the knowledge domain within the organization, it can be determined whether to use the knowledge in the knowledge domain within the organization when processing the reply to the user question; moreover, the present application does not limit the implementation manner of the confidence corresponding to the knowledge domain within the organization. For example, when the knowledge domain within the organization can include N professional knowledge domains, the confidence corresponding to the knowledge domain within the organization may include the confidence corresponding to the N professional knowledge domains. Among them, the confidence corresponding to the nth professional knowledge domain is used to represent the possibility that the user question belongs to the nth professional knowledge domain, so that subsequently, based on the confidence corresponding to the nth professional knowledge domain, it can be determined whether to use the knowledge in the nth professional knowledge domain when processing the reply to the user question, where n is a positive integer, n ≤ N, and N is a positive integer.

[0087] In addition, the present application does not limit the implementation manner of S1 above. For example, in order to improve the accuracy of domain recognition, S1 may specifically be: after receiving a user question, inputting the user question into a pre-constructed domain recognition model to obtain a domain recognition result output by the domain recognition model. Among them, the domain recognition model is used to perform domain recognition processing on the input data of the domain recognition model; moreover, the present application does not limit the implementation manner of the domain recognition model. For example, the domain recognition model may be implemented using any existing or future machine learning model. In addition, the present application also does not limit the construction process of the domain recognition model. For example, the training data used in constructing the domain recognition model may include public network samples and in-organization samples. Among them, the public network samples are used to represent the training data in the public network knowledge domain; moreover, the public network samples may include public network questions and the answer tags corresponding to the public network questions. Among them, the public network question refers to a question in the public network knowledge domain; the answer tag corresponding to the public network question refers to the answer corresponding to the public network question in the public network knowledge domain; moreover, the present application does not limit the public network question. For example, the public network question may refer to a question that satisfies the characteristics shown in (1)-(3) above. The in-organization samples are used to represent the training data in the in-organization knowledge domain; moreover, the in-organization samples may include in-organization questions and the answer tags corresponding to the in-organization questions. The in-organization question refers to a question in the in-organization knowledge domain; the answer tag corresponding to the in-organization question refers to the answer corresponding to the in-organization question in the in-organization knowledge domain; moreover, the present application does not limit the in-organization question. For example, the in-organization question may refer to a question that satisfies the characteristics shown in ①-③ above.

[0088] In addition, in some application scenarios, the above domain recognition result may be determined based on the knowledge base of the in-organization knowledge domain, so that the domain recognition result can better represent whether the user question belongs to the in-organization knowledge domain. Among them, the knowledge base of the in-organization knowledge domain refers to a knowledge base pre-constructed for the in-organization knowledge domain, such as Figure 2The knowledge base shown above, etc., so that the knowledge base can be used to provide knowledge in the knowledge fields within the organization. In addition, this application does not limit the implementation manner of the knowledge base in the knowledge fields within the organization. For example, when the knowledge fields within the organization include N professional knowledge fields, the knowledge base in the knowledge fields within the organization includes knowledge bases for N professional knowledge fields. The knowledge base for the nth professional knowledge field refers to the knowledge base pre-constructed for the nth professional knowledge field, so that the knowledge base for the nth professional knowledge field can be used to provide knowledge in the nth professional knowledge field, where n is a positive integer, n ≤ N, and N is a positive integer. In addition, this application does not limit the implementation manner of determining the domain recognition result based on the knowledge base. For example, in some application scenarios, the above-mentioned domain recognition model can be pre-constructed using the knowledge base, so that the domain recognition model can perform domain recognition processing with the help of the knowledge in the knowledge base, so that the domain recognition result can be obtained subsequently by using the domain recognition model to process the user question.

[0089] Based on the relevant content of S1 above, for some application scenarios, after obtaining the user question input by the user, use the pre-constructed domain recognition model to perform domain recognition processing on the user question to obtain a domain recognition result, so that the domain recognition result includes the confidence corresponding to the public network knowledge field and the confidence corresponding to the knowledge field within the organization, so that the domain recognition result can more accurately represent which field the user question belongs to, so that subsequently, based on the domain recognition result, it can be determined which field of knowledge to use to answer the user question.

[0090] S2: Determine the reference knowledge field of the user question according to the domain recognition result; the reference knowledge field includes at least one of the public network knowledge field and the knowledge field within the organization.

[0091] Among them, the reference knowledge field refers to the knowledge field that needs to be referred to when answering the above-mentioned user question; moreover, this application does not limit the determination process of the reference knowledge field.

[0092] In fact, in order to better improve the answer quality, this application also provides a possible implementation manner of S2 above. In this implementation manner, when the above-mentioned domain recognition result includes the confidence corresponding to the public network knowledge field and the confidence corresponding to the knowledge field within the organization, S2 may specifically include at least one of the following steps 11-step 13.

[0093] Step 11: In response to the confidence corresponding to the above-mentioned public network knowledge field being higher than the first threshold, determine the public network knowledge field as the reference knowledge field of the user question.

[0094] Among them, the first threshold refers to the lowest confidence value set in advance for the public network knowledge domain; moreover, the first threshold can be set in advance according to the actual application scenario. For example, the first threshold can be 0.5.

[0095] Based on the relevant content of step 11 above, after obtaining the confidence of the above user question in the public network knowledge domain, if the confidence is higher than the first threshold, it can be determined that the user question belongs to the question under the public network knowledge domain, so it can be determined that it is necessary to use the knowledge under the public network knowledge domain to reply to the user question. Therefore, the public network knowledge domain can be directly determined as the reference knowledge domain of the user question, so that the reference knowledge domain can indicate that the public network knowledge domain needs to be referred to when replying to the user question.

[0096] Step 12: In response to the confidence of the above organization's internal knowledge domain being higher than the second threshold, determine the organization's internal knowledge domain as the reference knowledge domain of the user question.

[0097] Among them, the second threshold refers to the lowest confidence value set in advance for the organization's internal knowledge domain; moreover, the second threshold can be set in advance according to the actual application scenario. For example, the second threshold can be 0.5.

[0098] In addition, the present application does not limit the implementation manner of step 12 above. For example, when the above organization's internal knowledge domain includes N professional knowledge domains, and the confidence of the above organization's internal knowledge domain includes the confidence of N professional knowledge domains, step 12 can specifically be: if the confidence of the nth professional knowledge domain is higher than the second threshold, then determine the nth professional knowledge domain as the reference knowledge domain of the user question, where n is a positive integer, n ≤ N, and N is a positive integer.

[0099] Based on the relevant content of step 12 above, after obtaining the confidence of the above user question in the organization's internal knowledge domain, if the confidence is higher than the second threshold, it can be determined that the user question belongs to the question under the organization's internal knowledge domain, so it can be determined that it is necessary to use the knowledge under the organization's internal knowledge domain to reply to the user question. Therefore, the organization's internal knowledge domain can be directly determined as the reference knowledge domain of the user question, so that the reference knowledge domain can indicate that the organization's internal knowledge domain needs to be referred to when replying to the user question.

[0100] Step 13: In response to the confidence of the above public network knowledge domain not being higher than the first threshold and the confidence of the above organization's internal knowledge domain not being higher than the second threshold, determine the reference knowledge domain of the user question based on the public network knowledge domain and the organization's internal knowledge domain.

[0101] In this application, after determining the domain recognition result for the above-mentioned user question, if the confidence level corresponding to the public network knowledge domain in the domain recognition result is not higher than the first threshold, and the confidence level corresponding to the intra-organization knowledge domain in the domain recognition result is not higher than the second threshold, such as the confidence levels corresponding to each professional knowledge domain in the intra-organization knowledge domain are not higher than the second threshold, it can be determined that the domain to which the user question belongs cannot be recognized. Thus, it can be determined that the user question belongs to a problem with an unclear domain. Therefore, in order to better improve the answer quality, the reference knowledge domain of the user question can be directly determined based on the public network knowledge domain and the intra-organization knowledge domain, so that the reference knowledge domain includes the public network knowledge domain and the intra-organization knowledge domain, thereby enabling the reference knowledge domain to indicate that the public network knowledge domain and the intra-organization knowledge domain need to be referred to when answering the user question. This is beneficial to solving the defects caused by the unclear domain of the user question and thus beneficial to improving the answer quality.

[0102] Based on the relevant content of the above steps 11 to 13, for some application scenarios, after determining the confidence level corresponding to the public network knowledge domain and the confidence level corresponding to the intra-organization knowledge domain for the above-mentioned user question, the reference knowledge domain of the user question can be determined based on the comparison result between the confidence level corresponding to the public network knowledge domain and the first threshold corresponding to the public network knowledge domain, and the comparison result between the confidence level corresponding to the intra-organization knowledge domain and the second threshold corresponding to the intra-organization knowledge domain, so that the reference knowledge domain can better indicate what knowledge domain needs to be referred to when answering the user question.

[0103] Based on the relevant content of the above S2, after determining the domain recognition result for the above-mentioned user question, based on the domain recognition result, the reference knowledge domain of the user question is determined, so that the reference knowledge domain includes at least one of the public network knowledge domain and the intra-organization knowledge domain, so that one or more domains of knowledge can be referred to subsequently to determine the answer to the user question.

[0104] S3: Determine the answer to the user question based on the reference knowledge domain.

[0105] Among them, the answer to the user question refers to the reply result determined for the user question.

[0106] In addition, this application does not limit the implementation manner of the above S3. For the sake of easy understanding, three examples are described below.

[0107] Example 1, if the reference knowledge domain of the above-mentioned user question includes the public network knowledge domain, then the above S3 may include the following steps 21 - step 22.

[0108] Step 21: Determine the answer corresponding to the public network knowledge domain based on the user's question and the search engine; the search engine is used to obtain the knowledge in this public network knowledge domain.

[0109] Among them, the search engine is used to obtain the knowledge in the public network knowledge domain; moreover, the implementation manner of this search engine is not limited in this application. For example, it can be implemented using any existing or future search engine that can perform search processing on the knowledge in the Internet. In addition, the number of this search engine is not limited in this application.

[0110] The answer corresponding to the public network knowledge domain refers to the result obtained by using the knowledge in the public network knowledge domain to reply to the above user's question; moreover, the determination process of the answer corresponding to the public network knowledge domain is not limited in this application. For example, specifically, it can be: first, use the search engine to perform search processing on the user's question to obtain some search results, so that these search results can represent the knowledge searched from the public network knowledge domain for this user's question; then, determine the answer corresponding to the public network knowledge domain based on these search results. For example, these search results can be directly determined as the answer corresponding to the public network knowledge domain. It can be seen that in a possible implementation manner, the above Step 21 can specifically be: use the search engine to perform search processing on the user's question to obtain the answer corresponding to the public network knowledge domain.

[0111] In addition, in order to better improve the answer quality, this application also provides a possible implementation manner of the above Step 21. In this implementation manner, Step 21 can specifically include the following Steps 211 - 213.

[0112] Step 211: Generate a search question based on the user's question.

[0113] Among them, the search question refers to the question required when using the search engine to perform search processing.

[0114] In addition, the implementation manner of the above Step 211 is not limited in this application. For example, specifically, it can be: directly determine part or all of the content in the user's question as the search question.

[0115] Another example is that, in order to better improve the search effect, the above Step 211 can specifically be: use a preset question conversion method to process the above user's question to obtain the search question. Among them, the preset question conversion method is used to convert the user's question into the search question; moreover, the preset question conversion method can be set in advance according to the actual application scenario. For example, the preset question conversion method can be implemented using a keyword extraction method or a question rewriting method, etc.

[0116] Also, to better improve the search effect, step 211 above can specifically be: input the above user question into a pre-constructed question generation model to obtain the search question output by the question generation model. Among them, the question generation model is used to perform question generation processing on the input data of the question generation model; moreover, the present application does not limit the implementation manner of the question generation model. For example, the question generation model can be implemented using any existing or future machine learning model.

[0117] Based on the relevant content of step 211 above, for some application scenarios, after obtaining the above user question, a pre-constructed question generation model can be used to perform question generation processing on the user question to obtain a search question, so that the search engine can perform search processing on the search question subsequently.

[0118] Step 212: Use the search engine corresponding to the public network knowledge field to perform search processing on the search question to obtain at least one search result.

[0119] Among them, at least one search result refers to the result obtained by the search engine performing search processing on the search question, so that the at least one search result can represent the knowledge searched from the public network knowledge field for the search question; moreover, the present application does not limit the at least one search result.

[0120] In addition, the present application does not limit the implementation manner of step 212 above. For example, in some application scenarios, this step 212 can be implemented by means of interface call.

[0121] Based on the relevant content of step 212 above, for some application scenarios, after determining the search question based on the above user question, the search question can be sent to the search engine through a search interface, so that the search engine can perform search processing based on the search question to obtain at least one search result, and feedback the at least one search result through the search interface. Among them, the search interface refers to the interface required when calling the search engine; moreover, the present application does not limit the implementation manner of the search interface.

[0122] Step 213: Determine the answer corresponding to the public network knowledge field based on at least one search result.

[0123] It should be noted that the present application does not limit the implementation manner of step 213 above. For example, it can specifically be: directly determining the above at least one search result as the answer corresponding to the public network knowledge field.

[0124] In addition, in some application scenarios, the search results provided by a search engine may be numerous and have relatively low accuracy. Therefore, in order to better improve the effect of answering questions, the present application also provides a possible implementation manner of step 213 above. In this implementation manner, step 213 may specifically include steps 2131-2132 below.

[0125] Step 2131: Determine a target result from at least one of the above search results according to the relevance characterization data between each search result and the user's question, where the relevance characterization data between the target result and the user's question is greater than the relevance characterization data between any other search result except the target result and the user's question among the at least one search result.

[0126] Among them, the relevance characterization data between the k-th search result and the user's question is used to characterize the degree of relevance between the k-th search result and the user's question; and if the relevance characterization data is larger, it means that the k-th search result is more relevant to the user's question, and thus it can be indicated that the k-th search result is more likely to provide some effective knowledge for the answering process of the user's question. k is a positive integer, k ≤ K, K is a positive integer, and K represents the number of results in at least one of the above search results.

[0127] In addition, the present application does not limit the determination process of the above relevance characterization data. For example, in order to improve the effect of relevance determination, the determination process of the relevance characterization data between the k-th search result and the user's question above may be: input the k-th search result and the user's question into a pre-constructed relevance determination model, and obtain the relevance characterization data output by the relevance determination model, such as a relevance prediction score, etc. Among them, the relevance determination model refers to a pre-set model that can measure the degree of relevance between two pieces of information; and the present application does not limit the implementation manner of the relevance determination model.

[0128] The target result refers to a search result that exists in at least one of the above search results and is relatively relevant to the user's question, such as Figure 2 or Figure 3 the public network information shown, etc.; and the present application does not limit the determination process of the target result. For example, specifically, it may be: according to the relevance characterization data between each search result and the user's question, sort the at least one of the above search results from high to low according to the degree of relevance to obtain a sorting result; then determine the top E search results in the sorting result as the target result, so as to be able to screen out the E search results that are most relevant to the user's question from the at least one search result. Among them, E is a positive integer, and the E can be set according to the actual application scenario.

[0129] Based on the relevant content of step 2131 above, after determining at least one search result based on the user's question above, first calculate the correlation characterization data between each search result and the user's question; then determine the target result from at least one search result above according to these correlation characterization data, so that the target result can represent a search result with a relatively high degree of relevance to the user's question, so as to be able to reply to the user's question based on the target result subsequently. In this way, it can effectively avoid the influence of useless content in the at least one search result on the question answering process, such as interfering with the answer quality and increasing the answer determination time, etc., thus being beneficial to the answer determination effect.

[0130] Step 2132: Generate an answer based on the target result and the user's question to obtain an answer corresponding to the public network knowledge field.

[0131] It should be noted that the present application does not limit the implementation manner of step 2132 above. For example, it can be implemented by using any existing or future method that can generate an answer based on a question and the relevant content of the question.

[0132] For another example, in order to better improve the answer quality, step 2132 above can specifically be: input the target result and the user's question into a pre-constructed answer generation model to obtain an answer corresponding to the public network knowledge field output by the answer generation model. Among them, the answer generation model is used to generate an answer for the input data of the answer generation model. In addition, the present application does not limit the working principle of the answer generation model. For example, in some application scenarios, when using the answer generation model to generate an answer for the target result and the user's question, the target result can be used as background knowledge. Based on this, it can be known that in a possible implementation manner, the answer generation model can be used to generate an answer for the user's question with the target result as background knowledge, and obtain and output the answer corresponding to the public network knowledge field. In addition, the present application does not limit the implementation manner of the answer generation model. For example, the answer generation model can adopt Figure 2 or Figure 4 the answer generation model for implementation.

[0133] Based on the relevant content from step 2131 to step 2132 above, for some application scenarios, after determining at least one search result based on the user question above, first screen out some search results that are most relevant to the user question from these search results as the public network information corresponding to the user question, so that the public network information can represent the effective knowledge existing in the public network knowledge field and related to the user question. In this way, it can effectively avoid the impact on the question answering process caused by the useless content in a large number of search results; then use the public network information as background knowledge to process the user question to obtain the answer corresponding to the public network knowledge field, so that the answer can better represent the answer result of the user question in the public network knowledge field. In this way, it is beneficial to improve the quality of the answer.

[0134] Based on the relevant content from step 211 to step 213 above, for some application scenarios, if it is determined that the knowledge in the public network knowledge field needs to be used to answer the user question, then a search question can be constructed based on the user question first; then call a search engine to perform a search process on the search question to obtain at least one search result; then, use all or part of these search results to process the user question to obtain the answer corresponding to the public network knowledge field, so that the answer can represent the answer result of the user question in the public network knowledge field.

[0135] Based on the relevant content from step 21 above, for some application scenarios, after determining the reference knowledge field of the user question above, if the reference knowledge field includes the public network knowledge field, it can be determined that the knowledge in the public network knowledge field needs to be used to answer the user question. Therefore, the answer process for the user question can be implemented by calling a search engine to obtain the answer corresponding to the public network knowledge field, so that the answer can represent the answer result of the user question in the public network knowledge field.

[0136] Step 22: Determine the answer to the user question above based on the answer corresponding to the public network knowledge field above.

[0137] It should be noted that the implementation method of step 22 above in this application is not limited. For example, if the reference knowledge field of the user question above is the public network knowledge field, it can be determined that only the knowledge in the public network knowledge field needs to be used to answer the user question. Therefore, this step 22 can specifically be: directly determine the answer corresponding to the public network knowledge field as the answer to the user question.

[0138] For another example, if the reference knowledge fields of the above user question include the public network knowledge field and the in-organization knowledge field, step 22 above may specifically be: Select the answer to the user question from the answers corresponding to the public network knowledge field and the answers corresponding to the in-organization knowledge field. Among them, the answer corresponding to the in-organization knowledge field refers to the result obtained by using the knowledge in the in-organization knowledge field to reply to the user question; and for the relevant content of the answer corresponding to the in-organization knowledge field, please refer to step 31 below.

[0139] Based on the relevant content of steps 21 to 22 above, for some application scenarios, after determining the reference knowledge fields of the above user question, if the reference knowledge fields include the public network knowledge field, the user question can be first replied to by means of a search engine to obtain the answer corresponding to the public network knowledge field; then, based on the answer corresponding to the public network knowledge field, the answer to the user question is determined, which is conducive to improving the quality of the answer.

[0140] Example 2, if the reference knowledge fields of the above user question include the in-organization knowledge field, S3 above may include steps 31 to 32 below.

[0141] Step 31: Determine the answer corresponding to the in-organization knowledge field according to the user question and the knowledge base of the in-organization knowledge field.

[0142] Among them, the answer corresponding to the in-organization knowledge field refers to the result obtained by using the knowledge in the in-organization knowledge field to reply to the above user question.

[0143] In addition, the present application does not limit the implementation manner of step 31 above. For example, it may specifically be: First, search the knowledge base of the above in-organization knowledge field for knowledge related to the user question; then, based on the searched knowledge, determine the answer corresponding to the in-organization knowledge field.

[0144] In addition, in order to better improve the quality of the answer, the answer corresponding to the above in-organization knowledge field may be obtained by processing the above user question using a pre-constructed in-organization knowledge model. Among them, the in-organization knowledge model is used to perform question-and-answer processing on the input data of the in-organization knowledge model; and the in-organization knowledge model is constructed based on the knowledge base of the in-organization knowledge field so that the in-organization knowledge model can perform question-and-answer processing on the user question according to the knowledge base. In addition, the present application does not limit the implementation manner of the in-organization knowledge model.

[0145] Based on the above content, in a possible implementation, when the above-mentioned in-organization knowledge model is constructed based on the knowledge base of the in-organization knowledge domain, step 31 above can specifically be: First, input the above-mentioned user question into the pre-constructed in-organization knowledge model to obtain the output data of the in-organization knowledge model, such as Figure 2 or Figure 4 the in-organization information shown, etc., so that the output data can represent the knowledge existing in the knowledge base and related to the user question, thereby enabling the output data to represent the knowledge required for answering the user question in the in-organization knowledge domain; then, perform answer generation processing based on the output data and the user question to obtain the answer corresponding to the in-organization knowledge domain.

[0146] It should be noted that this application does not limit the implementation manner of the step of "performing answer generation processing based on the output data and the user question to obtain the answer corresponding to the in-organization knowledge domain" above. For example, it is similar to the implementation manner of step 2132 above. Based on this, in a possible implementation, input the output data and the user question into the pre-constructed answer generation model, so that the answer generation model can perform answer generation processing on the user question with the output data as background knowledge, and obtain and output the answer corresponding to the in-organization knowledge domain.

[0147] In addition, in some application scenarios, when the above-mentioned in-organization knowledge domain includes N professional knowledge domains, and the reference knowledge domain of the above-mentioned user question includes the nth professional knowledge domain, step 31 can specifically be: Determine the answer corresponding to the in-organization knowledge domain based on the user question and the knowledge base of the nth professional knowledge domain.

[0148] It should be noted that this application does not limit the implementation manner of the step of "determining the answer corresponding to the in-organization knowledge domain based on the user question and the knowledge base of the nth professional knowledge domain" above. For example, it can be implemented by means of the professional knowledge model corresponding to the nth professional knowledge domain. Among them, the professional knowledge model corresponding to the nth professional knowledge domain is pre-constructed based on the knowledge base of the nth professional knowledge domain; and the professional knowledge model corresponding to the nth professional knowledge domain is used to perform question-and-answer processing on the input data of the model.

[0149] Based on the relevant content of step 31 above, for some application scenarios, after determining the reference knowledge area of the above user question, if the reference knowledge area includes the in-organization knowledge area, it can be determined that the knowledge in the in-organization knowledge area needs to be used to answer the user question. Therefore, the in-organization knowledge model constructed in advance can be used to process the user question to obtain the answer corresponding to the in-organization knowledge area, so that the answer can represent the answer result of the user question in the in-organization knowledge area. Among them, since the in-organization knowledge model is constructed based on the knowledge base of the in-organization knowledge area, the in-organization knowledge model can use the knowledge base for question-answering processing, so that the in-organization knowledge model can accurately determine the answer corresponding to the user question in the in-organization knowledge area, which is beneficial to improving the quality of the answer.

[0150] Step 32: Determine the answer to the user question based on the answer corresponding to the in-organization knowledge area above.

[0151] It should be noted that the implementation manner of step 32 above in this application is not limited. For example, if the reference knowledge area of the above user question is the in-organization knowledge area, it can be determined that only the knowledge in the in-organization knowledge area needs to be used to answer the user question. Therefore, this step 32 can specifically be: directly determine the answer corresponding to the in-organization knowledge area as the answer to the user question.

[0152] Another example, if the reference knowledge area of the above user question includes the public network knowledge area and the in-organization knowledge area, then step 32 above can specifically be: select the answer to the user question from the answer corresponding to the public network knowledge area and the answer corresponding to the in-organization knowledge area.

[0153] Based on the relevant content of steps 31 to 32 above, for some application scenarios, after determining the reference knowledge area of the above user question, if the reference knowledge area includes the in-organization knowledge area, the in-organization knowledge model constructed in advance for the in-organization knowledge area can be used to process the user question first to obtain the answer corresponding to the in-organization knowledge area; then, based on the answer corresponding to the in-organization knowledge area, determine the answer to the user question, which is beneficial to improving the quality of the answer.

[0154] In fact, in some application scenarios, to better improve the answer quality, the present application also provides a possible implementation manner of the answer to the above user question. In this implementation manner, if the above reference knowledge fields include the public network knowledge field and the intra-organization knowledge field, the answer to the user question can be determined based on the weight corresponding to the public network knowledge field, the answer corresponding to the public network knowledge field, the weight corresponding to the intra-organization knowledge field, and the answer corresponding to the intra-organization knowledge field. Among them, the answer corresponding to the public network knowledge field and the answer corresponding to the intra-organization knowledge field are both determined based on the user question. It should be noted that the present application does not limit the acquisition manner of these two weights. For example, when initializing and setting these two weights, the ratio between the weight corresponding to the intra-organization knowledge field and the weight corresponding to the public network knowledge field can be set to 10.

[0155] In addition, the present application does not limit the implementation manner of the determination process of the answer to the user question shown in the above paragraph. For the convenience of understanding, the following will be described in conjunction with Example 3.

[0156] Example 3, if the above reference knowledge fields include the public network knowledge field and the intra-organization knowledge field, then S3 above may include the following steps 41-step 43.

[0157] Step 41: Determine the answer corresponding to the public network knowledge field and the answer corresponding to the intra-organization knowledge field based on the user question.

[0158] It should be noted that for step 41 above, for the determination process of the answer corresponding to the public network knowledge field, please refer to step 21 above, and for the determination process of the answer corresponding to the intra-organization knowledge field, please refer to step 31 above.

[0159] Step 42: If the similarity between the answer corresponding to the public network knowledge field and the answer corresponding to the intra-organization knowledge field is higher than the preset similarity threshold, then randomly select one answer from the answer corresponding to the public network knowledge field and the answer corresponding to the intra-organization knowledge field as the answer to the user question.

[0160] Among them, the similarity between the answer corresponding to the public network knowledge field and the answer corresponding to the intra-organization knowledge field is used to represent the degree of similarity between the answer corresponding to the public network knowledge field and the answer corresponding to the intra-organization knowledge field; and the higher the similarity, the more similar the answer corresponding to the public network knowledge field and the answer corresponding to the intra-organization knowledge field are.

[0161] The preset similarity threshold refers to the lowest similarity value set in advance; and the preset similarity threshold can be set according to the actual application scenario.

[0162] Based on the relevant content of step 42 above, it can be known that if the reference knowledge fields of the user's question above include the public network knowledge field and the in-organization knowledge field, after obtaining the answer corresponding to the public network knowledge field and the answer corresponding to the in-organization knowledge field, first calculate the similarity between these two answers; then determine whether the similarity is higher than the preset similarity threshold. If it is higher, it can be determined that these two answers are relatively similar, so it can be determined that the semantic information carried by these two answers is roughly the same. Furthermore, it can be determined that the answers corresponding to the user's question in different fields are the same. Therefore, a random answer can be directly selected from the answer corresponding to the public network knowledge field and the answer corresponding to the in-organization knowledge field as the answer to the user's question.

[0163] Step 43: If the similarity between the answer corresponding to the public network knowledge field and the answer corresponding to the in-organization knowledge field is not higher than the preset similarity threshold, then determine the answer to the user's question based on the weight corresponding to the public network knowledge field, the answer corresponding to the public network knowledge field, the weight corresponding to the in-organization knowledge field, and the answer corresponding to the in-organization knowledge field.

[0164] It should be noted that the present application does not limit the implementation manner of step 43 above. For example, it can adopt any existing or future method for determining an answer from multiple answers based on weights, such as an answer fusion method or an answer selection method, etc., for implementation.

[0165] In addition, in order to better improve the answer quality, the present application also provides a possible implementation manner of step 43 above. In this implementation manner, when the above domain recognition result includes the confidence corresponding to the public network knowledge field and the confidence corresponding to the in-organization knowledge field, step 43 can specifically include the following steps 431 - step 433.

[0166] Step 431: Determine the answer usage probability corresponding to the public network knowledge field according to the product of the confidence corresponding to the public network knowledge field above and the weight corresponding to the public network knowledge field.

[0167] Among them, the answer usage probability corresponding to the public network knowledge field is used to represent the possibility of feeding back the answer corresponding to the public network knowledge field to the user. In addition, the answer usage probability is determined according to the product of the confidence corresponding to the public network knowledge field and the weight corresponding to the public network knowledge field, and the answer usage probability is positively correlated with the product.

[0168] Step 432: Determine the answer usage probability corresponding to the in-organization knowledge field according to the product of the confidence corresponding to the in-organization knowledge field above and the weight corresponding to the in-organization knowledge field.

[0169] Among them, the probability of the answer corresponding to the knowledge area within the organization is used to represent the possibility of feedback the answer corresponding to the knowledge area within the organization to the user. In addition, the probability of the answer is determined based on the product of the confidence corresponding to the knowledge area within the organization and the weight corresponding to the knowledge area within the organization, and the probability of the answer is positively correlated with the product.

[0170] Step 433: Based on the probability of the answer corresponding to the public network knowledge area and the probability of the answer corresponding to the knowledge area within the organization above, select the answer to the user's question from the answer corresponding to the public network knowledge area and the answer corresponding to the knowledge area within the organization.

[0171] It should be noted that the present application does not limit the implementation manner of step 433 above. For example, specifically, if the probability of the answer corresponding to the public network knowledge area above is greater than the probability of the answer corresponding to the knowledge area within the organization above, then determine the answer corresponding to the public network knowledge area as the answer to the user's question above; if the probability of the answer corresponding to the public network knowledge area is less than the probability of the answer corresponding to the knowledge area within the organization, then determine the answer corresponding to the knowledge area within the organization as the answer to the user's question; if the probability of the answer corresponding to the public network knowledge area is equal to the probability of the answer corresponding to the knowledge area within the organization, then randomly select an answer from the answer corresponding to the public network knowledge area and the answer corresponding to the knowledge area within the organization as the answer to the user's question.

[0172] Based on the relevant content of steps 431 to 433 above, it can be known that if the reference knowledge area of the user's question above includes the public network knowledge area and the knowledge area within the organization, then after obtaining the answer corresponding to the public network knowledge area and the answer corresponding to the knowledge area within the organization, if these two answers are similar, then a random answer can be selected from these two answers as the answer to the user's question; however, if the difference between these two answers is relatively large, then the answer to the user's question can be determined based on the confidence corresponding to the public network knowledge area, the weight corresponding to the public network knowledge area, the answer corresponding to the public network knowledge area, the confidence corresponding to the knowledge area within the organization, the weight corresponding to the knowledge area within the organization, and the answer corresponding to the knowledge area within the organization. In this way, the efficiency of answer determination can be provided on the premise of ensuring the quality of the answer, which is beneficial to improving the effect of answer determination.

[0173] Based on the relevant content of S1 to S3 above, for the answer determination method provided by the embodiments of the present application, after receiving a user question, first perform domain identification processing on the user question to obtain a domain identification result, so that the domain identification result can indicate which domain the user question belongs to; then, based on the domain identification result, determine the reference knowledge domain of the user question, so that the reference knowledge domain can indicate which domain of knowledge information needs to be referred to when answering the user question; then, based on the reference knowledge domain, determine the answer to the user question, so that the automatic determination of the answer can be realized. Among them, because the reference knowledge domain can indicate which domain of knowledge information needs to be referred to when answering the user question, so that the answer determined based on the reference knowledge domain can indicate the corresponding answer result of the user question in the reference knowledge domain, so that the answer can better meet the domain requirements, so that the accuracy impact caused by different answers to the same question in different domains can be effectively avoided, thus facilitating the improvement of the answer quality. In addition, because the reference knowledge domain includes at least one of the public network knowledge domain and the organization-internal knowledge domain, so that the reference knowledge domain can better indicate which domain of knowledge information needs to be referred to when answering the user question, so that the answer determined based on the reference knowledge domain is more accurate, and thus is conducive to improving the answer quality.

[0174] In addition, the present application does not limit the execution subject of the answer determination method provided by the embodiments of the present application. For example, the answer determination method provided by the embodiments of the present application can be applied to a terminal device. Another example is that the answer determination method provided by the embodiments of the present application can also be implemented by means of the data interaction process between the terminal device and the server. Among them, the terminal device can be a smart phone, a computer, a personal digital assistant (Personal Digital Assitant, PDA), a tablet computer, etc. The server can be an independent server, a cluster server or a cloud server.

[0175] In addition, in some application scenarios, in order to better improve the answer quality, the weights corresponding to the above-mentioned public network knowledge domain and the weights corresponding to the above-mentioned organization-internal knowledge domain can be dynamically updated based on user feedback. Based on this, the present application also provides a possible implementation manner of the above-mentioned answer determination method. In this implementation manner, when the answer to the above-mentioned user question is determined based on the weights corresponding to the public network knowledge domain and the weights corresponding to the organization-internal knowledge domain, the answer determination method can at least include the following steps 51-step 52. Among them, the execution time of step 51 is later than the execution time of S3 above.

[0176] Step 51: Display the answer to the above-mentioned user question.

[0177] It should be noted that the implementation manner of step 51 above is not limited in this application. For example, it can be implemented by using any existing or future method that can display the answer to the user.

[0178] Step 52: After obtaining the feedback information on the answer to the above user question, if the feedback information indicates that the answer to the user question is inaccurate, update the weights corresponding to the public network knowledge field and the weights corresponding to the knowledge field within the organization.

[0179] Among them, the feedback information is used to represent the feedback status of the user on the answer to the above user question, such as the status of approval or disapproval, etc.; and the implementation manner of this feedback information is not limited in this application. For example, the feedback information can be determined based on the operations triggered by the user on the answer to the user question, such as like operations or dislike operations, etc.

[0180] In addition, the implementation manner of the step of "updating the weights corresponding to the public network knowledge field and the weights corresponding to the knowledge field within the organization" in step 52 above is not limited in this application. For example, it may specifically include at least one of the following steps 521-step 522.

[0181] Step 521: If the answer to the above user question is the answer corresponding to the public network knowledge field, reduce the weight corresponding to the public network knowledge field and increase the weight corresponding to the knowledge field within the organization.

[0182] In this application, after determining to display the answer corresponding to the public network knowledge field as the answer to the user question based on the weights corresponding to the public network knowledge field and the weights corresponding to the knowledge field within the organization, if the feedback information given by the user for this answer indicates that the answer is inaccurate, it can be inferred that the user may want to obtain the answer corresponding to the knowledge field within the organization. Therefore, the weight corresponding to the public network knowledge field can be reduced and the weight corresponding to the knowledge field within the organization can be increased, so as to be able to select an answer that meets the user's needs as much as possible based on the updated weights in the next round of question-and-answer process, thereby helping to improve the answer quality.

[0183] Step 522: If the answer to the above user question is the answer corresponding to the knowledge field within the organization, reduce the weight corresponding to the knowledge field within the organization and increase the weight corresponding to the public network knowledge field.

[0184] In this application, after determining that the answer corresponding to the knowledge domain within the organization will be presented as the answer to the user's question based on the weights corresponding to the public network knowledge domain and the knowledge domain within the organization, if the feedback information given by the user regarding this answer indicates that the answer is inaccurate, it can be inferred that the user may want to obtain the answer corresponding to the public network knowledge domain. Therefore, the weight corresponding to the knowledge domain within the organization can be decreased, and the weight corresponding to the public network knowledge domain can be increased, so that in the next round of the Q&A process, an answer that meets the user's needs can be selected as much as possible based on the updated weights, thereby facilitating the improvement of the answer quality.

[0185] Based on the relevant content of steps 51 to 52 above, for some application scenarios, after determining that the answer corresponding to the public network knowledge domain will be the answer to the user's question based on the weights corresponding to the public network knowledge domain and the knowledge domain within the organization, the answer to the user's question can be presented first; then the feedback information of the user regarding this answer can be obtained, so that when the feedback information indicates that the answer is inaccurate, the weights corresponding to the public network knowledge domain and the knowledge domain within the organization can be updated, so that in the next round of the Q&A process, an answer that meets the user's needs can be selected as much as possible based on the updated weights, thereby facilitating the improvement of the answer quality.

[0186] In fact, in some application scenarios, in order to better improve the answer quality, some objects involved in the answer determination process, such as models, knowledge bases, etc., can be updated based on user feedback. Based on this, this application also provides a possible implementation manner of the above answer determination method. In this implementation manner, the answer determination method can at least include the following steps 61 - step 62. Among them, the execution time of step 61 is later than the execution time of S3 above.

[0187] Step 61: Present the answer to the above user's question.

[0188] It should be noted that for the relevant content of step 61, please refer to step 51 above.

[0189] Step 62: After receiving the feedback information regarding the answer to the above user's question, if the feedback information indicates that the answer to the user's question is inaccurate, a negative sample is constructed based on the user's question and the answer to the user's question, so that the negative sample is used to update some objects involved in the answer determination process, such as models, knowledge bases, etc.

[0190] Among them, a negative sample refers to the sample data required to be based on during the update process; moreover, this application does not limit the implementation manner of the negative sample. For example, if the feedback information regarding the answer to the above user's question indicates that the answer to the user's question is inaccurate, the negative sample can include the user's question and the answer to the user's question.

[0191] In addition, the present application does not limit the usage mode of the above-mentioned negative samples. For the sake of easy understanding, some situations will be described below.

[0192] Situation 1: If the above feedback information is used to indicate that the answer corresponding to the knowledge area within the organization is inaccurate, it can be inferred that the determination process of the answer corresponding to the knowledge area within the organization may need to be optimized, and it can be inferred that the above domain recognition process may need to be optimized. Among them, if both the determination process of the answer corresponding to the knowledge area within the organization and the domain recognition process are implemented based on the knowledge base of the knowledge area within the organization, it can be inferred that the knowledge base of the knowledge area within the organization may need to be optimized.

[0193] Situation 2: If the above feedback information is used to indicate that the answer corresponding to the public network knowledge area is inaccurate, it can be inferred that the determination process of the answer corresponding to the public network knowledge area may need to be optimized, and it can be inferred that the domain recognition process may need to be optimized. Among them, if the domain recognition process is implemented based on the knowledge base of the knowledge area within the organization, it can be inferred that the knowledge base of the knowledge area within the organization may need to be optimized. In addition, if the determination process of the answer corresponding to the public network knowledge area is implemented based on some models, it can be inferred that these models need to be optimized.

[0194] Based on the above two paragraphs, it can be seen that in some scenarios, such as scenarios where response processing is based on a single domain, if the answer is inaccurate, it can be inferred that this inaccuracy may be caused by defects in the knowledge base of the knowledge area within the organization. Therefore, the knowledge base of the knowledge area within the organization can be updated using the above negative samples. Based on this, in a possible implementation manner, if the answer to the above user question is determined based on the knowledge base of the knowledge area within the organization, such as the above domain recognition result is determined based on the knowledge base of the knowledge area within the organization, the answer corresponding to the knowledge area within the organization is determined based on the knowledge base of the knowledge area within the organization, etc., then the negative sample can be used to update the knowledge base of the knowledge area within the organization; moreover, the present application does not limit the update method of the knowledge base.

[0195] In addition, in some scenarios, if the above knowledge model within the organization is constructed based on the knowledge base of the knowledge area within the organization, after the knowledge base is updated, it is necessary to update the knowledge model within the organization based on the updated knowledge base, which is beneficial to improving the answer quality.

[0196] In addition, in some application scenarios, if the answer determination process for the public network knowledge domain and the answer determination process for the internal knowledge domain above both use an answer generation model, so that when the answer is inaccurate, it can be speculated that this inaccuracy may be caused by a defect in the answer generation model. Therefore, the negative samples above can be used to update the answer generation model. Based on this, it can be known that in a possible implementation manner, if the answer to the user question above is determined by using a pre-constructed answer generation model, then the negative sample can be used to update the answer generation model so that the updated answer generation model has better performance; and this application does not limit the update method of the answer generation model.

[0197] Furthermore, in some application scenarios, the phenomenon that the answer corresponding to the public network knowledge domain is inaccurate may be caused by a defect in the generation process of the search question. Therefore, the negative samples above can be used to optimize the generation process of the search question. Based on this, it can be known that in a possible implementation manner, if the answer to the user question above is determined based on the search question, and the search question is obtained by processing the user question using a pre-constructed question generation model, then the negative sample can be used to update the question generation model.

[0198] Based on the relevant content of steps 61 to 62 above, for some application scenarios, after obtaining the answer to the user question, the answer to the user question can be displayed first; then the feedback information of the user regarding the answer can be obtained, so that when the feedback information indicates that the answer is inaccurate, a negative sample can be constructed first based on the user question and the answer to the user question; then the negative sample can be used to update the knowledge base and model involved in the answer determination process of the user question, so that the answer determination process can be optimized based on user feedback, which is beneficial to improving the answer quality.

[0199] Based on the relevant content of the answer determination method above, the technical solution provided by this application has the advantages shown in (i) to (iv) below.

[0200] (i) Since this application can accurately determine which domain of knowledge needs to be used when answering a user question, it can effectively avoid problems such as answer confusion or inaccuracy caused by not distinguishing domains, which is beneficial to improving the answer quality.

[0201] (ii) Since this application can obtain complete public network information related to the user question through the relevance judgment process, it can effectively avoid interference caused by a large amount of irrelevant content carried by the knowledge obtained by searching through a search engine, which is beneficial to improving the answer quality.

[0202] (3) Since this application selects the final answer from multiple answers by means of weights, it is conducive to improving the answer determination efficiency on the premise of ensuring the answer quality.

[0203] (4) This application affects the answer determination process based on the user feedback closed-loop. Specifically, it intervenes in the weights corresponding to different fields in real time based on user feedback, which is conducive to achieving a quick response to the Q&A results; through the long-term feedback of feedback data, it realizes the incremental optimization of relevant models in the answer determination process and provides guiding information such as optimization, modification, and supplementation for the knowledge base corresponding to the knowledge fields within the organization, which is conducive to improving the answer quality.

[0204] Based on the above (1) to (4), it can be seen that the technical solution provided by this application can better realize knowledge Q&A under the complex knowledge background, which is conducive to providing more accurate knowledge Q&A services to users, thereby improving the user experience.

[0205] Based on the answer determination method provided by the embodiments of this application, the embodiments of this application also provide an answer determination device, which will be explained and described below in combination with Figure 5 Among them, Figure 5 is a schematic structural diagram of an answer determination device provided by the embodiments of this application. It should be noted that for the technical details of the answer determination device provided by the embodiments of this application, please refer to the relevant content of the above answer determination method.

[0206] As Figure 5 shown, the answer determination device 500 provided by the embodiments of this application includes:

[0207] A domain identification unit 501, configured to perform domain identification processing on the user question after receiving the user question, and obtain a domain identification result;

[0208] A first determination unit 502, configured to determine a reference knowledge domain of the user question according to the domain identification result; the reference knowledge domain includes at least one of a public network knowledge domain and an intra-organization knowledge domain;

[0209] A second determination unit 503, configured to determine an answer to the user question according to the reference knowledge domain.

[0210] In a possible implementation manner, the domain identification result includes a confidence level corresponding to the public network knowledge domain and a confidence level corresponding to the intra-organization knowledge domain;

[0211] The first determination unit 502 includes at least one of the following sub-units:

[0212] A first response subunit, configured to determine the public network knowledge domain as the reference knowledge domain of the user question in response to the confidence level corresponding to the public network knowledge domain being higher than a first threshold;

[0213] A second response subunit, configured to determine the in-organization knowledge domain as the reference knowledge domain of the user question in response to the confidence level corresponding to the in-organization knowledge domain being higher than a second threshold;

[0214] A third response subunit, configured to determine the reference knowledge domain of the user question based on the public network knowledge domain and the in-organization knowledge domain in response to the confidence level corresponding to the public network knowledge domain not being higher than the first threshold and the confidence level corresponding to the in-organization knowledge domain not being higher than the second threshold.

[0215] In a possible implementation manner, if the reference knowledge domain includes the public network knowledge domain, the second determination unit 503 includes:

[0216] A first determination subunit, configured to determine the answer corresponding to the public network knowledge domain based on the user question and a search engine; the search engine is used to obtain knowledge in the public network knowledge domain;

[0217] A second determination subunit, configured to determine the answer to the user question based on the answer corresponding to the public network knowledge domain.

[0218] In a possible implementation manner, the first determination subunit is specifically configured to: generate a search question based on the user question; perform a search process on the search question by using the search engine to obtain at least one search result; determine the answer corresponding to the public network knowledge domain based on the at least one search result.

[0219] In a possible implementation manner, the process of determining the answer corresponding to the public network knowledge domain includes: determining a target result from the at least one search result according to the correlation characterization data between each search result and the user question, where the correlation characterization data between the target result and the user question is greater than the correlation characterization data between any other search result except the target result and the user question among the at least one search result; performing answer generation processing based on the target result and the user question to obtain the answer corresponding to the public network knowledge domain.

[0220] In a possible implementation manner, if the reference knowledge domain includes the in-organization knowledge domain, the second determination unit 503 includes:

[0221] A third determination subunit, configured to determine the answer corresponding to the in-organization knowledge domain based on the user question and the knowledge base of the in-organization knowledge domain;

[0222] A fourth determination subunit, configured to determine an answer to the user question according to the answer corresponding to the knowledge domain within the organization.

[0223] In a possible implementation manner, the answer corresponding to the knowledge domain within the organization is obtained by processing the user question using a pre-constructed knowledge model within the organization; the knowledge model within the organization is constructed according to the knowledge base of the knowledge domain within the organization.

[0224] In a possible implementation manner, if the reference knowledge domain includes the public network knowledge domain and the knowledge domain within the organization, the answer to the user question is determined according to the weight corresponding to the public network knowledge domain, the answer corresponding to the public network knowledge domain, the weight corresponding to the knowledge domain within the organization, and the answer corresponding to the knowledge domain within the organization; the answer corresponding to the public network knowledge domain and the answer corresponding to the knowledge domain within the organization are both determined according to the user question.

[0225] In a possible implementation manner, the domain recognition result includes the confidence level corresponding to the public network knowledge domain and the confidence level corresponding to the knowledge domain within the organization;

[0226] The second determination unit 503 is specifically configured to: determine the usage probability of the answer corresponding to the public network knowledge domain according to the product of the confidence level corresponding to the public network knowledge domain and the weight corresponding to the public network knowledge domain; determine the usage probability of the answer corresponding to the knowledge domain within the organization according to the product of the confidence level corresponding to the knowledge domain within the organization and the weight corresponding to the knowledge domain within the organization; select the answer to the user question from the answer corresponding to the public network knowledge domain and the answer corresponding to the knowledge domain within the organization according to the usage probability of the answer corresponding to the public network knowledge domain and the usage probability of the answer corresponding to the knowledge domain within the organization.

[0227] In a possible implementation manner, the answer to the user question is determined according to the weight corresponding to the public network knowledge domain and the weight corresponding to the knowledge domain within the organization;

[0228] The answer determination device 500 further includes:

[0229] An answer display unit, configured to display the answer to the user question;

[0230] A weight update unit, configured to update the weight corresponding to the public network knowledge domain and the weight corresponding to the knowledge domain within the organization after obtaining feedback information on the answer to the user question, if the feedback information indicates that the answer to the user question is inaccurate.

[0231] In a possible implementation, if the answer to the user question is the answer corresponding to the public network knowledge domain, the weight update unit is specifically configured to: reduce the weight corresponding to the public network knowledge domain and increase the weight corresponding to the in-organization knowledge domain.

[0232] In a possible implementation, if the answer to the user question is the answer corresponding to the in-organization knowledge domain, the weight update unit is specifically configured to: reduce the weight corresponding to the in-organization knowledge domain and increase the weight corresponding to the public network knowledge domain.

[0233] In a possible implementation, the answer to the user question is determined based on the knowledge base of the in-organization knowledge domain;

[0234] The answer determination device 500 further includes:

[0235] An answer display unit, configured to display the answer to the user question;

[0236] A sample construction unit, configured to, after obtaining feedback information on the answer to the user question, if the feedback information indicates that the answer to the user question is inaccurate, construct a negative sample based on the user question and the answer to the user question; the negative sample is used to update the knowledge base of the in-organization knowledge domain.

[0237] In a possible implementation, the answer to the user question is determined by using a pre-constructed answer generation model, and the negative sample is further used to update the answer generation model;

[0238] And / or,

[0239] The answer to the user question is determined based on a search question, and the search question is obtained by processing the user question by using a pre-constructed question generation model, and the negative sample is further used to update the question generation model.

[0240] In a possible implementation, the in-organization knowledge domain includes at least one professional knowledge domain.

[0241] Based on the relevant content of the answer determination device 500, for the answer determination device 500 provided in the embodiments of the present application, after receiving a user question, the user question is first subjected to domain recognition processing to obtain a domain recognition result, so that the domain recognition result can indicate which domain the user question belongs to; then, based on the domain recognition result, a reference knowledge domain of the user question is determined, so that the reference knowledge domain can indicate which domain of knowledge information needs to be referred to when answering the user question; then, based on the reference knowledge domain, an answer to the user question is determined, so that automatic answer determination can be achieved. Among them, since the reference knowledge domain can indicate which domain of knowledge information needs to be referred to when answering the user question, so that the answer determined based on the reference knowledge domain can indicate the corresponding answer result of the user question in the reference knowledge domain, thereby enabling the answer to better meet the domain requirements, so that the accuracy impact caused by different answers to the same question in different domains can be effectively avoided, which is beneficial to improving the answer quality. In addition, since the reference knowledge domain includes at least one of the public network knowledge domain and the intra-organization knowledge domain, so that the reference knowledge domain can better indicate which domain of knowledge information needs to be referred to when answering the user question, so that the answer determined based on the reference knowledge domain is more accurate, which is beneficial to improving the answer quality.

[0242] In addition, an embodiment of the present application further provides an electronic device, the device includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any implementation manner of the answer determination method provided in the embodiments of the present application.

[0243] See Figure 6 , which shows a schematic structural diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitations to the functions and usage scopes of the embodiments of the present disclosure.

[0244] Such as Figure 6As shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0245] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 6 the electronic device 600 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Alternatively, more or fewer devices may be implemented or had.

[0246] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above functions defined in the method of the embodiment of the present disclosure are executed.

[0247] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept. The technical details not described in detail in this embodiment may be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0248] An embodiment of the present application also provides a computer-readable medium, in which instructions or a computer program are stored. When the instructions or the computer program run on a device, the device is caused to execute any implementation manner of the answer determination method provided by the embodiment of the present application.

[0249] It should be noted that the computer-readable medium described above in the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0250] In some embodiments, the client and the server may communicate using any currently known or future-developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0251] The above computer-readable medium may be included in the above electronic device; or it may exist separately without being assembled into the electronic device.

[0252] The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device can execute the above method.

[0253] Computer program code for performing the operations of this disclosure may be written in one or more programming languages or combinations thereof. The foregoing programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or alternatively, may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0254] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a special-purpose hardware-based system that performs the specified functions or operations, or by a combination of special-purpose hardware and computer instructions.

[0255] The units described in the embodiments of the present disclosure may be implemented in software or in hardware. Among them, the name of the unit / module does not constitute a limitation to the unit itself in some cases.

[0256] The functions described above herein may be performed, at least in part, by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on a chip (SOC), complex programmable logic devices (CPLD), and the like.

[0257] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0258] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For the same or similar parts among the various embodiments, reference may be made to each other. For the systems or apparatuses disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference may be made to the description in the method part for related parts.

[0259] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may mean: only A exists, only B exists, and both A and B exist simultaneously. Here, A and B may be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or similar expressions refer to any combination of these items, including any combination of single items (ones) or plural items (ones). For example, at least one (one) of a, b, or c may mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c may be single or multiple.

[0260] It should also be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0261] The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination thereof. The software modules may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0262] The foregoing description of the disclosed embodiments enables those skilled in the art within the organization to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art within the organization, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining an answer, characterized in that, The method includes: After receiving a user question, performing domain recognition processing on the user question to obtain a domain recognition result; According to the domain recognition result, determining a reference knowledge domain of the user question; the reference knowledge domain includes at least one of a public network knowledge domain and an in-organization knowledge domain; According to the reference knowledge domain, determining an answer to the user question.

2. The method according to claim 1, characterized in that, The domain recognition result includes a confidence level corresponding to the public network knowledge domain and a confidence level corresponding to the in-organization knowledge domain; The process of determining the reference knowledge domain includes at least one of the following: In response to the confidence level corresponding to the public network knowledge domain being higher than a first threshold, determining the public network knowledge domain as the reference knowledge domain of the user question; In response to the confidence level corresponding to the in-organization knowledge domain being higher than a second threshold, determining the in-organization knowledge domain as the reference knowledge domain of the user question; In response to the confidence level corresponding to the public network knowledge domain not being higher than the first threshold and the confidence level corresponding to the in-organization knowledge domain not being higher than the second threshold, determining the reference knowledge domain of the user question according to the public network knowledge domain and the in-organization knowledge domain.

3. The method according to claim 1, wherein If the reference knowledge domain includes the public network knowledge domain, the process of determining the answer to the user question includes: According to the user question and a search engine, determining an answer corresponding to the public network knowledge domain; the search engine is used to obtain knowledge in the public network knowledge domain; According to the answer corresponding to the public network knowledge domain, determining the answer to the user question.

4. The method according to claim 3, wherein The process of determining the answer corresponding to the public network knowledge domain includes: Generating a search question according to the user question; Using the search engine to perform a search process on the search question to obtain at least one search result; According to the at least one search result, determining the answer corresponding to the public network knowledge domain.

5. The method according to claim 4, wherein The determining the answer corresponding to the public network knowledge domain according to the at least one search result includes: According to the correlation characterization data between each search result and the user question, determining a target result from the at least one search result, and the correlation characterization data between the target result and the user question is greater than the correlation characterization data between any other search result except the target result and the user question among the at least one search result; Performing answer generation processing according to the target result and the user question to obtain the answer corresponding to the public network knowledge domain.

6. The method according to claim 1, wherein If the reference knowledge domain includes the in-organization knowledge domain, the process of determining the answer to the user question includes: According to the user question and the knowledge base of the in-organization knowledge domain, determining an answer corresponding to the in-organization knowledge domain; According to the answer corresponding to the in-organization knowledge domain, determining the answer to the user question.

7. The method according to claim 6, characterized in that, The answer corresponding to the in-organization knowledge domain is obtained by processing the user question using a pre-constructed in-organization knowledge model; the in-organization knowledge model is constructed according to the knowledge base of the in-organization knowledge domain.

8. The method according to claim 1, characterized in that If the reference knowledge field includes the public network knowledge field and the in-organization knowledge field, the answer to the user question is determined based on the weight corresponding to the public network knowledge field, the answer corresponding to the public network knowledge field, the weight corresponding to the in-organization knowledge field, and the answer corresponding to the in-organization knowledge field; both the answer corresponding to the public network knowledge field and the answer corresponding to the in-organization knowledge field are determined based on the user question.

9. The method according to claim 8, wherein The field recognition result includes the confidence level corresponding to the public network knowledge field and the confidence level corresponding to the in-organization knowledge field; The process of determining the answer to the user question includes: Determining the usage probability of the answer corresponding to the public network knowledge field based on the product of the confidence level corresponding to the public network knowledge field and the weight corresponding to the public network knowledge field; Determining the usage probability of the answer corresponding to the in-organization knowledge field based on the product of the confidence level corresponding to the in-organization knowledge field and the weight corresponding to the in-organization knowledge field; Selecting the answer to the user question from the answer corresponding to the public network knowledge field and the answer corresponding to the in-organization knowledge field based on the usage probability of the answer corresponding to the public network knowledge field and the usage probability of the answer corresponding to the in-organization knowledge field.

10. The method according to claim 1, characterized in that The answer to the user question is determined based on the weight corresponding to the public network knowledge field and the weight corresponding to the in-organization knowledge field; The method further includes: Displaying the answer to the user question; After obtaining the feedback information regarding the answer to the user question, if the feedback information indicates that the answer to the user question is inaccurate, updating the weight corresponding to the public network knowledge field and the weight corresponding to the in-organization knowledge field.

11. The method according to claim 10, wherein If the answer to the user question is the answer corresponding to the public network knowledge field, the updating the weight corresponding to the public network knowledge field and the weight corresponding to the in-organization knowledge field includes: Reducing the weight corresponding to the public network knowledge field and increasing the weight corresponding to the in-organization knowledge field; Or, If the answer to the user question is the answer corresponding to the in-organization knowledge field, the updating the weight corresponding to the public network knowledge field and the weight corresponding to the in-organization knowledge field includes: Reducing the weight corresponding to the in-organization knowledge field and increasing the weight corresponding to the public network knowledge field.

12. The method according to claim 1, wherein The answer to the user question is determined based on the knowledge base of the in-organization knowledge field; The method further includes: Displaying the answer to the user question; After obtaining the feedback information regarding the answer to the user question, if the feedback information indicates that the answer to the user question is inaccurate, constructing a negative sample based on the user question and the answer to the user question; the negative sample is used to update the knowledge base of the in-organization knowledge field.

13. The method according to claim 12, wherein The answer to the user question is determined using a pre-constructed answer generation model, and the negative sample is also used to update the answer generation model; And / or, The answer to the user question is determined based on the search question, which is obtained by processing the user question using a pre-constructed question generation model. The negative samples are also used to update the question generation model.

14. The method according to claim 1, wherein The knowledge fields within the organization include at least one professional knowledge field.

15. An answer determination device, characterized in that, Comprising: A field identification unit, configured to perform field identification processing on the user question after receiving the user question, to obtain a field identification result; A first determination unit, configured to determine a reference knowledge field of the user question according to the field identification result; The reference knowledge field includes at least one of a public network knowledge field and a knowledge field within the organization; A second determination unit, configured to determine an answer to the user question according to the reference knowledge field.

16. An electronic device, characterized in that, The device includes: a processor and a memory; The memory is configured to store instructions or computer programs; The processor is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the method according to any one of claims 1-14.

17. A computer-readable medium, characterized in that, Instructions or computer programs are stored in the computer-readable medium, and when the instructions or computer programs run on the device, the device executes the method according to any one of claims 1-14.

Images