Answer determination method and device, medium and program product

Abstract

The invention discloses an answer determination method and device, a medium and a program product, and relates to the field of artificial intelligence, and the method comprises the steps: obtaining user question information of a target user; inputting the user question information into a pre-trained semantic analysis model to obtain first user intention information output by the semantic analysis model; wherein the first user intention information comprises a target keyword; determining a target knowledge domain according to the target keyword, and determining at least one type of target knowledge information corresponding to the target keyword in the target knowledge domain; and determining a target answer according to the at least one type of target knowledge information, and sending the target answer to target user equipment of the target user. According to the scheme, the matching degree of the answers and the questions is improved by analyzing the semantic information of the user, the accuracy of outputting the answers is improved, and the use experience of the user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to an answer determination method, device, medium and program product. BACKGROUND

[0002] With the development of technology in the financial field, users are more and more used to using the question and answer function of the application program to understand the relevant information in the financial field.

[0003] At present, the existing question and answer function usually performs keyword matching on the keywords in the user's question after the user asks a question, returns a preset answer after the corresponding keywords are matched, and transfers the user to a manual service when the corresponding keywords cannot be matched.

[0004] However, the question answering mode of keyword matching is usually implemented based on a regular expression matching engine, which cannot understand complex semantic information, so that the correctness of the answer retrieval result is low, and the user's use experience is affected. SUMMARY

[0005] The present application provides an answer determination method, device, medium and program product to solve the problem that the question answering mode of keyword matching in the prior art is usually implemented based on a regular expression matching engine, which cannot understand complex semantic information, so that the correctness of the answer retrieval result is low, and the user's use experience is affected.

[0006] In a first aspect, the present application provides an answer determination method, which comprises:

[0007] obtaining user question information of a target user;

[0008] inputting the user question information into a pre-trained semantic analysis model to obtain first user intent information output by the semantic analysis model; wherein the first user intent information includes a target keyword;

[0009] determining a target knowledge domain according to the target keyword, and determining at least one target knowledge information corresponding to the target keyword in the target knowledge domain; wherein the target knowledge domain is one knowledge domain in a pre-constructed knowledge graph, the pre-constructed knowledge graph includes a plurality of knowledge domains, and the type of the target knowledge information includes structured knowledge and unstructured documents;

[0010] determining a target answer according to the at least one target knowledge information, and sending the target answer to a target user device of the target user.

[0011] In a second aspect, the present application provides an answer determination device, which comprises:

[0012] an acquisition module configured to acquire user question information of a target user;

[0013] an input module configured to input the user question information into a pre-trained semantic analysis model to obtain first user intent information output by the semantic analysis model, wherein the first user intent information comprises a target keyword;

[0014] a determination module configured to determine a target knowledge domain according to the target keyword, and determine at least one target knowledge information corresponding to the target keyword in the target knowledge domain, wherein the target knowledge domain is one knowledge domain in a pre-constructed knowledge graph, the pre-constructed knowledge graph comprises a plurality of knowledge domains, and the type of the target knowledge information comprises structured knowledge and unstructured document;

[0015] a sending module configured to determine a target answer according to the at least one target knowledge information, and send the target answer to a target user device of the target user.

[0016] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the answer determination method according to any one of the embodiments of the present application.

[0017] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein the program is executed by a processor to implement the answer determination method according to any one of the embodiments of the present application.

[0018] In a fifth aspect, the present application provides a computer program product, comprising a computer program, wherein the computer program is executed by a processor to implement the answer determination method according to any one of the embodiments of the present application.

[0019] The proposed solution involves: acquiring user question information from a target user; inputting the user question information into a pre-trained semantic analysis model to obtain first user intent information output by the semantic analysis model; wherein the first user intent information includes target keywords; determining a target knowledge domain based on the target keywords, and determining at least one type of target knowledge information corresponding to the target keywords within the target knowledge domain; wherein the target knowledge domain is one knowledge domain in a pre-constructed knowledge graph, the pre-constructed knowledge graph includes multiple knowledge domains, and the types of target knowledge information include structured knowledge and unstructured documents; determining a target answer based on at least one type of target knowledge information, and sending the target answer to the target user's target user device. In other words, the proposed solution inputs user question information into a semantic analysis model to obtain user intent, determines the knowledge information corresponding to the question based on the user intent, and then determines the answer corresponding to the question based on the knowledge information. This improves the matching degree between the answer and the question by analyzing the user's semantic information, avoids situations where complex semantic information cannot be understood, thereby improving the accuracy of the output answer and enhancing the user experience. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the method for determining the answer provided in this application;

[0022] Figure 2 This is a schematic diagram of the training process of the semantic analysis model of the answer determination method provided in this application;

[0023] Figure 3 This is a schematic diagram of the answer-determining device provided in this application;

[0024] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] In the technical solution of the present application, the collected information is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards of relevant countries and regions, necessary security measures are taken, public order and good customs are not violated, and appropriate operation portals are provided for the user to choose authorization or refusal.

[0027] Figure 1 is a flowchart of the answer determination method provided by the present application. The method can be executed by an answer determination device, which can be realized in the form of software and / or hardware. In a specific embodiment, the device can be applied in an electronic device, which can be a computer. The following embodiments will be described by taking the device applied in an electronic device as an example. Referring to Figure 1 , the method can specifically include the following steps:

[0028] Step 101: obtaining user question information of a target user.

[0029] Specifically, the electronic device executing the present embodiment can be a device integrated with a user interaction module. The user inputs a question through the user interaction module, and the electronic device executing the present embodiment receives the question, thereby obtaining the user question information of the target user. The user question information can be a question described by the user in natural language, for example, how to solve the problem of error code D0002 encountered when calling an Application Programming Interface (API) service. The user question information can also be a preset question determined by the user using a predefined instruction to quickly specify, for example, querying API related error information using a predefined instruction to quickly specify.

[0030] After obtaining the input information of the target user, the information can be preprocessed, for example, authenticating the identity information of the user, filtering sensitive words from the input information of the user through a natural language processing front-end module, or standardizing the input information of the user, for example, removing redundant words and correcting spelling errors, to improve the accuracy of subsequent intent recognition. In addition, the historical dialogue information of the target user can be obtained, and the historical questions and replies of the user in the same session are recorded to improve the interaction efficiency.

[0031] Step 102: inputting the user question information into a pre-trained semantic analysis model to obtain first user intent information output by the semantic analysis model.

[0032] The first user intent information includes a target keyword.

[0033] Specifically, the semantic analysis model is a pre-trained model, which can be a deep learning model or a large language model. The input content of the semantic analysis model is the pre-processed user question text, and the output content is the first user intent information. The first user intent information includes a target keyword. The target keyword is used as a retrieval key value to quickly narrow down the candidate set in the knowledge graph.

[0034] Optionally, step 102 can be implemented through steps 1021 to 1023.

[0035] Step 1021, identifying the question mark of the user question information to obtain an identification result.

[0036] The identification result includes an intent instruction mark and an absence of an intent instruction mark.

[0037] Specifically, the question mark refers to a specific symbol or keyword used by the user in the question to explicitly indicate the query intent or the identification of the target knowledge domain. For example, the specific symbol is @, and the user may use @API error or @document download in the question to indicate the direct retrieval of a specific type of knowledge. The intent instruction mark is used to explicitly specify the user's query intent and the corresponding target knowledge domain. For example, the @API error mark explicitly indicates the retrieval of content related to API error. Identifying the question mark of the user question information can be to use a pre-defined regular expression rule to identify the question mark in the user question, and the regular expression rule library contains multiple possible intent instruction mark patterns.

[0038] The identification result includes an intent instruction mark and an absence of an intent instruction mark. If the regular expression matches the intent instruction mark explicitly included in the user question, the identification result is marked as an intent instruction mark. For example, the user inputs @API error, D0002. The @API error is identified as an intent instruction mark, and the question is directly located to the API error knowledge domain. For questions with an intent instruction mark, the corresponding retrieval module is directly called to quickly retrieve related knowledge information in the target knowledge domain. If the user question does not have an explicit intent instruction mark, the identification result is marked as an absence of an intent instruction mark. For example, the user inputs API error how to solve, which is an absence of an intent instruction mark. By identifying the question mark in the user question information, it can be quickly determined whether the user has explicitly specified an intent instruction mark, and different processing strategies are adopted accordingly, improving the retrieval efficiency.

[0039] Step 1022, if the identification result is an intent instruction mark, determining the second user intent information according to the indication information of the user question information.

[0040] The second user intent information includes a target keyword.

[0041] Specifically, the indication information refers to the specific description or keywords in the user question related to the intent instruction identifier, which is used to further clarify the user's demand. If the identification result of the identifier is that there is an intent instruction identifier, other parts of the user question are determined, and the indication information related to the intent is extracted. This process can also be completed through regular expression matching. According to the extracted indication information, the second user intent information is determined. The second user intent information refers to the user's demand clarified through the indication information, which is usually in the form of target keywords. These target keywords are the core content of the user question, which is used for precise knowledge retrieval.

[0042] Step 1023, if the identification result of the identifier is that there is no intent instruction identifier, the user question information is input into the pre-trained semantic analysis model to obtain the first user intent information output by the semantic analysis model.

[0043] Specifically, when the identification result of the identifier shows that there is no intent instruction identifier in the user question, the user question information is input into the pre-trained semantic analysis model to obtain the first user intent information output by the model. This can be achieved through natural language processing technology, which is used to understand the semantic content of the user question and infer the user's potential demand. The semantic analysis model is a natural language processing model based on deep learning, which can be fine-tuned using a pre-trained language model to adapt to the semantic understanding needs of a specific field. This model can understand the semantic content of natural language text and extract key information to obtain the first user intent information.

[0044] Step 103, according to the target keywords, determine the target knowledge domain, and in the target knowledge domain, determine at least one target knowledge information corresponding to the target keywords.

[0045] Among them, the target knowledge domain is one of the knowledge domains in the pre-constructed knowledge graph, the pre-constructed knowledge graph includes multiple knowledge domains, and the types of target knowledge information include structured knowledge and unstructured documents.

[0046] Specifically, the pre-constructed knowledge graph includes multiple knowledge domains, and each knowledge domain corresponds to a specific business domain or knowledge category. For example, the knowledge graph can include knowledge domains such as user manual, problem solution, API error, business introduction, and hot issues. According to the semantics and context of the target keyword, and the corresponding relationship between the keyword and the knowledge domain, the target knowledge domain corresponding to the target keyword can be determined. For example, the keyword D0002 is identified as belonging to the API error knowledge domain. After determining the target knowledge domain, the target knowledge information related to the target keyword is retrieved in the knowledge domain. The types of target knowledge information include structured knowledge and unstructured documents. Structured knowledge exists in the form of frequently asked questions and is usually stored in the structured database of the knowledge graph. For example, for API error D0002, the retrieved structured knowledge can be that D0002 represents a network connection timeout, please check the network configuration. Unstructured documents exist in the form of documents and are stored in the file storage unit. For example, user manuals or solution documents related to API errors can be retrieved. By determining the target knowledge domain according to the target keyword, and retrieving structured knowledge and unstructured documents in the target knowledge domain, comprehensive and accurate knowledge services are provided for users.

[0047] Optionally, the pre-constructed knowledge graph is constructed according to steps 31 to 36.

[0048] Step 31, obtaining original structured data and original unstructured documents from multiple knowledge sources.

[0049] The multiple knowledge sources include but are not limited to user manuals, preset problems and problem solutions, business introduction information, and port error reason information.

[0050] Specifically, the knowledge source is the source of the original knowledge information, that is, the knowledge source corresponding to the field executed by the electronic device executing the embodiment. The user manual is knowledge including operation guide, usage instructions, etc. The preset problem and solution include common problems and their solutions. The business introduction information includes business processes, product introductions, etc. The port error reason information includes API error codes and their reasons and solutions. The original structured data and the original unstructured documents are obtained from the above multiple knowledge sources. Structured data can be obtained from tables or databases. Unstructured documents can be obtained from user manuals, solution documents, and business introduction documents.

[0051] Step 32, performing field standardization processing on the original structured data to obtain field data.

[0052] Specifically, the original structured data is subjected to field standardization processing to ensure the consistency of field names and formats, and field data is obtained, which will serve as the basic information of the knowledge graph.

[0053] Step 33: Extract entity relationships from the unstructured document to obtain entity relationship information.

[0054] Specifically, Natural Language Processing (NLP) techniques are used to extract entities and relationships from unstructured documents. For example, entities might be API error codes, business process names, or user manual sections. Relationships might be the relationship between error codes and solutions, or between business processes and user manual sections. The extracted entity-relationship information can be represented in the form of entity-relationship pairs.

[0055] Step 34: Determine the first tag information corresponding to the field data based on the field association information of the field data, and store the original structured data into the preset graph template based on the field association information to obtain the first knowledge graph.

[0056] The first label information is used to indicate the knowledge domain corresponding to the original structured data.

[0057] Specifically, based on the field association information of the field data, the first tag information corresponding to the field data is determined. For example, if the field data contains an error code field, the first tag information is "port error". The field data is stored in a preset graph template to obtain the first knowledge graph. For example, a graph database can be used to store the field data, and each field data node has first tag information indicating its knowledge domain.

[0058] Step 35: Based on the entity relationship information, determine the second tag information corresponding to the unstructured document, and store the unstructured document in the preset decision tree template according to the second tag information to obtain the second knowledge graph.

[0059] The second tag information is used to indicate the knowledge domain corresponding to the unstructured document.

[0060] Specifically, based on entity relationship information, the second tag information corresponding to the unstructured document is determined. For example, if the document contains entities related to cross-border business processes, the second tag information would be "business introduction." The unstructured document is then stored in a pre-defined decision tree template to obtain a second knowledge graph. For example, a decision tree model can be used to store the entity relationship information of the unstructured document, where each node represents an entity or relationship and carries the second tag information.

[0061] Step 36: Based on the first tag information and the second tag information, combine the first knowledge graph and the second knowledge graph into a pre-constructed knowledge graph.

[0062] Specifically, based on the first and second tag information, the first and second knowledge graphs are combined into a pre-constructed knowledge graph. For example, structured data nodes with port error tags are associated with unstructured document nodes with port error tags. The combined knowledge graph contains information from both structured data and unstructured documents, forming a complete knowledge system within which answers to user questions can be retrieved. Data is acquired from multiple knowledge sources, standardized, and entity relationship extracted to determine tag information. Structured data and unstructured documents are stored in graph templates and decision tree templates, respectively. Finally, the knowledge graph is constructed by combining the first and second knowledge graphs. This ensures the completeness and accuracy of the knowledge graph, providing a solid foundation for subsequent knowledge retrieval and intelligent question answering.

[0063] Optionally, the target knowledge domain can be determined based on the target keywords according to steps 1031 to 1032.

[0064] Step 1031: Vectorize the target keywords to obtain target vector features.

[0065] Specifically, keyword vectorization refers to the process of converting target keywords in text form into numerical vectors. These vectors capture the semantic information of the keywords, facilitating efficient semantic matching and similarity calculation. Target vector features are numerical vectors generated through the vectorization process, representing the semantic information of the target keywords. Keyword vectorization methods can include encoding target keywords using a pre-trained language model to generate target vector features, or using a word embedding-based model. This involves training a text corpus, mapping words to a low-dimensional vector space, assigning each word a vector, and then aggregating the vectors of multiple words into a target keyword vector using averaging or weighted averaging. Vectorizing target keywords generates target vector features, thereby enabling efficient semantic matching and knowledge retrieval.

[0066] Step 1032: Determine the target knowledge domain based on the target vector features and the vector index information of each knowledge domain.

[0067] Specifically, the vector index information of each knowledge domain refers to a vector set constructed in advance to represent the core semantics of the knowledge domain. These vectors can be vectorized representations of key documents, question and answer entries, or domain terms in the knowledge domain. The vector features of the target keyword are calculated for similarity with the vector index information of each knowledge domain, and the interface determines the target knowledge domain. The similarity calculation method can be cosine similarity calculation, Euclidean distance calculation, etc. According to the similarity calculation result, all knowledge domains are sorted, and the knowledge domain with the highest similarity is selected as the target knowledge domain. Alternatively, a confidence threshold is set in advance, and the knowledge domain with a similarity higher than the threshold is the target knowledge domain. By vectorizing the target keyword and combining the vector index information of each knowledge domain, the target knowledge domain can be efficiently determined, ensuring the accuracy and reliability of the determination result of the target knowledge domain.

[0068] Step 104, determining a target answer according to at least one target knowledge information, and sending the target answer to the target user device of the target user.

[0069] Specifically, the target knowledge information retrieved from the knowledge graph may include structured knowledge and unstructured documents, which need to be fused to generate the target answer. For structured knowledge, key information can be directly extracted and presented in a concise text form. For example, the question and answer entry D0002: Network connection timeout, please check the network configuration retrieved from the knowledge graph will be directly extracted. For unstructured documents, according to the relevance of the document content, key fragments or summaries are extracted, and a document download link is provided. The processed structured knowledge and unstructured document information are integrated into the target answer. For example, the integrated knowledge information is converted into a fluent and natural language text through natural language generation technology. After obtaining the target answer, the target answer is sent to the target user device of the target user to answer the user's question. The target answer sent is the answer rendered according to the device information of the target user, so that the answer can be displayed on the device.

[0070] Optionally, determining a target answer according to at least one target knowledge information can be achieved by steps 1041 to 1042.

[0071] Step 1041, determining the matching degree of at least one target knowledge information and the target keyword according to the target keyword and the preset matching information of the keyword and the knowledge information.

[0072] Specifically, the preset matching rule of the keyword and the knowledge information refers to a predefined mapping relationship for evaluating the relevance of the target keyword and the knowledge information in the knowledge base. According to the target keyword and the preset matching information of the keyword and the knowledge information, such as the matching value, the matching degree of at least one target knowledge information and the target keyword is determined.

[0073] Step 1042, according to the matching degree of at least one target knowledge information and the target keyword, determine the target answer.

[0074] Specifically, the matching degree refers to the relevance score between the target knowledge information and the target keyword, which is usually obtained through multi-dimensional evaluation such as semantic similarity, keyword matching, field relevance, and historical data statistics. The higher the matching degree, the stronger the relevance of the knowledge information and the user's demand. According to the matching degree, the retrieved target knowledge information is sorted, and the knowledge information with high matching degree is preferentially selected as the target answer. For example, a confidence threshold can be set, and only the knowledge information with a matching degree higher than the threshold will be selected as the target answer. The user's context information, such as historical query records, can also be combined to further optimize the target answer. By evaluating the matching degree of the target knowledge information and the target keyword, the most suitable answer for the user's demand can be selected from multiple knowledge information, ensuring the accuracy and practicality of the target answer.

[0075] Optionally, the user question information includes a document download instruction, and after determining the target answer according to at least one target knowledge information, steps 41 to 44 can be performed.

[0076] Step 41, obtain the user identification information of the target user, and determine the document download authority of the target user according to the user identification information and the preset document download authority information.

[0077] Specifically, when the user question information contains a document download instruction, after determining the target answer, the document download authority of the target user needs to be further verified. The document download instruction refers to the user's explicit expression of downloading the document in the question, which is usually represented by specific keywords or formats, such as @ download user manual. The document download instruction in the user question is identified through regular expression matching or semantic analysis. The user identification information refers to the information that can identify the user's identity, and the acquisition method can be to return a request for obtaining the user's identification information when the user initiates a query request, and then receive the user identification information returned by the user equipment. The preset document download authority information refers to the document download authority corresponding to the user's identity defined in advance. According to the user identification information and the preset document download authority information, the document download authority of the target user is determined.

[0078] Step 42, determine the target document according to the document download instruction, and obtain the document authority information corresponding to the target document.

[0079] Specifically, the target document is determined according to the information in the document download instruction, and the document authority information corresponding to the target document is obtained according to the information of the target document.

[0080] Step 43, according to the document permission information and the document download permission of the target user, determine the document execution strategy of the target document.

[0081] Wherein, the document execution strategy includes sending the target document to the target user equipment, or sending the preset insufficient permission information to the target user equipment.

[0082] Specifically, the document execution strategy refers to the strategy of deciding how to respond to the user's document download request according to the user's download permission and the document's permission information. If the user has the download permission, the target document is sent to the user equipment. If the user does not have the download permission, the user is informed of the insufficient permission information. According to the document permission information and the document download permission of the target user, the permission information of the target document is compared with the download permission of the user to verify whether the user has the right to download the document, thereby determining the document execution strategy of the target document. Sending the target document can be generating a temporary download link with a timestamp and user identification for the target document. The validity period of the link is usually short to ensure security. The download link of the target document is embedded in the target answer and sent to the user. A hash algorithm can be used to generate a temporary token to bind the user equipment and the time window to ensure the security of the link. The preset insufficient permission information is information that can inform the user of the reason why the document cannot be downloaded. Sending the preset insufficient permission information to the target user equipment means embedding the insufficient permission information in the target answer and sending it to the user.

[0083] Step 44, operate the target document according to the document execution strategy.

[0084] Specifically, after determining the document execution strategy, operate the target document according to the document execution strategy. That is, when the document execution strategy includes sending the target document to the target user equipment, send the target document to the target user equipment and convert the document to the document format corresponding to the target user equipment. When the document execution strategy includes sending the preset insufficient permission information to the target user equipment, send the preset insufficient permission information to the target user equipment so that the user can obtain the insufficient permission information. By verifying the document permission information and the download permission of the target user, the document execution strategy can be accurately determined, ensuring the security and compliance of document download.

[0085] The scheme of the present application obtains user question information of a target user; inputs the user question information into a pre-trained semantic analysis model to obtain first user intention information output by the semantic analysis model; wherein the first user intention information includes a target keyword; determines a target knowledge domain according to the target keyword, and determines at least one target knowledge information corresponding to the target keyword in the target knowledge domain; wherein the target knowledge domain is one knowledge domain in a pre-constructed knowledge graph, the pre-constructed knowledge graph includes multiple knowledge domains, and the types of the target knowledge information include structured knowledge and unstructured documents; determines a target answer according to the at least one target knowledge information, and sends the target answer to a target user device of the target user. That is, the scheme of the present application inputs the user question information into the semantic analysis model to obtain the user intention, and determines the knowledge information corresponding to the question according to the user intention, and then determines the answer corresponding to the question according to the knowledge information, that is, the matching degree of the answer and the question is improved by analyzing the semantic information of the user, the situation that the relatively complex semantic information cannot be understood is avoided, and the correctness of the output answer is improved, and the user experience is improved.

[0086] Figure 2 is a training process schematic diagram of the semantic analysis model of the answer determination method provided by the present application, and the embodiment in Figure 1 On the basis of the embodiments and various optional implementation schemes, the steps of training the semantic analysis model are described in detail. As Figure 2 The method can include the following steps:

[0087] Step 201, obtaining training question information and training intention information.

[0088] The training question information and the training intention information are a group of training data in a pre-set training database, and the pre-set training database includes multiple groups of training data, each group of training data including training question information and training intention information.

[0089] Specifically, the pre-set training database stores multiple groups of training data, each group of training data including training question information and training intention information corresponding to the training question information. The training intention information is the correct intention corresponding to the training question information. Therefore, after training the model according to the multiple training question information and the multiple training intention information, the model can obtain the most accurate user intention, thereby improving the accuracy of the user intention obtained by the model.

[0090] Step 202, inputting the training question information into the training model to obtain training user intention information output by the training model.

[0091] Specifically, the model to be trained can be any open-source large language model. By inputting the training question information into the model to be trained, the user intent information output by the model can be obtained.

[0092] Step 203: Based on the user intent information and training intent information, adjust the target model parameters of the model to be trained to obtain the model to be trained with adjusted parameters.

[0093] Specifically, the training intent information is the correct intent corresponding to the training question information. Therefore, based on the user intent information and the training intent information, the target model parameters of the training model are adjusted, such as adjusting the model's learning rate, to obtain the parameter-adjusted training model.

[0094] When the model to be trained is any open-source large language model, the target model parameters include the learning rate and / or the length of the soft cue embeddings. The target model parameters are determined to be the learning rate and / or the length of the soft cue embeddings because complex tasks require longer soft cue embeddings and / or smaller learning rates. A smaller learning rate can prevent the model's existing language understanding capabilities from being compromised by a larger learning rate. Choosing a smaller learning rate and a longer soft cue embedding can ensure the stability of the model's training process while avoiding overfitting.

[0095] Step 204: Use the parameter-adjusted training model as the new training model, and use the training question information from a preset training database that is not input into the training model as the new training question information. Return to step 202 until the preset iteration end condition is reached, and use the parameter-adjusted training model as the semantic analysis model.

[0096] Specifically, the parameter-adjusted model to be trained is used as the new model to be trained. A training question from a pre-defined training database that has not yet been input into the model is used as the new training question. The process then returns to the step of inputting the training question into the model to obtain the user intent information output by the model, until a pre-defined iteration termination condition is reached. For example, the pre-defined iteration termination condition could be stopping training when a certain cutoff condition is reached, i.e., stopping the return step. The cutoff condition could be reaching a pre-defined number of training rounds, or the training loss falling below a certain value, etc. The resulting parameter-adjusted model to be trained at this point is the semantic analysis model.

[0097] According to the scheme, the model parameters of the to-be-trained model are adjusted according to the to-be-trained user intention information and the training intention information, so that the output accuracy of the model is improved. After the to-be-trained model is trained according to the plurality of to-be-trained question information and the plurality of to-be-trained intention information, the most accurate user intention of the model can be obtained, thereby improving the accuracy of the user intention obtained by the model.

[0098] Figure 3 is a structural schematic diagram of an answer determination device provided by the present application, which is suitable for executing the answer determination method provided by the present application. As shown in Figure 3 , the device can specifically include:

[0099] The acquisition module 301 is configured to acquire user question information of a target user.

[0100] The input module 302 is configured to input the user question information into a pre-trained semantic analysis model to obtain first user intention information output by the semantic analysis model; wherein the first user intention information includes a target keyword.

[0101] The determination module 303 is configured to determine a target knowledge domain according to the target keyword, and determine at least one target knowledge information corresponding to the target keyword in the target knowledge domain; wherein the target knowledge domain is one knowledge domain in a pre-constructed knowledge graph, the pre-constructed knowledge graph includes a plurality of knowledge domains, and the type of the target knowledge information includes structured knowledge and unstructured document.

[0102] The sending module 304 is configured to determine a target answer according to the at least one target knowledge information, and send the target answer to a target user device of the target user.

[0103] In an embodiment, the input module 302 is specifically configured to: identify a question mark of the user question information to obtain an identification result; wherein the identification result includes an existing intention instruction mark and a non-existing intention instruction mark; if the identification result is the existing intention instruction mark, determine second user intention information according to the indication information of the user question information; wherein the second user intention information includes a target keyword; if the identification result is the non-existing intention instruction mark, input the user question information into a pre-trained semantic analysis model to obtain first user intention information output by the semantic analysis model.

[0104] In an embodiment, the sending module 304 is specifically configured to determine the target knowledge domain according to the target keyword, and determine the target answer according to the at least one target knowledge information and the matching degree of the target keyword.

[0105] In an embodiment, the determining module 303 is specifically configured to vectorize the target keyword to obtain a target vector feature, and determine the target knowledge domain according to the target vector feature and vector index information of each knowledge domain.

[0106] In an embodiment, the user question information includes a document download instruction, and the device further includes a permission determining module configured to, after the sending module 304 determines the target answer according to the at least one target knowledge information, acquire user identification information of the target user, and determine a document download permission of the target user according to the user identification information and preset document download permission information; determine a target document according to the document download instruction, and acquire document permission information corresponding to the target document; and determine a document execution strategy of the target document according to the document permission information and the document download permission of the target user; wherein the document execution strategy includes sending the target document to the target user device, or sending preset permission insufficient information to the target user device; and operating the target document according to the document execution strategy.

[0107] In an embodiment, the device further comprises a construction module configured to obtain original structured data and original unstructured documents from a plurality of knowledge sources, wherein the plurality of knowledge sources include, but are not limited to, user manuals, preset questions and solutions to the questions, business introduction information, and port error cause information; perform field standardization processing on the original structured data to obtain field data; perform entity relationship extraction on the unstructured documents to obtain entity relationship information; determine first label information corresponding to the field data according to field association information of the field data, and store the original structured data into a preset graph template according to the field association information to obtain a first knowledge graph; wherein the first label information is used to indicate a knowledge domain corresponding to the original structured data; determine second label information corresponding to the unstructured documents according to the entity relationship information, and store the unstructured documents into a preset decision tree template according to the second label information to obtain a second knowledge graph; wherein the second label information is used to indicate a knowledge domain corresponding to the unstructured documents; and combine the first knowledge graph and the second knowledge graph into the pre-constructed knowledge graph according to the first label information and the second label information.

[0108] In an embodiment, the device further comprises a semantic analysis model training module configured to obtain training question information and training intent information; wherein the training question information and the training intent information are a group of training data in a preset training database, the preset training database includes a plurality of groups of training data, and each group of the training data includes training question information and training intent information; input the training question information into a training model to obtain training user intent information output by the training model; adjust target model parameters of the training model according to the training user intent information and the training intent information to obtain a training model with adjusted parameters; use the training model with the adjusted parameters as a new training model, use training question information in the preset training database that has not been input into the training model as new training question information, return to execute the step of “input the training question information into a training model to obtain training user intent information output by the training model” until a preset iteration end condition is reached, and use the training model with the adjusted parameters as the semantic analysis model.

[0109] The apparatus of this application acquires user question information from a target user; inputs the user question information into a pre-trained semantic analysis model to obtain first user intent information output by the semantic analysis model; wherein the first user intent information includes target keywords; determines a target knowledge domain based on the target keywords, and determines at least one type of target knowledge information corresponding to the target keywords within the target knowledge domain; wherein the target knowledge domain is a knowledge domain in a pre-constructed knowledge graph, the pre-constructed knowledge graph includes multiple knowledge domains, and the types of target knowledge information include structured knowledge and unstructured documents; determines a target answer based on at least one type of target knowledge information, and sends the target answer to the target user's target user device. In other words, the scheme of this application inputs user question information into a semantic analysis model to obtain user intent, determines the knowledge information corresponding to the question based on the user intent, and then determines the answer corresponding to the question based on the knowledge information. This improves the matching degree between the answer and the question by analyzing the user's semantic information, avoids situations where complex semantic information cannot be understood, thereby improving the accuracy of the output answer and enhancing the user experience.

[0110] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the answer determination method provided in any of the above embodiments.

[0111] This application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the answer determination method provided in any of the above embodiments.

[0112] The following is for reference. Figure 4 It shows a schematic diagram of the structure of an electronic device 400 suitable for implementing the present application. Figure 4 The electronic device shown is merely an example and should not impose any limitations on the functionality and scope of this application.

[0113] like Figure 4 As shown, the electronic device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage section 408 into a random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the electronic device 400. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0114] The following components are connected to the I / O interface 405: an input part 406 including a keyboard, a mouse, etc.; an output part 407 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 408 including a hard disk, etc.; and a communication part 409 including a network interface card such as a LAN card, a modem, etc. The communication part 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as necessary. A removable media 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 410 as necessary, so that a computer program read out therefrom is installed in the storage part 408 as necessary.

[0115] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication part 409, and / or installed from the removable media 411. When the computer program is executed by the central processing unit (CPU) 401, the above-described functions defined in the system of the present disclosure are executed.

[0116] It should be noted that computer-readable media in this disclosure can be computer-readable storage media, computer-readable signal media or any combination thereof. Computer-readable storage media can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, computer-readable storage media can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus or device. In this disclosure, a computer-readable signal medium can include a computer-readable storage medium in baseband or propagated as a carrier wave in a propagated data signal, which can contain or store a program for use by or in connection with an instruction execution system, apparatus or device. The propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. Computer-readable signal media can be any computer-readable medium that is not a computer-readable storage medium, and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0117] The flow diagrams and block diagrams in the drawings are illustrations of possible architectures, functions, and operations of systems, methods, and computer program products in accordance with various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0118] The modules and / or units described in the present application can be implemented by software or by hardware. The described modules and / or units can also be arranged in a processor, for example, can be described as: a processor includes an acquisition module, an input module, a determination module and a sending module. In some cases, the names of these modules do not constitute a limitation on the modules themselves.

[0119] As another aspect, the present application also provides a computer readable medium, which can be included in the device described in the above embodiments; or can exist independently without being assembled into the device. The above computer readable medium carries one or more programs, when the one or more programs are executed by the device, the device performs the following operations:

[0120] acquiring user question information of a target user; inputting the user question information into a pre-trained semantic analysis model to obtain first user intention information output by the semantic analysis model; wherein the first user intention information includes a target keyword; determining a target knowledge domain according to the target keyword, and determining at least one kind of target knowledge information corresponding to the target keyword in the target knowledge domain; wherein the target knowledge domain is one knowledge domain in a pre-constructed knowledge graph, the pre-constructed knowledge graph includes multiple knowledge domains, and the types of the target knowledge information include structured knowledge and unstructured documents; determining a target answer according to the at least one kind of target knowledge information, and sending the target answer to a target user device of the target user.

[0121] According to the technical scheme of the present application, the user question information of the target user is acquired; the user question information is input into the pre-trained semantic analysis model to obtain the first user intention information output by the semantic analysis model; wherein the first user intention information includes a target keyword; a target knowledge domain is determined according to the target keyword, and at least one kind of target knowledge information corresponding to the target keyword in the target knowledge domain is determined; wherein the target knowledge domain is one knowledge domain in a pre-constructed knowledge graph, the pre-constructed knowledge graph includes multiple knowledge domains, and the types of the target knowledge information include structured knowledge and unstructured documents; a target answer is determined according to the at least one kind of target knowledge information, and the target answer is sent to a target user device of the target user. That is, the scheme of the present application inputs the user question information into the semantic analysis model to obtain the user intention, and determines the knowledge information corresponding to the question according to the user intention, and then determines the answer corresponding to the question according to the knowledge information, that is, the matching degree of the answer and the question is improved by analyzing the semantic information of the user, avoiding the situation that the relatively complex semantic information cannot be understood, thereby improving the correctness of the output answer and improving the user experience.

[0122] The embodiment of the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the answer determination method provided by any of the embodiments of the present application.

[0123] In implementing the computer program product, computer program code for performing the operations of the present application can be written in one or more programming languages, or combinations of languages, including object oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as the "C" programming language, or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network or a wide area network, or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0124] It should be understood that the steps shown in the above-described flow can be reordered, added, or deleted. For example, the steps described in the present application can be executed in parallel, in sequence, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, and the present application is not limited herein.

[0125] The specific embodiments described above are not meant to limit the scope of the present application. One skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions of the embodiments can occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application should be included in the scope of the present application.

Claims

1. An answer determination method, characterized by, The method comprises: obtaining user question information of a target user; inputting the user question information into a pre-trained semantic analysis model to obtain first user intent information output by the semantic analysis model; wherein the first user intent information comprises a target keyword; determining a target knowledge domain according to the target keyword, and determining at least one target knowledge information corresponding to the target keyword in the target knowledge domain; wherein the target knowledge domain is one knowledge domain in a pre-constructed knowledge graph, the pre-constructed knowledge graph comprises a plurality of knowledge domains, and the type of the target knowledge information comprises structured knowledge and unstructured documents; determining a target answer according to the at least one target knowledge information, and sending the target answer to a target user device of the target user.

2. The method of claim 1, wherein, The inputting of the user question information into the pre-trained semantic analysis model to obtain the first user intent information output by the semantic analysis model comprises: identifying a question mark of the user question information to obtain an identification result; wherein the identification result comprises an existing intent instruction mark and a non-existing intent instruction mark; if the identification result is the existing intent instruction mark, determining second user intent information according to indication information of the user question information; wherein the second user intent information comprises a target keyword; if the identification result is the non-existing intent instruction mark, inputting the user question information into the pre-trained semantic analysis model to obtain the first user intent information output by the semantic analysis model.

3. The method of claim 1, wherein, The determining of the target answer according to the at least one target knowledge information comprises: determining a matching degree of the at least one target knowledge information and the target keyword according to the target keyword and preset matching information of keywords and knowledge information; determining a target answer according to the matching degree of the at least one target knowledge information and the target keyword.

4. The method of claim 2, wherein, The determining of the target knowledge domain according to the target keyword comprises: vectorizing the target keyword to obtain a target vector feature; determining the target knowledge domain according to the target vector feature and vector index information of each knowledge domain.

5. The method of claim 2, wherein, The user question information comprises a document download instruction, and after the determining of the target answer according to the at least one target knowledge information, the method further comprises: obtaining user identification information of the target user, and determining a document download authority of the target user according to the user identification information and preset document download authority information; determining a target document according to the document download instruction, and obtaining document authority information corresponding to the target document; determining a document execution strategy of the target document according to the document authority information and the document download authority of the target user; wherein the document execution strategy comprises sending the target document to the target user device, or sending preset insufficient authority information to the target user device; operating the target document according to the document execution strategy.

6. The method of claim 1, wherein, The pre-constructed knowledge graph is constructed according to the following steps: The system acquires raw structured data and raw unstructured documents from multiple knowledge sources, including but not limited to user manuals, pre-set questions and solutions, business introduction information, and port error reason information. The original structured data is subjected to field standardization processing to obtain field data; Entity relation extraction is performed on the unstructured document to obtain entity relation information; Based on the field association information of the field data, the first tag information corresponding to the field data is determined, and the original structured data is stored in a preset graph template based on the field association information to obtain a first knowledge graph; wherein, the first tag information is used to indicate the knowledge domain corresponding to the original structured data; Based on the entity relationship information, the second tag information corresponding to the unstructured document is determined, and the unstructured document is stored in a preset decision tree template according to the second tag information to obtain a second knowledge graph; wherein, the second tag information is used to indicate the knowledge domain corresponding to the unstructured document; Based on the first tag information and the second tag information, the first knowledge graph and the second knowledge graph are combined into the pre-constructed knowledge graph.

7. The method of claim 1, wherein, The semantic analysis model was trained according to the following steps: Acquire training problem information and training intention information; wherein, the training problem information and training intention information are a set of training data in a preset training database, the preset training database includes multiple sets of data to be trained, and each set of data to be trained includes training problem information and training intention information; The training problem information is input into the model to be trained to obtain the user intent information to be trained output by the model to be trained. Based on the user intent information to be trained and the training intent information, the target model parameters of the model to be trained are adjusted to obtain the model to be trained after parameter adjustment. The model to be trained after parameter adjustment is used as the new model to be trained. A training question information from the preset training database that has not been input into the model to be trained is used as the new training question information. The process of "inputting the training question information into the model to be trained to obtain the training user intent information output by the model to be trained" is repeated until the preset iteration end condition is met. The model to be trained after parameter adjustment is then used as the semantic analysis model.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the answer determination method as described in any one of claims 1 to 7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the answer determination method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the answer determination method as described in any one of claims 1 to 7.

Images