Answer determination method and device, computer equipment and readable storage medium

Abstract

The invention relates to an answer determination method and device, computer equipment, a computer readable storage medium and a computer program product, relates to the technical field of artificial intelligence, and can be applied to the field of financial science and technology or other related fields. The method comprises the following steps: acquiring a question associated with a financial internal application; identifying entities and entity relationships from the question through a trained language processing model; the trained language processing model is obtained by iteratively training a pre-trained language processing model through a sample finance internal application knowledge graph; determining a target financial internal application entity satisfying the entity and the entity relationship from a pre-constructed financial internal application knowledge graph; and according to the target financial internal application entity, generating an answer to the question. By adopting the method, the answer determination efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an answer determination method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of financial business and computer technology, various financial internal applications emerge in an endless stream; when technical personnel need to understand the relevant content of financial internal applications, such as business background, system architecture, etc., they will raise questions related to financial internal applications.

[0003] In traditional technology, when faced with problems related to internal financial applications, the answer to the problem is generally determined by manually reviewing various application documents of the internal financial applications; however, each time the answer is determined, a large number of application documents of the internal financial applications need to be manually reviewed, and the process is relatively cumbersome, resulting in low efficiency in answer determination. Summary of the invention

[0004] Based on this, it is necessary to provide an answer determination method, apparatus, computer device, computer-readable storage medium and computer program product that can improve the efficiency of answer determination in response to the above technical problems.

[0005] In a first aspect, the present application provides an answer determination method, comprising:

[0006] Get issues related to internal financial applications;

[0007] Using the trained language processing model, entities and entity relationships are identified from the question; the trained language processing model is obtained by iteratively training a pre-trained language processing model using a sample financial internal application knowledge graph;

[0008] Determine, from a pre-built financial internal application knowledge graph, a target financial internal application entity that satisfies the entity and the entity relationship;

[0009] An answer to the question is generated according to the target financial internal application entity.

[0010] In one embodiment, determining the target financial internal application entity satisfying the entity and the entity relationship from the pre-built financial internal application knowledge graph includes:

[0011] Determine candidate financial internal application entities associated with the entity from a pre-constructed financial internal application knowledge graph;

[0012] From the candidate internal financial application entities, a target internal financial application entity whose association relationship with the entity satisfies the entity relationship is determined.

[0013] In one embodiment, determining the target financial internal application entity satisfying the entity and the entity relationship from the pre-built financial internal application knowledge graph includes:

[0014] Generate a knowledge graph query language according to the entity, the entity relationship and the question type of the question;

[0015] According to the knowledge graph query language, a pre-constructed financial internal application knowledge graph is queried to obtain the financial internal application entity corresponding to the knowledge graph query language as the target financial internal application entity.

[0016] In one embodiment, the pre-built financial internal application knowledge graph is obtained by:

[0017] Obtain financial internal application documents to be processed;

[0018] Using the trained language processing model, identifying new entities and new entity relationships from the internal financial application documents to be processed;

[0019] The new entity and the new entity relationship are added to the initial financial internal application knowledge graph to obtain the pre-built financial internal application knowledge graph.

[0020] In one embodiment, the trained language processing model is trained in the following manner:

[0021] Obtaining a sample sentence and the sample financial internal application knowledge graph;

[0022] By using the sample financial internal application knowledge graph, the sample sentence is converted into a corresponding tree structure;

[0023] Converting the tree structure into a sample sequence carrying a mask mark, and performing feature extraction processing on the sample sequence to obtain a feature vector of the sample sequence;

[0024] Inputting the feature vector of the sample sequence and the target matrix of the sample sequence into the pre-trained language processing model to obtain the predicted words corresponding to the mask mark; the target matrix is ​​used to indicate whether any two words in the sample sequence belong to the same triple;

[0025] According to the difference between the predicted words and the actual words corresponding to the mask mark, the pre-trained language processing model is iteratively trained to obtain the trained language processing model.

[0026] In one embodiment, the sample financial internal application knowledge graph is obtained by:

[0027] Get sample financial internal application documents;

[0028] identifying, by means of the pre-trained language processing model, a first sample entity and a first sample entity relationship from the sample financial internal application document;

[0029] The first sample entity and the first sample entity relationship are added to the initial financial internal application knowledge graph to obtain the sample financial internal application knowledge graph.

[0030] In one embodiment, converting the sample sentence into a corresponding tree structure through the sample financial internal application knowledge graph includes:

[0031] Identifying a second sample entity in the sample sentence;

[0032] Identifying, from the sample financial internal application knowledge graph, an associated entity of the second sample entity and a second sample entity relationship between the second sample entity and the associated entity;

[0033] A tree structure corresponding to the sample sentence is constructed according to the second sample entity in the sample sentence, the associated entity of the second sample entity, and the second sample entity relationship between the second sample entity and the associated entity.

[0034] In a second aspect, the present application further provides an answer determination device, comprising:

[0035] A problem acquisition module is used to acquire problems associated with internal financial applications;

[0036] An entity recognition module is used to identify entities and entity relationships from the question through a trained language processing model; the trained language processing model is obtained by iteratively training a pre-trained language processing model through a sample financial internal application knowledge graph;

[0037] An entity determination module, used to determine a target financial internal application entity that satisfies the entity and the entity relationship from a pre-built financial internal application knowledge graph;

[0038] The answer generation module is used to generate an answer to the question according to the target financial internal application entity.

[0039] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0040] Get issues related to internal financial applications;

[0041] Using the trained language processing model, entities and entity relationships are identified from the question; the trained language processing model is obtained by iteratively training a pre-trained language processing model using a sample financial internal application knowledge graph;

[0042] Determine, from a pre-built financial internal application knowledge graph, a target financial internal application entity that satisfies the entity and the entity relationship;

[0043] An answer to the question is generated according to the target financial internal application entity.

[0044] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0045] Get issues related to internal financial applications;

[0046] Using the trained language processing model, entities and entity relationships are identified from the question; the trained language processing model is obtained by iteratively training a pre-trained language processing model using a sample financial internal application knowledge graph;

[0047] Determine, from a pre-built financial internal application knowledge graph, a target financial internal application entity that satisfies the entity and the entity relationship;

[0048] An answer to the question is generated according to the target financial internal application entity.

[0049] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0050] Get issues related to internal financial applications;

[0051] Using the trained language processing model, entities and entity relationships are identified from the question; the trained language processing model is obtained by iteratively training a pre-trained language processing model using a sample financial internal application knowledge graph;

[0052] Determine, from a pre-built financial internal application knowledge graph, a target financial internal application entity that satisfies the entity and the entity relationship;

[0053] An answer to the question is generated according to the target financial internal application entity.

[0054] The above-mentioned answer determination method, device, computer equipment, computer-readable storage medium and computer program product obtain questions related to financial internal applications, and identify entities and entity relationships from the questions through a trained language processing model; the trained language processing model is obtained by iteratively training the pre-trained language processing model through a sample financial internal application knowledge graph; then, the target financial internal application entity that satisfies the entity and entity relationship is determined from the pre-constructed financial internal application knowledge graph, and finally, the answer to the question is generated based on the target financial internal application entity. In this way, when determining the answer to the question related to the financial internal application, the answer to the question is automatically generated directly based on the trained language processing model and the pre-constructed financial internal application knowledge graph, without the need to manually consult a large number of application documents of the financial internal application, thereby simplifying the answer determination process, thereby improving the answer determination efficiency, and avoiding the defect of low answer determination efficiency caused by manually consulting a large number of application documents of the financial internal application. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A schematic diagram of a flow chart of a method for determining an answer in one embodiment;

[0057] Figure 2 A schematic diagram of a sample financial internal application knowledge graph in an embodiment;

[0058] Figure 3 is a flowchart of the training steps of a language processing model in one embodiment;

[0059] Figure 4 A schematic flow chart of a method for determining an answer in another embodiment;

[0060] Figure 5 It is a flowchart of a method for constructing a bank internal application question-and-answer system based on a knowledge graph in one embodiment;

[0061] Figure 6 It is a structural block diagram of an answer determination device in one embodiment;

[0062] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that the answer determination method, apparatus, computer equipment, computer-readable storage medium and computer program product provided in this application can be used in the field of financial technology. For example, answers to questions can be automatically generated based on questions related to internal financial applications, thereby improving the efficiency of answer determination. They can also be used in other related fields, such as in the field of artificial intelligence technology, using knowledge graphs and language processing models to intelligently generate corresponding answers.

[0065] It should be noted that the information collected is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0066] In an exemplary embodiment, Figure 1 As shown, a method for determining an answer is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc.; the server refers to a bank server, which can be implemented with an independent server or a server cluster composed of multiple servers. In this embodiment, the following steps S101 to S104 are included. Among them:

[0067] Step S101, obtaining issues associated with financial internal applications.

[0068] Among them, financial internal applications refer to internal bank applications, such as internal bank application systems.

[0069] Among them, issues related to financial internal applications refer to issues related to financial internal applications, specifically issues involving relevant content of financial internal applications, such as business background, business architecture, data architecture, technical architecture, security architecture, etc. of financial internal applications. In actual scenarios, issues related to financial internal applications may include "which system does the credit card need to be registered in", "which system does the acquiring system need to notify", "what management information does the credit card system need to obtain", etc.

[0070] Exemplarily, the terminal displays a question input interface of the financial internal application question and answer system, and the user can enter questions on the question input interface, such as questions related to the financial internal application; the terminal responds to the input operation on the question input interface of the financial internal application question and answer system, and obtains the input question related to the financial internal application.

[0071] Step S102, identifying entities and entity relationships from the question through the trained language processing model; the trained language processing model is obtained by iteratively training the pre-trained language processing model through the sample financial internal application knowledge graph.

[0072] Among them, the trained language processing model refers to a network model that can identify entities and entity relationships in texts (such as questions, documents, etc.), such as the BERT (Bidirectional Encoder Representations from Transformer, bidirectional encoding based on Transformer) model, deep learning model, machine learning model, etc.

[0073] It should be noted that the trained language processing model is obtained by iteratively training the pre-trained language processing model based on the sample financial internal application knowledge graph, so the trained language processing model is more suitable for entity recognition in the financial field.

[0074] Entities refer to names of people, places, transactions, organizations, etc. Entity relationships refer to the relationship between two entities, such as the acquiring system notifying the credit card system, the credit card system registering to the accounting system, and the credit card system obtaining customer management information.

[0075] The sample financial internal application knowledge graph refers to the financial internal application knowledge graph used in model training, specifically the knowledge graph containing financial internal application entities and the entity relationships between financial internal application entities. Figure 2 Financial internal application entities refer to related entities in financial internal applications, such as acquiring systems, credit card systems, accounting systems, debit card applications, debit card accounts, debit card agreements, etc.

[0076] Among them, the pre-trained language processing model refers to the pre-trained BERT model.

[0077] Exemplarily, the terminal performs feature extraction processing on the question to obtain a feature vector of the question, and inputs the feature vector of the question into a trained language processing model to perform entity recognition processing to obtain entities and entity relationships in the question.

[0078] For example, for the question "In which system does a credit card need to be registered", the entities recognized by the trained language processing model are "credit card" and "system", and the entity relationship is "registration".

[0079] Step S103, determining the target financial internal application entity that satisfies the entity and entity relationship from the pre-constructed financial internal application knowledge graph.

[0080] The pre-built financial internal application knowledge graph refers to the financial internal application knowledge graph used when the model is applied, specifically the knowledge graph containing financial internal application entities and the entity relationships between financial internal application entities. Figure 2 similar.

[0081] It should be noted that the sample financial internal application knowledge graph and the pre-built financial internal application knowledge graph are two separate financial internal application knowledge graphs.

[0082] Among them, the target financial internal application entity that satisfies the entity and entity relationship refers to the financial internal application entity whose relationship with the entity satisfies the entity relationship. In addition, the pre-built financial internal application knowledge graph contains multiple financial internal application entities and entity relationships between financial internal application entities. Therefore, the target financial internal application entity that satisfies the entity and entity relationship can be found from the pre-built financial internal application knowledge graph.

[0083] Exemplarily, the terminal queries a pre-built financial internal application knowledge graph based on the entities and entity relationships in the question, and determines from the pre-built financial internal application knowledge graph a financial internal application entity whose relationship with the entity satisfies the entity relationship as the target financial internal application entity.

[0084] For example, for the question "In which system does the credit card need to be registered", the entities are "credit card" and "system", and the entity relationship is "registration". In the pre-built financial internal application knowledge graph, among the system entities connected to "credit card", the system entity with the entity relationship "registration" with "credit card" is "accounting system", indicating that the target financial internal application entity is "accounting system".

[0085] Step S104: Generate answers to the questions based on the target financial internal application entity.

[0086] The answer to the question refers to the answer statement containing the target financial internal application entity.

[0087] Exemplarily, the terminal generates answers to the questions according to the questions associated with the internal financial application and the target internal financial application entity.

[0088] For example, for the question "In which system should credit cards be registered?", and the target financial internal application entity is "accounting system", then the answer to the question is "Credit cards need to be registered in the accounting system."

[0089] In the above-mentioned answer determination method, by obtaining questions related to financial internal applications, and using the trained language processing model, entities and entity relationships are identified from the questions; the trained language processing model is obtained by iteratively training the pre-trained language processing model through the sample financial internal application knowledge graph; then, the target financial internal application entity that satisfies the entity and entity relationship is determined from the pre-constructed financial internal application knowledge graph, and finally, the answer to the question is generated based on the target financial internal application entity. In this way, when determining the answer to the question related to the financial internal application, the answer to the question is automatically generated directly based on the trained language processing model and the pre-constructed financial internal application knowledge graph, without the need to manually consult a large number of application documents of the financial internal application, thereby simplifying the answer determination process, thereby improving the answer determination efficiency, and avoiding the defect of low answer determination efficiency caused by manually consulting a large number of application documents of the financial internal application.

[0090] In an exemplary embodiment, the above step S103, determining the target financial internal application entity that satisfies the entity and entity relationship from the pre-constructed financial internal application knowledge graph, specifically includes the following contents: determining the candidate financial internal application entity associated with the entity from the pre-constructed financial internal application knowledge graph; determining the target financial internal application entity whose association relationship with the entity satisfies the entity relationship from the candidate financial internal application entities.

[0091] Among them, the candidate financial internal application entity associated with the entity refers to the financial internal application entity that has an association relationship (i.e., a connection relationship) with the entity in the pre-constructed financial internal application knowledge graph.

[0092] The target financial internal application entity refers to a financial internal application entity among the candidate financial internal application entities whose association relationship with the entity is an entity relationship.

[0093] Exemplarily, the terminal determines, from a pre-constructed financial internal application knowledge graph, financial internal application entities that have an association relationship with the entity as candidate financial internal application entities, and then determines, from the candidate financial internal application entities, financial internal application entities that have an entity relationship with the entity as target financial internal application entities.

[0094] For example, for the question "In which system should a credit card be registered", the entities are "credit card" and "system", and the entity relationship is "registration". In the pre-built financial internal application knowledge graph, the terminal finds the associated candidate system entity based on "credit card", and finds the target system entity with the association relationship "registration" with "credit card" from the candidate system entity, namely "accounting system".

[0095] In this embodiment, candidate financial internal application entities associated with the entity are first determined from a pre-constructed financial internal application knowledge graph, and then a target financial internal application entity whose association relationship with the entity satisfies the entity relationship is determined from the candidate financial internal application entities. This is beneficial to improving the accuracy of determining the target financial internal application entity, and is also beneficial for automatically generating answers to questions based on the target financial internal application entity, thereby saving answer determination time and improving answer determination efficiency.

[0096] In an exemplary embodiment, the above step S103, determining the target financial internal application entity that satisfies the entity and entity relationship from the pre-constructed financial internal application knowledge graph, specifically includes the following contents: generating a knowledge graph query language according to the entity, entity relationship and question type of the question; querying the pre-constructed financial internal application knowledge graph according to the knowledge graph query language, and obtaining the financial internal application entity corresponding to the knowledge graph query language as the target financial internal application entity.

[0097] The question type refers to the type of questions related to internal financial applications, such as concept explanation, relationship query, suggestion recommendation, comparison, etc. For example, for the question "In which system should a credit card be registered", the question type is relationship query.

[0098] The knowledge graph query language refers to a language specifically used to query the knowledge graph, which is generated by entities, entity relationships, and question types. It should be noted that the pre-built financial internal application knowledge graph can be regarded as a database.

[0099] Exemplarily, the terminal generates a knowledge graph query language that matches the question based on the entities, entity relationships, and question types, and in combination with a knowledge graph query language template, and queries a pre-built financial internal application knowledge graph based on the knowledge graph query language to obtain the financial internal application entity corresponding to the knowledge graph query language as the target financial internal application entity.

[0100] For example, the terminal queries the pre-built financial internal application knowledge graph based on the knowledge graph query language corresponding to the question "In which system should the credit card be registered", and obtains the financial internal application entity corresponding to the knowledge graph query language as "accounting system", indicating that the target financial internal application entity is "accounting system".

[0101] In addition, the terminal can also use the trained language processing model to semantically understand questions related to internal financial applications and obtain the corresponding intent; find entities that match the intent from the pre-built knowledge graph of internal financial applications, and then find the target entity relationship information that matches the intent from the entity relationship information corresponding to the entity; finally, based on the found target entity relationship information, generate an answer statement to the question to get the answer.

[0102] In this embodiment, a knowledge graph query language is first generated according to entities, entity relationships and question types, and then a pre-constructed financial internal application knowledge graph is queried based on the knowledge graph query language to obtain the financial internal application entity corresponding to the knowledge graph query language as the target financial internal application entity. In this way, the pre-constructed financial internal application knowledge graph is automatically queried using the knowledge graph query language without the need for manual query, which is conducive to improving the efficiency of determining the target financial internal application entity and further improving the efficiency of answer determination.

[0103] In an exemplary embodiment, the answer determination method provided in the present application also includes the step of constructing a financial internal application knowledge graph, which specifically includes the following contents: obtaining the financial internal application document to be processed; identifying new entities and new entity relationships from the financial internal application document to be processed through the trained language processing model; adding the new entities and new entity relationships to the initial financial internal application knowledge graph to obtain a pre-constructed financial internal application knowledge graph.

[0104] Among them, the financial internal application documents to be processed refer to application documents related to financial internal applications, specifically financial internal application documents other than sample financial internal application documents, such as requirement documents, design plans, test plans, database table structures, etc.

[0105] Among them, the initial financial internal application knowledge graph refers to the existing financial internal application knowledge graph, which is also constructed based on entities and entity relationships.

[0106] Among them, the pre-constructed financial internal application knowledge graph refers to the financial internal application knowledge graph obtained by adding new entities and new entity relationships identified from the financial internal application documents to be processed on the basis of the initial financial internal application knowledge graph.

[0107] Exemplarily, the terminal obtains the financial internal application document to be processed from the database, and performs feature extraction processing on the financial internal application document to be processed to obtain a feature vector of the financial internal application document to be processed, and then inputs the feature vector of the financial internal application document to be processed into the trained language processing model for entity recognition processing to obtain entities and entity relationships in the financial internal application document to be processed as new entities and new entity relationships; finally, the new entities and new entity relationships are added to the initial financial internal application knowledge graph to obtain a processed financial internal application knowledge graph as a pre-built financial internal application knowledge graph.

[0108] For example, new entities are inserted based on the relationship between the new entities and existing entities in the initial financial internal application knowledge graph. For example, the initial financial internal application knowledge graph already has the entity "credit card application". When parsing the text related to the accounting system, it is parsed as: "credit card application", "registration", "accounting system", and the connection between the credit card application and the accounting system will be established in the initial financial internal application knowledge graph, thereby obtaining an updated financial internal application knowledge graph as the pre-built financial internal application knowledge graph.

[0109] In this embodiment, the financial internal application document to be processed is first obtained, and then the new entities and new entity relationships are identified from the financial internal application document to be processed through the trained language processing model, and finally the new entities and new entity relationships are added to the initial financial internal application knowledge graph to obtain a pre-constructed financial internal application knowledge graph; in this way, the initial financial internal application knowledge graph is updated using the new entities and new entity relationships in the financial internal application document to be processed, which can improve the richness of the entities and entity relationships in the financial internal application knowledge graph, and ensure that the corresponding answers can be accurately obtained by subsequently querying the financial internal application knowledge graph.

[0110] In an exemplary embodiment, Figure 3 As shown, the answer determination method provided by the present application also includes a language processing model training step, specifically including the following steps S301 to S305. Among them:

[0111] Step S301, obtaining sample sentences and sample financial internal application knowledge graphs.

[0112] Step S302: Convert the sample sentence into a corresponding tree structure through the sample financial internal application knowledge graph.

[0113] Step S303: convert the tree structure into a sample sequence carrying a mask mark, and perform feature extraction processing on the sample sequence to obtain a feature vector of the sample sequence.

[0114] Step S304, input the feature vector of the sample sequence and the target matrix of the sample sequence into the pre-trained language processing model to obtain the predicted words corresponding to the mask mark; the target matrix is ​​used to represent whether any two words in the sample sequence belong to the same triple.

[0115] Step S305, iteratively training the pre-trained language processing model according to the difference between the predicted word and the actual word corresponding to the mask mark to obtain a trained language processing model.

[0116] The sample sentence refers to the sentence involved in the training, which includes multiple entities. For example, the sample sentence is represented as (h1, h2, h3), where h1, h2, and h3 are all entities in the sample sentence.

[0117] The tree structure corresponding to the sample sentence refers to the structure composed of the entity in the sample sentence and the triple corresponding to the entity in the sample financial internal application knowledge graph. For example, h1 has a triple of one relationship record (h1, r11, t11) in the sample financial internal application knowledge graph, and h3 has two triples of two relationship records in the sample financial internal application knowledge graph, namely (h3, r31, t31) and (h3, r32, t32). Then the tree structure corresponding to the sample sentence is [h1 (r11, t11), h2, h3 [(r31, t31) (r32, t32)]].

[0118] The sample sequence carrying the mask mark is obtained by converting the tree structure into a sequence structure and randomly replacing an entity in the sequence structure with a mask mark. In addition, the mask mark generally refers to MASK. For example, the sequence structure is (h1, r11, t11, h2, h3, r31, t31, r32, t32), after replacing h2 with MASK, the sample sequence carrying the mask mark is (h1, r11, t11, MASK, h3, r31, t31, r32, t32).

[0119] The feature vector of the sample sequence is obtained by concatenating the word vector, position vector, and segment vector of the words in the sample sequence.

[0120] Among them, the target matrix of the sample sequence is determined based on the words and their number in the sample sequence, and whether any two words in the sample sequence belong to the same triplet; the words and their number in the sample sequence are used to determine the elements in the target matrix; whether any two words in the sample sequence belong to the same triplet is used to determine the specific values ​​of the elements in the target matrix. For example, for the sample sequence (h1, r11, t11, MASK, h3, r31, t31, r32, t32), its target matrix is ​​a 9×9 matrix [Mij], i, j = 1, 2, 3, 4...9. Since h1 and t11 belong to the same triplet, M13 and M31 are both equal to 0; since t31 and t32 do not belong to the same triplet, M79 and M97 are both equal to -∞.

[0121] The predicted word corresponding to the mask mark refers to the word corresponding to the position of the mask mark predicted by the language processing model. The actual word corresponding to the mask mark refers to the word replaced by the mask mark, such as h2.

[0122] Exemplarily, the terminal obtains a sample sentence and a sample financial internal application knowledge graph from a database, and queries the sample financial internal application knowledge graph according to the entity in the sample sentence to obtain the triple corresponding to the entity in the sample sentence; according to the entity in the sample sentence and the triple corresponding to the entity in the sample sentence, the sample sentence is converted into a corresponding tree structure. Then, the terminal converts the tree structure into a sequence structure, and randomly replaces a certain entity in the sequence structure with a mask tag to obtain a sample sequence carrying the mask tag, and performs feature extraction processing on the sample sequence to obtain a feature vector of the sample sequence. Finally, the terminal inputs the feature vector of the sample sequence and the target matrix of the sample sequence into the pre-trained language processing model to obtain the probability distribution corresponding to the mask tag, and determines the predicted word corresponding to the mask tag according to the probability distribution corresponding to the mask tag, and determines the loss value according to the difference between the predicted word corresponding to the mask tag and the actual word, and adjusts the model parameters of the pre-trained language processing model according to the loss value, and continues to repeat the above training process until the loss value obtained according to the trained language processing model is less than the preset threshold, then stops the training, and uses the trained language processing model as the trained language processing model.

[0123] For example, the terminal defines the sample financial internal application knowledge graph as , a sentence to be parsed is defined as , , For entities in the sentence, put each entity into the sample financial internal application knowledge graph ; for example, exist There is a triple in which there is a relation record, specifically ; exist There are three tuples with two relationship records in and ,but It becomes a tree structure. Next, the terminal embeds the obtained tree structure. The embedding is divided into three parts: token embedding, position embedding, and segment embedding. Token embedding converts each word in the sentence into a vector of fixed dimension. Position embedding allows the program to know the order of input. Segment embedding can be used to handle the situation where a sentence contains two sentences and can distinguish them. After embedding, The tree structure is converted into a sequence, Since the semantics of the original sentence will be destroyed after serialization, causing the BERT model to not work properly, a visible matrix is ​​defined , when two points on the sequence are in the same branch, the matrix value is 0, indicating that the two points are visible; when two points on the sequence are in different branches, the value is -∞, indicating that the two points are invisible, avoiding the destruction of the semantics of the original sentence. When performing self-attention training of the BERT model, it is necessary to modify the self-attention formula, which is used to represent the hidden state of the i+1th mask segment, which is obtained by the hidden state of the i-th mask segment; the details are as follows:

[0124]

[0125]

[0126]

[0127] in, , , are the model training parameters, is the hidden state of the ith mask segment, is the vector dimension, Visible Matrix , is the attention score. When two points are not visible, the score of that point is also 0, so it will not affect the training results.

[0128] Finally, the terminal uses the information in the sample financial internal application knowledge graph to input the entity and relationship information and the corresponding context into the BERT model for fine-tuning, so that it can better understand the text of the banking application, provide more potential information for the training process, and improve the accuracy of subsequent entity and relationship extraction.

[0129] In this embodiment, a sample sentence and a sample financial internal application knowledge graph are first obtained, and the sample sentence is converted into a corresponding tree structure through the sample financial internal application knowledge graph, and then the tree structure is converted into a sample sequence carrying a mask mark, and the sample sequence is subjected to feature extraction processing to obtain a feature vector of the sample sequence, and then the feature vector of the sample sequence and the target matrix of the sample sequence are input into a pre-trained language processing model to obtain a predicted word corresponding to the mask mark, and finally, according to the difference between the predicted word corresponding to the mask mark and the actual word, the pre-trained language processing model is iteratively trained to obtain a trained language processing model. In this way, in the model training process, the sample financial internal application knowledge graph is used to enhance the training of the pre-trained language processing model, so that the trained language processing model is more suitable for entity recognition in the financial field, thereby improving the recognition accuracy of the trained language processing model for entities in the financial field.

[0130] In an exemplary embodiment, the above step S301, obtaining sample sentences and a sample financial internal application knowledge graph, specifically includes the following contents: obtaining a sample financial internal application document; identifying a first sample entity and a first sample entity relationship from the sample financial internal application document through a pre-trained language processing model; adding the first sample entity and the first sample entity relationship to the initial financial internal application knowledge graph to obtain a sample financial internal application knowledge graph.

[0131] Among them, sample financial internal application documents refer to application documents related to financial internal applications used in training, such as requirement documents, design plans, test plans, database table structures, etc.

[0132] The first sample entity refers to an entity in the sample financial internal application document; the first sample entity relationship refers to an entity relationship between entities in the sample financial internal application document.

[0133] The sample financial internal application knowledge graph refers to the financial internal application knowledge graph obtained by adding the first sample entity and the first sample entity relationship identified from the sample financial internal application document on the basis of the initial financial internal application knowledge graph.

[0134] Exemplarily, the terminal obtains a sample financial internal application document from a database, performs feature extraction processing on the sample financial internal application document, obtains a feature vector of the sample financial internal application document, then inputs the feature vector of the sample financial internal application document into a pre-trained language processing model for entity recognition processing, obtains entities and entity relationships in the sample financial internal application document as first sample entities and first sample entity relationships; finally, adds the first sample entity and the first sample entity relationship to the initial financial internal application knowledge graph, obtains a processed financial internal application knowledge graph as the sample financial internal application knowledge graph

[0135] For example, see Figure 2 The initial financial internal application knowledge graph already has the entity "credit card application". When parsing the relevant text of the acquiring system, it is parsed as: "acquiring application", "notification", and "credit card application". The connection between the acquiring application and the credit card application will be established in the initial financial internal application knowledge graph, thereby obtaining an updated financial internal application knowledge graph as a sample financial internal application knowledge graph.

[0136] In this embodiment, a sample financial internal application document is first obtained, and then a first sample entity and a first sample entity relationship are identified from the sample financial internal application document through a pre-trained language processing model. Finally, the first sample entity and the first sample entity relationship are added to the initial financial internal application knowledge graph to obtain a sample financial internal application knowledge graph. In this way, the initial financial internal application knowledge graph is updated using the first sample entity and the first sample entity relationship in the sample financial internal application document, which can improve the richness of entities and entity relationships in the sample financial internal application knowledge graph, and is conducive to improving the accuracy of entity recognition in the financial field by a language processing model subsequently trained based on the sample financial internal application knowledge graph.

[0137] In an exemplary embodiment, the above step S302 converts the sample sentence into a corresponding tree structure through the sample financial internal application knowledge graph, which specifically includes the following contents: identifying the second sample entity in the sample sentence; identifying the associated entity of the second sample entity and the second sample entity relationship between the second sample entity and the associated entity from the sample financial internal application knowledge graph; constructing a tree structure corresponding to the sample sentence according to the second sample entity in the sample sentence, the associated entity of the second sample entity, and the second sample entity relationship between the second sample entity and the associated entity.

[0138] The second sample entity refers to an entity in the sample sentence, which can be obtained through manual annotation or model recognition.

[0139] Among them, the associated entity of the second sample entity refers to the entity in the sample financial internal application knowledge graph that has an associated relationship with the second sample entity, such as t11, t31, and t32.

[0140] Among them, the second sample entity relationship refers to the entity relationship between the second sample entity and the associated entity in the sample financial internal application knowledge graph, for example, the second sample entity relationship between h1 and t11 is r11, the second sample entity relationship between h3 and t31 is r31, and the second sample entity relationship between h3 and t32 is r32.

[0141] Among them, the tree structure corresponding to the sample sentence is a tree structure constructed according to the second sample entity in the sample sentence, the associated entity of the second sample entity, and the second sample entity relationship between the second sample entity and the associated entity, such as [h1 (r11, t11), h2, h3 [(r31, t31) (r32, t32)]].

[0142] Exemplarily, the terminal identifies the second sample entity in the sample sentence; identifies an entity that has an association relationship with the second sample entity from the sample financial internal application knowledge graph as the associated entity of the second sample entity, and identifies the association relationship between the second sample entity and the associated entity as the second sample entity relationship; finally, based on the second sample entity in the sample sentence, the associated entity of the second sample entity, and the second sample entity relationship between the second sample entity and the associated entity, a tree structure corresponding to the sample sentence is constructed.

[0143] For example, for the sample sentence (h1, h2, h3), the second sample entity is h1, h2, h3; h1 has a triple of relationship records (h1, r11, t11) in the sample financial internal application knowledge graph, and h3 has two triples of relationship records in the sample financial internal application knowledge graph, namely (h3, r31, t31) and (h3, r32, t32), then the tree structure corresponding to the sample sentence is [h1 (r11, t11), h2, h3 [(r31, t31) (r32, t32)]].

[0144] In this embodiment, during the model training process, the sample sentences are converted into corresponding tree structures through the sample financial internal application knowledge graph, which can make full use of the content in the sample financial internal application knowledge graph to ensure that the language processing model finally trained is more adapted to the entity recognition in the financial internal application field, so that the language processing model finally trained can more accurately perform entity recognition in the financial internal application field.

[0145] In an exemplary embodiment, Figure 4As shown, another answer determination method is provided, which is described by taking the method applied to a terminal as an example, and includes the following steps S401 to S410. Among them:

[0146] Step S401, obtaining sample sentences and sample financial internal application knowledge graphs.

[0147] Step S402: Convert the sample sentence into a corresponding tree structure through the sample financial internal application knowledge graph.

[0148] Step S403: convert the tree structure into a sample sequence carrying a mask mark, and perform feature extraction processing on the sample sequence to obtain a feature vector of the sample sequence.

[0149] Step S404, input the feature vector of the sample sequence and the target matrix of the sample sequence into the pre-trained language processing model to obtain the predicted words corresponding to the mask mark; the target matrix is ​​used to represent whether any two words in the sample sequence belong to the same triple.

[0150] Step S405, iteratively training the pre-trained language processing model according to the difference between the predicted word and the actual word corresponding to the mask mark to obtain a trained language processing model.

[0151] Step S406, obtaining issues associated with financial internal applications.

[0152] Step S407: Identify entities and entity relationships from the question through the trained language processing model.

[0153] Step S408, generating a knowledge graph query language based on entities, entity relationships, and question types.

[0154] Step S409: query the pre-built financial internal application knowledge graph according to the knowledge graph query language, and obtain the financial internal application entity corresponding to the knowledge graph query language as the target financial internal application entity.

[0155] Step S410: Generate answers to questions based on the target financial internal application entity.

[0156] In the above-mentioned answer determination method, when determining the answers to questions related to financial internal applications, the answers to the questions are automatically generated based on the trained language processing model and the pre-built financial internal application knowledge graph, without the need to manually consult a large number of application documents of financial internal applications, thereby simplifying the answer determination process, thereby improving the answer determination efficiency, and avoiding the defect of low answer determination efficiency caused by manually consulting a large number of application documents of financial internal applications.

[0157] In an exemplary embodiment, in order to more clearly illustrate the answer determination method provided in the embodiment of the present application, the answer determination method is specifically described below with a specific embodiment. In one embodiment, Figure 5 As shown, this application also provides a method for constructing an internal bank application question-and-answer system based on a knowledge graph, combining knowledge graph and BERT technology to create an efficient internal bank application question-and-answer system; by leveraging the powerful semantic association capabilities of the knowledge graph and the natural language processing capabilities of BERT, semantic information is extracted from application documents, requirement documents, and test documents to build an intelligent internal application knowledge base, which can provide a better knowledge service platform for technical personnel. The specific contents are as follows:

[0158] (1) Automatically scan application-related documents and extract application document content as the initial document library.

[0159] (2) Use the pre-trained BERT model to scan the initial document library and extract the entities and entity relationships.

[0160] (3) Insert the entities and their entity relationships into the bank’s internal application knowledge graph to obtain a knowledge graph that covers the basic information and relationships of each application.

[0161] (4) Use the applied knowledge graph to assist the training of the BERT model, provide some potential knowledge for the BERT training process, and improve the training effect of BERT.

[0162] (5) Automatically scan application document information such as requirement documents, design plans, test plans, table structures, and extract document content.

[0163] (6) Use the BERT model enhanced by the knowledge graph to extract document content and insert new entities and entity relationships into the knowledge graph to build a banking question-answering system.

[0164] (7) When a user asks a question in the bank’s question-and-answer system, the system scans keywords and extracts relevant information from the knowledge graph to generate an answer statement.

[0165] The specific implementation methods are as follows:

[0166] (1) Bank internal systems usually use applications to divide functions. Each application will submit a corresponding application white paper, which covers the application's business architecture, data architecture, technical architecture, security architecture, etc., and introduces the overall situation of the application and the scope of its functions. Use crawlers to automatically obtain the content of the application white paper, perform text preprocessing on the document, extract the document content and unify the format, and generate an initial database.

[0167] (2) Use a pre-trained NLP model to scan the information in the initial database. For example, use a pre-trained BERT model as a classifier to label the text in the initial database. The BERT model can automatically extract entity content from the text and capture the relationship between entities.

[0168] (3) Insert entity information and relationship information into the bank’s application knowledge graph to obtain a knowledge graph that covers the basic information of each application and the connections between applications.

[0169] (4) Use the knowledge graph obtained in the above steps to assist the BERT model in enhanced training. Since the BERT model is an open pre-training model, the introduction of the applied knowledge graph can improve the accuracy of the BERT model in processing internal bank documents.

[0170] (5) Automatically scan application document information such as requirement documents, design plans, test plans, table structures, etc. (referring to internal bank documents outside the training set) and extract document content; expand the automated scanning mechanism to cover different types of documents, including requirement documents, design plans, test plans, and database table structures. Use domain-specific knowledge and natural language processing technology to extract information related to bank applications from these documents.

[0171] (6) Use the BERT model enhanced by the knowledge graph to extract document content, improve extraction accuracy, and insert it into the knowledge graph; use the entity and relationship information in the knowledge graph to further fine-tune the BERT model to adapt to new document types and content. Incorporate the contextual information of the knowledge graph into the document extraction process to improve the contextual awareness and accuracy of the extraction.

[0172] (7) When a user asks a question, the BERT model is used to identify entities, relationships, question types and other information in the question, and natural language is converted into a knowledge graph query language for query, to obtain relevant entity information, and to generate an answer statement using the relevant entity information.

[0173] (8) After receiving the answer, the user can evaluate and provide feedback on the system's answer. For example, an interactive interface can be provided to the user, allowing the user to evaluate and provide feedback on the answers provided by the system, so as to continuously improve the system's performance and user satisfaction. User feedback data can be collected for reference in system optimization and future improvements.

[0174] The above-mentioned method for constructing a question-and-answer system for internal bank applications based on knowledge graph can achieve the following technical effects: (1) As the cornerstone of this system, the knowledge graph integrates and associates information from various applications to form a comprehensive and structured knowledge network, which helps the question-and-answer system to better understand and analyze data in different fields, and improve the quality of the question-and-answer system's accurate understanding and answers to user questions; at the same time, the advantage of the knowledge graph is that it can also capture the relationship between entities, thereby providing more comprehensive and accurate application knowledge; (2) The introduction of BERT technology makes the system more intelligent and flexible in processing natural language; BERT's pre-trained model can better understand the context and meaning of language by learning bidirectional context information, which enables the question-and-answer system to better handle the complex semantic structure of documents and questions; (3) Combining knowledge graph and BERT to build a question-and-answer system for internal bank applications can greatly reduce the time it takes for technical personnel to understand the business and technical architecture of the application system and improve work efficiency; the system can effectively improve the intelligence level of internal information management in banks and provide more convenient and efficient work support for technical teams.

[0175] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0176] Based on the same inventive concept, the embodiment of the present application also provides an answer determination device for implementing the answer determination method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more answer determination device embodiments provided below can refer to the limitations of the answer determination method above, and will not be repeated here.

[0177] In an exemplary embodiment, Figure 6 As shown, an answer determination device is provided, comprising: a question acquisition module 610, an entity recognition module 620, an entity determination module 630 and an answer generation module 640, wherein:

[0178] The problem acquisition module 610 is used to acquire problems associated with financial internal applications.

[0179] The entity recognition module 620 is used to identify entities and entity relationships from the question through a trained language processing model; the trained language processing model is obtained by iteratively training a pre-trained language processing model through a sample financial internal application knowledge graph.

[0180] The entity determination module 630 is used to determine the target financial internal application entity that satisfies the entity and the entity relationship from the pre-constructed financial internal application knowledge graph.

[0181] The answer generation module 640 is used to generate an answer to the question according to the target financial internal application entity.

[0182] In an exemplary embodiment, the entity determination module 630 is also used to determine candidate financial internal application entities associated with the entity from a pre-constructed financial internal application knowledge graph; and to determine, from the candidate financial internal application entities, a target financial internal application entity whose association relationship with the entity satisfies the entity relationship.

[0183] In an exemplary embodiment, the entity determination module 630 is also used to generate a knowledge graph query language based on entities, entity relationships, and question types; based on the knowledge graph query language, a pre-built financial internal application knowledge graph is queried to obtain the financial internal application entity corresponding to the knowledge graph query language as the target financial internal application entity.

[0184] In an exemplary embodiment, the answer determination device also includes a first construction module, which is used to obtain a financial internal application document to be processed; identify new entities and new entity relationships from the financial internal application document to be processed through a trained language processing model; and add the new entities and new entity relationships to an initial financial internal application knowledge graph to obtain a pre-constructed financial internal application knowledge graph.

[0185] In an exemplary embodiment, the answer determination device also includes a model training module, which is used to obtain sample sentences and sample financial internal application knowledge graphs; convert the sample sentences into corresponding tree structures through the sample financial internal application knowledge graphs; convert the tree structures into sample sequences carrying mask tags, and perform feature extraction processing on the sample sequences to obtain feature vectors of the sample sequences; input the feature vectors of the sample sequences and the target matrix of the sample sequences into a pre-trained language processing model to obtain predicted words corresponding to the mask tags; the target matrix is ​​used to characterize whether any two words in the sample sequence belong to the same triplet; and iteratively train the pre-trained language processing model according to the difference between the predicted words corresponding to the mask tags and the actual words to obtain a trained language processing model.

[0186] In an exemplary embodiment, the answer determination device also includes a second construction module, which is used to obtain a sample financial internal application document; identify a first sample entity and a first sample entity relationship from the sample financial internal application document through a pre-trained language processing model; and add the first sample entity and the first sample entity relationship to an initial financial internal application knowledge graph to obtain a sample financial internal application knowledge graph.

[0187] In an exemplary embodiment, the model training module is also used to identify the second sample entity in the sample sentence; identify the associated entity of the second sample entity and the second sample entity relationship between the second sample entity and the associated entity from the sample financial internal application knowledge graph; and construct a tree structure corresponding to the sample sentence based on the second sample entity in the sample sentence, the associated entity of the second sample entity, and the second sample entity relationship between the second sample entity and the associated entity.

[0188] Each module in the above answer determination device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0189] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, an answer determination method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0190] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0191] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0192] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0193] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0195] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0196] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0197] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for determining an answer, characterized in that: The method comprises: Get issues related to internal financial applications; Using the trained language processing model, entities and entity relationships are identified from the question; the trained language processing model is obtained by iteratively training a pre-trained language processing model using a sample financial internal application knowledge graph; Determine, from a pre-built financial internal application knowledge graph, a target financial internal application entity that satisfies the entity and the entity relationship; An answer to the question is generated according to the target financial internal application entity.

2. The method according to claim 1, characterized in that The step of determining, from the pre-built financial internal application knowledge graph, a target financial internal application entity that satisfies the entity and the entity relationship comprises: Determine candidate financial internal application entities associated with the entity from a pre-constructed financial internal application knowledge graph; From the candidate internal financial application entities, a target internal financial application entity whose association relationship with the entity satisfies the entity relationship is determined.

3. The method according to claim 1, characterized in that The step of determining, from the pre-built financial internal application knowledge graph, a target financial internal application entity that satisfies the entity and the entity relationship comprises: Generate a knowledge graph query language according to the entity, the entity relationship and the question type of the question; According to the knowledge graph query language, a pre-constructed financial internal application knowledge graph is queried to obtain the financial internal application entity corresponding to the knowledge graph query language as the target financial internal application entity.

4. The method according to claim 1, characterized in that: The pre-built financial internal application knowledge graph is obtained in the following way: Obtain financial internal application documents to be processed; Using the trained language processing model, identifying new entities and new entity relationships from the internal financial application documents to be processed; The new entity and the new entity relationship are added to the initial financial internal application knowledge graph to obtain the pre-built financial internal application knowledge graph.

5. The method according to any one of claims 1 to 4, characterized in that: The trained language processing model is trained in the following way: Obtaining a sample sentence and the sample financial internal application knowledge graph; By using the sample financial internal application knowledge graph, the sample sentence is converted into a corresponding tree structure; Converting the tree structure into a sample sequence carrying a mask mark, and performing feature extraction processing on the sample sequence to obtain a feature vector of the sample sequence; Inputting the feature vector of the sample sequence and the target matrix of the sample sequence into the pre-trained language processing model to obtain the predicted words corresponding to the mask mark; The target matrix is ​​used to represent whether any two words in the sample sequence belong to the same triple; According to the difference between the predicted words and the actual words corresponding to the mask mark, the pre-trained language processing model is iteratively trained to obtain the trained language processing model.

6. The method according to claim 5, characterized in that The sample financial internal application knowledge graph is obtained in the following way: Get sample financial internal application documents; identifying, by means of the pre-trained language processing model, a first sample entity and a first sample entity relationship from the sample financial internal application document; The first sample entity and the first sample entity relationship are added to the initial financial internal application knowledge graph to obtain the sample financial internal application knowledge graph.

7. The method according to claim 5, characterized in that The converting the sample sentence into a corresponding tree structure through the sample financial internal application knowledge graph includes: Identifying a second sample entity in the sample sentence; Identifying, from the sample financial internal application knowledge graph, an associated entity of the second sample entity and a second sample entity relationship between the second sample entity and the associated entity; A tree structure corresponding to the sample sentence is constructed according to the second sample entity in the sample sentence, the associated entity of the second sample entity, and the second sample entity relationship between the second sample entity and the associated entity.

8. An answer determination device, characterized in that: The device comprises: A problem acquisition module is used to acquire problems associated with internal financial applications; An entity recognition module is used to identify entities and entity relationships from the question through a trained language processing model; the trained language processing model is obtained by iteratively training a pre-trained language processing model through a sample financial internal application knowledge graph; An entity determination module, used to determine a target financial internal application entity that satisfies the entity and the entity relationship from a pre-built financial internal application knowledge graph; The answer generation module is used to generate an answer to the question according to the target financial internal application entity.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images