A method and device for constructing a commercial bank knowledge graph and intelligent question answering based on a large language model, a medium and a product

Abstract

The application discloses a commercial bank knowledge graph construction and intelligent question and answer method and device based on a large language model, equipment, medium and product. Including: obtaining a data set, and performing entity extraction operation on the data set based on the large language model to obtain an extraction result; the extraction result includes each entity and the relationship between entities; the entity types include customers, products, transactions, organizations and events; the relationship types include owning accounts, purchasing or holding products, product types, subordinate relationships and existing transactions; a commercial bank knowledge graph is generated according to each entity and the relationship between entities; an input question is obtained, and semantic understanding is performed on the input question to identify a query intention; based on the query intention, the commercial bank knowledge graph is matched and queried to obtain a query result. Through the technical scheme of the application, the large language model technology in the field of artificial intelligence can be used to develop and design a knowledge graph for commercial banks, and intelligent question and answer and knowledge management are realized.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a method, apparatus, device, medium and product for constructing a knowledge graph for commercial banks and intelligent question answering based on a large language model. Background Technology

[0002] With the rapid development of fintech innovation and the demand for digital economic transformation, the complexity and workload of banking operations have increased significantly. At the same time, the requirements for internal management and external supervision within the banking system are becoming increasingly stringent. Traditional commercial bank knowledge base systems suffer from drawbacks such as lagging information updates, low search efficiency, information redundancy, and a lack of customized services, which urgently need to be addressed. In the process of building commercial bank knowledge bases, current industry solutions often fail to meet the demands due to issues such as lagging information updates and slow response times. Summary of the Invention

[0003] This invention provides a method, apparatus, device, medium, and product for constructing a commercial bank knowledge graph and providing intelligent question answering based on a large language model. This enables the development and design of knowledge graphs for commercial banks using large language model technology in the field of artificial intelligence, thereby achieving intelligent question answering and knowledge management.

[0004] According to one aspect of the present invention, a method for constructing a commercial bank knowledge graph and intelligent question answering based on a large language model is provided, comprising: Acquire a dataset and perform entity extraction on the dataset based on a large language model to obtain extraction results; the extraction results include: each entity and the relationships between entities; entity types include: customer, product, transaction, organization, and event; relationship types include: owning an account, purchasing or holding a product, product type, affiliation, and existing transaction; A commercial bank knowledge graph is generated based on the entities and the relationships between them. Obtain the input question, perform semantic understanding on the input question, and identify the query intent; Based on the query intent, the query is performed by matching the commercial bank knowledge graph to obtain the query results.

[0005] According to another aspect of the present invention, a device for constructing a commercial bank knowledge graph and providing intelligent question answering based on a large language model is provided, the device comprising: The acquisition and extraction module is used to acquire a dataset and perform entity extraction operations on the dataset based on a large language model to obtain extraction results. The extraction results include: each entity and the relationships between entities; entity types include: customer, product, transaction, organization, and event; relationship types include: owning an account, purchasing or holding a product, product type, affiliation, and existing transaction. The generation module is used to generate a commercial bank knowledge graph based on each entity and the relationships between entities. The acquisition and recognition module is used to acquire the input question, perform semantic understanding on the input question, and identify the query intent; The matching and query module is used to match the commercial bank knowledge graph based on the query intent and perform a query to obtain the query results.

[0006] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to execute the commercial bank knowledge graph construction and intelligent question answering method based on a large language model according to any embodiment of the present invention.

[0007] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the commercial bank knowledge graph construction and intelligent question answering method based on a large language model as described in any embodiment of the present invention.

[0008] According to another aspect of the present invention, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the commercial bank knowledge graph construction and intelligent question answering method based on a large language model as described in any embodiment of the present invention.

[0009] This invention first acquires a dataset and performs entity extraction on it based on a large language model to obtain extraction results. These results include various entities and their relationships. Entity types include customers, products, transactions, organizations, and events; relationship types include having an account, purchasing or holding a product, product type, affiliation, and existing transactions. Then, a commercial bank knowledge graph is generated based on the entities and their relationships. Next, an input question is obtained, and semantic understanding is performed to identify the query intent. Finally, based on the query intent, the commercial bank knowledge graph is matched and a query is performed to obtain the query results. This invention enables the development and design of a knowledge graph for commercial banks using large language model technology in the field of artificial intelligence, achieving intelligent question answering and knowledge management.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a method for constructing a commercial bank knowledge graph and intelligent question answering based on a large language model, as described in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a commercial bank knowledge graph construction and intelligent question answering device based on a large language model in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device that implements the commercial bank knowledge graph construction and intelligent question answering method based on a large language model according to an embodiment of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and their derivatives, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0016] Example 1 Figure 1 This is a flowchart illustrating a method for constructing a commercial bank knowledge graph and providing intelligent question answering based on a large language model, as described in this embodiment of the invention. This embodiment is applicable to the construction of a commercial bank knowledge graph and providing intelligent question answering based on a large language model. The method can be executed by the device for constructing a commercial bank knowledge graph and providing intelligent question answering based on a large language model, as described in this embodiment of the invention. This device can be implemented using software and / or hardware methods, such as... Figure 1 As shown, the method specifically includes the following steps: S101. Obtain the dataset and perform entity extraction on the dataset based on the large language model to obtain the extraction results.

[0017] The extracted results include: entities and relationships between entities; entity types include: customers, products, transactions, organizations, and events; relationship types include: having an account, purchasing or holding products, product type, affiliation, and having a transaction.

[0018] In this embodiment, the dataset may include: (1) internal data: customer information, transaction records, financial statements and internal documents, etc.; (2) external data: published economic data, commercial bank policies, industry reports and social media news, etc., to supplement the deficiencies of internal data and provide more comprehensive data support for the commercial bank knowledge graph.

[0019] Specifically, the data in this embodiment of the invention originates from an internal commercial database combined with external data (such as financial market reports, publicly available bank annual reports and data). External data can be scraped from bank websites, financial papers, and social media, while internal data is exported from the bank's internal database after privacy filtering. To remove noise and redundancy from the original data, deduplication, default handling, and data unification are performed. Simultaneously, text data is segmented and part-of-speech tagging is performed to provide data support for entity recognition and extraction in subsequent large language models (which can be simply referred to as large models).

[0020] Next, the data is cleaned and preprocessed to ensure quality and consistency for the next stage of knowledge extraction and optimization. In the knowledge extraction and optimization stage, this embodiment of the invention utilizes a large language model for entity recognition, relation extraction, and event extraction. Simultaneously, it can combine techniques such as prompting, data augmentation, and fine-tuning to improve the quality of knowledge extraction.

[0021] To enable the large model to learn and optimize better, this embodiment of the invention first establishes a set of ontology relationships for the commercial banking field, which mainly includes five entity categories: customers, products, transactions, organizations, and events. The detailed descriptions and relationship types of these five entity categories are shown in Table 1. This invention utilizes labeled supervised data to train an open-source large model to learn the relationships between entities, and then combines hints and fine-tuning techniques to optimize the large model so that it can identify entities specific to the banking domain.

[0022] S102. Generate a commercial bank knowledge graph based on each entity and the relationships between entities.

[0023] During the knowledge fusion and processing stage, commercial bank knowledge bases face challenges due to their broad data coverage, including inconsistent knowledge quality, knowledge duplication between commercial databases and external data, and unclear relationships between knowledge points. Therefore, the extraction of knowledge needs to be cleaned and integrated during the construction of the commercial bank knowledge graph to ensure its quality. Specifically, knowledge redundancy can be addressed through referencing resolution and entity disambiguation, unifying entity representations to obtain the final entity extraction results. Furthermore, a cyclical process of knowledge reasoning and quality assessment optimizes the commercial bank knowledge graph, ensuring its timeliness and accuracy.

[0024] S103. Obtain the input question and perform semantic understanding on the input question to identify the query intent.

[0025] The input question can be submitted in either text or voice format.

[0026] Specifically, it involves obtaining the user's input question, performing semantic understanding on the user's input question, and identifying the query intent.

[0027] S104. Based on the query intent, match the commercial bank knowledge graph and perform a query to obtain the query results.

[0028] Specifically, based on the query intent, the system performs knowledge graph matching and querying of commercial banks, and the query results can be output as text or voice; if the query results do not exist, a customized response is returned.

[0029] This invention first acquires a dataset and performs entity extraction on it based on a large language model to obtain extraction results. These results include various entities and their relationships. Entity types include customers, products, transactions, organizations, and events; relationship types include having an account, purchasing or holding a product, product type, affiliation, and existing transactions. Then, a commercial bank knowledge graph is generated based on the entities and their relationships. Next, an input question is obtained, and semantic understanding is performed to identify the query intent. Finally, based on the query intent, the commercial bank knowledge graph is matched and a query is performed to obtain the query results. This invention enables the development and design of a knowledge graph for commercial banks using large language model technology in the field of artificial intelligence, achieving intelligent question answering and knowledge management.

[0030] Optionally, a commercial bank knowledge graph is generated based on the entities and the relationships between them, including: Perform referential resolution operations on each entity to eliminate referential differences of the same entity in different contexts.

[0031] In the knowledge fusion and processing stage, knowledge redundancy is resolved through referential resolution and entity disambiguation, and the final ontology extraction result is obtained by unifying entity representation. Specifically, referential resolution addresses the differences in referencing the same entity in different contexts; entity disambiguation distinguishes between entities with the same name but different meanings. Afterwards, a synonym entity library is constructed to achieve a unified representation of the same or similar entities from different sources or representation methods.

[0032] Calculate the character similarity between any two entities, and replace the names of two entities with character similarity higher than the similarity threshold with the standard entity name.

[0033] In its implementation, entity alignment aims to build a synonym entity library to recognize the same or similar entities from different sources or representations. In this embodiment, the traditional character similarity calculation between two entities is used to evaluate their similarity. Specifically, entity names are mapped to standard objects in the database, and then the final similarity result between the two entities is calculated. Based on the similarity calculation result, entity names with high similarity (character similarity above a similarity threshold) are uniformly replaced. For example, in this embodiment, the Jaccard correlation coefficient can be used as an indicator to measure the similarity between two entities. The formula for calculating the Jaccard correlation coefficient is as follows: ; in, Represents the textual similarity between entities A and B. This indicates the number of characters that are the same between entity A and entity B. This represents the number of characters shared by entity A and entity B.

[0034] Calculating text similarity to determine whether two entities truly belong to the same entity, thereby achieving entity alignment, helps reduce redundancy in knowledge graphs and improves their quality.

[0035] The credibility of the merged information is assessed, and a commercial bank knowledge graph is generated based on entities with credibility values ​​above the credibility threshold and the relationships between entities.

[0036] In the construction of knowledge graphs, due to the diversity of knowledge sources, the acquired knowledge often contains a large amount of redundant information. Therefore, knowledge merging becomes a crucial step, aiming to handle similar or differing information between entities and establish a clear and consistent direct mapping relationship between the knowledge base and entities by integrating entity, relationship, and attribute information from commercial bank knowledge. When processing similar entities or attributes from different data sources, it is necessary to carefully compare their subtle differences and redundant content. To ensure the quality and accuracy of the knowledge base, preset rules or manual judgment can be used to assess the credibility of information based on professional domain knowledge and experience. Information with high credibility (credibility above the credibility threshold) is retained and integrated into the knowledge base; while information with low credibility or conflicting information needs to be discarded or further verified. For example, two different data sources describe customer contact information as "customer's phone number" and "customer's contact number," but since customers may leave different contact information, this data needs to be retained and merged.

[0037] Optionally, generate a commercial bank knowledge graph, including: A graph database is used to store the knowledge graph of commercial banks.

[0038] After the above steps, the entity, relationship, and attribute data required to construct the commercial bank's knowledge graph have been successfully obtained. Next, to effectively store the merged data, this embodiment of the invention uses a graph database to store the commercial bank's knowledge graph.

[0039] Import knowledge data into a graph database and insert triplet data using a third-party library.

[0040] In this context, a triple represents each entity and the relationships between entities.

[0041] Specifically, the database is first imported, and then a third-party library is used to efficiently insert triplet data. Triplets represent entities (banks, customers, accounts, etc.) and the relationships between them.

[0042] By creating entity nodes in the graph database using node classes and defining and connecting relationships between entities using relation classes, a knowledge graph of a commercial bank can be obtained.

[0043] Specifically, a graph database is used to store the knowledge graph of the commercial bank. Knowledge graph objects are created using the graph object creation interface provided by the graph database, entity nodes are created using the node creation interface, the relationship definition interface is used to establish the association between entities, and entity-relationship-entity data is efficiently imported through the triple insertion interface.

[0044] Optionally, during the process of matching the commercial bank's knowledge graph based on the query intent and obtaining the query results, the following operations are performed: Retrieval operation: Domain knowledge-intensive text content is treated as knowledge documents, and the knowledge documents are divided into multiple knowledge document blocks; the input question and knowledge document blocks are mapped into high-dimensional vectors using an embedding model, forming text vectors and storing them in an ordered vector knowledge base; the most semantically relevant document fragments to the query are retrieved from the vector knowledge base using vector search technology.

[0045] During the retrieval phase, the knowledge documents to be retrieved are prepared, typically domain-knowledge-intensive text content, and these documents are logically divided into blocks. Pre-trained embedding models (such as BERT (Bidirectional Encoder Representations from Transformers) or GLM (General Language Model)) are used to map user queries and document blocks in the knowledge base into high-dimensional vectors. These text vectors are then stored in an ordered vector knowledge base.

[0046] The process of quickly retrieving the most semantically relevant document fragments from a large-scale vector database using efficient vector search techniques typically employs a dual-tower model architecture, which includes two independent encoders that process the query and the document respectively, in order to achieve efficient similarity calculation.

[0047] Enhancement operation: Obtain the content of the most semantically relevant documents in the search results before sorting by a predetermined number, and integrate the retrieved knowledge with the input question through a weighted average strategy or a concatenation strategy to construct enhanced prompt words, which are then fed into the large language model as input context.

[0048] During the enhancement phase, based on actual needs, the top-K semantically most relevant document contents (K being a preset number that can be adjusted according to actual requirements) are retrieved and obtained. Using specific strategies (such as weighted averaging, concatenation, etc.), the retrieved knowledge is fused with the query to construct enhanced prompt words, which are then fed into the main model as input context. The fused prompt words provide richer contextual information to the generation module, thus more effectively guiding the model to generate answers.

[0049] Generation process: Using the enhanced prompts, the final text response is generated through a large language model.

[0050] During the generation phase, enhanced prompts are used to generate the final text response through a large language model. This process can also be fine-tuned according to specific tasks or domains to improve the professionalism and accuracy of the generated content.

[0051] Optional features also include: fine-tuning operations: The cross-entropy loss function is used to measure the difference between the model's generated results and the actual results, and the model parameters are optimized by minimizing the loss function.

[0052] During the model fine-tuning phase, a suitable loss function is first defined. In this embodiment, a cross-entropy loss function can be defined to measure the difference between the SQL (Structured Query Language) statements generated by the model and the actual SQL statements, and the model parameters are optimized by minimizing the loss function.

[0053] Choose an optimizer and set hyperparameters based on model characteristics and data size; hyperparameters include learning rate and weight decay.

[0054] Specifically, select an appropriate optimizer and set hyperparameters such as learning rate and weight decay reasonably according to the characteristics of the model and the scale of the data to ensure stable convergence of the model.

[0055] The training data is divided into multiple mini-batches for forward and backward propagation to update the model parameters.

[0056] During training, a batch training approach is used, dividing the training data into multiple mini-batches for forward and backward propagation to update the model parameters. The batch size is set appropriately based on hardware resources and data volume, such as 32, 64, or 128.

[0057] An early stopping mechanism is set up so that training stops when the loss on the validation set does not decrease within a consecutive target number of batches, and the model parameters with the best performance are selected as the final result.

[0058] Specifically, the model performance is evaluated periodically on the validation set, and metrics such as loss and accuracy are recorded. If the validation set performance does not improve within a certain number of steps, an early stopping strategy is adopted to prevent overfitting. For example, if the loss on the validation set does not decrease within 3 to 5 consecutive batches (target number, which can be adjusted according to actual needs or experience), training is stopped, and the model parameters with the best performance are selected as the final result.

[0059] Optional features include: fine-tuning the evaluation process. Build the test suite.

[0060] The test set data covers all key aspects of banking operations, including: counter services, credit management, financial accounting, customer management, and risk control and compliance.

[0061] Specifically, the evaluation process is rigorously based on a carefully constructed test set. The data selection for the test set covers all key aspects of banking operations, including but not limited to: counter services, credit management, financial accounting, customer management, risk control and compliance, ensuring data diversity and representativeness. In terms of test environment setup, efforts are made to simulate a real bank's database environment, including the configuration of the database management system and the maintenance of data integrity and consistency, to ensure that the evaluation results accurately reflect the model's performance in practical applications.

[0062] This invention establishes a structured and unstructured dedicated index library for bank products, risk control rules, counter services, and credit policies. Through semantic recall, reordering, and business relevance filtering algorithms, it achieves a strong binding between the output of the large model and the bank's real business data, eliminates the illusion problem of general large models, and ensures that knowledge output is compliant, accurate, and traceable.

[0063] The accuracy, execution accuracy, and target score of the large language model are calculated based on the test set.

[0064] Among them, accuracy is calculated by comparing the number of complete matches between the model and the real results on the test set; execution accuracy is based on testing the model-generated results against the bank's actual database to assess the feasibility of real-world scenarios; target score measures the semantic similarity between the generated results and the real results.

[0065] This invention constructs a multi-dimensional evaluation index system to conduct a detailed evaluation of the fine-tuned model. The accuracy is calculated by comparing the number of complete matches between the model and real SQL statements on the test set. The execution accuracy is based on the execution test of SQL statements generated by the large model on the actual bank database, which can reveal its feasibility in real scenarios. The target score measures the semantic similarity between the generated SQL and the real SQL, and can analyze the semantic understanding of the large model in complex relational queries.

[0066] Fine-tuning evaluation is performed based on accuracy, execution accuracy, and target score.

[0067] After recording the scores of various indicators, a meticulous manual review of the generated SQL statements is essential. This manual review, from a professional database management and business logic perspective, can keenly identify potential flaws in the model. For example, when reviewing SQL queries involving financial data statistics and analysis, it can be detected that while the model generates seemingly reasonable queries, subtle errors exist in handling complex financial function calculations, date range filtering, or data grouping and aggregation logic. These errors lead to deviations in financial statement data, thereby affecting the accuracy of bank decisions. Similarly, when handling related queries involving customer information, credit data, and risk control data, errors in processing multi-condition filtering or data join relationships can affect the accurate assessment of customer risk and business operations.

[0068] This invention, centered on a large language model, addresses the challenges of traditional commercial bank knowledge base systems, such as low search efficiency, information redundancy, and lack of customized services after information updates. From the perspective of large language models in the field of artificial intelligence, it utilizes large models and retrieval enhancement generation technology to construct a knowledge base for bank employees. The constructed commercial bank knowledge graph is applied to the large-model commercial bank knowledge base system, significantly improving the efficiency and accuracy of internal knowledge management within the bank. This provides banks and users with precise and efficient knowledge acquisition services, and strongly supports banking business processing and service quality.

[0069] Example 2 Figure 2 This is a schematic diagram of a device for constructing and intelligently answering knowledge graphs of commercial banks based on a large language model, according to an embodiment of the present invention. This embodiment is applicable to the construction of knowledge graphs of commercial banks based on large language models and intelligent question answering. The device can be implemented using software and / or hardware, and can be integrated into any device that provides functions for constructing and intelligently answering knowledge graphs of commercial banks based on large language models, such as… Figure 2 As shown, the commercial bank knowledge graph construction and intelligent question answering device based on a large language model specifically includes: an acquisition and extraction module 201, a generation module 202, an acquisition and recognition module 203, and a matching and query module 204.

[0070] The acquisition and extraction module 201 is used to acquire a dataset and perform entity extraction operations on the dataset based on a large language model to obtain extraction results. The extraction results include: each entity and the relationships between entities; entity types include: customer, product, transaction, organization, and event; relationship types include: owning an account, purchasing or holding a product, product type, affiliation, and existing transaction. Generation module 202 is used to generate a commercial bank knowledge graph based on each entity and the relationships between entities; The acquisition and recognition module 203 is used to acquire the input question, perform semantic understanding on the input question, and identify the query intent; The matching and query module 204 is used to match the commercial bank knowledge graph based on the query intent and perform a query to obtain the query result.

[0071] Optionally, the generation module 202 is specifically used for: Perform referential resolution operations on each entity to eliminate referential differences of the same entity in different contexts; Calculate the character similarity between any two entities, and replace the names of two entities with character similarity higher than the similarity threshold with the standard entity name; The credibility of the merged information is assessed, and a commercial bank knowledge graph is generated based on entities with credibility values ​​above the credibility threshold and the relationships between entities.

[0072] Optionally, the generation module 202 is specifically used for: The commercial bank knowledge graph is stored using a graph database; Import knowledge data into the graph database, and insert triplet data using a third-party library; the triplet represents each entity and the relationship between entities. By creating entity nodes in the graph database using node classes and defining and connecting relationships between entities using relation classes, a knowledge graph of a commercial bank can be obtained.

[0073] Optionally, the device is specifically used to perform: Retrieval operation: The domain knowledge-intensive text content is taken as a knowledge document, and the knowledge document is divided into multiple knowledge document blocks; the input question and the knowledge document blocks are mapped into high-dimensional vectors using an embedding model to form text vectors and stored in an ordered vector knowledge base; the most semantically relevant document fragments to the query are retrieved from the vector knowledge base using vector search technology. Enhancement operation: Obtain the content of the most semantically relevant documents in the search results before sorting by a preset number, and fuse the retrieved knowledge with the input question through a weighted average strategy or a concatenation strategy to construct enhanced prompt words, which are then fed into the large language model as input context; Generation process: Using the enhanced prompts, the final text response is generated through a large language model.

[0074] Optionally, the device is also specifically used to perform: fine-tuning operations: The cross-entropy loss function is used to measure the difference between the model-generated results and the true results, and the model parameters are optimized by minimizing the loss function; Select an optimizer and set hyperparameters based on model characteristics and data size; the hyperparameters include: learning rate and weight decay. The training data is divided into multiple mini-batches for forward and backward propagation to update the model parameters. An early stopping mechanism is set up so that training stops when the loss on the validation set does not decrease within a consecutive target number of batches, and the model parameters with the best performance are selected as the final result.

[0075] Optionally, the device is also specifically used to perform: fine-tuning evaluation operations: Construct a test suite; the test suite data covers all key aspects of banking operations, including: counter services, credit management, financial accounting, customer management, and risk control and compliance. The accuracy, execution accuracy, and target score of the large language model are calculated based on the test set. The accuracy is calculated by comparing the number of perfect matches between the model and the real results on the test set. The execution accuracy is tested on the model's generated results using a real bank database to evaluate its feasibility in real-world scenarios. The target score measures the semantic similarity between the generated results and the real results. Fine-tuning evaluation is performed based on accuracy, execution accuracy, and target score.

[0076] The above-mentioned products can execute the commercial bank knowledge graph construction and intelligent question answering method based on a large language model provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects of the execution method.

[0077] Example 3 Figure 3 A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0078] like Figure 3As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0079] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0080] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as the commercial bank knowledge graph construction and intelligent question answering method based on a large language model: Acquire a dataset and perform entity extraction on the dataset based on a large language model to obtain extraction results; the extraction results include: each entity and the relationships between entities; entity types include: customer, product, transaction, organization, and event; relationship types include: owning an account, purchasing or holding a product, product type, affiliation, and existing transaction; A commercial bank knowledge graph is generated based on the entities and the relationships between them. Obtain the input question, perform semantic understanding on the input question, and identify the query intent; Based on the query intent, the query is performed by matching the commercial bank knowledge graph to obtain the query results.

[0081] In some embodiments, the commercial bank knowledge graph construction and intelligent question answering method based on a large language model can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the commercial bank knowledge graph construction and intelligent question answering method based on a large language model described above can be performed. Alternatively, in other embodiments, processor 31 can be configured to execute the commercial bank knowledge graph construction and intelligent question answering method based on a large language model by any other suitable means (e.g., by means of firmware).

[0082] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0083] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0084] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0086] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0087] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0088] In one embodiment, the present invention further includes a computer program product, which includes a computer program that, when executed by a processor, implements the commercial bank knowledge graph construction and intelligent question answering method based on a large language model according to any embodiment of the present invention.

[0089] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0090] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0091] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for constructing a knowledge graph and intelligent question answering system for commercial banks based on a large language model, characterized in that, include: Obtain the dataset and perform entity extraction on the dataset based on the large language model to obtain the extraction results; The extraction results include: each entity and the relationships between entities; entity types include: customers, products, transactions, organizations, and events; relationship types include: owning an account, purchasing or holding a product, product type, affiliation, and the existence of a transaction; A commercial bank knowledge graph is generated based on the entities and the relationships between them. Obtain the input question, perform semantic understanding on the input question, and identify the query intent; Based on the query intent, the query is performed by matching the commercial bank knowledge graph to obtain the query results.

2. The method according to claim 1, characterized in that, A commercial bank knowledge graph is generated based on the entities and the relationships between them, including: Perform referential resolution operations on each entity to eliminate referential differences of the same entity in different contexts; Calculate the character similarity between any two entities, and replace the names of two entities with character similarity higher than the similarity threshold with the standard entity name; The credibility of the merged information is assessed, and a commercial bank knowledge graph is generated based on entities with credibility values ​​above the credibility threshold and the relationships between entities.

3. The method according to claim 1, characterized in that, Generate a knowledge graph of commercial banks, including: The commercial bank knowledge graph is stored using a graph database; Import knowledge data into the graph database, and insert triplet data using a third-party library; the triplet represents each entity and the relationship between entities. By creating entity nodes in the graph database using node classes and defining and connecting relationships between entities using relation classes, a knowledge graph of a commercial bank can be obtained.

4. The method according to claim 1, characterized in that, In the process of matching the commercial bank knowledge graph based on the query intent and performing a query to obtain the query results, the following operations are performed: Retrieval operation: The domain knowledge-intensive text content is taken as a knowledge document, and the knowledge document is divided into multiple knowledge document blocks; the input question and the knowledge document blocks are mapped into high-dimensional vectors using an embedding model to form text vectors and stored in an ordered vector knowledge base; the most semantically relevant document fragments to the query are retrieved from the vector knowledge base using vector search technology. Enhancement operation: Obtain the content of the most semantically relevant documents in the search results before sorting by a preset number, and fuse the retrieved knowledge with the input question through a weighted average strategy or a concatenation strategy to construct enhanced prompt words, which are then fed into the large language model as input context; Generation process: Using the enhanced prompts, the final text response is generated through a large language model.

5. The method according to claim 4, characterized in that, Also includes: fine-tuning operations: The cross-entropy loss function is used to measure the difference between the model-generated results and the true results, and the model parameters are optimized by minimizing the loss function; Select an optimizer and set hyperparameters based on model characteristics and data size; the hyperparameters include: learning rate and weight decay. The training data is divided into multiple mini-batches for forward and backward propagation to update the model parameters. An early stopping mechanism is set up so that training stops when the loss on the validation set does not decrease within a consecutive target number of batches, and the model parameters with the best performance are selected as the final result.

6. The method according to claim 5, characterized in that, This also includes: fine-tuning the evaluation process: Construct a test suite; the test suite data covers all key aspects of banking operations, including: counter services, credit management, financial accounting, customer management, and risk control and compliance. The accuracy, execution accuracy, and target score of the large language model are calculated based on the test set. The accuracy is calculated by comparing the number of perfect matches between the model and the real results on the test set. The execution accuracy is tested on the model's generated results using a real bank database to evaluate its feasibility in real-world scenarios. The target score measures the semantic similarity between the generated results and the real results. Fine-tuning evaluation is performed based on accuracy, execution accuracy, and target score.

7. A commercial bank knowledge graph construction and intelligent question-answering device based on a large language model, characterized in that, include: The acquisition and extraction module is used to acquire the dataset and perform entity extraction operations on the dataset based on the large language model to obtain the extraction results; The extraction results include: each entity and the relationships between entities; entity types include: customers, products, transactions, organizations, and events; relationship types include: owning an account, purchasing or holding a product, product type, affiliation, and the existence of a transaction; The generation module is used to generate a commercial bank knowledge graph based on each entity and the relationships between entities. The acquisition and recognition module is used to acquire the input question, perform semantic understanding on the input question, and identify the query intent; The matching and query module is used to match the commercial bank knowledge graph based on the query intent and perform a query to obtain the query results.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the commercial bank knowledge graph construction and intelligent question answering method based on a large language model as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the commercial bank knowledge graph construction and intelligent question answering method based on a large language model as described in any one of claims 1-6.

10. A computer program product, characterized in that, The method includes a computer program that, when executed by a processor, implements the commercial bank knowledge graph construction and intelligent question answering method based on a large language model according to any one of claims 1-6.

Images