Establishing method and device for enterprise credit information retrieval, retrieval method and device, electronic equipment and storage medium
By generating a knowledge graph through large language models and structured data injection, the problem of data fragmentation in enterprise credit information retrieval is solved, achieving efficient and accurate information retrieval while reducing costs and maintenance difficulties.
Patent Information
- Application Number
- CN202511050016.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-29
- Publication Date
- 2025-11-21
AI Technical Summary
Existing technologies for retrieving enterprise credit information suffer from problems such as high computational power consumption, weak information storage capacity, poor accuracy, low retrieval success rate, poor maintainability, and high application costs. In particular, the separation of structured and unstructured data leads to low retrieval efficiency.
This paper employs a large language model to extract relationships and covariates from enterprise credit information text, generates a knowledge graph through structured data injection, integrates unstructured and structured data, utilizes the Leiden algorithm to detect communities, generates community reports, and enables hierarchical queries to improve retrieval accuracy and efficiency.
It improves the accuracy of enterprise credit information retrieval, reduces the information retrieval costs for developers and users, simplifies the knowledge graph maintenance process, and enhances information carrying capacity and retrieval success rate.
Smart Images
Figure CN120994881A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of knowledge graph, and in particular to a building method and a retrieval method for enterprise credit information retrieval, a device, an electronic equipment and a storage medium. BACKGROUND
[0002] The credit status of an enterprise is influenced by many factors, including its own business situation, industry development situation, regional policy and economic situation, etc. When conducting enterprise credit analysis, analysts often need to spend a lot of time and effort to collect these related information from different channels, such as reading the prospectus published when issuing enterprise bonds, enterprise rating reports, enterprise annual reports published regularly, or checking enterprise financial data and economic data of enterprise operating areas in financial databases, etc.
[0003] In the industry, this data is generally processed separately from structured data. The text retrieval process consumes a lot of computing power, the returned information is not accurate, and the retrieval performance is not good. The development cost of structured data retrieval is high, and the returned information is not intuitive. SUMMARY
[0004] The embodiments of the present application provide a building method and a retrieval method for enterprise credit information retrieval, a device, an electronic equipment and a storage medium to build an enterprise credit information retrieval tool based on knowledge graph and large language model, fully integrating structured and non-structured data retrieval technology, and embedding knowledge graph as a retrieval enhancement tool RAG.
[0005] The embodiments of the present application adopt the following technical solutions:
[0006] In a first aspect, the embodiments of the present application provide a building method for enterprise credit information retrieval, wherein the building method comprises:
[0007] According to the large language model, the relationship and the covariant in the enterprise credit information text information are extracted; and
[0008] In a first aspect, the embodiments of the present application provide a building method for enterprise credit information retrieval, wherein the building method comprises:
[0009] In some embodiments, the extracting the relationship and the covariant in the enterprise credit information text information according to the large language model comprises
[0010] According to the large language model, the relationship and the covariant in the enterprise credit information text information are extracted; and
[0011] In some embodiments, the method of injecting structured data, in combination with the relationship and the covariate, generates a graph and community of enterprise credit information in response to a search request for enterprise credit information, comprising:
[0012] The method of injecting structured data annotates and describes entities, relationships, and covariates involved in structured data;
[0013] According to the large language model, it is determined whether the entity, the relationship, and the covariate exist;
[0014] If they exist, the existing relationships and covariates are checked and perfected. If they do not exist, new nodes or edges are added to obtain edges between each two nodes with a predetermined relationship. Each edge describes different events in two nodes with covariates, and the number of covariates defines the weight of the edge.
[0015] In some embodiments, the method further comprises:
[0016] Fusing unstructured data and structured data when generating the knowledge graph, and introducing the covariate as an auxiliary event record of node relationship when generating the graph.
[0017] In some embodiments, the method of injecting structured data, in combination with the relationship and the covariate, generates a graph and community of enterprise credit information in response to a search request for enterprise credit information, comprising:
[0018] Community detection is performed on the knowledge graph nodes and edges by the Leiden algorithm to generate a community;
[0019] After the community is generated, the community is traversed and summarized according to the large language model to generate a community report.
[0020] In some embodiments, the method further comprises:
[0021] Converting the unstructured text file into a pure text document;
[0022] Using a large language model to divide and slice the pure text document to obtain a divided text block;
[0023] Using the large language model to read the divided text block to extract entity names and related descriptions therefrom;
[0024] Similarity matching is performed on the existing entity names to find the highest similarity existing entity.
[0025] In a second aspect, the embodiments of the present application also provide a search method, wherein the search method comprises the method for building enterprise credit information search in the first aspect.
[0026] vectorizing the text to be searched, finding the closest multiple entities;
[0027] filtering out relevant entities among the multiple models through a large language model, and finding edges between entities after selecting entities;
[0028] querying direct relationship descriptions and covariant descriptions as first-level query retrieval;
[0029] querying indirect relationships and covariant descriptions as second-level query retrieval;
[0030] querying all relationships involving entities as third-level query retrieval;
[0031] querying all relationships within a community as fourth-level query retrieval;
[0032] global retrieval as fifth-level query retrieval;
[0033] wherein the third-level query retrieval to the fifth-level query retrieval does not do full query but does topK query.
[0034] In a third aspect, the embodiments of the present application also provide a building device for enterprise credit information retrieval, wherein the building device comprises:
[0035] an extraction module configured to extract relationships and covariants in enterprise credit information text information according to a large language model; and
[0036] a generation module configured to generate a graph and a community of enterprise credit information by using a structured data injection method in combination with the relationships and the covariants, in response to a retrieval request of enterprise credit information.
[0037] In a fourth aspect, the embodiments of the present application also provide an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, which when executed cause the processor to perform the above method.
[0038] In a fifth aspect, the embodiments of the present application also provide a computer readable storage medium storing one or more programs, which when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the above method.
[0039] The at least one technical solution adopted by the embodiment of the application can achieve the following beneficial effects: according to a large language model, relationship and covariates in enterprise credit information text information are extracted; then, in a structured data injection manner, a graph and a community of enterprise credit information are generated in combination with the relationship and the covariates, to respond to a search request of enterprise credit information. Through the search tool obtained by the building method for enterprise credit information search of the application, the information required for search can be accurately returned according to natural language instructions of a user, the accuracy of search is improved, and the development cost and the information search cost of the user are reduced. BRIEF DESCRIPTION OF DRAWINGS
[0040] The accompanying drawings, which are included to provide a further understanding of the application and constitute a part of this application, illustrate certain illustrative embodiments of the application and together with the description serve to explain the application. In the drawings:
[0041] Figure 1 A schematic diagram of the principle of the building method for enterprise credit information search in the embodiment of the application;
[0042] Figure 2 A schematic diagram of the flow of the building method for enterprise credit information search in the embodiment of the application;
[0043] Figure 3 A schematic diagram of the knowledge graph generated by the building method for enterprise credit information search in the embodiment of the application;
[0044] Figure 4 A schematic diagram of the information search flow of the building method for enterprise credit information search in the embodiment of the application;
[0045] Figure 5 A schematic diagram of the entity extraction flow of the building method for enterprise credit information search in the embodiment of the application;
[0046] Figure 6 A schematic diagram of the relationship and covariates extraction flow of the building method for enterprise credit information search in the embodiment of the application;
[0047] Figure 7 A schematic diagram of the structured data injection flow of the building method for enterprise credit information search in the embodiment of the application;
[0048] Figure 8 A schematic diagram of the structure of the building method for enterprise credit information search in the embodiment of the application;
[0049] Figure 9 A schematic diagram of the structure of the electronic device in the embodiment of the application. DETAILED DESCRIPTION
[0050] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in connection with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.
[0051] The technical terms involved in the embodiments of the present application are as follows:
[0052] Enterprise credit: refers to the performance ability of an enterprise, especially in the performance ability of debt in business activities such as borrowing, selling on credit, and advance payment. Enterprise credit risk refers to the risk that an enterprise cannot perform normally.
[0053] Knowledge graph (Knowledge Graph), is a series of various different graphs that show the development process and structural relationship of knowledge, which uses visualization technology to describe knowledge resources and their carriers, and mines, analyzes, constructs, draws and displays knowledge and their mutual relationships.
[0054] Large language model (Large Language Model, abbreviated as large model or LLM), refers to a deep learning model trained using a large amount of text data, which can generate natural language text or understand the meaning of language text.
[0055] NLP (Natural Language Processing): a discipline that uses computer technology to analyze, understand and process natural language, that is, uses computers as powerful tools for language research, and provides language descriptions that can be used by people and computers together under the support of computers.
[0056] GPT: (Generative Pre-Trained Transformer), a large language model algorithm.
[0057] RAG: (Retrieval-augmented Generation), a model that combines retrieval and generation techniques. It generates answers or content by referencing information from an external knowledge base, has strong explainability and customization capabilities, and is suitable for multiple natural language processing tasks such as question and answer systems, document generation, and intelligent assistants.
[0058] GraphRAG: (Graph based Retrieval-augmented Generation), knowledge graph-based retrieval-enhanced generation, is a text retrieval technology that combines large language models, knowledge graphs and retrieval algorithms.
[0059] Vectorization: A technique for converting text into vectors based on its semantics.
[0060] Cosine similarity: a method to measure the similarity between two vectors. The closer to 1, the higher the similarity; the closer to 0, the lower the similarity. In natural language, it is generally not less than 0.
[0061] Node: refers to a concept in a knowledge graph, generally a type of entity.
[0062] Edge: A concept in knowledge graphs, generally referring to the relationship between entities.
[0063] Covariate: A variable constructed in the embodiments of this application to record events that occur between entities.
[0064] Markdown: A text storage format that is easily understood by large language models.
[0065] Community: A concept in knowledge graphs, referring to an organization of entities that potentially have specific relationships. If some entities are closely related, then these entities may belong to the same community.
[0066] Leiden algorithm: A community detection algorithm that can automatically generate communities based on elements such as nodes, edges, and weights in a knowledge graph.
[0067] topK query: When querying, the samples and the query statement are sorted by similarity, and the K most similar samples are extracted.
[0068] Current enterprise credit information retrieval tools have the following shortcomings:
[0069] (1) Ordinary financial database retrieval methods can only process structured data. Unstructured data in enterprise-related manuals and reports still need to be read and retrieved manually, which is time-consuming and labor-intensive.
[0070] (2) For the scheme of simply reading various unstructured text data by a large model, the general scheme is to first perform vectorization processing on document slices, then perform vectorization processing on query statements, find the n document slices with the maximum cosine similarity of two vectors, and the large language model tries to obtain the required information by reading the n slices. This method has the following disadvantages: first, the information density of the original document slice without information extraction is low, which requires a large amount of computing power to retrieve the target information; second, the retrieval statement of natural language also has information redundancy, and the vectors formed by the redundant information of the retrieval statement and the redundant information in the document slice may have similarity, which makes the size of the vector cosine value dominated by the redundant information, making it difficult to extract effective information from the original document, and ultimately resulting in low retrieval accuracy and success rate; third, the slice document does not construct the potential relationship between documents, and the retrieved documents may ignore important intermediate variables, resulting in inability to perform multi-hop retrieval.
[0071] (3) For the existing large model and knowledge graph combination scheme, the main method is to construct a data knowledge graph through a structured database, or to extract triples through a large model to construct entities and relationships, thereby forming an entity knowledge graph with entities as nodes and entity relationships as edges, or generating them separately, and in the retrieval process, the nodes are retrieved and fused according to the preset rules, and all information related to the nodes is extracted, and then the method in (2) is used for slicing and similarity sorting. The above scheme has the following disadvantages: first, the knowledge graph formed by structured data and the knowledge graph formed by unstructured documents are not truly formed into one graph, but are two separate graphs that need to be fused with specific rules, and the graph and the rule are strongly coupled, which is difficult to maintain and consumes a large amount of computing power in the retrieval process; second, the knowledge graph composed of nodes and edges has disadvantages in information storage, as there is usually only one edge between nodes, making it difficult to record a large number of events between nodes; third, too much information is extracted from the knowledge graph in the retrieval process, and these information has to be processed again by method (2), so there is no way to avoid the disadvantages of scheme (2).
[0072] In summary, the related technical solutions have the problems of large computing power consumption, weak information storage capacity, poor accuracy, low retrieval success rate, poor maintainability, and high application cost.
[0073] To address the above shortcomings, the method for building an enterprise credit information retrieval tool based on a knowledge graph and a large language model is provided in the embodiments of the present application, which fully integrates structured and unstructured data retrieval technologies, embeds a knowledge graph as a retrieval enhancement tool, and accurately returns the required query information according to the user's natural language instructions, thereby improving the accuracy of retrieval and reducing the development cost and information retrieval cost of the user.
[0074] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0075] As shown in Figure 1 , specifically includes the steps of unstructured document parsing, document slicing, entity extraction, relationship and covariant extraction, structured data injection, graph generation, community generation, and information retrieval.
[0076] Among them, the document is sliced in the document slicing process, and the large language model is used to indicate the unknown elements in the sliced content according to the chapter title and the previous text in the document, so as to facilitate the extraction of entities and relationships, solve the problems of unknown entity elements and unknown relationship references in the process of extracting knowledge graph by the large language model, and improve the accuracy and information density of the knowledge graph.
[0077] Among them, in the process of entity extraction and relationship extraction, similarity query and homogeneity verification are performed on the existing entities to ensure that the same entity will not generate multiple nodes and the same relationship will not generate multiple edges, avoiding the problem of homogenization nodes and edges in the process of extracting knowledge graph by the large language model, reducing the redundancy of the graph, and reducing the computing power requirement in the query process.
[0078] Among them, the structured data is directly injected into the knowledge graph after being recognized by the large language model, fully integrating structured and unstructured data, and correcting the possible data inaccuracy of unstructured data, further improving the accuracy of the knowledge graph. The fusion graph scheme is easier to maintain than the two graph schemes, reducing the development and maintenance cost of the knowledge graph.
[0079] The embodiment of the application provides a building method for enterprise credit information retrieval, as shown in Figure 2 , a building method flow diagram for enterprise credit information retrieval in the embodiment of the application is provided, and the method at least includes the following steps S210 to S220:
[0080] Step S210, according to the large language model, extracting the relationship and covariant in the enterprise credit information text information.
[0081] The large language model is used to process the enterprise credit information text information, and the relationship and covariant are extracted. It can be understood that the covariant is a variable constructed in the embodiment of the application, which is used to record the events between entities. The relationship is the event between two entities.
[0082] Specifically, since the relationship between two entities is often stated in the text, the inherent relationship which is not limited by time, place and other factors can be recorded as the edge of the knowledge graph, and the specific event occurring at a specific time or place can be recorded as the covariant of the edge, which is used for auxiliary description of the edge.
[0083] Step S220, in response to the retrieval request of enterprise credit information, the graph and community of enterprise credit information are generated by using the structured data injection method combined with the relationship and the covariant.
[0084] The structured data is injected into the knowledge graph, so that the structured data and unstructured data are fused into a knowledge graph. First, the entities, relationships and covariants involved in the structured data are annotated and described, and then the large language model checks whether the entities and relationships exist. If they exist, no operation is performed, otherwise a node or edge is added. Finally, the covariant is queried. Since the accuracy of structured data is often higher than that of unstructured extraction, if the covariant already exists, the existing covariant should be corrected according to the structured data, and the unstructured data source should be marked as unmodifiable. If not, it is directly added.
[0085] As shown in Figure 3 , the generated knowledge graph example is shown in the figure. Each node has a definition and description, and there is an edge between two nodes with a given relationship, and each edge describes different events in two nodes with covariants. The number of covariants defines the weight of the edge.
[0086] The relationship / covariant is: relationship 1, and the corresponding description is: xx Binjiang Co., Ltd. and xx City Construction Co., Ltd. are upstream and downstream companies, and xx Binjiang Co., Ltd. purchases services from xx City Construction Co., Ltd.
[0087] The relationship / covariant is: covariant 1.1, and the corresponding description is: in 2023, xx City Construction Co., Ltd. has 2.2 billion yuan of other receivables from xx Binjiang Co., Ltd.
[0088] The relationship / covariant is: relationship 2, and the corresponding description is: the other receivables subject is one of the asset indicators in the balance sheet of xx Binjiang Co., Ltd.
[0089] The relationship / covariant is: covariant 2.1, and the corresponding description is: in 2022, the balance of other receivables of xx Binjiang Co., Ltd. is 10 billion yuan.
[0090] The relationship / covariant is: covariant 2.2, and the corresponding description is: in 2023, the balance of other receivables of xx Binjiang Co., Ltd. is 20 billion yuan.
[0091] Through the above method, according to the large language model, the relationship and covariant in the enterprise credit information text information are extracted, and the unstructured data and structured data are fused when generating the knowledge graph, which further improves the accuracy of the knowledge graph and reduces the maintenance cost of multiple graphs.
[0092] By the above method, the graph and community of enterprise credit information are generated by using the structured data injection method in combination with the relationship and the covariant to respond to the retrieval request of enterprise credit information, the covariant is introduced as an auxiliary event record of node relationship in the generation of the knowledge graph, and the information carrying capacity of the knowledge graph is greatly improved without improving the complexity of the knowledge graph.
[0093] Unlike the related art, the covariant is used as an intermediate variable for generating an edge, the covariant is used as part of the knowledge graph edge by the above method, and the time, place and other elements of event occurrence are explicitly recorded, which greatly enhances the information carrying capacity of the knowledge graph, and does not significantly increase the complexity of the graph, making the graph easy to understand and convenient to query and retrieve.
[0094] In an embodiment of the present application, the relationship and covariant in the enterprise credit information text information are extracted according to the large language model, including: reading the preset text block according to the large language model, recording the inherent relationship as the edge of the knowledge graph, and recording the specific event as the covariant of the edge, and extracting the relationship and covariant in the enterprise credit information text information.
[0095] It can be understood that the "preset text block" is the text block content obtained after document slicing, entity extraction and relationship and covariant extraction. The inherent relationship is recorded as the edge of the knowledge graph, and the specific event is recorded as the covariant of the edge according to the large language model reading the preset text block, and then the relationship and covariant in the enterprise credit information text information are extracted.
[0096] In an embodiment of the present application, the graph and community of enterprise credit information are generated by using the structured data injection method in combination with the relationship and the covariant to respond to the retrieval request of enterprise credit information, including: using the structured data injection method, the entities, relationships and covariants involved in the structured data are labeled and described, and the large language model is used to determine whether the entities, relationships and covariants exist; if they exist, the existing relationship and covariant are checked and improved, and if they do not exist, a node or an edge is added to obtain an edge between each two nodes with a predetermined relationship, and each edge describes different events in two nodes with covariants, and the number of covariants defines the weight of the edge.
[0097] As Figure 6As shown, the large model reads the text block to generate the relationship between entities and the covariant. Generally speaking, when the text states the relationship between two entities, it often states the event between the two entities at the same time. We record the inherent relationship without time, place, etc. as the edge of the knowledge graph, and record the specific event at a specific time or place as the covariant of the edge, which is used for auxiliary description of the edge. The large model reads the given text block, extracts the description of the event between the entities in the text block, called covariant, and searches the existing knowledge graph to see if the covariant exists. If there is no covariant, a new covariant is added, otherwise the large model reads the description of the existing covariant and the description of the newly discovered covariant. If there is a difference, the difference is supplemented, and if there is no substantial difference, the process is stopped. After the large model identifies the covariant in the text block, the edge involved is queried, and the stable relationship between the two entities that is not limited by a specific time is summarized as the edge between the nodes according to the newly added or modified covariant information and the existing covariant information, and the covariant is associated with the edge.
[0098] As shown in Figure 7 , the structured data is injected into the knowledge graph to integrate structured data and unstructured data into one knowledge graph. First, the entities, relationships and covariants involved in the structured data are annotated and described, and then the large model checks whether these entities and relationships exist. If they exist, no operation is performed, otherwise new nodes or edges are added. Finally, the covariant is queried. Since the accuracy of structured data is often higher than that of unstructured extraction, if the covariant already exists, the structured data should be used to modify the existing covariant, and the unstructured data source should be marked as unmodifiable. If it does not exist, it is directly added.
[0099] In an embodiment of the present application, the method further comprises: fusing unstructured data and structured data when generating the knowledge graph, and introducing the covariant as an auxiliary event record of node relationship when generating the graph.
[0100] The covariant is introduced as an auxiliary event record of node relationship when generating the graph, which greatly improves the information carrying capacity of the graph without increasing the complexity of the graph.
[0101] In an embodiment of the present application, the structured data injection method is used to combine the relationship and the covariant to generate a graph and a community of enterprise credit information in response to a search request for enterprise credit information: the Leiden algorithm is used to detect communities of nodes and edges of the knowledge graph to generate communities; after the community is generated, the large language model is used to traverse and summarize the community to generate a community report.
[0102] The Leiden algorithm is used to detect communities of nodes and edges in the knowledge graph. The Leiden algorithm calculates the relationship between nodes and edges in communities, ensures the effectiveness of internal connections, and achieves the separation of communities. When the Leiden algorithm is applied iteratively, it converges to a partition in which all subsets of all communities are optimally allocated locally, resulting in communities that guarantee connectivity.
[0103] It can be understood that the Leiden algorithm includes three stages: 1. Node local movement, quickly find the partition 2. Partition refinement 3. Network aggregation based on refined partitions, use non-refined partitions to create initial partitions for the aggregated network. Iterative steps until convergence. After the community is generated, a large model can be used to traverse the summary of the community, and a community report is generated for global retrieval in the information retrieval step.
[0104] In an embodiment of the present application, the method further comprises: converting the unstructured text file into a pure text document; using a large language model to slice the pure text document, obtaining sliced text blocks; using the large language model to read the sliced text blocks and extract entity names and related descriptions; similarity matching of existing entity names and finding the highest similarity existing entity.
[0105] Converting the unstructured text file into a pure text document mainly converts the PDF format unstructured document such as enterprise bond prospectus and enterprise rating report into a markdown format pure text document, which is convenient for large language model reading and understanding. First, restore it to a doc document through layout restoration technology, or if it is already a doc document, this step can be omitted. According to the title serial number and font elements, mark the markdown format hierarchical title, which is convenient for the subsequent document slicing.
[0106] Using a large language model to slice the pure text document, obtaining sliced text blocks, specifically slicing the markdown format text into 600-1200 word small text blocks. First, slice according to the markdown title chapter, and retain chapter level information. If a minimum level chapter exceeds the maximum number of text blocks, slice by paragraph, and if it still exceeds the number of words, slice by word number. Since part of the context is missing after document slicing, the large model needs to provide explanatory descriptions for the text blocks based on the retained chapter titles and previous text to ensure that all references in the text blocks can find specific entities. References such as "the company" and "controlling shareholder" should be specifically indicated.
[0107] As Figure 5As shown, the existing entity name is matched for similarity, and the existing entity with the highest similarity is found. The large model reads the segmented text block, extracts the entity name and related description, matches the existing entity name for similarity, and finds the existing entity with the highest similarity. If the matching degree is higher than the threshold, the large model is used to check whether the entity descriptions of the two are consistent, which is used to judge whether the two entities are the same entity; if the matching degree of the two is lower than the threshold, the similarity matching is performed according to the entity description, the entity with the highest description similarity is found, and the large model is used to check whether the descriptions of the two are consistent. By checking the entity description, the large model judges whether the entity should be merged into the existing entity, if it should be merged, the difference part of the entity description is merged, if it is not merged, a new entity is added, and the entity text and description text are vectorized, ready for similarity calculation with other new entities in the foregoing process. If a new entity is added, a node in the knowledge graph is added.
[0108] The application embodiment also provides a retrieval method, wherein the retrieval method comprises the following steps of:
[0109] Vectorizing the to-be-retrieved text to find the closest multiple entities;
[0110] Filtering out related entities in the multiple models through a large language model, and finding edges between entities after selecting the entities;
[0111] Querying direct relationship descriptions and covariant descriptions as first-level query retrieval; querying indirect relationships and covariant descriptions as second-level query retrieval; querying all relationships related to entities as third-level query retrieval;
[0112] Querying all relationships in the community as fourth-level query retrieval; querying globally as fifth-level query retrieval;
[0113] Among them, the third-level query retrieval to the fifth-level query retrieval does not make full query but topK query.
[0114] In the graph retrieval process, a hierarchical query scheme is adopted, first, the direct relationship between entities is queried, second, the indirect relationship is queried, third, all relationships of the entity are queried, fourth, all relationships in the community are queried, and finally, the global relationship is queried. By balancing the details and the overall problem answer, for the application scenario of enterprise credit analysis, the first-level and second-level retrieval are mainly concentrated, which can greatly save computing power and improve the accuracy and success rate of retrieval.
[0115] As Figure 4As shown, the retrieval process takes the form of hierarchical retrieval. First, the search text is vectorized, and then the cosine similarity of the vectors of entity names and descriptions is calculated to find the n closest entities. The large language model is used to filter out the accurate and relevant entities from the n models. After selecting the entities, the edges between the entities can be quickly found. The direct relationship description and covariant description are queried as the first-level retrieval, the indirect relationship and covariant description are queried as the second-level retrieval, all the relationships involved in the entity are queried as the third-level query, all the relationship queries in the community are queried as the fourth-level query, and the global retrieval is the fifth-level query. Starting from the third-level query, no full query is performed, but a topK query is performed. After querying the relevant information, the result may not be returned directly, and other tools such as calculators and statistical algorithms may be called for processing. The large model calls the processing tools according to the required processing method and tool interface description, and finally returns the processed answer to the retrieval party.
[0116] The embodiment of the present application also provides a device 800 for building enterprise credit information retrieval, as shown in the following figure: Figure 8 As shown, a structural schematic diagram of the device for building enterprise credit information retrieval in the embodiment of the present application is provided, and the device 800 at least includes an extraction module 810 and a generation module 820, wherein:
[0117] In an embodiment of the present application, the extraction module 810 is specifically configured to extract the relationship and the covariant variable in the enterprise credit information text information according to the large language model.
[0118] The large language model is used to process the enterprise credit information text information, and the relationship and the covariant variable are extracted therefrom. It can be understood that the covariant variable is a variable constructed in the embodiment of the present application, and is used to record the events occurring between entities. The relationship is the event between two entities.
[0119] Specifically, since the relationship between two entities is often stated in the text, the inherent relationship which is not limited by time, place and other factors can be recorded as the edge of the knowledge graph, and the specific event occurring at a specific time or place is recorded as the covariant variable of the edge, which is used for auxiliary description of the edge.
[0120] In an embodiment of the present application, the generation module 820 is specifically configured to generate the graph and the community of the enterprise credit information by using the structured data injection method in combination with the relationship and the covariant variable, so as to respond to the retrieval request of the enterprise credit information.
[0121] The structured data is used to inject the knowledge graph, so that the structured data and the unstructured data are fused into a knowledge graph. First, the entities, relationships and covariants involved in the structured data are annotated and described, then the large language model checks whether the entities and relationships exist, if they exist, no operation is performed, otherwise a node or an edge is added. Finally, the covariants are queried. Since the accuracy of the structured data is usually higher than that of the unstructured extraction, if the covariant exists, the existing covariant should be corrected according to the structured data, and the unstructured data source is annotated as unmodifiable, if not, it is directly added.
[0122] As shown in Figure 3 The generated knowledge graph example is shown in the figure. Each node has a definition and description, and there is an edge between two nodes with a given relationship, and each edge describes different events in two nodes with covariants. The number of covariants defines the weight of the edge.
[0123] In an embodiment of the present application, the extraction module 810 is further used to:
[0124] According to the large language model reading the preset text block, the inherent relationship is recorded as the edge of the knowledge graph, and the specific event is recorded as the covariant of the edge, and the relationship and the covariant in the enterprise credit information text information are extracted.
[0125] In an embodiment of the present application, the generation module 820 is further used to:
[0126] The entities, relationships and covariants involved in the structured data are annotated and described by using the structured data injection method,
[0127] According to the large language model, it is judged whether the entity, the relationship and the covariant exist;
[0128] If they exist, a node or an edge is added to obtain an edge between each two nodes with a given relationship, and each edge describes different events in two nodes with covariants, and the number of covariants defines the weight of the edge.
[0129] In an embodiment of the present application, it further includes: a structured data injection module, used to:
[0130] The unstructured data and the structured data are fused when the knowledge graph is generated, and the covariant is introduced as an auxiliary event record of the node relationship when the graph is generated.
[0131] In an embodiment of the present application, the generation module 820 is further used to:
[0132] The knowledge graph nodes and edges are detected by the Leiden algorithm to generate a community;
[0133] After the community is generated, a community report is generated by traversing and summarizing the community according to a large language model.
[0134] In an embodiment of the present application, the method further comprises: a preprocessing module configured to
[0135] Converting the unstructured text file into a pure text document;
[0136] Using a large language model to slice the pure text document, obtaining a sliced text block;
[0137] Using the large language model to read the sliced text block and extract entity names and related descriptions therefrom;
[0138] Performing similarity matching on the existing entity names and finding the highest similarity existing entity.
[0139] It can be understood that the above-described building device for enterprise credit information retrieval can realize each step of the building method for enterprise credit information retrieval provided in the foregoing embodiments, and the related explanations about the building method for enterprise credit information retrieval are all applicable to the building device for enterprise credit information retrieval, which will not be repeated here.
[0140] Figure 9 is a structural schematic diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 9 At the hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The memory can include a memory such as a high-speed random access memory (RAM), and can also include a non-volatile memory such as at least one disk memory. Of course, the electronic device can also include other hardware required by the business.
[0141] The processor, network interface, and memory can be connected to each other through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 In the figure, only one bidirectional arrow is used to represent, but it does not mean that there is only one bus or one type of bus.
[0142] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.
[0143] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a logical framework for retrieving enterprise credit information. The processor executes the program stored in memory and specifically performs the following operations:
[0144] Based on the large language model, relationships and covariates are extracted from the textual information of enterprise credit information; and
[0145] By employing structured data injection, and combining the aforementioned relationships and covariates, a graph and community of enterprise credit information are generated to respond to enterprise credit information retrieval requests.
[0146] The above is as stated in this application. Figure 2 The method executed by the device for building enterprise credit information retrieval disclosed in the illustrated embodiment can be applied to a processor, or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.
[0147] The electronic device can also perform Figure 2The method executed by the building device for enterprise credit information retrieval is used for the building device for enterprise credit information retrieval in Figure 2 The functions of the building device for enterprise credit information retrieval in the embodiment are not repeated here.
[0148] The embodiment of the present application also provides a computer readable storage medium, which stores one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device comprising a plurality of application programs, can enable the electronic device to perform the method executed by the building device for enterprise credit information retrieval in the embodiment of the building device for enterprise credit information retrieval. Figure 1 The method executed by the building device for enterprise credit information retrieval in the embodiment of the building device for enterprise credit information retrieval is used for executing the following steps in particular:
[0149] According to the large language model, relationship and covariate in the enterprise credit information text information are extracted; and
[0150] In response to the retrieval request of the enterprise credit information, the graph and the community of the enterprise credit information are generated by adopting the structured data injection mode, in combination with the relationship and the covariate.
[0151] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer usable program codes.
[0152] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The device for realizing the functions specified in one flow or multiple flows and / or blocks.
[0153] These computer program instructions can also be stored in a computer readable memory capable of guiding the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable memory produce a product comprising instruction devices, which realize the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1the function(s) specified in the block or blocks.
[0154] These computer program instructions can also be loaded into computer or other programmable data processing devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable devices provide steps for implementing the flowchart block(s) or flowchart flow(s) and / or portions thereof. Figure 1 the flowchart block(s) or flowchart flow(s) and / or portions thereof. the function(s) specified in the block or blocks.
[0155] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.
[0156] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information about an operating system, application software, and / or the like. Memory is an example of computer readable media.
[0157] Computer readable media includes permanent and non-permanent, moveable and non- moveable media that can be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that is accessible to a computing device. According to the definition provided herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.
[0158] It should also be noted that the terms "comprising," "including," and any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.
[0159] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code thereon for use by or in connection with an instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
[0160] The embodiments of the present application described above are merely intended to illustrate the present application, and are not intended to limit the present application. Various modifications and changes can be made by those skilled in the art to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A method for constructing a system for retrieving enterprise credit information, wherein, The construction method includes: Based on the large language model, relationships and covariates are extracted from the textual information of enterprise credit information; and By employing structured data injection, and combining the aforementioned relationships and covariates, a graph and community of enterprise credit information are generated to respond to enterprise credit information retrieval requests.
2. The method as described in claim 1, wherein, The step of extracting relations and covariates from enterprise credit information text based on a large language model includes: Based on the large language model, a preset text block is read, and the inherent relationships are recorded as edges of the knowledge graph, and specific events are recorded as covariates of the edges. The relationships and covariates in the enterprise credit information text are then extracted.
3. The method as described in claim 2, wherein, The method employs structured data injection, combining the relationships and covariates to generate a graph and community of enterprise credit information in response to enterprise credit information retrieval requests, including: The structured data injection method is used to annotate and describe the entities, relationships, and covariates involved in the structured data; Determine whether the entity, the relation, and the covariate exist based on the large language model; If they exist, the existing relationships and covariates are validated and improved. If they do not exist, new nodes or edges are added to obtain edges between every two nodes with a predetermined relationship. Each edge uses covariates to describe different events in the two nodes, and the number of covariates defines the weight of the edge.
4. The method as described in claim 2, wherein, The method further includes: When generating the knowledge graph, unstructured and structured data are integrated, and the covariates are introduced as auxiliary event records for node relationships during graph generation.
5. The method as described in claim 2, wherein, By employing structured data injection, and combining the aforementioned relationships and covariates, a graph and community of enterprise credit information are generated to respond to enterprise credit information retrieval requests, including: The Leiden algorithm is used to perform community detection on the nodes and edges of the knowledge graph to generate communities. After the community is generated, it is traversed and summarized according to the large language model to generate a community report.
6. The method of claim 1, wherein, The method further includes: Convert unstructured document parsing files into plain text documents; The plain text document is segmented into hierarchical segments using a large language model to obtain segmented text blocks; The large language model is used to read the segmented text blocks and extract the entity names and related descriptions. Perform similarity matching on existing entity names and find the existing entity with the highest similarity.
7. A retrieval method, wherein, The method for constructing enterprise credit information retrieval as described in any one of claims 1 to 6, wherein the retrieval method includes: The text to be searched is vectorized to find the most similar entities; The large language model is used to filter out relevant entities from the multiple models and find the edges between entities after the entities are selected. The query retrieves direct relationship descriptions and covariate descriptions as a first-level query. The query retrieves indirect relationships and covariate descriptions as secondary queries. All relationships involved in the query entity are retrieved as a third-level query. All relationship queries within the community are treated as level four queries. Global search is used as a fifth-level query retrieval method; Specifically, the third-level query retrieves the results of the fifth-level query without performing a full query, but instead performs a top-K query.
8. A device for constructing a system for retrieving enterprise credit information, wherein, The assembly device includes: The extraction module is used to extract relations and covariates from enterprise credit information text based on a large language model; and The generation module is used to generate a graph and community of enterprise credit information by using structured data injection, combining the relationships and covariates, in response to enterprise credit information retrieval requests.
9. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 6, and / or the method of claim 7.
10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 6, and / or the method of claim 7.