Atlas construction method and system based on neural network and sorting mechanism

Abstract

The invention provides a graph construction method and system based on a neural network and a sorting mechanism, and the method comprises the steps: enabling a preset large language model to generate a knowledge graph frame through a role instruction, and obtaining the language knowledge from the knowledge graph frame based on an autoregression language model, the knowledge graph framework is filled with language knowledge to form an initial knowledge graph, language variation and implicit relations are displayed, and then a preset double-tower structure is adopted for external knowledge expansion to obtain external knowledge matched with the initial knowledge graph, so that the universality of the knowledge graph is improved. And performing self-sorting on the external knowledge and the language knowledge through an unsupervised paragraph sorting mechanism to generate a target knowledge graph, thereby improving comprehensiveness and accuracy of the knowledge graph.

Description

Technical Field

[0001] The present invention relates to the technical field of graph construction, and more specifically, to a method and system for constructing a knowledge graph based on a neural network and a sorting mechanism. Background Art

[0002] As a structured knowledge representation method, knowledge graph effectively organizes and presents information through a network of nodes and edges, and has become an important tool for modern data analysis and intelligent applications. It not only plays a key role in search engines, recommendation systems, natural language processing and other fields, but also provides strong support for knowledge management and decision support. However, despite the increasing importance of knowledge graphs, existing construction methods still face many challenges.

[0003] Traditional knowledge graph construction methods mostly rely on manual annotation and rule-driven extraction techniques. Although these methods have achieved certain success in some applications, they often expose problems of low efficiency and insufficient accuracy when processing large-scale data. Manual annotation is not only time-consuming and labor-intensive, but also easily affected by subjective factors, resulting in incomplete or biased information in the knowledge graph. At the same time, rule-driven methods have difficulty dealing with complex language variations and implicit relationships, and often fail to capture the deep semantics and dynamic knowledge in the text.

[0004] Therefore, there is an urgent need for a graph construction method and system that can deeply capture information, overcome bias, and improve the scalability of knowledge graphs. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a graph construction method and system based on a neural network and a sorting mechanism to solve the technical problems that the existing technology is either easily affected by subjective factors, resulting in incomplete or biased information in the knowledge graph, or is difficult to cope with complex language variations and implicit relationships, and often fails to capture the deep semantics and dynamic knowledge in the text.

[0006] The present invention provides a graph construction method based on a neural network and a sorting mechanism, which includes:

[0007] Through role instructions, the preset large language model generates a knowledge graph framework;

[0008] Acquire language knowledge about the knowledge graph framework based on an autoregressive language model, and use the language knowledge to fill the knowledge graph framework to form an initial knowledge graph;

[0009] Using a preset double-tower structure to expand external knowledge to obtain external knowledge matching the initial knowledge graph;

[0010] The external knowledge and the language knowledge are self-ranked through an unsupervised paragraph ranking mechanism to generate a target knowledge graph.

[0011] Optionally, the generating a knowledge graph framework from a preset large language model through role instructions includes:

[0012] Receiving role instructions on ideal knowledge graph generation through the large language model;

[0013] Parsing the role instruction to obtain key elements and constraint conditions based on a preset conditional constraint template according to the role instruction;

[0014] Sorting out the relationships of the key elements based on the constraint conditions to form element relationships;

[0015] The key elements are used as nodes and the element relationships are used as edges to establish a graph to form a knowledge graph framework.

[0016] Optionally, the key elements include task definition, task purpose, task problem, relevant roles, time node, implementation location, task effect and implementation method;

[0017] The constraints include object constraints, motivation constraints, role constraints, time constraints, space constraints, quantitative constraints and method constraints.

[0018] Optionally, acquiring language knowledge from the knowledge graph framework based on the autoregressive language model, and filling the knowledge graph framework with the language knowledge to form an initial knowledge graph, includes:

[0019] Retrieving context information of the role instruction using a pre-fine-tuned autoregressive language model to obtain valid information, key element text, and element relationship text as language knowledge;

[0020] Establishing a mapping of language points corresponding to the key element text and language relationships corresponding to the element relationship text according to the language knowledge;

[0021] Based on the valid information, the language point is matched with the node on the knowledge graph framework, and the language knowledge that has a mapping relationship with the language point is filled into the node corresponding to the language point, and the element relationship text is matched with the edge of the knowledge graph framework; and if there is no node corresponding to the language point, a corresponding supplementary node is added to the knowledge graph framework, and the key element text corresponding to the supplementary node is filled into the supplementary node, and at the same time, a supplementary edge related to the supplementary node is added based on the valid information, and the supplementary edge is described with the element relationship text related to the supplementary node to form a filled graph;

[0022] The filled graph is optimized to form an initial knowledge graph; the optimization includes merging redundant nodes and correcting edges with unclear descriptions.

[0023] Optionally, the dual tower structure is connected to an external knowledge source;

[0024] The dual-tower structure includes a question encoder, a knowledge encoder and a retrieval component, wherein the knowledge encoder is used to convert the external knowledge source into an external source vector.

[0025] Optionally, the adopting a preset double-tower structure to perform external knowledge expansion to obtain external knowledge matching the initial knowledge graph includes:

[0026] Encoding the key element text into a dense element vector by the question encoder;

[0027] The retrieval component retrieves relevant external texts in the external knowledge source through the key element text, and performs similarity matching between the external source vector corresponding to the external text and the dense element vector to obtain a similarity value;

[0028] Arranging the external source vectors corresponding to the similarity values ​​from large to small according to the similarity values ​​to form an external source vector list;

[0029] Respective external texts corresponding to the external source vector list are obtained to form external knowledge.

[0030] Optionally, the self-sorting the external knowledge and language knowledge by an unsupervised paragraph sorting mechanism to generate a target knowledge graph includes:

[0031] Acquire the invisible relation r based on the role instruction;

[0032] The external knowledge and the language knowledge are integrated to form a text set P = {p1, p2, ..., p n}, and sort the text segment p based on the sorting strategy and the invisible relationship. n Perform optimization sorting to obtain combing order;

[0033] Adding supplementary nodes to the initial knowledge graph in accordance with the combing order to fill the external knowledge, and combing the original language knowledge and the external knowledge to form a target knowledge graph.

[0034] Optionally, the comparison of the invisible relationship to the text segment p n Perform optimization sorting to obtain a combing order, including:

[0035] The invisible relationship r and the text segment p are quantified by the preset BERT. n Encode to generate the text vector v pi and the relationship category vector v r ;

[0036] Calculate the text vector v pi and each relationship category vector v r and arranging the text segment vectors corresponding to each relationship category vector in descending order based on the value of the correlation score to obtain a text segment vector arrangement;

[0037] The text segments corresponding to the text segment vector arrangement are arranged in the same order to obtain a combing order.

[0038] The present invention also provides a graph construction system based on a neural network and a sorting mechanism, which implements the graph construction method based on a neural network and a sorting mechanism as described above, and the system includes:

[0039] A large language model is used to generate a knowledge graph framework based on the obtained character instructions;

[0040] An autoregressive language model, used to obtain language knowledge about the knowledge graph framework, and to fill the knowledge graph framework with the language knowledge to form an initial knowledge graph;

[0041] A double-tower structure is used to expand external knowledge to obtain external knowledge that matches the initial knowledge graph;

[0042] A ranking structure is used to self-rank the external knowledge and language knowledge through an unsupervised paragraph ranking mechanism to generate a target knowledge graph.

[0043] Optionally, the sorting structure includes:

[0044] A relation extraction unit, configured to obtain an invisible relation r based on the role instruction;

[0045] A sorting strategy unit is used to fuse the external knowledge and the language knowledge to form a text set P = {p1, p2, ..., p n}, and compare the invisible relationship with the text segment p based on the sorting strategy n Perform optimization sorting to obtain combing order;

[0046] A target combing unit is used to add supplementary nodes to the initial knowledge graph according to the combing order to fill the external knowledge, and comb the original language knowledge and the external knowledge to form a target knowledge graph.

[0047] From the above technical solution, it can be seen that the graph construction method and system based on neural network and sorting mechanism provided by the present invention enable the preset large language model to generate a knowledge graph framework through role instructions, and then obtain language knowledge from the knowledge graph framework based on the autoregressive language model, and use the language knowledge to fill the knowledge graph framework to form an initial knowledge graph, showing language variation and implicit relationships, and then use a preset double-tower structure to expand external knowledge to obtain external knowledge matching the initial knowledge graph to enhance the breadth of the knowledge graph, and then use an unsupervised paragraph sorting mechanism to self-sort the external knowledge and language knowledge to generate a target knowledge graph, thereby improving the comprehensiveness and accuracy of the knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more clear and easy to understand. In the accompanying drawings:

[0049] Figure 1 A flowchart of a graph construction method based on a neural network and a sorting mechanism according to an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of a double-tower structure in a graph construction method based on a neural network and a sorting mechanism according to an embodiment of the present invention;

[0051] Figure 3 A logical block diagram of a graph construction system based on a neural network and a sorting mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] Existing graph construction methods are either easily affected by subjective factors, resulting in incomplete or biased information in the knowledge graph, or are unable to cope with complex language variations and implicit relationships, and often fail to capture the deep semantics and dynamic knowledge in the text.

[0053] In view of the above problems, the present invention provides a graph construction method and system based on a neural network and a sorting mechanism. The specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

[0054] In order to illustrate the graph construction method and system based on neural network and sorting mechanism provided by the present invention, Figure 1-Figure 3 The embodiments of the present invention are exemplarily indicated.

[0055] The following description of exemplary embodiments is merely illustrative in nature and is not intended to limit the present invention and its application or use. Technologies and devices known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technologies and devices should be considered as part of the specification.

[0056] like Figure 1 As shown, the present invention provides a graph construction method based on a neural network and a sorting mechanism, comprising:

[0057] S1: Use role instructions to enable the preset large language model to generate a knowledge graph framework;

[0058] S2: Acquire language knowledge about the knowledge graph framework based on an autoregressive language model, and use the language knowledge to fill the knowledge graph framework to form an initial knowledge graph;

[0059] S3: using a preset double-tower structure to expand external knowledge to obtain external knowledge matching the initial knowledge graph;

[0060] S4: The external knowledge and the language knowledge are self-sorted through an unsupervised paragraph sorting mechanism to generate a target knowledge graph.

[0061] Step S1 is a process of generating a knowledge graph framework from a preset large language model through role instructions. In this embodiment, generating a knowledge graph framework from a preset large language model through role instructions includes:

[0062] S11: receiving a role instruction on ideal knowledge graph generation through the large language model;

[0063] S12: parsing the role instruction to obtain key elements and constraint conditions based on a preset conditional constraint template according to the role instruction;

[0064] S13: sorting out the relationships of the key elements based on the constraint conditions to form element relationships;

[0065] S14: Using the key elements as nodes and the element relationships as edges to establish a graph to form a knowledge graph framework;

[0066] Among them, the key elements include task definition, task purpose, task problem, relevant roles, time nodes, implementation location, task effect and implementation method; the constraints include object constraints, motivation constraints, role constraints, time constraints, space constraints, quantitative constraints and method constraints.

[0067] In this embodiment,

[0068] Object constraint: the subject is the task definition and the object is the task purpose;

[0069] Motivational constraints: the subject is the task definition, and the object is the task problem;

[0070] Role constraints: the subject is the task definition and the object is the related role;

[0071] Time constraint: the subject is the task definition and the object is the time node;

[0072] Spatial constraints: the subject is the task definition and the object is the implementation location;

[0073] Method constraint: the subject is the task definition and the object is the implementation method;

[0074] Quantitative constraints: the subject is the task definition, and the object is the task effect (output benefit).

[0075] To illustrate, take the role command "Developed a system for optimizing supply chain management, which aims to reduce inventory costs and solve the problem of logistics delays. The project is led by Zhang San and is expected to be implemented in Shenyang in 2024. It uses blockchain-based technology and is expected to save 10% of operating costs."

[0076] If the above example is input as a role command into the large language model, the following key elements are automatically identified:

[0077] · Task definition (system description)

[0078] · Mission objective (reduce inventory costs)

[0079] · Problem solved - Mission problem (logistics delay)

[0080] · Related roles (Zhang San)

[0081] · Time node (2024)

[0082] · Implementation location (Shenyang)

[0083] · Specific method (blockchain technology)

[0084] · Output efficiency (saving 10% of costs)

[0085] These key elements constitute the nodes in the knowledge graph (this can be understood as being stored as knowledge graph node objects, each node object has several edges associated with other node objects, which are edges).

[0086] According to the predefined constraint rules, corresponding constraint conditions are formed. The large language model organizes the extracted information into subject nodes, object nodes, and the relationship between them, and expresses them in the form of a graph. The specific constraint conditions and graph generation steps are as follows:

[0087] ·Object constraint: The subject is the task definition, and the object is the task purpose. For example, the node relationship generated by system → reduce inventory cost is: task definition → task purpose, where "system" is the subject node, "reduce inventory cost" is the object node, and the edge represents the "object constraint".

[0088] · Motivational constraints: The subject is the task definition, and the object is the problem to be solved. Example: System →

[0089] The node relationship generated by the logistics delay problem is: task definition → problem to be solved, the subject node is "system", the object node is "logistics delay", and the edge is "motivation constraint".

[0090] · Role constraint: The subject is the task definition, and the object is the related role. For example, the node relationship generated by system → Zhang San is: task definition → related role, where "system" is the subject, "Zhang San" is the object, and the edge is "role constraint".

[0091] · Time constraint: The subject is the task definition, and the object is the time node. For example, the node relationship generated by system → 2024 is: task definition → time node, where "system" is the subject,

[0092] "2024" is the object and the edge is the "time constraint".

[0093] · Spatial constraint: The subject is the task definition, and the object is the implementation location. For example, the node relationship generated by system → Shenyang is: task definition → implementation location, where "system" is the subject, "Shenyang" is the object, and the edge is "spatial constraint".

[0094] · Method constraint: The subject is the task definition, and the object is the specific method. Example: The node relationship generated by system → blockchain technology is: task definition → specific method, where "system" is the subject, "blockchain technology" is the object, and the edge is "method constraint".

[0095] · Quantitative constraint: The subject is the task definition, and the object is the output benefit. For example: System → Save 10% of operating costs generates a node relationship of: Task definition → Output benefit, where "system" is the subject, "Save 10% of costs" is the object, and the edge is "quantitative constraint".

[0096] More specifically, in a specific embodiment, the large language model (LLM) will parse the input text and parse out key elements such as task definition, task purpose, problem to be solved, relevant roles, time nodes, implementation locations, specific methods, and output benefits according to the constraint template. These key elements will serve as the basic components of nodes and edges in the knowledge graph. Prior to this, preprocessing must be performed, which includes sentence segmentation, word segmentation, named entity recognition (NER), syntactic analysis, and other steps to ensure that the key information in the text is accurately identified. After completing the text preprocessing, the LLM will generate nodes and edges of the graph framework based on the extracted key information. Each identified element, such as "task definition", "task purpose", etc., will become a node, and the relationship between these nodes (such as "object constraint", "motivation constraint", etc.) will connect these nodes as edges. During this process, LLM will follow the preset knowledge graph analysis rules to ensure that the generation of nodes and edges meets the constraint requirements; after generating the nodes and edges of the graph framework, LLM will optimize the structure of the knowledge graph to form a knowledge graph framework. These optimizations may include eliminating redundant nodes, merging duplicate edges, standardizing the naming of nodes and edges, and ensuring the overall logic and coherence of the graph. The optimized graph will be clearer and easier to read, and can accurately reflect the information structure in the text.

[0097] In this way, by setting LLM as a senior knowledge graph engineer, the knowledge graph is automatically extracted and constructed from the text. The main improvement of this framework is the introduction of detailed knowledge graph analysis rules, including seven types of constraints such as object, motivation, role, time, space, method and quantification, which makes the generated knowledge graph clearer and more structured, and can accurately reflect the information in the text, making the construction of the knowledge graph more automated and efficient.

[0098] Step S2 is a process of acquiring language knowledge from the knowledge graph framework based on the autoregressive language model, and filling the knowledge graph framework with the language knowledge to form an initial knowledge graph. In this embodiment, acquiring language knowledge from the knowledge graph framework based on the autoregressive language model, and filling the knowledge graph framework with the language knowledge to form an initial knowledge graph, includes:

[0099] S21: using a pre-fine-tuned autoregressive language model to retrieve context information of the role instruction to obtain valid information, key element text, and element relationship text as language knowledge;

[0100] S22: mapping out language points corresponding to the key element text and language relationships corresponding to the element relationship text according to the language knowledge;

[0101] S23: Based on the valid information, the language point is matched with the node on the knowledge graph framework, and the language knowledge that has a mapping relationship with the language point is filled into the node corresponding to the language point, and the element relationship text is matched with the edge of the knowledge graph framework; and if there is no node corresponding to the language point, a corresponding supplementary node is added to the knowledge graph framework, and the key element text corresponding to the supplementary node is filled into the supplementary node, and at the same time, a supplementary edge related to the supplementary node is added based on the valid information, and the supplementary edge is described with the element relationship text related to the supplementary node to form a filled graph;

[0102] S24: Optimizing the filled graph to form an initial knowledge graph; the optimization includes merging redundant nodes and correcting edges with unclear descriptions.

[0103] Specifically, in a specific embodiment, an autoregressive language model is selected to act as a data generator. The generator in the autoregressive language model is based on the Transformer architecture and is constructed using a multi-layer decoder-style Transformer block, and each layer of the block contains a self-attention mechanism and a feedforward neural network. In the process of filling the knowledge graph framework based on the autoregressive language model (ARLM), a complete initial knowledge graph is gradually generated by expanding and improving the knowledge graph framework generated above. In order to enable the autoregressive language model to accurately fill the knowledge graph, ARLM will first be fine-tuned, that is, it is trained and fine-tuned through an existing training data set (Dtr), which is usually a structured information sample obtained from business data. The data samples in Dtr help the model learn the structure and language patterns of a specific field, ensuring that the model has a good understanding of certain business logic and knowledge representation. The specific fine-tuning steps include: fine-tuning the ARLM model using the business data in Dtr to ensure that it can learn the complex relationships between elements such as task definition, purpose, role, time, etc.; optimizing the model parameters so that it can better predict and generate supplementary information based on these relationships.

[0104] The following is a detailed description of this process, with specific examples to illustrate how to gradually fill the knowledge graph framework, and the autoregressive language model receives the knowledge graph framework. The knowledge graph framework has generated the initial node and edge structure based on the key information extracted from the text in the previous step. For example:

[0105] Assuming that there is an incomplete knowledge graph framework, the content of the knowledge graph framework is as follows:

[0106] system

[0107] ├──Reduce inventory costs (object constraints)

[0108] ├──Logistics delay problem (motivation constraint)

[0109] ├──Zhang San (role constraint)

[0110] ├──2024 (time constraint)

[0111] ├──Shenyang (spatial constraints)

[0112] ├──Blockchain technology (method constraints)

[0113] └──Save 10% of costs (quantitative constraint)

[0114] At this point, the framework has only established preliminary nodes and edges based on the most basic information, but some nodes may not be detailed enough or have information gaps.

[0115] Therefore, it is necessary to fill in the nodes and edges that are not fully constructed. The autoregressive nature of ARLM means that it will predict and generate the next step of information based on the generated part. First, it retrieves the context information of the role instruction to obtain valid information, key element text and element relationship text as language knowledge. Based on the language knowledge, it maps the language points corresponding to the key element text and the language relationships corresponding to the element relationship text. For example, the language points and language relationships with key element text are as follows:

[0116] system

[0117] ├──Reduce inventory costs (object constraints)

[0118] ├──Logistics delay problem (motivation constraint)

[0119] ├──Zhang San (role constraint)

[0120] ├──2024 (time constraint)

[0121] ├──Shenyang (spatial constraints)

[0122] ├──Blockchain technology (method constraints)

[0123] ├──Real-time tracking and traceability (blockchain application scenarios)

[0124] └──Enhance supply chain transparency (goal)

[0125] └──Save 10% of costs (quantitative constraint);

[0126] It is easy to see that the language points and language relationships with key element texts have added some content compared to the above-mentioned knowledge graph framework content. Therefore, it is necessary to iterate, expand and verify. According to the effective information preset in advance, the language points can be corresponded to the nodes on the knowledge graph framework, and the language knowledge with a mapping relationship with the language points can be filled into the nodes corresponding to the language points, and the element relationship text can be corresponded to the edges of the knowledge graph framework. If there is no node corresponding to the language point, a corresponding supplementary node is added to the knowledge graph framework, and the key element text corresponding to the supplementary node is filled into the supplementary node. At the same time, based on the effective information, a supplementary edge related to the supplementary node is added, and the element relationship text related to the supplementary node is used to describe the supplementary edge to form a filled graph. It is iterated according to the generated results to continue to fill in other related nodes and edges. For example, with the appearance of the time node "2024", the model may generate more relevant future expectations or plan information. At the same time, the system will verify after each round of generation to ensure that the generated nodes and edges are logically consistent to avoid duplication and contradiction.

[0127] The final filled map example:

[0128] system

[0129] ├──Reduce inventory costs (object constraints)

[0130] ├──Logistics delay problem (motivation constraint)

[0131] ├──Zhang San (role constraint)

[0132] ├──2024 (time constraint)

[0133] ├──Shenyang (spatial constraints)

[0134] ├──Blockchain technology (method constraints)

[0135] ├──Real-time tracking and traceability (blockchain application scenarios)

[0136] ├──Enhance supply chain transparency (goal)

[0137] └──Smart Contract Automation (Technical Application)

[0138] └──Save 10% of costs (quantitative constraint);

[0139] At this point, ARLM has not only filled the application scenario of "blockchain technology", but also generated the goal of "enhancing supply chain transparency" and connected the two nodes.

[0140] The filled graph is then optimized and improved, such as merging redundant nodes and correcting unclear edges to ensure the overall logic and accuracy of the knowledge graph to form an initial knowledge graph. The final generated initial knowledge graph will contain rich knowledge structures extracted from business data to provide support for downstream tasks.

[0141] In addition, in the autoregressive language model, the relation label Rtr can be used as a generation prompt to generate synthetic data samples Dgen. The data generator receives a structured prompt of the form "Relation:r." and outputs structured data of "Relation:r.Context:s,Head Entity:eh,Tail Entity:et." that matches the sample structure in the dataset D.

[0142] Moreover, in this specific embodiment, the autoregressive language model is trained in an autoregressive manner and adopts the standard language modeling goal, that is, the next word prediction mechanism. Given any sequence x = [x1, x2, ..., xn], in order to accurately generate the following text, the training goal of the generator is to learn the conditional probability

[0143]

[0144] In this way, generalization is achieved with the help of the existing knowledge graph framework, providing support for the model to handle tasks that it has never encountered. Specific control signals or prompts are used to guide the text generation of the autoregressive language model, enhancing the accuracy and diversity of the content of the knowledge graph framework.

[0145] exist Figure 1 , Figure 2 In the embodiment shown together, step S3 is a process of using a preset double-tower structure to expand external knowledge to obtain external knowledge that matches the initial knowledge graph, wherein: Figure 2 The double-tower structure shown is connected to an external knowledge source; the double-tower structure includes a question encoder, a knowledge encoder and a retrieval component, wherein the knowledge encoder is used to convert the external knowledge source into an external source vector.

[0146] The preset double-tower structure is used to expand external knowledge to obtain external knowledge matching the initial knowledge graph, including:

[0147] S31: Encoding the key element text into a dense element vector through the question encoder;

[0148] S32: the retrieval component retrieves relevant external texts in the external knowledge source through the key element text, and performs similarity matching between the external source vector corresponding to the external text and the dense element vector to obtain a similarity value;

[0149] S33: Arranging the external source vectors corresponding to the similarity values ​​from large to small according to the similarity values ​​to form an external source vector list;

[0150] S34: Acquire each external text corresponding to the external source vector list to form external knowledge.

[0151] In a specific embodiment, a dense vector index is constructed with external knowledge data as an external knowledge source z, and a neural searcher is used to efficiently obtain the external knowledge source. For the relation label set Rte that is not in the visible relation class, a question template "What is the relation r?" is constructed as a strategy, where the label is selected from Rte, and the question is input as the source text x for retrieval.

[0152] In the previous step, the initial knowledge graph has been generated through the filling method based on the autoregressive language model. For example, the graph contains nodes such as "system", "blockchain technology", and "reduce inventory costs", but some relationship information may not be comprehensive or accurate enough. At this time, the knowledge graph can be improved by retrieving more supplementary information from external knowledge sources. (External retrieval requires the nodes and edges in the constructed knowledge graph as input, which are the objects of retrieval expansion, and the link to the external data source needs to be specified as the retrieval expansion target source).

[0153] The external knowledge source z is a dense vector index library that contains rich factual knowledge. We obtain supplementary information from the knowledge base through retrieval, which can be used to fill, expand and verify the relationships and nodes in the initial knowledge graph. That is, the retrieval component retrieves relevant external texts in the external knowledge source through key element texts, and performs similarity matching between the external source vectors corresponding to the external texts and the dense element vectors to obtain similarity values.

[0154] The dual-tower structure is the core part of the retrieval process. The dual-tower structure consists of two independent encoders: the question encoder and the knowledge encoder, which are used to encode the question and the external knowledge source respectively. The specific input and retrieval process is as follows:

[0155] a. Input build

[0156] In order to use the external knowledge source z to supplement the labels in the invisible relation class set Rte, the system constructs a question template in the form of: "What is the relation r?" Here, r is a specific relation label selected from the invisible relation class set Rte.

[0157] Assuming there is a set of labels in the invisible relationship class, such as "supply chain optimization", the system will combine these labels with the question template to construct a specific question. For example:

[0158] "What is the relation of supply chain optimization?"

[0159] This question template becomes the query source text x for external knowledge retrieval.

[0160] b. Encoding

[0161] Next, the two encoders of the dual-tower structure encode the question and the external knowledge source respectively: the question encoder (left tower) encodes the question x (e.g., "What is the relation of supply chain optimization?") into a dense vector representation v_x. This encoder is usually a Transformer or BERT-based model that can capture the semantic information of the question. The knowledge encoder (right tower) encodes the knowledge items in the external knowledge source z and generates a dense vector representation v_z. These knowledge items can be business-related documents, sentences, or factual data in an external database.

[0162] c. Vector search

[0163] After the encoding of the question and external knowledge is completed, the system will perform similarity matching between the vector representation v_x of the question and the vector representation v_z of the external knowledge source, usually by calculating the inner product or cosine similarity to find the most matching external source vector entry. At this time, the most similar external knowledge will be selected and returned as the search results, and the external source vectors corresponding to the similarity values ​​will be arranged from large to small according to the similarity value to form an external source vector list.

[0164] For example, for the question “What is the relation of supply chain optimization?”, the system may retrieve the following supplementary information from external knowledge sources:

[0165] “Supply chain optimization often involves using blockchain technology to improve transparency.”

[0166] “Real-time tracking and automated contract execution are common methods in supply chain management.”

[0167] The specific retrieval process can map the source text x and each article in the external knowledge database into 100-word paragraphs z respectively into vector representations. There are about 21 million paragraphs in total. The document encoder calculates their embedding vectors and constructs the maximum inner product search index of the approximate algorithm, making the retrieval process both accurate and fast. Then the retrieval component uses the question statement after splicing the relationship tags as the source text x to retrieve relevant documents in a targeted manner. In the process of building this index, the encoder in the dense text retriever DPR model is used as the starting point, and then the retriever is initialized and the document index is created:

[0168]

[0169] d(z)=BERT d (z)

[0170] q(x)=BERT q (x).

[0171] The dense vector representation d(z) of the document is obtained by a BERT-Based e The query representation q(x) of the source text x is generated by the document encoder of BERT-Base, while the query representation q(x) of the source text x is produced by the source text encoder also based on BERT-Base. The fusion of these two representations enables the model to use an efficient maximum inner product search algorithm to calculate and obtain an ordered list of K documents z with the highest conditional probability Pη(z|x). This probability highly indicates the relevance of these documents to the given source text x.

[0172] In this embodiment, after retrieving external knowledge, step S4 can be performed directly, or the initial knowledge graph framework can be updated with the acquired external knowledge before step S4 is performed. If the initial knowledge graph framework is updated, the external knowledge is used as supplementary information to update the initial knowledge graph. Assume that in the previously generated initial knowledge graph, the information about the application scenarios and effects of "blockchain technology" is incomplete, and the system supplements the new knowledge of "transparency improvement" and "automated execution" through retrieval. The following is the updated initial knowledge graph structure:

[0173] system

[0174] ├──Reduce inventory costs (object constraints)

[0175] ├──Logistics delay problem (motivation constraint)

[0176] ├──Zhang San (role constraint)

[0177] ├──2024 (time constraint)

[0178] ├──Shenyang (spatial constraints)

[0179] ├──Blockchain technology (method constraints)

[0180] ├──Real-time tracking and traceability (blockchain application scenarios)

[0181] ├──Enhance supply chain transparency (goal)

[0182] ├──Automated contract execution (application method)

[0183] └──Save 10% of costs (quantitative constraints),

[0184] That is, by filling in external knowledge, the graph becomes more complete and detailed, and can reflect richer relationship information.

[0185] In general, the dual-tower structure first constructs a question template: selects relation labels from the invisible relation class Rte to construct a question template, such as "What is the relation r?"; then encodes the question: the question is converted into a dense vector representation through the question encoder, and knowledge encoding is performed: the knowledge encoder is used to encode all knowledge items in the external knowledge source z, generate a dense vector representation and establish an index, and then the two are matched for similarity: the most relevant external knowledge is retrieved by calculating the similarity between the question vector and the knowledge vector. Then fill in the external knowledge to update the initial knowledge graph: combine the retrieved knowledge items with the initial knowledge graph, and update and expand the nodes and relationships in the initial knowledge graph.

[0186] For example, for relationship labels: the relationships in the initial knowledge graph may not be comprehensive or specific enough. For example, there may be a situation where the application scenario of "blockchain technology" is unclear. In this case, the system will input the relationship label (such as "blockchain technology application") as the source text into the source text encoder to retrieve external knowledge related to the relationship. For entity information: when the relationship between certain entities is unclear, the system can retrieve based on the entity information in the initial knowledge graph. For example, there is a "system" node and an "reduce inventory cost" edge in the graph, but the specific operation method is unclear. At this time, such entities and incomplete relationships can be used as source text for retrieval. For missing relationships or attributes: For missing attributes or unclear relationships in the initial knowledge graph, the system can retrieve by constructing question templates such as "What is the relationship about X?" or "How to implement X?" For example, "What is the application of blockchain technology for supply chain optimization?" can be used as source text to retrieve relevant documents in external knowledge bases.

[0187] Through such external retrieval and dual-tower structure, the knowledge graph can not only be filled according to the internally generated framework, but also external knowledge can be dynamically introduced to achieve efficient construction and optimization of the knowledge graph. In this way, the text retriever is used to extract relevant text segments from external knowledge sources to fill the knowledge gaps of the model in the synthetic data, thereby improving the accuracy and consistency of the data, effectively enhancing the model's ability to handle complex relationships and knowledge, and reflecting the importance of external knowledge acquisition in improving model performance.

[0188] Step S4 is a process of self-sorting the external knowledge and language knowledge through an unsupervised paragraph sorting mechanism to generate a target knowledge graph; wherein, self-sorting the external knowledge and language knowledge through an unsupervised paragraph sorting mechanism to generate a target knowledge graph includes:

[0189] S41: Acquire the invisible relationship r based on the role instruction;

[0190] S42: Fusing the external knowledge and the language knowledge to form a text set P = {p1, p2, ..., p n}, and sort the text segment p based on the sorting strategy and the invisible relationship. n Perform optimization sorting to obtain combing order;

[0191] S43: Adding supplementary nodes to the initial knowledge graph in accordance with the combing order to fill in the external knowledge, and combing the original language knowledge and external knowledge to form a target knowledge graph.

[0192] The step of optimizing and sorting the text segments by referring to the invisible relationship to obtain a sorting order includes:

[0193] S421: Using a preset BERT to analyze the invisible relationship r and the text segment p n Encode to generate the text vector v pi and the relationship category vector v r ;

[0194] S422: Calculate the text vector v pi and each relationship category vector v r and arranging the text segment vectors corresponding to each relationship category vector in descending order based on the value of the correlation score to obtain a text segment vector arrangement;

[0195] S423: Arrange the text segments corresponding to the text segment vector arrangement in the same order to obtain a combing order.

[0196] The goal of this unsupervised paragraph ranking mechanism is to fuse and sort the synthetic data generated for each invisible relationship category with the data set of the retrieved documents, and finally select the top k ranked data as the extended knowledge graph of the specific relationship class. The specific method is to first combine the retrieved document data with the synthetic data for each relationship category, and obtain a total of n comprehensive data, forming a paragraph set P = {p1, p2, ..., p n The task of the paragraph sorting module is to optimize the order of the paragraphs in the set P, ensuring that the paragraphs that accurately reflect the knowledge of the corresponding relationship are ranked higher in the list. i The sorting is based on its relevance score p(pi| r ).

[0197] In a more specific embodiment, the detailed steps are:

[0198] 1. Fusion of synthetic data and retrieved documents

[0199] In the previous steps, based on the autoregressive language model, we have generated the initial knowledge graph framework. For example, "The application of blockchain technology in supply chain management" may generate a series of sentences or paragraphs to describe relevant scenarios and information. At the same time, through the double-tower retrieval mechanism, documents obtained from external knowledge sources also provide a large amount of external knowledge. This external knowledge may contain more detailed facts, examples or methods. For example, the retrieved external documents may involve the application of blockchain technology in real-time tracking, transparency improvement and other fields. In order to enrich the knowledge graph, the content of these language knowledge and external knowledge will be integrated to form a new set of paragraphs, denoted as P = {p1, p2, ..., p n}, where each paragraph p iIt can be a segment from synthetic data or a segment retrieved from an external knowledge base.

[0200] 2. Goal of paragraph sorting

[0201] The main task of the paragraph sorting module is to reorder the paragraphs in the paragraph set P to ensure that the paragraphs that best reflect the invisible relationship category r (such as "the application of blockchain technology") are ranked first. In this way, the top k paragraphs are selected after sorting to ensure that the information retained in the extended knowledge graph is the most relevant and accurate.

[0202] 3. Sorting basis: relevance score

[0203] Each passage is ranked based on its relevance score with the target relation category r, denoted as p(pi|r). This score measures the degree of match between passage pi and relation r. The relevance is calculated by the following steps:

[0204] a. Text coding and relationship category coding

[0205] First, each paragraph p i and the target relation category r are respectively vectorized by an encoder (e.g., BERT): i Encoded into a vector representation through the text encoder, v pi The relation category r is encoded into another vector representation v by the relation encoder r .

[0206] b. Similarity calculation

[0207] Calculate the paragraph vector v pi and the relationship category vector v r The similarity between them. Cosine similarity or dot product can be used to quantify their relevance. The higher the similarity, the higher the match between the text segment and the relationship category. The formula can be expressed as:

[0208] p(pi|r)=cos(v pi ,v r ).

[0209] c. Sorting Optimization

[0210] According to the calculated relevance score p(pi|r), all the texts in the text set P are sorted to ensure that the texts most relevant to the target relationship category are ranked first. The text sorting optimization module may dynamically adjust the order of the texts based on a learned sorting strategy to improve the accuracy of the sorting.

[0211] 4. Select the first k data

[0212] After the sorting is completed, the system will select the top k data and arrange the text vectors corresponding to each relationship category vector to obtain the text vector arrangement. The corresponding texts have been verified and can better supplement the nodes and relationships in the initial knowledge graph, improving the comprehensiveness and accuracy of the graph.

[0213] For example, suppose we are constructing an extended knowledge graph for the invisible relationship category “Application of blockchain technology in supply chain management”. The steps are as follows:

[0214] Synthetic data in language knowledge: Multiple synthetic data were generated through the autoregressive language model, such as "Blockchain technology improves the transparency of the supply chain" and "Blockchain can be used for real-time tracking".

[0215] Retrieve documents from external knowledge: External documents are retrieved through the BERT dual-tower structure, such as "The use of blockchain in logistics can reduce human errors" and "Blockchain in supply chain management can realize automatic contracts."

[0216] Fusion text: Combine the synthesized data with the retrieved documents to form a text set P.

[0217] Sorting process: Using the relational category “Application of blockchain technology in supply chain management” as the target, the text segments are vectorized, the relevance scores are calculated, and the text segments are sorted according to the relevance.

[0218] Select the top k text segments: Select the top k most relevant text segments and sort them out to form the target knowledge graph.

[0219] In the above ranking method, an unsupervised learning strategy without task-specific training can be adopted. For example, the language model T0 can be used as a scoring tool to calculate the probability that any given paragraph p potentially implies a relation category r. The implied relation estimation model uses zero-shot learning, allowing paragraph re-ranking to be performed without relying on a specific dataset. It interprets each token of the input relation label during the model training process, integrating the cross-attention between the relation category and the paragraph token. It can be seen that the proposed paragraph re-ranking strategy has higher explanation performance than simply using a dense retriever, although in essence, both methods are built on similar models. In order to exhaustively evaluate the relevance between paragraphs and relation labels, the focus is on the establishment and application of probability models, which can estimate the probability that the relation label r is correctly implied given the paragraph text pi, that is, p(r|pi). Furthermore, by applying Bayes' theorem, it is possible to convert from p(r|pi) to the posterior probability p(pi|r), which directly reflects the relevance of a specific paragraph given a certain relation label.

[0220] log p(p i|r)=log p(r|p i )+log p(p i )+c;

[0221] where p(pi) is the prior value of the retrieved paragraph, and c is a common constant for all pi. Trying to simplify this assumption, we can consider the prior logp(pi) of the paragraphs as uniform and negligible during re-ranking. Thus, the above expression can be simplified to

[0222]

[0223] The large language model is then used to quantitatively evaluate the conditional probability log likelihood logp(r|pi) that a given text pi contains a relation tag r. This language model can capture complex contextual dependencies and accurately estimate the probability of generating a specific relation token in a specific text context in the form of mathematical log likelihood. This method is crucial in improving the performance of sorting tasks and shows its strong prediction and application potential.

[0224]

[0225] Where θ represents the parameter set of the language model used in the study, and |r| is used to indicate the number of tokens contained in a specific question. As shown in the figure, by directly splicing the natural language instruction "What kind of relationship is implied in the above passage?" to the paragraph token, the language model is used to actively generate the possible implicit relationships in the paragraph in a zero-shot learning manner. This processing step does not involve any type of fine-tuning process. Finally, all generated and retrieved paragraphs in each relationship category are ranked according to the calculated conditional probability log-likelihood value log p(r|pi), and a certain amount of top-ranked data is selected as the dataset for the relationship extractor. This re-ranking strategy can reasonably determine the mixed ratio of enhanced data obtained by generation and retrieval, aiming to provide model training with as much information as possible containing the target relationship domain knowledge, while reducing the noise problem that may be caused by the introduction of low-quality data.

[0226] In this way, the technical problem that the generated synthetic data may present low quality when the data generator has errors in semantic understanding of certain relationship labels or insufficient coverage of relevant knowledge in the prior art is solved. The knowledge quality judgment method based on the unsupervised paragraph sorting mechanism can identify as much information containing target relationship domain knowledge as possible, while reducing the noise problem that may be caused by the introduction of low-quality data.

[0227] As described above, the graph construction method and system based on neural network and sorting mechanism provided by the present invention enable the preset large language model to generate a knowledge graph framework through role instructions, and then acquire language knowledge from the knowledge graph framework based on the autoregressive language model, and use the language knowledge to fill the knowledge graph framework to form an initial knowledge graph to obtain the deep semantics of the knowledge graph, display language variation and implicit relationships, and then use the preset double-tower structure to expand external knowledge to obtain external knowledge matching the initial knowledge graph, so as to obtain dynamic knowledge and improve the breadth of the knowledge graph, and then use the unsupervised paragraph sorting mechanism to self-sort the external knowledge and language knowledge to generate a target knowledge graph, thereby improving the comprehensiveness and accuracy of the knowledge graph.

[0228] like Figure 3 As shown, the present invention also provides a graph construction system 100 based on a neural network and a sorting mechanism, which implements the graph construction method based on a neural network and a sorting mechanism as described above, and the system includes:

[0229] A large language model 101, used to generate a knowledge graph framework based on the acquired role instructions;

[0230] An autoregressive language model 102, used to obtain language knowledge about the knowledge graph framework, and use the language knowledge to fill the knowledge graph framework to form an initial knowledge graph;

[0231] The double tower structure 103 is used to perform external knowledge expansion to obtain external knowledge matching the initial knowledge graph;

[0232] The sorting structure 104 is used to self-sort the external knowledge and language knowledge through an unsupervised paragraph sorting mechanism to generate a target knowledge graph.

[0233] The sorting structure 104 includes:

[0234] A relation extraction unit 1041, configured to obtain an invisible relation r based on the role instruction;

[0235] The sorting strategy unit 1042 is used to merge the external knowledge and the language knowledge to form a text segment set P = {p1, p2, ..., p n}, and compare the invisible relationship with the text segment p based on the sorting strategy n Perform optimization sorting to obtain combing order;

[0236] The target combing unit 1043 is used to add supplementary nodes to the initial knowledge graph in a combing order to fill the external knowledge, and comb the original language knowledge and the external knowledge to form a target knowledge graph.

[0237] A more specific implementation of the graph construction system based on a neural network and a sorting mechanism refers to the above-mentioned embodiment of the graph construction method based on a neural network and a sorting mechanism, which will not be described in detail here.

[0238] As described above with reference to the accompanying drawings, the graph construction method and system based on the neural network and sorting mechanism proposed in the present invention are described by way of example. However, those skilled in the art should understand that various improvements can be made to the graph construction method and system based on the neural network and sorting mechanism proposed in the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.

Claims

1. A graph construction method based on neural network and sorting mechanism, characterized in that: include: Through role instructions, the preset large language model generates a knowledge graph framework; Acquire language knowledge about the knowledge graph framework based on an autoregressive language model, and use the language knowledge to fill the knowledge graph framework to form an initial knowledge graph; Using a preset double-tower structure to expand external knowledge to obtain external knowledge matching the initial knowledge graph; The external knowledge and the language knowledge are self-ranked through an unsupervised paragraph ranking mechanism to generate a target knowledge graph.

2. The graph construction method based on a neural network and a sorting mechanism as claimed in claim 1, characterized in that: The step of generating a knowledge graph framework from a preset large language model through role instructions includes: Receiving role instructions on ideal knowledge graph generation through the large language model; Parsing the role instruction to obtain key elements and constraint conditions based on a preset conditional constraint template according to the role instruction; Sorting out the relationships of the key elements based on the constraint conditions to form element relationships; The key elements are used as nodes and the element relationships are used as edges to establish a graph to form a knowledge graph framework.

3. The graph construction method based on neural network and sorting mechanism as claimed in claim 2, characterized in that: The key elements include task definition, task purpose, task problem, relevant roles, time node, implementation location, task effect and implementation method; The constraints include object constraints, motivation constraints, role constraints, time constraints, space constraints, quantitative constraints and method constraints.

4. The graph construction method based on neural network and sorting mechanism as claimed in claim 2, characterized in that: The method of acquiring language knowledge from the knowledge graph framework based on the autoregressive language model and using the language knowledge to fill the knowledge graph framework to form an initial knowledge graph includes: Retrieving context information of the role instruction using a pre-fine-tuned autoregressive language model to obtain valid information, key element text, and element relationship text as language knowledge; Establishing a mapping of language points corresponding to the key element text and language relationships corresponding to the element relationship text according to the language knowledge; Based on the valid information, the language point is matched with the node on the knowledge graph framework, and the language knowledge that has a mapping relationship with the language point is filled into the node corresponding to the language point, and the element relationship text is matched with the edge of the knowledge graph framework; and if there is no node corresponding to the language point, a corresponding supplementary node is added to the knowledge graph framework, and the key element text corresponding to the supplementary node is filled into the supplementary node, and at the same time, a supplementary edge related to the supplementary node is added based on the valid information, and the supplementary edge is described with the element relationship text related to the supplementary node to form a filled graph; The filled graph is optimized to form an initial knowledge graph; the optimization includes merging redundant nodes and correcting edges with unclear descriptions.

5. The graph construction method based on neural network and sorting mechanism as claimed in claim 1, characterized in that: The double tower structure is connected to an external knowledge source; The dual-tower structure includes a question encoder, a knowledge encoder and a retrieval component, wherein the knowledge encoder is used to convert the external knowledge source into an external source vector.

6. The graph construction method based on neural network and sorting mechanism as claimed in claim 5, characterized in that: The method of using a preset double-tower structure to expand external knowledge to obtain external knowledge matching the initial knowledge graph includes: Encoding the key element text into a dense element vector by the question encoder; The retrieval component retrieves relevant external texts in the external knowledge source through the key element text, and performs similarity matching between the external source vector corresponding to the external text and the dense element vector to obtain a similarity value; Arranging the external source vectors corresponding to the similarity values ​​from large to small according to the similarity values ​​to form an external source vector list; Respective external texts corresponding to the external source vector list are obtained to form external knowledge.

7. The graph construction method based on neural network and sorting mechanism according to claim 1, characterized in that: The self-sorting of the external knowledge and language knowledge by an unsupervised paragraph sorting mechanism to generate a target knowledge graph includes: Acquire the invisible relation r based on the role instruction; The external knowledge and the language knowledge are integrated to form a text set P = {p1, p2, ..., p n }, and sort the text segment p based on the sorting strategy and the invisible relationship. n Perform optimization sorting to obtain combing order; Adding supplementary nodes to the initial knowledge graph in accordance with the combing order to fill the external knowledge, and combing the original language knowledge and the external knowledge to form a target knowledge graph.

8. The graph construction method based on neural network and sorting mechanism as claimed in claim 7, characterized in that: The comparison of the invisible relationship to the text segment p n Perform optimization sorting to obtain a combing order, including: The invisible relationship r and the text segment p are quantified by the preset BERT. n Encode to generate the text vector v pi and the relationship category vector v r ; Calculate the text vector v pi and each relationship category vector v r and arranging the text segment vectors corresponding to each relationship category vector in descending order based on the value of the correlation score to obtain a text segment vector arrangement; The text segments corresponding to the text segment vector arrangement are arranged in the same order to obtain a combing order.

9. A graph construction system based on neural network and sorting mechanism, characterized in that: To implement the graph construction method based on a neural network and a sorting mechanism as described in any one of claims 1 to 8, the system comprises: A large language model is used to generate a knowledge graph framework based on the obtained character instructions; An autoregressive language model, used to obtain language knowledge about the knowledge graph framework, and to fill the knowledge graph framework with the language knowledge to form an initial knowledge graph; A double-tower structure is used to expand external knowledge to obtain external knowledge that matches the initial knowledge graph; A ranking structure is used to self-rank the external knowledge and language knowledge through an unsupervised paragraph ranking mechanism to generate a target knowledge graph.

10. The graph construction system based on neural network and sorting mechanism as claimed in claim 9, characterized in that: The sorting structure includes: A relation extraction unit, configured to obtain an invisible relation r based on the role instruction; A sorting strategy unit is used to fuse the external knowledge and the language knowledge to form a text set P = {p1, p2, ..., p n }, and compare the invisible relationship with the text segment p based on the sorting strategy n Perform optimization sorting to obtain combing order; A target combing unit is used to add supplementary nodes to the initial knowledge graph according to the combing order to fill the external knowledge, and comb the original language knowledge and the external knowledge to form a target knowledge graph.

Images