Information processing method and system based on large model and knowledge graph

Abstract

The invention discloses an information processing method and system based on a large model and a knowledge graph, and belongs to the technical field of information processing. Comprising the following steps: extracting knowledge information from a data source through structured information knowledge extraction, semi-structured information knowledge extraction and unstructured information knowledge extraction; integrating the entity information and the relation information in the extracted knowledge information to construct a knowledge graph; converting tasks in the knowledge graph into cue words, and inputting the cue words into the large model for fine tuning to obtain a fine-tuned large model; and performing information retrieval and generating a result in combination with the knowledge graph and the fine-tuned large model. Compared with the prior art, the method has the beneficial effects that the knowledge graph containing rich entities and relationship information is combined, so that a wider knowledge field can be covered in the processing process, and more diversified, more comprehensive and more accurate retrieval results are generated by combining the generation capability of the large language model.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and more specifically, to an information processing method and system based on a large model and a knowledge graph. Background Art

[0002] With the rapid advancement of Internet technology and the sharp increase in data volume, information retrieval (IR) technology plays a vital role in modern society. Traditional information retrieval methods, such as keyword-based search technology and vector space model (VSM), have gradually exposed a series of challenges when facing massive data. These methods mainly rely on literal matching of keywords and document indexing technology. Users enter keywords or phrases, and the system will search and return documents containing these keywords in the pre-built document index library.

[0003] Although this technical solution is highly efficient in processing simple queries, its limitations become apparent when faced with queries with complex semantics or ambiguity. Traditional methods are mainly based on literal matching of keywords and lack an in-depth understanding of query semantics. Due to the large number of polysemous words and synonyms in natural language, the same keyword may have completely different meanings in different contexts. However, traditional retrieval methods find it difficult to effectively distinguish these different semantics, making it difficult to return accurate and relevant results when faced with complex or ambiguous queries. In addition, traditional information retrieval usually only focuses on the direct match between query keywords and documents, while ignoring the contextual relationship between queries and documents. This context-ignoring processing method limits the system's accurate understanding of query intent, which in turn affects the retrieval effect. In practical applications, user queries often carry specific contextual backgrounds, and traditional methods cannot effectively capture and utilize this information.

[0004] In the related technology, for example, Chinese patent CN115730083A provides a recommendation method based on text content knowledge graph, which processes the text content to form triples and constructs the knowledge graph of the corresponding text; calculates the knowledge graph vector and obtains the knowledge representation vector of the text by multi-vector fusion; uses the attention mechanism to weight the vectors of the knowledge representation vectors of the query text and the candidate text; calculates the click probability between the query text and the candidate text through a neural network model, and makes recommendations based on the click probability. The disadvantage of this solution is that it still relies on keyword matching and document indexing technology, and fails to effectively solve the semantic gap problem in traditional information retrieval. Summary of the invention

[0005] 1. Technical issues to be solved In response to the problem of how to improve the accuracy of search results in information retrieval in the prior art, the present invention provides an information processing method and system based on a large model and a knowledge graph, which can generate more diverse and comprehensive retrieval results and return more accurate retrieval results by combining the generation capability of a large language model during information retrieval.

[0006] 2. Technical solution The purpose of the present invention is achieved through the following technical solutions.

[0007] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present application propose an information processing method and system based on a big model and a knowledge graph to solve the technical problems mentioned in the above background technology section. As a first aspect of the present application, some embodiments of the present application provide an information processing method based on a big model and a knowledge graph, including the following steps: extracting knowledge information from a data source through structured information knowledge extraction, semi-structured information knowledge extraction, and unstructured information knowledge extraction; integrating the entity information and relationship information extracted from the knowledge information to construct a knowledge graph; converting the tasks in the knowledge graph into prompt words and inputting them into the big model for fine-tuning to obtain a fine-tuned big model; combining the knowledge graph and the fine-tuned big model to perform information retrieval and generate results.

[0009] Furthermore, the process of structured information knowledge extraction is: for structured information, the records in the data table are mapped to entities and relationships in the knowledge graph through SQL queries.

[0010] Furthermore, the process of semi-structured information knowledge extraction is: for semi-structured information, regular expressions are used to extract information by matching specific patterns in semi-structured data.

[0011] Furthermore, the process of unstructured information knowledge extraction is: for unstructured information, knowledge is extracted by constructing a Bi-GRU+CRF neural network model; the Bi-GRU+CRF neural network model includes a word embedding layer, a Bi-GRU layer and a CRF layer.

[0012] Furthermore, the process of building a knowledge graph includes: defining the nodes and edges of the graph, storing the graph, and fine-tuning the large model.

[0013] Furthermore, in the process of graph storage, Neo4j graph database is used to store the knowledge graph.

[0014] Furthermore, the steps of fine-tuning the large model include: converting the knowledge graph information into prompt words; converting the knowledge graph information into prompt words and using LoRA to fine-tune the large model.

[0015] Furthermore, the calculation process of fine-tuning the large model specifically includes: converting the knowledge graph information into prompt words, the expression is as follows: ; Among them, T represents the knowledge graph information, Indicates the converted prompt word, Represents the process of converting knowledge graph information into prompt words; The build includes input as And the output is The training dataset D is used to fine-tune the large model LM using the dataset D and the loss function L. The expression of the fine-tuned large model is as follows: ; ; in, Indicates input, Indicates the output, represents the model parameters, Represents a large model In the model parameters The output of the following, represents the fine-tuned parameters; It represents the fine-tuned large model, that is, the result of fine-tuning the large model.

[0016] Furthermore, the information retrieval step includes: converting the knowledge graph structure into prompt words of the big model; the big model analyzes the input question and the prompt words corresponding to the graph structure, generates Cypher search statements, and the big model generates answers based on the search results of the knowledge graph.

[0017] As the second aspect of the present application, some embodiments of the present application provide a system for an information processing method based on a big model and a knowledge graph, including a multi-information knowledge extraction module: extracting knowledge information from a data source through structured information knowledge extraction, semi-structured information knowledge extraction, and unstructured information knowledge extraction; a knowledge graph construction module: integrating entity information and relationship information extracted from the knowledge information to construct a knowledge graph; a fine-tuning module: converting tasks in the knowledge graph into prompt words and inputting them into the big model for fine-tuning to obtain a fine-tuned big model; an information retrieval module: combining the knowledge graph and the fine-tuned big model to perform information retrieval and generate results.

[0018] 3. Beneficial effects Compared with the prior art, the advantages of the present invention are: the information processing method and system based on the big model and knowledge graph of the present invention constructs structured data of entities, relationships and attributes through the knowledge graph, which can accurately express and understand the semantic information in the query; combined with the deep understanding ability of the big language model for natural language text, it can more accurately understand the user's query intention, thereby returning more accurate search results. In addition, since the knowledge graph contains rich entity and relationship information, the present invention, by combining the knowledge graph containing rich entity and relationship information, can cover a wider range of knowledge fields in the processing process, and combined with the generation ability of the big language model, it can achieve the generation of more diverse and more comprehensive and more accurate search results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of an information processing method based on a large model and a knowledge graph in one embodiment of the present invention; Figure 2 An unstructured text knowledge extraction model based on Bi-GRU+CRF neural network in one embodiment of the present invention; Figure 3 This is a flowchart of information retrieval combined with a knowledge graph and a large model after preliminary fine-tuning in one embodiment of the present invention; Figure 4 Schematic diagram of the network structure of GRU in one embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0021] Combination Figures 1 to 3 , an information processing method based on a large model and a knowledge graph of the present invention comprises the following steps: Extract knowledge information from data sources through structured information knowledge extraction, semi-structured information knowledge extraction and unstructured information knowledge extraction; Integrate the entity information and relationship information extracted from the knowledge information to construct a knowledge graph; The tasks in the knowledge graph are converted into prompt words and input into the large model for fine-tuning to obtain the fine-tuned large model; Combine the knowledge graph and the fine-tuned large model to perform information retrieval and generate results.

[0022] In a specific embodiment, the information processing method based on the big model and the knowledge graph is as follows: S1. Multi-information knowledge extraction Multivariate information knowledge extraction is to extract knowledge information from diverse data sources through structured information knowledge extraction, semi-structured information knowledge extraction, and unstructured information knowledge extraction, ensuring that valuable knowledge is captured and transformed from various data sources, laying the foundation for building a knowledge graph. Specifically, it includes the following steps: S101, structured information knowledge extraction: For structured information, SQL query technology is used to map records in the data table to entities and relationships in the knowledge graph. This process achieves seamless mapping of records in the data table to entities and relationships in the knowledge graph, which not only ensures accurate data conversion, but also greatly improves the efficiency of information extraction, providing a solid foundation for subsequent information retrieval.

[0023] Specifically, structured information usually exists in the form of relational databases, tables, or data files with fixed formats, such as CSV, JSON, etc., and its data fields and records are clear and ordered. Structured information is usually stored in relational databases, tables, or data files with fixed formats, such as CSV, JSON, etc. The data fields and records in these data sources are clear and ordered. For relational databases, SQL query statements can be used to directly access and extract these data and convert them into entities and relationships in the knowledge graph.

[0024] Through the process of structured information knowledge extraction, the accuracy of data conversion can be improved, and the efficiency of information extraction can be significantly improved, providing a reliable data foundation for building a knowledge graph.

[0025] S102, semi-structured information knowledge extraction: For semi-structured information, regular expression technology is used to extract the required key information by matching specific patterns in semi-structured data. This process fully adapts to the diversity of semi-structured information and improves the flexibility and accuracy of information extraction.

[0026] Specifically, semi-structured information usually has a certain structure, but this structure is not completely fixed, such as tables and lists in HTML web pages. For semi-structured information, regular expressions are used to match specific patterns in semi-structured data to extract the required key information from these semi-structured data. This processing process improves the flexibility of information extraction and ensures the accuracy of extracted information, providing a rich data source for the construction of knowledge graphs.

[0027] Specific patterns refer to recurring elements or combinations of elements with a specific format or structure in semi-structured data. These patterns define the arrangement and organization of key information in the data so that they can be used to accurately locate and extract the required information. Specific patterns include: (1) Markup patterns: such as HTML or XML tags, which define the beginning and end of data, as well as the hierarchical structure of data. For example, <title> The tag might be used to identify the title of a web page, while< / title> The tag may be used to define a hyperlink.

[0028] (2) Delimiter pattern: such as commas, tabs, and newlines, which are used to separate different parts of the data. In CSV files, commas are usually used as field separators, while newlines are used to separate different records.

[0029] (3) Format patterns: Such as date format (YYYY-MM-DD), phone number format (XXX-XXXX-XXXX), etc. They define the specific format of the data. By matching these formats, the correctness of the data can be verified and useful information can be extracted.

[0030] (4) Content patterns: such as the occurrence patterns of specific words, phrases, or sentences, which may indicate a specific type or attribute of data. For example, in a web page, a string containing "price:" may indicate the price information of a product.

[0031] Regular expressions can be used to match these specific patterns. By defining appropriate regular expressions, key information in semi-structured data can be accurately captured, thereby achieving efficient information extraction.

[0032] S103, Unstructured Information Knowledge Extraction: For unstructured information, a Bi-GRU+CRF neural network model is constructed for knowledge extraction. The model consists of three parts: word embedding layer, bidirectional gated recurrent unit (Bi-GRU) layer, and conditional random field (CRF) layer. The construction of the Bi-GRU+CRF neural network model can greatly improve the accuracy and efficiency of unstructured information extraction.

[0033] Bi-GRU is a bidirectional gated recurrent unit, which is a variant of the GRU model; GRU is a type of recurrent neural network (RNN); compared with the unidirectional GRU, Bi-GRU can consider both forward and backward context information and capture the characteristics of sentences more accurately. CRF is a conditional random field, which is an effective method for modeling time series. The CRF layer is used to decode the features extracted by the Bi-GRU layer and output the label sequence with the highest probability, thereby obtaining the global optimal labeling result.

[0034] Specifically, unstructured information usually refers to free text data, including news reports and books, which have no fixed structure or pattern. A Bi-GRU+CRF neural network model is constructed to extract knowledge from the annotated power network security text, where the Bi-GRU+CRF neural network model consists of three parts: word embedding layer, Bi-GRU layer, and CRF layer. The text knowledge is vectorized through the word embedding layer while retaining the original semantic information; the Bi-GRU layer makes full use of contextual information to obtain a complete feature representation; the CRF layer further optimizes the label sequence, solves the labeling bias problem, and ensures the output of the globally optimal label sequence. Multivariate information knowledge extraction technology provides solid technical support for the construction of accurate and comprehensive knowledge graphs through the meticulous implementation and innovative practice of structured, semi-structured and unstructured information extraction.

[0035] More specifically, the process of building the Bi-GRU+CRF neural network model is as follows: Construct word embedding layer: The word embedding layer is the first step in model construction. Its core function is to vectorize text knowledge and achieve efficient conversion from text to vector.

[0036] Specifically, the word embedding layer uses word vector training tools to vectorize text knowledge and realize the conversion from text to vector. Word embedding is a low-dimensional vectorized expression of words. The trained word vector can learn the original semantic features from the input text knowledge, and word embedding can solve the problem of dimensionality curse.

[0037] In this embodiment, each word in the text is converted into a low-dimensional vector representation through the word embedding layer using the word vector training tool Word2Vec, thereby retaining the original semantic features.

[0038] In a specific embodiment, a set of word vectors are trained through the Word2Vec tool, which can accurately reflect the semantic information of each word in the text. Word embedding technology not only solves the problem of dimensionality curse, but also provides rich input features for subsequent neural network layers. The word embedding layer provides high-quality input features for the model, significantly improving the accuracy and efficiency of knowledge extraction.

[0039] Construct the Bi-GRU layer: The Bi-GRU layer is the core part of the model, which consists of two independent GRU (Gated Recurrent Unit) neural networks, which process the forward and reverse order of the input sequence respectively to capture the bidirectional contextual information in the text.

[0040] GRU is a variant of LSTM with a simpler network structure. GRU only contains a reset gate and an update gate.

[0041] like Figure 4 As shown, this is the network structure of GRU. It can be seen from the figure that and It is calculated by two paths, namely different weights and biases. represents the GRU output, Represents the reset gate.

[0042] The reset gate is used to control the extent to which the state quantity of the previous moment is retained to the current state, while the update gate is used to determine the GRU output at the current moment. .

[0043] Specifically, the reset gate Used to control the state of the last moment The candidate state retained to the current moment is the current state degree; among which, the current state It consists of two parts, namely the state quantity of the last moment and the current input information , reset gate The calculation expression is as follows: (1) Among them, the current status The calculation expression is as follows: (2) like Figure 4 As shown, through the network structure of GRU, we can know: and is calculated by two paths, i.e. different weights and biases, where W r , W c Represents the network pairs in the two paths respectively Different weight matrices learned; U r , U c Represents the network pairs in the two paths respectively Different weight matrices learned; b r Representatives The network bias of b c Representatives The network bias of represents the sigmoid activation function, The value is [0, 1]; when =0, indicating All abandoned; =1 indicates Keep all.

[0044] Specifically, the update gate is used to determine the GRU output , It consists of two parts, namely the state of the last moment and current status , update gate The calculation expression is as follows: (3) GRU Output The calculation expression is as follows: (4) in, W z During the training process, the network The learned weight matrix, U z During the training process, the network The learned weight matrix; b z Express The network bias of represents the sigmoid activation function, The value is [0, 1]; when =0, Z t is equal to 0, that is All abandoned; =1, Z t Equal to 1, indicating Keep all.

[0045] The Bi-GRU neural network structure model is divided into two independent GRUs. The input sequence is input into the two GRU neural networks in forward and reverse order for feature extraction. The word vector formed by concatenating the two output vectors (i.e. the extracted feature vectors) is used as the final feature expression of the word. The Bi-GRU layer significantly improves the model's ability to understand text features by capturing the bidirectional contextual information of the text, thereby improving the accuracy of knowledge extraction.

[0046] Construct the CRF layer: The CRF layer is used to solve the labeling bias problem caused by the Bi-GRU network and ensure the output of the globally optimal labeling sequence.

[0047] The CRF layer uses the interdependence between adjacent tags to optimize the global output label. For a given input X, the CRF layer calculates the probability of output y and finds the optimal tag sequence by maximizing the probability. The input X is the output of the Bi-GRU layer, and the output y is the global optimal tag sequence.

[0048] like Figure 2 The network design shown in the figure, the output of the Bi-GRU layer is used as the input of the CRF layer, and the output of the Bi-GRU layer is used as the input of the CRF layer. Among them, the input X of the CRF layer is [x 1 , x 2 , …, x n ]; the output of the CRF layer y =[ y 1 ,y 2 , …, y n ] .

[0049] For a given input X, the probability that the prediction result is correct It is expressed as: (5) in, represents the emission probability; represents the transition probability; the value of i is [1, n], and n represents the sequence length.

[0050] Simplifying expression (5) into logarithmic form, it can be expressed as: (6) In the prediction stage, the output label sequence with the largest overall probability, i.e. the optimal label sequence It is expressed as: (7) Here, argmax represents the function that predicts y given an input.

[0051] The CRF layer significantly improves the model's global optimization capability for tag sequences by considering the interdependence between adjacent tags, thereby improving the performance of knowledge extraction.

[0052] The construction process of the Bi-GRU+CRF neural network model carefully designs the word embedding layer, Bi-GRU layer and CRF layer. Each layer carries a specific function and works together in the task of knowledge extraction from unstructured information. Through the construction of the model, efficient and accurate knowledge extraction of unstructured information is achieved, providing strong support for the construction of knowledge graphs.

[0053] S2. Building a knowledge graph After completing the multi-information knowledge extraction, this solution further integrates the extracted key information such as entities and relationships into a structured knowledge graph. This step aims to achieve effective organization and efficient storage of information, providing strong support for subsequent information retrieval, intelligent question answering and other applications. The specific construction process is as follows: S201. Define the nodes and edges of the graph: Nodes play a vital role in knowledge graphs, representing entities or abstract concepts that exist in the real world. These entities can be concrete objects (such as people, places, and things) or more abstract concepts (such as ideas and theories). To ensure the uniqueness and identifiability of each node in the graph, this solution assigns a unique identifier to each node. In addition, nodes may also contain detailed information or attributes related to the entity or concept, such as name, type, and description, which further enriches the content of the graph.

[0054] Edges are used to represent the relationship between nodes, that is, the semantic connection between entities. Edges are bridges connecting two nodes, and they are used to represent the semantic connection or relationship between nodes. In the knowledge graph, edges indicate the type of association between two entities. In order to clearly identify the type of each relationship, each edge is assigned a label to identify the type of relationship. At the same time, edges may also contain attributes or weights that describe this relationship. This additional information helps to understand the relationship between nodes more deeply.

[0055] S202, graph storage: In order to efficiently store and manage knowledge graph data, this solution uses Neo4j graph database to store knowledge graph. In Neo4j, nodes and edges can be described and operated through Cypher query language, which makes data storage and retrieval extremely simple. Node creation is implemented through CREATE command. When creating a node, you can attach attributes to store additional information related to the node. Similarly, edge creation establishes connections between nodes through MATCH and CREATE commands and specifies the type and attributes of the relationship. This flexible data creation method enables the knowledge graph to be quickly expanded and updated.

[0056] S3. Fine-tune the large model The large model is initially fine-tuned through the constructed knowledge graph. First, the information contained in the knowledge graph is divided into three tasks: "classification", "relationship prediction" and "entity prediction". These tasks are then converted into prompt words and input into the large model for fine-tuning, allowing the large model to learn the graph information.

[0057] "Task" refers to the specific goal or activity of training a large model using knowledge graph information. These tasks are designed to enable the large model to understand, process and apply the information in the knowledge graph, thereby improving its performance and accuracy in related fields or problems.

[0058] Specifically, three tasks: "classification", "relationship prediction" and "entity prediction" constitute the core content of fine-tuning the large model. Among them, the classification task involves verifying whether the structure (entity-relationship-entity) in the knowledge graph correctly expresses the knowledge in the real world. The model needs to judge whether the given structure is logically correct and consistent with the facts. For example, judge whether the structure of "a certain brand manufacturer-production-drone" is correct. In the relationship prediction task, given the head entity and the tail entity, the model needs to infer the connection or relationship between the two. For example, given "a certain brand manufacturer" and "drone", the model needs to recognize that the connection between them is "production". The entity prediction task is divided into two subtasks. One is to predict the corresponding tail entity given the head entity and the relationship; the other is to reversely predict the head entity given the relationship and the tail entity. For example, given "a certain brand manufacturer" and "production", the model needs to predict "drone" as the tail entity.

[0059] These tasks are converted into prompt words and input into the large model, allowing the model to learn the information in the knowledge graph during the training process. The design and execution of tasks are key steps in fine-tuning the large model, which determines what types of knowledge the model can understand and apply, and how to apply this knowledge in different situations.

[0060] More specifically, the computational process for fine-tuning a large model is as follows: Convert knowledge graph information into prompt words: Use T to represent the information of the knowledge graph, such as the relationship between entities; use Indicates the converted prompt word, The expression is as follows: (8) Among them, T represents the knowledge graph information, Indicates the converted prompt word, Represents the process of converting knowledge graph information into prompt words; Construct the training dataset D required for fine-tuning: The training data set D contains a series of input-output pairs; Output: prompt word + question are task-specific goals, such as target entities and entity categories.

[0061] Fine-tuning the large model: Fine-tuning the large model means using the training dataset D and loss function L to fine-tune the large model LM. In the fine-tuning process, the model parameters are usually optimized. To minimize the value of the loss function L. The expression of fine-tuning the large model is as follows: (9) (10) in, Indicates input, Indicates the output, represents the model parameters, Represents a large model In the model parameters The output of the following, represents the fine-tuned parameters; It represents the fine-tuned large model, that is, the result of fine-tuning the large model.

[0062] In a specific embodiment, the step of fine-tuning the large model includes: S301, converting the knowledge graph information into prompt words, converting the knowledge graph information into prompt words, the conversion aims to convert the complex knowledge graph structure into a natural language form that can be understood and processed by the large model. The specific conversion process is as follows: (1) Refine the classification task: The knowledge graph is composed of the structure of (entity, relationship, entity). In the classification task, it is necessary to verify whether such a structure correctly expresses the knowledge in the real world. For example, when faced with the structure: (a certain brand manufacturer-production-drone), the task is to verify whether this combination is logically correct and consistent with the facts. The prompt word template for the conversion is not limited to directly asking for correctness, but can also be designed to be more interactive, such as "Please judge the truth of the following statement: Does a certain brand manufacturer produce drones? Your answer should be 'correct' or 'wrong'." The large model should reply "correct".

[0063] (2) Deepening the relationship prediction task: Given a head entity and a tail entity, the purpose of the relationship prediction task is to infer the relationship between the two. In the relationship prediction task, two entities are known - the head entity and the tail entity, and the goal is to infer the possible relationship between them. For example, given a head entity (a certain brand manufacturer) and a tail entity (drone), the system needs to recognize that the relationship between them is "production". The prompt word template for the conversion is "Please infer based on the context that the relationship between a certain brand manufacturer and drones best fits which of the following? 'Birth', 'competition'...'production'". The output of the large model should be "production".

[0064] (3) Entity prediction task expansion: The entity prediction task is divided into two subtasks: one is to predict the corresponding tail entity given the head entity and relationship; the other is to reversely predict the head entity given the relationship and tail entity. For example, given the head entity (a certain brand manufacturer) and the relationship (production), the task is that the system needs to predict the tail entity "drone". In order to improve the flexibility and creativity of the model, the prompt words can be designed to be more open, such as "What does a certain brand manufacturer produce?" The output of the large model is "drone". For reverse prediction, you can design questions such as "Which company or brand produces drones, and this production relationship matches the word 'production'?" The model needs to reply "a certain brand manufacturer" and so on.

[0065] S302. Using the prompt words and answers constructed in S301, the large model is preliminarily fine-tuned using the LoRA (Low-Rank Adaptation) method.

[0066] At this stage, the carefully designed prompt words and their accurate answers in S301 constitute valuable training data in the fine-tuning process. As a lightweight and efficient model fine-tuning method, LoRA is suitable for adaptive adjustment for specific tasks while maintaining the original capabilities of the large model.

[0067] Specifically, LoRA introduces low-rank weight adjustment matrices in certain layers of the large model. The dimensions of these matrices are much lower than the original weight matrices, which significantly reduces the number of parameters that need to be trained. In this way, accurate fine-tuning of the large model can be achieved without significantly increasing the computational burden. During the fine-tuning process, the prompt words and their answers constructed in S301 are first input into the large model as training samples, and then these samples are iteratively trained using LoRA technology. During the training process, the model gradually adjusts its internal weights according to the correspondence between the prompt words and the answers to better adapt to the needs of specific tasks. It is worth noting that fine-tuning the large model not only focuses on the accuracy of the model, but also on the generalization ability of the model. Not only does it improve the performance of the model on specific tasks, but it also maintains the stability and reliability of the model when processing other tasks.

[0068] S4. Information Retrieval Combine the knowledge graph and the large model after preliminary fine-tuning for information retrieval. Based on the constructed knowledge graph, this solution further combines the large model for information retrieval to deepen the efficiency and accuracy of information retrieval. With its powerful representation and learning capabilities, the large model can accurately understand the user's query intent and quickly locate relevant information in the vast network of the knowledge graph. Therefore, combining the knowledge graph and the large model for information retrieval not only improves the accuracy and efficiency of information retrieval, but also greatly enriches the diversity and depth of the retrieval results. The steps for combining the knowledge graph and the large model for information retrieval include: S401, converting the knowledge graph structure into prompt words of the big model. The big model reads the knowledge graph, records the entities, attributes and relationships in the graph, and uses this information as prompt words of the big model together with the question raised by the user as input.

[0069] Specifically, the rich structure of the knowledge graph is carefully converted into prompt words that the big model can understand. The big model first reads the knowledge graph in depth, carefully recording the entities, detailed attributes and complex relationships in the graph. Subsequently, this information is cleverly woven into prompt words, which together with the user's specific query questions constitute the input of the big model. Through this transformation, the big model is able to have a more comprehensive and in-depth knowledge background when understanding user queries. This not only enhances the model's ability to grasp user intent, but also provides strong support for it to quickly locate relevant information in the knowledge graph.

[0070] S402, the big model analyzes the prompt words corresponding to the user input question and the graph structure, and generates a Cypher search statement. The big model will first analyze the semantics of the user's question and identify the key information and intent. Then, it will determine the entities, relationships, and possible attributes to be queried based on the prompt words converted in step 3-1. Finally, the big model will automatically generate one or more Cypher search statements based on this information, which are used to retrieve the result data related to the user's question from the knowledge graph.

[0071] Specifically, the big model first performs a deep semantic analysis on the questions input by the user to identify the key information and intent. Then, based on the prompt words generated in step S401, the entities, relationships and possible attributes to be queried are accurately locked. On this basis, the big model automatically generates one or more Cypher search statements with its powerful generation capabilities, which can accurately retrieve data closely related to the user's questions from the knowledge graph. In this process, the intelligent analysis and generation capabilities of the big model ensure the accuracy and efficiency of the search statements. This not only greatly improves the speed of information retrieval, but also significantly improves the pertinence and relevance of the search results.

[0072] S403. The big model generates answers to questions based on the search results of the knowledge graph. After retrieving results related to the user's question through the knowledge graph, the big model will enter the stage of answer generation. In this step, the big model will conduct in-depth analysis and processing of the search results to extract the information that the user is most concerned about, and summarize, sort or format the search results to ensure the accuracy and readability of the answers. At the same time, the big model will also appropriately interpret and reason the answers based on the context and intent of the question to provide a more comprehensive and in-depth answer.

[0073] Specifically, after successfully retrieving the results related to the user's question, the big model enters the critical stage of answer generation. It first conducts in-depth analysis and processing of the search results to accurately extract the information that the user is most concerned about. Subsequently, the big model summarizes, sorts, or formats the search results to ensure that the answers are clear, accurate, and easy to understand. At the same time, the big model will also appropriately explain and reason about the answers based on the context and intent of the question, thereby providing users with more comprehensive and in-depth answers. The big model not only provides users with accurate and concise answers, but also enhances the depth and breadth of the answers through explanation and reasoning. This not only improves the user experience, but also further demonstrates the huge potential of the combination of knowledge graphs and big models in the field of information retrieval.

[0074] By using the large model to search the graph, the large model can complete multimodal information processing.

[0075] In a specific embodiment, the large model can process information queries in different modalities, such as text, images, and voice, and achieve information integration.

[0076] Among them, the input includes text, images, audio and video. For text input, the large model can directly process natural language text and identify entities and relationships; for image input, using image recognition technology (such as convolutional neural network CNN), the large model can extract key information from the image, such as objects, scenes, actions, etc., which can be regarded as part of the entity or relationship; for audio or video input, through audio processing (such as speech recognition) and video analysis (such as frame extraction, action recognition), the large model can extract useful information from audio and video, which can also be mapped to entities and relationships in the knowledge graph. The large model is able to integrate information from different modalities to form a comprehensive understanding. For example, it can combine text descriptions with objects in images to identify more complex entities and relationships.

[0077] Combination Figures 1 to 3 , an information processing system based on a large model and a knowledge graph of the present invention comprises: Multi-information knowledge extraction module: extracts knowledge information from data sources through structured information knowledge extraction, semi-structured information knowledge extraction and unstructured information knowledge extraction; Constructing knowledge graph module: Integrate the entity information and relationship information extracted from the knowledge information to construct a knowledge graph; Fine-tuning module: converts the tasks in the knowledge graph into prompt words and inputs them into the large model for fine-tuning to obtain the fine-tuned large model; Information retrieval module: combines the knowledge graph and the fine-tuned large model to perform information retrieval and generate results.

[0078] The information processing method and system based on the big model and knowledge graph of the present invention improves the way of knowledge extraction in the process of building the knowledge graph and combines the knowledge graph with the big model. On the one hand, the big model is fine-tuned through the knowledge graph so that the big model can learn the information in the knowledge graph; on the other hand, the big model helps the knowledge graph to realize multimodal input, Cypher statements and result polishing. It realizes efficient and accurate extraction of knowledge from multivariate information, and construction of knowledge graph, and then combines the big model for efficient information retrieval, improving the accuracy and efficiency of information retrieval.

[0079] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. An information processing method based on a large model and a knowledge graph, characterized in that: The following steps are involved: Extract knowledge information from data sources through structured information knowledge extraction, semi-structured information knowledge extraction and unstructured information knowledge extraction; Integrate the entity information and relationship information extracted from the knowledge information to construct a knowledge graph; The tasks in the knowledge graph are converted into prompt words and input into the large model for fine-tuning to obtain the fine-tuned large model; Combine the knowledge graph and the fine-tuned large model to perform information retrieval and generate results.

2. The information processing method based on a large model and a knowledge graph according to claim 1, characterized in that: The specific process of structured information knowledge extraction is: for structured information, the records in the data table are mapped to entities and relationships in the knowledge graph through SQL queries.

3. The information processing method based on a large model and a knowledge graph according to claim 1, characterized in that: The specific process of semi-structured information knowledge extraction is: for semi-structured information, regular expressions are used to extract information by matching specific patterns in semi-structured data.

4. The information processing method based on a large model and a knowledge graph according to claim 1, characterized in that: The specific process of unstructured information knowledge extraction is as follows: for unstructured information, knowledge is extracted by constructing a Bi-GRU+CRF neural network model; The Bi-GRU+CRF neural network model includes a word embedding layer, a Bi-GRU layer, and a CRF layer.

5. The information processing method based on a large model and a knowledge graph according to claim 1, characterized in that: The process of building a knowledge graph includes: defining the nodes and edges of the graph, storing the graph, and fine-tuning the large model.

6. The information processing method based on a large model and a knowledge graph according to claim 5, characterized in that: In the process of graph storage, Neo4j graph database is used to store the knowledge graph.

7. The information processing method based on a large model and a knowledge graph according to claim 5, characterized in that: The steps of fine-tuning the large model include: converting the knowledge graph information into prompt words; converting the knowledge graph information into prompt words and using LoRA to fine-tune the large model.

8. The information processing method based on a large model and a knowledge graph according to claim 5, characterized in that: The calculation process of fine-tuning the large model specifically includes: The knowledge graph information is converted into prompt words. The expression is as follows: ； T represents the knowledge graph information, Indicates the converted prompt word, Represents the process of converting knowledge graph information into prompt words; The build includes input as And the output is The training dataset D is used to fine-tune the large model LM using the dataset D and the loss function L. The expression of the fine-tuned large model is as follows: ； ； Indicates input, Indicates the output, represents the model parameters, Represents a large model In the model parameters The output of the following, represents the fine-tuned parameters; It represents the fine-tuned large model, that is, the result of fine-tuning the large model.

9. The information processing method based on a large model and a knowledge graph according to claim 1, characterized in that: The steps of information retrieval include: converting the knowledge graph structure into prompt words of the big model; the big model analyzes the input question and the prompt words corresponding to the graph structure, generates Cypher search statements, and the big model generates answers based on the search results of the knowledge graph.

10. A system based on the information processing method based on a large model and a knowledge graph according to any one of claims 1 to 9, characterized in that: It includes a multi-information knowledge extraction module: extracting knowledge information from data sources through structured information knowledge extraction, semi-structured information knowledge extraction and unstructured information knowledge extraction; Constructing knowledge graph module: Integrate the entity information and relationship information extracted from the knowledge information to construct a knowledge graph; Fine-tuning module: converts the tasks in the knowledge graph into prompt words and inputs them into the large model for fine-tuning to obtain the fine-tuned large model; Information retrieval module: combines the knowledge graph and the fine-tuned large model to perform information retrieval and generate results.

Images