A query method integrating AI big model and knowledge graph

Abstract

The present invention discloses a query method integrating an AI big model and a knowledge graph, and relates to the field of knowledge graph construction and query technology, including collecting multimodal data and integrating a knowledge base; performing entity recognition and relationship extraction based on the multimodal data in the knowledge base to obtain structured triple data, and preprocessing the triple data; constructing a knowledge graph based on the preprocessed triple data; constructing a query parsing model based on the knowledge graph, converting natural language queries into machine-understandable query languages, and obtaining parsed query instructions; constructing a query path based on the parsed query instructions, and optimizing the query path; obtaining the most relevant result set in the optimized query path, and outputting the most relevant result set as the final query result.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph construction and query technology, and in particular to a query method integrating an AI big model and a knowledge graph. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the progress of natural language processing (NLP) and deep learning algorithms, the fields of information retrieval and knowledge management have ushered in great changes. Traditional information retrieval systems mainly rely on keyword matching and simple semantic analysis. However, this method often cannot accurately understand the user's query intention, especially when faced with complex multimodal data and deep knowledge associations. In recent years, knowledge graphs, as a structured form of knowledge representation, have been favored for their powerful semantic representation capabilities and efficient query performance. Knowledge graphs can not only capture the complex relationships between entities, but also provide a solid foundation for intelligent question and answer, recommendation systems and personalized services.

[0003] However, the existing knowledge graph construction and query technologies still have several limitations. First, the construction process of the knowledge graph often relies on a single modal data source, such as text data, which limits the comprehensiveness and richness of the knowledge graph. The fusion of multimodal data, especially the integration of text, images, audio and video, can provide a more complete and intuitive knowledge representation. Secondly, the accuracy of natural language query parsing directly affects the quality of the query results. The existing query parsing models produce errors because they cannot fully understand complex natural language expressions. In addition, the efficiency of knowledge graph queries and the relevance of results are also issues to be resolved. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a query method that integrates AI big models and knowledge graphs to solve several limitations of existing knowledge graph construction and query technologies. First, the construction process of the knowledge graph often relies on a single modal data source, such as text data, which limits the comprehensiveness and richness of the knowledge graph. The fusion of multimodal data, especially the integration of text, images, audio and video, can provide a more complete and intuitive knowledge representation. Secondly, the accuracy of natural language query parsing directly affects the quality of the query results. Existing query parsing models produce errors because they cannot fully understand complex natural language expressions. In addition, the efficiency of knowledge graph queries and the relevance of results are also problems to be solved.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a query method integrating an AI big model and a knowledge graph, which includes:

[0008] Collect multimodal data and integrate knowledge base;

[0009] Based on the multimodal data in the knowledge base, entity recognition and relationship extraction are performed to obtain structured triple data, and the triple data is preprocessed;

[0010] Build a knowledge graph based on the preprocessed triple data;

[0011] Build a query parsing model based on the knowledge graph, convert natural language queries into machine-understandable query language, and obtain parsed query instructions;

[0012] Build a query path based on the parsed query instructions and optimize the query path;

[0013] The most relevant result set is obtained in the optimized query path, and the most relevant result set is output as the final query result.

[0014] As a preferred solution of the query method integrating AI big model and knowledge graph described in the present invention, wherein: the multimodal data is collected and the knowledge base is integrated, and the specific steps are as follows:

[0015] Collect multimodal data of text, images, audio, and video;

[0016] Calculate the data points in multimodal data Relevance score , the expression is:

[0017] ;

[0018] in, Represents data points The embedding vector of represents the embedding vector of the target topic, represents the cosine similarity, A weight factor indicating the relative importance of topic relevance, represents the latent Dirichlet allocation model LDA;

[0019] Calculate data points Quality score , the expression is:

[0020] ;

[0021] in, Represents the output of the grammatical error detection model, ranging from 0 to 1, 1 means no grammatical errors at all, and vice versa, it means all grammatical errors. Represents the output of the definition detection model, ranging from 0 to 1, 1 means the image definition is the highest, and vice versa, it means the image definition is the lowest. Represents the output of the noise suppression model, ranging from 0 to 1, 1 means the audio has the least noise, and vice versa, the audio has the most noise. Represents the output of the resolution detection model, ranging from 0 to 1, 1 means the video quality is the best, and vice versa, the worst;

[0022] Calculate data points Safety score , using pre-trained sensitive word filtering models and privacy compliance models to detect data points Whether it contains sensitive information and elements that violate the privacy policy. The calculation expression is:

[0023] ;

[0024] in, Represents the output of the sensitive word filtering model, ranging from 0 to 1, 1 means it contains the most sensitive words, and vice versa, it does not contain any. Represents the output of the privacy compliance model, ranging from 0 to 1, where 1 means full compliance with the privacy policy, and vice versa, no compliance with the privacy policy at all;

[0025] Score the collected data and get a comprehensive score, expressed as:

[0026] ;

[0027] in, represents the comprehensive score, Represents data points The relevance score, Represents data points The quality score, Represents data points Safety score;

[0028] The qualified data will be included in the knowledge base based on the comprehensive scores and requirements.

[0029] As a preferred solution of the query method integrating AI big model and knowledge graph described in the present invention, wherein: entity recognition and relationship extraction are performed based on multimodal data in the knowledge base to obtain structured triple data, and the triple data is preprocessed. The specific steps are:

[0030] Extract multimodal data including text, images, audio and video from the knowledge base;

[0031] For text data, the pre-trained named entity recognition model Et and dependency analysis model Dt are used to identify entities in the text and the relationships between them;

[0032] For image data, a deep convolutional neural network Ei is used for object detection and recognition, and an image relationship detection model Di is used to capture the relationship between entities in the image;

[0033] For audio data, after being converted into text by the speech recognition model Ea, the named entity recognition model Et and the dependency analysis model Dt are used for entity recognition and relationship extraction;

[0034] For video data, it is first segmented into frames by the video segmentation model Sv, and then Ei and Di are applied to each frame, combined with time series analysis Tv to capture the relationship between consecutive frames.

[0035] As a preferred solution of the query method integrating the AI ​​big model and the knowledge graph described in the present invention, the knowledge graph is constructed based on the preprocessed triple data, and the specific steps are as follows:

[0036] Use triple construction to convert all identified data into unified structured triple data ,in, Indicates the subject, Indicates the predicate, Indicates the object;

[0037] Preprocess the triplet data;

[0038] The preprocessed triple data is mapped to the graph database to form a queryable knowledge graph structure.

[0039] As a preferred solution of the query method integrating the AI ​​big model and the knowledge graph described in the present invention, wherein: a query parsing model is constructed based on the knowledge graph, natural language queries are converted into machine-understandable query languages, and the parsed query instructions are obtained. The specific steps are:

[0040] Set the knowledge graph as KG, the natural language query as NLQ, and build a query parsing model based on the knowledge graph KG;

[0041] The natural language query NLQ is input into the natural language understanding function, which parses the query and returns the intent, entity set, and relationship set. The expression is:

[0042] ;

[0043] in, represents the natural language understanding function, Indicates the intent of the query. represents the set of all entities extracted from the query, Represents a set of relations;

[0044] The query mapping function receives the parsed components and the entire knowledge graph KG, and converts them into a specific query instruction Q executed on the knowledge graph. The expression is:

[0045] ;

[0046] in, Represents the query mapping function;

[0047] Mapping the query to a function and natural language understanding functions Combined, the expression is:

[0048] ;

[0049] in, Indicates the query result. Represents the query parsing model, Represents a natural language query Applying Natural Language Understanding Functions As a result, Indicates the use of query mapping function ,Will Output and knowledge graph Combine to generate the final query instruction .

[0050] As a preferred solution of the query method integrating AI big model and knowledge graph described in the present invention, wherein: the query path is constructed based on the parsed query instruction, and the specific steps are as follows:

[0051] Use natural language understanding (NLU) to parse the query instruction Q and identify the entities and relationships in the query;

[0052] Map the entities and relationships in the query to the corresponding nodes and edges in the knowledge graph KG;

[0053] Based on the mapped entities and relationships, find the path from the starting entity to the target entity in the knowledge graph, i.e., the candidate path;

[0054] Set the starting node and target node in the query;

[0055] Starting from the starting node, explore all connected nodes and check whether they match the query intent and entities;

[0056] The path is expanded by traversing all edges through the knowledge graph until the target node is reached. In this process, the nodes and edges passed are recorded to form a path set.

[0057] As a preferred solution of the query method integrating AI big model and knowledge graph described in the present invention, the specific steps of optimizing the query path are as follows:

[0058] An evaluation function is constructed based on the path set to calculate the evaluation score of each path. The evaluation function expression is:

[0059] ;

[0060] in, Indicates the path The evaluation score of Represents an index variable, e represents a path One edge in Indicates the path The set of edges contained in , represents the relevance score of edge e, Indicates the path Length;

[0061] Will The path to the maximum value as the initial query path.

[0062] As a preferred solution of the query method integrating the AI ​​big model and the knowledge graph described in the present invention, wherein: the most relevant result set is obtained in the optimized query path, and the most relevant result set is output as the final query result. The specific steps are:

[0063] The result set scoring function is constructed based on the optimized path. The expression is:

[0064] ;

[0065] in, Representation Node The result set scoring function is Represents an index variable, Indicates The weight coefficient of the edge, represents the similarity function, Indicates The type of edge, Indicates the query intent, α e Indicates The attenuation coefficient of the edge, Represents edge To Node The distance, that is, the position on the query path, Representation Node The shortest path distance to the query start node;

[0066] Traverse all nodes on the path, that is, calculate the initial query path for each node value;

[0067] Set the threshold θ;

[0068] All >The nodes of θ are selected into the result set;

[0069] For nodes in the result set, The values ​​are sorted in descending order, and the top-ranked nodes are the results most relevant to the query;

[0070] Select the first K nodes from the sorted result set as the final query result output.

[0071] In a second aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the query method integrating the AI ​​big model and the knowledge graph as described in the first aspect of the present invention.

[0072] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the query method integrating an AI big model and a knowledge graph as described in the first aspect of the present invention.

[0073] The beneficial effects of the present invention are as follows: the present invention constructs a comprehensive knowledge base by collecting and integrating multimodal data, and utilizes advanced entity recognition and relationship extraction techniques to convert unstructured data into structured triple data, thereby constructing a multimodal knowledge graph. In addition, the present invention also introduces a query parsing model, which can efficiently convert natural language queries into machine-understandable query instructions, while optimizing the query path to improve the query speed and the accuracy of the results. By obtaining the most relevant result set in the optimized query path, the present invention improves the efficiency and accuracy of the query, especially when processing complex queries and large-scale knowledge graphs. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0075] Figure 1 This is a flowchart of the query method integrating the AI ​​big model and the knowledge graph in Example 1.

[0076] Figure 2 This is a flow chart of incorporating triple data into the knowledge base in Example 1. DETAILED DESCRIPTION

[0077] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0078] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0079] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0080] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a query method integrating an AI big model and a knowledge graph, comprising the following steps:

[0081] S1 collects multimodal data and integrates the knowledge base;

[0082] Collect multimodal data of text, images, audio, and video;

[0083] Calculate the data points in multimodal data Relevance score , the expression is:

[0084] ;

[0085] in, Represents data points The embedding vector of represents the embedding vector of the target topic, represents the cosine similarity, A weight factor indicating the relative importance of topic relevance, represents the latent Dirichlet allocation model LDA;

[0086] Calculate data points Quality score , the expression is:

[0087] ;

[0088] in, Represents the output of the grammatical error detection model, which uses NLP technology to identify grammatical errors in sentences. Represents the output of the clarity detection model, which uses edge detection to detect the clarity of the image. Represents the output of the noise suppression model, which uses a GRU-based sequence model to identify and eliminate noise in audio. Represents the output of the resolution detection model, which is based on the CNN model to evaluate the resolution and quality of the video;

[0089] Calculate data points Safety score , using pre-trained sensitive word filtering models and privacy compliance models to detect data points Whether it contains sensitive information and elements that violate the privacy policy. The calculation expression is:

[0090] ;

[0091] in, Represents the output of the sensitive word filtering model, Represents the output of the privacy compliance model; the collected data is scored to obtain a comprehensive score, which is expressed as:

[0092] ;

[0093] in, represents the comprehensive score, Represents data points The relevance score, Represents data points The quality score, Represents data points Safety score;

[0094] According to the comprehensive scores and requirements, the qualified data will be included in the knowledge base;

[0095] when >1, indicating a data point Highly relevant and of good quality, as well as highly secure, are formally included in the knowledge base;

[0096] When 0< When ≤1, it indicates a data point It has certain value and is manually reviewed to decide whether to adopt it;

[0097] when When ≤0, it means the data point Those with poor relevance, low quality and security risks will not be included in the knowledge base.

[0098] S2 performs entity recognition and relationship extraction based on the multimodal data in the knowledge base to obtain structured triple data and preprocess the triple data;

[0099] Extract multimodal data including text, images, audio and video from the knowledge base;

[0100] For text data, the pre-trained named entity recognition model Et and dependency analysis model Dt are used to identify entities in the text and the relationships between them;

[0101] For image data, a deep convolutional neural network Ei is used for object detection and recognition, and an image relationship detection model Di is used to capture the relationship between entities in the image;

[0102] For audio data, after being converted into text through the speech recognition model Ea, Et and Dt are used for entity recognition and relationship extraction;

[0103] For video data, it is first segmented into frames by the video segmentation model Sv, and then Ei and Di are applied to each frame, combined with time series analysis Tv to capture the relationship between consecutive frames.

[0104] S3 builds a knowledge graph based on the preprocessed triple data;

[0105] Use triple construction to convert all identified data into unified structured triple data ,in, Indicates the subject, Indicates the predicate, Indicates the object;

[0106] Preprocess the triplet data, which includes cleaning redundant information, eliminating outliers and standardizing;

[0107] Map the preprocessed triple data into a graph database to form a queryable knowledge graph structure;

[0108] Initialize the graph database using Neo4j to ensure that it supports heterogeneous node and relationship types;

[0109] Each preprocessed triple data Mapped to nodes and relationships in the graph database, subject and object Become a node, predicate Become the relationship connecting these two nodes;

[0110] Store additional properties for each node and relationship, including type and source data type;

[0111] Apply the uniqueness of the graph database to constrain and prevent data duplication;

[0112] Use the predicate as an index search to speed up the query process.

[0113] S4 builds a query parsing model based on the knowledge graph, converts natural language queries into machine-understandable query language, and obtains parsed query instructions;

[0114] Set the knowledge graph as KG, the natural language query as NLQ, and build a query parsing model based on the knowledge graph KG;

[0115] The natural language query NLQ is input into the natural language understanding function, which parses the query and returns the intent, entity set, and relationship set. The expression is:

[0116] ;

[0117] in, Represents the natural language understanding function, which is responsible for processing and parsing natural language queries and identifying the semantic information of intent, entities, and relationships. Indicates the intent of the query, including search, calculation, and comparison. represents the set of all entities extracted from the query, including names, places, and dates, Represents a set of relationships, which are relationships between entities, including belongs to, is located in, and created by;

[0118] The query mapping function receives the parsed components and the entire knowledge graph KG, and converts them into a specific query instruction Q executed on the knowledge graph. The expression is:

[0119] ;

[0120] in, Represents the query mapping function, which maps the result of natural language understanding to the structure of the knowledge graph and generates query instructions that can be executed on the knowledge graph;

[0121] Mapping the query to a function and natural language understanding functions Combined, the expression is:

[0122] ;

[0123] in, Indicates the query result. Represents the query parsing model, Represents a natural language query Applying Natural Language Understanding Functions The result is that The set of intents, entities, and relations extracted from Indicates the use of query mapping function ,Will Output and knowledge graph Combine to generate the final query instruction ;

[0124] The natural language understanding function is responsible for processing and parsing the natural language query NLQ and identifying the semantic information of intent, entities and relationships;

[0125] The query mapping function matches the results of natural language understanding with the structure of the knowledge graph KG to generate query instructions that can be executed on the knowledge graph.

[0126] S5 builds a query path based on the parsed query instructions and optimizes the query path;

[0127] Use natural language understanding (NLU) to parse the query instruction Q and identify the entities and relationships in the query;

[0128] Map the entities and relationships in the query to the corresponding nodes and edges in the knowledge graph KG;

[0129] Based on the mapped entities and relationships, find the path from the starting entity to the target entity in the knowledge graph, i.e., the candidate path;

[0130] Set the starting node and target node in the query, which can be set by yourself;

[0131] Starting from the starting node, explore all connected nodes and check whether they match the query intent and entities;

[0132] Expand the path by traversing all edges through the knowledge graph until the target node is reached. In this process, the nodes and edges passed through are recorded to form a path set.

[0133] Not all candidate paths are valid. An evaluation function is constructed based on the path set to calculate the evaluation score of each path. The higher the evaluation score, the more effective the path is. That is, as the initial query path, the evaluation function expression is:

[0134] ;

[0135] in, Indicates the path The evaluation score of Represents an index variable, e represents a path One edge in Indicates the path The set of edges contained in , represents the relevance score of edge e, Indicates the path Length;

[0136] Will The path to the maximum value As the initial query path;

[0137] The relevance score of edge e , the calculation expression is:

[0138] ;

[0139] in, Indicates The weight coefficient of the edge, Indicates The type of edge, Indicates the intent of the query. represents the similarity function, The value range is from 0 to 1. The closer to 1, the more similar they are, and vice versa.

[0140] S6 obtains the most relevant result set in the optimized query path, and outputs the most relevant result set as the final query result;

[0141] The result set scoring function is constructed based on the optimized path. The expression is:

[0142] ;

[0143] in, Representation Node The result set scoring function is used to measure the node Relative to the relevance and importance of the query, higher Value means node Contains the information required by the user's query, Represents an index variable, Indicates The weight coefficient of the edge, represents the similarity function, Indicates The type of edge, Indicates the query intent, α e Indicates The attenuation coefficient of the edge, Represents edge To Node The distance, that is, the position on the query path, Representation Node The shortest path distance to the query start node;

[0144] Traverse all nodes on the path, that is, calculate the initial query path for each node value;

[0145] Set the threshold θ;

[0146] All Nodes >θ are selected into the result set, and the threshold θ can be adjusted according to needs to control the size and quality of the result set;

[0147] For nodes in the result set, The values ​​are sorted in descending order, and the top-ranked nodes are the results most relevant to the query;

[0148] The first K nodes are selected from the sorted result set as the final query result output, and the size of K can be adjusted according to specific user needs.

[0149] This embodiment also provides a computer device, which is suitable for the query method of integrating AI big model and knowledge graph, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the query method of integrating AI big model and knowledge graph proposed in the above embodiment.

[0150] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0151] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by the processor, the query method for integrating the AI ​​large model and the knowledge graph proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, disk or optical disk.

[0152] In summary, the present invention collects and integrates multimodal data to build a comprehensive knowledge base, uses advanced entity recognition and relationship extraction techniques to convert unstructured data into structured triple data, and then builds a multimodal knowledge graph. In addition, the present invention also introduces a query parsing model, which can efficiently convert natural language queries into machine-understandable query instructions, and optimize the query path to improve the query speed and the accuracy of the results. By obtaining the most relevant result set in the optimized query path, the present invention improves the efficiency and accuracy of the query, especially when processing complex queries and large-scale knowledge graphs.

[0153] Example 2, referring to Table 1, is the second example of the present invention. In order to further verify the technical solution of the present invention, experimental simulation data of the query method integrating the AI ​​big model and the knowledge graph are provided.

[0154] In order to verify the effectiveness of the content of this invention, an experiment was conducted to collect multimodal data, build a knowledge graph, and optimize the query path by integrating an AI big model to improve the query efficiency and the relevance of the results. The following steps were adopted in the experiment:

[0155] Collect multimodal data including text, images, audio and video from the Internet, totaling 5,000 samples, including 1,000 text data, 1,500 images, 1,000 audio data and 1,500 video data. Each type of data is evaluated using the relevance score R(x), quality score Q(x) and security score S(x). Finally, calculate the comprehensive score F(x) and select the qualified data for subsequent processing.

[0156] Use named entity recognition models and dependency analysis models to process text data; use deep convolutional neural networks to detect objects in image data; use speech recognition models to convert audio into text and then perform entity recognition; and segment video data into frames before performing object detection and relationship analysis.

[0157] Preprocess the identified and extracted triple data, construct a knowledge graph, and store it in the Neo4j graph database;

[0158] Develop query parsing models to convert natural language queries into machine-understandable query instructions, optimize query paths through evaluation functions, and obtain the most relevant result sets.

[0159] The details are shown in Table 1:

[0160] Table 1 Experimental record table

[0161]

[0162] The above are some of the data samples collected in the experiment and their evaluation results;

[0163] The data ID is used to identify each piece of data. The data types include Text, Image, Audio, and Video. The relevance, quality, and security scores are all between 0 and 1, with 1 being the best.

[0164] By analyzing the above data, we can observe that:

[0165] The text data D001 has high relevance, quality, and security scores, indicating that its content is closely related to the target topic, the text quality is high, and there is no sensitive information, making it suitable for inclusion in the knowledge base.

[0166] Although the relevance of image data D002 is slightly lower than that of text data, its quality and security scores remain at a high level, indicating that the image has good clarity and complies with the privacy policy. It is high-quality material for building a knowledge graph and is suitable for inclusion in the knowledge base.

[0167] Audio data D003 and video data D004 also show good comprehensive scores, reflecting the comprehensiveness and effectiveness of the method of the present invention in processing multimodal data, and are suitable for inclusion in the knowledge base.

[0168] In summary, the query method integrating AI big model and knowledge graph proposed in the present invention can not only screen out high-quality, high-relevance and high-security data from massive multimodal data, but also show advantages in constructing knowledge graphs and optimizing query paths. Compared with the traditional single-modal data processing method, the present invention can more comprehensively understand the user's query intention, provide more accurate query results, and improve the efficiency and accuracy of information retrieval.

[0169] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A query method integrating AI big model and knowledge graph, characterized by: include, Collect multimodal data and integrate knowledge base; Based on the multimodal data in the knowledge base, entity recognition and relationship extraction are performed to obtain structured triple data, and the triple data is preprocessed; Build a knowledge graph based on the preprocessed triple data; Build a query parsing model based on the knowledge graph, convert natural language queries into machine-understandable query language, and obtain parsed query instructions; Build a query path based on the parsed query instructions and optimize the query path; Obtain the most relevant result set in the optimized query path, and output the most relevant result set as the final query result; The specific steps of collecting multimodal data and integrating the knowledge base are as follows: Collect multimodal data of text, images, audio, and video; Calculate the relevance score R(x) of a data point x in multimodal data, expressed as: R(x)=cOs(V x ,V T )×α·LDA(x); Among them, V x represents the embedding vector of data point x, V T represents the embedding vector of the target topic, cOs represents the cosine similarity, α represents the weight factor of the relative importance of topic relevance, and LDA represents the latent Dirichlet allocation model LDA; Calculate the quality score Q(x) of data point x, the expression is: Among them, G(x) represents the output of the syntax error detection model, C(x) represents the output of the clarity detection model, N(x) represents the output of the noise suppression model, and V(x) represents the output of the resolution detection model; Calculate the security score S(x) of data point x. The calculation expression is: S(x)=min(1,1-F s (x))×F p (x); Among them, F s (x) represents the output of the sensitive word filtering model, F p (x) represents the output of the privacy compliance model; Score the collected data and get a comprehensive score, expressed as: Among them, F(x) represents the comprehensive score, R(x) represents the relevance score of data point x, Q(x) represents the quality score of data point x, and S(x) represents the security score of data point x; According to the comprehensive scores and requirements, the qualified data will be included in the knowledge base; The query parsing model is constructed based on the knowledge graph, natural language queries are converted into machine-understandable query languages, and the parsed query instructions are obtained. The specific steps are: Set the knowledge graph as KG, the natural language query as NLQ, and build a query parsing model based on the knowledge graph KG; The natural language query NLQ is input into the natural language understanding function, which parses the query and returns the intent, entity set, and relationship set. The expression is: g under (NLQ)={I, E, R}; Among them, g under represents the natural language understanding function, I represents the intent of the query, E represents the set of all entities extracted from the query, and R represents the set of relations; The query mapping function receives the parsed components and the entire knowledge graph KG, and converts them into a specific query instruction Q executed on the knowledge graph. The expression is: Q6 g map ({I,E,R},KG) Among them, g map Represents the query mapping function; Map the query to function g map and the natural language understanding function g under Combined, the expression is: Q=g(KG,NLQ)=g map (g under (NLQ),KG); Among them, Q represents the query result, g represents the query parsing model, and g under (NLQ) represents the application of the natural language understanding function g to the natural language query NLQ under The result, g map (g under (NLQ), KG) indicates the use of query mapping function g map , g under The output of (NLQ) is combined with the knowledge graph KG to generate the final query instruction Q.

2. The query method integrating the AI ​​big model and the knowledge graph as claimed in claim 1, characterized in that: The entity recognition and relationship extraction are performed based on the multimodal data in the integrated knowledge base to obtain structured triple data, and the triple data is preprocessed. The specific steps are: Extract multimodal data including text, images, audio and video from the knowledge base; For text data, the pre-trained named entity recognition model Et and dependency analysis model Dt are used to identify entities in the text and the relationships between them; For image data, a deep convolutional neural network Ei is used for object detection and recognition, and an image relationship detection model Di is used to capture the relationship between entities in the image; For audio data, after being converted into text by the speech recognition model Ea, the named entity recognition model Et and the dependency analysis model Dt are used for entity recognition and relationship extraction; For video data, it is first segmented into frames by the video segmentation model Sv, and then Ei and Di are applied to each frame, combined with time series analysis Tv to capture the relationship between consecutive frames.

3. The query method integrating the AI ​​big model and the knowledge graph as claimed in claim 2, characterized in that: The specific steps of constructing a knowledge graph based on the preprocessed triple data are as follows: All identified data are converted into unified structured triple data (s, p, o) using triple construction, where s represents the subject, p represents the predicate, and o represents the object; Preprocess the triplet data; The preprocessed triple data is mapped to the graph database to form a queryable knowledge graph structure.

4. The query method integrating the AI ​​big model and the knowledge graph as claimed in claim 3, characterized in that: The query path is constructed based on the parsed query instruction, and the specific steps are: Use natural language understanding (NLU) to parse the query instruction Q and identify the entities and relationships in the query; Map the entities and relationships in the query to the corresponding nodes and edges in the knowledge graph KG; Based on the mapped entities and relationships, find the path from the starting entity to the target entity in the knowledge graph, i.e., the candidate path; Set the starting node and target node in the query; Starting from the starting node, explore all connected nodes and check whether they match the query intent and entities; The path is expanded by traversing all edges through the knowledge graph until the target node is reached. In this process, the nodes and edges passed are recorded to form a path set.

5. The query method integrating the AI ​​big model and the knowledge graph as claimed in claim 4, characterized in that: The optimization query path comprises the following specific steps: An evaluation function is constructed based on the path set to calculate the evaluation score of each path. The evaluation function expression is: Among them, A(P) represents the evaluation score of path P, P represents the index variable, e represents an edge in path P, EI(P) represents the set of edges contained in path P, R(e) represents the relevance score of edge e, and L(P) represents the length of path P; The path P with the maximum value of A(P) is used as the initial query path.

6. The query method integrating the AI ​​big model and the knowledge graph as claimed in claim 5, characterized in that: The most relevant result set is obtained in the optimized query path, and the most relevant result set is output as the final query result. The specific steps are: The result set scoring function is constructed based on the optimized path. The expression is: Among them, F RS (n) represents the result set scoring function of node n, n represents the index variable, ω e represents the weight coefficient of the e-th edge, sim represents the similarity function, t e represents the type of the e-th edge, q represents the query intent, and α e represents the attenuation coefficient of the e-th edge, D(n, e) represents the distance from edge e to node n, that is, the position on the query path, and D(n) represents the shortest path distance from node n to the query start node; Traverse all nodes on the path, that is, calculate the F of each node in the initial query path RS (n) value; Set the threshold θ; All F RS Nodes with (n)>θ are selected into the result set; Press F for a node in the result set RS (n) values ​​are sorted in descending order, and the top-ranked nodes are the results most relevant to the query; Select the first K nodes from the sorted result set as the final query result output.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the query method integrating the AI ​​big model and the knowledge graph as described in any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the query method integrating the AI ​​big model and the knowledge graph as described in any one of claims 1 to 6 are implemented.

Images