Data lake query system and method based on semantics
By introducing semantic-based query methods into the data lake query system, using data lake metadata for semantic query and data resource calls, the problem of lack of automated index construction and intelligent sorting in the existing technology is solved, and the accurate understanding of user intentions and comprehensive response to search results is achieved.
Patent Information
- Application Number
- CN202111522647.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-13
- Publication Date
- 2025-04-04
- Estimated Expiration
- 2041-12-13
AI Technical Summary
The existing data lake query technology lacks an automated index construction mechanism, cannot effectively utilize the implicit associations and knowledge in the original data, and is difficult to achieve intelligent sorting of search results, and cannot accurately analyze user intentions and return query results in a comprehensive manner.
A semantic-based data lake query system is proposed. Through the tag identification module, entity search module, instance inference module and query result fusion module, the data lake metadata is used to perform semantic query and data resource calls, and intelligent response to user query requests is realized.
On the premise of ensuring data consistency, improve user retrieval flexibility and comprehensiveness of search results, be able to accurately understand user intentions and return knowledge instance results, and adapt to the multi-source heterogeneity, dynamic changes and complex correlation characteristics of the original data in the data lake.
Smart Images

Figure CN114218400B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a technology in the field of information processing, specifically a semantic-based data lake query system and method. Background Art
[0002] As a data storage method, data lakes can manage massive amounts of heterogeneous raw data in a flat manner, support the discovery of information and knowledge in the data, and thus fully tap the value of the data. However, the rapid generation, dynamic changes, and complex associations of large-scale data pose challenges to intelligent and effective retrieval and analysis in data lakes.
[0003] Existing data lake query solutions are mainly supported by manually building indexes. However, this type of technical means lacks flexibility, has low performance in dynamic access to data and intelligent response to queries, and cannot adapt to the continuous generation of raw data in different modes, accurately understand query requests and match results. The many-to-many mapping between query indexes and data is mainly carried out through multi-mode matching on the front-end nodes, which is insufficient in mining data semantics, making it difficult to support data sharing for complex businesses and to achieve in-depth analysis of data relationships. As a result, it is impossible to accurately analyze user intent and return query results comprehensively. Summary of the invention
[0004] In view of the shortcomings of existing data lake retrieval technologies, such as the lack of an automated index construction mechanism, the lack of utilization of implicit associations and knowledge in raw data, and the lack of intelligent sorting of retrieval results, the present invention proposes a semantic-based data lake query system and method, which, with the support of data lake metadata, performs semantic-based data query and data resource call, and can effectively adapt to the characteristics of multi-source heterogeneity, dynamic changes, and complex associations of raw data in the data lake. On the premise of ensuring data consistency, it improves the flexibility of user retrieval and the comprehensiveness of retrieval results, and intelligently returns the retrieved knowledge instance results for the query request input by the user.
[0005] The present invention is achieved through the following technical solutions:
[0006] The present invention relates to a semantic-based data lake query system, comprising: a label recognition module, an entity retrieval module, an instance reasoning module and a query result fusion module, wherein: the label recognition module relies on the data lake meta-model to index the top-level classification label, performs named entity recognition on the query statement text input by the user, and constructs a syntax tree, and outputs the semantic label set involved in the query request and the screening conditions for the label; the entity retrieval module accesses and filters entities according to the label, and deeply matches the entity subset of the screened instance knowledge graph with the query semantic block, links to the entity semantically related to the query pattern, and generates a candidate entity set after semantic expansion; the instance reasoning module accesses and fuses the attribute data of the instance associated with the entity output in the previous step, and performs multi-dimensional comprehensive sorting on the instance, and prunes it in combination with constraint conditions, thereby narrowing the result set; the query result fusion module reads the original data according to the information template of the summary query semantic label information and the sorting evaluation dimension, encapsulates it into a knowledge instance, and returns the structured instance knowledge to the user.
[0007] The data lake metadata includes: a unified management semantic metamodel layer, an instance knowledge graph layer and a raw data layer, wherein: mapping is established between layers through foreign keys to support accurate extraction; the semantic metamodel layer is the concept, attribute and association model of classification and stratification described in OWL format, which is the global directory index of data; the instance knowledge graph layer is the associated data described in the uniform resource management framework (RDF), and is identified by the uniform resource descriptor (URI) to store the instance knowledge extracted and updated in real time in a globally semantically consistent manner. Among them, the instance knowledge includes the instance data extracted from the raw data layer and the historical query statement structure model. And the extraction granularity and encapsulation structure are adjusted through the historical query statement structure model; the raw data layer is the original data of text, pictures, voice, video and other modalities and multi-source heterogeneity.
[0008] The label recognition module identifies coarse-grained entities through dictionary matching based on the top-level entity classification labels composed of the user query statement and the directory index in the unified management semantic metamodel; then, after fine-tuning the pre-trained BERT model on the annotation set, each statement in the query input by the user is embedded as a word vector and a character vector, and the optimal label sequence is determined by the conditional random field layer (CRF) using the scores and transition probability matrix output by the previous layer, thereby obtaining the semantic label set L that can be directly parsed from the query statement in ; Then, the matching results of the dictionary model and the deep learning model are integrated to build a syntax tree, and the filtering conditions for the tag are identified based on the grammatical dependencies.
[0009] The entity retrieval module screens entities from the tag classification one by one according to the parsed tags, and uses the context of the entities in the instance knowledge graph to supplement the semantic information in the user query through deep entity matching based on graph embedding; expands the entity set through the semantic expansion algorithm, thereby expanding the recall, generating a candidate entity set, and inputting it into the instance reasoning module for pruning.
[0010] The instance knowledge graph is a network knowledge structure formed by the named entities, associations, and attributes automatically identified in the original data of the data lake after unified standardization, and is stored in the form of associated data. The construction of the instance knowledge graph is achieved by preprocessing various types of original data and extracting information of different granularities, then encapsulating knowledge, and finally performing word sense disambiguation and entity matching to solve the problem of merging a large amount of knowledge, and then performing knowledge fusion processing to ensure the uniqueness and accuracy of knowledge.
[0011] The entity deep matching is achieved by maximizing the cosine similarity of the semantic block embedding vectors of the query statement and the filtered entity in the knowledge graph, that is, deep matching based on embedding: using the G2S (Graph-to-Segment, semantic parsing of knowledge graph based on semantic blocks) model, under the strategy of integrating the information of instance knowledge nodes and association relationships in the instance knowledge graph of multimodal data, the semantic blocks of the label screening results and query requests are embedded, aligned based on the attention mechanism, and sequence decoded to obtain embedded vectors, perform similarity matching, and then use the semantic expansion algorithm to expand the result set. The specific steps are as follows:
[0012] Step a) Knowledge graph node entity embedding is the concatenation of its adjacent nodes
[0013] Step b) forming an embedding of the semantic block subgraph by maximum pooling of the nodes in the semantic block;
[0014] Step c) uses a bidirectional gated recurrent network (GRU) to decode the semantic block sequence, and the hidden state s 1 = s i+1 =GRU([Φ (y) (y i ), c i ],s i ,), where: Φ (y) (y i ) is the embedding of the semantic block;
[0015] Step d) Align the vectors based on the attention mechanism: node attention context vector of the knowledge graph in: e ij=a(s i-1 ,h j ), a ij is the weight represented by each node, and a is a feedforward neural network as an alignment function, which scores the degree of match between the input nodes around position j and the output at position i.
[0016] Step e) Maximize the calculation of the user's current search sentence embedding s q and knowledge graph semantic block embeddings h The cosine similarity of Where: q i ,h i Represent each component of the vector respectively, so as to match the query to the semantic block in the knowledge graph.
[0017] Step f) Perform semantic expansion across concept and instance layers: Use label propagation (LPA) to expand the candidate entity set RS into an extended set ES, where: |ES| = γ·|RS|, γ is the expansion coefficient, γ∈[1, min(|neighborSet|)], i.e., the interest node e (e∈ES) sends a message to the neighboring node e' (e′∈instance knowledge graph KG) with synonymous or hyponymous relationship. The neighboring node receives the message and updates the entity label set of the node to the label or label set with the largest cumulative weight. When the weights are equal, a random selection is made. After the label is stable, the entity is associated with the label. A sliding window with window attenuation is used to select the entity e with the highest matching weight. 1 Starting from the neighbors of , the result set is expanded to improve the recall rate of retrieval in the knowledge graph, including:
[0018] Step 1. In e 1 Select neighbor node Nei(e 1 ), Nei(e 1 )≠e 1 As a listening node;
[0019] Step 2: Nei(e 1 ) receives messages from its neighbor nodes, with message weight W = s × w split , where: the node weight s is the similarity value of the semantic block where the node is located after being embedded, calculated in step e), representing the relevance between the entity and the search formula, and w split The split coefficient is the inverse of the node out-degree. The node labels are normalized in each iteration;
[0020] Step 3: Process the neighborhood of the next matching entity node in turn;
[0021] Step 4: Return to step 1 until the predefined maximum number of iterations T is reached;
[0022] Step 5: Traverse the RS node neighborhood and sort the window size with the highest weight The nodes are added to the expansion set ES. The processing ends.
[0023] The extended result entity set ES obtained by the window-decayed semantic instance expansion method is a superset of the user's query target. After pruning in the subsequent path sorting step, the sorted instance set returned to the user is obtained. For the screening conditions containing time information, the time constraints are matched to six semantics: equal, contain, overlap, before, after, and close to. In this way, the query is matched to an entity that conforms to the semantics of the knowledge graph, and the semantics of the search input by the user are expanded based on the label propagation method to automatically generate a candidate entity set.
[0024] The instance reasoning module performs multi-dimensional combination sorting on the candidate entity set ES, and returns the instances associated with the query intent under the restriction of the user-defined clipping coefficient k. The specific method is to select k instance data that meet the rule constraints, have high multi-dimensional combination score sorting, good data quality description, and high node response in the entity set, prune the linked instances by dynamically setting the clipping threshold, and input them into the subsequent steps for fusion processing.
[0025] The multi-dimensional combined scoring sorting means: routing to instance data according to the candidate entity set; then fusing the multi-dimensional attributes and features in the instance data, and obtaining the entity score through weighted aggregation: F[f 1 (e), f 2 (e), ..., f n (e)]·φ(e), where: the numerical score f of entity e on dimension i i (e) = μ i ·i(e),μ i is the weight of the feature dimension, i(e)
[0026] Aggregate results for instance attributes that make up the dimension labels φ(e)=Πrule λ , rule λ is the satisfiability of the attribute λ of the instance to the data rules, time semantics, and permission rules, φ(e)∈[0,1]; F is a scoring function, including but not limited to summation and weighted average.
[0027] The query result fusion module generates an information template based on the classification semantic labels formed by the directory index in the data lake meta-model and the attribute dimension labels involved in the scoring; reads the original data for aggregation and compression, and fills it into the information template to fill in key attributes, thereby projecting the relevant original data onto a feature view related to the query intent topic, and encapsulating it through the information template to obtain candidate entity information returned to the user.
[0028] The information template generation is to select tags related to query and sorting semantics at different abstract levels. The original data reading is routed to the instance original data according to the URI corresponding to the result instance set. The attribute filling is based on the information items corresponding to the information template, and is supplemented and extracted from the original data according to the predefined interface, and the semantic information of different scales and modes at high and low levels is integrated and hierarchically compressed according to the semantic metamodel to form attributes and features of different granularities and then filled into the corresponding slots of the returned knowledge template.
[0029] The present invention relates to a semantic-based data lake query method, which performs semantic analysis on the request input by the user and identifies the tags contained in the query; under the support of the metadata of the data lake, the tags are screened and matched layer by layer in the knowledge graph, and the query is expanded through semantic tag propagation to generate a candidate entity set; then, multiple evaluation dimensions are integrated in the instance reasoning module to sort and prune the instances; finally, the original data is called, and an information template is generated according to the parsed semantic tags and sorted dimension tags, and the instance data is filled into each slot in the template, and the instance data is summarized, aggregated and knowledge encapsulated. Through the above steps, the user's query intention is accurately understood, and on the basis of full semantic fusion, the instance information is comprehensively summarized, and the variable query requests are responded to, and the candidate instance information sorted and combined by different dimensions is returned.
[0030] Technical Effects
[0031] Compared with the prior art, the present invention parses user intent with the support of data lake metadata; then performs screening and semantic expansion in the entity retrieval module to recall matching entities; then in the instance reasoning module, data semantic association reasoning is performed by performing multi-dimensional attribute fusion and weighted aggregation, so as to sort the recalled result set according to the scores of instance data in multiple evaluation dimensions; finally, in the query result fusion module, an information template is generated, the original data is called to fill in the key tags in the information template, the sorting tags are explained, and the constructed result instance information is returned to the user. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a schematic diagram of the system of the present invention;
[0033] Figure 2 It is a schematic diagram of an embodiment application. DETAILED DESCRIPTION
[0034] This embodiment relates to a semantic-based data lake query method based on the above system. During the user's query and retrieval process of data, it flexibly and accurately responds to various query requests based on a unified management semantic metamodel and instance knowledge graph with a high degree of data fusion, strong semantic relevance of fused data, and real-time fusion and update of new data.
[0035] like Figure 2 As shown in the figure, the query input by the user is encapsulated as JSON and input into the tag recognition module through the RESTful interface. First, in the application layer tag recognition module, the classification semantic tags and filtering conditions in the query statement are identified through the named entity recognition algorithm and syntax tree construction; then, in the entity retrieval module of the logic layer, the entities are filtered layer by layer by label, and the knowledge graph semantic block embedding model is used to accurately link to some entities with the label. Based on the hierarchical semantic metamodel, semantic expansion is performed through the label algorithm to expand the scope of recalled entities; then, in the instance reasoning module, the query results are obtained through instance knowledge fusion and multi-dimensional sorting, so as to accurately fill in the knowledge instances as the return results based on the comprehensive acquisition of the description data of the entity to be queried and the user's query intention. Finally, in the knowledge application layer, the relevant instance information is returned to the front end in the form of HTML through the RESTful interface to complete the intelligent query.
[0036] At the data layer, the graph database Neo4J stores data lake metadata that uniformly manages semantic metamodels, instance knowledge graphs, historical query statements, and models; the relational database MySQL, the non-relational database MongoDB, the distributed file system HDFS, and the object storage system store multimodal raw data.
[0037] This system implements intelligent retrieval of the data lake through the following steps:
[0038] Step 1: Parse the query entered by the system user and identify the semantic tags and filter conditions contained in the query with the help of the unified management semantic metamodel catalog index of the data lake;
[0039] Step 2: Filter the knowledge entities according to the labels output in step 1, and obtain the semantic blocks representing the query intent in combination with the context of the entities in the knowledge graph. Then, expand the semantics through label propagation, discover related entities, and expand the candidate entity set.
[0040] Step 3: According to the candidate entity set, the indicator attributes of different dimensions and different semantic levels are integrated in the instance data to construct a multi-dimensional sorting function. The data is sorted according to constraints such as time constraints and size constraints, and pruned according to the custom threshold to obtain a small query result set.
[0041] Step 4: Call relevant files from the original data for aggregation and information template filling; return candidate instance information.
[0042] The comparison of the technical characteristics of the above work with similar technical products at home and abroad is shown in Table 1.
[0043] Table 1 Technical characteristics comparison:
[0044]
[0045]
[0046]
[0047] Compared with the prior art, the technical effects of the present invention include:
[0048] The present invention parses and infers user query statements with the support of the data lake semantic metamodel, and realizes comprehensive and intelligent recall and sorting of relevant data in the process of users querying the data lake: for real-time updated data, after semantic fusion, it is uniformly managed by metadata; under the support of the continuously generated data lake metadata, in the process of semantic retrieval, the relevant instances are obtained by label screening entities and association relationship reasoning; the attributes and features of the instance data extracted by cross-modal fusion are used to intelligently expand the knowledge, and then the instance object formed by the knowledge fusion related to the query in the data lake is returned as the query result. In this way, the user does not need to understand the complex association between the specific fields and instance data in the business data, and can directly describe the query intent through natural language, and can link to the relevant entity set through the dynamic candidate entity generation method. In addition, the global unified representation of the data model by the knowledge graph modeling method enables the original data and the reasoning results to be associated with the uniformly managed metamodel under the premise of meeting the semantic constraints, so as to ensure the data consistency of the query and relevance analysis.
[0049] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.
Claims
1. A semantic-based data lake query system, characterized in that: include: The label recognition module, entity retrieval module, instance reasoning module and query result fusion module include: the label recognition module relies on the data lake meta-model to index the top-level classification label, performs named entity recognition on the query text input by the user, and constructs a syntax tree, and outputs the semantic label set involved in the query request and the filtering conditions for the label; the entity retrieval module accesses and filters entities by label, and deeply matches the entity subset of the filtered instance knowledge graph with the query semantic block, links to the entity semantically related to the query pattern, and generates a candidate entity set after semantic expansion; the instance reasoning module accesses and fuses the attribute data of the instance associated with the entity output in the previous step, and performs multi-dimensional comprehensive sorting on the instance, and prunes it in combination with the constraint conditions, thereby narrowing the result set; the query result fusion module reads the original data according to the information template of the query semantic label information and the sorting evaluation dimension, encapsulates it into a knowledge instance, and returns the structured instance knowledge to the user; The data lake metadata includes: a unified management semantic metamodel layer, an instance knowledge graph layer and a raw data layer, wherein: mapping is established between layers through foreign keys to support accurate extraction, the semantic metamodel layer is the concept, attribute and association model of classification and stratification described in OWL format, and is the global directory index of data; the instance knowledge graph layer is the associated data described in the Uniform Resource Framework (RDF), and is identified by the Uniform Resource Descriptor (URI) to store the instance knowledge extracted and updated in real time in a globally semantically consistent manner; wherein the instance knowledge includes the instance data extracted from the raw data layer and the historical query statement structure model; and the extraction granularity and encapsulation structure are adjusted through the historical query statement structure model; the raw data layer is modal, multi-source heterogeneous raw data.
2. The semantic-based data lake query system according to claim 1, characterized in that: The label recognition module identifies coarse-grained entities through dictionary matching based on the top-level entity classification labels composed of the user query statement and the directory index in the unified management semantic metamodel; then, after fine-tuning the pre-trained BERT model on the annotation set, each statement in the query input by the user is embedded as a word vector and a character vector, and the optimal label sequence is determined by the conditional random field layer (CRF) using the scores and transition probability matrix output by the previous layer, thereby obtaining the semantic label set L that can be directly parsed from the query statement in ; Then, the matching results of the dictionary model and the deep learning model are integrated to build a syntax tree, and the filtering conditions for the tag are identified based on the grammatical dependencies.
3. The semantic-based data lake query system according to claim 1, characterized in that: The entity retrieval module screens entities from the tag classification one by one according to the parsed tags, and uses the context of the entities in the instance knowledge graph to supplement the semantic information in the user query through deep entity matching based on graph embedding; expands the entity set through the semantic expansion algorithm, thereby expanding the recall, generating a candidate entity set, and inputting it into the instance reasoning module for pruning; The instance knowledge graph is a network knowledge structure formed by unified standardization of named entities, association relationships and attributes automatically identified in the original data of the data lake, and stored in the form of associated data. The instance knowledge graph is obtained by performing specific preprocessing on various types of original data and extracting information of different granularities, and then encapsulating knowledge. Finally, it is processed by word sense disambiguation and entity matching to solve the problems existing when merging a large amount of knowledge, and then performing knowledge fusion processing to ensure the uniqueness and accuracy of knowledge.
4. The semantic-based data lake query system according to claim 3 is characterized in that: The entity deep matching is achieved by maximizing the cosine similarity of the semantic block embedding vectors of the query statement and the screened entity in the knowledge graph, that is, based on the embedded deep matching, the G2S model is used to embed, align and sequence decode the semantic blocks of the label screening results and query requests under the strategy of integrating the instance knowledge nodes and association relationship information of the instance knowledge graph of multimodal data, and obtain the embedded vectors for similarity matching, and then use the semantic expansion algorithm to expand the result set. The specific steps are as follows: Step a) Knowledge graph node entity embedding is the concatenation of its adjacent nodes Step b) forming an embedding of the semantic block subgraph by maximum pooling of the nodes in the semantic block; Step c) uses a bidirectional gated recurrent network (GRU) to decode the semantic block sequence, and the hidden state s i+1 =GRU([Φ (y) (y i ),c i ],s i ,), where: Φ (y) (y i ) is the embedding of the semantic block; Step d) Align the vectors based on the attention mechanism: node attention context vector of the knowledge graph in: e ij =a(s i-1 ,h j ), a ij is the weight represented by each node, a is a feedforward neural network as an alignment function, which scores the matching degree between the input nodes around position j and the output at position i; Step e) Maximize the calculation of the user's current search sentence embedding s q and knowledge graph semantic block embeddings h Cosine similarity of Where: q i ,h i Represent each component of the vector separately, so as to match the query to the semantic block in the knowledge graph; Step f) Through label propagation (LPA), the candidate entity set RS is expanded into an expanded set ES, where: |ES| = γ·|RS|, γ is the expansion coefficient, γ∈[1, min(|neighborSet|)], that is, the interest node e (e∈ES) sends a message to the neighboring node e' associated with synonymous or hyponymous relationship, e'∈ instance knowledge graph KG, the neighboring node receives the message, and updates the entity label set of the node to the label or label set with the largest cumulative weight. When the weights are equal, they are randomly selected; after the label is stable, the entity is associated with the label; a sliding window with window decay is used to expand the result set starting from the neighbor of the entity e1 with the highest matching weight, thereby improving the recall rate of retrieval in the knowledge graph.
5. The semantic-based data lake query system according to claim 4 is characterized in that: The expanded result set specifically includes: Step 1: Select the neighbor node Nei(e1) in the neighborhood of e1, Nei(e1)≠e1 as the monitoring node, where: s is the similarity value of the semantic block where the node is located after embedding; Step 2: Nei(e1) receives messages from its neighbor nodes, with message weight W = s × w split , where: the node weight s is the similarity value of the semantic block where the node is located after being embedded, calculated in step e), representing the relevance between the entity and the search formula, and w split The split coefficient is the inverse of the node out-degree, and the node labels are normalized in each iteration; Step 3: Process the neighborhood of the next matching entity node in turn; Step 4: Return to step 1 until the predefined maximum number of iterations T is reached; Step 5: Traverse the RS node neighborhood and sort the window size with the highest weight The nodes are added to the extended set ES; the processing ends.
6. The semantic-based data lake query system according to claim 1, characterized in that: The constraints are specifically screening conditions that include time information, namely, time constraints, which are matched into six semantics: equal, contain, overlap, precede, follow, and close to.
7. The semantic-based data lake query system according to claim 1, characterized in that: The instance reasoning module performs multi-dimensional combination sorting on the candidate entity set ES, and after returning the instances associated with the query intent under the restriction of the user-defined clipping coefficient k, selects k instance data that meet the rule constraints, have high multi-dimensional combination score sorting, good data quality description, and high node response, and dynamically sets the clipping threshold to prune the linked instances, and inputs them into the subsequent steps for fusion processing; The multi-dimensional combined scoring sorting means: routing to instance data according to the candidate entity set; then fusing the multi-dimensional attributes and features in the instance data, and obtaining the entity score after weighted aggregation: F(f1(e), f2(e),…, f n (e)]·φ€, where: the numerical score f of entity e on dimension i i (e) = μ i ·i(e),μ i is the weight of the dimension i, which is the aggregation result of the instance attributes that make up the dimension label φ(e)=∏rule k ,rule k is the satisfiability of the attribute k of the instance to the data rules, time semantics, and permission rules, φ(e)∈[0,1]; F is the scoring function.
8. The semantic-based data lake query system according to claim 1, characterized in that: The query result fusion module generates an information template based on the classification semantic labels formed by the directory index in the data lake meta-model and the attribute dimension labels involved in the scoring; reads the original data for aggregation and compression, and fills it into the information template to fill in key attributes, thereby projecting the relevant original data onto a feature view related to the query intent topic, and encapsulating it through the information template to obtain candidate entity information returned to the user.
9. The semantic-based data lake query system according to claim 1, characterized in that: The information template generation is to select tags related to query and sorting semantics at different abstract levels. The original data reading is routed to the instance original data according to the URI corresponding to the result instance set. The attribute filling is based on the information items corresponding to the information template, and is supplemented and extracted from the original data according to the predefined interface, and the semantic information of different scales and modes at high and low levels is integrated and hierarchically compressed according to the semantic metamodel to form attributes and features of different granularities and then filled into the corresponding slots of the returned knowledge template.
10. A semantic-based data lake query method according to the semantic-based data lake query system according to any one of claims 1 to 9, characterized in that: The request input by the user is semantically parsed to identify the tags contained in the query; with the support of the metadata of the data lake, the tags are screened and matched layer by layer in the knowledge graph, and the query is expanded through semantic tag propagation to generate a set of candidate entities; then multiple evaluation dimensions are integrated in the instance reasoning module to sort and prune the instances; finally, the original data is called, and an information template is generated according to the parsed semantic tags and sorted dimension tags, and the instance data is filled into each slot in the template, and the instance data is summarized, aggregated and knowledge encapsulated. By accurately understanding the user's query intention and based on full semantic fusion, the instance information is comprehensively summarized, responding to changing query requests, and returning candidate instance information sorted and combined by different dimensions.
Citation Information
Patent Citations
Ontology matching based case search system
CN106021306A
Autonomous data lake construction system and method based on associated data
CN110941612A