Multi-source heterogeneous data fusion analysis method and system

Abstract

The invention provides a multi-source heterogeneous data fusion analysis method and system, and relates to the technical field of data processing. The method comprises the following steps: acquiring multi-source heterogeneous event knowledge, and extracting entity information and attribute information of multi-source heterogeneous data; fusing the attribute information, constructing a multi-label classification model, and extracting a sequential relationship of the multi-source heterogeneous data through the multi-label classification model; a time sequence label is added to the time sequence relation, and then an initial knowledge graph of the multi-source heterogeneous data is constructed; obtaining an entity time sequence state sequence of the multi-source heterogeneous data through the initial knowledge graph and the entity information; extracting features of the entity time sequence state sequence; and processing the characteristics of the entity time sequence state sequence to update the initial knowledge graph to obtain the time sequence knowledge graph of the multi-source heterogeneous data. Massive and diversified knowledge is organized and expressed orderly, uniformly and associatively through an entity and attribute extraction technology of multi-source heterogeneous information, a time sequence multi-label relation extraction technology and a time-space big data standardization expression technology.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a multi-source heterogeneous data fusion analysis method and system. Background Art

[0002] Multi-source data intelligent fusion technology is a technology that integrates, processes and analyzes data from different sources, different formats and different semantics to achieve comprehensive, accurate and in-depth information acquisition and knowledge discovery.

[0003] Intelligent multi-source data fusion technology uses artificial intelligence algorithms to systematically integrate and analyze heterogeneous data from multiple sources. Its core goal is to overcome the limitations of traditional single-modal data processing and achieve cross-domain information synergy and efficiency. Current mainstream frameworks feature three layers of fusion: data-layer fusion employs distributed storage and federated learning to address the challenges of physical integration of multi-source data; feature-layer fusion uses graph convolution of deep neural networks to achieve cross-modal feature embedding; and decision-layer fusion utilizes knowledge graphs and reinforcement learning to build dynamic reasoning mechanisms. Current technology enables the intelligent fusion of multi-dimensional information, including text, images, and time series signals.

[0004] With the development of internet technology, knowledge graph technology, a technique for organizing knowledge resources in the form of information, has facilitated the rapid analysis of relationships between various types of data. Current research on knowledge graphs in specialized fields primarily focuses on objective entities such as personnel, institutions, facilities, and equipment. However, event knowledge graphs represent knowledge at a higher granularity than objective entities, possessing temporal, spatial, and dynamic characteristics, and can carry more valuable battlefield information. Research using events or incidents as the basic unit of knowledge representation and constructing event-centric knowledge graphs can effectively organize and manage massive amounts of heterogeneous and multimodal data, deeply correlate and mine key information, and provide auxiliary support for understanding and judging behavioral patterns of events.

[0005] Event information is fragmented, diverse, and chaotic, making it difficult to unify, correlate, and utilize multi-source heterogeneous knowledge. Therefore, how to organize and express this massive amount of diverse event knowledge in an orderly, unified, and correlated manner becomes a key issue. Summary of the Invention

[0006] In view of the above problems, the present invention provides a multi-source heterogeneous data fusion analysis method and system.

[0007] On the one hand, the present invention provides a multi-source heterogeneous data fusion analysis method, including: acquiring multi-source heterogeneous event knowledge, extracting entity information and attribute information of the multi-source heterogeneous data; constructing a multi-label classification model after fusing the attribute information, and extracting the time series relationship of the multi-source heterogeneous data through the multi-label classification model; constructing an initial knowledge graph of the multi-source heterogeneous data after adding time series labels to the time series relationship; obtaining the entity time series state sequence of the multi-source heterogeneous data through the initial knowledge graph and entity information; extracting the features of the entity time series state sequence; processing the features of the entity time series state sequence to update the initial knowledge graph and obtain the time series knowledge graph of the multi-source heterogeneous data.

[0008] According to an embodiment of the present invention, entity elements of multi-source heterogeneous data are extracted, and event information of the multi-source heterogeneous data is obtained through the entity elements; the event information is detected and analyzed to obtain entity attributes; attribute information of each node of the multi-source heterogeneous data is obtained through the entity attributes; and the attribute information is classified and analyzed to obtain entity relationships.

[0009] According to an embodiment of the present invention, multi-source heterogeneous data includes multiple entities, and a multi-label classification model can obtain event relationships between the multiple entities.

[0010] According to an embodiment of the present invention, the features of multi-source heterogeneous data are obtained and learned through a convolutional layer to obtain a learning result vector of the convolutional layer; the learning result vector is segmented and pooled through a pooling layer to obtain a final output vector; the similarity between the final output vector and the event relationship is calculated to obtain the event association relationship between multiple entities; and time series labels are added to the event association relationship to obtain the time series relationship of multi-source heterogeneous data.

[0011] According to an embodiment of the present invention, an event ontology library is constructed through knowledge graphs and entity information; an entity set of multi-source heterogeneous data is constructed through the event ontology library; and a current state feature vector of multi-source heterogeneous data is constructed through the entity set to form an entity time series state sequence.

[0012] According to an embodiment of the present invention, an entity temporal state sequence is converted into a label sequence; the label sequence is fine-tuned using an open base code model; and the fine-tuned label sequence is trained and evaluated using the open base code model to obtain features of the entity temporal state sequence.

[0013] According to an embodiment of the present invention, the evaluation indicators include: code accuracy, grammatical standardization, and semantic consistency.

[0014] According to an embodiment of the present invention, multiple proxy modules are obtained based on a language model programming framework; a translation model is constructed through the multiple proxy modules; the features of the entity temporal state sequence are processed through the translation model to obtain the entity information missing in the initial knowledge graph; the missing entity information is trained and then input into the initial knowledge graph to obtain a temporal knowledge graph of multi-source heterogeneous data.

[0015] According to an embodiment of the present invention, the update includes: the nonlinear relationship between entities, relations and timestamps in the initial knowledge graph, the reasoning path in the initial knowledge graph, and the time information and event association relationship in the initial knowledge graph.

[0016] Another aspect of the present invention provides a system for multi-source heterogeneous data fusion analysis, including: a data input module for inputting multi-source heterogeneous data; a data extraction module for extracting entity information and attribute information of multi-source heterogeneous data; a data processing module for processing the entity information and attribute information to obtain an initial knowledge graph and an entity time series state sequence; a data update module for updating the initial knowledge graph to obtain a time series knowledge graph of multi-source heterogeneous data.

[0017] The multi-source heterogeneous data fusion analysis method and system provided by the present invention can achieve the following beneficial effects:

[0018] (1) Through the entity and attribute extraction technology of multi-source heterogeneous information, the time series multi-label relationship extraction technology and the normalized representation technology of spatiotemporal big data, the massive and diverse knowledge is organized and expressed in an orderly, unified and related manner, and the scattered, diverse and messy data is unified, associated and utilized, thus solving the problem that data is difficult to organize and manage in a unified manner;

[0019] (2) Through the normalized representation technology of spatiotemporal big data and the expert knowledge extraction and rule construction technology, time series labels are introduced into the traditional event association relationship on the basis of the traditional big data information representation knowledge graph, thereby constructing a temporal knowledge graph framework structure with the ability to represent temporal information, solving the problem that the traditional knowledge graph lacks spatiotemporal dimensions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0021] Figure 1 A flowchart of a multi-source heterogeneous data fusion analysis method according to an embodiment of the present invention is schematically shown;

[0022] Figure 2 The following schematically shows a principle diagram of a multi-source heterogeneous data fusion analysis method according to an embodiment of the present invention;

[0023] Figure 3 A flowchart of extracting multi-source heterogeneous data information according to an embodiment of the present invention is schematically shown;

[0024] Figure 4 A flowchart of extracting the temporal relationship of multi-source heterogeneous data according to an embodiment of the present invention is schematically shown;

[0025] Figure 5 Schematically shows a principle diagram of extracting entity temporal state sequence features according to an embodiment of the present invention;

[0026] Figure 6 A block diagram schematically illustrates language model programming according to an embodiment of the present invention;

[0027] Figure 7 The block diagram of the multi-source heterogeneous data fusion analysis system according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise," "include," etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0030] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] Before describing specific embodiments of the present invention in detail, technical terms are first explained to facilitate a better understanding of the present invention.

[0032] Multi-source heterogeneous data refers to a data set obtained from different sources that differs in structure, format, type or storage method. This data is usually difficult to integrate or analyze directly and requires preprocessing before it can be effectively used.

[0033] Temporal relationship: refers to the correlation between data, events or phenomena in the time dimension, emphasizing characteristics such as time sequence, duration, periodicity and causality.

[0034] Translation model: Captures the relationship between two objects through translation in vector space (such as addition or subtraction).

[0035] In view of this, the present invention provides a multi-source heterogeneous data fusion analysis method and system.

[0036] Figure 1 A flowchart of a multi-source heterogeneous data fusion analysis method according to an embodiment of the present invention is schematically shown; Figure 2 The principle diagram of the multi-source heterogeneous data fusion analysis method according to an embodiment of the present invention is schematically shown.

[0037] like Figure 1 and Figure 2 As shown, the multi-source heterogeneous data fusion analysis method according to an embodiment of the present invention includes steps S1 to S6.

[0038] In step S1, multi-source heterogeneous event knowledge is acquired, and entity information and attribute information of the multi-source heterogeneous data are extracted.

[0039] Extract entity attributes, entity relationships, and entity elements from the accessed multi-source heterogeneous data, output information points in a fixed format based on the data type, identify a large number of information points from various documents, and integrate them in a unified form for easy inspection and comparison. Through multi-source data, complete the automatic extraction of multi-source relationships between entities, and support customized relationships, providing data support for the construction of relationship maps.

[0040] The attribute information of each node entity of multi-source heterogeneous data is used for analysis. According to the data content, the various attributes in the entity attributes are classified. The attribute information of each category is used for in-depth analysis. By adopting machine learning, deep learning and statistical methods, an attribute mining model is constructed to learn the attribute information of each node in the biological knowledge network, analyze the association between the attributes of the nodes, obtain the similarity of the attributes between different entities, and then discover the association relationship between entities when the attributes are similar, and mine the potential entity relationship.

[0041] In step S2, attribute information is fused to construct a multi-label classification model, and the temporal relationship of multi-source heterogeneous data is extracted through the multi-label classification model.

[0042] After fusion, the relevant multi-source heterogeneous data is classified using a Softmax regression model. The inputs are mapped to real numbers between 0 and 1 and normalized to sum to 1, resulting in a sum of 1 for the probabilities of the multiple classifications. Using event-attribute-based temporal multi-label relationship extraction technology, the initially extracted entities are compared with simple relationships preset in the ontology library. Event entities and relationships are equated to vocabulary. A sentence classification model is then used to classify indirectly related entities, thereby extracting temporal relationships within the multi-source heterogeneous data.

[0043] For example, when dealing with multi-label classification (number of categories >2) When the classifier has the problem, the final output unit needs to be numerically processed by the Soflmax function. The definition of the Soflmax function is as follows:

[0044]

[0045] Where, Indicates the scale of the output index of the current element in the sum of the output indexes of all elements. Represents the mapping output of the previous output unit, Represents the category index, Indicates the total number of categories.

[0046] For example, give a specific example to illustrate that when =4, 4 output values ​​are obtained through the linear classifier model The result is expressed as:

[0047]

[0048] The 4 output values ​​obtained As a result, after Soflmax processing, the relative probability after numerical conversion is obtained. :

[0049]

[0050] It can be seen from the above formula that ( =2) has the highest corresponding probability, and is more likely to be classified as Category II.

[0051] For example, when using Soflmax to implement multi-label classification, you should pay attention to whether the value overflows. If the value of the mine is very large, the value after the exponential operation is often likely to overflow. You can process the mine data and subtract the maximum value of the mine from each element in the mine. , specifically expressed as:

[0052]

[0053]

[0054] Where, Indicates the scale of the output index of the current element in the sum of the output indexes of all elements. Represents the mapping output of the previous output unit, Represents the category index, Indicates the total number of categories.

[0055] In step S3, after adding time series labels to the time series relationships, an initial knowledge graph of multi-source heterogeneous data is constructed.

[0056] By adding time series labels to temporal relationships, we obtain multi-label associations with temporal information. Given the complexity of event spatiotemporal structures and spatiotemporal associations, associations between event entities in task-oriented scenarios often have multiple labels and multiple categories. Combined with actual event information extracted from electronic texts, we construct a temporal knowledge graph based on the ontology library to understand the spatiotemporal evolution patterns of events.

[0057] For example, remote supervision is used to complete the classification and extraction of multi-label association relationships, and multiple association relationships between events are modeled in the form of an undirected graph of relationship labels. The relationship label classification based on the undirected graph is implemented in the form of a graph structure traversal algorithm. By finding the connected components in the relationship connectivity graph, all relationship labels that may belong to the current sentence expression are collected. Finally, K-means clustering is used to complete the screening and division of association relationship labels. This can solve the problems of incomplete instances in the knowledge base, high spatial and temporal discreteness of information, diverse relationship types, and single association patterns.

[0058] For example, specific methods for collecting event information include: remote sensing image event extraction, recognition and retrieval, information generation, information analysis combined with historical information, image retrieval of events for images obtained from the Internet, and keyword extraction and retrieval of text data.

[0059] In step S4, the entity temporal state sequence of multi-source heterogeneous data is obtained through the initial knowledge graph and entity information.

[0060] Using cluster analysis and deep neural network feature classification methods, event entities from multi-source heterogeneous data sources such as networks, images, and signals are extracted based on the unique identifying attribute information in the initial knowledge graph ontology library, and an event entity set is constructed. The event state feature vectors at each temporal state of the event entity set and the state vectors of adjacent event entities are transformed and constructed into event current state feature vectors with stronger spatiotemporal contextual associations, combined with the LTR (Learning to Rank) feature space latent optimization method, thereby forming a temporal state sequence of event entities.

[0061] In step S5, features of the entity temporal state sequence are extracted.

[0062] We built a CodeLlama (Open Code Base Model) large-scale language model (with a long context memory and attention mechanism that can capture and understand long-term dependencies in sequences). We input the generated token sequence into the CodeLlama model for fine-tuning to clarify the expectations of the event entity, including code structure, grammatical specifications, function logic, etc. We controlled the diversity and quality of the generated code by adjusting the sampling temperature, and encoded and extracted features from the generated token sequence.

[0063] In step S6, the features of the entity temporal state sequence are processed to update the initial knowledge graph and obtain a temporal knowledge graph of multi-source heterogeneous data.

[0064] Design a LangChain (language model programming framework) multi-agent architecture. According to the LangChain architecture design, develop each agent module separately and define the path of data flow from input to output, which can include code input, data transmission and processing between agents, and the final generated code output. Map the tuples of events in the initial knowledge graph to a low-dimensional vector space, use the time order of the relationship to model the knowledge evolution in the time dimension, apply LM (language model) to the initial knowledge graph to obtain its implicit semantic information for knowledge reasoning, and mine the association information between multiple quadruples in the graph, capture semantic knowledge, and quickly adapt to new entities and relationships to obtain an updated time series knowledge graph for multi-source heterogeneous data. This solves the problems of insufficient time information extracted from timestamps in existing completion and update methods and insufficient information mining of association relationships in time series knowledge graphs.

[0065] Figure 3 The flowchart of extracting multi-source heterogeneous data information according to an embodiment of the present invention is schematically shown.

[0066] like Figure 3 As shown, the extraction of multi-source heterogeneous data information according to an embodiment of the present invention includes steps S11 to S14.

[0067] Step S11 , extracting entity elements of multi-source heterogeneous data, and obtaining event information of the multi-source heterogeneous data through the entity elements.

[0068] For example, entity element recognition and extraction includes information recognition and automatic identification of concepts such as personnel, unit names, weapons and equipment, countries, and organizations in the text. The identification results can be used to distinguish concept categories.

[0069] For example, event information includes the time, location, participating roles, and related actions or status changes.

[0070] Using deep learning models, we pre-train the input multi-source heterogeneous data to obtain the corresponding annotation sequences. We then post-process the annotation results (for example, merging labels) to obtain the final entity elements. Event information is extracted from this large amount of entity element data and presented in a structured form, generating structured event knowledge in batches. Based on the output data hierarchy, event knowledge construction is divided into event detection and event extraction. By building an event knowledge base, we provide knowledge support for discovering time clues or analyzing event development trends.

[0071] Step S12: Detect and analyze the event information to obtain entity attributes.

[0072] Extracting event element information from loose unstructured information and generating refined structured event data. The main tasks of event extraction include three aspects: (1) Text understanding, the event description text can be divided into text units with independent semantics through syntactic component analysis, and the semantic role of the text unit is understood; (2) Event parsing, identifying an event data including element units, such as entities, relationships, time, geographic information, and attribute information such as the number of people involved and event type, which can be set manually or automatically generated based on the text understanding results; (3) Element filling, according to the filling requirements of the element unit, the text unit is converted into an attribute value that meets the specification to obtain the entity attribute.

[0073] Step S13: obtaining attribute information of each node of multi-source heterogeneous data through entity attributes.

[0074] The attribute information of a specific entity is obtained from the entity attribute information, and mining and analysis are performed according to the attribute characteristics. By sorting out the same attribute content and improving the attribute information, the attribute information of each node of multi-source heterogeneous data is obtained.

[0075] For example, the attribute information of each node includes biological attributes (name, release year and month, country of affiliation, etc.), association attributes (associated entity name, associated content, associated nature, etc.), event attributes (event time, event location, event type, etc.) and organizational attributes (organization name, number of organization members, organization nature, content of organizational activities) and other information.

[0076] Step S14: classify the attribute information and analyze it to obtain entity relationships.

[0077] Based on the attribute information of each node, an attribute mining model is constructed using machine learning, deep learning and statistical methods to learn the attribute information of each node in the biological knowledge network. By analyzing the entity attributes between similar entities, the potential association between the attributes of entities is discovered. Through a large amount of data attribute information, the similarity of attributes between different entities is obtained, and then the association relationship between entities under similar attributes is discovered, and the potential entity relationship is mined.

[0078] Figure 4 The flowchart for extracting the temporal relationship of multi-source heterogeneous data according to an embodiment of the present invention is schematically shown.

[0079] like Figure 4 As shown, the method of extracting the temporal relationship of multi-source heterogeneous data according to an embodiment of the present invention includes steps S21 to S24.

[0080] Step S21: Acquire the features of multi-source heterogeneous data through the convolution layer and learn them to obtain the learning result vector of the convolution layer.

[0081] A convolutional neural network, which excels at extracting features, is used to learn all the features obtained by the vector representation layer. It learns sentences in which two entities co-occur to predict the relationship between them. Feature extraction is achieved by performing convolution operations on the vectors within a sliding window. To learn multiple features, multiple convolution kernels are typically used in the model. Assume that multiple convolution kernels are used in the model.

[0082] For example, assuming that n convolution kernels are used in the model, the convolution matrix is , the convolution operation is expressed as follows:

[0083]

[0084] Where, represents the result vector of the convolution kernel, Represents the length of the convolution kernel sliding window, Represents a sequence.

[0085] Step S22: After the learning result vector is segmented and pooled through the pooling layer, the final output vector is obtained.

[0086] The pooling layer further extracts the features learned by the convolutional layer and adopts the maximum pooling strategy. This strategy selects the maximum value from a series of features learned by each convolution kernel in the convolutional layer to extract the most valuable features. The entire sentence is divided into three segments with two event entities as the dividing points, and the maximum pooling operation is performed on each segment separately.

[0087] For example, the result vector of each convolution kernel is It will be divided into three sections , combine the three pooled vectors together to get the vector . Segmented max pooling operation It can be expressed as:

[0088]

[0089] For example, concatenate all vectors Get the total vector , and perform nonlinear function operations to obtain the final output vector of the pooling layer for:

[0090]

[0091] Step S23: Calculate the similarity between the final output vector and the event relationship to obtain the event association relationship between multiple entities.

[0092] Sentences that correctly express event relationships will receive higher weights, while those that are incorrectly labeled will receive very low weights. The weight of a sentence is obtained by calculating the similarity between the sentence feature representation vector and the event relationship, thereby obtaining the event association relationship between multiple entities.

[0093] For example, for a set of entity pairs , all the sentences in which they appear together Composition Collection , the attention mechanism layer calculates the corresponding weight vector for the set . Then, the characteristics of the set T can be calculated as follows:

[0094]

[0095] Step S24: adding a time series tag to the event association relationship to obtain the time series relationship of the multi-source heterogeneous data.

[0096] Since the knowledge graph needs to reflect the dynamic characteristics of events and the complex associations between events, there are multiple associations between some entities, and the relationship categories are also multi-label and multi-attribute. The temporal multi-label relationship extraction technology based on event attributes is used to combine the preliminary extracted entities with the simple association relationships preset in the ontology library, and the event entities and association relationships are equivalent to vocabulary. The sentence classification model is used to classify the relationships of indirectly associated entities, and time series labels are added to obtain the temporal relationships of multi-source heterogeneous data.

[0097] Figure 5 Schematically shows a principle diagram of extracting entity temporal state sequence features according to an embodiment of the present invention; Figure 6 A block diagram of language model programming according to an embodiment of the present invention is schematically shown.

[0098] like Figure 5 and Figure 6 As shown in Figure 2, the time-series-based knowledge graph relationship update mainly includes: (1) logical rule-based relationship update and completion, which predicts the links of missing elements in the knowledge graph according to a series of inference rules formulated by experts or rules obtained through mining algorithms; (2) tensor decomposition-based update and completion, which models the fact quadruple as a fourth-order tensor and then decomposes the fourth-order tensor so that the quadruple can be embedded in the Euclidean space in the form of a low-dimensional matrix to facilitate link prediction.

[0099] The features of extracting entity temporal state sequences include:

[0100] According to the characteristics and structure of event expert knowledge in the CodeLlama model, the entity time series is converted into a tag sequence suitable for input into the CodeLlama model. The rules and representation of tags are defined according to domain knowledge, and the key information of each expert knowledge is extracted as a tag, or the entire knowledge is used as part of the tag sequence.

[0101] The generated token sequence is encoded and features extracted for representation and learning in the model. Word vectors, embedding layers, or other encoding techniques can be used to convert the token sequence into a numerical representation that the model can understand and process.

[0102] The generated labeled sequences are fed into the CodeLlama model for fine-tuning. During fine-tuning, the model learns how to effectively handle long-range dependencies to better understand and translate event expert knowledge. Model training can be guided by supervised learning tasks with labeled sequences, such as sequence classification, sequence generation, or sequence labeling.

[0103] Define indicators for evaluating generative models Evaluation indicators: Evaluation indicators are used to measure the degree of match between the generated code and event expert knowledge. For each generated code sample, an evaluation is performed based on the expectations of event expert knowledge to check whether the generated code meets the expectations, including code structure, syntax specifications, function logic and other aspects.

[0104] Processing the features of the entity temporal state sequence to update the initial knowledge graph includes:

[0105] Based on the language model programming framework, multiple agent modules are obtained. Each agent module should implement its specific functions and tasks, such as syntax analysis, semantic analysis, program logic, etc. Appropriate programming languages ​​and tools are used to implement the agent modules. The developed agent modules are integrated into the LangChain architecture to ensure smooth collaboration and information exchange between agent modules.

[0106] Define the data flow path from input to output. This can include code input, data transmission and processing between agents, and the final generated code output. Ensure the correct flow of data between agents and perform necessary data processing and conversion. This may involve data structure design, data format conversion, error handling, etc., and manage the control logic and process in the data flow, such as determining the order of agent execution, handling exceptions, and decision-making logic, to ensure the correctness and reliability of the data flow.

[0107] The tuples of events in the temporal knowledge graph are mapped to a low-dimensional vector space, and the temporal order of relations is used to model the evolution of knowledge in the time dimension, so that the entity vectors, relationship vectors and timestamp vectors can be translated in space, the temporal order of relations is used to model the evolution of knowledge in the time dimension, and the observed relationship ordering of head entities is used to regularize the traditional embedding score function.

[0108] Adaptively learn the nonlinear relationships between entities, relations, and timestamps in the knowledge graph, as well as the dynamic transformations between them, fully capturing their semantic features and thus enabling link prediction in the knowledge graph. Neural network-based completion methods can extract knowledge sequence features and mine implicit semantic information.

[0109] A multi-layer network structure is used to learn the features in the knowledge graph, and the learned important features are used to complete the missing parts of the graph. Using a multi-hop reasoning model, a policy network is used to learn the multi-hop reasoning path and continuously adjust the reasoning path in the temporal knowledge graph to update the initial knowledge graph and obtain the temporal knowledge graph.

[0110] In summary, the present invention provides a method for fusion analysis of multi-source heterogeneous data, which has the following beneficial effects:

[0111] (1) Through the entity and attribute extraction technology of multi-source heterogeneous information, the time series multi-label relationship extraction technology and the normalized representation technology of spatiotemporal big data, the massive and diverse knowledge is organized and expressed in an orderly, unified and related manner, and the scattered, diverse and messy data is unified, associated and utilized, thus solving the problem that data is difficult to organize and manage in a unified manner;

[0112] (2) Through the normalized representation technology of spatiotemporal big data and the expert knowledge extraction and rule construction technology, time series labels are introduced into the traditional event association relationship on the basis of the traditional big data information representation knowledge graph, thereby constructing a temporal knowledge graph framework structure with the ability to represent temporal information, solving the problem that the traditional knowledge graph lacks spatiotemporal dimensions.

[0113] Based on the method disclosed in the above embodiment, the present invention also provides a multi-source heterogeneous data fusion analysis system. Figure 7 The system is described in detail.

[0114] Figure 7 The block diagram of the multi-source heterogeneous data fusion analysis system according to an embodiment of the present invention is schematically shown.

[0115] like Figure 7 As shown, the multi-source heterogeneous data fusion analysis system 700 according to an embodiment of the present invention includes a data input module 710 , a data extraction module 720 , a data processing module 730 and a data update module 740 .

[0116] The data input module 710 is used to input multi-source heterogeneous data.

[0117] The data extraction module 720 is used to extract entity information and attribute information from multi-source heterogeneous data.

[0118] The data processing module 730 is used to process entity information and attribute information to obtain an initial knowledge graph and entity time series state sequence.

[0119] The data update module 740 is used to update the initial knowledge graph to obtain a temporal knowledge graph of multi-source heterogeneous data.

[0120] It should be noted that the embodiment of the device part is similar to the embodiment of the method part, and the technical effects achieved are also similar. For specific details, please refer to the above-mentioned method embodiment part, which will not be repeated here.

[0121] According to an embodiment of the present invention, any multiple of the data input module 710, data extraction module 720, data processing module 730, and data update module 740 may be combined into a single module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in a single module. According to an embodiment of the present invention, at least one of the data input module 710, data extraction module 720, data processing module 730, and data update module 740 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or may be implemented in any one of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, at least one of the data input module 710 , the data extraction module 720 , the data processing module 730 and the data update module 740 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architectures, functions and operations of the devices and methods according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] The features described in the various embodiments of the present invention may be combined or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations or couplings fall within the scope of the present invention.

[0124] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A multi-source heterogeneous data fusion analysis method, characterized in that: include: Acquire multi-source heterogeneous event knowledge and extract entity information and attribute information of the multi-source heterogeneous data; After fusing the attribute information, a multi-label classification model is constructed, and the temporal relationship of the multi-source heterogeneous data is extracted through the multi-label classification model; After adding time series labels to the time series relationships, an initial knowledge graph of the multi-source heterogeneous data is constructed; Obtaining an entity temporal state sequence of the multi-source heterogeneous data through the initial knowledge graph and the entity information; Extracting features of the entity temporal state sequence; The features of the entity temporal state sequence are processed to update the initial knowledge graph to obtain the temporal knowledge graph of the multi-source heterogeneous data.

2. The method according to claim 1, wherein The entity information includes entity attributes, entity relationships, and entity elements. The extraction of entity information and attribute information from the multi-source heterogeneous data includes: Extracting entity elements of the multi-source heterogeneous data, and obtaining event information of the multi-source heterogeneous data through the entity elements; Detecting and parsing the event information to obtain the entity attributes; Obtaining attribute information of each node of the multi-source heterogeneous data through the entity attributes; The attribute information is classified and analyzed to obtain the entity relationship.

3. The method according to claim 1, characterized in that The multi-source heterogeneous data includes multiple entities, and the multi-label classification model can obtain event relationships between the multiple entities.

4. The method according to claim 3, wherein: The multi-label classification model includes a convolution layer and a pooling layer, and extracting the temporal relationship of the multi-source heterogeneous data through the multi-label classification model includes: Acquire the features of the multi-source heterogeneous data through the convolution layer and perform learning to obtain a learning result vector of the convolution layer; After segmenting and pooling the learning result vector through the pooling layer, a final output vector is obtained; Calculating the similarity between the final output vector and the event relationship to obtain the event association relationship between the multiple entities; Adding a time series tag to the event association relationship obtains the time series relationship of the multi-source heterogeneous data.

5. The method according to claim 1, wherein The entity time series state sequence of the multi-source heterogeneous data obtained by using the knowledge graph and the entity information includes: Constructing an event ontology library through the knowledge graph and the entity information; Constructing an entity set of the multi-source heterogeneous data through the event ontology library; The current state feature vector of the multi-source heterogeneous data is constructed through the entity set to form the entity time series state sequence.

6. The method according to claim 1, wherein The extracting of the features of the entity temporal state sequence comprises: Converting the entity temporal state sequence into a tag sequence; fine-tuning the tag sequence using an open base code model; The fine-tuned tag sequence is trained and evaluated using the open base code model to obtain features of the entity temporal state sequence.

7. The method according to claim 6, characterized in that The evaluation indicators include: code accuracy, grammatical standardization, and semantic consistency.

8. The method according to claim 1, wherein Processing the features of the entity temporal state sequence to update the initial knowledge graph to obtain the temporal knowledge graph of the multi-source heterogeneous data includes: Multiple agent modules are obtained according to the language model programming framework; constructing a translation model through the plurality of agent modules; Processing the features of the entity temporal state sequence through the translation model to obtain entity information missing in the initial knowledge graph; After the missing entity information is trained, it is input into the initial knowledge graph to obtain the temporal knowledge graph of the multi-source heterogeneous data.

9. The method according to claim 8, characterized in that The update includes: the nonlinear relationship between entities, relationships and timestamps in the initial knowledge graph, the reasoning path in the initial knowledge graph, and the time information and event association relationship in the initial knowledge graph.

10. A multi-source heterogeneous data fusion analysis system, characterized in that: include: Data input module, used to input multi-source heterogeneous data; A data extraction module, configured to extract entity information and attribute information from the multi-source heterogeneous data; A data processing module, configured to process the entity information and the attribute information to obtain an initial knowledge graph and an entity temporal state sequence; A data updating module is used to update the initial knowledge graph to obtain a temporal knowledge graph of the multi-source heterogeneous data.

Images