Multi-view information processing method based on knowledge graph

Abstract

The invention provides a multi-view information processing method based on a knowledge graph, which comprises the following steps: collecting data, and carrying out preprocessing and feature extraction to obtain multi-view information; obtaining structured data in the multi-view information, and constructing an initial knowledge graph; establishing cross-view association and alignment among data of different views based on the initial knowledge graph, and performing cross-view information fusion; performing cross-domain and cross-modal information reasoning by using a cross-modal neural network based on the fused cross-view information to obtain a multi-view knowledge graph; migration learning and self-supervised learning are introduced into the multi-view knowledge graph, reinforcement learning is carried out, and a feedback mechanism is introduced to update the multi-view knowledge graph in real time. According to the method, multi-source and multi-dimensional data can be efficiently processed, information islands can be broken through, data values can be fully mined, the knowledge graph can be dynamically updated and optimized, the self-adaptability and accuracy of the system are improved, and the method has wide application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a multi-view information processing method based on a knowledge graph. Background Art

[0002] In the information age, there are more and more data sources and the types of data are also becoming more and more diverse. Multi-view information refers to multiple information sources obtained from different angles or perspectives of the same object or scene. In the fields of computer vision and machine learning, multi-view information is often used to improve the performance of tasks such as object recognition and target detection. At present, in dealing with multi-view information, existing multi-view information processing methods face problems such as data redundancy, low processing efficiency, and information silos, resulting in the inability to effectively and efficiently utilize multi-source data, which seriously restricts the performance of big data analysis and intelligent systems. How to effectively fuse information from different views so that multi-view information can complement and cooperate with each other to improve the performance of the model, and how to enable different view information to be fully represented and utilized, and establish effective associations and interactions between different views, are problems that need to be urgently solved in current multi-view information processing.

[0003] A knowledge graph is a data structure that represents the relationships between entities and concepts as a graph, which can provide rich semantic information and association information. By using the knowledge graph, it can help solve the problems existing in multi-view information processing. Therefore, the present invention proposes a multi-view information processing method based on a knowledge graph to effectively solve the above problems. Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art, the purpose of the present invention is to provide a multi-view information processing method based on a knowledge graph, which can efficiently process multi-source and multi-dimensional data, break through information silos, fully explore the data value, and can dynamically update and optimize the knowledge graph, improving the self-adaptability and accuracy of the system, and having broad application prospects.

[0005] To achieve the above object, the present invention provides the following solution: A multi-view information processing method based on a knowledge graph, comprising the following steps:

[0006] Collect multi-dimensional and multi-source data, preprocess and extract features from the collected data to obtain multi-view information;

[0007] Obtain structured data from the multi-view information and construct an initial knowledge graph;

[0008] Based on the initial knowledge graph, establish cross-view associations and alignments between data of different views to perform cross-view information fusion;

[0009] Based on the fused cross-view information, using a cross-modal neural network, perform cross-domain and cross-modal information reasoning between different views to obtain a multi-view knowledge graph;

[0010] Introduce transfer learning and self-supervised learning in the multi-view knowledge graph, perform reinforcement learning to achieve multi-view information processing, and introduce a feedback mechanism to update the multi-view knowledge graph in real time.

[0011] Optionally, the construction process of the initial knowledge graph is as follows:

[0012] Identify the entities in the multi-view information, unify the representation of different types of entities according to the characteristics of different data views, and determine the relationships between entities;

[0013] Pre-define the relationships between entities, and construct a relationship graph between entities according to the association information in different data views;

[0014] Finally, integrate the constructed entities and the pre-defined relationships, store them in a graph database to form a structured knowledge representation, and obtain the initial knowledge graph.

[0015] Optionally, the process of performing cross-view association and alignment using the initial knowledge graph is as follows:

[0016] Map the entities and attributes in different views to the entities and relationships in the initial knowledge graph respectively, and establish the corresponding relationships between different views and the initial knowledge graph;

[0017] Based on the entity similarity and association information in the initial knowledge graph, perform similarity calculation to align the entities between different views and obtain the entity alignment relationship between different views;

[0018] Based on the entity alignment relationship, use the association information of entities and attributes in the initial knowledge graph to map and align the attributes in different views, and then use the relationship information in the multi-view knowledge graph to align and match the relationships in different views to obtain the relationship alignment relationship between different views;

[0019] Utilize the aligned entity and relationship information to achieve cross-view information fusion.

[0020] Optionally, the process of performing cross-domain and cross-modal information reasoning between different views using a cross-modal neural network is as follows:

[0021] Based on the entities associated between different views, establish cross-view knowledge links, and through these knowledge links, perform cross-view information transmission and reasoning to achieve information interaction and fusion between different views;

[0022] Based on the fused cross-view information, complement the information between different views to achieve the supplementation of different view information;

[0023] Design a cross-modal neural network that can process different view data simultaneously. Introduce aligned entity and relationship information in the cross-modal neural network to learn the interaction relationships between different view information, and achieve cross-domain and cross-modal information reasoning to obtain the multi-view knowledge graph.

[0024] Optionally, the process of introducing transfer learning and self-supervised learning for reinforcement learning in the multi-view knowledge graph is as follows:

[0025] Use the graph neural network model to learn the representation vectors of each entity and relationship in the multi-view knowledge graph to obtain the low-dimensional vector representations of each entity and relationship;

[0026] Take the low-dimensional vector representations as prior knowledge, combine the information of other views, and transfer the information learned in the multi-view knowledge graph to the representation learning tasks of other views through transfer learning to improve the representation ability of multi-view information;

[0027] Define self-supervised learning tasks on the low-dimensional vector representations for node attribute prediction and link prediction to improve the generalization and promotion ability of knowledge representation;

[0028] Use the low-dimensional vector representations as a medium to integrate different view information into the same representation space to achieve reinforcement learning of multi-view information.

[0029] Optionally, the process of introducing a feedback mechanism to update the multi-view knowledge graph in real time is as follows:

[0030] Introduce a real-time update mechanism, use a data monitoring and acquisition system to monitor the changes of multiple data sources and update the data in real time;

[0031] Introduce a user feedback mechanism to receive the feedback and suggestions of users on the information in the knowledge graph, and adjust the content of the multi-view knowledge graph according to the user feedback;

[0032] Regularly conduct data quality assessment to ensure the accuracy of the data.

[0033] The present invention discloses the following technical effects by providing a multi-view information processing method based on a knowledge graph:

[0034] 1. By constructing a knowledge graph, the present invention can utilize the entities and relationships in the knowledge graph to help with precise matching and alignment between different views, achieve the fusion of multi-view information, help the model better understand and utilize cross-modal information, and improve the comprehensive utilization efficiency and performance of multi-view information. Moreover, it can effectively utilize the knowledge in the knowledge graph to achieve cross-domain and cross-modal information reasoning, improve the utilization efficiency and performance of multi-view information, help the model more comprehensively understand data in different domains and modalities, and achieve more accurate information reasoning and prediction.

[0035] 2. By introducing transfer learning and self-supervised learning into the knowledge graph, the present invention can effectively combine information from different views and enhance the learning effect of multi-view information. Such a method can improve the model's comprehensive utilization ability of multi-modal information and enhance the generalization performance of information representation, thereby enhancing the model's learning and reasoning ability and helping the model more comprehensively understand and utilize diverse data information.

[0036] 3. The present invention can achieve efficient processing of multi-source and multi-dimensional data, break through information silos, fully explore data value, and improve the system's self-adaptability and accuracy by dynamically updating and optimizing the knowledge graph. The present invention will have broad application prospects in the fields of intelligent systems, data analysis, human-computer interaction, etc., and can significantly improve the system's performance and user experience.

[0037] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the accompanying drawings required for use in the embodiments. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0039] Figure 1 It is a flowchart of the method provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0041] To make the above objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] As Figure 1 shown, the present invention provides a multi-view information processing method based on a knowledge graph, including the following steps:

[0043] 1. Collect multi-dimensional and multi-source data, preprocess and extract features from the collected data to obtain multi-view information.

[0044] The collected data information may include text files, pictures, videos, and audios, etc.

[0045] The preprocessing process includes cleaning the collected data, removing noise, eliminating redundant information, converting the data format, and annotating and classifying the data to ensure the reliability and consistency of the data.

[0046] For different types of data views, feature extraction is performed; for text data, natural language processing techniques can be used to extract entities and relationships; for image data, techniques such as convolutional neural networks can be used to extract image feature vectors; for audio data, sound processing techniques can be used to extract sound features.

[0047] These steps contribute to subsequent knowledge graph construction and data analysis.

[0048] 2. Obtain structured data from the multi-view information and construct an initial knowledge graph. The specific process is as follows:

[0049] Identify the entities in the multi-view information, unify the representation of different types of entities according to the characteristics of different data views, and determine the relationships between the entities. These entities can represent different roles and objects in the data.

[0050] Pre-define the relationships between entities, and construct a relationship graph between the entities according to the association information in different data views. The relationships can include semantic associations, syntactic associations, etc., to help understand the connections between the entities.

[0051] Finally, integrate the constructed entities and the pre-defined relationships and store them in a graph database to form a structured knowledge representation, obtaining an initial knowledge graph. The graph database query language can be used for retrieval and query.

[0052] Constructing the initial knowledge graph can help the model better understand and utilize diverse data information, improving the integration and utilization effect of the information.

[0053] 3. Based on the initial knowledge graph, establish cross-view associations and alignments between the data of different views to perform cross-view information fusion; the specific process is as follows:

[0054] Map the entities and attributes in different views to the entities and relationships in the initial knowledge graph respectively, establish the corresponding relationship between different views and the initial knowledge graph, and the corresponding relationship between views and the knowledge graph can be established by matching information such as entity names and attribute features.

[0055] Based on the entity similarity and association information in the initial knowledge graph, perform similarity calculation to align the entities between different views and obtain the entity alignment relationship between different views.

[0056] Based on the entity alignment relationship, use the association information of entities and attributes in the initial knowledge graph to map and align the attributes in different views, and the attribute mapping can be performed through information such as semantic similarity of attributes and similarity of attribute values.

[0057] Then use the relationship information (relationship paths, relationship types, etc.) in the multi-view knowledge graph to align and match the relationships in different views and obtain the relationship alignment relationship between different views.

[0058] Utilize the aligned entity and relationship information to achieve cross-view information fusion and improve the representation and fusion effect of multi-view information.

[0059] 4. Based on the fused cross-view information, use a cross-modal neural network to perform cross-domain and cross-modal information reasoning between different views to obtain a multi-view knowledge graph; the specific process is as follows:

[0060] Based on the entities associated between different views, establish cross-view knowledge links, and through the knowledge links, perform cross-view information transmission and reasoning to achieve information interaction and fusion between different views;

[0061] Based on the fused cross-view information, complement the information between different views, realize the complement of different view information, and improve the comprehensive utilization efficiency of multi-view information.

[0062] Design a cross-modal neural network that can process different view data simultaneously, introduce the aligned entity and relationship information into the cross-modal neural network, learn the interaction relationship between different view information, and achieve cross-domain and cross-modal information reasoning to obtain the multi-view knowledge graph, thereby improving the efficiency and accuracy of information reasoning.

[0063] 5. Introduce transfer learning and self-supervised learning into the multi-view knowledge graph to perform reinforcement learning to achieve multi-view information processing, and introduce a feedback mechanism to update the multi-view knowledge graph in real time.

[0064] 5.1 The process of introducing transfer learning and self-supervised learning into the multi-view knowledge graph for reinforcement learning is as follows:

[0065] Using a graph neural network model, learn the representation vectors of each entity and relationship in the multi-view knowledge graph to obtain the low-dimensional vector representations of each entity and relationship.

[0066] Taking the low-dimensional vector representations as prior knowledge, combining the information of other views, and through the method of transfer learning, transfer the information learned in the multi-view knowledge graph to the representation learning tasks of other views to improve the representation ability of multi-view information.

[0067] Define self-supervised learning tasks on the low-dimensional vector representations, such as node attribute prediction and link prediction, to improve the generalization and promotion ability of knowledge representation, and thus enhance the learning effect of multi-view information.

[0068] Using the low-dimensional vector representations as a medium, integrate the information of different views into the same representation space to achieve the reinforcement learning of multi-view information.

[0069] 5.2 The process of introducing a feedback mechanism to update the multi-view knowledge graph in real time is as follows:

[0070] Introduce a real-time update mechanism. Use a data monitoring and acquisition system to monitor the changes of multiple data sources and update the data in real time; regularly detect the update requirements and frequencies of the data in the knowledge graph. By monitoring the changes of the data sources, determine when to update the entity and relationship information in the knowledge graph.

[0071] Introduce a user feedback mechanism to receive the feedback and suggestions of users on the information in the knowledge graph, and adjust the content of the multi-view knowledge graph according to the user feedback; optimize the data quality and accuracy of the knowledge graph.

[0072] Regularly conduct data quality assessment to ensure the accuracy of the data, discover problems and repair them in time to ensure the effectiveness and reliability of the knowledge graph.

[0073] Therefore, by providing a multi-view information processing method based on a knowledge graph, the present invention can efficiently process multi-source and multi-dimensional data, break through information islands, fully explore the data value, and can dynamically update and optimize the knowledge graph, improving the self-adaptability and accuracy of the system, and having a wide range of application prospects.

[0074] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other.

[0075] In this article, specific examples are used to illustrate the principles and implementation modes of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation modes and application scopes. To sum up, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A multi-view information processing method based on a knowledge graph, characterized in that It includes the following steps: Collect multi-dimensional and multi-source data, preprocess and extract features from the collected data to obtain multi-view information; Obtain structured data from the multi-view information and construct an initial knowledge graph; Based on the initial knowledge graph, establish cross-view associations and alignments between data in different views to perform cross-view information fusion; Based on the fused cross-view information, use a cross-modal neural network to perform cross-domain and cross-modal information reasoning between different views to obtain a multi-view knowledge graph; Introduce transfer learning and self-supervised learning into the multi-view knowledge graph to perform reinforcement learning to achieve multi-view information processing, and introduce a feedback mechanism to update the multi-view knowledge graph in real time.

2. The multi-view information processing method based on a knowledge graph according to claim 1, wherein, The construction process of the initial knowledge graph is as follows: Identify entities in the multi-view information, unify the representation of different types of entities according to the characteristics of different data views, and determine the relationships between entities; Pre-define the relationships between entities, and construct a relationship graph between entities according to the association information in different data views; Finally, integrate the constructed entities and the pre-defined relationships and store them in a graph database to form a structured knowledge representation and obtain an initial knowledge graph.

3. A multi-view information processing method based on a knowledge graph according to claim 2, characterized in that The process of using the initial knowledge graph for cross-view association and alignment is as follows: Map entities and attributes in different views to entities and relationships in the initial knowledge graph respectively to establish the corresponding relationship between different views and the initial knowledge graph; Based on the entity similarity and association information in the initial knowledge graph, perform similarity calculation to align entities between different views and obtain the entity alignment relationship between different views; Based on the entity alignment relationship, use the association information of entities and attributes in the initial knowledge graph to map and align attributes in different views, and then use the relationship information in the multi-view knowledge graph to align and match relationships in different views to obtain the relationship alignment relationship between different views; Use the aligned entity and relationship information to achieve cross-view information fusion.

4. A method for processing multi-view information based on a knowledge graph according to claim 3, characterized in that, The process of using a cross-modal neural network to perform cross-domain and cross-modal information reasoning between different views is as follows: Based on the entities associated between different views, establish cross-view knowledge links, and through the knowledge links, perform cross-view information transfer and reasoning to achieve information interaction and fusion between different views; Based on the fused cross-view information, complement the information between different views to achieve the complement of different view information; Design a cross-modal neural network that can process data from different views simultaneously, introduce the aligned entity and relationship information into the cross-modal neural network, learn the interaction relationship between different view information, and achieve cross-domain and cross-modal information reasoning to obtain the multi-view knowledge graph.

5. A multi-view information processing method based on a knowledge graph according to claim 4, characterized in that The process of introducing transfer learning and self-supervised learning into the multi-view knowledge graph for reinforcement learning is as follows: Use a graph neural network model to learn the representation vectors of each entity and relationship in the multi-view knowledge graph to obtain the low-dimensional vector representations of each entity and relationship; Taking the low-dimensional vector representation as prior knowledge, combining the information of other views, and through the method of transfer learning, transferring the information learned in the multi-view knowledge graph to the representation learning tasks of other views to improve the representation ability of multi-view information; Defining self-supervised learning tasks on the low-dimensional vector representation for node attribute prediction and link prediction to improve the generalization and promotion ability of knowledge representation; Using the low-dimensional vector representation as a medium to integrate different view information into the same representation space to achieve reinforcement learning of multi-view information.

6. A method for processing multi-view information based on a knowledge graph according to claim 5, characterized in that The process of introducing a feedback mechanism to update the multi-view knowledge graph in real time is as follows: Introducing a real-time update mechanism, using a data monitoring and acquisition system to monitor the changes of multiple data sources and update the data in real time; Introducing a user feedback mechanism to receive the feedback and suggestions of users on the information in the knowledge graph and adjust the content of the multi-view knowledge graph according to the user feedback; Regularly conducting data quality assessment to ensure the accuracy of the data.

Images