A data processing method, apparatus and storage medium

Abstract

A data processing method, apparatus, and storage medium, relating to the field of communication technology, are disclosed to address the problem of low effectiveness in the use of knowledge graphs in general technologies. The method includes: acquiring data to be associated with a target data node; the target data node being any data node in a pre-constructed knowledge graph; determining the relevance between the data to be associated and at least one data node in the knowledge graph; the at least one data node including the target data node; and determining the data to be associated as associated data of the target data node when the relevance is greater than an association threshold.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a data processing method, apparatus and storage medium. Background Technology

[0002] A knowledge graph is considered a semantic network describing the relationships between entities, capable of describing real-world things and their interrelationships. External search platforms can improve the intelligence of their search engines, optimize search recommendations, and enhance user experience by building knowledge graphs. Compared to external search platforms, internal knowledge management platforms possess more proprietary knowledge, which is more conducive to employee work and learning. Therefore, knowledge graphs built through internal knowledge management platforms can effectively improve the intelligence of internal platforms, prevent the idleness of internal knowledge resources, and help employees quickly search for relevant skills and knowledge.

[0003] A knowledge graph contains multiple data nodes with logical relationships. The server can determine the associated data of any data node, and subsequently, the client can request the associated data of any data node in the knowledge graph from the server.

[0004] The server can typically identify data related to a data node as associated data for that data node. However, this common method has the problem of associated data being either irrelevant or only marginally relevant to the subject matter of the knowledge graph. For example, when the same data expresses different meanings in multiple different subjects, the server might identify data related to that data in one subject as associated data for the corresponding data node in another subject's knowledge graph. This results in the client requesting associated data through the knowledge graph that is irrelevant to the subject matter, affecting the effectiveness of the knowledge graph. Summary of the Invention

[0005] This application provides a data processing method, apparatus, and storage medium to address the problem of low effectiveness of knowledge graphs in general technologies.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, a data processing method is provided, comprising: acquiring data to be associated with a target data node; the target data node being any data node in a pre-constructed knowledge graph; determining the relevance between the data to be associated and at least one data node in the knowledge graph; at least one data node including the target data node; and determining the data to be associated as associated data of the target data node when the relevance is greater than an association threshold.

[0008] Optionally, the data processing method further includes: acquiring raw data for constructing a knowledge graph; determining multiple entity data and multiple relation features of the raw data; using entity data to represent the data content of the raw data; using relation features to represent the logical relationship between multiple entity data; determining multiple data nodes of the knowledge graph based on the multiple entity data; and constructing the knowledge graph based on the correspondence between the multiple data nodes and the multiple relation features.

[0009] Optionally, the original data includes structured data or unstructured data; when the original data includes unstructured data, multiple entity data of the original data are determined, including: determining the first entity data based on the entity extraction algorithm and the unstructured data; determining the high-frequency data in the unstructured data content that appears more frequently than a first frequency threshold as the second entity data; and fusing the first entity data and the second entity data according to a preset data processing strategy to obtain multiple entity data.

[0010] Optionally, when the original data includes unstructured data, multiple relational features of the original data are determined, including: determining a first entity relation based on the relation extraction algorithm and the unstructured data; identifying entity relations in the unstructured data content that appear more frequently than a second frequency threshold as second entity relations; and fusing the first entity relation and the second entity relation according to the data processing strategy to obtain multiple relational features.

[0011] Optionally, based on multiple entity data, multiple data nodes of the knowledge graph are determined, including: when the relevance between the target entity data and at least one entity data other than the target entity data in the multiple entity data is greater than or equal to a relevance threshold, the target entity data is determined as a data node of the knowledge graph; the target entity data is any one of the multiple entity data.

[0012] Optionally, the data processing method further includes: receiving a request message from the client; the request message is used to request the acquisition of associated data of the target data node; and in response to the request message, sending the associated data of the target data node to the client.

[0013] In a second aspect, a data processing apparatus is provided, comprising: an acquisition unit and a processing unit; the acquisition unit is configured to acquire data to be associated with a target data node; the target data node is any data node in a pre-constructed knowledge graph; the processing unit is configured to determine the relevance between the data to be associated and at least one data node in the knowledge graph; the at least one data node includes the target data node; the processing unit is further configured to determine that the data to be associated is associated data of the target data node when the relevance is greater than an association threshold.

[0014] Optionally, the acquisition unit is further configured to acquire the original data used to construct the knowledge graph; the processing unit is further configured to determine multiple entity data and multiple relation features of the original data; the entity data is used to represent the data content of the original data; the relation features are used to represent the logical relationship between the multiple entity data; the processing unit is further configured to determine multiple data nodes of the knowledge graph based on the multiple entity data; the processing unit is further configured to construct the knowledge graph based on the correspondence between the multiple data nodes and the multiple relation features.

[0015] Optionally, the original data includes structured data or unstructured data. When the original data includes unstructured data, the processing unit is specifically used to: determine the first entity data based on the entity extraction algorithm and the unstructured data; determine the high-frequency data in the unstructured data content that appears more frequently than a first frequency threshold as the second entity data; and fuse the first entity data and the second entity data according to a preset data processing strategy to obtain multiple entity data.

[0016] Optionally, when the original data includes unstructured data, the processing unit is specifically used to: determine a first entity relationship based on the relation extraction algorithm and the unstructured data; identify entity relationships in the unstructured data whose frequency of occurrence is higher than a second frequency threshold as second entity relationships; and fuse the first entity relationship and the second entity relationship according to the data processing strategy to obtain multiple relation features.

[0017] Optionally, the processing unit is specifically used to: determine the target entity data as a data node of the knowledge graph when the relevance between the target entity data and at least one entity data other than the target entity data among multiple entity data is greater than or equal to the relevance threshold; the target entity data is any one of the multiple entity data.

[0018] Optionally, the data processing apparatus further includes: a receiving unit and a sending unit; the receiving unit is used to receive a request message from the client; the request message is used to request the acquisition of associated data of the target data node; the sending unit is used to send the associated data of the target data node to the client in response to the request message.

[0019] Thirdly, a data processing apparatus is provided, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is connected to the memory via a bus; when the data processing apparatus is running, the processor executes the computer-executable instructions stored in the memory to cause the data processing apparatus to perform the data processing method described in the first aspect.

[0020] The data processing apparatus may be a network device or a component of a network device, such as a chip system within the network device. The chip system supports the network device in implementing the functions involved in the first aspect and any of its possible implementations, such as acquiring, determining, and transmitting data and / or information involved in the aforementioned data processing method. The chip system includes a chip and may also include other discrete devices or circuit structures.

[0021] Fourthly, a computer-readable storage medium is provided, comprising computer-executable instructions that, when executed on a computer, cause the computer to perform the data processing method described in the first aspect.

[0022] Fifthly, a computer program product is also provided, comprising computer instructions that, when executed on a data processing apparatus, cause the data processing apparatus to perform the data processing method as described in the first aspect above.

[0023] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on the first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the data processing device, or it may be packaged separately from the processor of the data processing device; this application embodiment does not limit this.

[0024] The descriptions of the second, third, fourth, and fifth aspects in this application can be referenced to the detailed description of the first aspect; and the beneficial effects of the second, third, fourth, and fifth aspects can be referenced to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0025] In the embodiments of this application, the names of the aforementioned data processing devices do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalents.

[0026] These or other aspects of this application will become more readily apparent in the following description.

[0027] The technical solution provided in this application brings at least the following beneficial effects:

[0028] Based on any of the above aspects, embodiments of this application provide a data processing method. After the server obtains the data to be associated with a target data node, it can determine the relevance between the data to be associated and at least one data node in the knowledge graph. When the relevance is greater than the association threshold, the server can determine that the data to be associated is associated data of the target data node. Since the relevance between the associated data and at least one data node is greater than the association threshold, it indicates that the associated data belongs to the profession of the knowledge graph. Compared with general technologies, this solves the problem that the associated data is unrelated or has a low relevance to the profession of the knowledge graph, thus improving the effectiveness and accuracy of finding associated data in the knowledge graph. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application;

[0030] Figure 2 A schematic diagram of the server structure provided in an embodiment of this application;

[0031] Figure 3 This is a schematic diagram of the hardware structure of a data processing device provided in an embodiment of this application;

[0032] Figure 4 A schematic diagram of the hardware structure of another data processing device provided in the embodiments of this application;

[0033] Figure 5 A flowchart illustrating a data processing method provided in this application embodiment. Figure 1 ;

[0034] Figure 6 A schematic diagram of a knowledge graph provided in an embodiment of this application;

[0035] Figure 7 A flowchart illustrating a data processing method provided in this application embodiment. Figure 2 ;

[0036] Figure 8 A flowchart illustrating a data processing method provided in this application embodiment. Figure 3 ;

[0037] Figure 9 A flowchart illustrating a data processing method provided in this application embodiment. Figure 4 ;

[0038] Figure 10 A flowchart illustrating a data processing method provided in this application embodiment. Figure 5 ;

[0039] Figure 11 A flowchart illustrating a data processing method provided in this application embodiment. Figure 6 ;

[0040] Figure 12 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0042] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0043] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0044] As shown in the background section, general technologies suffer from the problem that the related data and knowledge graphs are either not professionally relevant or have a low degree of relevance.

[0045] This application provides a data processing method. After the server obtains the data to be associated with a target data node, it can determine the relevance between the data to be associated and at least one data node in the knowledge graph. When the relevance is greater than the association threshold, the server can determine that the data to be associated is associated data of the target data node. Since the relevance between the associated data and at least one data node is greater than the association threshold, it indicates that the associated data belongs to the profession of the knowledge graph. Compared with general technologies, this solves the problem that the associated data is not related to or has a low relevance to the profession of the knowledge graph, thus improving the effectiveness and accuracy of finding associated data in the knowledge graph.

[0046] This data processing method is applicable to data processing systems. Figure 1 A schematic diagram of the structure of a data processing system is shown. For example... Figure 1 As shown, the data processing system includes a server 110 and a client 120. The server 110 and the client 120 are connected.

[0047] In one embodiment, server 110 can connect to multiple clients. For ease of understanding, this application uses the example of "connection between server 110 and one client 120" for illustration.

[0048] Specifically, server 110 can pre-create the knowledge graph. Subsequently, users can request access to the knowledge graph from server 110 through client 120.

[0049] In one embodiment, the server 110 may be one of the servers in a server cluster (composed of multiple servers), a chip in the server, a system-on-a-chip in the server, or a virtual machine (VM) deployed on a physical machine. This application embodiment does not limit this.

[0050] In one embodiment, the client 120 can be either a wireless terminal or a wired terminal. Examples include smart devices such as mobile phones, personal computers (PCs), desktop computers, tablets, laptops, netbooks, and personal digital assistants (PDAs).

[0051] Combination Figure 1 ,like Figure 2 As shown, the functional modules of the server 110 may include: ontology management module 210, information extraction module 220 (including entity extraction module 221 and relation extraction module 222), knowledge fusion module 230, knowledge graph visualization module 240 and knowledge quality assessment module 250.

[0052] The entity management module 210 is used for manually setting up datasets and mechanically filtering data. Users can enter, modify, or delete data from various disciplines within the dataset. Mechanically filtered data can pre-select data from its content.

[0053] The information extraction module 220 is used to determine entity data and relationship features based on preset entity extraction strategies and relationship extraction strategies.

[0054] The entity extraction module 221 is used to perform entity recognition on the data pre-screened by the ontology management module 210 in the original data to obtain entity data.

[0055] The relation extraction module 222 is used to identify relations in the data pre-screened by the ontology management module 210 in the original data and obtain relation features.

[0056] The knowledge fusion module 230 is used to further process the entity data determined by the information extraction module 220, including data aggregation, ambiguity resolution, and other processing on historical data, internally extracted data, and externally crawled data, to obtain information such as synonyms and related words, thereby obtaining high-quality information extraction results.

[0057] The knowledge graph visualization module 240 is used to display knowledge graphs.

[0058] The knowledge quality assessment module 250 is used to evaluate the data to be associated, and the data that passes the assessment can be identified as associated data in the knowledge graph.

[0059] Combination Figure 1 The server 110 and client 120 in the data processing system both include Figure 3 or Figure 4 The components included in the communication device shown. The following are examples... Figure 3 and Figure 4 Taking the communication device shown as an example, the hardware structure of the server 110 and the client 120 is introduced.

[0060] like Figure 3 The diagram shown is a hardware structure schematic of a communication device provided in an embodiment of this application. The communication device includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, the memory 22, and the communication interface 23 are connected via the bus 24.

[0061] Processor 21 is the control center of the communication device. It can be a single processor or a collective term for multiple processing elements. For example, processor 21 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0062] As one embodiment, processor 21 may include one or more CPUs, for example Figure 3 CPU 0 and CPU 1 are shown in the diagram.

[0063] The memory 22 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0064] In one possible implementation, the memory 22 can exist independently of the processor 21. The memory 22 can be connected to the processor 21 via a bus 24 and is used to store instructions or program code. When the processor 21 calls and executes the instructions or program code stored in the memory 22, it can implement the data processing method provided in the following embodiments of the present invention.

[0065] In another possible implementation, the memory 22 can also be integrated with the processor 21.

[0066] Communication interface 23 is used for connecting the communication device to other devices via a communication network, such as Ethernet, wireless access network, wireless local area network (WLAN), etc. Communication interface 23 may include a receiving unit for receiving data and a transmitting unit for sending data.

[0067] Bus 24 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0068] Figure 4 Another hardware structure of the communication device in an embodiment of the present invention is shown. For example... Figure 4 As shown, the communication device may include a processor 31 and a communication interface 32. The processor 31 is coupled to the communication interface 32.

[0069] The functions of processor 31 can be referred to in the description of processor 21 above. In addition, processor 31 also has a storage function, and can perform the functions of memory 22 mentioned above.

[0070] The communication interface 32 is used to provide data to the processor 31. The communication interface 32 can be an internal interface of the communication device or an external interface of the communication device (equivalent to communication interface 23).

[0071] It should be pointed out that, Figure 3 (or Figure 4 The structure shown in the diagram does not constitute a limitation on the communication device, except... Figure 3 (or Figure 4 In addition to the components shown in the diagram, the communication device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0072] The data processing method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0073] The data processing method includes: a process where the server determines multiple data nodes and multiple relation features based on the original data to construct a knowledge graph (hereinafter referred to as the "knowledge graph construction process"), and a process where the server determines the associated data of the knowledge graph based on the relevance between the data to be associated and at least one data node in the knowledge graph (hereinafter referred to as the "associated data determination process").

[0074] The following describes the knowledge graph construction process. For example... Figure 5 As shown, the data processing method includes:

[0075] S501. The server obtains the raw data used to build the knowledge graph.

[0076] Optionally, the raw data may include structured data, such as relational databases, unstructured data, such as audio and video, and semi-structured data such as Extensible Markup Language (XML), JavaScript Object Notation (JSON), and encyclopedias.

[0077] Optionally, the data format of the raw data may include editable data formats, such as text (Word) format, plain text (TXT) format, etc., and may also include non-editable data formats, such as portable document format (PDF) format, etc.

[0078] In one feasible approach, after the server obtains the raw data in an uneditable format, it can convert the raw data into an editable format.

[0079] In one possible approach, the server can obtain the raw data used to construct the knowledge graph by either importing the raw data from the system or receiving the raw data sent by the client.

[0080] S502, The server determines multiple entity data and multiple relational features of the original data.

[0081] Entity data represents the content of the original data. Relationship features represent the logical relationships between multiple entity data.

[0082] Optionally, the data content may include: text, images, videos, audio, etc.

[0083] In one feasible way, combining Figure 2 The method for the server to determine multiple entity data and multiple relation features of the original data may include: After the server obtains the original data, the ontology management module 210 can identify the data content of the original data, determine the data type of the original data, and perform pre-screening. Then, the ontology management module 210 can determine the predefined relations of the original data based on the data type of the original data. Subsequently, the entity extraction module 221 in the information extraction module 220 can further determine the entity data based on the pre-screened data. The relation extraction module 222 can further determine the relation features based on the predefined relations.

[0084] In one feasible way, combining Figure 2 When the original data includes structured data, the information extraction module 220 can determine entity data and relational features based on the structured configuration.

[0085] When the original data includes unstructured data, the information extraction module 220 can determine the entity data according to the preset entity extraction algorithm and rules, and determine the relationship features according to the preset relationship extraction algorithm and rules.

[0086] S503: The server determines multiple data nodes of the knowledge graph based on multiple entity data.

[0087] Specifically, since the multiple entity data identified from the original data content may contain ambiguous data or other data with low relevance to the profession to which the original data belongs, it is necessary to further determine multiple data nodes of the knowledge graph based on the multiple entity data.

[0088] In one feasible way, combining Figure 2The knowledge fusion module 230 can process multiple entity data, including data aggregation, ambiguity elimination, and other processing of historical data, internally extracted data, and externally crawled data, and obtain information such as synonyms and related words.

[0089] In one feasible way, combining Figure 2 The knowledge fusion module 230 can process multiple entity data through a preset data processing model.

[0090] Optionally, the data processing model may include: word to vector (word2vec), named entity recognition (NER), etc.

[0091] S504. The server constructs a knowledge graph based on the correspondence between multiple data nodes and multiple relational features.

[0092] Specifically, multiple data nodes are connected based on the correspondence of multiple relational features to obtain a network-like knowledge graph. When a client requests access to the knowledge graph, it is displayed on the client's screen. Each data node can be associated with multiple related data, facilitating user searching and knowledge graph maintenance.

[0093] Optionally, the knowledge graph can be stored in a Neo4j graph database.

[0094] For example, such as Figure 6 As shown, the knowledge graph includes data nodes A, B, C, D, E, and F. Data nodes A and B satisfy relation feature a, data nodes A and C satisfy relation feature b, data nodes A and D satisfy relation feature c, data nodes A and F satisfy relation feature b, data nodes B and C satisfy relation feature c, data nodes B and E satisfy relation feature a, and data nodes C and D satisfy relation feature b.

[0095] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S501-S504, after the server obtains the original data used to construct the knowledge graph, it can determine multiple entity data and multiple relation features of the original data. Then, the server can determine multiple data nodes of the knowledge graph based on the multiple entity data, and construct the knowledge graph based on the correspondence between the multiple data nodes and the multiple relation features. In this way, the server can construct a knowledge graph based on the original data, so that associated data can be added to each data node, which facilitates user searching and knowledge graph maintenance.

[0096] In one alternative embodiment, when the original data includes unstructured data, the method for the server to determine multiple entity data in the original data is as follows: Figure 5 Based on the illustrated method embodiments, this embodiment provides a possible implementation, such as... Figure 7 As shown in S502, the methods by which the server determines multiple entity data of the original data include:

[0097] S701. The server determines the first entity data based on the entity extraction algorithm and unstructured data.

[0098] Optionally, the entity extraction algorithm may include a long short-term memory (LSTM) conditional random field algorithm (CRF) based on a pre-trained bidirectional encoder representation from transformers (BERT) algorithm.

[0099] In one feasible way, combining Figure 2 The entity extraction module 221 can identify the data content of unstructured data using the BERT-LSTM-CRF algorithm and determine the first entity data from the data content.

[0100] S702. The server identifies high-frequency data in the unstructured data content that appears more frequently than the first frequency threshold as the second entity data.

[0101] Optionally, the first frequency threshold can be determined based on the frequency of occurrence of multiple data points in the data content.

[0102] For example, the server can determine the average frequency of occurrence of multiple data in the data content as the first frequency threshold.

[0103] In one feasible way, combining Figure 2 The entity extraction module 221 can use a partitioning tool to partition unstructured data and determine the frequency of occurrence of the partitioned data in the data content. When the frequency of occurrence is higher than a first frequency threshold, it indicates that the data corresponding to the frequency of occurrence is high-frequency data, and this data is identified as the second entity data.

[0104] S703. The server merges the first entity data and the second entity data according to the preset data processing strategy to obtain multiple entity data.

[0105] In one feasible approach, the server can merge the first entity data and the second entity data to obtain multiple entity data, which can then be processed to obtain data nodes.

[0106] Optionally, data processing strategies may include data aggregation and ambiguity resolution on historical data and internally extracted and externally crawled data, and obtaining information such as synonyms and related words.

[0107] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S701-S703, the server can determine the first entity data based on the entity extraction algorithm and unstructured data. The server can also determine high-frequency data (those appearing more frequently than a first frequency threshold) in the unstructured data content as the second entity data. Then, the server can fuse the first entity data and the second entity data according to a preset data processing strategy to obtain multiple entity data. In this way, the server can obtain entity data obtained by multiple methods, making the entity data more objective and reasonable.

[0108] In one alternative embodiment, when the original data includes unstructured data, the method for the server to determine multiple relational features of the original data is as follows: Figure 5 Based on the illustrated method embodiments, this embodiment provides a possible implementation, such as... Figure 8 As shown in S502, the methods by which the server determines multiple relational features of the original data include:

[0109] S801. The server determines the first entity relationship based on the relationship extraction algorithm and unstructured data.

[0110] Optionally, the relation extraction algorithm may include the BERT classification algorithm.

[0111] In one feasible way, combining Figure 2 The relation extraction module 222 can identify the predefined relations determined by the ontology management module 210 based on the BERT classification algorithm, and further determine the first entity relation.

[0112] S802. The server identifies entity relationships that appear more frequently than the second frequency threshold in the content of unstructured data as second entity relationships.

[0113] Optionally, the second frequency threshold can be determined based on the frequency of occurrence of multiple entity relationships in the data content.

[0114] For example, the server can determine the average frequency of occurrence of multiple entity relationships in the data content as the second frequency threshold.

[0115] In one feasible way, combining Figure 2 The relation extraction module 222 can extract entity relations based on preset relation extraction rules and determine the frequency of occurrence of the extracted entity relations in the data content. When the frequency of occurrence is higher than a second frequency threshold, the entity relation is identified as the second entity relation.

[0116] Optionally, the preset relationship extraction rules may include: extracting data using a preset data extraction template; or, determining the attribute relationships between multiple entity data as entity relationships based on the structural relationships of the data.

[0117] S803. The server merges the data of the first entity relationship and the second entity relationship according to the data processing strategy to obtain multiple relationship features.

[0118] In one feasible approach, the server can fuse data from the first entity relationship and the second entity relationship to obtain multiple relationship features.

[0119] Optionally, data processing strategies may include data aggregation and ambiguity resolution on historical data and internally extracted and externally crawled data, and obtaining information such as synonyms and related words.

[0120] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S801-S803, the server can determine the first entity relationship based on the relationship extraction algorithm and unstructured data. The server can also identify entity relationships in the unstructured data content that have a frequency higher than a second frequency threshold as the second entity relationship. Then, the server can fuse the first and second entity relationships according to a data processing strategy to obtain multiple relationship features. In this way, the server can obtain relationship features obtained by multiple methods, making the relationship features more objective and reasonable.

[0121] In one optional embodiment, the method by which the server determines multiple data nodes of the knowledge graph based on multiple entity data is as follows: Figure 5 Based on the illustrated method embodiments, this embodiment provides a possible implementation, such as... Figure 9 As shown in S503, the method by which the server determines multiple data nodes of the knowledge graph based on multiple entity data includes:

[0122] S901. When the relevance between the target entity data and at least one entity data other than the target entity data is greater than or equal to the relevance threshold, the server determines the target entity data as a data node of the knowledge graph.

[0123] The target entity data can be any one of multiple entity data.

[0124] Specifically, when the relevance between the target entity data and at least one entity data other than the target entity data is greater than or equal to the relevance threshold, it indicates that the target entity data belongs to the same profession as the original data. Therefore, the server can identify the target entity data as a data node of the knowledge graph.

[0125] Optionally, the relevance can be the average or sum of the relevance between the target entity data and each entity data in at least one entity data.

[0126] S902. When any entity data other than multiple entity data has a relevance greater than or equal to the relevance threshold with at least one of the multiple entity data, the server will identify any entity data as a data node of the knowledge graph.

[0127] Specifically, when any entity data other than multiple entity data has a relevance greater than or equal to a relevance threshold with at least one of the multiple entity data, it indicates that any entity data belongs to the same profession as the original data. Therefore, the server can identify any entity data as a data node of the knowledge graph.

[0128] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S901-S902, when the relevance between the target entity data and at least one entity data other than the target entity data among multiple entity data is greater than or equal to a relevance threshold, the server determines the target entity data as a data node of the knowledge graph. When any entity data other than the multiple entity data has a relevance greater than or equal to a relevance threshold with at least one entity data among multiple entity data, the server determines that entity data as a data node of the knowledge graph. In this way, the server can determine highly relevant entity data, as well as synonyms and related words, as data nodes of the knowledge graph, making the data nodes of the knowledge graph more comprehensive and more in line with professional requirements.

[0129] The process for determining associated data is described below. For example... Figure 10 As shown, the data processing method includes:

[0130] S1001, The server obtains the data to be associated from the target data node.

[0131] The target data node is any data node in the pre-built knowledge graph.

[0132] Optionally, the data to be associated may include structured data, such as relational databases, unstructured data, such as audio and video, and semi-structured data such as XML, JSON, and encyclopedias.

[0133] Optionally, the data format of the data to be associated can include editable data formats, such as Word format, TXT format, etc., and can also include non-editable data formats, such as PDF format, etc.

[0134] In one feasible approach, after the server obtains the data to be associated in a non-editable data format, it can convert the data to be associated into an editable data format.

[0135] In one possible approach, the server can obtain the data to be associated by either importing it from the system or receiving the data sent by the client.

[0136] S1002. The server determines the relevance between the data to be associated and at least one data node in the knowledge graph.

[0137] At least one data node includes the target data node.

[0138] Optionally, relevance can include metrics such as the in-degree, out-degree, average distance to the core nodes of the knowledge graph, and degree betweenness of the data content to be associated in the knowledge graph.

[0139] Optionally, the core nodes of a knowledge graph may include the data node with the largest number of corresponding relational features.

[0140] S1003. When the relevance is greater than the association threshold, the server determines the data to be associated as the associated data of the target data node.

[0141] Specifically, when the relevance is greater than the association threshold, it indicates that the data to be associated belongs to the profession of the knowledge graph. Therefore, the server can determine that the data to be associated is the associated data of the target data node.

[0142] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S1001-S1003, after the server obtains the data to be associated with the target data node, it can determine the relevance between the data to be associated and at least one data node in the knowledge graph. When the relevance is greater than the association threshold, the server can determine that the data to be associated is associated data of the target data node. Since the relevance between the associated data and at least one data node is greater than the association threshold, it indicates that the associated data belongs to the profession of the knowledge graph. Compared with general technologies, this solves the problem that the associated data is not related to or has a low relevance to the profession of the knowledge graph, thus improving the effectiveness and accuracy of finding associated data in the knowledge graph.

[0143] In one optional embodiment, after the server determines that the data to be associated is the associated data of the target data node, in Figure 10Based on the illustrated method embodiments, this embodiment provides a possible implementation, such as... Figure 11 As shown, after S1003, the data processing method further includes:

[0144] S1101, The server receives the request message from the client.

[0145] The request message is used to request the associated data of the target data node.

[0146] In one possible implementation, the client can send a request message to the server requesting access to the knowledge graph. In response to this request message, the server sends the knowledge graph to the client, causing the client's display page to show the knowledge graph. Then, in response to performing a trigger operation on a target data node in the knowledge graph on the display page, the client can send a request message to the server requesting the retrieved associated data of the target data node.

[0147] S1102. The server responds to the request message by sending the associated data of the target data node to the client.

[0148] In one possible implementation, the server responds to a request message sent by the client for retrieving the associated data of the target data node by sending the associated data of the target data node to the client, so that the client's display page displays the associated data.

[0149] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S1101-S1102, the server receives the request message from the client. In response to the request message, the server sends the associated data of the target data node to the client. In this way, the client can obtain the associated data of the data node in the knowledge graph through the server.

[0150] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0151] This application embodiment can divide the data processing device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0152] like Figure 12 The diagram shown is a structural schematic of a data processing apparatus provided in an embodiment of this application. This data processing apparatus can be used to perform... Figures 5-11 The data processing method shown. Figure 12 The data processing device shown includes: an acquisition unit 1201 and a processing unit 1202;

[0153] The acquisition unit 1201 is used to acquire the data to be associated with the target data node; the target data node is any data node in the pre-constructed knowledge graph.

[0154] Processing unit 1202 is used to determine the relevance between the data to be associated and at least one data node in the knowledge graph; the at least one data node includes the target data node;

[0155] The processing unit 1202 is also used to determine the data to be associated as the associated data of the target data node when the relevance is greater than the association threshold.

[0156] Optionally, the acquisition unit 1201 is also used to acquire the raw data used to construct the knowledge graph;

[0157] The processing unit 1202 is also used to determine multiple entity data and multiple relational features of the original data; the entity data is used to represent the data content of the original data; the relational features are used to represent the logical relationship between the multiple entity data.

[0158] The processing unit 1202 is also used to determine multiple data nodes of the knowledge graph based on multiple entity data;

[0159] The processing unit 1202 is also used to construct a knowledge graph based on the correspondence between multiple data nodes and multiple relational features.

[0160] Optionally, the raw data includes structured data or unstructured data; when the raw data includes unstructured data, the processing unit 1202 is specifically used for:

[0161] Based on the entity extraction algorithm and unstructured data, determine the first entity data;

[0162] High-frequency data in the unstructured data content that appears more frequently than the first frequency threshold is identified as the second entity data.

[0163] According to the preset data processing strategy, the first entity data and the second entity data are fused to obtain multiple entity data.

[0164] Optionally, when the original data includes unstructured data, the processing unit 1202 is specifically used for:

[0165] Based on the relation extraction algorithm and unstructured data, determine the first entity relation;

[0166] Entity relationships that appear more frequently than a second frequency threshold in the content of unstructured data are identified as second entity relationships.

[0167] Based on the data processing strategy, the first entity relationship and the second entity relationship are fused to obtain multiple relationship features.

[0168] Optionally, the processing unit 1202 is specifically used for:

[0169] When the relevance between the target entity data and at least one entity data other than the target entity data among multiple entity data is greater than or equal to the relevance threshold, the target entity data is determined as a data node of the knowledge graph; the target entity data can be any one of the multiple entity data.

[0170] Optionally, the data processing device further includes: a receiving unit 1203 and a transmitting unit 1204;

[0171] The receiving unit 1203 is used to receive request messages from clients; the request messages are used to request the acquisition of associated data of the target data node.

[0172] The sending unit 1204 is used to send the associated data of the target data node to the client in response to the request message.

[0173] This application also provides a computer-readable storage medium, which includes computer-executable instructions. When the computer-executable instructions are run on a computer, the computer performs the data processing method provided in the above embodiments.

[0174] This application also provides a computer program that can be directly loaded into a memory and contains software code. After being loaded and executed by a computer, the computer program can implement the data processing method provided in the above embodiments.

[0175] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0177] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and other division methods may exist in actual implementation. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate; components shown as units may be one physical unit or multiple physical units, i.e., they may be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the general technology, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0179] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A data processing method, characterized by, The method comprises: acquiring target data to be associated with a target data node; the target data node is any data node in a pre-constructed knowledge graph; determining a correlation degree of the target data to at least one data node in the knowledge graph; the at least one data node comprises the target data node; when the correlation degree is greater than an association threshold, determining that the target data is association data of the target data node; the correlation degree of the target data to at least one data node in the knowledge graph is used to represent a probability that the target data belongs to a professional field to which the knowledge graph belongs; the method further comprises: acquiring original data used to construct the knowledge graph; determining a plurality of entity data and a plurality of relationship features of the original data; the entity data is used to represent data content of the original data; the relationship features are used to represent logical relationships between the plurality of entity data; when a target entity data has a correlation degree greater than or equal to a correlation degree threshold to at least one entity data in the plurality of entity data except the target entity data, the target entity data is determined as a data node of the knowledge graph; the target entity data is any one of the plurality of entity data; and / or, when any entity data except the plurality of entity data has a correlation degree greater than or equal to a correlation degree threshold to at least one entity data in the plurality of entity data, the any entity data is determined as a data node of the knowledge graph; constructing the knowledge graph according to a correspondence between the plurality of data nodes and the plurality of relationship features.

2. The data processing method according to claim 1, characterized in that, The original data comprises structured data or unstructured data; when the original data comprises the unstructured data, the determining of the plurality of entity data of the original data comprises: determining first entity data according to an entity extraction algorithm and the unstructured data; determining high-frequency data with a frequency higher than a first frequency threshold in data content of the unstructured data as second entity data; performing data fusion on the first entity data and the second entity data according to a preset data processing strategy to obtain the plurality of entity data.

3. The data processing method according to claim 2, characterized in that, When the original data comprises the unstructured data, the determining of the plurality of relationship features of the original data comprises: determining first entity relationship according to a relationship extraction algorithm and the unstructured data; determining entity relationship with a frequency higher than a second frequency threshold in data content of the unstructured data as second entity relationship; performing data fusion on the first entity relationship and the second entity relationship according to the data processing strategy to obtain the plurality of relationship features.

4. The data processing method according to any one of claims 1 to 3, characterized in that, Further comprising: receiving a request message of a client; the request message is used to request to acquire association data of the target data node; in response to the request message, sending the association data of the target data node to the client.

5. A data processing apparatus, characterized by, The method comprises: an acquisition unit and a processing unit; the acquisition unit is configured to acquire target data to be associated with a target data node; the target data node is any data node in a pre-constructed knowledge graph; The processing unit is configured to determine a relevance of the data to be associated with at least one data node in the knowledge graph; the at least one data node includes the target data node; The processing unit is further configured to determine that the data to be associated is the associated data of the target data node when the relevance is greater than an association threshold; the relevance of the data to be associated with at least one data node in the knowledge graph is used to represent a probability that the data to be associated belongs to a professional field to which the knowledge graph belongs. The acquisition unit is further configured to acquire original data used to construct the knowledge graph. The processing unit is further configured to determine a plurality of entity data and a plurality of relationship features of the original data; the entity data is used to represent data content of the original data; and the relationship feature is used to represent a logical relationship between the plurality of entity data. The processing unit determines that a target entity data is a data node of the knowledge graph when a relevance of the target entity data to at least one entity data in the plurality of entity data except the target entity data is greater than or equal to a relevance threshold; the target entity data is any one of the plurality of entity data, and / or when a relevance of any entity data except the plurality of entity data to at least one entity data in the plurality of entity data is greater than or equal to a relevance threshold, the any entity data is determined to be a data node of the knowledge graph. The processing unit is further configured to construct the knowledge graph according to a correspondence between the plurality of data nodes and the plurality of relationship features.

6. The data processing apparatus according to claim 5, characterized in that, The original data includes structured data or unstructured data; when the original data includes the unstructured data, the processing unit is specifically configured to: determine first entity data according to an entity extraction algorithm and the unstructured data; determine high-frequency data with a frequency higher than a first frequency threshold in data content of the unstructured data as second entity data; perform data fusion on the first entity data and the second entity data according to a preset data processing strategy to obtain the plurality of entity data.

7. The data processing apparatus according to claim 6, characterized in that, When the original data includes the unstructured data, the processing unit is specifically configured to: determine a first entity relationship according to a relationship extraction algorithm and the unstructured data; determine an entity relationship with a frequency higher than a second frequency threshold in data content of the unstructured data as a second entity relationship; perform data fusion on the first entity relationship and the second entity relationship according to the data processing strategy to obtain the plurality of relationship features.

8. The data processing apparatus according to any one of claims 5-7, characterized in that, Further comprising: a receiving unit and a sending unit; The receiving unit is configured to receive a request message of a client; The request message is used to request to acquire associated data of the target data node; The sending unit is configured to send the associated data of the target data node to the client in response to the request message.

9. A data processing apparatus, characterized by, The data processing device comprises a memory and a processor; the memory is used for storing computer execution instructions, and the processor is connected with the memory through a bus; when the data processing device is running, the processor executes the computer execution instructions stored in the memory, so that the data processing device executes the data processing method according to any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises computer execution instructions, when the computer execution instructions are running on the computer, so that the computer executes the data processing method according to any one of claims 1-4.

Images