An automatic data cleaning method and system based on knowledge graph

Abstract

The present invention relates to the field of electronic digital data processing, and more specifically, to an automatic data cleaning method and system based on a knowledge graph. The present invention constructs a knowledge graph corresponding to each data source based on a data table corresponding to each data source; classifies the entities in all knowledge graphs to obtain multiple groups of similar entities; for any group of similar entities, classifies them according to the names of each entity and classifies the entities with the same name into one category, and determines the name of any category of entities with the largest number as the name of the target entity of the group of similar entities; for any group of similar entities, determines the related entities of the group of similar entities. For each group of similar entities, a target data table is constructed and output based on the name text of its target entity and the name text of the target entity of similar entities corresponding to all its group-related entities. The present invention realizes the integration of the knowledge graph composed of multi-source data to realize the cleaning of multi-source data.

Description

Technical Field

[0001] The present invention relates to the field of electronic digital data processing, and more specifically, to an automatic data cleaning method and system based on a knowledge graph. Background Art

[0002] With the development of information technology, the generation of data is growing exponentially. In modern information systems, data comes from a variety of different sources, such as multiple databases, file systems, API interfaces, etc., and these data are often in various formats.

[0003] The existing method of data cleaning is usually that the data cleaning personnel use SQL to write codes, verify the data one by one through manual inspection, or filter and verify according to preset rules. The Chinese patent application document with publication number CN116166650A discloses a multi-source heterogeneous data cleaning method based on a generative adversarial network, which synthesizes each multi-source heterogeneous data into a multi-source heterogeneous data fusion table, and analyzes the redundant data, abnormal data and missing data remaining in the multi-source heterogeneous data fusion table through a clustering analysis algorithm, and then determines the multi-source heterogeneous data with missing data, so that the multi-source heterogeneous data with missing data can be accurately and effectively divided, thereby improving the accuracy of multi-source heterogeneous data cleaning. However, multi-source data may have inconsistent formats, irregular naming or data quality differences, resulting in a large amount of time and manpower required for data cleaning and integration in applications such as data analysis and machine learning, which not only reduces the efficiency of obtaining data, but also increases the possibility of errors in the obtained data.

[0004] Furthermore, the data formats and standards of different data sources are inconsistent. When merging and deduplicating data from multiple data sources, manual review and intervention are usually required to ensure that duplicate records in the data are correctly identified and merged.

[0005] Based on this, how to reduce the human resources consumed in the integration and cleaning of multi-source data, and thus improve the efficiency of cleaning multivariate data, is an important research direction in the field of data processing. Summary of the invention

[0006] The present invention provides an automatic data cleaning method and system based on knowledge graph, aiming to solve the problem in the related art that the efficiency of obtaining data is reduced and the possibility of error in the obtained data is increased due to inconsistent formats, irregular naming or data quality differences in multi-source data.

[0007] In a first aspect, the present invention provides an automatic data cleaning method based on a knowledge graph, the method comprising: constructing a knowledge graph corresponding to each data source according to a data table corresponding to each data source; classifying entities in all knowledge graphs to obtain multiple groups of similar entities, wherein for a first associated entity of any entity in any group of similar entities, and a second associated entity of another entity, the first associated entity and the second associated entity satisfy: n >1 / 2max( N 1, N 2), N 1 is the number of the first associated entity, N 2 is the number of the second associated entity, n is the number of pairs of entities with the same name, where any entity pair includes one entity in the first associated entity and one entity in the second associated entity; for any group of similar entities, entities with the same name are classified into one category and the name of any category with the largest number of entities is determined as the name of the target entity of the group of similar entities; for any group of similar entities, in response to the group of similar entities satisfying any other group of similar entities m >min( N 3, N 4) Determine that this group of similar entities and any other group of similar entities are related entities, where m is the number of entity pairs in the group of similar entities and any group of related entities that belong to the same knowledge graph and are related to each other. N 3 is the number of entities in this group of similar entities, N 4 is the number of entities in any group of related entities; for each group of similar entities, a target data table is constructed and output according to the name text of its target entity and the name text of the target entity of similar entities corresponding to all its group-related entities.

[0008] The beneficial effects of the present invention are:

[0009] The present invention converts the data of each data source into a knowledge graph, and classifies each entity according to the associated entities of each entity in each knowledge graph, and obtains multiple groups of similar entities with similar associated entities, so that entities with less outliers and greater correlations are selected as one category in the classification process to filter out the data corresponding to the data source with missing or abnormal data. The present invention constructs triples based on the correlation between each group of similar entities, and obtains a data table through the triples to obtain cleaned data. Based on this, the present invention realizes the automatic integration and cleaning of multi-source data, reduces the human resources consumed in the integration and cleaning of multi-source data, and thus improves the cleaning efficiency of multivariate data.

[0010] Preferably, the classification of entities in all knowledge graphs to obtain multiple groups of similar entities includes: obtaining all associated entities of each entity in each knowledge graph; selecting any one entity, obtaining a group of similar entities including the entity, and marking all entities in the group of similar entities as selected entities; selecting any entity other than the selected entity, obtaining a group of similar entities including the entity, and marking all entities in the group of similar entities as selected entities, and iterating the above steps to obtain multiple groups of similar entities.

[0011] The beneficial effect is that the present invention marks an entity that has been classified into a group of similar entities as a selected entity, thereby preventing one entity from being classified into multiple groups of entities.

[0012] Preferably, for each group of similar entities, constructing a data table based on the name text of its target entity and the name text of the target entity of similar entities corresponding to all its group-related entities includes: constructing a target triple based on each group of similar entities, based on the name text of its target entity and the name text of the target entity of similar entities corresponding to all its group-related entities; and constructing a target data table based on the target triple.

[0013] The beneficial effect is that by constructing target triples and data tables, similar entities and their relationships can be structured, thereby realizing the storage and management of the information contained in the similar entities.

[0014] Preferably, for each group of similar entities, constructing a target triple based on the name text of its target entity and the name text of the target entity of similar entities corresponding to all its related entities includes: wherein, the name text of the target entity of the group of similar entities is taken as the first text, and the name text of the target entity corresponding to any group of related entities corresponding to the group of similar entities is taken as the second text; obtaining triples in all knowledge graphs with the first text as the text of one entity and the second text as the text of another entity, recorded as preliminary triples; classifying preliminary triples with the same relationship into one category; obtaining the basic unit of any category of preliminary triples with the largest number, and determining the basic unit as the basic unit of the target triple corresponding to the group of related entities of the group of similar entities; obtaining the target triples corresponding to each group of related entities of each group of similar entities.

[0015] The beneficial effect is that the relationship between the name texts of each group of similar entities and each group of related entities of each group of similar entities is obtained through all triples of all knowledge graphs, thus avoiding the time and cost consumed in manually obtaining the relationship between entities.

[0016] Preferably, for each group of similar entities, constructing a triple based on the name text of its target entity and the name text of the target entity of similar entities corresponding to all its related entities includes: wherein, the name text of the target entity of the group of similar entities is taken as the first text, and the name text of the target entity corresponding to any group of related entities corresponding to the group of similar entities is taken as the second text; obtaining the predicted relationship between the first text and the second text through a large language model; for the target triple corresponding to the group of related entities of the group of similar entities, taking the predicted relationship as the relationship of the target triple, taking the first text as the name of one of the entities of the target triple, and taking the second text as the name of another entity of the target triple; obtaining the target triple corresponding to each group of related entities of each group of similar entities.

[0017] The beneficial effect is that the relationship between the name texts of each group of similar entities and each group of related entities of each group of similar entities is obtained through a large language model, thereby avoiding the time and cost consumed in manually obtaining the relationship between entities.

[0018] Preferably, constructing the knowledge graph corresponding to each data source based on the data table corresponding to each data source includes: in response to the data in the data source being unstructured data or semi-structured data, extracting knowledge from the data in the data source to obtain structured data, and performing knowledge processing on the structured data to obtain a data table; in response to the data in the data source being structured data, performing knowledge processing on the data in the data source to obtain a data table; constructing triples corresponding to each data source based on the data table corresponding to each data source; and constructing the knowledge graph corresponding to each data source based on the triples corresponding to each data source.

[0019] Preferably, the data source includes a predetermined number of databases and a predetermined number of data files.

[0020] In a second aspect, the present invention provides an automatic data cleaning system based on a knowledge graph, comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement an automatic data cleaning method based on a knowledge graph as described in any one of the above invention contents.

[0021] The beneficial effects of the present invention are:

[0022] The present invention integrates multi-source data in a unified format by converting it into a knowledge graph. The present invention also automatically classifies entities according to their relevance, significantly improving the efficiency of data cleaning and integration and reducing the need for manual intervention. This method achieves effective management and information storage of similar entities through systematic triple construction and relationship modeling, thereby reducing the time and cost of processing data relationships and achieving automatic cleaning of multi-source heterogeneous data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0024] Figure 1 is a flowchart schematically illustrating an automatic data cleaning method based on a knowledge graph according to an embodiment of the present invention;

[0025] Figure 2 is a flowchart schematically illustrating step S5 according to an embodiment of the present invention;

[0026] Figure 3 is a flowchart schematically illustrating step S501 according to an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of a predetermined number of entities and their associated entities according to one embodiment of the present invention;

[0028] Figure 5 is a schematic diagram of a predetermined number of entities and their associated entities according to another embodiment of the present invention;

[0029] Figure 6 It is a block diagram schematically showing the structure of the automatic data cleaning system based on the knowledge graph in this embodiment. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0031] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0032] Figure 1 It is a flowchart schematically showing the steps of an automatic data cleaning method based on a knowledge graph according to an embodiment of the present invention.

[0033] like Figure 1 As shown, an automatic data cleaning method based on knowledge graph includes steps S1 to S5.

[0034] Step S1: Construct the knowledge graph corresponding to each data source based on the data table corresponding to each data source.

[0035] In one embodiment, in response to the data in the data source being unstructured data or semi-structured data, knowledge is extracted from the data in the data source to obtain structured data, and the structured data is knowledge processed to obtain a data table; in response to the data in the data source being structured data, knowledge is processed from the data source to obtain a data table; a triple corresponding to each data source is constructed based on the data table corresponding to each data source; and a knowledge graph corresponding to each data source is constructed based on the triple corresponding to each data source.

[0036] Among them, knowledge extraction refers to the automatic extraction of meaningful structured information from unstructured or semi-structured data. Knowledge extraction includes entity extraction, relationship extraction and attribute extraction. Entity extraction, also known as named entity recognition (NER), is to identify specific named entities from text. Relation extraction uses technologies such as linguistics, statistics and information science to discover semantic relationships between entities from text. Attribute extraction focuses on extracting specific attributes or features of entities from text.

[0037] In one embodiment, the data source includes a predetermined number of databases and a predetermined number of data files. A knowledge graph can be constructed based on a data source, so a data source corresponds to a knowledge graph.

[0038] Step S2: Classify all entities in the knowledge graph to obtain multiple groups of similar entities.

[0039] Wherein, for a first associated entity of any entity in any group of similar entities, and a second associated entity of another entity, the first associated entity and the second associated entity satisfy: n >1 / 2max( N 1, N 2), N 1 is the number of the first associated entity, N 2 is the number of the second associated entity, n is the number of entity pairs with the same name, where any entity pair includes one entity in the first associated entity and one entity in the second associated entity.

[0040] It should be noted that an associated entity refers to an entity with a relationship between two or more entities, and an entity and any associated entity of the entity belong to the same triple. That is, any entity in any group of similar entities mentioned above and the first associated entity belong to a triple in a knowledge graph; the other entity and the second associated entity belong to a triple in a knowledge graph.

[0041] Since for two entities, the same name does not mean that they represent the same meaning in the data source, but the entity itself has multiple attributes, relationships and characteristics, and the attributes and characteristics of the entity can be reflected through all its associated entities. Based on this, the present invention classifies entities according to all their associated entities, groups entities with similar associated entities, and then obtains multiple groups of similar entities.

[0042] Furthermore, for any two entities in a group of similar entities, the number of entity pairs with the same name of the two entities, which includes an associated entity of one of the entities and an associated entity of the other entity, is greater than one-half of the number of any one of the two entities (also the number of the larger entity of the two entities). Based on this, most of the associated entities of the entities in a group of similar entities are the same, that is, the characteristics and attributes of the entities in a group of similar entities are similar.

[0043] In one embodiment, Figure 4 As shown, the first entity Q1 has 4 associated entities, whose names are A2, A3, A4, and A5; the second entity Q2 has 4 associated entities, whose names are A2, A3, A4, and A6; the third entity Q3 has 4 associated entities, whose names are A2, A3, A4, and A7; the fourth entity Q4 has 4 associated entities, whose names are A2, A3, A4, and A5; the fifth entity Q5 has 1 associated entity, whose name is A2; the sixth entity Q6 has 4 associated entities, whose names are A2, A6, A8, and A9.

[0044] The first entity Q1, the second entity Q2, the third entity Q3, and the fourth entity Q4 belong to the same group of similar entities, and the associated entities of the four entities all include the entity named A2, the entity named A3, and the entity named A4. The associated entities of the four entities all include the above three entities, and the number of the above three entities is greater than half of the number of associated entities of any one of the four entities.

[0045] Step S3: For any group of similar entities, classify them according to their names and group the entities with the same name into one category, and determine the name of any category with the largest number of entities as the name of the target entity in the group of similar entities.

[0046] In one embodiment, Figure 5 As shown, the seventh entity Q7 , the eighth entity Q8 , the ninth entity Q9 , the tenth entity Q10 , the eleventh entity Q11 , and the twelfth entity Q12 are entities in a group of similar entities.

[0047] The names of the five entities including the seventh entity Q7, the eighth entity Q8, the ninth entity Q9, the tenth entity Q10 and the twelfth entity Q12 are all B1, and the name of the eleventh entity Q11 is B2, so the name of the target entity of the group of similar entities is B2.

[0048] In another embodiment, if the names of three entities in a group of similar entities are B1 and the names of the remaining three entities are B2, B1 or B2 is used as the name of the target entity of the group of similar entities. At this time, relevant personnel can manually select one of B1 and B2 as the name of the target entity of the group of similar entities.

[0049] Step S4: For any group of similar entities, in response to the group of similar entities satisfying any other group of similar entities m >min( N 3, N 4) Determine whether the group of similar entities and any other group of similar entities are related entities.

[0050] in m is the number of entity pairs in the group of similar entities and any group of related entities that belong to the same knowledge graph and are related to each other. N 3 is the number of entities in this group of similar entities, N 4 is the number of entities in any group of related entities. That is, when the group of similar entities and another group of similar entities meet the following conditions: the number of entity pairs that belong to the same knowledge graph and are related entities is greater than half of the number of entities in the group of similar entities or another group of similar entities, with one entity in the group of similar entities and an entity in the other group of similar entities as an entity pair.

[0051] Based on this, for any group of similar entities, any group of similar entities that are mutually related entities is a group of related entities of the similar entity, which has a predetermined number of similar entities of other groups that are mutually related entities, so the group of similar entities has a predetermined number of related entities.

[0052] Furthermore, for any group of similar entities, the name texts of some entities in any group of related entities are semantically related to the name texts of some entities in the group of similar entities (can form triples, i.e., the two entities are related), so that the texts of the two groups of entities are related.

[0053] Step S5: For each group of similar entities, a target data table is constructed and outputted according to the name text of its target entity and the name text of the target entity of similar entities corresponding to all group-related entities.

[0054] Figure 2 is a step flow chart schematically illustrating step S5 according to an embodiment of the present invention.

[0055] like Figure 2 As shown, step S5 includes step S501 to step S502.

[0056] Step S501: construct a target triple according to each group of similar entities, the name text of its target entity and the name text of the target entity of similar entities corresponding to all groups of related entities.

[0057] In one embodiment, a predicted relationship between a first text and a second text is obtained through a large language model, wherein the first text is the name text of a target entity of the group of similar entities, and the second text is the name text of a target entity corresponding to any group of related entities corresponding to the group of similar entities; for a target triple corresponding to the group of related entities of the group of similar entities, the predicted relationship is the relationship of the target triple, the first text is the name of one of the entities of the target triple, and the second text is the name of another entity of the target triple; and the target triple corresponding to each group of related entities of each group of similar entities is obtained.

[0058] Step S502: construct a target data table according to the target triples.

[0059] In one embodiment, a target data table is constructed by N target triples, wherein for any triple, the subject-object relationship of the two entities included in it is obtained through the semantic recognition model. For the i-th target triple, the name text of the entity belonging to the subject is C i , the name text of the entity belonging to the predicate is D i , the text of the relation (i.e. predicate) is E i , change the name text C i Fill in the first column of the i+1th row of the target data table and replace the text E of the relationship i Fill in the second column of the i+1th row of the target data table and replace the name text D i Fill in the third column of the i+1th row of the target data table. The target data table of this embodiment is shown in Table 1.

[0060] Table 1

[0061]

[0062] Figure 3 is a flowchart schematically illustrating step S501 according to an embodiment of the present invention.

[0063] like Figure 3 As shown, step S501 includes steps S5011 to S5014.

[0064] Step S5011: Obtain triples in all knowledge graphs that have the first text as one entity and the second text as another entity, and record them as preliminary triples.

[0065] The first text is the name text of the target entity of the group of similar entities, and the second text is the name text of the target entity corresponding to any group of related entities corresponding to the group of similar entities. All triples of the knowledge graph are traversed, and the triples with the name text of one entity in the first text and the name text of another entity in the second entity are selected to obtain a set of preliminary triples of the group of similar entities and the group of related entities of the similar entities.

[0066] Based on this, any group of similar entities and any group of related entities of the similar entities correspond to a group of preliminary triples.

[0067] Step S5012: Classify according to the relationship between the prepared triples.

[0068] Among them, the relationship between various types of preliminary triples is the same.

[0069] Step S5013: obtaining the basic unit of any type of preliminary triples with the largest number, and determining the basic unit as the basic unit of the target triples corresponding to the group of related entities of the group of similar entities.

[0070] The basic unit of a triple is "entity-relationship-entity", that is, two entities and a relationship. The name text of one of the entities in the triple is the subject, the name text of the other entity in the triple is the object, and the relationship of the triple is the predicate. For example, Mr. H was born in country h, where "Mr. H" is the name text of one of the entities in the triple, "born in" is the predicate of the triple, and "country h" is the name text of the other entity in the triple; Company C is located in City C, "Company C" is the name text of one of the entities in the triple, "located in" is the predicate of the triple, and "City C" is the name text of the other entity in the triple.

[0071] Step S5014: Obtain target triples corresponding to each group of related entities of each group of similar entities.

[0072] Figure 6 It is a block diagram schematically showing the structure of the automatic data cleaning system based on the knowledge graph in this embodiment.

[0073] The present invention also provides an automatic data cleaning system based on knowledge graph. Figure 6 As shown, the system includes a processor and a memory, the memory stores computer program instructions, and when the computer program instructions are executed by the processor, an automatic data cleaning method based on a knowledge graph according to the first aspect of the present invention is implemented.

[0074] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device. Any application or module described in the present invention may be implemented using computer-readable / executable instructions that may be stored or otherwise maintained by such a computer-readable medium.

[0075] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0076] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0077] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent application. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. An automatic data cleaning method based on knowledge graph, characterized in that: include: Build the knowledge graph corresponding to each data source based on the data table corresponding to each data source; Classify all entities in the knowledge graph to obtain multiple groups of similar entities, where for the first associated entity of any entity in any group of similar entities, and the second associated entity of another entity, the first associated entity and the second associated entity satisfy: n >1 / 2max( N 1, N 2), N 1 is the number of the first associated entity, N 2 is the number of the second associated entity, n is the number of entity pairs with the same name, where any entity pair includes one entity in the first associated entity and one entity in the second associated entity; For any group of similar entities, entities with the same name are grouped together and the name of any class of entities with the largest number is determined as the name of the target entity of the group of similar entities; for any group of similar entities, in response to the group of similar entities satisfying any other group of similar entities m >min( N 3, N 4) Determine that this group of similar entities and any other group of similar entities are related entities, where m is the number of entity pairs in the group of similar entities and any group of related entities that belong to the same knowledge graph and are related to each other. N 3 is the number of entities in this group of similar entities, N 4 is the number of entities in any set of related entities; For each group of similar entities, construct and output a target data table according to the name text of the target entity and the name text of the target entity of similar entities corresponding to all group-related entities, including: constructing a target triple according to the name text of the target entity and the name text of the target entity of similar entities corresponding to all group-related entities according to each group of similar entities; Construct a target data table according to the target triples; Among them, for each group of similar entities, constructing a triple according to the name text of its target entity and the name text of the target entity of the similar entity corresponding to all its related entities includes: The name text of the target entity of the group of similar entities is the first text, and the name text of the target entity corresponding to any group of related entities corresponding to the group of similar entities is the second text; Obtaining a predicted relationship between a first text and a second text through a large language model; For a target triplet corresponding to the group of related entities of the group of similar entities, the predicted relationship is the relationship of the target triplet, the first text is the name of one of the entities of the target triplet, and the second text is the name of another entity of the target triplet; Obtain target triples corresponding to each group of related entities of each group of similar entities.

2. The automatic data cleaning method based on knowledge graph according to claim 1 is characterized in that: The classification of entities in all knowledge graphs to obtain multiple groups of similar entities includes: Get all related entities of each entity in each knowledge graph; Select an entity, obtain a group of similar entities including the entity, and mark all entities in the group of similar entities as selected entities; Select any entity other than the selected entity, obtain a group of similar entities including the entity, and mark all entities in the group of similar entities as selected entities, and iterate the above steps to obtain multiple groups of similar entities.

3. The automatic data cleaning method based on knowledge graph according to claim 1 is characterized in that: For each group of similar entities, a target triple is constructed based on the name text of the target entity and the name text of the target entity of the similar entities corresponding to all its related entities. include: The name text of the target entity of the group of similar entities is the first text, and the name text of the target entity corresponding to any group of related entities corresponding to the group of similar entities is the second text; Obtaining all triples of texts in the knowledge graph that have the first text as one entity and the second text as another entity, which are recorded as preliminary triples; and classifying preliminary triples with the same relationship into one category; The basic unit of any type of preliminary triples with the largest number is obtained, and the basic unit is determined as the basic unit of the target triples corresponding to the group of related entities of the group of similar entities; the target triples corresponding to each group of related entities of each group of similar entities are obtained.

4. The automatic data cleaning method based on knowledge graph according to claim 1 is characterized in that: The step of constructing the knowledge graph corresponding to each data source according to the data table corresponding to each data source includes: In response to the data in the data source being unstructured data or semi-structured data, performing knowledge extraction on the data in the data source to obtain structured data, and performing knowledge processing on the structured data to obtain a data table; In response to the data in the data source being structured data, performing knowledge processing on the data in the data source to obtain a data table; Construct the triples corresponding to each data source according to the data tables corresponding to each data source; Construct the knowledge graph corresponding to each data source based on the triples corresponding to each data source.

5. The automatic data cleaning method based on knowledge graph according to claim 1 is characterized in that: The data sources include a predetermined number of databases and a predetermined number of data files.

6. An automatic data cleaning system based on knowledge graph, comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: The processor executes the computer program to implement the automatic data cleaning method based on the knowledge graph as described in any one of claims 1-5.

Images