Data organization and storage method, system and storage medium based on knowledge graph

By configuring metadata and building knowledge graph data storage layers and links, the problem of inefficient data organization storage in the existing technology is solved, efficient data storage and adaptive domain characteristics adaptation are achieved, and the advantage utilization of knowledge graphs is improved .

CN113849591BActive Publication Date: 2024-11-01BEIJING WANGPIN CONSULTING CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111111743.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-23
Publication Date
2024-11-01
Estimated Expiration
2041-09-23

AI Technical Summary

Technical Problem

The existing knowledge graph-based data organization storage method has failed to fully utilize its advantages and failed to adapt to data characteristics in different fields, resulting in inefficiency.

Method used

By configuring metadata, we can gradually build a data storage layer based on knowledge graphs, including the basic SPO layer, entity data normalization layer and wide table service application layer, and automatically build a knowledge graph data link to support various data services.

Benefits of technology

The efficiency of data organization and storage is improved, the advantages of knowledge graphs are fully utilized, and adaptively adapted to the characteristics of the data field through self-learning of the model, further improving efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113849591B_ABST
    Figure CN113849591B_ABST
Patent Text Reader

Abstract

A method, device and computer-readable medium for organizing and storing data based on a knowledge graph. The method includes: S1, configuring metadata; S2, progressively constructing a data storage layer based on a knowledge graph, wherein the data storage layer includes at least one or more basic SPO layers, one or more entity data aggregation layers, and one or more wide table service application layers; S3, automatically constructing a link for knowledge graph data, wherein the link is used to support various data services. The method can optimize the data storage structure, improve the efficiency of data retrieval and query, and achieve a two-way improvement in data recall and matching.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of data organization and storage, and in particular to a data organization and storage method, system and storage medium based on a knowledge graph. Background Art

[0002] In the current Internet, massive amounts of information are distributed in different information sources, and the relevance is relatively sparse. It is becoming increasingly difficult to use traditional search engines to accurately and quickly obtain valuable information. Traditional manual knowledge management methods cannot meet users' needs for knowledge acquisition in real time and efficiently. This is especially true in the application field of accurately and efficiently identifying user needs and accurately matching search query results, such as accurate recruitment needs based on big data. Among them, information organization and storage based on knowledge graphs, through the introduction of natural language processing technology, can conveniently and efficiently search for relevant results from knowledge graphs and push them to users to assist users in learning relevant knowledge.

[0003] Knowledge graph is a modern technology that combines the theories and methods of applied mathematics, graphics, information visualization technology, information science and other disciplines with methods such as metrology citation analysis and co-occurrence analysis, and uses visual graphs to vividly display the core structure, development history, frontier fields and overall knowledge architecture of the discipline to achieve the purpose of multidisciplinary integration. Knowledge graph provides a more effective way to express, organize, manage and utilize massive, heterogeneous and dynamic big data on the Internet, making the network more intelligent and closer to human cognitive thinking. For example, after the user enters the query, the search engine not only looks for keywords, but first understands the semantics. After the query is segmented, the description of the query is normalized so that it can be matched with the knowledge base. The query return result is the complete knowledge system given by the search engine after retrieving the corresponding entity in the knowledge base.

[0004] In the existing technology, data organization and storage based on knowledge graphs simply stores data in relational databases and organizes them into knowledge graphs, but fails to effectively organize and store data and fully utilize the advantages of knowledge graphs. At the same time, data in different fields have their own field characteristics. When using knowledge graphs to describe and organize data, they also have field characteristics. The organization and storage of data in the existing technology also fails to adapt to the field characteristics. New technical solutions are urgently needed to solve this technical problem. Summary of the invention

[0005] In view of the above technical problems, in order to improve the efficiency of data organization and storage based on knowledge graphs and give full play to the advantages of knowledge graphs, the present invention proposes a data organization and storage method based on knowledge graphs. The method can improve the efficiency of data organization and storage based on knowledge graphs and give full play to the advantages of knowledge graphs; further, through model self-learning, it can adaptively adapt to data domain characteristics, further improve the efficiency of data organization and storage, and give full play to the advantages of knowledge graphs.

[0006] In addition, a knowledge graph-based data organization storage system and a computer-readable medium are provided.

[0007] In order to achieve the above object, according to one aspect of the present disclosure, the following technical solution is provided:

[0008] A method for organizing and storing data based on a knowledge graph, the method comprising: S1, configuring metadata; S2, progressively constructing a data storage layer based on a knowledge graph, the data storage layer comprising at least one or more basic SPO layers, one or more entity data aggregation layers, and one or more wide table service application layers; S3, automatically constructing a link for knowledge graph data, the link being used to support various data services.

[0009] In order to achieve the above object, according to another aspect of the present disclosure, the following technical solution is also provided:

[0010] A data organization and storage system based on knowledge graph, comprising:

[0011] Configuration module, used to configure metadata;

[0012] A data storage layer construction module is used to progressively construct a data storage layer based on a knowledge graph, wherein the data storage layer includes at least one or more basic SPO layers, one or more entity data aggregation layers, and one or more wide table service application layers;

[0013] The data link construction module is used to automatically build links of knowledge graph data, and the links are used to support various data services.

[0014] In order to achieve the above object, according to another aspect of the present disclosure, the following technical solution is also provided:

[0015] A computer-readable storage medium for storing non-temporary computer-readable instructions. When the non-temporary computer-readable instructions are executed by a computer, the computer executes the above-mentioned data organization and storage method based on the knowledge graph.

[0016] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following specifically cites the preferred embodiments and describes them in detail with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a process of a method for organizing and storing data based on a knowledge graph according to an embodiment of the present disclosure;

[0018] Figure 2 A schematic diagram of the structure of a data organization and storage system based on a knowledge graph according to an embodiment of the present disclosure;

[0019] Figure 3 A knowledge graph-based data storage layer built for the present disclosure;

[0020] Figure 4 On the basis of the knowledge graph-based data storage layer constructed for this disclosure, links between various application services and data are constructed, and a schematic diagram of links between system internal services and data is constructed;

[0021] Figure 5 is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present disclosure;

[0022] FIG6(A) is a schematic diagram showing an abstract kg_calss entity category in the data organization and storage method based on the knowledge graph according to an embodiment of the present disclosure;

[0023] FIG6(B) is a schematic representation of the abstracted kg_calss entity category in the data organization and storage method based on the knowledge graph according to an embodiment of the present disclosure;

[0024] FIG6(C) is a metadata table for constructing kg_spo_property_schedule abstracted in the data organization and storage method based on the knowledge graph according to an embodiment of the present disclosure;

[0025] FIG6(D) is a record traceability table constructed by abstracting kg_spo_track data in the data organization and storage method based on the knowledge graph according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0027] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0028] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0029] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0030] In order to improve the efficiency of data organization and storage based on knowledge graphs and give full play to the advantages of knowledge graphs, the following embodiments provide a method for data organization and storage based on knowledge graphs. Figure 1 The present invention is a flowchart of a method for organizing and storing data based on a knowledge graph according to an embodiment of the present invention.

[0031] like Figure 1 As shown in the figure, the data organization and storage method based on knowledge graph mainly includes the following steps:

[0032] S1. Configure metadata.

[0033] In order to realize the automated and configurable data organization and storage link, a metadata table is abstracted for data attributes according to the concepts in the knowledge graph model. This application uses the Schema table to describe the metadata.

[0034] In one embodiment of the present invention, in a talent recruitment scenario, a metadata table process is abstracted from a real-world entity (e.g., Company) based on a knowledge graph model. Figures 6(A) to 6(D) shown.

[0035] Figure 6 (A) is a schematic diagram of the kg_calss entity category table abstracted in the knowledge graph-based data organization and storage method disclosed in the present invention; as shown in the figure, the kg_calss entity category table mainly includes entity category number, category name, level, and parent category number.

[0036] Figure 6(B) is a representation of the abstracted kg_property entity attribute in the knowledge graph-based data organization and storage method disclosed in the present invention; as shown in the figure, the kg_property entity attribute table constrains the attributes (basic attributes and relationship attributes) of the entity, mainly including the attribute name, the category to which the attribute belongs, whether it is multi-valued, etc.

[0037] Figure 6(C) is a metadata table for constructing kg_spo_property_schedule that is abstracted from the data organization and storage method based on the knowledge graph disclosed in the present invention; as shown in the figure, the metadata table for constructing kg_spo_property_schedule describes which attributes are constructed for which entity, mainly including task number, attribute name, data source, field mapping, relationship attribute constraints, and whether to construct a reverse relationship.

[0038] Figure 6 (D) is the kg_spo_track data construction record traceability table abstracted from the knowledge graph-based data organization and storage method disclosed in the present invention; as shown in the figure, the kg_spo_track data construction record traceability table records the process information and detailed configuration information during the data construction process to facilitate data traceability, mainly including trackID, entity category, construction time, type, data source, version number, etc.

[0039] In addition, according to the data organization and storage method based on the knowledge graph disclosed in the present invention, there are other auxiliary tables to improve data quality, such as kg_property_constraint property constraint table, kg_source data source table, kg_spo_supply customized wide table conversion configuration table, etc.

[0040] This application abstracts the concept into a metadata table, called KG Schema, which uniformly standardizes, constrains and manages data from the source. This application creates a KG metadata management system, including entity category organization management, entity attribute organization management, and data source organization management.

[0041] Entity category organization management: It is the organization management of the category to which the entity belongs. In one embodiment, in the talent, application, and recruitment scenario, such as the management of resumes, candidates, positions, companies, schools, work experience, etc., there is a blood relationship between entity categories, for example, a company or school is a subcategory of an organization.

[0042] Entity attribute organization management: It is the management of basic attributes of entities and relationship attributes, and the inheritance management of parent-child entity attributes. In one embodiment, in talent, application, and recruitment scenarios, basic attributes, such as name, kgID, alias, etc.; relationship attributes, such as workedIn, itsIndustry, etc.

[0043] Data source organization management: It is the management of data sources. Different data sources have different credibility. The better the data quality, the higher the credibility. In one embodiment, in talent, application, and recruitment scenarios, for example, a company's internal data, label manually annotated data, Baidu, credit, etc.

[0044] In one embodiment, metadata configuration can be performed through deep learning models, such as various convolutional neural network models CNN, deep trust network models DBN, etc. An initial sample set is constructed, and a neural network model is trained to abstract metadata from initial data obtained from multiple data sources; metadata is continuously abstracted and updated through continuous training and learning of the neural network model. Through continuous data training, errors are transmitted from top to bottom to fine-tune the neural network model. Based on the obtained parameters of each layer, the parameters of the entire multi-layer model are further optimized to form a supervised training process. Better metadata configuration effects can be achieved, and the configuration is more accurate.

[0045] S2. Progressively build a data storage layer based on the knowledge graph, where the data storage layer includes at least one or more basic SPO layers, one or more entity data aggregation layers, and one or more wide table service application layers.

[0046] Basic SPO layer: In the process of data organization and storage, the basic SPO layer is first constructed for the data obtained from various data sources. The data organized and stored in this layer is the data that has not yet been normalized to entities and is the basic data of all data. According to the knowledge graph theoretical model, the SPO (Subject-Predicate-Object) triple is used to represent the knowledge of the basic data, and the basic data is organized and stored through the (entity, attribute, relationship) triple. Through the basic SPO layer, the data traceability of the basic data is realized, that is, the generation of traceability IDs, the automatic construction of inverse relationships, and reliable data source attributes.

[0047] Entity data normalization layer: This layer organizes and stores data by deduplicating and normalizing the entity data of the base layer, achieving data normalization, data sorting, deduplication based on single value / multi-value, cleaning of invalid data, and control of data sources.

[0048] Wide table service application layer: This layer organizes the stored data and provides one-stop service to data users, making it convenient for users who do not understand SPO. It mainly implements the mapping of attribute names and data types to wide tables.

[0049] The data storage layer based on the knowledge graph is constructed as follows Figure 3 shown.

[0050] In one embodiment, a deep learning model, such as various convolutional neural network models CNN, deep trust network models DBN, etc., can be used to construct a sample set, train a neural network model, and construct a preliminary single basic SPO layer from the initial data obtained from multiple data sources, a layer of entity data, and a wide table service application layer; through continuous training and learning of the neural network model, the data storage layer is continuously learned and updated. Through continuous data training, errors are transmitted from top to bottom to fine-tune the neural network model. Based on the obtained parameters of each layer, the parameters of the entire multi-layer model are further optimized to form a supervised training process. A better configuration and construction effect of the data storage layer can be achieved, and the hierarchical construction configuration is more reasonable.

[0051] In one embodiment, a ResNet (residual network) model can be used to extract data features from the training set, and the extracted data features are input into the trained decision tree model to classify the basic SPO layer, the entity data layer, and the wide table service application layer. Furthermore, the MultiTaskLoss loss function is used to replace and improve the last layer of ResNet. The MultiTaskLoss loss function is used to evaluate the performance of the model. The expression of the MultiTaskLoss loss function is:

[0052]

[0053] Among them, L({pi},{t i}) is the size of the multi-objective loss;

[0054] i is the sequential index value of the target in the mini-batch sample,

[0055] p i is the predicted probability that sample i belongs to the correctly classified sample; p i * is the reference standard, if it is 1, it is a positive sample, if it is 0, it is a negative sample;

[0056] t i A vector representing the predicted parameterized coordinates;

[0057] t i * is the reference associated with the positive sample;

[0058] λ is a balance parameter that can be used to give the regularization coefficient N cls and N reg Add corresponding weights;

[0059] L cls is the log classification error between the two classes;

[0060] L reg is a regression loss and can only be activated by positive samples.

[0061] In one embodiment, for example, for data in talent, application, and recruitment scenarios, according to the knowledge graph model, the data can be preprocessed first, the missing data can be repaired, unified and standardized, and basic data can be formed. For example, the company address, profile and other data can be repaired, unified and standardized into basic data such as skills, majors, job categories, and industries, represented as (entity, attribute, relationship) triples, and the basic SPO layer can be built; based on the basic data, analysis and calculation are performed, data deduplication and normalization are performed, data sorting is achieved, and according to single value / multi-value deduplication, invalid data is cleaned, and an entity data normalization layer is built; for example, resumes are associated with schools and companies, and relationships are established between each other to accurately characterize the characteristics of user education managers and work experience; attribute names and data types are mapped to wide tables, and a wide table service application layer is built, and users are provided with one-stop services to facilitate users who do not understand SPO. For example, when providing search services to users, the user's needs are accurately understood, and the needs of the client and enterprise are accurately understood based on the knowledge graph relationships of multiple dimensions such as establishment, position, and company.

[0062] S3. Automatically build links of knowledge graph data, which are used to support various data services.

[0063] Figure 4For this disclosure, on the basis of the constructed data storage layer based on the knowledge graph, links between various application services and data are constructed, and links between internal system services and data are constructed. Create tasks that can run in parallel, on the basis of the constructed data storage layer based on the knowledge graph, links between various application services and data are constructed, and links between internal system services and data are constructed. Figure 4 shown.

[0064] Through the above-mentioned data organization and storage method based on knowledge graph, the efficiency of data organization and storage based on knowledge graph can be improved, and the advantages of knowledge graph can be fully utilized; further, through model self-learning, it can adaptively adapt to the characteristics of data domain, further improve the efficiency of data organization and storage, and give full play to the advantages of knowledge graph.

[0065] In the above, although the various steps in the embodiment of the data organization and storage method based on the knowledge graph are described in the above order, those skilled in the art should be aware that the steps in the embodiments of the present disclosure are not necessarily executed in the above order, and they can also be executed in other orders such as reverse order, parallel order, and cross order. Moreover, on the basis of the above steps, those skilled in the art can also add other steps. These obvious variations or equivalent replacement methods should also be included in the protection scope of the present disclosure and will not be repeated here.

[0066] The following is an embodiment of the device disclosed in the present invention. The embodiment of the device disclosed in the present invention can be used to execute the steps implemented by the embodiment of the method disclosed in the present invention. For the convenience of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the embodiment of the method disclosed in the present invention.

[0067] In order to improve the efficiency of data organization and storage based on knowledge graphs and give full play to the advantages of knowledge graphs, the following embodiments provide a data organization and storage system based on knowledge graphs. The device can perform the steps described in the above-mentioned data organization and storage method embodiment based on knowledge graphs.

[0068] Figure 2 FIG. 1 is a schematic diagram of a data organization and storage system based on a knowledge graph according to an embodiment of the present disclosure. Figure 2 As shown, the system 2 mainly includes:

[0069] Configuration module, used to configure metadata.

[0070] In order to realize the automated and configurable data organization and storage link, a metadata table is abstracted for data attributes according to the concepts in the knowledge graph model. This application uses the Schema table to describe the metadata.

[0071] 5. In one embodiment, for example, for data in talent, application, and recruitment scenarios, the metadata table is abstracted according to the concept in the knowledge graph model. Please refer to the above Figures 6(A) to 6(D) shown.

[0072] This application abstracts the concept into a metadata table, called KG Schema, which uniformly standardizes, constrains and manages data from the source. This application creates a KG metadata management system, including entity category organization management, entity attribute organization management, and data source organization management.

[0073] Entity category organization management: It is the organization management of the category to which the entity belongs. In one embodiment, in the talent, application, and recruitment scenario, such as the management of resumes, candidates, positions, companies, schools, work experience, etc., there is a blood relationship between entity categories, for example, a company or school is a subcategory of an organization.

[0074] Entity attribute organization management: It is the management of basic attributes of entities and relationship attributes, and the inheritance management of parent and child entity attributes. In the talent recruitment scenario, basic attributes include "name", "kgID", "alias", etc.; relationship attributes include "workedIn" (employed in), "itsIndustry" (industry), etc.

[0075] Data source organization and management: It is the management of data sources. Different data sources have different credibility. The better the data quality, the higher the credibility. In the talent recruitment scenario, data sources include: zhilian: internal company data (high credibility), label: manually annotated data (high credibility), baidu, credit (average credibility), etc.

[0076] In one embodiment, metadata configuration can be performed through deep learning models, such as various convolutional neural network models CNN, deep trust network models DBN, etc. An initial sample set is constructed, and a neural network model is trained to abstract metadata from initial data obtained from multiple data sources; metadata is continuously abstracted and updated through continuous training and learning of the neural network model. Through continuous data training, errors are transmitted from top to bottom to fine-tune the neural network model. Based on the obtained parameters of each layer, the parameters of the entire multi-layer model are further optimized to form a supervised training process. Better metadata configuration effects can be achieved, and the configuration is more accurate.

[0077] The data storage layer construction module is used to progressively construct a data storage layer based on the knowledge graph, and the data storage layer includes at least one or more basic SPO layers, one or more entity data aggregation layers, and one or more wide table service application layers.

[0078] Basic SPO layer: In the process of data organization and storage, the basic SPO layer is first constructed for the data obtained from various data sources. The data organized and stored in this layer is the data that has not yet been normalized to entities and is the basic data of all data. According to the knowledge graph theoretical model, the SPO (Subject-Predicate-Object) triple is used to represent the knowledge of the basic data, and the basic data is organized and stored through the (entity, attribute, relationship) triple. Through the basic SPO layer, the data traceability of the basic data is realized, that is, the generation of traceability IDs, the automatic construction of inverse relationships, and reliable data source attributes.

[0079] Entity data normalization layer: This layer organizes and stores data by deduplicating and normalizing the entity data of the base layer, achieving data normalization, data sorting, deduplication based on single value / multi-value, cleaning of invalid data, and control of data sources.

[0080] Wide table service application layer: This layer organizes the stored data and provides one-stop service to data users, making it convenient for users who do not understand SPO. It mainly implements the mapping of attribute names and data types to wide tables.

[0081] Figure 3 A knowledge graph-based data storage layer constructed for the present disclosure.

[0082] In one embodiment, a deep learning model, such as various convolutional neural network models CNN, deep trust network models DBN, etc., can be used to construct a sample set, train a neural network model, and construct a preliminary single basic SPO layer from the initial data obtained from multiple data sources, a layer of entity data, and a wide table service application layer; through continuous training and learning of the neural network model, the data storage layer is continuously learned and updated. Through continuous data training, errors are transmitted from top to bottom to fine-tune the neural network model. Based on the obtained parameters of each layer, the parameters of the entire multi-layer model are further optimized to form a supervised training process. A better configuration and construction effect of the data storage layer can be achieved, and the hierarchical construction configuration is more reasonable.

[0083] In one embodiment, a ResNet (residual network) model can be used to extract data features from the training set, and the extracted data features are input into the trained decision tree model to classify the basic SPO layer, the entity data layer, and the wide table service application layer. Furthermore, the MultiTaskLoss loss function is used to replace and improve the last layer of ResNet. The MultiTaskLoss loss function is used to evaluate the performance of the model. The expression of the MultiTaskLoss loss function is:

[0084]

[0085] Among them, L({pi},{t i}) is the size of the multi-objective loss;

[0086] i is the sequential index value of the target in the mini-batch sample,

[0087] p i is the predicted probability that sample i belongs to the correctly classified sample; p i * is the reference standard, if it is 1, it is a positive sample, if it is 0, it is a negative sample;

[0088] t i A vector representing the predicted parameterized coordinates;

[0089] t i * is the reference associated with the positive sample;

[0090] λ is a balance parameter that can be used to give the regularization coefficient N cls and N reg Add corresponding weights;

[0091] L cls is the log classification error between the two classes;

[0092] L reg is a regression loss and can only be activated by positive samples.

[0093] In one embodiment, for example, for data in talent, application, and recruitment scenarios, according to the knowledge graph model, the data can be preprocessed first, the missing data can be repaired, unified and standardized, and basic data can be formed. For example, the company address, profile and other data can be repaired, unified and standardized into basic data such as skills, majors, job categories, and industries, represented as (entity, attribute, relationship) triples, and the basic SPO layer can be built; based on the basic data, analysis and calculation are performed, data deduplication and normalization are performed, data sorting is achieved, and according to single value / multi-value deduplication, invalid data is cleaned, and an entity data normalization layer is built; for example, resumes are associated with schools and companies, and relationships are established between each other to accurately characterize the characteristics of user education managers and work experience; attribute names and data types are mapped to wide tables, and a wide table service application layer is built, and users are provided with one-stop services to facilitate users who do not understand SPO. For example, when providing search services to users, the user's needs are accurately understood, and the needs of the client and enterprise are accurately understood based on the knowledge graph relationships of multiple dimensions such as establishment, position, and company.

[0094] The data link construction module is used to automatically build links of knowledge graph data, and the links are used to support various data services.

[0095] Figure 4On the basis of the knowledge graph-based data storage layer constructed for this disclosure, links between various application services and data are constructed, and a schematic diagram of links between system internal services and data is constructed.

[0096] Figure 5 is a schematic diagram illustrating a computer-readable storage medium according to an embodiment of the present disclosure. Figure 5 As shown, according to the computer-readable storage medium 50 of the embodiment of the present disclosure, non-transitory computer-readable instructions 51 are stored thereon. When the non-transitory computer-readable instructions 51 are executed by the processor, all or part of the steps of the data organization and storage method based on the knowledge graph of each embodiment of the present disclosure are executed.

[0097] The computer-readable storage medium 50 includes but is not limited to optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0098] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0099] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0100] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0101] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0102] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0103] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0104] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0105] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A data organization and storage method based on knowledge graph, characterized by: The method comprises: S1, configuration metadata; S2. Progressively build a data storage layer based on the knowledge graph, where the data storage layer includes at least one or more basic SPO layers, one or more entity data aggregation layers, and one or more wide table service application layers; The ResNet model is used to extract data features from the training set, and the extracted data features are input into the trained decision tree model to classify the basic SPO layer, the entity data layer, and the wide table service application layer; Among them: Use MultiTaskLoss loss function to replace and improve the last layer of the ResNet model. The MultiTaskLoss loss function is used to evaluate the performance of the model; wherein the expression of the MultiTaskLoss loss function is: Among them, L({p i },{t i }) is the size of the multi-objective loss; i is the sequential index value of the target in the mini-batch sample, p i is the predicted probability that sample i belongs to the correctly classified sample; p i * is the reference standard, if it is 1, it is a positive sample, if it is 0, it is a negative sample; t i A vector representing parameterized coordinates; t i * is the reference associated with the positive sample; λ is a balance parameter that can be used to give the regularization coefficient N cls and N reg Add corresponding weights; L cls is the log classification error between the two classes; L reg is a regression loss and can only be activated by positive samples; S3. Automatically build links of knowledge graph data, which are used to support various data services.

2. As claimed in claim 1, the data organization and storage method based on the knowledge graph, wherein the configuration metadata is configured by abstracting the metadata from the attributes of the data according to the concept of the knowledge graph model.

3. As claimed in claim 2, the data organization and storage method based on the knowledge graph, wherein the metadata organization and storage system is constructed based on the Schema through the configuration of metadata to realize the organization and management of entity categories, the organization and management of entity attributes, and the organization and management of data sources.

4. The data organization and storage method based on knowledge graph as claimed in claim 1, wherein a sample set is constructed, a deep learning model is trained, and metadata is abstracted from sample data obtained from multiple data sources; metadata is continuously abstracted and updated through continuous training and learning of the deep learning model.

5. The data organization and storage method based on knowledge graph as claimed in claim 1, wherein: In the basic SPO layer, SPO triples are used to represent the knowledge of basic data, and <entity, attribute, relationship> triples are used to organize and store basic data.

6. The method for organizing and storing data based on a knowledge graph as claimed in claim 1, wherein the entity data is normalized to a layer of organized and stored data, and the basic layer entity data is deduplicated, normalized, sorted, and cleaned to achieve normalization, sorting, cleaning, and organize and manage data sources; The wide table service application layer organizes the stored data to provide a one-stop service to data users, and realizes the mapping of attribute names and data types to wide tables.

7. A data organization and storage system based on knowledge graph, characterized in that: include: Configuration module, used to configure metadata; A data storage layer construction module is used to progressively construct a data storage layer based on a knowledge graph, wherein the data storage layer includes at least one or more basic SPO layers, one or more entity data aggregation layers, and one or more wide table service application layers; The ResNet model is used to extract data features from the training set, and the extracted data features are input into the trained decision tree model to classify the basic SPO layer, the entity data layer, and the wide table service application layer; Among them: Use MultiTaskLoss loss function to replace and improve the last layer of the ResNet model. The MultiTaskLoss loss function is used to evaluate the performance of the model; wherein the expression of the MultiTaskLoss loss function is: Among them, L({p i },{t i }) is the size of the multi-objective loss; i is the sequential index value of the target in the mini-batch sample, p i is the predicted probability that sample i belongs to the correctly classified sample; p i * is the reference standard, if it is 1, it is a positive sample, if it is 0, it is a negative sample; t i A vector representing parameterized coordinates; t i * is the reference associated with the positive sample; λ is a balance parameter that can be used to give the regularization coefficient N cls and N reg Add corresponding weights; L cls is the log classification error between the two classes; L reg is a regression loss and can only be activated by positive samples; The data link construction module is used to automatically build links of knowledge graph data, and the links are used to support various data services.