User data graph construction method, system, electronic device and storage medium

Abstract

The user data graph construction method, system, electronic device and storage medium disclosed in the embodiment of the present invention relate to the field of data analysis technology. The method includes: analyzing user data, establishing an index and field attribute structure corresponding to the user data, and creating a primary key for each index; wherein the user data includes data on entities and relationships between entities; importing the user data according to the established index and field attribute structure; mapping the index and field attributes with the preset index and field attributes in the graph database to obtain a user data graph. The present invention facilitates improving the efficiency of user data graph construction and is applicable to various data analysis scenarios such as network asset security, corporate relationships, and internal organizational structures of units.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a method, system, electronic device and storage medium for constructing a user data graph. Background Art

[0002] A graph database is a data storage engine that combines the basic graph structure of vertices and edges with persistence technology and a traversal (query) language to create a database optimized for storing and quickly retrieving highly connected data. Currently, known graph databases, including Neo4j and Hugegraph, support importing unstructured data in formats such as CSV files.

[0003] However, the inventors of this application found in the process of realizing the invention that for the import of the above-mentioned unstructured data, it is generally necessary to manually sort out the exported files, distinguish between vertices (Vertex, representing entities) and edges (Edge, representing the relationship between entities), and define the fields in the CSV file according to the requirements of the graph database. The workload is large and it is easy to make import errors, resulting in low efficiency in building user data graphs. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, system, electronic device, and storage medium for constructing a user data graph, so as to improve the efficiency of constructing a user data graph.

[0005] In order to achieve the above-mentioned purpose of the invention, the following technical solutions are adopted:

[0006] In a first aspect, an embodiment of the present invention provides a method for constructing a user data graph, the method comprising the steps of: analyzing user data, establishing an index and field attribute structure corresponding to the user data, and creating a primary key for each index; wherein the user data includes data on entities and relationships between entities;

[0007] Importing the user data according to the established index and field attribute structure;

[0008] The index and field attributes are mapped with the preset index and field attributes in the graph database to obtain a user data graph.

[0009] Optionally, before analyzing the user data, the method further comprises: acquiring the user data; the user data comprises: unstructured data;

[0010] The method further includes: when analyzing the user data, determining whether there is an incremental field in the unstructured data;

[0011] If it does not exist, add an auto-incrementing long integer field as the increment field;

[0012] The incrementally imported user data is determined according to the incremental field.

[0013] Optionally, the user data further includes: structured data stored in a standard database;

[0014] The method further includes: acquiring the structured data from the standard database; the structured data is stored in the form of tables, each table has fields, and each field carries field attributes;

[0015] According to the table, field and field attributes, mapping is performed with the preset index and field attributes in the graph database to obtain the graph of the structured data; the preset index is maintained in the form of a table.

[0016] Optionally, before mapping the index and field attributes with the preset index and field attributes in the graph database, the method also includes: configuring the mapping relationship in the graph database; the mapping relationship includes: index and field attributes, the index is used to represent the entity, and the field attributes include foreign keys used to represent the relationship between entities.

[0017] Optionally, after configuring the mapping relationship in the graph database, the method further includes: periodically and incrementally importing user data into the graph database:

[0018] When incrementally importing user data, the graph database is called to create an entity interface according to the mapping relationship, and field attributes are added to the entity;

[0019] Also, call the graph database to create an entity relationship interface and add field attributes representing the relationship between entities to the graph database.

[0020] Optionally, after obtaining the user data graph, the method further includes: displaying or discovering the topological structure of entities and relationships between entities based on the user data graph.

[0021] In the second aspect, an embodiment of the present invention also provides a user data graph construction system, including: a data processing program module, used to analyze user data, establish an index and field attribute structure corresponding to the user data, and create a primary key for each index; wherein, the user data includes data on entities and the relationships between entities; an import program module, used to import the user data according to the established index and field attribute structure; a mapping program module, used to map the index and field attributes with the preset index and field attributes in the graph database to obtain a user data graph.

[0022] In a third aspect, an embodiment of the present invention provides an electronic device comprising: one or more processors; a memory; one or more executable programs stored in the memory, the one or more processors reading the executable program code stored in the memory, and running the program corresponding to the executable program code to execute any of the user data graph construction methods described in the first aspect.

[0023] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement any user data graph construction method described in the first aspect.

[0024] The user data graph construction method, system, electronic device and storage medium provided by the embodiments of the present invention can automatically analyze the user data after receiving user instructions when it is necessary to construct a user data graph based on user data, establish an index and field attribute structure corresponding to the user data, and create a primary key for each index; wherein the user data includes data on entities and the relationships between entities; import the user data according to the established index and field attribute structure; map the index and field attributes to the preset index and field attributes in the graph database to obtain a user data graph. Through the above steps, the formatting, importing and mapping of user data are automatically realized to obtain a user data graph, which is not prone to errors and facilitates improving the efficiency of user data graph construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 This is a flowchart of a method for constructing a user data graph according to an embodiment of the present invention;

[0027] Figure 2 This is a flowchart of a method for constructing a user data graph according to another embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the structure of a user data graph according to an embodiment of the present invention;

[0029] Figure 4 A schematic diagram of vertex configuration mapping in a graph database according to an embodiment of the present invention;

[0030] Figure 5 A schematic diagram of a mapping configuration between entities and entity relationships in a graph database according to an embodiment of the present invention;

[0031] Figure 6 This is a schematic block diagram of the user data graph construction system architecture according to one embodiment of the present invention;

[0032] Figure 7 The figure is a schematic block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0034] It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0035] The method for constructing a user data graph provided by an embodiment of the present invention can be applied to various analysis scenarios that can display entities and entity relationships using graph data ("graph data" is also commonly known as "graph" in the industry), such as network asset security analysis, enterprise relationship analysis, internal organizational structure analysis, business behavior analysis, online shopping analysis, etc. For scenarios where unstructured data such as CSV files provided by users are disordered and cannot be directly imported into a graph database, the data is first imported into the Elasticsearch retrieval server for formatting through a data cleansing service, and the indexes and fields required by the preset graph database are added, and mapped to the preset indexes and field attributes in the graph database on which the user data graph (which can also be understood as a knowledge graph) is based. This facilitates the rapid and efficient construction of a user data graph, which can intuitively display the entities in the user data and the relationships between entities, making it easier for analysts to analyze the target analysis object, such as asset security.

[0036] It should be noted that the method can be solidified in a certain manufactured physical product in the form of software, and when the user uses the product, the method flow of the present application can be reproduced.

[0037] Before describing the embodiments of the present invention in detail, to help understand the technical solutions of the embodiments of the present invention, the technical terms or nouns involved in one or more embodiments of the present invention are explained as follows:

[0038] Neo4j graph database: Neo4j is a high-performance NOSQL graph database that stores structured data on the network instead of in tables.

[0039] Hugegraph graph database: HugeGraph is an open source graph database system developed by Baidu that can store massive amounts of vertices and edges. It is based on the Apache TinkerPop 3 framework and supports the Gremlin query language.

[0040] CSV file: A file in the Comma-Separated Values ​​(CSV) format, sometimes also called Character-Separated Values ​​because the delimiter character can be other than a comma. The file stores tabular data (numbers and text) in plain text.

[0041] ElasticSearch search server, abbreviated as ES: Elasticsearch is a search (retrieval) server based on Lucene. It provides a distributed multi-user full-text search engine based on the RESTful web interface (a web server based on the REST architecture can be used to replace a web server based on the SOAP protocol in front-end development. In a RESTful service system, the server uses URIs to expose resources, and the client uses four HTTP verbs to access resources).

[0042] A graph database is a data storage engine that combines a basic graph structure of vertices (representing entities) and edges (representing relationships between entities) with persistence technology and a traversal (query) language to create a database optimized for storing and quickly retrieving highly connected data. A graph database can intuitively display the complete structure of the relationships between entities in the data, making it easy to observe and manipulate.

[0043] Figure 1 This is a flow chart of a method for constructing a user data graph according to an embodiment of the present invention; Figure 1 As shown, the user data graph construction method includes the following steps:

[0044] S110: Analyze user data, establish indexes and field attribute structures corresponding to the user data, and create a primary key for each index.

[0045] The user data includes data on entities and relationships between entities.

[0046] The entity is a tangible or intangible thing that exists objectively and can be distinguished from each other. For example, an entity can be a computer, a router, computer software, etc., and is generally represented by a vertex in a graph database.

[0047] The relationship between entities refers to the connection between entity sets of different entity types. The connection between entities includes one-to-one, one-to-many, many-to-many and other types. For example, a department is divided into multiple computers, which can be represented as a computer entity set of the department, marked as A. In an attack incident, the computers in the aforementioned department are attacked, which can be represented as multiple entities in the attack incident and the attack process entity set, marked as B. Then, a mapping relationship can be established between A and B, and the relationship between A and B, as well as the relationship between entities in A and B, can be represented by graph data.

[0048] Index, as used in this article, refers to the term "index" in the Elasticsearch (ES) system. In earlier Elasticsearch (ES) versions, it was similar to the concept of a library in MySQL. Later versions removed the concept of a type class, and the concept of an index reverted to a table. In this article, index can be understood as equivalent to a table. Simply put, an index is a document container, a collection of documents of a certain type, logically categorized and physically partitioned, containing fields and their value ranges.

[0049] For example, a table in Mysql is as follows:

[0050] ID Name Age Sex 1 Zhang San 22 male 2 Li Si 35 female 3 Wang Wu 22 female

[0051] If you store the data in the preceding table in ES, ES will create the following indexes:

[0052] Term Posting List Zhang San 1 Li Si 2 Wang Wu 3 22 [1,3] 35 2 male 1 female [2,3]

[0053] Term is the field value, and Posting List is the value range of the field value.

[0054] To help understand the embodiments of the present invention, the following briefly introduces the technical terms involved: Attribute: An entity has many characteristics, each of which is called an attribute. Each attribute has a value range, which can be an integer, a real number, or a string. For example, a student (entity) has attributes such as student ID, name, age, and gender, and the corresponding value ranges are character, string, integer, and string.

[0055] In ES indexes (tables), fields are the named units that mark entity attributes. Fields are the smallest unit of information that can be named, and are therefore also called data elements or elementary items. Field names are often the same as entity attributes, such as a student's ID, name, age, and gender. Therefore, fields and field attributes are sometimes used interchangeably in this article.

[0056] Record: An ordered collection of fields is called a record. A record generally describes an entity, so a record can also be defined as the set of fields that fully describes an entity. For example, a student record consists of an ordered set of fields (student ID, name, age, gender, etc.).

[0057] Primary Key: A field or combination of fields that uniquely identifies an index or table.

[0058] S120: Import the user data according to the established index and field attribute structure.

[0059] In this embodiment, after establishing corresponding indexes and field attribute structures based on user data, the user data is imported into the corresponding indexes based on the indexes and field attribute structures. For example, user data may include data on departments, employees, computers, CPUs, memory, software, vulnerabilities, and so on. Based on the entities and relationships between entities in the user data, corresponding indexes are established, and fields and corresponding value range identifiers are created in each index. The user data is then imported into the indexes to form structured data.

[0060] S130. Map the index and field attributes with the preset index and field attributes in the graph database to obtain a user data graph.

[0061] The graph database can be Baidu's open source graph database HugeGraph, in which corresponding index and field attribute mapping relationships are pre-configured. After the structured data is formed in step S120, it is mapped with the corresponding index and field attributes in the graph database, and the user data is automatically imported into the graph database to obtain a user data graph. In this way, the relationship between entities can be intuitively displayed or topologically discovered (abbreviated as topology).

[0062] The user data graph construction method provided by the embodiment of the present invention, when it is necessary to construct a user data graph based on user data, can automatically analyze the user data after receiving user instructions, establish an index and field attribute structure corresponding to the user data, and create a primary key for each index; wherein the user data includes data on entities and the relationships between entities; according to the established index and field attribute structure, the user data is imported; the index and field attributes are mapped with the preset index and field attributes in the graph database to obtain a user data graph. Through the above steps, the formatting, importing and mapping of user data are automatically realized to obtain a user data graph, reducing the error probability and thus avoiding import errors or even the inability to import, and facilitating the improvement of the efficiency of user data graph construction.

[0063] Please see Figure 2As shown, before analyzing the user data, the method further includes: obtaining user data; the user data includes: unstructured data; the unstructured data mainly exists in Excel table or CSV text format, or other text formats, etc. If it is in Excel table format, it can be converted into a CSV format file first.

[0064] In this embodiment, during the data acquisition and cleaning stage, the big data component Elasticsearch can be used for data storage and retrieval, which improves the speed of data retrieval and matching. After retrieving and acquiring the data, Elasticsearch acts as a data cleaning middleware, and its main function is to store the user's original data to be analyzed.

[0065] The user data is initially cleaned using the Elasticsearch data cleaning tool, including formatting and standardizing the structured processing of the aforementioned data, making the data further usable, relatable, and analyzable.

[0066] To ensure data uniqueness and real-time performance during the analysis of user data, the method further includes: determining whether an incremental field exists in the unstructured data; if not, adding an auto-incrementing long integer field as the incremental field; the incremental field may be time. Based on the incremental field, user data to be incrementally imported is determined. Incremental import involves periodically importing data within a range of specified fields into the target data source.

[0067] In this embodiment of the present invention, adding a long integer auto-increment field to the original data ensures data uniqueness and real-time updates, and also ensures the feasibility of incremental data writing. Because the ES search server supports distributed deployment, data retrieval and processing performance can be improved by adding nodes, disks, or memory.

[0068] Furthermore, in terms of data conversion, Logstash can be used to collect and transform data from multiple sources, then send it to a "repository" such as the aforementioned Elasticsearch. This data can then be filtered and matched through visual association configuration, while providing an intuitive overview of the accuracy of data mapping. Dynamic adjustments and optimizations are also supported, making it easier for analysts to use.

[0069] Please see Figure 2As shown, user data can generally be roughly divided into two categories: structured data and unstructured data. The processing of unstructured data has been described above and will not be repeated here. In the case where the user data also includes structured data stored in a standard database, in some embodiments, the method further includes: obtaining the structured data from the standard database; the structured data is stored in the form of a table, each table has fields, and each field carries field attributes; mapping the table and field attributes with the preset index and field attributes in the graph database to obtain a graph of the structured data; the preset index is maintained in the form of a table.

[0070] The structured data may be user data stored in a database such as Mysql, PostgreSQL, Oracle, or SqlServer.

[0071] In this embodiment, it supports docking with multiple user data sources, supports not only unstructured data but also incremental import of structured data, realizes incremental data fusion processing, and can check the accuracy of data fusion.

[0072] In some embodiments, before mapping the index and field attributes to pre-set indexes and field attributes in the graph database, the method further includes configuring a mapping relationship in the graph database; the mapping relationship includes an index and field attributes, the index being used to represent an entity, and the field attributes including a foreign key representing a relationship between entities. The foreign key is used to establish a connection with another table.

[0073] Mapping is the process of defining a document, including which fields it contains, whether these fields are saved, indexed, and word segmented.

[0074] It is understandable that a mapping must be defined before adding data. After the mapping is defined, user data can be automatically added to the graph database according to the mapping relationship to form a user data graph.

[0075] In an embodiment of the present invention, after configuring the mapping, it supports mapping the fields in the data source and the field attributes in the graph database; it also supports associating the relationships in the data source and mapping the relationships with the relationships in the graph database to realize automatic import of user data, thereby quickly building a user data graph.

[0076] As an optional embodiment, after configuring the mapping relationship in the graph database, the method further includes: regularly and incrementally importing user data into the graph database: when incrementally importing user data, calling the graph database to create an entity interface according to the mapping relationship, and adding field attributes to the entity; and calling the graph database to create an entity relationship interface, and adding field attributes representing the relationship between entities to the graph database.

[0077] In this embodiment, the configured mapping relationship can be added to the system's scheduled tasks. The system executes the scheduled tasks as required and writes data into the graph database regularly to update the user data graph.

[0078] Further, see Figure 2 As shown, after obtaining the user data graph, the method further includes: displaying or discovering the topological structure of entities and the relationships between entities based on the user data graph.

[0079] In this embodiment, all user data graphs may be displayed at once, or may be gradually discovered and displayed as needed.

[0080] To help understand the technical solutions provided by the embodiments of the present invention, the technical solutions and technical effects of the embodiments of the present invention are described in detail below with reference to specific examples:

[0081] Suppose an enterprise wants to use a knowledge graph to display its internal organizational structure, departments, employees, hosts, and the relationships between them, while also identifying which hosts are at risk. The enterprise provides asset information in Excel.

[0082] Considering that asset information involves multiple entities and complex relationships, as well as issues such as query efficiency, the graph database uses the open source Hugegraph graph database to store and display user data.

[0083] There are two situations for the data provided by users. One is that the user allows the method of this embodiment to be solidified in the system formed by the physical product to connect to their standard database; the other is that the user does not allow direct connection to their data source library, but can export part of the data in Excel format to analyze the description of the data provided by the party and the relationship between the data.

[0084] After analyzing the user's Excel data, the entities that need to be displayed and the relationships between entities are clarified, including: departments, users, hosts, software attributes and the relationships between them.

[0085] For the first case (generally referring to unstructured user data), analysts leverage the Elasticsearch service, which supports data storage and retrieval, as well as distributed deployment. After analyzing user documents, they establish a corresponding index and field structure in Elasticsearch, with a primary key for each entity. Then, using its built-in tools, they batch-import user data based on this established index and field structure. After importing, they use visual configuration to map Elasticsearch index fields to conceptual attributes in the graph database, enabling automated, scheduled data writes and improving the efficiency of building user data graphs.

[0086] For the other case (referring to the standard database that allows docking with users), the analyst can support the configuration of the data source through the system and directly connect to the database without performing operations such as data format processing. In addition, through visual configuration, the fields in the data source and the conceptual attributes in the graph database can be mapped to achieve automatic writing of scheduled data, which is convenient for improving the efficiency of building user data maps.

[0087] in, Figure 4 This diagram shows the attribute mapping between the source data firewall and the firewall concept in the library. The source fields on the right come from the source data fields and display the firewall field details, while the attributes on the right come from the library attributes and display the firewall concept field details in the library. You can see that the source fields and attributes have a one-to-one correspondence, ensuring the accuracy of the data attribute mapping.

[0088] Figure 5 This diagram shows the relationship between vulnerabilities and software in the source data and maps this relationship to the relationship defined in the library. The "software_name" field in the source field on the right comes from the vulnerability's source data field, the "software_name" field in the target field on the right comes from the software's target data field, and the "Vulnerability_VulnerAffect_ApplicationSoftware" relationship on the right comes from the library relationship. You can see that the source data relationships map exactly to the library's defined relationships, ensuring the accuracy of the data relationship mapping.

[0089] After clarifying the data source and graph data structure, the above configuration can be used to regularly import user data into the graph database, helping operation and maintenance personnel analyze user data using knowledge graphs.

[0090] In order to achieve uniqueness and dynamic update of user data, the user data can be imported into the ElasticSearch big data retrieval engine after adding a primary key auto-increment field.

[0091] The front-end can directly configure the imported entities and the relationships between them on the page, add them to the scheduled tasks, and execute the data import according to the scheduled time. After generating the user data map, the relationship between the entities can be intuitively discovered (topology discovery) in the map analysis interface, such as Figure 3 shown.

[0092] Based on the data provided by the user, Figure 3 The graph shows the user's internal departments, employees, computers, CPUs, memory, software, vulnerabilities, and other concepts, and illustrates the relationships between them. With the data source (user data) and graph data structure clearly defined, user data can be configured to be imported into the graph database on a scheduled basis, helping operations personnel analyze user data using knowledge graphs.

[0093] The user data graph construction method provided by the embodiments of the present invention offers a customizable, configurable, and visual way to automatically import data. This improves the efficiency of user data graph construction compared to manual data entry or manual data organization followed by import into a graph database. Furthermore, to address the current issue of slow data export, which requires user data to be exported to a CSV format and stored locally, the present invention provides a method for directly connecting to user data, pulling data from libraries, tables, and fields in real time, further improving the efficiency of user data graph construction.

[0094] Furthermore, by formatting unstructured data and configuring mapping relationships in the graph database, the problem of dynamic integration of various types of user data into the graph database is solved.

[0095] Furthermore, by setting an auto-increment field and configuring a scheduled write task, the memory overflow problem when dynamically writing data was solved. Furthermore, the visualization configuration tool in the ES search server was used to solve the visibility problem of data mapping.

[0096] Example 2

[0097] Figure 6 This is a schematic diagram of the user data graph construction system architecture according to an embodiment of the present invention. Figure 6 As shown, based on the same technical concept as the embodiments in the aforementioned embodiment one, a user data graph construction system is also provided, including: a data processing program module 210, used to analyze user data, establish an index and field attribute structure corresponding to the user data, and create a primary key for each index; wherein the user data includes entities and data on the relationship between entities; an import program module 220, used to import the user data according to the established index and field attribute structure; a mapping program module 230, used to map the index and field attributes with the preset index and field attributes in the graph database to obtain a user data graph.

[0098] The system of this embodiment can be used to perform Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects to those of embodiment 1, which will not be described in detail here and can be referenced by each other.

[0099] In addition, it is understandable that Figure 6 The system shown is also applicable to the execution step processes of other embodiments in Example 1. For details, please refer to the relevant description in Example 1, which will not be repeated here.

[0100] like Figure 2 As shown, in some embodiments, the system includes a server and a graph database. The server includes an Elasticsearch (ES) retrieval service running in its environment. The graph database establishes a communication connection with the server, which in turn establishes a communication connection with one or more user databases. The server can retrieve user data from a standard database or Elasticsearch (ES). Elasticsearch (ES) formats the unstructured data and imports the corresponding data in batches.

[0101] Subsequently, a scheduled import task can be executed to read the data to be imported and its relationships from the database and import them into the graph database to build a user data graph. When it is necessary to display the entities and entity relationships in the user data, the data is read from the user data graph and displayed on the front end.

[0102] The user data graph construction system provided by the embodiment of the present invention provides a customized, configurable, and visual operation method to automatically import data. Compared with the solution of manually entering or manually organizing data and then importing it into a graph database, it is convenient to improve the efficiency of user data graph construction.

[0103] Example 3

[0104] The present invention also provides an electronic device according to an embodiment, comprising: one or more processors; a memory; one or more executable programs stored in the memory, and the one or more processors read the executable program code stored in the memory and run the program corresponding to the executable program code to execute the user data graph construction method described in any one of the first embodiments.

[0105] Figure 7 FIG. 1 is a schematic structural diagram of an embodiment of an electronic device of the present invention, which can implement any of the methods described in Embodiment 1 of the present invention, such as Figure 7As shown, as an optional embodiment, the above-mentioned electronic device may include: a shell 41, a processor 42, a memory 43, a circuit board 44 and a power supply circuit 45, wherein the circuit board 44 is placed inside the space enclosed by the shell 41, and the processor 42 and the memory 43 are arranged on the circuit board 44; the power supply circuit 45 is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory 43 is used to store executable program code; the processor 42 runs the program corresponding to the executable program code by reading the executable program code stored in the memory 43, and is used to execute the user data mapping construction method described in any of the above-mentioned embodiments one.

[0106] The specific execution process of the above steps by the processor 42 and the steps further executed by the processor 42 by running the executable program code can be found in the description of the user data graph construction method embodiment 1 of the present invention, and will not be repeated here.

[0107] The electronic device exists in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by having mobile communication functions and are mainly aimed at providing voice and data communications. Such terminals include: smart phones, multimedia phones, functional phones, and low-end phones. (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. Such terminals include: PDA, MID and UMPC devices, such as iPad. (3) Portable entertainment devices: These devices can display and play multimedia content. Such devices include: audio and video playback modules (such as iPod), handheld game consoles, e-books, as well as smart toys and portable car navigation devices. (4) Servers: Devices that provide computing services. The server's composition includes processors, hard disks, memory, system buses, etc. The server is similar to the general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability. (5) Other electronic devices with data interaction functions.

[0108] An embodiment of the present invention also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the user data graph construction method described in any of the above-mentioned embodiments.

[0109] In summary, according to the above embodiments, the user data graph construction method and system provided by the embodiments of the present invention provide a customized, configurable, and visual operation method to automatically import data. Compared with the solution of manually entering or manually organizing data and then importing it into a graph database, it is convenient to improve the efficiency of user data graph construction.

[0110] Furthermore, by formatting unstructured data and configuring mapping relationships in the graph database, the problem of dynamic integration of various types of user data into the graph database is solved.

[0111] Furthermore, by setting an auto-increment field and configuring a scheduled write task, the memory overflow problem when dynamically writing data was solved. Furthermore, the visualization configuration tool in the ES search server was used to solve the visibility problem of data mapping.

[0112] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0113] Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0114] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can also be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0115] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for constructing a user data graph, characterized in that: The method comprises the steps of: Analyze user data, establish indexes and field attribute structures corresponding to the user data, and create a primary key for each index; wherein the user data includes data on entities and relationships between entities; Importing the user data according to the established index and field attribute structure; Mapping the index and field attributes with the preset index and field attributes in the graph database to obtain a user data graph; Before analyzing the user data, the method further includes: acquiring the user data; the user data includes: unstructured data; When analyzing the user data, the method further includes: determining whether there is an incremental field in the unstructured data; If it does not exist, add an auto-incrementing long integer field as the increment field; Determining the incrementally imported user data according to the incremental field; Before mapping the index and field attributes with the preset index and field attributes in the graph database, the method also includes: configuring the mapping relationship in the graph database; the mapping relationship includes: index and field attributes, the index is used to represent the entity, and the field attributes include foreign keys used to represent the relationship between entities.

2. The method for constructing a user data graph according to claim 1, characterized in that: The user data also includes: structured data stored in a standard database; The method further includes: acquiring the structured data from the standard database; the structured data is stored in the form of tables, each table has fields, and each field carries field attributes; The table and field attributes are mapped with the preset index and field attributes in the graph database to obtain the graph of the structured data; the preset index is maintained in the form of a table.

3. The method for constructing user data graph according to claim 1, characterized in that: After configuring the mapping relationship in the graph database, the method further includes: periodically incrementally importing user data into the graph database: When incrementally importing user data, the graph database is called to create an entity interface according to the mapping relationship, and field attributes are added to the entity; Also, call the graph database to create an entity relationship interface and add field attributes representing the relationship between entities to the graph database.

4. The method for constructing a user data graph according to claim 3, characterized in that: After obtaining the user data graph, the method further includes: displaying or discovering the topological structure of entities and relationships between entities based on the user data graph.

5. An electronic device, characterized in that: include: One or more processors; Memory; One or more executable programs are stored in the memory, and the one or more processors read the executable program code stored in the memory and run the program corresponding to the executable program code to execute the user data graph construction method described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the user data graph construction method described in any one of claims 1 to 4.

Images