Atlas construction method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a graph construction method and device, electronic equipment and a storage medium, and belongs to the technical field of resource big data. The method comprises the steps of obtaining text data, and splitting the text data to obtain sentence data; performing semantic analysis on the sentence data to obtain a resource data model; obtaining a target label, and associating a field in the resource data model with the target label to form a genetic map; and detecting and merging repeated nodes of the gene map, and fusing the nodes to obtain a resource gene pool map. According to the embodiment of the invention, the construction period of the gene pool can be shortened, and the data accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of resource big data technology, and in particular to a graph construction method and device, electronic equipment and storage medium. Background Art

[0002] Cultural data are core information resources, especially cultural gene data, which are as important as biological gene data. How to understand these cultural data from the massive existing literature and the latest literature that is constantly being produced has become an extremely challenging problem.

[0003] In the related technology, the existing methods of analyzing cultural data to construct gene library maps have the problems of long construction cycle, consumption of a large amount of human resources and difficulty in ensuring the accuracy of data.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to propose a map construction method and device, electronic equipment and storage medium, aiming to shorten the construction period of the gene library map and improve the accuracy of the data.

[0006] To achieve the above object, an embodiment of the present application provides a method for constructing a graph, the method comprising the following steps:

[0007] Acquire text data, and split the text data to obtain sentence data;

[0008] Performing semantic analysis on the sentence data to obtain a resource data model;

[0009] Obtaining a target tag, and associating a field in the resource data model with the target tag to form a gene map;

[0010] The gene map is subjected to repeated node detection and merging, and then the nodes are fused to obtain a resource gene pool map.

[0011] In some embodiments, the performing semantic analysis on the sentence data to obtain a resource data model includes:

[0012] Performing word segmentation processing on the sentence data to obtain verbs, nouns and adjectives;

[0013] A first time value is identified from the sentence data, and an entity is formed based on the first time value, the verb, the noun, and the adjective.

[0014] In some embodiments, the quadruple includes a time sequence, an entity, an attribute of the entity, and an attribute value of the entity, and the semantic analysis of the sentence data to obtain a resource data model includes:

[0015] Extracting attribute values ​​of the entity from the text data;

[0016] forming a basic entity model in the form of the four-tuple according to the entity and the attribute value of the entity;

[0017] A resource data model is constructed based on the basic entity model.

[0018] In some embodiments, the four-tuple includes a time sequence, a first entity, a relationship, and a second entity, and the semantic analysis of the sentence data to obtain a resource data model includes:

[0019] extracting the relationship between the entities from the text data;

[0020] forming a basic entity model in the form of the four-tuple according to the entities and the relationships between the entities;

[0021] A resource data model is constructed based on the basic entity model.

[0022] In some embodiments, the obtaining of the target tag and associating the field in the resource data model with the target tag to form a gene map includes:

[0023] Aggregate and retrieve the first time value of the entity to form a unique list, and store the unique list in a time queue;

[0024] Data fusion of the resource data model is performed according to the time queue, and the data fusion includes data merging, data upper and lower relationship merging, and data attribute definition merging.

[0025] In some embodiments, the obtaining of the target tag and associating the field in the resource data model with the target tag to form a gene map includes:

[0026] Retrieving the resource data model according to the target time to obtain a retrieval result, wherein the target time belongs to the time queue;

[0027] generating a target label according to the target time;

[0028] The target tag is added to the search result in the resource data model.

[0029] In some embodiments, the detecting and merging of duplicate nodes on the gene map, and then fusing the nodes to obtain a resource gene pool map, includes:

[0030] Performing a repeat detection on the gene map to obtain a plurality of repeat nodes;

[0031] Calculating the weight of the repeated node according to a weight calculation formula;

[0032] The repeated node with the largest weight is taken as the result node, and the attributes and association relationships of other nodes and the result node are merged to obtain a resource gene pool map, wherein the other nodes are the nodes in all the repeated nodes except the result node.

[0033] To achieve the above object, another aspect of the embodiment of the present application provides a graph construction device, the device comprising:

[0034] A text splitting module, used to obtain text data and split the text data to obtain sentence data;

[0035] A semantic analysis module, used for performing semantic analysis on the sentence data to obtain a resource data model;

[0036] The tag association module is used to obtain the target tag, associate the fields in the resource data model with the target tag, and form a gene map.

[0037] The merging module is used to detect and merge duplicate nodes in the gene map, and then fuse the nodes to obtain a resource gene pool map.

[0038] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method when executing the computer program.

[0039] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0040] The embodiments of the present application include at least the following beneficial effects: the present application provides a graph construction method and device, an electronic device and a storage medium. The scheme obtains text data, splits the text data to obtain sentence data, reduces manual intervention through automated splitting and improves processing speed; performs semantic analysis on sentence data to obtain a resource data model, which is conducive to understanding and analyzing text content and reducing manpower consumption; obtains target tags, associates fields in the resource data model with target tags, forms a gene map, and automatically associates them using algorithms, which is conducive to shortening the gene library construction cycle; detects and merges duplicate nodes on the gene map, and then fuses the nodes to obtain a resource gene library map, which improves the accuracy and information density of data in the gene map and helps prevent data duplication. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of the method for constructing a graph provided in an embodiment of the present application;

[0042] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0043] Figure 3 yes Figure 1 Flow chart of step S103 in FIG.

[0044] Figure 4 yes Figure 1 Flow chart of step S104 in FIG.

[0045] Figure 5 This is an optional flow chart of the graph construction method provided in the embodiment of the present application applied to a text processing system;

[0046] Figure 6 is an optional flow chart of the method for constructing a graph provided in an embodiment of the present application;

[0047] Figure 7 It is a schematic diagram of paragraph splitting provided in an embodiment of the present application;

[0048] Figure 8 It is a schematic diagram of a four-tuple solution provided in an embodiment of the present application;

[0049] Fig. 9 is a schematic diagram of a quadruple solution after adding a timing attribute provided in an embodiment of the present application;

[0050] Fig.10 It is a schematic diagram of a basic entity model provided in an embodiment of the present application;

[0051] Fig.11 It is a schematic diagram of the resource gene library map provided in the embodiment of the present application;

[0052] Fig.12 It is a structural schematic diagram of a graph construction device provided in an embodiment of the present application;

[0053] Fig.13 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0055] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0056] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0059] 1) Optical Character Recognition (OCR), which is used to convert text in images into editable and searchable data.

[0060] Cultural data are core information resources, especially cultural gene data, which are as important as biological gene data. How to understand these cultural data from the massive existing literature and the latest literature that is constantly being produced has become an extremely challenging problem.

[0061] In the related technology, the existing methods of analyzing cultural data to construct gene library maps have the problems of long construction cycle, consumption of a large amount of human resources and difficulty in ensuring the accuracy of data.

[0062] In summary, the technical problems existing in the relevant technologies need to be improved.

[0063] In view of this, a method, device, equipment and medium for constructing a graph are provided in the embodiments of the present application. The scheme obtains text data, splits the text data to obtain sentence data, reduces manual intervention through automated splitting and improves processing speed; performs semantic analysis on the sentence data to obtain a resource data model, which is conducive to understanding and analyzing text content and reducing manpower consumption; obtains target tags, associates the fields in the resource data model with the target tags, and forms a gene map, which is automatically associated with the algorithm, which is conducive to shortening the construction cycle of the gene library; detects and merges duplicate nodes on the gene map, and then fuses the nodes to obtain a resource gene library map, which improves the accuracy and information density of the data in the gene map and helps prevent data duplication.

[0064] The graph construction method provided in the embodiment of the present application relates to the field of resource big data technology. The graph construction method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms and other basic cloud computing services. The cloud server, the server can also be a node server in the blockchain network; the software can be an application that implements the graph construction method, etc., but is not limited to the above forms.

[0065] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0066] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0067] Figure 1 is an optional flow chart of the method for constructing a graph provided in an embodiment of the present application, Figure 1 The method may include but is not limited to steps S101 to S104.

[0068] Step S101, obtaining text data, and splitting the text data to obtain sentence data;

[0069] In some embodiments, keyword data is retrieved from the Internet, and resource pages are obtained through search websites; text data or resource files of the resource pages are obtained through reading technology.

[0070] Here, OCR is performed on the resource file to obtain text data.

[0071] Furthermore, the acquired text data is stored to form a text file in txt format.

[0072] The present application does not specifically limit the method of splitting text data, and can be flexibly selected in combination with actual processing needs. Optionally, text data in different languages ​​can be split by different splitting methods.

[0073] Exemplarily, the text may be split into sentences using natural language processing (NLP) technology, or the text may be split into separate sentences using punctuation marks (such as periods, question marks, exclamation marks) and context information.

[0074] Furthermore, for large-scale data, distributed or parallel computing is used for text splitting.

[0075] In this embodiment, text data is acquired and split into sentence data. The automated splitting reduces manual intervention, improves processing speed, and lays a foundation for subsequent analysis and processing.

[0076] Step S102: perform semantic analysis on the sentence data to obtain a resource data model.

[0077] In some embodiments, natural language processing technology is used to perform semantic analysis on sentence data to obtain key information and then obtain a resource data model.

[0078] Optionally, entity recognition is performed on the sentence data, and the recognized data is classified using a probabilistic method.

[0079] Among them, the probabilistic method maps the input text (the data obtained by recognition) to a separate vector. Specifically, the vector tends to be "dense" rather than "sparse" (i.e., smaller and with fewer zeros).

[0080] Furthermore, the TF IDF algorithm is used to combine word frequency and inverse document frequency to calculate text similarity.

[0081] In some embodiments, a directed acyclic graph of sentences is generated by comparing with a dictionary, and then depending on the selected mode, the sentences are truncated after finding the shortest path according to the dictionary or directly truncated, so as to clarify the verbs, nouns, and adjectives, thereby forming corresponding basic entities.

[0082] Further, a first time value is identified from the sentence data, and an entity is formed according to time, verbs, nouns, and adjectives, wherein the first time value corresponds to the verbs, nouns, and adjectives.

[0083] Exemplarily, entities include time, place, people, events, and artifacts.

[0084] Optionally, the quadruple adopts the format of <time series, entity, relationship, entity> or <time series, entity, attribute, attribute value>.

[0085] In some embodiments, when the quadruple adopts the format of <time series, entity 1, relationship, entity 2>, the relationship in the quadruple represents the relationship between entity 1 and entity 2.

[0086] The time series of the quadruple can be timestamp, duration, change rate, etc. The time series of the quadruple is used to represent the evolution of entities and relationships over time.

[0087] Optionally, the basic entity model is formed in the form of a quad.

[0088] In some embodiments, NLP technology is used to analyze and extract, and the frequency of each word in the text is counted to form a feature vector; the word frequency-inverse document frequency is calculated to evaluate the importance of the word in the text; the semantic relationship between words is captured by Word2Vec. Furthermore, corresponding entities and connections are created for the semantic relationship in the data model, and the database model is used to describe the relationship, thereby forming respective basic entity models.

[0089] Optionally, a resource data model is constructed based on the basic entity model.

[0090] In step S102 of some embodiments, the sentence data is segmented to obtain verbs, nouns and adjectives, and entities are formed according to the first time value, the verbs, nouns and adjectives, wherein the first time value is extracted from the sentence data.

[0091] Optionally, extract the attribute value of the entity from the text data; form a basic entity model in the form of a quadruple according to the entity and the attribute value of the entity; and construct a resource data model based on the basic entity model, wherein the quadruple includes a time series, an entity, an attribute of the entity, and an attribute value of the entity.

[0092] Optionally, extract the relationship between entities from the text data; form a basic entity model in the form of a quadruple according to the entities and the relationship between the entities; and construct a resource data model based on the basic entity model. The quadruple includes a time sequence, a first entity, a second entity, and a relationship between the first entity and the second entity.

[0093] In this embodiment, semantic analysis is performed on sentence data to obtain a resource data model, which is conducive to understanding and analyzing text content, reducing manpower consumption, shortening the gene library construction cycle, and improving data accuracy.

[0094] Step S103, obtaining a target tag, associating the field in the resource data model with the target tag, and forming a gene map;

[0095] Specifically, the target label includes the attribute label of the entity.

[0096] Exemplarily, the target tag may include attribute tags of entities such as time, person, event, and cultural relic.

[0097] In some embodiments, attributes of entities in the resource data model are matched and associated with target tags.

[0098] Optionally, automatic tag association can be achieved through rule matching, classification models, or deep learning models.

[0099] Furthermore, the data of the associated resource data model is organized into a graph structure to obtain a gene graph, wherein each node of the gene graph represents a data entity and the edges represent the semantic relationship between them.

[0100] In step S103 of some embodiments, the first time value of the search entity is aggregated to form a unique list, and the unique list is stored in a time queue; data fusion of the resource data model is performed according to the time queue, and the resource data model is searched according to the target time to obtain a search result; a target tag is generated according to the target time; and a target tag is added to the resource data model for the search result. Among them, data fusion includes data merging, data upper and lower relationship merging, and data attribute definition merging, and the target time belongs to the time queue.

[0101] In this embodiment, a target tag is obtained, and the fields in the resource data model are associated with the target tag to form a gene map. The algorithm is used to automatically associate the association, which is beneficial to shortening the construction cycle of the gene library, facilitating the understanding of text content, and being able to intuitively display the relationship between entities and their characteristics in the map.

[0102] Step S104, detecting and merging duplicate nodes on the gene map, and then fusing the nodes to obtain a resource gene pool map.

[0103] In some embodiments, duplicate nodes are detected on the genetic map by similarity calculation.

[0104] Optionally, the text similarity is calculated using the TF IDF algorithm in combination with the term frequency and the inverse document frequency.

[0105] Optionally, use Word2Vec for semantic-based similarity calculation.

[0106] Further, once duplicate nodes are identified, the attributes and relationships of the duplicate nodes are merged.

[0107] Optionally, an incremental learning method can be used to merge nodes and fuse attributes based on the existing gene map.

[0108] Furthermore, for complex genetic maps, the map structure can be optimized through graph databases to achieve more efficient query and update.

[0109] In step S104 of some embodiments, a gene map is subjected to a duplication detection to obtain a plurality of duplicate nodes; the weights of the duplicate nodes are calculated according to a weight calculation formula; the duplicate node with the largest weight is taken as the result node, and the attributes and association relationships of the remaining duplicate nodes are merged with the result node to obtain a resource gene pool map.

[0110] In this embodiment, duplicate nodes are detected and merged on the gene map, and then the nodes are fused to obtain a resource gene library map. By removing redundant nodes and fusing attributes, the accuracy and information density of the data in the gene map are improved, which helps to prevent data duplication.

[0111] In steps S101 to S104 shown in the embodiment of the present application, text data is acquired and split into sentence data, and manual intervention is reduced through automated splitting, thereby improving processing speed; semantic analysis is performed on the sentence data to obtain a resource data model, which is beneficial to understanding and analyzing text content and reducing manpower consumption; target tags are acquired, and fields in the resource data model are associated with target tags to form a gene map, which is automatically associated using an algorithm, thereby shortening the gene library construction cycle; duplicate nodes are detected and merged on the gene map, and then nodes are merged to obtain a resource gene library map, thereby improving the accuracy and information density of data in the gene map and preventing data duplication.

[0112] See also Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S207:

[0113] Step S201, performing word segmentation processing on sentence data to obtain verbs, nouns and adjectives.

[0114] In step S201 of some embodiments, a word segmentation tool is used to perform word segmentation processing on the sentence data.

[0115] Furthermore, on the basis of word segmentation, part-of-speech tagging is further performed to identify the part of speech (noun, verb, adjective) of each word.

[0116] Optionally, word segmentation and part-of-speech tagging are performed through a training dictionary. A directed acyclic graph of sentences is generated by comparing the dictionary, and then the sentences are intercepted after finding the shortest path according to the dictionary or the sentences are directly intercepted according to different selected modes.

[0117] It should be noted that for unregistered words (words not in the dictionary), HMM is used for new word discovery and they are sorted in descending order according to keyword weights.

[0118] Among them, verbs, nouns, and adjectives are determined from the word segmentation results.

[0119] In this embodiment, the sentence data is segmented to obtain verbs, nouns, and adjectives, which can extract useful words from the sentence and prepare for subsequent entity generation.

[0120] Step S202: Identify the first time value from the sentence data, and form entities based on the first time value, verbs, nouns, and adjectives.

[0121] In step S202 of some embodiments, after word segmentation and part-of-speech tagging, entity information such as people, events, cultural relics, locations, and times is identified through the named entity recognition (NER) method, and entities are formed based on the entity information.

[0122] Exemplarily, the entities include <YYYYMM, City J, hold a meeting, Cultural Relic A>. Among them, YYYYMM is the time (the first time value), City J is the location, hold a meeting is the event, and Cultural Relic A is the cultural relic.

[0123] Optionally, extract time-related words from the text, and construct specific entities according to the combination of time, verbs, nouns, and adjectives.

[0124] Exemplarily, nouns can represent "people", "cultural relics", or "locations", verbs can represent "events", and adjectives are used to modify nouns.

[0125] Optionally, entity recognition can be performed through a machine learning model.

[0126] In this embodiment, identifying the first time value from the sentence data and forming entities based on the first time value, verbs, nouns, and adjectives, extracting information from the sentence and forming entities is beneficial to the subsequent construction of the knowledge graph.

[0127] Step S203: Extract the attribute values of the entities from the text data.

[0128] In step S203 of some embodiments, automatic matching is performed on entities such as people and cultural relics, and the attribute values of people and cultural relics are automatically extracted from the text data by means of regular expressions.

[0129] Exemplarily, in "YY is an honest person", "YY" is the entity and "honest" is the attribute value.

[0130] Step S204: Form a basic entity model in the form of a quadruple according to the entities and the attribute values of the entities.

[0131] Specifically, the quadruple includes time sequence, entity, attribute of the entity, and attribute value of the entity.

[0132] In step S204 of some embodiments, a basic entity model is formed in the form of <time sequence, entity, attribute, attribute value> according to the entity and the attribute value of the entity. Optionally, the time sequence can be a timestamp.

[0133] Step S205: extracting the relationship between entities from the text data.

[0134] In step S205 of some embodiments, the relationship between entities is identified through technologies such as dependency parsing and semantic role labeling.

[0135] Optionally, Word2Vec can be used to map words into a high-dimensional vector space to capture the semantic relationship between words.

[0136] For example, in the text "22 attended the meeting", the "attendance" relationship between "22" and "meeting" is extracted.

[0137] Optionally, deep learning models can be used for relation extraction to identify more complex and abstract entity relationships.

[0138] Step S206: forming a basic entity model in the form of a quaternary according to the entities and the relationships between the entities.

[0139] In step S206 of some embodiments, a basic entity model is formed in the form of <time sequence, first entity, relationship, second entity> according to the entities and the relationships between the entities.

[0140] Among them, a basic entity model is formed according to the relationship between entities.

[0141] Step S207: construct a resource data model based on the basic entity model.

[0142] In some embodiments, attribute values ​​of people and cultural relics are automatically extracted from text data by means of regular expressions.

[0143] Optionally, sentence data is analyzed and extracted using NLP technology to count the frequency of occurrence of each word in the text.

[0144] In some embodiments, term frequency - inverse document frequency is calculated to assess the importance of a term in a text.

[0145] Optionally, Word2Vec is used to map words into a high-dimensional vector space to capture the semantic relationship between words.

[0146] In step S206 of some embodiments, the potential relationships between people and cultural relics are analyzed, corresponding entities and connections are created for these relationships in the data model, and database models are used to describe these relationships to form respective basic entity models, thereby constructing a resource data model.

[0147] See also Figure 3 In some embodiments, step S103 may include but is not limited to steps S301 to S305:

[0148] Step S301 : Aggregate the first time values ​​of the search entities to form a unique list, and store the unique list in a time queue.

[0149] In step S301 of some embodiments, the aggregated search of the time field (ie, the first time value) of the entity is performed to form a unique list on the time column, and the unique list is stored in a time queue.

[0150] The first time value and related entities are sorted into a list, and the time value of each entity is made unique through deduplication.

[0151] Step S302: performing data fusion of the resource data model according to the time queue.

[0152] Specifically, data fusion includes data merging, data hierarchical relationship merging, and data attribute definition merging.

[0153] It can be understood that data merging means merging the data of the resource data model itself. Data hierarchical relationship merging means merging the hierarchical relationship of data. Data attribute definition merging means merging the attributes and attribute values ​​of data.

[0154] Optionally, machine learning or data mining techniques are used to handle inconsistent data in data fusion.

[0155] Optionally, you can use the ETL (Extract, Transform, Load) process to transform the data into a unified format and then merge it.

[0156] Step S303, searching the resource data model according to the target time to obtain the search result.

[0157] Specifically, the target time belongs to the time queue.

[0158] In step S303 of some embodiments, the resource data model table is searched for records based on the time in the list, and records matching the time are retrieved (ie, search results).

[0159] Step S304: Generate a target tag according to the target time.

[0160] In step S304 of some embodiments, the target time is recorded in a tag field in the resource data model table.

[0161] Step S305: adding a target tag to the search result in the resource data model.

[0162] In step S305 of some embodiments, the updated resource data table is submitted to the database system for physical storage to form an association relationship with the target tag.

[0163] Therein, the target tag information is added to the resource data model.

[0164] Optionally, corresponding data in the resource data model is updated according to the target time point, so that the resource data has a time-related label.

[0165] It is understandable that the resource data model is not limited to time labels, but also includes labels of people, events, cultural relics, etc., and labeling operations can also be performed on resources such as people, events, and cultural relics in the resource data model.

[0166] See also Figure 4 In some embodiments, step S104 may include but is not limited to steps S401 to S403:

[0167] Step S401, performing duplication detection on the gene map to obtain multiple duplicate nodes.

[0168] In some embodiments, a gene map is subjected to duplication detection by a similarity measurement method to obtain a plurality of duplicate nodes.

[0169] A similarity threshold may be set, and nodes whose similarity exceeds the threshold are determined as duplicate nodes.

[0170] Optionally, the similarity measurement method may include, but is not limited to, a gene sequence-based alignment algorithm, a graph structure-based similarity algorithm, and a feature-based clustering algorithm.

[0171] It is understandable that the update of the gene map adopts a dynamic detection mechanism so that newly added nodes can also be detected.

[0172] Step S402, calculating the weight of the repeated node according to the weight calculation formula.

[0173] In step S402 of some embodiments, a weight calculation formula is designed for conflict detection. The weight values ​​of similar nodes are obtained according to the thickness of the nodes (sub-nodes, attributes, etc.), attribute values, association levels, etc. The weights of sub-nodes are calculated according to the weight value calculation formula of each node and the displayed attributes and association relationships.

[0174] Optionally, machine learning methods (such as support vector machines, decision trees, etc.) can be used to automatically adjust the weight calculation formula according to existing gene or node characteristics to improve prediction accuracy.

[0175] Furthermore, a weighted graph algorithm is used to process the relationship between nodes, so that the upstream and downstream relationships of the nodes are fully considered during the calculation process.

[0176] In this embodiment, the weights of duplicate nodes are calculated according to the weight calculation formula, so that the importance of each duplicate node can be quantified, which is beneficial for removing low-weight nodes in subsequent operations and avoiding interference from redundant information.

[0177] Step S403, taking the repeated node with the largest weight as the result node, and merging the attributes and association relationships of the remaining repeated nodes with the result node to obtain a resource gene pool map.

[0178] In step S403 of some embodiments, nodes with small weight values ​​are removed, and nodes with high weight values ​​are retained as main nodes. At the same time, the attributes and association relationships of nodes with small values ​​are merged with those with large values ​​to finally form a resource gene library map.

[0179] Among them, all repeated nodes are sorted according to their weight values, and the node with the highest weight is selected as the result node.

[0180] Further, the attributes and association relationships of the remaining repeated nodes are merged with the result node.

[0181] Optionally, the fusion method includes but is not limited to data fusion, data hierarchical relationship fusion, data attribute definition fusion, feature weighted fusion, graph structure fusion and graph update.

[0182] Furthermore, dynamic update technology can be used to update the resource gene library map as new data is introduced.

[0183] In this embodiment, the repeated node with the largest weight is used as the result node, and the attributes and associations of the remaining repeated nodes are fused with the result node to obtain a resource gene library map, which can effectively reduce redundant information, make the fused map more concise and highly representative, and improve the integration and accuracy of the map.

[0184] Figure 5 is an optional flow chart of the graph construction method provided in the embodiment of the present application applied to a text processing system. Figure 6 is an optional flow chart of the method for constructing a graph provided in an embodiment of the present application, Figure 6 The method may include, but is not limited to, the following steps:

[0185] Step 1: Collect and organize cultural resource data through the resource selection unit, obtain cultural resources from designated channels, and organize the resources into text data format.

[0186] In some embodiments, obtaining cultural resources and organizing the resources into a text data format includes:

[0187] 1) Search for keyword data on the Internet, such as through Baidu or other search websites, and obtain resource pages.

[0188] 2) Obtain the text content of the page or the resource file through reading technology (crawler).

[0189] 3) Perform OCR on the resource files in 2), such as PDF, Word, etc., to obtain the text content.

[0190] 4) The text contents obtained in 2) and 3) are stored to form a text file in txt format as preparation for the input data of the next unit.

[0191] Step 2: Segment the text data into units such as sentences and paragraphs through resource decomposition units.

[0192] In some embodiments, splitting the paragraphs includes:

[0193] The text file output by the resource selection unit is read, the text content is read into the memory, and the read content is separated by Chinese periods to perform sentence splitting. For example, the paragraph splitting schematic diagram is as follows: Figure 7 shown.

[0194] Step 3: The sentence outputted by the resource decomposition unit is semantically parsed by the resource identification unit to form a data model of the resource.

[0195] 1) The sentences split from the above decomposition units form a data model with sentences as the smallest unit, and the data in the data model is entity-recognized, semantically segmented, and the input text vector is classified using a probabilistic method, which maps each word to a separate vector, where the vector tends to be "dense" rather than "sparse" (i.e. smaller, with fewer zeros). During the mapping process, each word and the words around it are considered. The resulting dense vector can analyze and compare words and context. The TFIDF algorithm is used to combine word frequency and inverse document frequency to calculate text similarity.

[0196] 2) Use the established rules in the text corpus for 1) to identify the purpose of words in the whole sentence speech. The triple method is adopted in this embodiment. First, read the text data and perform word segmentation processing using the word segmentation component. The word segmentation and part-of-speech tagging are mainly carried out through the training dictionary. The same dictionary is used in this embodiment. First, generate a directed acyclic graph of the sentence by referring to the dictionary, and then according to different selected modes, intercept the sentence after finding the shortest path according to the dictionary or directly intercept the sentence. For the out-of-vocabulary words (words not in the dictionary), use HMM for new word discovery and sort them in descending order according to the keyword weights. Finally, clarify the use of verbs, nouns, and adjectives, so as to form the corresponding basic entities. According to the triple method, form entities (Ent ity), attributes (Attr ibute), and relationships (Re l ation), in the form of <entity, relationship, entity> or <entity, attribute, attribute value>. At the same time, based on the particularity of the resources, creatively add a unary, that is, time sequence, to form a quadruple scheme, that is, <time sequence, first entity, relationship, second entity> or <time sequence, entity, attribute, attribute value>. Finally, form data according to the gene data model. Among them, the schematic diagram of the quadruple scheme is as Figure 8 shown. Exemplarily, according to the entity data: <YYYYMM, City J (xxx), hold a meeting, Cultural Relic A>.

[0197] Fig. 9 shown.

[0198] Fig.10 3) Automatically match entities such as people and cultural relics in the data model in 2). Through regular expressions, automatically extract the attribute values of people and cultural relics from the text data. For structured data, the attribute values can be directly read from the corresponding fields. The text data is analyzed and extracted through NLP technology. Count the frequency of each word in the text to form a feature vector; calculate the word frequency-inverse document frequency to evaluate the importance of words in the text; use Word2Vec to map words into a high-dimensional vector space to capture the semantic relationships between words. Analyze the potential relationships between people and cultural relics, create corresponding entities and connections for these relationships in the data model, and use the database schema to describe these relationships. Thus, form their respective basic entity models. Exemplarily, the schematic diagram of the basic entity model is as Fig.10 shown.

[0199] After automatically extracting the entity names of people and cultural relics from text data, the entity data collected from multiple data sources need to be matched to determine which records refer to the same entity, and the string similarity algorithm is used to calculate the similarity between entity names. Based on the extracted entity names and attributes, a unique entity identifier is defined for each person and cultural relic.

[0200] Step 4: associate different data entries in each field of the data model through the resource label unit, configure the element label for each data, and form the final gene library.

[0201] 1) Aggregate and retrieve the time field from the data queue of the previous unit (i.e., data content formed according to a certain combination, the data queue is an unprocessed entity, such as time, person, place, era, and event, etc.) to form a unique list on the time column, and store the unique list in the time queue; realize data model layer fusion during the operation, including data merging, data hierarchical relationship merging, and data attribute definition merging.

[0202] 2) Using the time in the list of step 1 as a condition, perform a record search on the resource data model table to retrieve records that match the conditional time, and record the time in the label field in the resource data model table.

[0203] 3) Submit the updated resource data table to the database system for physical storage to form an association relationship with the tag.

[0204] 4) Loop 1)-3) to add a time tag to the tag field in each resource data model table. There may be a record without a time tag, in which case the record is ignored and can be manually labeled later.

[0205] 5) Loop through steps 1)-4), label the resources such as time, people, events, and cultural relics in the resource data model table in turn. For each main node in the gene map, it is necessary to detect and merge duplicate nodes, merge multiple attributes of each node, and detect conflicts between the nodes themselves. After detecting conflicting nodes, merge nodes and merge node attributes. A weight calculation formula is designed for conflict detection. The weight values ​​of similar nodes are obtained according to the thickness of the nodes (sub-nodes, attributes, etc.), attribute values, association levels, etc. According to the weight value calculation formula of each node and the weight of the sub-nodes of the displayed attributes and association relationships, nodes with small weight values ​​are removed, and nodes with high weight values ​​are retained as main nodes. At the same time, the attributes and association relationships of nodes with small values ​​are merged with those with large values ​​to finally form a resource gene pool map. Exemplarily, the schematic diagram of the resource gene pool map is as follows Fig.11 shown.

[0206] It is understandable that the present invention, by segmenting the disordered text content, learning and recognizing nouns based on NLP, and classifying the input text vectors by probabilistic methods, maps each word to a separate vector, wherein the vector tends to be "dense" rather than "sparse" (i.e., smaller, with fewer zeros). In the mapping process, the meaning of each word and the words around it are considered. The resulting dense vector can help to better analyze and compare words and contexts, and according to the triple theory, realize the model theory of entities, attributes, attribute values ​​or entities, relations, and entities, thereby forming entities such as time, place, person, event, and cultural relics, and forming the basic data of the gene bank.

[0207] It should be noted that the present application performs entity merging, entity hierarchical and subordinate relationship merging, and entity attribute definition merging on the implemented knowledge graph gene sequence. During the merging process, conflict detection, node merging, and node attribute fusion are implemented. A weight calculation formula is designed for conflict detection. According to the thickness of the node (child nodes, attributes, etc.), attribute value, association level, etc., the weight value of similar nodes is obtained, and finally the unrelated data is merged into a unified knowledge graph.

[0208] Among them, the present invention creatively proposes the concept of embedded timing information, namely, quadruple, which is expanded to quadruple by adding time series on the basis of the original triple, providing additional information about when the facts are established or when they occur. The quadruple is expanded by adding the time range [ts, te] to the triple, where ts and te can be the same time. This creativity effectively satisfies the most important time element in the actual scene presentation of resources.

[0209] See also Fig.12 The present application also provides a graph construction device, which can implement the above-mentioned graph construction method, and the device includes:

[0210] A text splitting module 1201 is used to obtain text data and split the text data to obtain sentence data;

[0211] Semantic analysis module 1202, used to perform semantic analysis on sentence data to obtain a resource data model;

[0212] The tag association module 1203 is used to obtain the target tag, associate the fields in the resource data model with the target tag, and form a gene map.

[0213] The merging module 1204 is used to detect and merge duplicate nodes in the gene map, and then fuse the nodes to obtain a resource gene pool map.

[0214] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0215] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned graph construction method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0216] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0217] See also Fig.13 , Fig.13 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0218] The processor 1301 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0219] The memory 1302 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1302 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1302, and the processor 1301 is called to execute the graph construction method of the embodiment of this application;

[0220] Input / output interface 1303, used to implement information input and output;

[0221] The communication interface 1304 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0222] A bus 1305 that transmits information between the various components of the device (e.g., the processor 1301, the memory 1302, the input / output interface 1303, and the communication interface 1304);

[0223] The processor 1301 , the memory 1302 , the input / output interface 1303 and the communication interface 1304 are connected to each other in communication within the device via a bus 1305 .

[0224] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned graph construction method is implemented.

[0225] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0226] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0227] The graph construction method, graph construction device, electronic device and storage medium provided in the embodiments of the present application obtain text data, split the text data to obtain sentence data, reduce manual intervention through automatic splitting and improve processing speed; perform semantic analysis on the sentence data to obtain a resource data model, which is conducive to understanding and analyzing text content and reducing manpower consumption; obtain target tags, associate fields in the resource data model with target tags to form a gene map, and use an algorithm to automatically associate them, which is conducive to shortening the construction cycle of the gene library; detect and merge duplicate nodes on the gene map, and then fuse the nodes to obtain a resource gene library map, which improves the accuracy and information density of the data in the gene map and helps prevent data duplication.

[0228] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0229] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0230] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0231] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0232] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0233] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0234] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0235] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0236] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0237] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0238] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for constructing a graph, characterized in that: The method comprises the following steps: Acquire text data, and split the text data to obtain sentence data; Performing semantic analysis on the sentence data to obtain a resource data model; Obtaining a target tag, and associating a field in the resource data model with the target tag to form a gene map; The gene map is subjected to repeated node detection and merging, and then the nodes are fused to obtain a resource gene pool map.

2. The method according to claim 1, characterized in that The semantic analysis of the sentence data to obtain a resource data model includes: Performing word segmentation processing on the sentence data to obtain verbs, nouns and adjectives; A first time value is identified from the sentence data, and an entity is formed based on the first time value, the verb, the noun, and the adjective.

3. The method according to claim 2, characterized in that The semantic analysis of the sentence data to obtain a resource data model includes: Extracting attribute values ​​of the entity from the text data; According to the entity and the attribute value of the entity, a basic entity model is formed in the form of a quaternary group, wherein the quaternary group includes a time sequence, the entity, the attribute of the entity and the attribute value of the entity; A resource data model is constructed based on the basic entity model.

4. The method according to claim 2, characterized in that: The semantic analysis of the sentence data to obtain a resource data model includes: extracting the relationship between the entities from the text data; According to the entities and the relationships between the entities, a basic entity model is formed in the form of a quaternary group, wherein the quaternary group includes a time sequence, a first entity, the relationship, and a second entity; A resource data model is constructed based on the basic entity model.

5. The method according to claim 1, characterized in that The step of obtaining the target tag and associating the field in the resource data model with the target tag to form a gene map includes: Aggregate and retrieve the first time value of the entity to form a unique list, and store the unique list in a time queue; Data fusion of the resource data model is performed according to the time queue, and the data fusion includes data merging, data upper and lower relationship merging and data attribute definition merging.

6. The method according to claim 5, characterized in that The step of obtaining the target tag and associating the field in the resource data model with the target tag to form a gene map includes: Retrieving the resource data model according to the target time to obtain a retrieval result, wherein the target time belongs to the time queue; generating a target label according to the target time; The target tag is added to the search result in the resource data model.

7. The method according to claim 1, characterized in that The detecting and merging of repeated nodes on the gene map, and then fusing the nodes to obtain a resource gene pool map, includes: Performing a repeat detection on the gene map to obtain a plurality of repeat nodes; Calculating the weight of the repeated node according to a weight calculation formula; The repeated node with the largest weight is taken as the result node, and the attributes and association relationships of other nodes and the result node are merged to obtain a resource gene pool map, wherein the other nodes are the nodes in all the repeated nodes except the result node.

8. A graph construction device, characterized in that: The device comprises: A text splitting module, used to obtain text data and split the text data to obtain sentence data; A semantic analysis module, used for performing semantic analysis on the sentence data to obtain a resource data model; The tag association module is used to obtain the target tag, associate the fields in the resource data model with the target tag, and form a gene map. The merging module is used to detect and merge duplicate nodes in the gene map, and then fuse the nodes to obtain a resource gene pool map.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images