A knowledge storage device and method for thesaurus in knowledge fusion

Abstract

A knowledge storage device and method for the thesaurus of synonyms in knowledge fusion, including a word splitting and encoding module in knowledge fusion, an improved tree storage structure, and a linked list structure in a multi-tuple storage form; the word splitting and encoding module in knowledge fusion performs word splitting, Chinese character encoding, and hash table storage on the Chinese text; the improved tree storage structure constructs a tree storage structure for the text characters and words in the scenario to complete the text tree-level index; the linked list structure in the multi-tuple storage form performs disk storage on the thesaurus of synonyms in the knowledge fusion, innovatively constructs a thesaurus of synonyms triple storage structure based on the linked list storage form, and designs a front and back pointer index. The present invention has the characteristics of high retrieval efficiency and high accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical fields of computer artificial intelligence and Internet technology, and particularly relates to a knowledge storage device and method for a synonym thesaurus in knowledge fusion. Background Art

[0002] In the field of computer artificial intelligence, in natural language processing, synonyms are often used in knowledge fusion, such as synonym entity disambiguation, synonym entity reference, etc. Most of the existing synonym thesaurus management is realized through coding technology, and the functions of intelligent storage and intelligent retrieval of the synonym thesaurus in artificial intelligence applications are not provided. The implementation methods all adopt loop iteration, resulting in some synonym thesauruses being difficult to make full use of, or the query efficiency of the synonym thesaurus being low. In the applications of artificial intelligence, natural language processing and knowledge graphs, for example, in the fields of intelligent recommendation, knowledge graphs, etc., synonyms need to be frequently and quickly extracted, and it is difficult for the system to implement related functions by using a synonym thesaurus without intelligent management technology. Summary of the Invention

[0003] In order to overcome the above technical problems, the purpose of the present invention is to provide a knowledge storage device and method for a synonym thesaurus in knowledge fusion, which can achieve fast retrieval of synonyms, quickly realize knowledge fusion, ontology alignment, and entity alignment during the construction of a knowledge graph, and can be used as a means to judge semantic similarity between entity and entity, relationship and relationship, and attribute and attribute, with the characteristics of high retrieval efficiency and high accuracy.

[0004] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0005] A knowledge storage device for a synonym thesaurus in knowledge fusion includes a word splitting and encoding module in knowledge fusion, an improved tree storage structure, and a linked list structure in a multi-tuple storage form;

[0006] The word splitting and encoding module in knowledge fusion performs word splitting, Chinese character encoding, and hash table storage on the Chinese text;

[0007] The improved tree storage structure constructs a tree storage structure for the text characters and words in the scenario to complete the text tree-level index;

[0008] The linked list structure in the multi-tuple storage form performs disk storage on the synonym thesaurus in knowledge fusion, innovatively constructs a synonym thesaurus triple storage structure based on the linked list storage form, and designs a front and back pointer index.

[0009] Furthermore, the word splitting and encoding module in knowledge fusion completes the word splitting function, Chinese character encoding process, and hash storage function for the synonym thesaurus;

[0010] The word splitting function includes: uniformly numbering the words in the Chinese text thesaurus, and sequentially completing the splitting of synonymous words and recording their positions.

[0011] The Chinese character encoding process includes encoding Chinese characters according to "National Standard Compilation of Character Sets and Information Encoding" (compiled by China Standard Press in 1998);

[0012] The hash table storage function uses hash hashing technology to determine the unique mapping address of the Chinese character encoding, and stores the Chinese character encoding and the mapping address in a continuous storage space, that is, the hash table. To avoid storage collision phenomena in the hash table, the chaining method is adopted for processing: continue to search in the singly linked list at this place until the necessary stop conditions for the program are reached. Its address field Key stores the Chinese character hash address, and the data field Value corresponding to Key stores the root node address of the tree.

[0013] Furthermore, the improved tree storage structure includes the conventional definition based on the ordinary tree and the definition of the leaf nodes of the B+ tree, where the root node address of the tree is stored in the data field Value of the hash table;

[0014] In the design of the non-leaf nodes of the tree, the storage format of the data content of each layer of nodes is formulated, that is, the non-leaf nodes do not use a unified storage format.

[0015] Furthermore, the improved tree storage structure is used to store, index, sort, and save a large amount of vocabulary data; among them, the root node (layer 0) stores a single Chinese character, the first-layer child nodes store the number of Chinese characters in the word, indicating how many Chinese characters the vocabulary consists of; the second layer is the node that stores abstract words, indicating the position of the root node in the vocabulary; the third-layer leaf nodes are designed separately, and their storage structure is designed as two fields, one is the data field, used to store the specific vocabulary of the word; the other is the pointer field, used to store the storage position of the synonym vocabulary set of the word in the doubly linked list or doubly circular linked list.

[0016] Furthermore, the linked list structure of the tuple storage form is a doubly linked list structure, and the nodes of the linked list are divided into three fields, including: pointer field 1, data field, pointer field 2, where the data field in the doubly linked list structure is used to save a specific vocabulary, and when this word is stored in data, it adopts the form of a triple;

[0017] The two pointer fields of the doubly linked list respectively save the storage addresses of a forward synonym node and a backward synonym node of the word, where the predecessor address of the head node of the linked list is stored as empty, and the successor address of the tail node of the linked list is stored as empty.

[0018] The storage structure of the data domain part in the doubly linked list is in the form of triples, and its storage structure is (s, a, f), with the search word v as input, where s represents the synonym of the search word v, a represents the storage code (storage address) of the synonym s of the search word v, and f represents the number of searches or retrievals between the search word and the synonym s.

[0019] A method for using a knowledge storage device for a synonym dictionary in knowledge fusion, comprising the following steps:

[0020] Synonym dictionary storage construction process;

[0021] (1) The word splitting function realizes word splitting of the original document; all Chinese characters are uniformly encoded according to the Chinese character encoding part, wherein the encoding is based on the "National Standard Compilation of Character Sets and Information Coding"; taking the Chinese character "人" as an example, it is assumed to be represented as: key = 00100;

[0022] According to the formula Address=H(key), the hash storage function hash address calculation is implemented, and the Chinese character is stored in the corresponding storage space. The hash address Address of the Chinese character is stored in the corresponding sequence, and the Chinese character code key is stored in the corresponding sequence. Taking the Chinese character "人" as an example, ki stores the hash address H(00100) of the Chinese character "人", and vi stores the Chinese character code value 00100;

[0023] (2) After completing the storage of the above key sequence, enter the improved tree storage structure. Each Chinese character in the key sequence is used as the root node of a tree. Several trees are constructed. The root node (the 0th layer of the tree) represents the Chinese character "人". The data domain of the node is H (00100);

[0024] The first-level nodes represent how many Chinese characters a word is made up of (e.g., 1 character, 2 characters, 3 characters, etc.);

[0025] The second-level nodes represent the abstract structures that make up Chinese words. The root node is the Chinese character "人". For example, there are two abstract structures: "人*" and "*人". "人*" means that the word is composed of two Chinese characters, with "人" in the first position, such as: "humanity", "people", "people", etc.; "*人" means that the word is composed of two Chinese characters, with "人" in the second position, such as: "老人", "好人", "美人", etc.

[0026] The third-level nodes represent leaf nodes of the tree. There are two storage methods. One is the storage method of "character", which has two storage spaces A1 and A2. A1 is the address of "character" in the doubly linked list, and A2 is the memory address of the word "character". The other is that the node information only contains the address A1 in the doubly linked list, and its memory address is not stored in the node 507, but in the node 509.

[0027] (3) Taking the word "person" as an example, in the above, A1 represents the address of a node in a doubly linked list, and 602 represents the node in the linked list where the word that is a synonym of "person" is located. Taking "role" as an example, the format of 602 can be expressed as: <pointer field 1, (role, address of "role", number of synonym lookups) pointer field 2>.

[0028] Furthermore, taking the encoding - hash storage of "person", the root node of the improved tree - shaped storage structure storing "person", "2" in the tree, "person*" in the tree, "person" in the tree, and 602 in the doubly linked list as an example, this storage process is described.

[0029] Advantages of the present invention.

[0030] This method can be used in the construction of intelligent recommendation and information retrieval systems. If used in an information retrieval system, users can quickly and effectively query for synonyms, near - synonyms, and antonyms of a certain word by inputting the word. If applied to a recommendation system, it can quickly and effectively recommend products of the same type for users to choose according to the user's search content.

[0031] Using this storage device can quickly find the required number of synonyms in synonym retrieval; it can be extended to near - synonyms, and even use this storage method to store antonyms, thereby completing the construction of an antonym thesaurus, having the effects of being extensible and migratable. Constructing the storage of the synonym thesaurus in knowledge fusion is also convenient for achieving entity alignment, knowledge fusion, and semantic similarity judgment in the construction of a knowledge graph. Brief Description of the Drawings

[0032] Figure 1 It is a schematic structural diagram of a knowledge storage method and device for the hardware operating environment involved in the embodiment solution of the present invention;

[0033] Figure 2 It is a schematic flowchart of the knowledge storage in the present invention during the construction storage process;

[0034] Figure 3 It is a functional diagram of the knowledge storage of the present invention;

[0035] Figure 4 It is a schematic diagram of Chinese character hash storage in the knowledge storage of the present invention;

[0036] Figure 5 It is a schematic diagram of the improved tree - shaped storage structure in the knowledge storage of the present invention;

[0037] Figure 6 It is a schematic diagram of the triple doubly linked list storage structure in the knowledge storage of the present invention. Detailed Embodiments

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The present invention provides a knowledge storage method and device for synonymous word forests in knowledge fusion, to address the problems of insufficient word forest storage and insufficient retrieval technology in existing synonym word forest storage technologies. The present invention provides a knowledge storage method and device for synonymous word forests in knowledge fusion, and the technical solution is as follows:

[0040] A knowledge storage method and device for synonymous word forests in knowledge fusion, including: a word splitting and encoding module in knowledge fusion, an improved tree storage structure, and a linked list in the form of multi-tuple storage.

[0041] The above functions can store and retrieve synonyms used for knowledge fusion in natural language processing and knowledge graphs.

[0042] The device has the function of searching for synonyms in knowledge fusion, and uses the same steps to search for corresponding synonyms or synonym sequences according to the above storage process, and displays the search results in the form of tuple data.

[0043] Taking the word "老" as an example, the process of searching for synonyms is described. According to the code, the code value key2 of the Chinese character "老" is obtained, and then the hash address of the Chinese character "老" is obtained using Address2=H(key2). According to the address in the sequence where 401 is located in the hash table, v2 is found. According to v2, a tree with "老" as the root node is found, such as Figure 5 As shown in . Since the word to be searched is composed of two Chinese characters, and its abstract structure is "老*", it will be traversed to Figure 5 At node 509, the position of the doubly linked list of synonyms found in A1 in node 509 is used to traverse the doubly linked list to find all synonyms of the searched word "老老." Finally, the triples in the list containing the synonyms are output. For example, (老翁, address, 5) indicates that the synonym of the searched word "老老" is "老翁," the storage address of the synonym "老翁" is address, and the synonym "老翁" of "老翁" has been searched five times. Due to the characteristics of the doubly linked list, a bidirectional search for triples can be supported starting from the current node, outputting the required number of synonym triples.

[0044] like Figure 1As shown in the figure. Among them, 1001 represents the user who uses this method and device; 1002 is the user interface for the internal interaction between the user and the computer; 1003 and 1004 respectively represent devices such as computer processors, processing services, or plugins; 1005 represents the network interface installed on the computer; 1012 represents the internal storage device of the system, which can generally be understood as a memory. 1002 includes: 1006 operating system OS module; 1007 system communication module; 1008 business service module, which can be programmatically operated for different services; 1009 represents the knowledge storage structured program of this patent, which is the key to innovative applications; 1010 and 1011 represent the registers and cache devices in the storage.

[0045] As Figure 1 shown: After the user 1001 initiates an instruction to query synonyms, it enters 1003 and 1004 through the user interface 1002, converts the query action into a network instruction, and enters the memory through 1005. Through the internal operating system 1006 and related communication modules, business services, etc. cooperate, and the query instruction enters the structured program 1009. According to the word to be searched, it enters the word forest storage devices 1010 and 1011, then retrieves the synonyms, and returns them to the user 1001 according to the request route. After entering 1009, its execution process is as Figure 2 shown.

[0046] As Figure 2 shown. Among them, 201 represents three modules of word splitting, encoding, and hash storage in knowledge fusion; 205 represents an improved tree storage structure process; 206 represents a linked list structure of a multi-tuple storage form. Through the design of knowledge storage in this process, this process has the functions of adding, deleting, modifying, and querying knowledge storage for the synonym thesaurus in knowledge fusion.

[0047] As Figure 3 shown, where 301 represents the knowledge storage structure of this patent, and the functions designed based on this structure include: 302 adding words, 303 deleting words, 304 modifying words, and 305 word retrieval function.

[0048] Figure 2 In the operation process of 201, it includes 202 word splitting function, 202 Chinese character encoding function, and 203 hash storage function. The specific implementation steps of this operation process are as follows:

[0049] Step 1: In the word splitting module, based on the Chinese sentence, complete word splitting, and at the same time record the positions after splitting;

[0050] Step 2: In the Chinese character encoding module, based on Step 1, complete Chinese character encoding; the encoding method is not unique and not limited, and the GB2312-80 character set is used as an example for description.

[0051] Step 3: Save the Chinese character code into the hash table; use the hash algorithm H(key) (as shown in the following formula 1) to calculate the storage location of the Chinese character in the hash table, and save the Chinese character and its storage address into the corresponding position of the hash table. Figure 4 As shown, 401 represents the storage location sequence of H(key), and the storage sequence where 402 is located is the storage address information of the specific Chinese character, and is also the address information of the root node of the tree in step 4.

[0052] Address=H(key) Formula 1

[0053] When calculating the hash address, a one-way cryptographic system is used. The key represents the Chinese character code. The storage address of the character is calculated from the Chinese character plain text and stored in the hash table. Suppose the number of duplicate Chinese characters contained in the Chinese synonyms dictionary is N, then in the storage space, a space of size M is opened up, which can satisfy M>N or M≥N; the calculation is performed according to H(key)=key%p, where p satisfies p≤M. In addition, in the above calculation process, the key information is not limited to the GB2312-80 encoding character set, and the value of p is also not limited to a specific number. However, it is recommended that when using this patent, the value of p is a number that is closest to and smaller than the prime number M.

[0054] Figure 2 205 represents an improved tree storage structure, which is specifically implemented as shown in steps 4 and 5 below.

[0055] Step 4: If Figure 5 As shown in FIG, it is the design of the improved tree storage structure; the 0th level of the Chinese character tree is the root node, which stores a single Chinese character, and its address is stored in the hash table, that is, Figure 4 The position corresponding to the storage sequence where 402 is located.

[0056] The first layer stores the number of Chinese characters in the Chinese character synonym vocabulary, that is, the number of Chinese characters that the vocabulary is composed of. Generally, the value range is 1 to 4; Figure 5 502 represents a tree node whose value is 1 Chinese character, and 503 represents a tree node composed of other numbers of Chinese characters.

[0057] The second layer stores the nodes of abstract words in the Chinese character synonym vocabulary, such as Figure 5 As shown, taking the character "人" as the root node, 501 represents the character "人" at the 0th level; the Chinese character number nodes of the first level are 1, 2, ...; the child node of the first level 1 node is the character "人", as shown in 504; the word nodes of the first level 2 nodes are "人*" and "*人"; the word nodes of the first level 3 nodes are "人**", "*人*", "**人"; and so on. The structure of the Chinese character tree is defined as follows:

[0058]

[0059] Step 5: The third layer of the Chinese character tree is the leaf node. As shown in Figure 5 Figure 5 such as 505 shown therein, its storage structure is one of the innovative design points of this patent; this node is divided into two categories. The first category is the storage address node of the word itself, such as the node "task" where 505 is located. The second category is the index address of the word, as shown in 507. This node only stores its position information in the Chinese character tree. For example, 507 stores the position information of 509, where 508 represents the mapping relationship and has no specific physical pointer.

[0060] The leaf node of the Chinese character tree includes two field values, A1 and A2 as shown in 506. A1 represents the pointer address of the doubly linked list in the knowledge storage of the Thesaurus, that is, Figure 6 the address of 601 in Figure 6 . A2 represents the memory address of the stored word. Only the leaf nodes of the words that meet the above first category have the A2 address. Its storage structure is as follows:

[0061]

[0062] Figure 2 In Figure 2 , 206 represents the linked list structure processing flow of the tuple storage form, and the corresponding storage form includes two address fields and one data field. Its specific implementation is as follows in Step 6.

[0063] Step 6: Design of the doubly linked list and the triple knowledge storage structure; in the linked list of the leaf nodes of the improved tree-shaped storage structure, the nodes of the linked list are divided into three fields, including: pointer field 1, data field, and pointer field 2. Pointer field 1 and pointer field 2 point to the predecessor and successor nodes of the node respectively. That is, pointer field 1 and pointer field 2 store the storage addresses of the predecessor or successor nodes of the current node respectively.

[0064] In the leaf linked list of the improved tree-shaped storage structure, the data field part is mainly to design a storage structure in the form of a triple (s, a, f). Taking the search word v as the input, where s represents the synonym word of the word v to be searched, a represents the storage code (storage address) of the synonym s of the word v to be searched, and f represents the number of times of searching or retrieving between the word to be searched and the synonym s.

[0065] This device has the retrieval function of the Thesaurus in the face of knowledge fusion and displays the retrieval results in the form of tuple data.

[0066] When the user inputs the word for which synonyms are to be queried, the knowledge storage method first determines the Chinese character list and the hash address, then locates the corresponding word search tree. Based on the tree where the root node of the search is located, and the abstract structure of the word to be searched (abstract structures such as X*, *X, X**, **X, etc.), it retrieves the address field stored in the leaf node of the word search tree. It maps to the doubly linked list through the retrieved address field and directly locates the position of the current word to be searched, and then obtains the number of synonyms according to the parameter requirement of the number of requested synonyms. If the default value is adopted, all synonyms of the word will be output. The method for outputting the synonyms of the word to be searched is to traverse according to the position of the current word to be searched, making use of the characteristics of the pointer fields of the doubly linked list. One points to the address field of the synonym in front of the word, and the other points to the address field of the synonym behind the word; in addition, the head node and the last tail node of the doubly linked list are checked. It retrieves in both directions to obtain the required number of relevant synonyms, or all synonyms.

Claims

1. A knowledge storage device for thesaurus in knowledge fusion, characterized in that, It includes a word splitting and encoding module in knowledge fusion, an improved tree storage structure, and a linked list structure in the form of a multi-tuple storage; The linked list structure in the form of a multi-tuple storage stores the thesaurus of synonyms in knowledge fusion on disk. Based on the linked list storage form, a thesaurus of synonyms triple storage structure is innovatively constructed, and a front and back pointer index is designed; The word splitting and encoding module in knowledge fusion completes the word splitting function, Chinese character encoding process, and hash table storage function for the thesaurus of synonyms; The word splitting function includes: uniformly numbering the vocabulary in the Chinese text thesaurus, and sequentially completing the splitting of synonyms and recording their positions; The Chinese character encoding process includes encoding Chinese characters according to the "National Standard Compilation of Character Sets and Information Encoding"; The hash table storage function uses hash hashing technology to determine the unique mapping address of the Chinese character encoding, and stores the Chinese character encoding and the mapping address in a continuous storage space, that is, the hash table. To avoid storage collision phenomena in the hash table, the open addressing method is used for processing: by continuing to search in the singly linked list of the hash table until the necessary stop condition for the program is reached; its address field Key stores the Chinese character hash address, and the data field Value corresponding to Key stores the root node address of the tree; The improved tree storage structure includes the conventional definition based on the ordinary tree and the definition of the leaf node of the B+ tree, where the root node address of the tree is stored in the data field Value of the hash table; In the design of the non-leaf nodes of the tree, the storage format of the data content of each layer of nodes is formulated, that is, the non-leaf nodes do not use a unified storage format; The improved tree storage structure is used to store, index, sort, and save a large amount of vocabulary data; among them, the root node (layer 0) stores a single Chinese character, the first-layer child nodes store the number of characters of the word, indicating how many Chinese characters the vocabulary consists of; the second layer is the node that stores abstract words, indicating the position of the root node in the vocabulary; the third layer of leaf nodes is designed separately, and its storage structure is designed as two fields. One is the data field, which is used to store the specific vocabulary of the word; the other is the pointer field, which is used to store the storage position of the synonym vocabulary set of the word in the doubly linked list or doubly circular linked list.

2. The knowledge storage device for a synonym thesaurus in knowledge fusion according to claim 1, wherein, The linked list structure in the form of a multi-tuple storage is a doubly linked list structure. The nodes of the linked list are divided into three fields, including: pointer field 1, data field, and pointer field 2. In the doubly linked list structure, the data field is used to save a specific vocabulary. When the word is stored in data, it adopts the form of a triple; The two pointer fields of the doubly linked list respectively save the storage addresses of a forward synonym node and a backward synonym node of the word. Among them, the predecessor address of the head node of the linked list is stored as empty, and the successor address of the tail node of the linked list is stored as empty.

3. The knowledge storage device for thesaurus in knowledge fusion according to claim 2, wherein The storage structure of the data field part in the doubly linked list uses a triple as the storage form, and its storage structure is (s, a, f). Taking the search word v as the input, where s represents the synonym word of the word v to be searched, a represents the storage encoding of the synonym s of the word v to be searched, and f represents the number of times of searching or retrieving between the word v to be searched and the synonym s; 4. A method for using a knowledge storage device for a thesaurus in knowledge fusion according to any one of claims 1-3, characterized in that, It includes the following steps; The word splitting function realizes the word splitting of the original text document; all Chinese characters are uniformly encoded according to the Chinese character encoding part, where the encoding is based on the "National Standard Compilation of Character Sets and Information Encodings"; the assumption is expressed as: key = 00100; According to the formula Address = H(key), the hash storage function hash address calculation is realized, and the Chinese characters are stored in the corresponding storage space. The hash address Address of the Chinese characters is stored in the sequence where 401 is located, and the Chinese character encoding key is stored in the sequence where 402 is located; After completing the storage of the above key sequence, enter the improved tree storage structure. Each Chinese character in the key sequence serves as the root node of a tree, and several trees are constructed. The data field of the node is H(00100); The first-layer node indicates how many Chinese characters the word is composed of; The second-layer node indicates the abstract structure of the Chinese character word; The third-layer node represents the leaf node of the tree, and there are two storage methods. One is that there are two storage spaces A1 and A2. A1 is the address of the node in the doubly linked list, and A2 is the memory address. The other is that the node information only includes the address A1 in the doubly linked list, and its memory address is not stored in this node 507, but stored in the 509 node.

Images