Method and apparatus for constructing generative neural network tree structure, and storage medium

Abstract

Provided in the present invention is a method for constructing a generative neural network tree structure. The method comprises the following method steps: S1, acquiring a word sequence of a plurality of natural language segments, and generating a word vector of each word of each natural language segment; S2, training a generative neural network with a word vector of each word of a natural language segment, and taking the trained generative neural network as a path of the natural language segment; and S3, repeating the step S2 to generate a plurality of paths corresponding to the plurality of natural language segments, selecting word vectors of the same word from all the paths as common nodes, so as to construct a tree structure. In the present invention, dynamic natural language sequences are used for training corresponding paths, and the paths are added to a generative neural network tree structure, such that the memory of subsequent corresponding paths is enhanced, which makes the subsequent corresponding paths have the advantages of the tree structure gradually, thereby greatly improving the effects of the memory and search capabilities of a generative neural network.

Description

A method, device and storage medium for constructing a generative neural network tree structure Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device and storage medium for constructing a generative neural network tree structure. Background Art

[0002] The tree structure is a classic structure in traditional data structures. It has good compression and information storage capabilities. The specific advantages are as follows:

[0003] 1. Hierarchical organization: The tree structure is a hierarchical data structure that can naturally represent various hierarchical relationships, which is very useful for organizing and managing data.

[0004] 2. Efficient Data Access: Tree structures offer high search and access efficiency. Due to the hierarchical nature of tree structures, operations such as search, insertion, and deletion typically have a time complexity of O(logn), where n is the number of nodes in the tree. This makes tree structures well-suited for efficient search and retrieval operations.

[0005] 3. Sorting and range queries: In some tree structures, such as binary search trees, the order of nodes can be used to achieve efficient sorting. In addition, tree structures can also support range queries, that is, finding nodes that meet a certain range of conditions.

[0006] 4. Recursive nature: Tree structures have a natural recursive nature, which allows the use of recursive algorithms when processing tree structures. This simplifies problem solving in many cases. For example, tree traversal operations can be implemented recursively.

[0007] 5. Storing hierarchical relationships: Tree structures can effectively store hierarchical relationships, such as organizational structures, family tree relationships, etc. This makes tree structures highly practical in practical applications and can accurately reflect various hierarchies and relationships in the real world.

[0008] 6. Balance: Some tree structures, such as balanced binary search trees (such as AVL trees and red-black trees), have the ability to automatically adjust to maintain the tree's balance. This helps prevent the tree structure from degenerating into a linked list, thereby maintaining efficient operational performance.

[0009] 7. Graph Simplification: A tree can be viewed as a special type of graph that has no cycles or loops. In some cases, trees can be used to simplify graph problems, converting complex graph structures into more manageable tree structures.

[0010] Most existing generative neural networks have weak memory capabilities. Current approaches to memory management based on large models primarily combine external storage with the neural network structure so that the model can read and write information while performing tasks. While this approach has some application effects, it has the following drawbacks:

[0011] (1) Storage and computational costs: Existing networks with certain memory capabilities generally need to maintain external storage, which may lead to increased storage costs and computational complexity, limiting the scalability of the network.

[0012] (2) Training Difficulty: Training a generative neural network with memory can be relatively complex. Effective methods need to be designed to initialize storage, determine attention weights, and balance write and read operations during training. Different tasks may require different storage and attention configurations, which increases the difficulty of training and parameter adjustment.

[0013] (3) Limited memory capacity: Generative networks with memory often have limited external storage capacity, which may limit the model’s ability to process long texts or large amounts of information. Memory may not perform well when the task requires long-term dependencies that exceed the storage capacity.

[0014] Summary of the Invention

[0015] The present invention provides a method, device and storage medium for constructing a generative neural network tree structure to solve the technical problems of high storage and computing costs, difficult training and limited memory capacity of generative neural networks in the prior art.

[0016] One aspect of the present invention is to provide a method for constructing a generative neural network tree structure, characterized in that the construction method includes the following steps:

[0017] S1. Obtain multiple natural language word sequences and generate word vectors for each word in each natural language sequence.

[0018] S2. Train a generative neural network using the word vector of each word in a natural language segment, and use the trained generative neural network as a path for the natural language segment;

[0019] S3. Repeat step S2 to generate multiple paths corresponding to multiple segments of natural language, select the word vectors of the same words in all paths as common nodes, and build a tree structure.

[0020] In a preferred embodiment, the construction method further comprises the following steps:

[0021] S4. When a natural language path needs to be added to the tree structure, a word sequence of the natural language to be added is obtained, and a word vector for each word of the natural language to be added is generated;

[0022] S5. Repeat step S2 to generate a natural language path that needs to be added, select the word vectors of the same words in the natural language path that needs to be added as those in the tree structure as common nodes, and add the natural language path that needs to be added to the tree structure.

[0023] In a preferred embodiment, in step S1, a unique and non-repeating ID number is assigned to each word in each natural language segment to obtain a word sequence of the natural language;

[0024] For each word ID number, randomly generate an m-bit code and generate a word vector for each word.

[0025] In a preferred embodiment, in step S2, the word vector of each word is input into the generative neural network in sequence according to the ID number of each word in the natural language, and the predicted word vector of each word corresponding to the word is output to train the generative neural network.

[0026] Another aspect of the present invention is to provide a device for constructing a generative neural network tree structure, the device comprising:

[0027] The word vector generation module is used to obtain the word sequence of natural language and generate the word vector of each word in each natural language segment;

[0028] A generative neural network training module is used to train a generative neural network using the word vector of each word in a natural language segment, and use the trained generative neural network as a path for the natural language segment;

[0029] The tree structure construction module is used to select the word vectors of the same words in all paths as common nodes to build a tree structure.

[0030] In a preferred embodiment, the construction device further comprises:

[0031] The path adding module is used to select the word vectors of the same words in the natural language path to be added as the common nodes, and add the natural language path to be added to the tree structure.

[0032] In a preferred embodiment, the word vector generation module assigns a unique and non-repeating ID number to each word in each natural language segment to obtain a word sequence of the natural language;

[0033] For each word ID number, randomly generate an m-bit code and generate a word vector for each word.

[0034] In a preferred embodiment, the generative neural network training module inputs the word vector of each word into the generative neural network in sequence according to the ID number of each word in the natural language, outputs the predicted word vector of each word corresponding to the word, and trains the generative neural network.

[0035] Another aspect of the present invention is to provide a storage medium, which is used to store computer-executable instructions, and the computer-executable instructions are used to execute a method for constructing a generative neural network tree structure provided by the present invention.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention provides a method, device and storage medium for constructing a generative neural network tree structure. By training a generative neural network, the trained generative neural network is used as a path for the natural language segment to dynamically construct the generative neural network tree structure.

[0038] The present invention provides a method, device, and storage medium for constructing a generative neural network tree structure. The natural language path (generative neural network) is trained through dynamic natural language sequences rather than manually specified training paths. Compared with traditional neural network tree structures, the entire process is unsupervised and does not require updating all parameters of the entire tree result. Only the corresponding path (generative neural network) needs to be trained according to the needs of the natural language.

[0039] The present invention provides a method, device and storage medium for constructing a generative neural network tree structure. The construction process of the generative neural network tree structure does not require supervision of the entire tree structure. It only needs to use dynamic natural language sequences to train the corresponding paths (generative neural networks) and add them to the generative neural network tree structure to enhance the memory of subsequent corresponding paths (generative neural networks). The subsequent corresponding paths (generative neural networks) gradually possess the advantages of the tree structure, which greatly improves the memory and search capabilities of the generative neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] FIG1 is a flow chart of a method for constructing a generative neural network tree structure according to the present invention.

[0042] FIG2 is a schematic diagram of a training process of a natural language path in one embodiment of the present invention.

[0043] FIG3 is a schematic diagram of a structured neural network tree constructed in one embodiment of the present invention.

[0044] FIG4 is a schematic diagram of adding a path to a formulated neural network tree structure according to an embodiment of the present invention.

[0045] FIG5 is a structural block diagram of a device for constructing a generative neural network tree structure according to the present invention. DETAILED DESCRIPTION

[0046] In order to make the above and other features and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.

[0047] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0048] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0049] As shown in FIG1 , according to an embodiment of the present invention, a method for constructing a generative neural network tree structure is provided, comprising the following steps:

[0050] Step S1: Obtain word sequences of multiple natural language segments and generate word vectors for each word in each natural language segment.

[0051] In this embodiment, two natural language segments are taken as an example, such as the two natural language segments given in the natural language list shown in Figure 1.

[0052] Table 1 Natural Language List

[0053] As shown in Table 1, in this embodiment, the two natural languages of the natural language [Go to the park today.] and the natural language [What's the weather like today?] are taken as examples.

[0054] For each word in each natural language, assign a unique and non-repeating ID number, obtain the word sequence of the natural language, and for the ID number of each word, randomly generate an m-bit code to generate the word vector of each word.

[0055] Taking the natural language [Go to the park today.] as an example, it is segmented into 6 words: [Today, go, to, the, park,.] and a unique and non-repeating ID number is assigned to each word.

[0056] For example, "Today" is assigned the ID number 1, "go" is assigned the ID number 2, "to" is assigned the ID number 3, "the" is assigned the ID number 4, "park" is assigned the ID number 5, and "." is assigned the ID number 6, and the natural language sequence [1, 2, 3, 4, 5, 6] of the natural language [Go to the park today.] is obtained.

[0057] For the ID number of each word, randomly generate an m-bit code. For example, for the ID number of each word, randomly generate a 4-bit code, then the word "Today" with the ID number 1 generates a 4-bit code word vector (Embedding): x 15 ,

[0059] , 13 , 11 ,

[0058] ,

[0062] ,

[0061] ,

[0060] , 14 , 12 , , , , , , the word vector x 11 (Embedding) has four elements. For example, x 11 = [0.2, 0.3, 0.4, 0.5], and the four elements of the word vector x 11 are randomly generated.

[0058] Similarly, the word "go" with the ID number 2 generates a 4-bit code word vector (Embedding): x 12 ;

[0059] Similarly, the word "to" with the ID number 3 generates a 4-bit code word vector (Embedding): x 13 ;

[0060] Similarly, the word "the" with the ID number 4 generates a 4-bit code word vector (Embedding): x 14 ;​​​​​​​Similarly, the word "." with ID number 6 generates a 4-bit encoded word vector (Embedding): x 16 , and add the sentence terminator <eos>。

[0063] Perform the same processing on the natural language "[What's the weather like today?]" to generate word vectors (Embeddings) for each word.

[0064] It should be understood that for the sake of brief description, only two paragraphs of natural language are given in this embodiment. In actual applications, there should be more paragraphs of natural language.

[0065] Step S2: Use the word vectors of each word in a paragraph of natural language to train the generative neural network, and use the trained generative neural network as a path for this paragraph of natural language.

[0066] Specifically, according to the ID numbers of each word in the natural language, input the word vectors of each word into the generative neural network in sequence, output the predicted word vectors of the corresponding words for each word, and train the generative neural network.

[0067] As shown in Figure 2, taking the natural language "[Go to the park today.]" as an example, the word vector (Embedding) x of the word "今" with ID number 1 11 , the word vector (Embedding) x of the word "天" with ID number 2 12 , the word vector (Embedding) x of the word "去" with ID number 3 13 , the word vector (Embedding) x of the word "公" with ID number 4 14 , the word vector (Embedding) x of the word "园" with ID number 5 15 , the word vector (Embedding): x of the word "。" with ID number 6 16 , are input into each layer of the generative neural network in the order of the ID numbers (ID number 1, ID number 2, ID number 3, ID number 4, ID number 5, ID number 6) to train the generative neural network. [[ID=...]]

[0068] Specifically as shown in Figure 2, the word vector (Embedding) x of the word "今" with ID number 1 11 , is input into the first layer of the generative neural network, and the predicted word vector y of the word "今" corresponding to the word "天" is output 12 .

[0069] The word vector (Embedding) x of the word "天" with ID number 2 12 , is input into the second layer of the generative neural network, and the predicted word vector y of the word "天" corresponding to the word "去" is output 13 .

[0070] The word vector (Embedding) x of the word "go" with ID number 3 13 , is input into the third layer of the generative neural network, and the predicted word vector y corresponding to the word "go" and the word "public" is output 14 .

[0071] The word vector (Embedding) x of the word "public" with ID number 4 14 , is input into the fourth layer of the generative neural network, and the predicted word vector y corresponding to the word "public" and the word "garden" is output 15 .

[0072] The word vector (Embedding) x of the word "garden" with ID number 5 15 , is input into the fifth layer of the generative neural network, and the predicted word vector y corresponding to the word "garden" and the word "." is output 16 .

[0073] The word vector (Embedding) x of the word "." with ID number 6 16 , is input into the sixth layer of the generative neural network, and the sentence end symbol added to the word "." is output <eos>.

[0074] The generative neural network is trained using the above method, and the trained generative neural network is used as a path for the natural language [Let’s go to the park today.].

[0075] Step S3: Repeat step S2 to generate multiple paths corresponding to multiple segments of natural language, select the word vectors of the same words in all paths as common nodes, and build a tree structure.

[0076] Similarly, according to the method of step S2, the natural language [How is the weather today?] is processed in the same way, and the trained generative neural network is used as a path of the natural language [How is the weather today?].

[0077] The word vectors of the same words in all paths are selected as common nodes to construct a tree structure. As shown in Figure 3, for example, in the path corresponding to the natural language "Let's go to the park today," and the path corresponding to the natural language "What's the weather like today?", the word vector of the same word "day" is selected as a common node to construct a tree structure. Each path in the constructed tree structure is a generative neural network trained using the corresponding natural language.

[0078] According to an embodiment of the present invention, a method for constructing a generative neural network tree structure further includes the following method steps:

[0079] Step S4: When it is necessary to add a natural language path to the tree structure, obtain the word sequence of the natural language to be added, and generate a word vector for each word of the natural language to be added.

[0080] When it is necessary to add a natural language path (generative neural network) to the already constructed tree structure, the word sequence of the natural language to be added is obtained according to the method of step S1, and a word vector for each word of the natural language to be added is generated.

[0081] For example, in this embodiment, if a path (generative neural network) for the natural language "The weather is great today!" needs to be added to the already constructed tree structure, the method of step S1 is used to obtain the word sequence of the natural language "The weather is great today!" to be added, and a word vector for each word of the natural language "The weather is great today!" is generated.

[0082] Step S5: Repeat step S2 to generate the natural language path that needs to be added, select the word vectors of the same words in the natural language path that needs to be added as those in the tree structure as common nodes, and add the natural language path that needs to be added to the tree structure.

[0083] For example, in this embodiment, according to the method of step S2, the word vectors of each word in the natural language [What a nice day today!] to be added are used to train the generative neural network, and the trained generative neural network is used as a path of the natural language [What a nice day today!] to be added.

[0084] Select the word vector of the same word "qi" in the path of the natural language [What a nice day today!] to be added as the common node in the already constructed tree structure, and add the path of the natural language [What a nice day today!] to be added to the tree structure, as shown in Figure 4.

[0085] The present invention uses a dynamic natural language sequence to train the corresponding path (generative neural network) and adds it to the generative neural network tree structure, enhancing the memory of the subsequent corresponding path (generative neural network), enabling the subsequent corresponding path (generative neural network) to gradually possess the advantages of the tree structure, and greatly improving the memory ability and search ability of the generative neural network.

[0086] As shown in Figure 5, according to an embodiment of the present invention, there is provided a device for constructing a generative neural network tree structure, including:

[0087] A word vector generation module 100, configured to obtain a word sequence of natural language and generate word vectors of each word in each segment of natural language.

[0088] Specifically, the word vector generation module 100 assigns a unique and non-repeating ID number to each word in each segment of natural language, obtains the word sequence of natural language, and randomly generates an m-bit code for the ID number of each word to generate the word vector of each word.

[0089] A generative neural network training module 200, configured to train the generative neural network with the word vectors of each word in a segment of natural language, and use the trained generative neural network as a path of this segment of natural language.

[0090] The generative neural network training module 200 inputs the word vectors of each word into the generative neural network in sequence according to the ID number of each word in the natural language, outputs the predicted word vector of the word corresponding to each word, and trains the generative neural network.

[0091] A tree structure construction module 300, configured to select the word vectors of the same word as the common node among all paths and construct a tree structure.

[0092] A path addition module 400, configured to select the word vector of the same word in the path of the natural language to be added as the common node in the tree structure, and add the path of the natural language to be added to the tree structure.

[0093] According to an embodiment of the present invention, a storage medium is provided for storing computer-executable instructions, and the computer-executable instructions are used to execute a method for constructing a generative neural network tree structure provided by the present invention.

[0094] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.< / eos> < / eos>

Claims

1. A method for constructing a generative neural network tree structure, characterized in that: The construction method comprises the following steps: S1. Obtain word sequences of multiple natural languages ​​and generate word vectors for each word in each natural language. S2. Train a generative neural network using the word vector of each word in a natural language, and use the trained generative neural network as a path of the natural language; S3. Repeat step S2 to generate multiple paths corresponding to multiple segments of natural language, select word vectors of the same words in all paths as common nodes, and build a tree structure.

2. The construction method according to claim 1, characterized in that: The construction method also includes the following steps: S4. When it is necessary to add a natural language path to the tree structure, obtain a word sequence of the natural language to be added, and generate a word vector for each word of the natural language to be added; S5. Repeat step S2 to generate a natural language path that needs to be added, select the word vectors of the same words in the natural language path that needs to be added as those in the tree structure as common nodes, and add the natural language path that needs to be added to the tree structure.

3. The construction method according to claim 1, characterized in that: In step S1, a unique and non-repetitive ID number is assigned to each word in each natural language segment to obtain a word sequence of the natural language; For each word ID number, randomly generate an m-bit code and generate a word vector for each word.

4. The construction method according to claim 1, characterized in that: In step S2, according to the ID number of each word in the natural language, the word vector of each word is input into the generative neural network in turn, and the predicted word vector of each word corresponding to the word is output to train the generative neural network.

5. A device for constructing a generative neural network tree structure, characterized in that: The construction device comprises: The word vector generation module is used to obtain the word sequence of natural language and generate the word vector of each word in each natural language segment; A generative neural network training module, used to train a generative neural network using the word vector of each word in a natural language segment, and use the trained generative neural network as a path of the natural language segment; The tree structure construction module is used to select the word vectors of the same words in all paths as common nodes to construct a tree structure.

6. The construction device according to claim 5, characterized in that The construction device also includes: The path adding module is used to select the word vectors of the same words in the tree structure in the natural language path to be added as common nodes, and add the natural language path to be added to the tree structure.

7. The construction device according to claim 5, characterized in that The word vector generation module assigns a unique and non-repetitive ID number to each word in each natural language segment to obtain a word sequence of the natural language; For each word ID number, randomly generate an m-bit code and generate a word vector for each word.

8. The construction device according to claim 5, characterized in that The generative neural network training module inputs the word vector of each word into the generative neural network in sequence according to the ID number of each word in the natural language, outputs the predicted word vector of each word corresponding to the word, and trains the generative neural network.

9. A storage medium, characterized in that: The storage medium is used to store computer-executable instructions, and the computer-executable instructions are used to execute the construction method described in any one of claims 1 to 4.

Images