Dynamic hierarchical semantic memory method

Abstract

The invention provides a dynamic hierarchical semantic memory method. The dynamic hierarchical semantic memory method comprises a hierarchical encoder, a dynamic memory management and access mechanism and context-based semantic resolution and fusion. A multi-level and structured dynamic semantic memory system can be constructed, surface sequence information of a text can be captured at the same time, the method is used for local coherence and deep semantic architecture and global understanding, and the most relevant part can be dynamically activated according to a current task.

Description

Technical Field

[0001] This invention relates to the field of large model LLM technology, and in particular to a dynamic hierarchical semantic memory method. Background Technology

[0002] Instead of viewing context as a flat, continuous sequence of tokens, this approach simulates how humans process long texts: constructing a multi-layered, structured, dynamic semantic memory system. This system can simultaneously capture the surface sequence information of the text (for local coherence) and the deep semantic architecture (for global understanding), and can dynamically activate the most relevant parts based on the current task.

[0003] The key issues to be addressed: Information flattening: Traditional vector representation flattens important and secondary information into the same dimension, causing key signals to be buried.

[0004] Missing structure: Unable to explicitly represent the logical structure of the text, such as chapters, arguments, facts, and entity relationships.

[0005] Static representation: Once the context is encoded, it is fixed and cannot be dynamically corrected or focused during the generation process. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of the prior art by providing a dynamic hierarchical semantic memory method, thereby constructing a multi-level, structured dynamic semantic memory system.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides a dynamic hierarchical semantic memory method, comprising the following steps: S1, Hierarchical encoder; S2, Dynamic Memory Management and Access Mechanism; S3. Context-based semantic resolution and fusion.

[0008] Furthermore, S1 specifically refers to: S101, Surface Sequence Layer: Preserves the original or compressed token-level sequence information for tasks requiring local grammar and coherence; divides long sequences into multiple blocks, each block extracts local features through a combination of lightweight convolution and attention, and implements a hierarchical pooling strategy: word-level, phrase-level, and sentence-level, preserving multi-granularity information; S102, Semantic Concept Layer: Automatically extracts key entities, events, concepts and core relationships from text to build a lightweight real-time knowledge graph or concept network; S103, Abstract Intent Layer: Generates high-level abstracts at the paragraph or chapter level and infers the author's intent, sentiment, or overall argument of the text.

[0009] Furthermore, the semantic concept layer is the skeleton of global understanding, including: real-time entity recognition and relation extraction, using a lightweight BERT model for incremental NER and RE; graph structure learning: dynamically updating the relationship between new nodes based on the message passing mechanism of GNN; and time-aware graph embedding: adding time decay weights to each node to distinguish the importance of new and old information.

[0010] Furthermore, the abstract intent layer provides strategic guidance, including multi-scale abstract generation: simultaneously generating paragraph-level, chapter-level, and document-level abstracts; intent classifier: identifying the dialogue behavior of text fragments; and sentiment trajectory modeling: tracking the sentiment changes throughout the text and constructing a sentiment evolution map.

[0011] Furthermore, S2 specifically refers to: Memory Units: Information from the three levels—surface sequence layer, semantic concept layer, and summary intent layer—is stored in different memory units, with metadata appended to each unit; Read operation: When the next token needs to be generated, a two-stage retrieval is performed; Importance score: calculated based on information entropy, frequency of occurrence, and positional weight; Freshness decay: an exponential decay function, where recently accessed information has a higher weight; Confidence calibration: Calculate confidence based on source reliability and consistency; Routing: Based on the current generation state, a lightweight routing network determines which memory layer has the highest priority for retrieving information; Precision: Within the selected memory layer, perform efficient similarity retrieval or attention mechanisms to obtain the most relevant information fragments; Write operation: The memory unit is dynamically updated during the reading and generation process.

[0012] Furthermore, S3 specifically involves: explicitly solving the long-distance referential problem by utilizing the constructed concept layer; When a change in contextual semantics is detected, a new node or version of the concept is created in the concept layer and linked together through relational edges, thereby clarifying the semantic evolution path.

[0013] The beneficial effects of this invention are: it can construct a multi-level, structured dynamic semantic memory system, which can simultaneously capture the surface sequence information of text (for local coherence) and the deep semantic architecture (for global understanding), and can dynamically activate the most relevant parts according to the current task. Attached Figure Description

[0014] Figure 1 This is a flowchart of a dynamic hierarchical semantic memory method. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0016] Please see Figure 1 A dynamic hierarchical semantic memory method includes the following steps: S1, Hierarchical encoder; S2, Dynamic Memory Management and Access Mechanism; S3. Context-based semantic resolution and fusion.

[0017] Specifically, S1 is: S101, Surface Sequence Layer: Preserves the original or compressed token-level sequence information, used for tasks requiring local grammar and coherence; for example, using a variant of Locality Sensitive Hashing (LSH) attention. Long sequences are divided into multiple blocks, each block extracts local features through a combination of lightweight convolution and attention, implementing a hierarchical pooling strategy: word-level, phrase-level, and sentence-level, preserving multi-granularity information. S102, Semantic Concept Layer: Automatically extracts key entities, events, concepts and core relationships from text to build a lightweight real-time knowledge graph or concept network; S103, Abstract Intent Layer: Generates high-level abstracts at the paragraph or chapter level and infers the author's intent, sentiment, or overall argument of the text.

[0018] The semantic concept layer is the skeleton of global understanding, including: real-time entity recognition and relation extraction, using a lightweight BERT model for incremental NER and RE; graph structure learning: dynamically updating node relationships based on the message passing mechanism of GNN; and time-aware graph embedding: adding time decay weights to each node to distinguish the importance of new and old information.

[0019] The abstract intent layer provides strategic guidance, including multi-scale abstract generation: simultaneously generating paragraph-level, chapter-level, and document-level summaries; intent classifier: identifying the dialogue behavior (questions, statements, instructions, etc.) of text fragments; and sentiment trajectory modeling: tracking the sentiment changes throughout the text and constructing a sentiment evolution map.

[0020] Specifically, S2 is: Memory Units: Information from the three levels of surface sequence layer, semantic concept layer, and summary intent layer is stored in different memory units, and metadata (such as confidence, importance weight, and last accessed timestamp) is attached to each unit.

[0021] Read operation: When the next token needs to be generated, a two-stage retrieval is performed; Importance score: calculated based on information entropy, frequency of occurrence, and positional weight; Freshness decay: an exponential decay function, where recently accessed information has a higher weight; Confidence calibration: Calculate confidence based on source reliability and consistency; Routing: Based on the current generation state, a lightweight routing network determines which memory layer (sequence layer, concept layer, or summary layer) has the highest priority for retrieving information; Precision: Within the selected memory layer, perform efficient similarity retrieval or attention mechanisms to obtain the most relevant information fragments; Write operation: The memory unit is dynamically updated during the reading and generation process.

[0022] Specifically, S3 involves: using the constructed concept layer to explicitly solve the long-distance referential problem; When a change in contextual semantics is detected (e.g., "apple" changes from fruit to company), a new node or version of the concept is created in the concept layer and linked by relational edges, thereby clarifying the semantic evolution path.

[0023] The embodiments described above are merely illustrative of implementation methods of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be defined by the appended claims.

Claims

1. A dynamic hierarchical semantic memory method, characterized in that: Includes the following steps: S1, Hierarchical encoder; S2, Dynamic Memory Management and Access Mechanism; S3. Context-based semantic resolution and fusion.

2. The dynamic hierarchical semantic memory method according to claim 1, characterized in that, Specifically, S1 is: S101, Surface Sequence Layer: Preserves the original or compressed token-level sequence information for tasks requiring local grammar and coherence; divides long sequences into multiple blocks, each block extracts local features through a combination of lightweight convolution and attention, and implements a hierarchical pooling strategy: word-level, phrase-level, and sentence-level, preserving multi-granularity information; S102, Semantic Concept Layer: Automatically extracts key entities, events, concepts and core relationships from text to build a lightweight real-time knowledge graph or concept network; S103, Abstract Intent Layer: Generates high-level abstracts at the paragraph or chapter level and infers the author's intent, sentiment, or overall argument of the text.

3. The dynamic hierarchical semantic memory method according to claim 2, characterized in that: The semantic concept layer is the skeleton of global understanding, including: real-time entity recognition and relation extraction, using a lightweight BERT model for incremental NER and RE; graph structure learning: dynamically updating the relationship between new nodes based on the message passing mechanism of GNN; and time-aware graph embedding: adding time decay weights to each node to distinguish the importance of new and old information.

4. The dynamic hierarchical semantic memory method according to claim 2, characterized in that: The abstract intent layer provides strategic guidance, including multi-scale abstract generation: simultaneously generating paragraph-level, chapter-level, and document-level summaries; and an intent classifier: identifying the conversational behavior of text fragments. Emotional trajectory modeling: Tracking emotional changes throughout the text and constructing an emotional evolution map.

5. The dynamic hierarchical semantic memory method according to claim 1, characterized in that, Specifically, S2 is: Memory Units: Information from the three levels—surface sequence layer, semantic concept layer, and summary intent layer—is stored in different memory units, with metadata appended to each unit; Read operation: When the next token needs to be generated, a two-stage retrieval is performed; Importance score: calculated based on information entropy, frequency of occurrence, and positional weight; Freshness decay: an exponential decay function, where recently accessed information has a higher weight; Confidence calibration: Calculate confidence based on source reliability and consistency; Routing: Based on the current generation state, a lightweight routing network determines which memory layer has the highest priority for retrieving information; Precision: Within the selected memory layer, perform efficient similarity retrieval or attention mechanisms to obtain the most relevant information fragments; Write operation: The memory unit is dynamically updated during the reading and generation process.

6. The dynamic hierarchical semantic memory method according to claim 5, characterized in that, Specifically, S3 involves: using the constructed concept layer to explicitly solve the long-distance referential problem; When a change in contextual semantics is detected, a new node or version of the concept is created in the concept layer and linked together through relational edges, thereby clarifying the semantic evolution path.

Images