Research and development-oriented long-short-term memory framework construction method and system

Abstract

The invention belongs to the technical field of software development, and particularly provides a research and development-oriented long and short-term memory framework construction method and system, which adopts a layered memory architecture to construct four core modules including a short-term memory compressor, a medium-term memory aggregator, a long-term memory graph and a cross-layer memory router. The system takes multi-source input such as research and development dialogues, code snippets and project documents as a starting point, extracts research and development elements through semantic analysis and entity recognition technologies, compresses lengthy dialogues into structured short-term memory by utilizing an attention distillation mechanism, upgrades high-frequency short-term memory into medium-term knowledge fragments based on a time sequence attenuation algorithm, and improves the research and development efficiency. And constructing a long-term knowledge graph containing developer portraits, project dependence and normative standards by adopting a graph convolutional network. Context understanding accuracy, multi-round dialogue continuity and personalized service quality of a large model in a research and development scene are remarkably improved, and the method is suitable for mainstream research and development tool scenes such as IDE plug-ins, code review and architecture design.

Description

Technical Field

[0001] This disclosure relates to the field of software development technology, and in particular to a method and system for constructing a long short-term memory framework for research and development. Background Technology

[0002] In the AI-driven revolution of R&D efficiency, long short-term memory (LSTM) technology is becoming a core technological breakthrough for intelligent R&D assistants. As the demands for AI in R&D scenarios such as code generation, defect analysis, and architecture design deepen, existing technologies are revealing significant limitations in memory management.

[0003] Current mainstream AI assistants for R&D primarily rely on piling up raw dialogue history to achieve multi-turn interactions. This simple memory mechanism has several technical shortcomings in real-world R&D scenarios. At the short-term memory level, due to the limitations of the LLM context window, critical API documentation or error stacks are easily truncated in complex code review scenarios, preventing the model from obtaining the complete problem context. During architecture design discussions, after a certain number of dialogue rounds, the model's attention to system constraints significantly decreases, affecting the consistency of design solutions. Furthermore, unprocessed raw Git history dilutes the model's ability to judge core requirements.

[0004] In terms of long-term memory, existing technologies lack specific designs for the R&D field. Long-term memory in R&D scenarios needs to store structured information such as developer profiles (technology stack preferences, coding style patterns), project knowledge graphs (microservice dependencies, module interfaces), and organizational standards (secure coding rules, cloud-native architecture standards), as well as semi-structured content such as vectorized code snippet libraries and defect pattern databases. However, existing solutions typically use simple vector databases for storage, lacking the ability to semantically model and analyze the relationships between R&D knowledge and other information.

[0005] More critically, current technologies lack an effective memory hierarchy management mechanism. There is no smooth transition between short-term and long-term memory, hindering the automatic accumulation and upgrading of knowledge. Furthermore, the lack of a dynamic forgetting mechanism means that outdated or erroneous memories continue to interfere with model judgments. In terms of retrieval performance, in real-time interactive scenarios such as IDE plugins, current technologies struggle to complete memory retrieval and generation within milliseconds, impacting user experience.

[0006] These technical issues severely restrict the evolution of intelligent assistants from "one-time dialogue tools" to "continuously growing R&D partners," and there is an urgent need for a technical framework with hierarchical memory management, dynamic knowledge updates, and efficient semantic retrieval capabilities. Summary of the Invention

[0007] This disclosure aims to address at least one of the technical problems existing in the prior art, and proposes a method and system for constructing a long short-term memory framework for research and development.

[0008] This disclosure provides a method for constructing a long short-term memory framework for research and development, including: S1, converts multi-source heterogeneous data into structured memory data; S2, the structured memory data is divided into short-term memory, medium-term memory and long-term memory and stored separately; S3 utilizes a cross-layer memory router for cross-layer retrieval and fusion output.

[0009] Preferably, S1 specifically includes: S11, different methods are used to parse and extract key text information from different types of multi-source heterogeneous data; S12, combined with a dictionary specifically for the R&D field, identifies key entity information in the R&D field; S13, the key text information and key entity information are converted into numerical semantic vector representations to obtain standardized structured memory data.

[0010] Preferably, S11 specifically includes: For code input data, abstract syntax tree parsing technology is used to extract function structure, variable definitions, and call relationships; For the document input data, a document structure analysis algorithm is used to extract heading levels, key paragraphs, and code examples; For dialogue input data, natural language understanding technology is used to identify user intent and determine whether it is a code generation request, a question inquiry, or an error investigation.

[0011] Preferably, the short-term memory storage process in S2 specifically includes: The complete dialogue history is segmented into fixed-length semantic blocks. Then, the attention score between each semantic block and the current user query is calculated using the formula A_i = softmax((Q·K_i) / d), where Q represents the vector representation of the current query, K_i represents the key vector of the i-th historical semantic block, and d is the vector dimension. The attention scores are then ranked by importance, and the top-K key semantic blocks with the highest scores are retained while redundant information irrelevant to the current task is filtered out. Receive key information fragments filtered by the distillation engine, and organize and classify them according to a predefined structured template; The compression strategy and retention ratio are adaptively adjusted based on the complexity and urgency of the current task, and the compressed short-term memory is output.

[0012] Preferably, the storage process of intermediate memory in S2 specifically includes: A dynamic weight is assigned to each knowledge fragment entering the intermediate memory. This weight will naturally decrease over time to decay. The decay function is designed as W(t) = W_0 × e^(-λt) × (1 + γ×freq), where W_0 represents the initial importance weight of the knowledge fragment, λ is the time decay coefficient, γ is the frequency enhancement coefficient, freq represents the access frequency of the knowledge fragment, and t is the time interval. Semantic clustering analysis is performed regularly on the content in medium-term memory to merge multiple similar knowledge fragments into more abstract and comprehensive knowledge items. The distance metric function of semantic clustering analysis, d(x,y) = α×semantic_sim(x,y) +β×temporal_sim(x,y), considers both semantic similarity and temporal relevance. Here, x and y represent two knowledge fragments, α and β are weight coefficients, semantic_sim is the semantic similarity calculation function, and temporal_sim is the temporal relevance calculation function. The evaluation continuously assesses whether knowledge fragments in intermediate memory have the value and conditions to be upgraded to long-term memory. The evaluation criteria include cumulative access frequency, number of references across multiple sessions, and the strength of association with existing long-term memory entities. The evaluation model score = w_1×frequency + w_2×persistence + w_3×relevance is used for comprehensive evaluation, where w_1, w_2, and w_3 are weight coefficients, frequency is the access frequency score, persistence is the persistence score, and relevance is the association score.

[0013] Preferably, the long-term memory storage process in S2 specifically includes: The complex knowledge in the R&D field is abstracted into developer entities, project entities, specification entities, and experience entities, with each type of entity carrying information in different dimensions. Graph convolutional networks are used to achieve dynamic learning and updating of knowledge graphs, enabling each entity node in the knowledge graph to learn information from its neighboring entity nodes, thereby continuously improving its own feature representation. Node updates are implemented through the information propagation mechanism of the formula h_v^((L)+1) = σ(∑_(u∈N(v)) (1 / (|N(v)||N(u)|)) × W^(L) × h_u^(L)), where v represents the currently updated node, L represents the number of graph convolutional layers, h_v^((L)+1) is the feature representation of node v in the L+1 layer, u represents the neighboring nodes of node v, N(v) is the set of neighbors of node v, W^(L) is the learnable weight matrix of the Lth layer, and σ is the activation function. By comprehensively analyzing multiple signals, the degree of association between entities in the knowledge graph is quantified, entity relationships are calculated and established, and finally a dynamically updated knowledge graph is formed. Among them, the association strength formula strength(e_1,e_2) = log(1+co_occurrence) × semantic_sim(e_1,e_2) × decay_factor is used to evaluate the strength of the relationship between two entities e_1 and e_2, where co_occurrence is the co-occurrence frequency value, semantic_sim is the semantic similarity calculation function, and decay_factor is the time decay factor.

[0014] Preferably, the developer entity includes personalized information such as the developer's skill tree, coding style preferences, and historical project experience; the project entity includes technical information such as inter-module dependencies, API interface definitions, and architectural component structures; the specification entity includes organizational-level specification requirements for coding standards, design patterns, and best practices; and the experience entity includes practical knowledge such as historical bug patterns, solution templates, and performance optimization experience.

[0015] Preferably, S3 specifically includes: S31 uses a pre-trained multi-label text classification model to automatically classify user query requests into four basic types: immediate task, context continuation, knowledge query, and comprehensive analysis. S32 employs specially optimized retrieval algorithms for different query types. Short-term memory retrieval uses a fast matching algorithm based on vector similarity, with a time complexity of O(n), suitable for scenarios with high real-time requirements. Mid-term memory retrieval combines time decay weights for weighted sorting, prioritizing the return of knowledge fragments with higher weights to balance the timeliness and relevance of information. Long-term memory retrieval is based on a graph structure to achieve multi-hop reasoning queries. S33 uses a result fusion algorithm to intelligently merge retrieval results from different memory levels into a final answer for output.

[0016] Preferably, the result fusion algorithm in S33 specifically includes: When a user's query requires access to short-term, medium-term, and long-term memory simultaneously, multiple candidate results are obtained from each level, and then it is necessary to decide which results are most valuable and how to combine them. The value of each candidate result is evaluated by the fusion scoring formula: score = ∑_i (w_i × sim_i × confidence_i × freshness_i), where i represents the i-th candidate result, w_i is the query type weight, sim_i is the semantic similarity score, confidence_i is the retrieval confidence score, and freshness_i is the information freshness score. Query type weights are assigned different levels of importance based on the specific characteristics of the current query. Semantic similarity measures the degree of content matching between candidate results and user queries, retrieval confidence reflects the system's confidence level in the accuracy of the result, and information freshness considers the timeliness of the result. The weight allocation is dynamically adjusted based on the specific query. When a user asks about the latest technology trends, the weight of freshness_i will increase; when a user needs to debug code errors, the weight of confidence_i will increase.

[0017] This disclosure also provides a long short-term memory (LSTM) framework construction system for research and development, which can be used to implement the above-mentioned LSM framework construction method for research and development. The system includes: The input preprocessing module is configured to convert multi-source heterogeneous data into structured memory data. The hierarchical storage module is configured to divide the structured memory data into short-term memory, medium-term memory, and long-term memory for separate storage; The cross-layer memory output module is configured to utilize a cross-layer memory router for cross-layer retrieval and fusion output.

[0018] Beneficial Effects: Addressing the technical shortcomings of existing R&D intelligent assistants in memory management, this invention proposes a method for constructing a long short-term memory framework for R&D. This framework employs an innovative hierarchical memory architecture, utilizing four core modules—a short-term memory compressor, a medium-term memory aggregator, a long-term memory graph, and a cross-layer memory router—to achieve intelligent management and efficient retrieval of R&D knowledge.

[0019] The core innovation of this invention lies in establishing an organic collaborative mechanism for short-term, medium-term, and long-term memory. The short-term memory layer uses attention distillation technology to compress lengthy dialogues into structured key information, breaking through the limitations of the context window; the medium-term memory layer is based on a temporal decay algorithm to achieve dynamic management and automatic upgrading of knowledge fragments; and the long-term memory layer uses graph convolutional networks to construct a multi-dimensional knowledge graph, deeply integrating R&D elements such as developer profiles, project dependencies, and organizational norms.

[0020] This framework significantly improves the accuracy of contextual understanding, the coherence of multi-turn dialogues, and the quality of personalized services for large models in R&D scenarios. In terms of performance, it supports millisecond-level memory retrieval, meeting the real-time interaction needs of IDE plugins and other applications. Regarding knowledge management, it enables automatic memory accumulation, dynamic updates, and intelligent forgetting, giving the R&D intelligent assistant continuous learning and growth capabilities. Attached Figure Description

[0021] Figure 1 This is a flowchart of an energy efficiency assessment method provided in an embodiment of the present disclosure. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions of this disclosure, the disclosure will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure are not intended to indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms are not intended to limit the quantity, but rather to indicate the presence of at least one. The terms “comprising,” “including,” or “including,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects.

[0024] The English abbreviations used in this invention are explained as follows: DevAgent (Development Agent) is an AI assistant specifically designed for the software development field, possessing core capabilities such as code generation, defect analysis, and architecture design. Unlike general chatbots, DevAgent requires a deep understanding of specific knowledge in areas such as code semantics, project structure, and development processes. In this invention, DevAgent is the primary application carrier of the Long Short-Term Memory (LSTM) framework, and through enhanced memory capabilities, it evolves from a "one-time conversation tool" to a "continuously growing development partner."

[0025] STM (Short-Term Memory) refers to the temporary information storage maintained by the system in the current dialogue session, mainly including the dialogue history of recent rounds, the current task context, and active code snippets. In R&D scenarios, the main challenge of STM is how to retain critical information within a limited context window, avoiding the truncation of important API documentation, error stacks, etc. This invention uses attention distillation technology to intelligently compress and structure STM.

[0026] Medium-Term Memory (MTM) is an intermediate storage layer between short-term and long-term memory, used to maintain session-level project context, temporary knowledge fragments, and phased task states. The lifespan of MTM typically ranges from several hours to several days, supporting context continuation across sessions. In this invention, MTM employs a temporal decay algorithm to manage the lifespan of knowledge fragments, enabling dynamic knowledge upgrading and forgetting.

[0027] Long-Term Memory (LTM) is a persistent knowledge store for a system, containing stable information such as developer profiles, project knowledge graphs, organizational norms, and historical experience. LTM has three storage formats: structured, semi-structured, and unstructured, and supports semantic retrieval and associative reasoning. This invention uses graph convolutional networks to construct the core knowledge graph of LTM, achieving deep fusion of multi-source information.

[0028] Attention distillation is a technique for extracting key information from long sequences by analyzing the distribution of attention weights in a model to identify the most important contextual fragments for the current task. In this invention, attention distillation is used for intelligent compression of STM (Simulation-Based Transmission) to refine lengthy dialogue histories into structured summaries of key information.

[0029] Graph Convolutional Networks (GCNs) are deep learning models specifically designed for processing graph-structured data, enabling information propagation and feature learning at the nodes and edges of a graph. In this invention, GCNs are used to construct and update knowledge graphs in long-term memory, achieving correlated modeling of multi-dimensional information such as developer profiles, project dependencies, and code patterns.

[0030] Semantic routing is a technique that automatically selects the most suitable memory level and retrieval strategy based on the semantic features of the input query. This invention designs a cross-layer memory router that can intelligently determine whether a query should access short-term, medium-term, or long-term memory, and combine multi-level results to form the final answer.

[0031] This invention provides a method for constructing a long short-term memory (LSTM) framework for research and development. This framework employs a hierarchical memory architecture to achieve intelligent management and efficient retrieval of research and development knowledge. The overall technical solution follows the design principles of "multi-layered memory collaboration, dynamic knowledge management, and semantic routing retrieval," and the specific technical process is as follows: Figure 1 As shown.

[0032] A four-layer architecture is adopted, with clear data flow relationships between each layer. The input preprocessing layer is responsible for converting heterogeneous data from multiple sources into a standardized format, laying the foundation for subsequent processing. The hierarchical memory management layer realizes the collaborative storage of STM, MTM, and LTM memory layers, with each layer undertaking knowledge management responsibilities with different timeliness requirements. The semantic retrieval layer selects the optimal retrieval strategy through an intelligent routing mechanism to achieve accurate information retrieval. The output generation layer merges the retrieval results from the multiple memory layers to generate high-quality answers. The entire data flow follows the process: raw input → preprocessing standardization → storage in the corresponding memory layer → intelligent retrieval based on the query → fusion and output of the final result. Specifically, it includes the following processes: The first step, input preprocessing and feature extraction, transforms multi-source heterogeneous data into structured memory data. This is a critical task in converting diverse heterogeneous data from R&D scenarios into a unified format that the system can process, providing standardized input for subsequent memory storage and retrieval. The module employs a three-level processing architecture to achieve a complete transformation from raw data to structured features.

[0033] The multimodal parser, as the first-level processing unit, is specifically responsible for identifying and parsing different types of input data. For code input, the parser extracts key information such as function structure, variable definitions, and call relationships using abstract syntax tree parsing technology. For document input, the system employs document structure analysis algorithms to extract content such as heading levels, key paragraphs, and code examples. When processing dialogic input, the parser uses natural language understanding technology to identify the user's intent and determine the specific task type, such as a code generation request, a question inquiry, or error debugging. This categorized processing ensures that the system can accurately understand the structural features and semantic meaning of different input content.

[0034] The entity recognizer, serving as the second-level processing unit, is specifically designed to identify key entity information within the R&D domain. Based on a pre-trained named entity recognition model and enhanced with rules using a domain-specific dictionary, this recognizer accurately identifies technical entities such as API names, framework components, class names, variable names, error types, and architectural components. These entities not only form the foundation for building a long-term memory knowledge graph but also serve as crucial indexes for subsequent semantic retrieval. Through domain-specific entity recognition technology, the system can deeply understand the semantic relationships and dependencies between technical concepts.

[0035] The semantic feature encoder, as the third-level processing unit, converts the parsing results of the first two submodules into numerical semantic vector representations. The encoder employs a 768-dimensional high-dimensional vector space to ensure the full expression of semantic information. Simultaneously, the system retains and structurally stores important meta-information such as timestamps, file paths, author information, and task types. This unified encoding format enables information from different data sources to undergo similarity calculations, association analysis, and clustering within the same semantic space. The three submodules work collaboratively in a strict processing order, ultimately outputting a standardized data structure: {semantic vectors, meta-information dictionary, entity list}, providing high-quality input data for subsequent processing within the entire memory system.

[0036] Logical relationship: The three sub-modules are processed in sequence: multimodal parser → entity recognizer → semantic encoder, and finally output a standardized structure: {semantic vector, meta-information, entity list}, which serves as the input to the memory system.

[0037] The second step is to divide the structured memory data into short-term memory, medium-term memory, and long-term memory for separate storage.

[0038] Existing technical solutions primarily manage historical information using a simple context splicing method, linearly storing all dialogue history and lacking a hierarchical memory design. As the number of dialogue rounds increases, the system can only handle context overflow issues through truncation or sliding windows, leading to the loss of crucial information. Mainstream tools like GitHub Copilot also have similar limitations, failing to distinguish the importance and timeliness of information, treating all historical information equally. This invention innovatively proposes a three-layer memory architecture (STM-MTM-LTM), achieving automatic hierarchical management and lifecycle control of knowledge. Short-term memory intelligently compresses current conversation information using attention distillation technology; medium-term memory manages conversation-level knowledge fragments using a temporal decay algorithm; and long-term memory constructs a persistent knowledge graph. More importantly, the system establishes an automatic upgrade mechanism between memory layers, allowing high-value short-term knowledge to be automatically deposited into long-term memory through an upgrade decision mechanism, forming an organic knowledge growth channel. This dynamic hierarchical architecture not only solves the context window limitation problem but also achieves intelligent accumulation and forgetting of knowledge. Each type of structured memory data is stored using different methods, as detailed below: (1) Short-term memory compressor; The Short-Term Memory Compressor is a core module for addressing the context window limitation problem in large language models. Its main task is to intelligently compress the lengthy dialogue history of the current session into refined, structured information, significantly saving computational resources while ensuring that key context is not lost. This module employs an attention-driven compression strategy, which adaptively preserves the historical information most valuable to the current task.

[0039] The attention distillation engine analyzes the attention weight distribution patterns of large language models when processing long dialogue sequences to accurately identify the dialogue segments that contribute most to the current task. The system first segments the complete dialogue history into fixed-length semantic blocks, then calculates the attention score between each semantic block and the current user query, using the formula A_i = softmax((Q·K_i) / d), where Q represents the vector representation of the current query, K_i represents the key vector of the i-th historical semantic block, and d is the vector dimension. Based on the calculated attention scores, the system ranks the attention blocks by importance, retaining the top-K key semantic blocks with the highest scores while filtering out redundant information irrelevant to the current task.

[0040] The context structurator receives key information fragments filtered by the distillation engine and organizes and categorizes them according to a predefined structured template. This template includes standardized categories such as task description, relevant code snippets, error messages, API usage examples, preconditions, and expected results. Through this structured processing, the compressed short-term memory is not only concise in its information content but also has a unified format, making it easier for subsequent language models to understand and effectively utilize.

[0041] The dynamic window manager adaptively adjusts the compression strategy and retention ratio based on the complexity and urgency of the current task. For simple query tasks such as syntax checking or API queries, the system retains more historical context information to provide a more comprehensive background. For complex architecture design or system optimization tasks, the system prioritizes retaining key information directly related to the core task. The compression ratio is calculated using the dynamic formula ratio = α × complexity + β × urgency, where complexity is determined based on the syntactic complexity of the query statement and the breadth of the technology stack involved, while urgency is assessed based on user interaction frequency and task priority.

[0042] Logical relationship: When the dialogue exceeds the context window limit, the attention distillation engine first identifies important segments → the context structurator organizes them into a standard format → the dynamic window manager adjusts the retention strategy according to the current task and outputs compressed short-term memory.

[0043] (2) Intermediate memory aggregator; The intermediate memory aggregator manages temporary knowledge with a session-level lifecycle, enabling a smooth transition from short-term to long-term memory. The core value of this module lies in preventing the immediate loss of important information after a session ends, while also preventing the infinite accumulation of useless information that could degrade system performance. The module achieves dynamic lifecycle management of knowledge through a dual mechanism of time decay and value assessment.

[0044] The temporal decay algorithm assigns dynamic weights to each knowledge fragment entering intermediate memory. These weights naturally decrease over time, mimicking the forgetting process in human memory. However, unlike simple temporal decay, the system gives weight enhancement to frequently accessed and persistently relevant content. The decay function is designed as W(t) = W_0 × e^(-λt) × (1 + γ×freq), where W_0 represents the initial importance weight of the knowledge fragment, λ is the temporal decay coefficient, γ is the frequency enhancement coefficient, freq represents the access frequency of the knowledge fragment, and t is the time interval. This design ensures that truly valuable and active knowledge maintains a high presence in intermediate memory, while outdated or irrelevant information is gradually and automatically faded by the system.

[0045] The knowledge fragment aggregator periodically performs semantic clustering analysis on content in medium-term memory, merging multiple similar knowledge fragments into more abstract and comprehensive knowledge entries, effectively avoiding redundant storage of duplicate information. The system employs an improved clustering algorithm, whose distance metric function d(x,y) = α×semantic_sim(x,y) + β×temporal_sim(x,y) considers both semantic similarity and temporal relevance. Here, x and y represent two knowledge fragments, α and β are weighting coefficients, semantic_sim is the semantic similarity calculation function, and temporal_sim is the temporal relevance calculation function. This dual metric ensures that the aggregation process considers not only the semantic association of content but also temporal relevance, making the aggregation results more reasonable and meaningful.

[0046] The upgrade decision-maker continuously evaluates whether knowledge fragments in medium-term memory possess the value and conditions to be upgraded to long-term memory. Evaluation criteria include multiple dimensions such as cumulative access frequency, number of references across multiple sessions, and the strength of association with existing long-term memory entities. The system uses a multi-factor scoring model: score = w_1×frequency + w_2×persistence + w_3×relevance for comprehensive evaluation, where w_1, w_2, and w_3 are weighted coefficients, frequency is the access frequency score, persistence is the persistence score, and relevance is the relevance score. When the score exceeds a preset threshold, the knowledge fragment is automatically upgraded to the long-term memory layer. This automatic upgrade mechanism enables dynamic identification and hierarchical management of knowledge value, forming a knowledge accumulation channel from short-term to long-term.

[0047] Logical relationship: High-frequency content in short-term memory enters medium-term memory → Temporal decay algorithm continuously adjusts weights → Knowledge aggregator periodically merges similar content → Upgrade judge filters out valuable knowledge and upgrades it to long-term memory, forming a knowledge lifecycle management system.

[0048] (3) Long-term memory map builder; The long-term memory graph builder is responsible for constructing and maintaining the system's core knowledge base, using a graph structure to store stable knowledge assets in the R&D field. This module not only stores structured information such as developer profiles, project dependencies, and organizational standards, but also supports complex relational queries and multi-hop reasoning analysis, providing the system with in-depth domain knowledge support.

[0049] Multidimensional entity modeling abstracts the complex knowledge in the R&D field into four standardized entity types, each carrying information in different dimensions. Developer entities record personalized information such as individual skill trees, coding style preferences, and historical project experience. Project entities describe technical information such as dependencies between modules, API interface definitions, and architectural component structures. Specification entities contain organizational-level specification requirements such as coding standards, design patterns, and best practices. Experience entities accumulate practical knowledge such as historical bug patterns, solution templates, and performance optimization experience. Each entity type has specially designed attribute templates and constraint rules to ensure the consistency and completeness of knowledge representation.

[0050] The graph convolutional update algorithm leverages the powerful capabilities of graph convolutional networks to achieve dynamic learning and updating of knowledge graphs. The core idea of ​​this algorithm is to enable each entity node in the knowledge graph to learn information from its neighboring entity nodes, thereby continuously improving its own feature representation. Node updates are implemented through the formula h_v^((L)+1) = σ(∑_(u∈N(v)) (1 / (|N(v)||N(u)|)) × W^(L) × h_u^(L)), where v represents the currently updated node, L represents the graph convolutional layer number, h_v^((L)+1) is the feature representation of node v at layer L+1, u represents the neighboring nodes of node v, N(v) is the set of neighbors of node v, W^(L) is the learnable weight matrix at layer L, and σ is the activation function. When the system needs to update an entity, it collects information from all neighboring entities associated with that entity, then weights and fuses this neighbor information to generate a new feature representation for that entity. The weight allocation follows a balance principle: it considers both the number of neighbors of the current entity and the connection complexity of the neighboring entities themselves. This update mechanism enables associative learning of knowledge. For example, when the system discovers that a developer frequently uses a specific combination of APIs, the related API entities will strengthen their associations through graph convolution updates. When new code patterns are added to the knowledge graph, related entities automatically adjust their feature representations to reflect the latest usage patterns and dependencies, maintaining the timeliness and accuracy of the knowledge graph.

[0051] Association strength calculation quantifies the degree of association between entities in a knowledge graph by comprehensively analyzing multiple signals. The system uses the association strength formula strength(e_1,e_2) = log(1+co_occurrence) × semantic_sim(e_1,e_2) × decay_factor to evaluate the strength of the relationship between two entities e_1 and e_2, where co_occurrence is the co-occurrence frequency, semantic_sim is the semantic similarity calculation function, and decay_factor is the time decay factor. Co-occurrence frequency analyzes the number of times two entities appear simultaneously in actual development scenarios. For example, if an API function and a specific error handling pattern frequently appear in the same code, it indicates a close usage association between them in practice. Semantic similarity assesses the relevance of two entities at the conceptual level, calculating the degree of similarity in the conceptual space by analyzing the semantic vector representation of the entities. The time decay factor ensures that the association reflects the latest development trends, gradually reducing the weight of older associations over time while enhancing the influence of newly established associations. This multi-dimensional association strength assessment provides a reliable data foundation for intelligent retrieval and association recommendation of knowledge graphs, enabling the system to accurately identify important relationships between entities and perform reasonable knowledge reasoning.

[0052] Logical Relationship: Knowledge upgraded in mid-term memory and externally imported knowledge → Multi-dimensional entity modeling and standardization processing → Graph convolution update algorithm to learn entity representation → Association strength calculation to establish entity relationships, ultimately forming a dynamically updated knowledge graph.

[0053] Existing technologies use static knowledge bases to store test templates and rules, resulting in a flat knowledge structure and a lack of modeling of relationships between entities. While large models like OpenAI's CodeX and Anthropic's Claude possess powerful code understanding capabilities, their knowledge primarily originates from the pre-training phase and cannot dynamically learn personalized user needs and project-specific knowledge. Existing vector database solutions, although supporting semantic retrieval, lack structured modeling of knowledge in the R&D domain and cannot handle complex concept relationships and dependencies. This invention designs a four-dimensional entity model specifically for the R&D domain, including developer entities (skill profiles, coding style), project entities (module dependencies, API interfaces), specification entities (coding standards, architectural patterns), and experience entities (bug patterns, solutions), achieving a deep, structured representation of R&D knowledge. Compared to static knowledge bases, the dynamic graph of this invention can continuously learn personalized knowledge such as project-specific architectural patterns, API usage habits, and coding style preferences.

[0054] The third step is to utilize a cross-layer memory router for cross-layer retrieval and fusion output.

[0055] The cross-layer memory router, as the intelligent retrieval core of the entire system, automatically determines which memory layer to access based on the semantic features of the user's query and the characteristics of the task, and selects the optimal combination of retrieval strategies to achieve accurate and efficient information retrieval. The core value of this module lies in making the complex multi-layer memory system transparent to the user, providing a unified and intelligent query interface.

[0056] The query intent classifier employs a pre-trained multi-label text classification model to automatically categorize user queries into four basic types. Immediate task queries require current context information and primarily access short-term memory to obtain the session state. Context-dependent queries involve cross-session historical information and require mid-term memory to provide session-level background knowledge. Knowledge-based queries require domain-specific knowledge and are primarily retrieved from the knowledge graph in long-term memory. Comprehensive analysis queries require the fusion of information from multiple levels to provide a complete answer. The classifier supports multi-label recognition and can handle complex situations where a query belongs to multiple types simultaneously.

[0057] The hierarchical retrieval strategy employs specially optimized retrieval algorithms for different query types. Short-term memory retrieval uses a fast matching algorithm based on vector similarity, with a time complexity of O(n), suitable for scenarios with high real-time requirements. Mid-term memory retrieval combines time-decay weights for weighted sorting, prioritizing the return of knowledge fragments with higher weights to balance the timeliness and relevance of information. Long-term memory retrieval uses a graph structure to implement multi-hop reasoning queries, which can not only directly match related entities but also discover indirectly related knowledge through associations, supporting deeper semantic understanding.

[0058] The result fusion algorithm intelligently merges retrieval results from different memory levels into a final answer. When a user's query requires access to short-term, medium-term, and long-term memory simultaneously, the system obtains multiple candidate results from each level and then determines which results are most valuable and how to combine them. The system evaluates the value of each candidate result using the fusion scoring formula: score = ∑_i(w_i × sim_i × confidence_i × freshness_i), where i represents the i-th candidate result, w_i is the query type weight, sim_i is the semantic similarity score, confidence_i is the retrieval confidence score, and freshness_i is the information freshness score. The query type weight assigns different levels of importance based on the specific characteristics of the current query. For example, for code debugging problems, error stack information in short-term memory has a higher weight, while for architecture design problems, design pattern knowledge in long-term memory has a greater weight. Semantic similarity measures the degree of content matching between candidate results and the user's query, retrieval confidence reflects the system's confidence level in the accuracy of the result, and information freshness considers the timeliness of the result. The weight allocation strategy is dynamically adjusted according to the specific circumstances of the query. When users inquire about the latest technology trends, the weight of `freshness_i` increases significantly; when users need to debug code errors, the weight of `confidence_i` is strengthened to ensure that the provided solutions are reliable. Through this intelligent fusion mechanism, the system can provide users with accurate and comprehensive answers, allowing them to obtain a high-quality service experience without needing to understand complex underlying memory architectures.

[0059] Logical relationship: User query input → Intent classifier determines query type → Hierarchical retrieval strategy retrieves results in the corresponding memory layer → Result fusion algorithm integrates multi-layer results → Outputs the final answer, achieving end-to-end intelligent routing of the queried answer.

[0060] Existing technologies generally employ a single retrieval strategy in information retrieval. Traditional IDE plugins primarily rely on syntax analysis for code completion, lacking semantic understanding capabilities. While vector-based semantic retrieval systems support semantic matching, their retrieval strategies are fixed and cannot adaptively adjust to query characteristics. Existing RAG (Retrieval-Augmented Generation) systems, although combining retrieval and generation, typically only access single knowledge sources, lacking the ability to intelligently fuse multi-level information. This invention innovatively designs a cross-layer memory router, achieving intelligent matching between query and memory layers. The system first automatically categorizes user requests into four types using a query intent classifier: immediate tasks, contextual continuation, knowledge queries, and comprehensive analysis. Then, it employs specially optimized retrieval strategies for different types: short-term memory uses fast vector matching, medium-term memory combines temporal weighting, and long-term memory uses graph-structured multi-hop reasoning.

[0061] Furthermore, existing technologies primarily employ methods such as truncation, sliding windows, or simple summarization when handling long contexts. While dialogue systems like ChatGPT support longer contexts, they can only use a FIFO (First-In-First-Out) truncation strategy when the window limit is exceeded, failing to guarantee the preservation of important information. Some research has attempted to compress dialogue history using TextRank or BERT-based summarization techniques, but these methods lack consideration for the relevance to the current task and are prone to losing crucial contextual information.

[0062] This invention proposes an intelligent compression technique based on attention distillation. By analyzing the attention weight distribution A_i = softmax((Q·K_i) / √d) of a large language model when processing long dialogues, it accurately identifies the most important historical segments for the current task. This technique not only considers the semantic relevance of information but also incorporates the model's actual attention patterns, ensuring that the compressed information retains as much valuable context as possible for model inference. Furthermore, the system designs a dynamic window management mechanism that adaptively adjusts the compression ratio according to task complexity, achieving a personalized context management strategy.

[0063] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.

Claims

1. A method for constructing a long short-term memory framework for research and development, characterized in that, include: S1, converts multi-source heterogeneous data into structured memory data; S2, the structured memory data is divided into short-term memory, medium-term memory and long-term memory and stored separately; S3 utilizes a cross-layer memory router for cross-layer retrieval and fusion output.

2. The method for constructing a long short-term memory framework for R&D as described in claim 1, characterized in that, S1 specifically includes: S11, different methods are used to parse and extract key text information from different types of multi-source heterogeneous data; S12, combined with a dictionary specifically for the R&D field, identifies key entity information in the R&D field; S13, the key text information and key entity information are converted into numerical semantic vector representations to obtain standardized structured memory data.

3. The method for constructing a long short-term memory framework for R&D according to claim 2, characterized in that, S11 specifically includes: For code input data, abstract syntax tree parsing technology is used to extract function structure, variable definitions, and call relationships; For the document input data, a document structure analysis algorithm is used to extract heading levels, key paragraphs, and code examples; For dialogue input data, natural language understanding technology is used to identify user intent and determine whether it is a code generation request, a question inquiry, or an error investigation.

4. The method for constructing a long short-term memory framework for R&D as described in claim 1, characterized in that, The short-term memory storage process in S2 specifically includes: The complete dialogue history is segmented into fixed-length semantic blocks. Then, the attention score between each semantic block and the current user query is calculated using the formula A_i = softmax((Q·K_i) / d), where Q represents the vector representation of the current query, K_i represents the key vector of the i-th historical semantic block, and d is the vector dimension. The attention scores are then ranked by importance, and the top-K key semantic blocks with the highest scores are retained while redundant information irrelevant to the current task is filtered out. Receive key information fragments filtered by the distillation engine, and organize and classify them according to a predefined structured template; The compression strategy and retention ratio are adaptively adjusted based on the complexity and urgency of the current task, and the compressed short-term memory is output.

5. The method for constructing a long short-term memory framework for R&D according to claim 1, characterized in that, The storage process of intermediate memory in S2 specifically includes: A dynamic weight is assigned to each knowledge fragment entering the intermediate memory. This weight will naturally decrease over time to decay. The decay function is designed as W(t) = W_0 × e^(-λt) × (1 + γ×freq), where W_0 represents the initial importance weight of the knowledge fragment, λ is the time decay coefficient, γ is the frequency enhancement coefficient, freq represents the access frequency of the knowledge fragment, and t is the time interval. Regularly perform semantic clustering analysis on the content in intermediate memory to merge multiple similar knowledge fragments into more abstract and comprehensive knowledge items. The distance metric function of semantic clustering analysis, d(x,y) = α×semantic_sim(x,y) + β×temporal_sim(x,y), considers both semantic similarity and temporal relevance. Here, x and y represent two knowledge fragments, α and β are weight coefficients, semantic_sim is the semantic similarity calculation function, and temporal_sim is the temporal relevance calculation function. The evaluation continuously assesses whether knowledge fragments in intermediate memory have the value and conditions to be upgraded to long-term memory. The evaluation criteria include cumulative access frequency, number of references across multiple sessions, and the strength of association with existing long-term memory entities. The evaluation model score = w_1×frequency + w_2×persistence + w_3×relevance is used for comprehensive evaluation, where w_1, w_2, and w_3 are weight coefficients, frequency is the access frequency score, persistence is the persistence score, and relevance is the association score.

6. The method for constructing a long short-term memory framework for R&D according to claim 1, characterized in that, The long-term memory storage process in S2 specifically includes: The complex knowledge in the R&D field is abstracted into developer entities, project entities, specification entities, and experience entities, with each type of entity carrying information in different dimensions. Graph convolutional networks are used to achieve dynamic learning and updating of knowledge graphs, enabling each entity node in the knowledge graph to learn information from its neighboring entity nodes, thereby continuously improving its own feature representation. Node updates are implemented through the information propagation mechanism of the formula h_v^((L)+1) = σ(∑_(u∈N(v)) (1 / (|N(v)||N(u)|)) × W^(L) × h_u^(L)), where v represents the currently updated node, L represents the number of graph convolutional layers, h_v^((L)+1) is the feature representation of node v in the L+1 layer, u represents the neighboring nodes of node v, N(v) is the set of neighbors of node v, W^(L) is the learnable weight matrix of the Lth layer, and σ is the activation function. By comprehensively analyzing multiple signals, the degree of association between entities in the knowledge graph is quantified, entity relationships are calculated and established, and finally a dynamically updated knowledge graph is formed. Among them, the association strength formula strength(e_1,e_2) = log(1+co_occurrence) × semantic_sim(e_1,e_2) × decay_factor is used to evaluate the strength of the relationship between two entities e_1 and e_2, where co_occurrence is the co-occurrence frequency value, semantic_sim is the semantic similarity calculation function, and decay_factor is the time decay factor.

7. The method for constructing a long short-term memory framework for R&D according to claim 6, characterized in that, The developer entity includes personalized information such as the developer's skill tree, coding style preferences, and historical project experience; the project entity includes technical information such as inter-module dependencies, API interface definitions, and architectural component structures; the specification entity includes organizational-level specification requirements for coding standards, design patterns, and best practices; and the experience entity includes practical knowledge such as historical bug patterns, solution templates, and performance optimization experience.

8. The method for constructing a long short-term memory framework for R&D according to claim 1, characterized in that, S3 specifically includes: S31 uses a pre-trained multi-label text classification model to automatically classify user query requests into four basic types: immediate task, contextual continuation, knowledge query, and comprehensive analysis. S32 employs specially optimized retrieval algorithms for different query types. Short-term memory retrieval uses a fast matching algorithm based on vector similarity, with a time complexity of O(n), suitable for scenarios with high real-time requirements. Mid-term memory retrieval combines time decay weights for weighted sorting, prioritizing the return of knowledge fragments with higher weights to balance the timeliness and relevance of information. Long-term memory retrieval is based on a graph structure to achieve multi-hop reasoning queries. S33 uses a result fusion algorithm to intelligently merge retrieval results from different memory levels into a final answer for output.

9. The method for constructing a long short-term memory framework for R&D according to claim 8, characterized in that, The result fusion algorithm in S33 specifically includes: When a user's query requires access to short-term, medium-term, and long-term memory simultaneously, multiple candidate results are obtained from each level, and then it is necessary to decide which results are most valuable and how to combine them. The value of each candidate result is evaluated by the fusion scoring formula score = ∑_i (w_i × sim_i × confidence_i × freshness_i), where i represents the i-th candidate result, w_i is the query type weight, sim_i is the semantic similarity score, confidence_i is the retrieval confidence score, and freshness_i is the information freshness score. Query type weights are assigned different levels of importance based on the specific characteristics of the current query. Semantic similarity measures the degree of content matching between candidate results and user queries, retrieval confidence reflects the system's confidence level in the accuracy of the result, and information freshness considers the timeliness of the result. The weight allocation is dynamically adjusted based on the specific query. When a user asks about the latest technology trends, the weight of freshness_i will increase; when a user needs to debug code errors, the weight of confidence_i will increase.

10. A long short-term memory framework construction system for research and development, characterized in that, The system can be used to implement the long short-term memory framework construction method for R&D as described in any one of claims 1 to 9, and the system includes: The input preprocessing module is configured to convert multi-source heterogeneous data into structured memory data. The hierarchical storage module is configured to divide the structured memory data into short-term memory, medium-term memory, and long-term memory for separate storage; The cross-layer memory output module is configured to utilize a cross-layer memory router for cross-layer retrieval and fusion output.

Images