Question answering method and device based on memory slot and semantic alignment, equipment and medium

Abstract

The invention relates to the technical field of semantic analysis, can be applied to business scenes such as financial science and technology and medical health, and discloses a question and answer method, device and equipment based on memory slots and semantic alignment and a medium. The method comprises the steps that a document set is acquired and stored in a knowledge base, semantic aggregation is conducted on the document set to generate a memory slot set, and a mapping relation is established; performing semantic alignment on the memory slot set to generate global target embedding representation; receiving a query question to generate a question embedding representation; matching the problem embedding representation with the global target embedding representation to determine a target memory slot and obtain candidate document fragments; generating a fragment target embedded representation, and fusing the fragment target embedded representation with the question embedded representation to form a question and answer input representation; and generating a question-answer response through the question-answer model. According to the method, context compression and information fidelity are realized through semantic aggregation and embedding alignment, the long text reasoning overhead is reduced, and the information positioning and generating efficiency of the question and answer model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semantic parsing, and in particular to a question and answer method and device based on memory slots and semantic alignment, equipment and medium. BACKGROUND

[0002] In the field of financial technology business, the existing large language model question and answer system has been applied to intelligent customer service, risk research and judgment, claim question and answer, and insurance consulting, etc. However, there are significant long text processing problems for financial and insurance documents (such as insurance policy clauses, claim rules, regulatory guidelines, etc.). The traditional question and answer system based on large language model needs to input the complete text into the model for context analysis in the reasoning stage, and the computational complexity increases exponentially with the increase of input length, resulting in high reasoning delay and high consumption of computing resources. Especially in edge node scenarios such as front-end terminals and mobile applications, limited by computing power and storage conditions, the model is difficult to load and execute efficiently, and cannot meet the real-time response requirements of business. In addition, the existing system based on retrieval enhancement generation usually uses text truncation, keyword retrieval or sparse recall to reduce the input size, but this kind of method often ignores the logical constraint relationship and context dependence between documents, causing the factuality of the question and answer results to decline, the logic to break and the semantic to drift, thereby weakening the explainability and credibility of the question and answer system in the financial and insurance field.

[0003] In the field of medical and health business, the question and answer system based on large language model is also widely used in medical knowledge question and answer, health consultation and medical record information analysis, etc. However, medical and health documents (such as guidelines, medical records, medication instructions) have complex structure and dense terminology, and the existing long text processing mechanism is prone to context compression distortion and key information omission when facing multi-level semantic dependence. In particular, in the electronic medical record system or mobile health terminal, the model calculation is limited, which makes it impossible to process the complete medical record context, resulting in one-sided answers or semantic misplacement. At the same time, the general text summary or knowledge compression model is difficult to maintain the integrity and causal consistency of clinical semantics in the medical field, and cannot effectively support doctor decision assistance or patient inquiry service. In addition, the lack of context compression mechanism for domain document features makes the generalization performance of the model insufficient in medical semantic understanding, and it is difficult to balance the compression efficiency and semantic fidelity. SUMMARY

[0004] The main purpose of the present application is to provide a question and answer method, device, equipment and storage medium based on memory slots and semantic alignment, aiming to solve the technical problem that the existing technology lacks a mechanism that can realize efficient context compression and semantic alignment while ensuring semantic integrity, resulting in insufficient reasoning efficiency and information fidelity of the question and answer system in long text processing.

[0005] To achieve the above object, the application provides a question and answer method based on memory slots and semantic alignment, comprising: A document set is acquired, and the document set and corresponding metadata are stored in a knowledge base; The document set in the knowledge base is subjected to multi-layer semantic aggregation processing by a compressor module, a memory slot set corresponding to the document set is generated, and a mapping relationship is established between the memory slot set and document indexes of the document set; The memory slot set is subjected to semantic alignment processing by a converter module, the memory slot set is mapped to an input space of a question and answer model, and a global target embedding representation is generated; A query question is received, and the query question is subjected to semantic representation processing, and a question embedding representation is generated; The question embedding representation is matched with the global target embedding representation by a retrieval module, a target memory slot is determined from the global target embedding representation according to a matching result, and a candidate document segment corresponding to the target memory slot is acquired according to the mapping relationship; The candidate document segment is input into the compressor module and the converter module, and a segment target embedding representation corresponding to the candidate document segment is generated; The segment target embedding representation is fused with the question embedding representation to form a question and answer input representation; The question and answer input representation is processed by a question and answer model, and a question and answer response is generated.

[0006] Further, to achieve the above object, the application provides a question and answer device based on memory slots and semantic alignment, comprising: A knowledge base management module is configured to acquire a document set, and store the document set and corresponding metadata in a knowledge base; A compressor module is configured to subject the document set in the knowledge base to multi-layer semantic aggregation processing by the compressor module, generate a memory slot set corresponding to the document set, and establish a mapping relationship between the memory slot set and document indexes of the document set; A converter module is configured to subject the memory slot set to semantic alignment processing by the converter module, map the memory slot set to an input space of a question and answer model, and generate a global target embedding representation; A question representation module is configured to receive a query question, and subject the query question to semantic representation processing to generate a question embedding representation; A retrieval module is configured to match the question embedding representation with the global target embedding representation by the retrieval module, determine a target memory slot from the global target embedding representation according to a matching result, and acquire a candidate document segment corresponding to the target memory slot according to the mapping relationship; The fragment embedding generation module is configured to input the candidate document fragment into the compressor module and the converter module to generate a fragment target embedding representation corresponding to the candidate document fragment. The feature fusion module is configured to fuse the fragment target embedding representation and the question embedding representation to form a question and answer input representation. The question and answer model module is configured to process the question and answer input representation through a question and answer model to generate a question and answer response.

[0007] Further, to achieve the above object, the present application also provides a computer device, which comprises a memory, a processor and a memory slot and semantic alignment based question and answer program stored in the memory and executable on the processor, and the memory slot and semantic alignment based question and answer program, when executed by the processor, implements the steps of the memory slot and semantic alignment based question and answer method as described above.

[0008] Further, to achieve the above object, the present application also provides a non-volatile computer readable storage medium, which stores a memory slot and semantic alignment based question and answer program, and the memory slot and semantic alignment based question and answer program, when executed by a processor, implements the steps of the memory slot and semantic alignment based question and answer method as described above.

[0009] Beneficial effects: The present application relates to the technical field of semantic analysis, and can be applied to business scenarios such as financial technology and medical health. A memory slot and semantic alignment based question and answer method, device, equipment and medium are disclosed, which comprise: obtaining a document set and storing it in a knowledge base, performing multi-layer semantic aggregation on the document set in the knowledge base to generate a memory slot set and establishing a mapping relationship; performing semantic alignment processing on the memory slot set to generate a global target embedding representation; receiving a query question and generating a question embedding representation; matching the question embedding representation and the global target embedding representation through a retrieval module, determining a target memory slot and obtaining a candidate document fragment; compressing and converting the candidate document fragment to generate a fragment target embedding representation; fusing the fragment target embedding representation and the question embedding representation to form a question and answer input representation; and processing the question and answer input representation through a question and answer model to generate a question and answer response. The present application introduces a multi-layer semantic aggregation and embedding alignment mechanism in the question and answer generation process, realizes context compression and semantic information fidelity of document content, and reduces the computational complexity of long text reasoning. Through the embedding space based retrieval and fusion process, the positioning and semantic matching ability of the question and answer model to key knowledge is improved, so that the reasoning efficiency is significantly improved while the question and answer accuracy and consistency are maintained. BRIEF DESCRIPTION OF DRAWINGS

[0010] The present application will be further described below with reference to the accompanying drawings and embodiments. In the drawings: Figure 1An application environment schematic diagram of the question and answer method based on memory slot and semantic alignment in an embodiment of the present application; Figure 2 A flowchart of the question and answer method based on memory slot and semantic alignment in an embodiment of the present application; Figure 3 A function module schematic diagram of the question and answer device based on memory slot and semantic alignment in a preferred embodiment of the present application; Figure 4 A structure schematic diagram of a computer device in an embodiment of the present application; Figure 5 Another structure schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0011] It should be understood that the specific embodiments described herein are merely exemplary and are not intended to limit the present application.

[0012] The question and answer method based on memory slot and semantic alignment provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 , wherein the client communicates with the server through the network. The server can obtain the document set through the client and store it in the knowledge base, perform multi-layer semantic aggregation on the document set in the knowledge base to generate a memory slot set and establish a mapping relationship, perform semantic alignment processing on the memory slot set to generate a global target embedding representation, receive a query question and generate a question embedding representation, match the question embedding representation and the global target embedding representation through a retrieval module, determine a target memory slot and obtain a candidate document segment, perform compression and conversion on the candidate document segment to generate a segment target embedding representation, fuse the segment target embedding representation and the question embedding representation to form a question and answer input representation, and process the question and answer input representation through a question and answer model to generate a question and answer response. The present application introduces multi-layer semantic aggregation and embedding alignment mechanism in the question and answer generation process, realizes context compression and semantic information fidelity of the document content, and reduces the computational complexity of long text reasoning. Through the retrieval and fusion process based on the embedding space, the positioning and semantic matching ability of the question and answer model to the key knowledge is improved, so that the reasoning efficiency is significantly improved while the accuracy and consistency of the question and answer are maintained. The client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be realized by an independent server or a server cluster composed of multiple servers. The present application will be described in detail through specific embodiments.

[0013] Please refer to Figure 2 , Figure 2A flowchart of an embodiment of the memory slot and semantic alignment based question answering method provided by the present application is shown. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that shown.

[0014] As shown in Figure 2 the memory slot and semantic alignment based question answering method provided by the present application includes the following steps: S10, obtaining a document set and storing the document set and corresponding metadata in a knowledge base; In this embodiment, in the data processing system, first, the document set is obtained and stored in the knowledge base together with the metadata. The document set generally refers to a plurality of text data sets that have relevance or can be used together for semantic reasoning. These texts can come from structured databases, business archives, policy provisions, operation instructions or standard specifications, etc. The document set can be obtained through various input paths, including batch import from a data storage system, automatic crawling through a network interface, or selection and upload by a user interface. Each document needs to be verified for integrity and checked for coding format before being imported, to ensure that the character set, paragraph level and data field are consistent in the storage system.

[0015] Metadata is descriptive information of the document set, including the identifier, creation time, author, version number, topic label, language and scope of application of the document, etc., used to support subsequent semantic indexing and retrieval process. The metadata can be automatically generated by a parsing engine when the document is imported, or supplemented by a manual annotation system. To improve data retrieval and update efficiency, the metadata and the document content are bound in a key-value mapping manner to form an indexable document description table.

[0016] The knowledge base storage module undertakes the task of structured organization and management of the document set and metadata. The knowledge base can be designed with a hierarchical data structure, with the document content, semantic index and metadata information stored in different logical layers, and the multi-layer mapping realized through a unique document identifier. During data storage, a partitioning strategy is introduced, and the documents are stored in groups according to the document type, source or update time, for parallel retrieval and distributed computing. To maintain data consistency and traceability, the knowledge base management module performs a verification process on the imported data, including hash verification, redundancy detection and version conflict judgment, to ensure that the data can be accurately referenced after being written.

[0017] The semantic index structure of the knowledge base can be constructed based on an inverted list or a vector space model, and an efficient semantic retrieval path is established by binding the keywords or embedding vectors of the document with their index positions. The data writing process usually includes text parsing, feature extraction, index construction and storage operation. The parsing engine determines the text boundary through morphological and syntactic analysis, the feature extraction module generates multi-layer semantic representation, the index module generates retrievable index nodes according to the text weight calculation, and finally the storage manager writes into the database.

[0018] The embodiment realizes centralized management and efficient indexing of information resources by constructing a unified knowledge structure containing a document set and metadata in the data import stage; combined with semantic indexing and version control strategy, the subsequent question and answer model can quickly access the required context in a large-scale text environment, reduce the computational burden of long text processing and improve the semantic consistency.

[0019] S20, a plurality of semantic aggregation processes are performed on the document set in the knowledge base through the compressor module to generate a memory slot set corresponding to the document set, and a mapping relationship is established between the memory slot set and the document index of the document set; In the embodiment, the compressor module first performs semantic blocking on the document set, and divides each document into independent segment units. This process can determine the blocking boundary by combining syntactic structure analysis and information density measurement to ensure that each segment contains a relatively complete semantic unit. Each segment is embedded and represented by a semantic aggregation network, which uses a multi-head attention mechanism to calculate global dependencies from word vector sequences and generates high-dimensional semantic vectors. Subsequently, the semantic core is extracted at the sentence level, section level and chapter level through layer-by-layer aggregation strategy, so that the multi-layer representation forms convergent features in the semantic space.

[0020] The generated memory slot set is a high-density semantic storage unit output by the compressor module, and each memory slot corresponds to a piece of semantically aggregated document information. The memory slot can adopt a fixed-dimensional vector structure for subsequent retrieval, comparison and semantic reasoning. To facilitate indexing and tracking, the system extracts the corresponding document index from the knowledge base while generating the memory slot set. The document index usually includes a document identifier, a segment start and end position, a timestamp and a metadata reference address.

[0021] After the aggregation is completed, the compressor module generates a mapping relationship between the memory slot set and the document index through the mapping generation unit. The mapping relationship is stored in the form of a hash table or a key-value mapping, which ensures that each memory slot can be quickly traced back to the original document segment, and realizes the consistency of semantic compression and physical positioning. The mapping relationship construction process includes index coding, hash key generation and bidirectional reference binding. The mapping table supports both semantic-based vector retrieval and structured index-based fast path access.

[0022] By establishing the multi-layer semantic aggregation and mapping relationship, the embodiment can significantly compress the input size while maintaining the integrity of the document semantics, and realize the structured storage of the context semantics. The processing process not only reduces the reasoning complexity when a long text is input, but also enables the question and answer model to quickly locate the relevant information through the memory slot, thereby improving the reasoning efficiency and the consistency of the question and answer.

[0023] In S30, the converter module performs semantic alignment processing on the memory slot set, maps the memory slot set to an input space of a question and answer model, and generates a global target embedding representation; In the embodiment, in the input stage, the memory slot set is composed of a plurality of high-dimensional semantic vectors, and each vector contains paragraph-level semantic compression information. The converter module first transforms these high-dimensional representations to a target dimensional space through a linear projection layer to match the vector dimension required by the input of the question and answer model. The linear projection layer is usually composed of a weight matrix and a bias term, and realizes the linear mapping of the feature space through matrix multiplication, so that the semantic distribution tends to the feature distribution of the model input in a statistical sense. This process can be regarded as a joint operation of feature dimension reduction and channel recalibration.

[0024] Subsequently, a nonlinear activation function performs nonlinear transformation on the linear projection result to enhance the expression ability of the model for complex semantic boundaries. Common activation functions include ReLU, GELU or SiLU, etc., which control the smoothness and information sparsity of the mapping in different scenarios, so that the high-dimensional semantic information still maintains the distinguishability after mapping. Through this process, the converter module can reconstruct the semantic energy distribution in the compressed low-dimensional space, avoiding feature degradation caused by linear mapping.

[0025] In the dimension mapping stage, the system maps the activated embedding representation to the input dimension system of the question and answer model using a mapping matrix. The mapping matrix can be pre-trained or dynamically learned according to the encoding specification of the question and answer model, and is used to correct the scale difference of different semantic levels. Dimension mapping not only ensures the consistency of numerical dimensions, but also realizes feature alignment between different memory slots through scale normalization operation, so that the overall semantic space distribution is smooth, which is beneficial to subsequent context fusion.

[0026] To further ensure semantic consistency, the converter module introduces a semantic co-distillation mechanism, which aligns the learning of the semantic distribution with the teacher model, so that the converted embedding can approximate the standard semantic domain in the statistical semantic distribution. This mechanism measures the difference between the output of the student model (the current converter module) and the teacher model distribution by minimizing the distribution distance function (such as KL divergence or cosine similarity). The semantic co-distillation process is gradually optimized in continuous training, so that the output global target embedding representation has both compression efficiency and semantic interpretability.

[0027] Finally, the memory slot set after multi-layer transformation and semantic alignment is integrated into a global target embedding representation. The representation is a high-level abstraction of the overall semantic structure of the knowledge base, which can be directly called in the question and answer model for matching and semantic reasoning with the question embedding representation.

[0028] Through the semantic alignment and mapping mechanism, the embodiment can establish a unified expression system between the compressed semantic space and the input space of the question and answer model, so that the question and answer model can directly understand the high-density information from the compressed semantics, thereby improving the context association and semantic consistency in the question and answer generation process, significantly reducing the semantic drift problem and reducing the input complexity of the model.

[0029] S40, receiving a query question, and performing semantic representation processing on the query question to generate a question embedding representation; In the embodiment, in the semantic representation processing process, the system first performs word segmentation and tokenization operation to split the query text into basic semantic units that can be recognized by the encoder. The word segmentation stage combines the field dictionary and the subword segmentation rule to ensure that the field-specific terms (such as medical terms and contract clause terms) remain semantically complete during processing. The tokenized sequence is sent to the embedding layer, which maps each token to a fixed-dimensional word vector through table lookup or neural embedding calculation. The word vector not only reflects the word meaning, but also captures the intra-sentence dependency through the context window, thereby providing a basic representation for subsequent semantic encoding.

[0030] The embedded word vector sequence is input into the encoder module of the pre-trained language model to perform context encoding processing. The encoder can use the Transformer structure to calculate the dependency weight between words and words through the multi-head self-attention mechanism to extract context features from a global range. The encoder performs weighted aggregation and residual normalization on the input sequence in each layer to capture semantic level information and inter-sentence logic. Finally, the model outputs a context-enhanced representation of each token, so that the same semantic unit obtains differentiated encoding under different contexts.

[0031] The context-enhanced representation is compressed into a single vector through sequence pooling operation. The pooling method can be average pooling or weighted pooling, which generates an overall semantic vector by aggregating all token features. To avoid statistical bias introduced by different sequence lengths, a mask mechanism can be used to ignore invalid padding positions during aggregation. The generated initial question vector then enters the normalization stage, which adjusts its distribution through layer normalization or batch normalization method to keep the dimensions stable in numerical scale.

[0032] The normalized question vector is dimensionally adjusted by a linear projection layer to be fully consistent with the global target embedding representation in vector dimension. The projection layer can realize vector space rotation and scale mapping through the learned weight matrix, ensuring that the semantic representation can be compared in the same embedding domain. The final output of the question embedding representation is the input form of the question and answer model for matching and reasoning, which retains the semantic logical structure and context dependency of the query text.

[0033] Through the semantic representation processing process, the query question is converted into a vector representation with context dependency and domain semantics, enabling the question and answer model to achieve accurate matching and semantic reasoning in the embedding space. This processing method effectively improves the model's understanding ability for complex queries, avoids the semantic drift problem caused by keyword matching, and maintains semantic stability and logical consistency under long sentence structure.

[0034] S50, match the question embedding representation with the global target embedding representation through the retrieval module, determine the target memory slot from the global target embedding representation according to the matching result, and obtain the candidate document segment corresponding to the target memory slot according to the mapping relationship; In this embodiment, the system pairs the input question embedding representation with the global target embedding representation one by one and inputs them into the similarity analysis unit. This unit usually uses cosine similarity, dot product similarity or Euclidean distance as a measure function to calculate the semantic similarity between the question vector and each memory slot vector. Cosine similarity reflects the semantic direction consistency by calculating the cosine of the angle between two vectors, which is more suitable for distribution comparison in high-dimensional embedding space. Dot product similarity retains vector length information and can strengthen the influence of significant semantic dimensions.

[0035] After the calculation is completed, the retrieval module forms a similarity score set, where each score corresponds to the matching degree of a memory slot. Then, the sorting unit sorts the scores in descending order to generate a memory slot similarity sorting list. The sorting process can consider the word meaning matching confidence, context dependency strength and semantic level weight in combination with the weighting strategy to improve the sorting accuracy. To prevent noise samples from interfering with the results, the system can use a Top-K screening mechanism to only retain the top several candidate memory slots with the highest similarity to form a candidate set.

[0036] The candidate memory slot set is then passed to the selection unit. The selection unit determines the final target memory slot by setting a similarity threshold or a dynamic threshold mechanism based on the cluster center distance. This process can use a two-stage screening method: first, determine the coarse matching candidates in the global space, and then perform fine-grained discrimination in the local semantic space to ensure that the selected memory slot is consistent with the query question in terms of semantic structure and context logic.

[0037] After the target memory slot is determined, the system calls the mapping query unit to obtain the document index corresponding to the memory slot from the knowledge base according to the pre-established mapping relationship. The mapping relationship is composed of a key-value pair structure, where the key is the memory slot identifier, and the value is the document identifier and location information. The retrieval module uses the index to locate the specific document segment and extracts the original text or vectorized content of the segment from the knowledge base storage to form a candidate document segment set. These segments serve as input for subsequent semantic fusion and question answering generation, providing high-relevance context support for the model.

[0038] By performing matching and mapping queries in the embedding space, the embodiment can achieve cross-level semantic alignment and accurate retrieval, significantly improving the efficiency of question answering response in a long document environment. This process effectively reduces the error of traditional keyword-based retrieval, enhances the semantic recognition and context positioning ability of the question answering model in domain text, and makes the question answering generation more accurate and logically consistent.

[0039] S60, inputting the candidate document segment into the compressor module and the converter module to generate a segment target embedding representation corresponding to the candidate document segment; In the embodiment, the compressor module first receives the original text content of the candidate document segment, performs word segmentation and word embedding processing, and converts the text into a multi-dimensional vector representation. The same word segmentation rules and dictionaries as in the knowledge base construction stage are used in this stage to ensure consistency of the embedding space in different stages. After word segmentation, the module extracts the semantic dependency structure within the segment through a context encoding mechanism. Common implementations are Transformer encoder or bidirectional LSTM, which calculate the correlation weights between word units through multi-head attention to obtain the context-enhanced representation of the segment, enabling the model to capture the logical relationships and context dependencies between terms.

[0040] After completing the context encoding, the compressor module performs semantic aggregation operations to integrate word-level and sentence-level feature vectors at the paragraph level to generate a segment memory slot representation. Semantic aggregation can be achieved through attention-weighted pooling or hierarchical aggregation, where the attention weights are dynamically determined by the importance of internal information in the segment, thereby strengthening the representation of key semantic areas, such as strengthening the representation of restrictive clauses in financial contracts or strengthening the representation of disease descriptions and diagnosis information in medical records. The output of this stage is the compressed and aggregated segment memory slot.

[0041] After receiving the segment memory slot, the converter module performs semantic mapping and feature reconstruction operations to ensure that the segment-level semantics can be effectively utilized in the input space of the question and answer model. Inside the module, the segment memory slot is first dimensionally mapped by a linear projection layer to convert it into an embedding dimension acceptable to the question and answer model. Then, a nonlinear activation function such as GELU or ReLU is used to transform the projection result, enhancing the nonlinear features of the semantic distribution. This process not only improves the model's ability to recognize complex semantic patterns, but also prevents feature collapse caused by linear projection.

[0042] Next, the converter module further adjusts the embedding structure by a high-dimensional semantic mapping matrix to align the segment semantics with the global target embedding space. To improve alignment accuracy, the system can introduce a semantic distillation mechanism to make the segment target embedding representation close to the statistical structure of the global embedding in terms of semantic distribution. Through this cross-layer mapping and distillation, the segment target embedding representation preserves local semantic details while achieving global interpretability and domain consistency. The final generated segment target embedding representation is the input unit for context fusion by the question and answer model.

[0043] By jointly using the compressor module and the converter module to process the candidate document segments, the present embodiment can achieve high-density information embedding while preserving semantic integrity, so that the segment target embedding representation contains both domain context information and perfect alignment with the semantic space of the question and answer model. This mechanism effectively reduces redundant text input, reduces computational cost, and significantly improves the context consistency and semantic accuracy of the question and answer generation.

[0044] S70, fusing the segment target embedding representation and the question embedding representation to form a question and answer input representation; In the present embodiment, after receiving the question embedding representation and the segment target embedding representation, the fusion module first performs splicing processing to concatenate the two groups of vectors in the feature dimension. The splicing operation preserves the complete features of the two semantic sources and provides full input to the subsequent fusion layer. Splicing can use simple vector splicing or a double-channel splicing structure to distinguish the two groups of embeddings as different semantic channels to enhance the model's ability to recognize source differences. The spliced vector forms a fusion input sequence.

[0045] The linear transformation layer of the fusion module performs dimension adjustment on the fusion input sequence. This transformation layer linearly combines the splicing result through a weight matrix to project the semantic features of different sources to a unified dimension in a new space. The linear transformation layer learns the optimal mapping ratio between the question embedding and the segment embedding in the training stage, thereby balancing the importance of the two semantics and avoiding semantic drift caused by input imbalance.

[0046] The dimension-transformed embedding representation is input to an activation function layer, which enhances the feature expression capability through nonlinear transformation. Common activation functions include GELU, Tanh, or ReLU, which introduce nonlinear mapping to capture complex semantic interaction relationships. For example, when the question is "Does insurance claim include a specific accident?", the activation function can strengthen the nonlinear dependence features of the "claim" and "accident" semantic dimensions, making it easier for the model to focus on relevant information in subsequent stages.

[0047] The activated embedding representation is input to an attention mechanism layer, which is used to weight the fused semantic vectors. This layer assigns attention weights by calculating the relevance scores between the question embedding and the segment embedding, with higher weights for highly relevant parts, thereby highlighting the segment information directly related to the query question. Attention weight calculation can be based on self-attention structure or bidirectional interactive attention mechanism, where interactive attention can significantly improve the fusion accuracy of cross-semantic source features.

[0048] Finally, the fusion module performs standardization processing through the output layer to stabilize the fusion results in numerical distribution. Standardization processes such as layer normalization or batch normalization ensure that the question and answer input representation maintains gradient balance and convergence stability during subsequent model inference. After the above processing, the system generates a question and answer input representation that fuses the question intent and candidate document semantics, serving as the input basis for the question and answer model to perform inference and response generation.

[0049] The embodiment achieves dynamic alignment of question intent and context content at the semantic level by multi-level fusion of segment target embedding representation and question embedding representation, making the question and answer input representation have both semantic integrity and information relevance. This fusion mechanism significantly improves the understanding accuracy and response coherence of the question and answer model, reducing information loss and semantic drift problems, and laying a high-quality input foundation for subsequent question and answer generation.

[0050] S80, processing the question and answer input representation through the question and answer model to generate a question and answer response.

[0051] In this embodiment, the question and answer model first receives the question and answer input representation and extracts deep features from the input through the encoder layer. The encoder layer is usually based on the Transformer architecture and consists of multi-head self-attention mechanism and feedforward network, which is used to capture the dependency relationships between different semantic units in the question and answer input. The multi-head attention mechanism dynamically assigns semantic focus to key words or high-relevance context areas by calculating the attention weight matrix between vectors, allowing the model to accurately identify information focus in long sequences. For example, in insurance clause question and answer, the model will automatically focus on high-weight semantic areas such as "compensation conditions" and "exemption range".

[0052] After encoding, the model enters the attention mechanism layer. This layer establishes an information channel between the encoded representation and the decoded input through a cross-attention mechanism, enabling the decoder to reference the semantic content of the question and context in real time when generating the answer. This process strengthens the linkage between question intent and fragmented knowledge, ensuring that the generated answer does not deviate from the contextual logic. The cross-attention score is normalized using the Softmax function to determine the semantic contribution ratio of each input tag in the output generation process.

[0053] Subsequently, the decoder layer performs sequence generation based on the attention-weighted context vector. The decoder uses an autoregressive structure to progressively output the word prediction probability distribution; each word generated in each step serves as input for the next, forming a word-by-word generation chain. The model maximizes the conditional probability... The most likely sequence of lexical terms is generated in a way that forms a continuous semantic sentence.

[0054] During the generation process, the output layer maps the probability distribution obtained from the decoder to a vocabulary space and selects the candidate word with the highest probability as the output. To prevent repetition and semantic incoherence, the system can employ beam search or temperature sampling strategies to ensure that the output is both diverse and logically consistent.

[0055] The generated initial question-and-answer responses are optimized by a post-processing module. This module includes format normalization, entity standardization, and semantic consistency verification. Format normalization ensures the output conforms to a preset language template; entity standardization identifies and standardizes domain-specific terms, such as medical drug names or insurance policy names; semantic consistency verification compares the semantic matching between the model-generated content and candidate document fragments, eliminating potentially misleading or logically contradictory answers. The final output question-and-answer responses are structured, accurate, and semantically consistent.

[0056] Through the aforementioned structure, this embodiment achieves a collaborative fusion of semantic understanding, logical reasoning, and natural language generation in the question-answering model. This mechanism significantly improves the response accuracy and interpretability of the question-answering system, enabling the generation of consistent and logically coherent answers in highly complex document environments, while reducing the burden of manual intervention and post-review.

[0057] In one embodiment, step S20 above includes: S201, The document set in the knowledge base is segmented by the compressor module to generate word-level embedded representations; S202, the word-level embedding representations are aggregated through the attention mechanism in the compressor module to generate sentence-level embedding representations; S203, aggregating the sentence-level embedding representations through an attention mechanism in the compressor module to generate paragraph-level embedding representations; S204, aggregating the paragraph-level embedding representations through a pooling function in the compressor module to generate a set of memory slots; S205, extracting a document index of the set of documents; S206, establishing a mapping relationship table between the set of memory slots and the document index, which records the association of each memory slot with a corresponding document index.

[0058] In this embodiment, the compressor module first receives the original text stream and structured annotations of the set of documents, performs character-to-token segmentation to obtain computable units. Tokenization processing can be based on dictionary segmentation, subword segmentation or mixed segmentation strategy, combined with regularization rules and punctuation retention rules to ensure that numbers, currency symbols and time expressions remain in place and can be traced back. After completing the segmentation, the compressor module retrieves a vector word table for each token or generates a vector representation via an embedding projection layer to form a word-level embedding representation; this representation retains word shape, meaning and position information, which can be enhanced by position encoding to inject sequence bit order and intra-segment offset to enhance sequence distinguishability. To reduce the influence of noise and spelling variants, the word-level embedding representation can be superimposed with normalization and light noise reduction projection to stabilize the convergence of subsequent attention calculations.

[0059] On the basis of the word-level representation, the attention mechanism inside the compressor module weights and aggregates the word-level embedding representations within the same sentence to generate a sentence-level embedding representation. This attention mechanism uses a multi-head structure to depict different semantic relationships such as synonym substitution, modification dependency and long-distance reference, calculates attention weights through three sets of linear mappings of query, key and value, uses a mask matrix to limit cross-sentence interference, and uses residual connection and feedforward network to enhance non-linear representation. The sentence-level embedding representation not only covers word surface information, but also explicitly encodes syntactic dependency and semantic focus; to improve robustness, random dropout and layer normalization can be added during the training phase to control the drift of internal activation distribution.

[0060] For the semantic organization within the paragraph, the compressor module continues to call the attention mechanism to hierarchically aggregate the sentence-level embedding representations to obtain a paragraph-level embedding representation. This stage encodes the clues of sentence order, theme progression and causal transition into the attention graph within the paragraph; to suppress redundancy, sparse attention or windowed attention is introduced into the mechanism to concentrate the weights on adjacent or theme-consistent sentence pairs; at the same time, low-contribution sentences are suppressed through a gating unit to retain key argument links within the paragraph, and the paragraph-level embedding representation thus has higher semantic density and lower redundancy, which is suitable for subsequent compression.

[0061] To complete the extraction from the section to the long-term memory, the compressor module applies a pooling function on the paragraph-level embedded representation to generate a set of memory slots. The pooling function can adopt average pooling, maximum pooling, attention pooling or a combination of multi-scale pooling, the former emphasizes overall statistics, the latter captures peak clues, attention pooling is used to further highlight the core points within the section; the multi-scale strategy converges at the sentence, section and chapter three granularities in parallel, and finally splices or weights to obtain the vector of each memory slot. The memory slot set is output in fixed dimensions and configured with parameters such as information entropy threshold and aggregation temperature to adjust the balance between compression rate and semantic fidelity; when it is detected that the redundancy of a certain slot is high or the mutual information is lower than the threshold, the compressor module triggers the re-aggregation or merging operation to reduce the occupation of invalid slots.

[0062] In order to establish a traceable link across vectors and original text positions, a document index is extracted from the document set, which contains a document identifier and position information covering paragraph start and end, sentence sequence number, word offset and version label. Then a mapping relationship table is constructed to establish one-to-one or one-to-many association records between each slot in the memory slot set and the corresponding document index entry. The mapping relationship table adopts a key-value structure or a columnar storage structure to support high concurrency retrieval, and contains redundancy check fields such as timestamp, source channel and checksum to ensure consistency. Integrity check and constraint check are performed during writing, such as uniqueness constraint to ensure that the same memory slot does not conflict with different document identifiers, and referential integrity constraint to ensure that each mapping can be traced back to a valid document index entry; for cross-version scenarios, the mapping relationship table records version evolution chain to support historical playback and differential update.

[0063] The embodiment extracts information points from bottom to top through multi-layer semantic aggregation, and combines attention and pooling at word level, sentence level and paragraph level, which significantly reduces redundancy while maintaining semantic continuity; the memory slot set carries key information across documents in fixed dimensions, and the mapping relationship table establishes a traceable link from vectors to original text positions, thereby achieving adjustable balance among compression rate, retrieval efficiency and semantic fidelity. Subsequent retrieval and question answering generation can efficiently locate evidence sources and maintain consistent expression.

[0064] In one embodiment, the above step S30 comprises: S301, performing linear transformation processing on the memory slot set through a linear projection layer in the converter module to generate an intermediate embedded representation; S302, performing non-linear transformation processing on the intermediate embedded representation through a non-linear activation function in the converter module to generate an activated embedded representation; S303, performing dimension mapping processing on the activated embedded representation through a mapping matrix in the converter module to generate a preliminary target embedded representation; S304, aligning the preliminary target embedding representation with semantic distributions of the teacher model through a semantic co-distillation mechanism in the converter module, to generate a global target embedding representation.

[0065] In this embodiment, after the converter module receives the vector sequence of the memory slot set, it first performs an affine mapping through a linear projection layer to unify the input vector to the alignment base before transformation, obtaining an intermediate embedding representation. The parameters of the linear projection layer are obtained through end-to-end training, and the correspondence between batch samples and time positions is not disturbed during projection; to suppress the gradient instability caused by numerical amplification, a light normalization and residual bypass can be connected before and after the linear projection layer, so that the intermediate embedding representation is stable and controllable in scale and variance. The intermediate embedding representation still retains the semantic principal components of the memory slot set, but completes the preliminary rearrangement in the channel dimension, creating space for nonlinear modeling.

[0066] The nonlinear activation function applies a nonlinear transformation to the intermediate embedding representation element by element to obtain an activated embedding representation. Activation can use piecewise linear or smooth activation, or can combine a gating mechanism to form selective enhancement and inhibition in the channel dimension; to avoid extreme value saturation, dynamic amplitude modulation can be performed with learnable scaling and translation parameters. The activated embedding representation introduces high-order feature combination capability without changing the sequence timing structure, so that the subsequent dimension mapping can work in a more separated semantic coordinate.

[0067] The mapping matrix completes the dimension mapping to the target space based on the activated embedding representation, outputting a preliminary target embedding representation. The mapping matrix can be designed as a single-layer full connection, or can use block sparse or low-rank approximation to reduce computational overhead; in the cross-model collaboration scenario, shared sub-matrices and dedicated sub-matrices can be introduced for memory slots of different sources to simultaneously obtain cross-domain consistency and source domain specificity. To improve numerical stability, scale calibration and channel re-labeling are combined before and after mapping, and structured dropout is used to improve generalization in the training phase. The preliminary target embedding representation is consistent in dimension with the input space of the question and answer model, but the semantic distribution is not fully aligned and needs to be further corrected at the distribution level.

[0068] The semantic co-distillation mechanism aligns the preliminary target embedding representation with the semantic distribution of the teacher model to output the global target embedding representation. The teacher model provides a semantic distribution reference for the same input corpus, which can be a static frozen teacher or a moving teacher updated by an exponential smoothing method. To avoid excessive dependence on the teacher and reduce diversity, the co-distillation sets temperatures and weights at the sequence level and the sample level, respectively, to increase alignment strength at high uncertainty locations and preserve the freedom of student representation at confident locations. The alignment process measures the distribution difference and uses it to build a training signal. Combined with a domain-oriented masking strategy, strong alignment is performed only in channels closely related to retrieval and question answering decisions, and weak alignment is performed in noise-sensitive channels. The global target embedding representation meets the geometric requirements of the question answering model input space and is statistically compatible with the prior of the teacher model, enabling stable entry into subsequent attention and decoding processes during the reasoning phase. To ensure traceability, the converter module caches the global target embedding representation and the source memory slot set index, and records the parameter snapshot identifier of the linear projection layer, the activation function, and the mapping matrix at this time, so that it can be quickly restored and compared when drift occurs.

[0069] The three-level transformation composed of the linear projection layer, the nonlinear activation function, and the mapping matrix gradually integrates the vector scale, expression ability, and dimension consistency layer by layer, so that the memory slot set quickly falls into the input space available to the question answering model; the semantic co-distillation mechanism further aligns at the distribution level, so that the global target embedding representation has both geometric accessibility and statistical compatibility, thereby reducing the reasoning fluctuations caused by mismatches at the cross-model interface, reducing the dependence on direct input of long text, and improving the available information density and response stability of compressed context entering the question answering model.

[0070] In one embodiment, the above step S40 includes: S401, receiving a query question, performing word segmentation and tokenization processing on the query question to generate a token sequence; S402, performing vector mapping processing on the token sequence by the embedding layer of the pre-trained language model to generate a token embedding sequence; S403, performing context encoding processing on the token embedding sequence by the Transformer encoder of the pre-trained language model to generate a context-enhanced sequence representation; S404, performing sequence pooling processing on the context-enhanced sequence representation to generate an initial question vector; S405, performing distribution standardization processing on the initial question vector by the layer normalization module to generate a normalized question vector; S406, performing dimension adjustment processing on the normalized question vector by the linear projection layer to generate a question embedding representation matching the dimension of the global target embedding representation.

[0071] In this embodiment, after the system receives the query question, firstly, the word segmentation and tokenization processing is performed, and the string input in natural language is converted into a calculable token sequence. The word segmentation stage adopts the word segmentation device corresponding to the language model vocabulary table to split the input text at the sub-word level, so that the same word can be consistently coded in different contexts; special marks such as start, end and separator are retained in the tokenization process to ensure that the model can distinguish the query range and the structure boundary. Through this standardization preprocessing, the query text is converted into stable and reproducible discrete input units.

[0072] Subsequently, the embedding layer of the pre-trained language model performs vector mapping processing on the token sequence. The embedding layer maps the discrete index into a continuous dense vector through table lookup operation or linear mapping, and each token vector carries potential information of word meaning, part of speech and context dependence in the semantic space. In order to improve the semantic discrimination ability, the embedding layer combines position encoding and semantic label encoding, so that the model can capture word sequence and semantic proximity at the same time in the encoding stage. The embedding output constitutes a token embedding sequence, which is the input of subsequent context encoding.

[0073] In the Transformer encoder stage, the token embedding sequence is input into the multi-layer self-attention network for context modeling. Each layer of the encoder contains a multi-head self-attention unit, a feedforward network and a layer normalization module. The self-attention mechanism captures long-distance dependency relationships by calculating the attention weight between any two tokens, so as to integrate the global context at one time. The multi-head mechanism focuses on different semantic subspaces such as syntax, sentiment or logical relationship through multiple independent attention projection matrices. After multi-layer stacking, a context-enhanced sequence representation is generated, and the representation of each position not only reflects the meaning of the token itself, but also integrates the influence of the whole sentence semantic environment.

[0074] In order to obtain a unified problem representation, the context-enhanced sequence representation is processed by sequence pooling. The pooling method can use average pooling, maximum pooling or attention-based weighted pooling, the former focuses on overall balanced information, the latter highlights key word information, and attention pooling dynamically focuses on the semantic core area. The pooling operation compresses the time dimension, and converts the variable-length sequence into a fixed-dimension initial problem vector. The vector condenses the core intent and semantic features of the query question in the feature space, which is convenient for subsequent model comparison and matching.

[0075] The layer normalization module performs distribution standardization on the initial problem vector, so that the input under different batches and different semantic distributions remains numerically stable. Layer normalization realizes zero mean and unit variance constraint by subtracting mean value and dividing by standard deviation, and introduces learnable scaling and translation parameters to retain distribution flexibility. The normalized problem vector has stable gradient characteristics and high training convergence speed.

[0076] Finally, the linear projection layer maps the normalized problem vector to the dimension matching the global target embedding representation. The parameter matrix of this layer is set according to the dimension of the input space of the question-answering model, and the vector space alignment is achieved through affine transformation. The dimension adjustment not only guarantees the geometric consistency, but also reduces the parameter redundancy through the sparse constraint of the weight matrix. After projection, the problem embedding representation is completely consistent with the global target embedding representation in terms of spatial dimension and semantic scale, and can be directly used in the embedding retrieval or similarity matching stage to realize the same space mapping of the problem semantics and the knowledge base semantics.

[0077] The embodiment realizes continuous and measurable semantic representation of natural language input at the word, sentence and global levels by performing hierarchical embedding and context encoding on the query question. Tokenization and word tokenization ensure the standardization and interpretability of the input, the Transformer encoder realizes global modeling of context dependency, sequence pooling and layer normalization balance between compression and stability, and linear projection realizes semantic space alignment.

[0078] In one embodiment, the above step S50 comprises: S501, determining the cosine similarity between the problem embedding representation and the embedding representation corresponding to each memory slot in the global target embedding representation through the similarity analysis unit in the retrieval module, and generating a set of similarity scores; S502, performing descending order sorting processing on the set of similarity scores through the sorting unit in the retrieval module, and generating a memory slot similarity sorting list; S503, selecting the first pre-set number of memory slots with high similarity scores from the memory slot similarity sorting list through the selection unit in the retrieval module, and generating a candidate memory slot set; S504, determining the target memory slot from the candidate memory slot set according to the mapping relationship through the mapping query unit in the retrieval module; S505, extracting the candidate document fragment from the knowledge base according to the document index corresponding to the target memory slot through the fragment acquisition unit in the retrieval module.

[0079] In this embodiment, the retrieval module receives the question embedding representation and the global target embedding representation. After the similarity analysis unit first performs vector preprocessing, including length normalization and numerical clipping, to avoid the influence of extreme values on the calculation stability. Length normalization projects two vectors onto the unit sphere, and the cosine similarity can be directly given by the vector dot product; numerical clipping limits the input range to reduce the cumulative error of hardware instructions. To improve throughput, similarity calculation adopts batch matrix multiplication, which expands the question embedding representation to batch dimension, and performs parallel operation on the vector library composed of the embedding representation corresponding to each memory slot in the global target embedding representation. The similarity analysis unit outputs a set of similarity scores, along with a position mask and an availability flag, which are used to mask invalid or expired memory slot entries.

[0080] The sorting unit reads the set of similarity scores and sorts the scores in descending order using a stable sorting strategy to generate a memory slot similarity sorting list. Stable sorting ensures that entries with the same score maintain the input order, facilitating alignment with parallel processing rules in downstream selection logic. To reduce latency, the sorting unit can use a local selection algorithm or partial sorting, which skips the tail of the full sorting under the premise of ensuring the order of the first pre-set number of entries. The sorting unit also records the threshold boundary, which is used to quickly determine whether the candidate set needs to be expanded or contracted to adapt to different query difficulties and resource occupancy.

[0081] The selection unit performs truncation on the first pre-set number of memory slots in the memory slot similarity sorting list to generate a candidate memory slot set. If the sorting list is not long enough, the selection unit fills in the missing entries according to the availability flag; if the scores are tied and cross the truncation boundary, the selection unit processes according to the preset tie-breaking rules, such as prioritizing the most recently updated memory slot or prioritizing the larger span of the source document index set. To improve information coverage, the selection unit can introduce lightweight diversity suppression, which ensures that high-score entries are prioritized while avoiding candidate entries from the same document location window, thereby reducing redundancy.

[0082] The mapping query unit reads the candidate memory slot set and retrieves the document index corresponding to each memory slot according to the mapping relationship. The mapping relationship maintains the association between memory slots, document identifiers, and location information in a structured table, supporting one-to-one or one-to-many mapping. The mapping query unit first performs integrity checking to check if there are missing fields or invalid links in the mapping entries; then it attempts to locate the target memory slot according to the similarity sorting order. If one-to-many mapping is established, it determines the final target memory slot and locks the corresponding document index set based on the similarity upper and lower threshold values and the source consistency rules. To maintain traceability, the mapping query unit includes a mapping link summary in the output, which records the correspondence between the memory slot identifier and the document index.

[0083] The fragment acquisition unit extracts the candidate document fragments from the knowledge base according to the document index corresponding to the target memory slot. The extraction process includes position analysis and window segmentation operations. Position analysis reads document identifiers and position information, supporting multiple positioning methods such as character offset, paragraph number, or chapter path. Window segmentation expands the context forward and backward from the positioning point, uses an adaptive window length strategy, and combines document types and sentence boundary identifiers to control fragment length while ensuring semantic integrity. The fragment acquisition unit performs light cleaning on the extracted content, unifies the encoding format, removes invisible characters, and preserves structural markers. For continuous content across pages or sections, the unit performs fragment splicing and deduplication, outputs the candidate document fragments and their associated document indexes, and ensures that subsequent processing can directly associate with the original source.

[0084] The similarity analysis unit performs batch cosine calculation and numerical stabilization processing, and the matching link maintains controllable latency in a high-concurrency scenario. The sorting unit performs partial sorting and threshold boundary recording, significantly reducing unnecessary full sorting overhead. The selection unit performs parallel resolution and diversity suppression, improving information coverage while hitting highly relevant memory slots. The mapping query unit performs integrity verification and mapping link summary, ensuring that the correspondence from memory slots to document indexes is verifiable. The fragment acquisition unit performs adaptive window segmentation and cleaning splicing, outputting candidate document fragments that maintain semantic integrity and meet length constraints for downstream processing.

[0085] In one embodiment, the above step S60 includes: S601, inputting the candidate document fragments into the compressor module and the converter module to generate a fragment target embedding representation corresponding to the candidate document fragments, including: S602, performing word segmentation processing on each candidate document fragment by the compressor module to generate a fragment word-level embedding representation; S603, performing context encoding processing on the fragment word-level embedding representation by the compressor module to generate a fragment context-enhanced representation; S604, performing semantic aggregation processing on the fragment context-enhanced representation by the compressor module to generate a fragment memory slot; S605, performing linear projection processing on the fragment memory slot by the converter module to generate a fragment intermediate embedding representation; S606, performing nonlinear activation processing on the fragment intermediate embedding representation by the converter module to generate a fragment activated embedding representation; S607, performing dimension mapping processing on the fragment activated embedding representation by the converter module to generate a fragment target embedding representation.

[0086] In this embodiment, after the candidate document segment enters the compression and alignment pipeline, the first thing is to complete the word segmentation processing in the compressor module to form a segment word-level embedding representation. The word segmentation processing performs cleaning and uniform coding on the white space, punctuation, and special symbols in the natural language text, and then cuts the continuous character sequence into a token sequence according to the tokenization rule. The tokenization rule can be based on a dictionary-driven or sub-word segmentation mechanism, and the corresponding relationship with the original text position is preserved during segmentation to generate a token boundary index, which is used for alignment and backtracking later. The segment word-level embedding representation is generated by combining embedding lookup and linear mapping. The embedding lookup maps each token to a fixed-length vector, and the linear mapping unifies the distribution range of different source vocabularies to obtain a segment word-level embedding representation with consistent shape and stable numerical range.

[0087] The compressor module then performs context encoding processing on the segment word-level embedding representation to obtain a segment context-enhanced representation. The context encoding is implemented in a structure alternating self-attention stacks and feedforward units, with position hints added to characterize relative and absolute sequence relationships, and masks introduced to mask invalid padding bits. To reduce the impact of sequence length on computation, a sliding window and overlap splicing strategy is used for long segments, local context modeling is completed within the window, and the window boundary is merged with weighted overlap in the overlap area to avoid information break caused by sentence boundary crossing. Normalization and residual addition are introduced at the numerical level to improve gradient propagation stability and parallel efficiency, and the output is a segment context-enhanced representation sequence consistent with the segment length.

[0088] After obtaining the segment context-enhanced representation, the compressor module performs semantic aggregation processing to generate a segment memory slot with coarser granularity and directly indexable. Semantic aggregation is implemented in a combination of hierarchical weighting and compression: first, the structure is divided according to the sentence boundary or paragraph boundary, then the representative vector is extracted within the divided unit through attention weighting and pooling function; subsequently, at a higher level, the multi-unit information is compressed into a fixed number of aggregation units through gated fusion, each aggregation unit being a candidate representation of the segment memory slot. To maintain traceability, the segment memory slot retains the start and end offsets and source identification from the segment, and is accompanied by local confidence and coverage description, facilitating subsequent mapping query and quality control. During the aggregation process, repeated content is removed, extremely short units are merged to improve information density, and long-distance dependencies across paragraphs and sentences are supplemented through cross-unit attention to enhance expression ability.

[0089] After the segment memory slot enters the converter module, the first thing is to generate a segment intermediate embedding representation through linear projection processing. Linear projection uses a parameter matrix to perform affine transformation on the segment memory slot in the feature dimension, completes distribution alignment and scale adjustment, and uses parallel matrix multiplication in the batch dimension to improve throughput. To improve numerical stability, light normalization can be combined before and after linear projection to suppress the amplitude difference between different slots, and the segment intermediate embedding representation is obtained from this to obtain a consistent feature distribution.

[0090] After obtaining the segment intermediate embedding representation, the converter module processes the segment intermediate embedding representation with a non-linear activation to generate a segment activation embedding representation. The non-linear activation is used to introduce separability of feature transformation, and to improve the ability to depict complex semantic patterns. To balance expressiveness and stability, an activation function selection and a threshold smoothing strategy are used together, the former provides a non-linear mapping, and the latter suppresses saturation and gradient disappearance caused by abnormally large amplitude. The activation processing is realized in an element-by-element form, maintaining the same parallel granularity as the linear projection, and ensuring the consistency of pipeline execution.

[0091] Finally, the converter module performs a dimension mapping process on the segment activation embedding representation to generate a segment target embedding representation. The dimension mapping is completed by a learnable mapping matrix to align with the question and answer side space, so that the segment target embedding representation, the global target embedding representation and the question embedding representation are comparable and can be fused in the same input space. The dimension mapping stage calibrates different batches and different sources of segments in a unified scale, and selects the appropriate numerical precision and parallel strategy according to the running environment, to ensure stable latency performance in edge deployment and central deployment. The generated segment target embedding representation is output together with its source offset and coverage range, facilitating the downstream fusion module to perform splicing, weighting and attention aggregation while maintaining traceable evidence.

[0092] In this embodiment, the compressor module is used to complete the hierarchical aggregation from the segment word-level embedding representation to the segment memory slot, the text length is significantly converged, and the long-distance dependency is condensed and reserved in a cross-cell weighting manner; through the linear projection, the non-linear activation and the dimension mapping of the converter module, the segment memory slot is robustly mapped to the input space on the question and answer side, and the same expression scale and geometric structure as the global target embedding representation and the question embedding representation are obtained.

[0093] In one embodiment, before the step S20, the above-mentioned steps further include: S2001, obtaining a training text set; S2002, randomly selecting a part of text content from the training text set for covering processing to generate a covered text; S2003, performing multi-layer semantic aggregation processing on the covered text by the compressor module to generate a training memory slot set; S2004, performing decoding processing on the training memory slot set by the decoder module to reconstruct the original text and determine a text reconstruction loss; S2005, performing language completion processing on the training memory slot set by the language model to predict the covered text content and determine a language completion loss; S2006, assigning weight coefficients to the text reconstruction loss and the language completion loss respectively to generate a weighted text reconstruction loss and a weighted language completion loss; S2007, add the weighted text reconstruction loss and the weighted language completion loss to generate a joint loss; S2008, update parameters of the compressor module according to the joint loss to obtain a trained compressor module.

[0094] In the embodiment, a training text set is used to construct an original source of a compressed learning corpus. Text collection can cover various styles such as clauses, rules, question and answer corpus, policy abstracts, and the like, unified character encoding, removal of control symbols, and retention of necessary punctuation and structure identifiers; for super-long samples, segmented slicing is used and position index within the segment and inter-segment concatenation markers are maintained to ensure that subsequent reconstruction and alignment do not lose bits. In order to reduce corpus bias, de-duplication and difficult example sampling are introduced, and difficult examples can be screened according to indicators such as cross-sentence reference density, long-distance dependency span, numerical and conditional constraint density, etc. to improve the carrying capacity of the training memory slot set for complex semantics.

[0095] A portion of the text content is randomly selected for masking processing to generate a masked text. The masking strategy is jointly executed at three granularities of word, phrase and cross-sentence fragment. The word-level masking is asynchronously set at a random ratio within the sample, the phrase-level masking is determined based on the part of speech and dependency relationship to determine the continuous fragment, and the cross-sentence fragment is positioned according to the reference chain and logical connection word span. Masking markers use a special placeholder, and the original text offset and granularity label are recorded to form a fillable note table. In order to avoid training leakage, the masked fragments do not explicitly appear within the context window; in order to improve robustness, three disturbance methods of replacement, deletion and retention are designed: a small number of word replacements are replaced by random words, a small number of deletions are used to simulate missing, and a large number of retention masks are used. The three-way proportion is randomized by batch.

[0096] The compressor module performs multi-layer semantic aggregation on the masked text to generate a training memory slot set. First, word embedding and local context encoding are completed, and position hints are uniformly represented by a combination of relative and absolute to balance sequence sensitivity; then attention weighting is performed within the sentence to obtain a sentence-level expression, and segment-level expression is extracted through gated fusion, pooling and cross-sentence attention within the segment; finally, a fixed number of aggregation units are used to carry key information at the segment level, each aggregation unit carries the source paragraph, coverage range, aggregation confidence and redundancy ratio index, and constitutes a training memory slot set. In order to control the redundancy between slots, mutual information suppression and orthogonal projection are introduced to improve the separation degree of different slots in the semantic space, while retaining causal clues and constraint conditions that can support reasoning.

[0097] The decoder module reconstructs based on the set of training memory slots, calculates a text reconstruction loss. The reconstruction process is implemented in two paths: the autoregressive path generates the reconstructed compressed text in sequence, and the non-autoregressive path predicts a sequence fragment at a time in parallel; both use the set of training memory slots as conditional input, align to the original text length space using a mask, and then connect to the unmasked part without loss. The text reconstruction loss is mainly based on sequence-level cross-entropy, supplemented by character-level edit distance regularization and coverage penalty, the former constrains language generation accuracy, and the latter prevents omission or excessive redundancy; the covered and uncovered areas are weighted separately to highlight the reconstruction difficulty of the covered area.

[0098] The language model performs language completion processing on the set of training memory slots to predict the covered text and calculate a language completion loss. The completion model locates the prediction target using the cover annotation table and uses a double-tower or single-tower conditional generation structure with the set of training memory slots as a conditional input channel; to reduce position drift, a pointer backfill mechanism is introduced in the prediction head to align the probability mass to the start and end offsets. The language completion loss is mainly based on mask cross-entropy, supplemented by span consistency constraints and term consistency constraints, the former requires the entire predicted segment to be continuous, and the latter sets up a word list or regular template matching item for professional terms, amount identifier values, and time expressions, thereby reducing semantic drift.

[0099] The text reconstruction loss and the language completion loss are respectively assigned weight coefficients to generate a weighted text reconstruction loss and a weighted language completion loss. The weight coefficients support two mechanisms: static preset and dynamic scheduling. Static preset uses a fixed ratio in a stable data distribution scenario; dynamic scheduling automatically adjusts based on the current batch coverage ratio, average span length, and reconstruction perplexity to balance gradient contributions in different difficulty sections. The two weighted losses are added to obtain a joint loss, and the joint loss can be further stacked with lightweight regularization terms such as memory slot sparsity constraints and representation norm constraints to suppress slot redundancy and large magnitudes.

[0100] The compressor module parameters are updated based on the joint loss to obtain the trained compressor module. The optimizer can use a first-order method with momentum or a segmented learning rate plan, focusing on stable convergence in the early stage and introducing fine-grained tuning in the later stage; to balance throughput and stability, batch normalization and layer normalization are used in different sublayers, mixed precision training reduces memory usage and avoids underflow by loss scaling; to adapt to long texts, gradient accumulation and gradient clipping are enabled together to improve effective batch size and suppress gradient explosion. After updating, the set of training memory slots is verified offline, including reconstruction fidelity, coverage completion accuracy, cross-sentence dependency retention rate, and slot redundancy reduction rate, to evaluate the training effectiveness in a closed loop and fine-tune the weight coefficients and coverage strategy parameters accordingly.

[0101] For example, the compressor module is responsible for encoding long texts (such as insurance contracts, claims guidelines, and insurance policies) into multi-dimensional embedding vectors. Its core component is a lightweight Transformer encoder , defined as follows:

[0102]

[0103] where, is the input text sequence, represents a single word in the text, corresponding to the input unit of the "word-level embedding representation", represents the word-level embedding result obtained by the encoder, is a set of memory slots. The compressor adopts a multi-layer semantic aggregation strategy, i.e., extracting embedding representations at the word level, sentence level, and paragraph level, respectively, and implementing information condensation through attention pooling functions To ensure domain consistency, the compressor optimizes the dual-task objective function during the pre-training phase:

[0104] where, is the text reconstruction loss (to ensure content reversibility), is the language completion loss (to ensure context coherence), and the two losses are jointly optimized to improve the semantic fidelity of the model.

[0105] Since different LLMs (such as ChatGLM, Qwen, and Claude) have different input embedding space dimensions, the converter module is responsible for semantic alignment and vector mapping. The converter adopts a linear projection + semantic adjustment form:

[0106] where, is the converter, is a learnable mapping matrix, is a nonlinear activation function, , are trainable bias terms used to enhance the stability of the mapping, and Z represents the result after semantic mapping and activation transformation, i.e., the global target embedding representation.

[0107] In addition, to further improve cross-model adaptability, a semantic co-distillation mechanism is introduced: i.e., synchronously optimizing semantic distribution between different models (such as a financial specialized LLM and a general LLM) to make the embedding space compatible, thereby ensuring that the compressed results can be transferred and used.

[0108] The domain alignment module fine-tunes with a small amount of financial and insurance domain data (about 10-50 thousand) to improve the system's sensitivity to domain language. Specifically, two-stage optimization is adopted: Stage one: clause logic fine-tuning uses logical relationship annotation data within insurance clauses (such as "exemption clause → claim limitation → exception of payment"), and optimizes the model's logical consistency in the embedding layer.

[0109] Stage two: regulatory alignment introduces financial regulatory policy summary data, and through contrastive learning optimization, the model can maintain consistent interpretation when facing different policy constraints.

[0110] The loss function is defined as:

[0111] Where represents the logical consistency loss, which is used to strengthen the clause causal consistency, represents the domain normalization loss, which is used to normalize the domain semantics, represents the overall semantic alignment loss, which is used to guide the converter module to maintain semantic consistency in the cross-model space.

[0112] In the reasoning phase, the system combines context compression and RAG mechanisms to realize an efficient question answering process: User input question ; The retriever selects relevant document fragments from the knowledge base; The compressor converts these documents into compact embeddings; The converter adjusts the embedding dimension and then inputs it into the LLM along with the question embedding; LLM generates answer , and the information source can be visualized through attention heat maps.

[0113] Unlike traditional RAG, the system can compress the original document to 1 / 20 to 1 / 50 of the original size while maintaining key semantics, significantly reducing reasoning costs.

[0114] Further, the goal of the compressor is to compress information without losing semantics, so a joint loss function needs to be defined:

[0115] Text reconstruction loss : Ensure that the compressed "memory slot vector" can fully express the original text content.

[0116] Input the original text to the compressor , and get the compressed embedding . Then pass through a light-weight decoder (typically Transformer Decoder or LSTM) to restore the text . Compute the cross-entropy loss of both:

[0117] If contrastive loss is adopted, we can further define:

[0118] where is the intermediate semantic representation of the original text, is the decoding mapping function.

[0119] Language completion loss : Let the compressor not only reconstruct the existing text, but also assist the language model in predicting the subsequent content (reflecting "context coherence").

[0120] Randomly mask a part of the original text (mask), such as the end of the sentence or the end of the paragraph; input the compressed embedding into the language model head (or frozen LLM) to predict the masked token; use cross-entropy or KL divergence as the loss function:

[0121] This is similar to the MLM task of BERT or span-masking of T5, but the compressor makes up for the prediction through sparse embedding, so it can strengthen its ability to capture "context dynamics".

[0122] Further, the multi-layer semantic aggregation strategy (word level -> sentence level -> paragraph level) is the key to context compression, which enables the compressor to capture both fine-grained information (financial terms, numbers) and macro-logical relationships (clause causality).

[0123] Overall structure: input long text is divided into: Word-level representation: token embeddings

[0124] Sentence-level representation: sentence vector

[0125] Paragraph-level representation: paragraph vector

[0126] The compressor uses a hierarchical attention mechanism internally to achieve hierarchical information aggregation.

[0127] Word-level aggregation: for sentences :

[0128] where BiLSTM (Bidirectional Long Short-Term Memory) denotes a bidirectional long short-term memory network, is the semantic projection vector of the i-th word, which is obtained by linearly transforming the word-level vector with a trainable weight matrix and a bias term The process of linear transformation and nonlinear mapping of the word-level vector and are the same type of vectors, which are used to represent the semantic projection vector of the word at any position t in the sentence; while is a trainable context vector; is the attention weight, which is used to aggregate word-level features to generate sentence-level representations.

[0129] Each word is aggregated into a sentence vector by attention weight, retaining the prominent features of financial terms such as "deductible" and "annualized yield".

[0130] Sentence-level aggregation: Similarly, for the sentences within a paragraph:

[0131] This step mainly captures the logical relationship between sentences (such as "if A occurs, B will not be paid").

[0132] Paragraph-level aggregation (final compression): Embed all paragraphs and integrate them through a Transformer layer to obtain a global embedding :

[0133] Here, is usually attention pooling or grouped max pooling to generate a fixed number of memory slots (such as 64 or 128). This "multi-granularity aggregation" can maintain the complete semantics of the clause while achieving a compression rate of 95%.

[0134] Further, semantic co-distillation is used to improve the robustness of the compressed embedding in cross-model and cross-task scenarios. The idea is to align the embedding distribution of the compressor to the high-dimensional semantic distribution of a "teacher model".

[0135] The architecture includes: Teacher model: a pre-trained large language model (such as ChatGLM, Qwen) Student Model: Compressor Training objectives: For the same input text : The teacher model generates a hidden semantic distribution.

[0136] Compressor produces embedded

[0137] Define distribution alignment loss: in: This is a linear mapping layer that projects the teacher feature dimension onto the student space; This represents the KL divergence of the output probability distributions of the two. Control the distillation intensity.

[0138] Furthermore, to adapt to the financial and insurance sectors, "dual-teacher distillation" can be used: General-purpose teachers: such as ChatGLM2, used to maintain language fluency; Domain-specific instructors: such as FinGPT and insurance question-answering models, used to enhance professionalism; Total loss: This structure ensures that the compressed memory slots achieve a balance between semantic consistency, domain representation, and cross-platform migration.

[0139] This embodiment minimizes both text reconstruction loss and language completion loss in random occlusion scenarios. The compressor module is guided to retain the key information necessary to drive reconstruction and completion within a fixed number of slots. The training memory slot set achieves comprehensive optimization in semantic density, causal constraints, and terminology consistency. The static and dynamic joint scheduling of weight coefficients ensures that the joint loss maintains a balanced gradient contribution on samples of different difficulty levels, avoiding representation degradation caused by bias towards a single objective. The resulting trained compressor module maintains high semantic fidelity and refillability while significantly compressing the input length, providing stable and compact evidence representation for subsequent mapping, retrieval, fusion, and generation stages, reducing inference computation overhead, and improving answer consistency and interpretability.

[0140] In one embodiment, a question-answering device based on memory slots and semantic alignment is provided, which corresponds one-to-one with the question-answering methods based on memory slots and semantic alignment described in the above embodiments. (Refer to...) Figure 3 , Figure 3FIG. 1 is a schematic diagram of a functional module of a preferred embodiment of the question-answering device based on memory slots and semantic alignment of the present application. Knowledge base management module 10, compressor module 20, converter module 30, question representation module 40, retrieval module 50, fragment embedding generation module 60, feature fusion module 70, and question-answering model module 80. The detailed description of each functional module is as follows: Knowledge base management module 10 is used to obtain a document set and store the document set and corresponding metadata into a knowledge base; Compressor module 20 is used to perform multi-layer semantic aggregation processing on the document set in the knowledge base through the compressor module, generate a memory slot set corresponding to the document set, and establish a mapping relationship between the memory slot set and the document index of the document set; Converter module 30 is used to perform semantic alignment processing on the memory slot set through the converter module, map the memory slot set to the input space of a question-answering model, and generate a global target embedding representation; Question representation module 40 is used to receive a query question and perform semantic representation processing on the query question to generate a question embedding representation; Retrieval module 50 is used to match the question embedding representation with the global target embedding representation through the retrieval module, determine a target memory slot from the global target embedding representation according to the matching result, and obtain a candidate document fragment corresponding to the target memory slot according to the mapping relationship; Fragment embedding generation module 60 is used to input the candidate document fragment into the compressor module and the converter module to generate a fragment target embedding representation corresponding to the candidate document fragment; Feature fusion module 70 is used to fuse the fragment target embedding representation with the question embedding representation to form a question-answering input representation; Question-answering model module 80 is used to process the question-answering input representation through a question-answering model to generate a question-answering response.

[0141] In an embodiment, compressor module 20 is specifically used for: Performing word segmentation processing on the document set in the knowledge base through the compressor module to generate a word-level embedding representation; Aggregating the word-level embedding representation through the attention mechanism in the compressor module to generate a sentence-level embedding representation; Aggregating the sentence-level embedding representation through the attention mechanism in the compressor module to generate a paragraph-level embedding representation; Aggregating the paragraph-level embedding representation through the pooling function in the compressor module to generate a memory slot set; Extracting the document index of the document set; A mapping table recording the association of each memory slot with a corresponding document index is established between the memory slot set and the document index.

[0142] In an embodiment, the converter module 30 is specifically configured to: perform linear transformation processing on the memory slot set through a linear projection layer in the converter module to generate an intermediate embedding representation; perform non-linear transformation processing on the intermediate embedding representation through a non-linear activation function in the converter module to generate an activated embedding representation; perform dimension mapping processing on the activated embedding representation through a mapping matrix in the converter module to generate a preliminary target embedding representation; perform alignment processing on the preliminary target embedding representation with a semantic distribution of a teacher model through a semantic co-distillation mechanism in the converter module to generate a global target embedding representation.

[0143] In an embodiment, the question representation module 40 is specifically configured to: receive a query question, perform word segmentation and tokenization processing on the query question to generate a token sequence; perform vector mapping processing on the token sequence through an embedding layer of a pre-trained language model to generate a token embedding sequence; perform context encoding processing on the token embedding sequence through a Transformer encoder of the pre-trained language model to generate a context-enhanced sequence representation; perform sequence pooling processing on the context-enhanced sequence representation to generate an initial question vector; perform distribution standardization processing on the initial question vector through a layer normalization module to generate a normalized question vector; perform dimension adjustment processing on the normalized question vector through a linear projection layer to generate a question embedding representation matching the dimension of the global target embedding representation.

[0144] In an embodiment, the retrieval module 50 is specifically configured to: determine the cosine similarity between the question embedding representation and the embedding representation corresponding to each memory slot in the global target embedding representation through a similarity analysis unit in the retrieval module to generate a similarity score set; perform descending order sorting processing on the similarity score set through a sorting unit in the retrieval module to generate a memory slot similarity sorting list; select the top pre-set number of memory slots with high similarity scores from the memory slot similarity sorting list through a selection unit in the retrieval module to generate a candidate memory slot set; determining, by a mapping query unit in the retrieval module, a target memory slot from the candidate memory slot set according to the mapping relationship; extracting, by a fragment acquisition unit in the retrieval module, a candidate document fragment from a knowledge base according to a document index corresponding to the target memory slot.

[0145] In an embodiment, the fragment embedding generation module 60 is specifically configured to: performing word segmentation processing on each candidate document fragment by the compressor module to generate a fragment word-level embedding representation; performing context encoding processing on the fragment word-level embedding representation by the compressor module to generate a fragment context-enhanced representation; performing semantic aggregation processing on the fragment context-enhanced representation by the compressor module to generate a fragment memory slot; performing linear projection processing on the fragment memory slot by the converter module to generate a fragment intermediate embedding representation; performing nonlinear activation processing on the fragment intermediate embedding representation by the converter module to generate a fragment activated embedding representation; performing dimension mapping processing on the fragment activated embedding representation by the converter module to generate a fragment target embedding representation.

[0146] In an embodiment, the compressor module 20 is specifically configured to: obtain a training text set; randomly select a part of text content from the training text set for masking processing to generate a masked text; perform multi-layer semantic aggregation processing on the masked text by the compressor module to generate a training memory slot set; perform decoding processing on the training memory slot set by the decoder module to reconstruct the original text and determine a text reconstruction loss; perform language completion processing on the training memory slot set by a language model to predict the masked text content and determine a language completion loss; assign weight coefficients to the text reconstruction loss and the language completion loss respectively to generate a weighted text reconstruction loss and a weighted language completion loss; add the weighted text reconstruction loss and the weighted language completion loss to generate a joint loss; update parameters of the compressor module according to the joint loss to obtain a trained compressor module.

[0147] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 4As shown in the figure. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide determination and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media, internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external client through the network connection. The computer program is executed by the processor to realize the functions or steps of the server side of the memory slot and semantic alignment-based question and answer method.

[0148] In one embodiment, a computer device is provided, which can be a client, and its internal structure diagram can be as Figure 5 As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide determination and control capabilities. The memory of the computer device includes non-volatile storage media, internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of the client side of the memory slot and semantic alignment-based question and answer method.

[0149] In one embodiment, a computer device is provided, including a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to realize the following steps: Obtain a document set, and store the document set and corresponding metadata into a knowledge base; Perform multi-layer semantic aggregation processing on the document set in the knowledge base through a compressor module, generate a memory slot set corresponding to the document set, and establish a mapping relationship between the memory slot set and the document index of the document set; Perform semantic alignment processing on the memory slot set through a converter module, map the memory slot set to the input space of a question and answer model, and generate a global target embedding representation; Receive a query question, and perform semantic representation processing on the query question to generate a question embedding representation; Match the question embedding representation with the global target embedding representation through a retrieval module, determine a target memory slot from the global target embedding representation according to the matching result, and obtain a candidate document segment corresponding to the target memory slot according to the mapping relationship; inputting the candidate document segment into the compressor module and the converter module to generate a segment target embedding representation corresponding to the candidate document segment; fusing the segment target embedding representation with the question embedding representation to form a question-answering input representation; processing the question-answering input representation through a question-answering model to generate a question-answering response.

[0150] In an embodiment, a non-volatile computer readable storage medium is provided, which can be non-volatile or volatile, and has stored thereon a computer program, which, when executed by a processor, implements the following steps: obtaining a document set and storing the document set and corresponding metadata into a knowledge base; performing multi-layer semantic aggregation processing on the document set in the knowledge base through a compressor module to generate a memory slot set corresponding to the document set, and establishing a mapping relationship between the memory slot set and document indexes of the document set; performing semantic alignment processing on the memory slot set through a converter module to map the memory slot set to an input space of a question-answering model to generate a global target embedding representation; receiving a query question and performing semantic representation processing on the query question to generate a question embedding representation; matching the question embedding representation with the global target embedding representation through a retrieval module, determining a target memory slot from the global target embedding representation according to a matching result, and obtaining a candidate document segment corresponding to the target memory slot according to the mapping relationship; inputting the candidate document segment into the compressor module and the converter module to generate a segment target embedding representation corresponding to the candidate document segment; fusing the segment target embedding representation with the question embedding representation to form a question-answering input representation; processing the question-answering input representation through a question-answering model to generate a question-answering response.

[0151] It should be noted that the functions or steps that the above computer readable storage medium or computer device can implement can correspond to the related descriptions of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0154] It should be noted that if non-company software tools or components appear in the embodiments of the present application, they are only used for example introduction and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

[0155] The user personal information involved in the embodiments of the present application is authorized (known and agreed) by the relevant object or fully authorized by all parties, and the execution subject can obtain it through various public, legal and compliant ways. The collection, storage, use, processing, transmission, provision and disclosure of the information, data and signals involved comply with the relevant laws and regulations of the relevant countries and regions, and do not violate public order and good customs.

Claims

1. A question-answering method based on memory slots and semantic alignment, characterized in that, Includes the following steps: Obtain the document set and store the document set and its corresponding metadata into the knowledge base; The document collection in the knowledge base is subjected to multi-layer semantic aggregation processing by the compressor module to generate a memory slot set corresponding to the document collection, and a mapping relationship is established between the memory slot set and the document index of the document collection. The memory slot set is semantically aligned using a converter module, mapping the memory slot set to the input space of the question-answering model to generate a global target embedding representation. Receive query questions and perform semantic representation processing on the query questions to generate question embedding representations; The retrieval module matches the question embedding representation with the global target embedding representation, determines the target memory slot from the global target embedding representation based on the matching result, and obtains the candidate document fragment corresponding to the target memory slot according to the mapping relationship. The candidate document fragments are input into the compressor module and the converter module to generate a fragment target embedding representation corresponding to the candidate document fragments; The fragment target embedding representation is fused with the question embedding representation to form a question-and-answer input representation; The question-and-answer input representation is processed by a question-and-answer model to generate a question-and-answer response.

2. The question-answering method based on memory slots and semantic alignment as described in claim 1, characterized in that, The document collection in the knowledge base undergoes multi-layer semantic aggregation processing through a compressor module to generate a memory slot set corresponding to the document collection, and a mapping relationship is established between the memory slot set and the document index of the document collection, including: The document set in the knowledge base is segmented using a compressor module to generate word-level embedded representations. The word-level embedding representations are aggregated using the attention mechanism in the compressor module to generate sentence-level embedding representations. The sentence-level embedding representations are aggregated using the attention mechanism in the compressor module to generate paragraph-level embedding representations. The paragraph-level embedding representations are aggregated using the pooling function in the compressor module to generate a set of memory slots; Extract the document index of the document collection; A mapping table is established between the memory slot set and the document index to record the association between each memory slot and the corresponding document index.

3. The question-answering method based on memory slots and semantic alignment as described in claim 1, characterized in that, The memory slot set is semantically aligned using a converter module, mapping it to the input space of the question-answering model to generate a global target embedding representation, including: The set of memory slots is linearly transformed by the linear projection layer in the converter module to generate an intermediate embedded representation. The intermediate embedding representation is nonlinearly transformed by the nonlinear activation function in the converter module to generate an activated embedding representation. The activation embedding representation is dimension-mapped using the mapping matrix in the converter module to generate a preliminary target embedding representation. The initial target embedding representation is aligned with the semantic distribution of the teacher model through the semantic co-distillation mechanism in the converter module to generate a global target embedding representation.

4. The question-answering method based on memory slots and semantic alignment as described in claim 1, characterized in that, Receive a query question and perform semantic representation processing on the query question to generate a question embedding representation, including: Receive a query question, perform word segmentation and word grouping on the query question, and generate a word group sequence; The word sequence is vector-mapped through the embedding layer of a pre-trained language model to generate a word embedding sequence. The lexical embedding sequence is context-encoded using a Transformer encoder of a pre-trained language model to generate a context-enhanced sequence representation. The context-enhanced sequence representation is subjected to sequence pooling to generate an initial question vector; The initial problem vector is standardized by a layer normalization module to generate a normalized problem vector. The normalized problem vector is adjusted in dimension by a linear projection layer to generate a problem embedding representation that matches the dimension of the global target embedding representation.

5. The question-answering method based on memory slots and semantic alignment as described in claim 1, characterized in that, The retrieval module matches the question embedding representation with the global target embedding representation, determines the target memory slot from the global target embedding representation based on the matching result, and obtains the candidate document fragment corresponding to the target memory slot according to the mapping relationship, including: The cosine similarity between the question embedding representation and the embedding representation corresponding to each memory slot in the global target embedding representation is determined by the similarity analysis unit in the retrieval module, and a similarity score set is generated. The similarity score set is sorted in descending order by the sorting unit in the retrieval module to generate a memory slot similarity sorting list. The selection unit in the retrieval module selects a preset number of memory slots with the highest similarity scores from the memory slot similarity sorting list to generate a candidate memory slot set. The target memory slot is determined from the candidate memory slot set by the mapping query unit in the retrieval module according to the mapping relationship; The fragment acquisition unit in the retrieval module extracts candidate document fragments from the knowledge base based on the document index corresponding to the target memory slot.

6. The question-answering method based on memory slots and semantic alignment as described in claim 1, characterized in that, The candidate document fragments are input into the compressor module and the converter module to generate a fragment target embedding representation corresponding to the candidate document fragments, including: The compressor module performs word segmentation on each candidate document fragment to generate a fragment word-level embedding representation. The compressor module performs context encoding on the fragment word-level embedding representation to generate a fragment context-enhanced representation. The compressor module performs semantic aggregation processing on the enhanced context representation of the fragment to generate fragment memory slots. The converter module performs linear projection processing on the fragment memory slot to generate a fragment intermediate embedding representation. The converter module performs nonlinear activation processing on the intermediate embedding representation of the segment to generate a segment activation embedding representation. The converter module performs dimension mapping on the fragment activation embedding representation to generate the fragment target embedding representation.

7. The question-answering method based on memory slots and semantic alignment as described in claim 1, characterized in that, Before performing multi-layer semantic aggregation processing on the document set in the knowledge base through the compressor module to generate a memory slot set corresponding to the document set, and before establishing a mapping relationship between the memory slot set and the document index of the document set, the process further includes: Obtain the training text set; A portion of the text content is randomly selected from the training text set and masked to generate masked text; The compressed text is subjected to multi-layer semantic aggregation processing by the compressor module to generate a training memory slot set. The training memory slot set is decoded by the decoder module to reconstruct the original text and determine the text reconstruction loss. The training memory slot set is processed by a language model to perform language completion, predict the content of the masked text, and determine the language completion loss. Weight coefficients are assigned to the text reconstruction loss and the language completion loss respectively to generate weighted text reconstruction loss and weighted language completion loss; The weighted text reconstruction loss and the weighted language completion loss are added together to generate a joint loss; The parameters of the compressor module are updated based on the joint loss to obtain the trained compressor module.

8. A question-answering device based on memory slots and semantic alignment, characterized in that, The question-answering device based on memory slots and semantic alignment includes: The knowledge base management module is used to acquire a document collection and store the document collection and its corresponding metadata into the knowledge base. The compressor module is used to perform multi-level semantic aggregation processing on the document set in the knowledge base, generate a memory slot set corresponding to the document set, and establish a mapping relationship between the memory slot set and the document index of the document set. The converter module is used to perform semantic alignment processing on the memory slot set, map the memory slot set to the input space of the question answering model, and generate a global target embedding representation. The question representation module is used to receive query questions, perform semantic representation processing on the query questions, and generate question embedding representations; The retrieval module is used to match the question embedding representation with the global target embedding representation, determine the target memory slot from the global target embedding representation according to the matching result, and obtain the candidate document fragment corresponding to the target memory slot according to the mapping relationship. A fragment embedding generation module is used to input the candidate document fragments into the compressor module and the converter module to generate a fragment target embedding representation corresponding to the candidate document fragments; The feature fusion module is used to fuse the fragment target embedding representation with the question embedding representation to form a question-answering input representation; The question-answering model module is used to process the question-answering input representation through a question-answering model and generate a question-answering response.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a slot-based and semantically aligned question-answering program stored in the memory and executable on the processor, wherein the slot-based and semantically aligned question-answering program, when executed by the processor, implements the steps of the slot-based and semantically aligned question-answering method as described in any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that, The storage medium stores a question-answering program based on memory slots and semantic alignment, which, when executed by a processor, implements the steps of the question-answering method based on memory slots and semantic alignment as described in any one of claims 1-7.

Images