Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

110 results about "Context processing" patented technology

Multi-modal large language model dialogue generation method based on natural language understanding

The invention is suitable for the technical field of artificial intelligence, and provides a multi-modal large language model dialogue generation method based on natural language understanding, comprising the following steps: receiving multi-modal information input by a user through a multi-modal large language model, the multi-modal information comprising a plurality of modal data; preprocessing modal data in the multi-modal information, and extracting to obtain multi-modal features; performing unified fusion feature representation on the extracted multi-modal features based on a dynamic weighted fusion strategy of a multi-head attention mechanism; determining the dialogue state of the long context based on a long context processing algorithm, and ensuring that the generated reply content is associated with the dialogue history; and according to the unified fusion feature representation and the dialogue state, a natural language is generated through an RAG retrieval enhancement generation technology for replying. According to the method, input of multiple modes can be processed and understood, the ability of understanding the intention of the user is improved, the continuity of the context is maintained, and the generated reply is more natural.
Owner:BEIJING LIANPING TECH CO LTD

Method and system for generating bionic hierarchical memory fusion document of large electric semantic model

The invention provides an electric power semantic large model bionic hierarchical memory fusion document generation method and system, and relates to the technical field of document generation, and the method comprises the steps: extracting knowledge in a multi-modal data form, building a long-term memory library through simulating a memory mechanism of a human hippocampus, and generating a multi-modal data form; the knowledge in the long-term memory library is stored and dynamically updated in a hierarchical manner according to three levels of sentences, paragraphs and chapters; inputting a new document generation task into the electric power semantic large model, and taking the long-term memory library as an external knowledge source of the electric power semantic large model; dynamically searching knowledge of each level related to the current task from a long-term memory library; and iteratively fusing the context background knowledge and the knowledge dynamically searched from the long-term memory library layer by layer, and finally generating complete document content. By dynamically calling long-term memory and multi-level context processing, logically coherent and information-accurate contents are generated, multi-modal data are effectively integrated, and the method is suitable for efficient generation of complex documents such as construction schemes and audit reports.
Owner:STATE GRID INTELLIGENCE TECHNOLOGY CO LTD

Context-aware, artificial intelligence-based system for increasing employee engagement and automating the integration of business processes.

A system for improving employee engagement and automating the integration of business processes. The system includes: a user interaction module configured to receive multimodal inputs, including natural language text and voice requests from employees via enterprise portals, mobile applications, voice-activated devices, and collaboration platforms, and to generate real-time responses via conversational output channels; a context processor that is operationally coupled with the user interaction module, wherein the context processor stores the interaction history, employee preferences, organizational role metadata, and task results in both short-term and long-term memory and dynamically derives context for controlling responses and initiating workflows; A natural language and intent processor that is communicatively linked to the context processor. The intent engine consists of large language models trained on company-specific lexicons to perform intent detection, entity extraction, ambiguity resolution, and sentiment or urgency classification from employee queries; a workflow orchestration layer in communication with the intent engine and the context processor, wherein the workflow orchestration layer includes a rule-based and AI-powered process execution engine configured to automatically initiate, route, and complete cross-functional business tasks, including approvals, escalations, and compliance checks, with workflows defined using modular templates that include conditional logic and time-based triggers; Webhooks and authentication protocols provide secure interoperability between the workflow orchestration layer and external enterprise platforms; a feedback and learning module that is operationally coupled with the workflow orchestration layer and the natural language understanding engine, wherein the feedback and learning module is configured to analyze task completion rates, response accuracy, latency metrics, and user feedback signals, and to retrain underlying language and workflow models for adaptive improvement in real time; and an administration console with role-based access controls, compliance dashboards, audit trails and interfaces for workflow configuration, whereby the administration console enables authorized personnel to monitor system operations, adjust interaction rules and enforce data protection restrictions across departments.
Owner:KURAPATI SURESH KHAMMAM +3

Two-level context caching and eviction for scatter-gather DMA

One aspect of the instant disclosure may provide a system and method for processing scatter-gather direct memory access (S-G DMA) instructions. During operation, the system may receive an S-G DMA instruction associated with a message and gather instruction context for the S-G DMA instruction. An S-G DMA processor may process the S-G DMA instruction based on the gathered instruction context and determine whether there exists a pending S-G DMA instruction associated with the message. In response to the presence of the pending S-G DMA instruction, the system stores the instruction context in a hot context cache at an address corresponding to the pending S-G DMA instruction. In response to the absence of the pending S-G DMA instruction, the system stores the instruction context in a cold context cache.
Owner:HEWLETT PACKARD ENTERPRISE DEV LP

Voice control implementation method and system based on large model

The invention discloses a voice control implementation method and system based on a large model, belongs to the technical field of artificial intelligence, and aims to solve the technical problems of insufficient recognition precision, low response speed and limited complex context processing capability in voice control. Comprising the following steps: analyzing a user voice instruction into a text instruction through a voice recognition model; performing semantic feature extraction on an input text instruction through a pre-trained text embedding model, and outputting a text embedding vector as a to-be-recognized text embedding vector; searching whether a text embedding vector similar to the to-be-identified text embedding vector exists in a pre-constructed vector database, if not, executing text analysis, and if yes, returning corresponding control information from the vector database; and taking the text instruction and the question statement as input, performing semantic analysis through a pre-trained large language model to obtain control information, and storing a text embedding vector corresponding to the current user voice instruction and the control information into a vector database.
Owner:SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD

Traffic influence for initial EAS selection

A method performed by a User Equipment having a first functional component implementing an enabler function. The method comprises performing an Edge Application Server (EAS) discovery to discover information regarding one or more first network functions implementing EASs as part of an initial service provisioning. The method further comprises selecting at least one first network function for implementing an initial EAS of the one or more first network functions, for providing service to a second functional component implementing application function in the UE. The method further comprises determining a second network function implementing Edge Enabler Server (EES) according to the selected at least one first network function as an initial EAS. The method further comprises transmitting, to the determined second network function, a message for triggering at least one of service session context handling and traffic influence. The message comprises an EAS endpoint of the selected initial EAS.
Owner:TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)

Leader decision-making-oriented intelligent data question and answer and visualization system and method

The invention discloses an intelligent data question answering and visualization system and method for leader decision making, and relates to the technical field of artificial intelligence, and the system comprises a knowledge base unit, an intelligent question answering module, a dynamic visualization unit, a quality control unit and a performance optimization unit. According to the method, the customized knowledge base is constructed through the knowledge base unit, cross-modal alignment and layered long context processing are weakly supervised during data fusion, the defects that cross-modal data labeling cost is high and long context processing is redundant can be overcome, and an interaction platform of the knowledge base and a data engine in the knowledge base unit is constructed through the intelligent question and answer module; the model mixing module mixes an NLP model, constructs a combined architecture of an LLM + field fine tuning model, ensures understanding of natural languages and intention analysis precision, introduces an online expert error correction mechanism, and reduces misjudgment during question answering, so that the processing capacity of long texts or complex logic can be ensured, missing of key information in question answering is avoided, and then the answering quality is ensured.
Owner:JIANGXI WEIBO TECH CO LTD

Intelligent sensing of screen content updates for user context processing

Various systems and methods for contextual capture and processing of screen capture data are disclosed. An example method for screen capture data processing in a computing device may include: determining an active screen of the computing device based on a user interaction event; identifying a graphics rendering event associated with a software application presented in the active screen; identifying screen capture data in a buffer (e.g., of graphics processing circuitry such as a GPU) that corresponds to the graphics rendering event; and communicating a contextual screen update event via an application programming interface (e.g., a an API operated by a GPU driver). The receipt of this contextual screen update event can be used by an AI engine to control whether to perform contextual processing on particular frames of the screen capture data.
Owner:INTEL PRODUCTS IP LLC

Large model context processing method and device, equipment, storage medium and program product

The invention provides a large model context processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the steps of obtaining a new action step request, updating a context view tree and a source interaction data tree based on the new action step request, and determining a pressure coefficient according to the updated context view tree; according to the pressure coefficient, determining whether a to-be-compressed node exists in the updated context view tree or not; if the to-be-compressed node exists, combining the updated source interaction data tree and the updated context view tree to perform back-to-source compression on the to-be-compressed node so as to update the updated context view tree and the updated source interaction data tree, and returning to the step of determining the pressure coefficient according to the updated context view tree until the to-be-compressed node does not exist; and based on the updated context view tree, expanding the context view tree into context content and inputting the context content into the large language model. By adopting the method, the large language model can accurately understand and respond to the new action step request.
Owner:ELECTRIC POWER RES INST CHINA SOUTHERN POWER GRID CO LTD

Retrieval recall method and system integrating document fragmentation and authority control

The invention provides a retrieval recall method and system fusing document fragmentation and authority control, and relates to the technical field of artificial intelligence and information retrieve.The method comprises the steps that a role of a user is obtained according to the identity of the user, and a corresponding authority strategy is determined in combination with a preset multi-level authority mapping relation; the permission strategy comprises a label allowed to be accessed, a data set and a document set; the query request is converted into a semantic vector, and initial candidate document fragments are retrieved from a vector database in combination with an associated data set or tag identifier; according to the method, the candidate results are subjected to access control filtering on the basis of the authority strategy, and the final retrieval recall result is generated, so that role-based fine-grained access control is realized while high retrieval precision is ensured, the problems of rough authority management and weak context processing capability in traditional retrieval are solved, and the security and efficiency of retrieval are remarkably improved.
Owner:DIGITAL ZHEJIANG TECH OPERATION CO LTD

An intelligent question answering system and method based on large language model

The present application discloses an intelligent question-answering system and method based on a large language model, and relates to the field of intelligent question-answering systems. The intelligent question-answering system includes: a user question acquisition module, a context processing module, a question classification module, a knowledge base management module, a question-answering processing module, and a reply module. The user question acquisition module acquires the user's current question, the context processing module adopts a large language model, and generates a search statement according to the user's historical question information and the user's current question. The question classification module classifies the user's question according to the search statement, the knowledge base management module stores technical solutions, the question-answering processing module retrieves the technical solutions stored in the knowledge base management module based on the question classification results, and the reply module adopts a large language model to generate reply content based on the retrieved technical solutions. The present application can achieve efficient and accurate answers to customers' technical inquiries, and can automatically generate personalized solutions according to the specific needs of customers.
Owner:CHINA COMMERCE NETWORKS (SHANGHAI) CO LTD

Multi-round dialogue system and method based on conversion from natural language to SQL

The invention discloses a multi-round dialogue system and method based on conversion from a natural language to an SQL, and relates to the technical field of artificial intelligence, and the system comprises a user interaction module which is used for supporting a user to input a query problem and displaying a corresponding SQL query result; the context processing module is used for screening the effective dialogue history of the current question and re-integrating the effective dialogue history into a complete dialogue; the ambiguity processing module is used for identifying an entity from a dialogue history to define a user intention, processing a fuzzy keyword in a current question and guiding the user to complement a query condition; the re-splicing module is used for reconstructing a complete dialogue according to a time sequence, eliminating ambiguity, generating a summary through a large model, and combining the summary with a current problem to construct a complete problem; the SQL generation module is used for performing semantic analysis on the complete problem and outputting an SQL query instruction with the highest matching degree; and the execution and rendering module is used for executing the instruction on the target database and returning a query result. The conversation processing capability can be improved.
Owner:SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD

Artificial intelligence (AI) agents based data processing in enterprise applications developed by codeless platform

The present invention provides a system and method for multi AI agent driven data processing in enterprise application developed by codeless platform. The invention includes one or more AI agents configured for processing one or more input received on conversational assistant interface. The invention includes triggering contextual processing of received input by orchestration agent interacting with multi-AI agents for executing a task identified from the received input.
Owner:NB VENTURES INC DBA GEP

Automatic long text fine tuning instruction set construction method based on large language model

The invention provides an automatic long text fine-tuning instruction set construction method based on a large language model, which comprises the following steps: acquiring an input text, and segmenting the input text by adopting a recursive character segmentation method to generate a paragraph set; aiming at the generated paragraph set, the large language model generates a question set and an answer set by adopting a self-guidance learning method through a preset question type set and a prompt template according to the task type; and generating an instruction set based on the generated question set and answer set, evaluating the quality of the instruction set in multiple dimensions, and optimizing the instruction set according to an evaluation result to obtain an optimized instruction set. According to the method, the high-quality long text fine tuning instruction set can be automatically generated, so that the performance of a large language model is improved, and meanwhile, the challenge of long context processing is solved. By automatically constructing the long text fine-tuning instruction set, the requirement of manual annotation is reduced, the cost is reduced, and meanwhile, the efficiency of the fine-tuning process and the long text processing capacity of the model are improved.
Owner:INST OF AUTOMATION CHINESE ACAD OF SCI +3

Self-adaptive context processing method and system for large language model

The invention relates to the technical field of artificial intelligence, in particular to an adaptive context processing method and system for a large language model. The method comprises the following steps: collecting multi-source candidate context segments related to current user query; performing utility evaluation on each candidate fragment, outputting a quantized utility score, and performing multi-granularity compression on each fragment based on the utility score and a preset quality constraint to obtain a compressed fragment set; taking the maximization of the total utility score as a target, taking the condition that the total token cost is less than or equal to the global token budget as a constraint, and carrying out optimization selection to obtain an optimal context combination; and inputting the optimal context combination into a large language model to generate an output result for the current query. By applying the method, the context window management of the large language model can be optimized, the adaptive context processing capability of the information utilization efficiency is improved, and the problems of context window limitation, value judgment and the like are solved.
Owner:RONGZHITONG TECH BEIJING

Dynamic structured adaptive training method

The invention relates to the technical field of psychological intervention adjustment, in particular to a dynamic structured self-adaptive training method which is applied to a training framework comprising an acquisition terminal, a processing terminal and an execution terminal. The acquisition terminal is used for acquiring a physiological signal, a behavior signal and an environment context of an individual, the processing terminal fuses the information and calculates an anxiety index and an uncertainty probability value, and when the uncertainty probability value is lower than a preset threshold value, an initial task graph is constructed according to a target task and the environment context. The initial task graph is subjected to structure and parameter updating through a preset rule generator in combination with a reward function and an anxiety index, an individualized task graph is generated, cross-age migration is further carried out based on physiological and behavior signals, an execution graph bound with an execution terminal is obtained, and the execution terminal carries out intervention according to the execution graph. The problem that an existing training method is difficult to give consideration to behavior complexity and psychological bearing capacity is solved, and training individuation and stability are improved.
Owner:SUZHOU YUANMENG BARRIER FREE TECH CO LTD

Business process processing algorithm and system based on multi-condition rule driving

The invention provides a business process processing algorithm and system based on multi-condition rule driving. The business process processing algorithm comprises the following steps: receiving to-be-processed data from a data source; obtaining a rule list matched with the current data source; performing condition judgment processing on the to-be-processed data by applying the matched rule; analyzing the content of the rule, and establishing an OR logical relationship among condition groups; an AND logic relation is established among the conditions in the same condition group; processing each condition group in parallel, packaging judgment logic of each condition group into a task, sending the task to a thread pool for asynchronous execution, and judging whether a rule is triggered or not; a context processing object is constructed, judgment results of the condition groups are assembled into a trigger result object, and the context processing object is constructed; distributing the received to-be-processed data for processing the context object; the method has the advantages that computing power resources are concentrated to process calculation of multi-condition rule judgment, waste of the computing power resources is avoided, and the response speed and expandability of the system are improved.
Owner:NINGBO SHUYI GONGLIAN TECH CO LTD

Systems and methods for sub-graph processing in executable graph-based models

A method for dynamic execution of sub-graphs within executable graph-based models is provided. Processing circuitry obtains an executable graph-based model comprising a plurality of sub-graphs and an overlay structure comprising processing logic associated with the plurality of sub-graphs. Each sub-graph defines a hierarchical structure of related nodes. The processing circuitry receives a stimulus and a context associated with the stimulus. In response to the stimulus being received and based on the context, the processing circuitry maps the stimulus to a first sub-graph of the executable graph-based model. The processing circuitry causes execution of processing logic within the overlay structure based on the mapping. The processing logic is associated with one or more nodes of the first sub-graph.
Owner:INVERTIT INC +1

Mechanism to enable out-of-order packet processing in a datatype engine

ActiveUS12229048B2Data classData pack
A network interface card (NIC) receives packets corresponding to a read or write request, the packets associated with a datatype descriptor stored in a datatype engine of the NIC, and each packet associated with a precomputed context which indicates a value for each dimension of a multi-dimensional array and a start location of the respective packet within a host memory block. The NIC generates, for a respective packet, a datatype handle corresponding to the datatype descriptor and an offset indicating a position of the respective packet within the packets. The NIC determines, based on the datatype handle and the offset, a cached context for the respective packet and initializes the datatype engine based on the cached context. The datatype engine generates, based on the cached context, read or write requests comprising addresses and lengths, thereby allowing the NIC to process out-of-order packets based on the precomputed and cached context.
Owner:HEWLETT PACKARD ENTERPRISE DEV LP

A system for context-dependent language translation

A system (100) for context-sensitive language translation, comprising: an input interface module (102) configured to allow the user to input text for language translation, wherein the input interface module (102) includes an interface that allows the user to input text and select the language for translation; a context processing module (104) connected to the input interface module (102) and configured to enable context-dependent extraction of information from the provided input text, wherein the context processing module (104) comprises the following: an input processing module (106) configured to decompose the input text into tokens and identify context-sensitive segments; and a context extraction module (108) connected to the input processing module (106), wherein the context extraction unit (108) comprises an extraction processor configured to generate context vectors from source input text; a translation processing unit (110) comprising a machine learning-based processor configured to translate the provided input text, wherein the translation processing unit comprises: an attention mechanism (112) configured to select words based on their relevance to the context; and a translation generation module (114) configured to generate translations using a neural network that incorporates the attention-weighted context vectors; a post-processing unit (116) connected to the translation processing unit (110) and configured to refine the translated output to ensure fluency and coherence, refining the translation grammar, ensuring semantic consistency, validating contextual appropriateness, and generating a fluency score; and an output module (118) connected to the post-processing unit (116) and comprising a display with an output interface on which the input text and the translated output text are displayed to the user.
Owner:GAIKWAD RAJESH NALLASOPARA +7

Method and system for quickly identifying tiny foreign matters in reactor pool

The invention relates to the technical field of nuclear power station monitoring, and relates to a method and system for rapidly identifying small foreign matters in a reactor pool, and the method comprises the steps: firstly, processing a reference image and a matched image based on an RAFT optical flow method combined with an attention mechanism to obtain an optical flow graph, and then carrying out the segmentation of the optical flow graph to extract a motion region; and the motion tends to be mapped to the matched image to obtain a foreign matter recognition result. According to the invention, by adding an attention mechanism, the context processing capability of the RAFT method is enhanced, and the optical flow of the water surface ripples can be calculated, so that the precision of optical flow estimation is improved.
Owner:CHINA GENERAL NUCLEAR POWER OPERATION

Efficient context handling for RRC-inactive in 5G

ActiveUS12232202B2Computer networkEngineering
An apparatus, method and computer program product of a new functional entity, namely an RRC-Inactive Context Hosting (RICH) entity, receiving, from a radio access node, a Push UE context notification or a Fetch UE Context Request. In response to the receiving of the Push UE context notification, the RICH entity sends the radio access node an ACK or NACK, while in response to the receiving of the Fetch UE Context Request, the RICH entity retrieves information requested. In response to the sending of the ACK, the RICH entity stores information contained in the notification, while in response to the retrieving, the RICH entity sends the radio access node the retrieved information.
Owner:NOKIA SOLUTIONS & NETWORKS OY

Key value neural network architecture

The present disclosure relates to techniques for improving inference efficiency and memory utilization in transformer-based neural networks. A modified architectural design is introduced that decouples key and value matrix generation from inter-layer dependencies, enabling statically computed or parallelizable projections across layers. The disclosed approach may eliminate the need for layer-wise prefilling, support linear-time inference, and substantially reduce the memory footprint associated with key-value (KV) caching. The disclosed architecture may use shared or layer-specific projections, with a single KV-cache serving all or subsets of layers. In some embodiments, a non-linear transformation (e.g., implemented via a feed-forward network), may preprocess input embeddings prior to query, key, and value generation. A lookup table of transformed embeddings may be precomputed to further accelerate inference. The disclosed system can enhance scalability and may allow deployment of large models on resource-constrained hardware, offering practical benefits for latency-sensitive applications and long-context processing in transformer-based models.
Owner:WRITER INC

Automatic user interface customization based on machine learning processing

A processor may receive a request to display a user interface (UI) for a user account. The processor may determine a context for the UI from information specific to the user account. The processor may select one of a plurality of UI elements based on the determining. In some embodiments, the selecting may include evaluating a value function taking the information specific to the user account and information about the context as inputs. The processor may cause the UI including the selected one of the plurality of UI elements to be displayed in response to the request.
Owner:INTUIT INC

Multi-context dynamic line buffer management for image processing

A system includes an image signal processor (ISP). The ISP is a discrete hardware unit and includes a line buffer, a memory controller, and one or more image processors. The memory controller is configured to assign blocks within the line buffer to a plurality of contexts, and for each of the contexts: receive one or more lines of image data associated with the respective context, and storing the one or more lines of the image data associated with the context in a respective block assigned to the respective context. The one or more image processors of the ISP are configured to process, for each respective context of the plurality of contexts, the one or more lines of the image data stored in the line buffer associated with the respective context.
Owner:QUALCOMM INC

Real-time job intelligent recommendation system based on instant messaging context

The application relates to the technical field of recruitment, and discloses a real-time position intelligent recommendation system based on an instant communication context, which comprises an instant communication access module, a real-time context processing engine, a multi-dimensional semantic understanding module, a dynamic position matching engine, a real-time recommendation pushing module, a data storage module, a permission management module, a business opportunity conversion module and a compliance processing module, and each module cooperatively works; the application solves the context split problem, realizes seamless experience of "communication is data", through efficient communication architecture of the instant communication access module and the back end, real-time capture of bidirectional chat messages of HR and candidates, customers, breaking of the split state of the communication scene and the recommendation system in the prior art, inclusion of each interaction into the recommendation consideration, complete capture of dynamic demands and implicit demands in the communication process, and avoidance of demand information omission.
Owner:AOYE HUMAN RESOURCES (GUANGDONG) CO LTD

Image segmentation method based on edge-to-global feature fusion network

According to the EGANet polyp image segmentation method provided by the invention, the omission ratio and the error detection ratio can be reduced, and the colonoscopy accuracy is improved. The method is designed for three challenges: a ResNet50-SPD encoder is combined with space-to-depth layer downsampling, and a deconvolution decoder is matched, so that the problem of missing detection due to too small polyp size is solved; designing an edge attention module, supervising an advanced feature layer by using edge information, and coping with missing detection and under-segmentation caused by morphological differences; through data set reflection removal and color shift preprocessing, a global attention module is combined to capture long-distance dependence and channel context, and error detection caused by polyp and background similarity is processed. And finally fusing edge and global attention features to relieve over-segmentation. Experiments show that the segmentation precision of the scheme is superior to that of an existing model.
Owner:UNIV OF JINAN

Edge computing based environmental context processing and model optimization method and system

The present application relates to the technical field of data processing, and more particularly to an environment context processing and model optimization method and system based on edge computing, which stores initial data in each time period together with historical data in a target storage, guarantees the continuity and integrity of data in the time dimension, and fuses data of different sources and times to mine the association between current data and past environment states, more comprehensively grasps the state of the target physical place and each target object at the end of each time period, so that the target storage stores continuous historical data over time, facilitates understanding the dynamic changes of the target physical place and the target object in combination with context data, enables the edge world model to better adapt to the characteristics and changes of the target physical place, improves the adaptability between the edge world model and the target physical place, and improves the completion efficiency and completion effect of the target application task.
Owner:BEIJING QIDAISONG TECH CO LTD

Robust enhancement method and system for long context processing based on dynamic adjustment of neurons

The application discloses a long context processing robust enhancement method and system based on neuron dynamic adjustment, and belongs to the technical field of text sequence processing, and comprises the following steps: embedding a nerve regulation unit in each Transform layer of a large language model used for a long context processing task; and the nerve regulation unit sequentially performs the following steps: dynamically activating abnormal perception, that is, adaptively calculating an abnormal detection threshold of a current layer according to activation value statistical characteristics of the current layer; activating a hard-gate truncation, that is, generating a binary hard mask according to the abnormal detection threshold, and the binary hard mask realizes deterministic isolation of abnormal components in the activation value while keeping normal components unchanged; and performing hierarchical response compression, that is, performing learnable dimension-level linear compression on the isolated abnormal components, re-integrating the compressed abnormal components and the normal components through gate fusion, and outputting an activation tensor after suppressing the abnormality. The application can prevent rank collapse and gradient oscillation, and enhance the robustness of long context task execution.
Owner:ZHEJIANG UNIV