A multimodal cognitive input driven adaptive memory engine construction method and optimization system

Abstract

The application discloses a kind of multi-modal cognitive input driven adaptive memory engine construction method and optimization system, it is related to artificial intelligence technical field, including the following steps, S1, acquisition and pre-processing multi-modal cognitive input data, the multi-modal cognitive input data includes text semantic data, visual perception data, speech interaction data and environment state data, to each kind of data is respectively normalized noise reduction processing, feature extraction and modal feature alignment processing between it.This open and close tank cap assembly and tire curing tank, by collecting and pre-processing text, vision, voice and environment state and so on Multi-modal data, modal feature alignment is realized using cross-modal attention mechanism, eliminate modal heterogeneity, can accurately capture user multidimensional cognitive demand, at the same time, through hierarchical intent recognition and scene understanding, combined with the hierarchical retrieval strategy of three-layer memory structure, the real-time and integrity of memory response are considered.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and optimization system for constructing an adaptive memory engine driven by multimodal cognitive input. Background Technology

[0002] With the rapid development of artificial intelligence technology, multimodal interaction has become the mainstream trend of human-computer interaction, and multimodal cognitive input scenarios such as text, vision, and voice are becoming increasingly widespread.

[0003] Existing memory engines are mostly designed for single-modal input, which has problems such as poor modal adaptability, lag in memory updates, and insufficient scene adaptability. They are difficult to accurately capture users' multi-dimensional cognitive needs. At the same time, traditional memory engines lack dynamic adaptive mechanisms, with fixed memory weights and unadjustable association strengths, which can easily lead to invalid memory redundancy and omission of key memories. This results in low memory response accuracy and poor real-time performance, making it impossible to meet the cognitive interaction needs in complex scenarios.

[0004] Therefore, in view of this, we study and improve the existing structure and its shortcomings, and propose a method and optimization system for constructing an adaptive memory engine driven by multimodal cognitive input. Summary of the Invention

[0005] The purpose of this invention is to provide a method and optimization system for constructing an adaptive memory engine driven by multimodal cognitive input, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for constructing an adaptive memory engine driven by multimodal cognitive input, comprising the following steps, S1. Collect and preprocess multimodal cognitive input data, which includes text semantic data, visual perception data, voice interaction data and environmental state data. Normalize and denoise the various types of data, extract features and align features between modalities to eliminate modal heterogeneity and form a multimodal feature vector with a unified dimension and format to ensure the consistency of subsequent cognitive analysis. S2. Based on the multimodal feature vectors obtained in step S1, perform cognitive intent parsing and scene understanding. Deeply encode the multimodal feature vectors through a pre-trained cognitive model to extract the user's explicit basic interaction intent, implicit task objectives and real-time interaction scene information. Integrate multi-dimensional features to construct a representation of the current cognitive state, thereby achieving an accurate depiction of the user's cognitive needs. S3. Based on the cognitive state representation constructed in step S2, dynamically activate memory units. Using a feature similarity matching algorithm, accurately match and retrieve relevant memory fragments from the three-layer memory structure of short-term memory cache, medium-term associative memory, and long-term knowledge memory, filter out invalid memory information, and form an initial memory activation set. S4. Adaptive weight update and association enhancement are performed on the initial memory activation set obtained in step S3. Based on the importance score of cognitive input, the frequency of interaction repetition and the scene relevance coefficient, the storage weight and association strength of each memory segment are dynamically adjusted in real time to construct a dynamic memory network that can be dynamically iterated and improve the adaptability of memory matching. S5. Based on the updated dynamic memory network, output memory response results that adapt to the current cognitive needs. Before outputting, perform consistency verification and conflict resolution on the response results, remove contradictory memory fragments, and ensure the logical coherence of the response. At the same time, write the newly added cognitive data into the corresponding memory level according to the importance level, complete the iterative update and closed-loop optimization of the memory engine, and achieve adaptive improvement of memory ability.

[0007] Furthermore, in S1, when performing intermodal feature alignment on multimodal cognitive input data, a cross-modal attention mechanism is used to achieve feature fusion and unified representation mapping of text, visual, and speech data.

[0008] Furthermore, in S2, the cognitive intent parsing adopts a hierarchical intent recognition model, which first identifies basic interaction intents, then deeply infers potential task requirements and cognitive preferences, and outputs refined scene understanding results.

[0009] Furthermore, in S3, the three-layer memory structure adopts a hierarchical retrieval strategy, prioritizing the retrieval of short-term memory cache to ensure response speed, and then progressively retrieving medium-term and long-term memory to improve memory integrity.

[0010] Furthermore, in S4, the memory weight update introduces a forgetting mechanism and a reinforcement mechanism. A weight decay threshold and an enhancement threshold are set. Low-relevance, low-frequency memories below the decay threshold are subject to gradient weight decay until they are eliminated. High-frequency key cognitive memories above the enhancement threshold are subject to weight enhancement, and their association strength with related memory fragments is strengthened, thereby achieving a reasonable allocation of memory resources.

[0011] Furthermore, in S5, consistency verification and conflict resolution are performed before the memory response result is output, and contradictory memory fragments are eliminated.

[0012] Furthermore, the cognitive state representation adopts a vector encoding form, which maintains the same vector space as the index encoding of the memory unit.

[0013] Furthermore, contextual temporal information is introduced during the scene understanding process, and multiple rounds of cognitive input are modeled as a whole sequence.

[0014] Furthermore, the dynamic memory network supports incremental updates and adopts a modular design, enabling the rapid insertion of new memory segments and the orderly elimination of old memory segments without reconstructing the overall network structure or affecting existing memory data.

[0015] An optimization system is applied to the aforementioned method for constructing an adaptive memory engine driven by multimodal cognitive input. The system includes a multimodal acquisition module and a weight update module. The output of the multimodal acquisition module is electrically connected to a cognitive parsing module, and the output of the cognitive parsing module is electrically connected to a memory scheduling module. The weight update module is electrically located at the output of the memory scheduling module, and the output of the weight update module is electrically connected to a response output module.

[0016] This invention provides a method and optimization system for constructing an adaptive memory engine driven by multimodal cognitive input, which has the following beneficial effects: By collecting and preprocessing multimodal data including text, vision, speech, and environmental status, and employing a cross-modal attention mechanism to align features between modalities and eliminate modal heterogeneity, the system can accurately capture users' multidimensional cognitive needs. Simultaneously, through hierarchical intent recognition and scene understanding, combined with a graded retrieval strategy based on a three-layer memory structure, it balances the real-time performance and completeness of memory responses. Combined with weight updates via forgetting and reinforcement mechanisms, it achieves rational allocation of memory resources, reduces redundant invalid memories, and improves the accuracy and adaptability of memory responses, significantly outperforming traditional fixed-weight memory engines. Through multi-module collaborative design, it perfectly adapts to the aforementioned construction method, achieving closed-loop operation and adaptive iteration of the memory engine. The multimodal acquisition module ensures comprehensive data collection, the cognitive analysis module accurately characterizes needs, the memory scheduling module and weight update module collaboratively construct a dynamic memory network, and the response output module ensures the accuracy and consistency of results. The system supports incremental updates, completing memory iterations without reconstructing the entire network, reducing operating and maintenance costs. Furthermore, each module is functionally independent yet collaborative, flexibly adapting to different complex interaction scenarios, effectively improving the intelligence level of human-computer interaction, and possessing broad application prospects and practical value. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the optimization system of the multimodal cognitive input-driven adaptive memory engine construction method and optimization system of the present invention.

[0018] In the diagram: 1. Multimodal acquisition module; 2. Cognitive parsing module; 3. Memory scheduling module; 4. Weight update module; 5. Response output module. Detailed Implementation

[0019] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0020] A method for constructing a multimodal cognitive input-driven adaptive memory engine includes the following steps: S1. Collect and preprocess multimodal cognitive input data, which includes text semantic data, visual perception data, voice interaction data, and environmental state data. Normalize and denoise each type of data, extract features, and perform intermodal feature alignment to eliminate modal heterogeneity and form a multimodal feature vector with a unified dimension and format to ensure consistency in subsequent cognitive analysis. In S1, when performing intermodal feature alignment on the multimodal cognitive input data, a cross-modal attention mechanism is used to achieve feature fusion and unified representation mapping of text, visual, and voice data. S2. Based on the multimodal feature vectors obtained in step S1, cognitive intent parsing and scene understanding are performed. The multimodal feature vectors are deeply encoded through a pre-trained cognitive model to extract the user's explicit basic interaction intent, implicit task objectives, and real-time interaction scene information. Multi-dimensional features are integrated to construct the current cognitive state representation, thereby achieving an accurate characterization of the user's cognitive needs. In S2, the cognitive intent parsing adopts a hierarchical intent recognition model, which first identifies basic interaction intents and then deeply infers potential task needs and cognitive preferences, outputting refined scene understanding results. The cognitive state representation adopts a vector encoding form, which maintains the same vector space as the index encoding of the memory unit. Contextual temporal information is introduced during the scene understanding process, and multiple rounds of cognitive input are modeled as a whole sequence. S3. Based on the cognitive state representation constructed in step S2, the memory units are dynamically activated. The feature similarity matching algorithm is used to accurately match and retrieve relevant memory fragments from the three-layer memory structure of short-term memory cache, medium-term associative memory, and long-term knowledge memory, filter out invalid memory information, and form an initial memory activation set. In S3, the three-layer memory structure adopts a hierarchical retrieval strategy, prioritizing the retrieval of short-term memory cache to ensure response speed, and then retrieval of medium-term and long-term memory level by level to improve memory integrity. S4. Adaptive weight update and association reinforcement are performed on the initial memory activation set obtained in step S3. Based on the importance score of cognitive input, the frequency of interaction repetition and the scene relevance coefficient, the storage weight and association strength of each memory segment are dynamically adjusted in real time to construct a dynamically iterative dynamic memory network and improve the adaptability of memory matching. In S4, the memory weight update introduces forgetting mechanism and reinforcement mechanism, sets weight decay threshold and reinforcement threshold, performs gradient weight decay on low relevance and low frequency memories below the decay threshold until they are eliminated, and enhances the weight of high frequency key cognitive memories above the reinforcement threshold and strengthens their association strength with related memory segments to achieve reasonable allocation of memory resources. The dynamic memory network supports incremental updates and adopts a modular design to achieve rapid insertion of new memory segments and orderly elimination of old memory segments without reconstructing the overall network structure or affecting existing memory data. S5. Based on the updated dynamic memory network, output memory response results that adapt to the current cognitive needs. Before outputting, perform consistency verification and conflict resolution on the response results, remove contradictory memory fragments, and ensure the logical coherence of the response. At the same time, write the newly added cognitive data into the corresponding memory level according to the importance level, complete the iterative update and closed-loop optimization of the memory engine, and realize the adaptive improvement of memory ability. In S5, consistency verification and conflict resolution are performed before the memory response results are output, and contradictory memory fragments are removed. like Figure 1 As shown, an optimization system includes a multimodal acquisition module 1, a cognitive parsing module 2, a memory scheduling module 3, a weight update module 4, and a response output module 5. The output of the multimodal acquisition module 1 is electrically connected to the cognitive parsing module 2, and the output of the cognitive parsing module 2 is electrically connected to the memory scheduling module 3. The weight update module 4 is electrically located at the output of the memory scheduling module 3, and the output of the weight update module 4 is electrically connected to the response output module 5. The multimodal acquisition module 1 ensures the comprehensiveness of data acquisition, the cognitive parsing module 2 achieves accurate characterization of requirements, the memory scheduling module 3 and the weight update module 4 collaboratively construct a dynamic memory network, and the response output module 5 ensures the accuracy and consistency of results. The system supports incremental updates and can complete memory iteration without reconstructing the entire network, reducing operating and maintenance costs. Moreover, each module is functionally independent yet collaborative, which can flexibly adapt to different complex interaction scenarios, effectively improving the intelligence level of human-computer interaction, and has broad application prospects and practical value.

[0021] In summary, the proposed method and optimization system for constructing and optimizing a multimodal cognitive input-driven adaptive memory engine firstly collects multimodal cognitive input data, including text, visual, speech, and environmental states, through a multimodal acquisition module 1. This data is preprocessed to eliminate modal heterogeneity, forming a unified format of multimodal feature vectors to provide a foundation for subsequent cognitive analysis. Secondly, a cognitive analysis module 2 uses a pre-trained cognitive model to deeply encode the feature vectors, analyzing the user's explicit and implicit intentions, identifying interaction scenarios, and constructing accurate cognitive state representations. Finally, a memory scheduling module 3, based on these cognitive state representations, employs a hierarchical retrieval strategy to retrieve data from three layers of memory. The system retrieves relevant memory fragments to form an initial memory activation set. Then, the weight update module 4 introduces a forgetting and reinforcement mechanism, dynamically adjusting the weights and association strengths of memory fragments based on the importance, frequency, and contextual relevance of cognitive input, thus constructing an iterative dynamic memory network. Finally, the response output module 5 performs consistency verification and conflict resolution on the memory response results, outputting results adapted to the current cognitive needs. At the same time, it hierarchically writes newly added cognitive data into the corresponding memory level, completing the engine's iterative update. All modules work together to achieve adaptive operation of the memory engine, improving the accuracy and real-time performance of human-computer interaction.

[0022] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for constructing a multi-modal cognitive input driven adaptive memory engine, the method comprising: Includes the following steps, ​ S1. Collect and preprocess multimodal cognitive input data, which includes text semantic data, visual perception data, voice interaction data and environmental state data. Normalize and denoise the various types of data, extract features and align features between modalities to eliminate modal heterogeneity and form a multimodal feature vector with a unified dimension and format to ensure the consistency of subsequent cognitive analysis. S2. Based on the multimodal feature vectors obtained in step S1, perform cognitive intent parsing and scene understanding. Deeply encode the multimodal feature vectors through a pre-trained cognitive model to extract the user's explicit basic interaction intent, implicit task objectives and real-time interaction scene information. Integrate multi-dimensional features to construct a representation of the current cognitive state, thereby achieving an accurate depiction of the user's cognitive needs. S3. Based on the cognitive state representation constructed in step S2, dynamically activate memory units. Using a feature similarity matching algorithm, accurately match and retrieve relevant memory fragments from the three-layer memory structure of short-term memory cache, medium-term associative memory, and long-term knowledge memory, filter out invalid memory information, and form an initial memory activation set. S4. Adaptive weight update and association enhancement are performed on the initial memory activation set obtained in step S3. Based on the importance score of cognitive input, the frequency of interaction repetition and the scene relevance coefficient, the storage weight and association strength of each memory segment are dynamically adjusted in real time to construct a dynamic memory network that can be dynamically iterated and improve the adaptability of memory matching. S5. Based on the updated dynamic memory network, output memory response results that adapt to the current cognitive needs. Before outputting, perform consistency verification and conflict resolution on the response results, remove contradictory memory fragments, and ensure the logical coherence of the response. At the same time, write the newly added cognitive data into the corresponding memory level according to the importance level, complete the iterative update and closed-loop optimization of the memory engine, and achieve adaptive improvement of memory ability.

2. The method of claim 1, wherein the method further comprises: In S1, when performing intermodal feature alignment on multimodal cognitive input data, a cross-modal attention mechanism is used to achieve feature fusion and unified representation mapping of text, visual, and speech data.

3. The method of claim 1, wherein the method further comprises: In S2, the cognitive intent parsing adopts a hierarchical intent recognition model, which first identifies basic interaction intents, then deeply infers potential task requirements and cognitive preferences, and outputs refined scene understanding results.

4. The method of claim 3, wherein the method further comprises: In S3, the three-layer memory structure adopts a hierarchical retrieval strategy, prioritizing the retrieval of short-term memory cache to ensure response speed, and then progressively retrieving medium-term and long-term memory to improve memory integrity.

5. The method of claim 3, wherein the method further comprises: In S4, the memory weight update introduces a forgetting mechanism and a reinforcement mechanism. A weight decay threshold and an enhancement threshold are set. Low-relevance, low-frequency memories below the decay threshold are subject to gradient weight decay until they are eliminated. High-frequency key cognitive memories above the enhancement threshold are subject to weight enhancement and their association with related memory fragments is strengthened, thereby achieving a reasonable allocation of memory resources.

6. The method of claim 5, wherein the method further comprises: In step S5, consistency verification and conflict resolution are performed before the memory response result is output, and contradictory memory fragments are eliminated.

7. The method of claim 3, wherein the method further comprises: The cognitive state representation adopts a vector encoding form, which maintains the same vector space as the index encoding of the memory unit.

8. The method for constructing a multimodal cognitive input-driven adaptive memory engine according to claim 3, characterized in that, The scenario understanding process incorporates contextual temporal information, and models multiple rounds of cognitive input as a whole sequence.

9. The method for constructing a multimodal cognitive input-driven adaptive memory engine according to claim 5, characterized in that, The dynamic memory network supports incremental updates and adopts a modular design, enabling the rapid insertion of new memory segments and the orderly elimination of old memory segments without reconstructing the overall network structure or affecting existing memory data.

10. An optimization system, characterized in that, Its application includes the multimodal cognitive input-driven adaptive memory engine construction method according to any one of claims 1-9, including a multimodal acquisition module (1) and a weight update module (4), wherein the output end of the multimodal acquisition module (1) is electrically connected to a cognitive parsing module (2), and the output end of the cognitive parsing module (2) is electrically connected to a memory scheduling module (3), the weight update module (4) is electrically located at the output end of the memory scheduling module (3), and the output end of the weight update module (4) is electrically connected to a response output module (5).

Images