Intelligent agent decision-making model based on memory-learning cooperation and application thereof

Abstract

The invention discloses an agent decision-making model based on memory-learning collaboration and application thereof, and the method comprises the steps: S1, constructing an agent feature attribute module which comprises the static attribute feature and the dynamic attribute feature of each agent; s2, constructing an intelligent agent behavior set module which is used for selecting intelligent agent individual behaviors and organizing behaviors, and S3, constructing an intelligent agent behavior mode module which combines the state of the intelligent agent behavior mode module with perceived information and refers to an accumulated memory set of the intelligent agent behavior mode module, generating an action plan of the next step under the guidance of the behavior rule according to the following formula, so as to update the own state; step S4, constructing an interaction mechanism between intelligent agents, wherein the interaction mechanism comprises a memory mechanism formed by an individual memory library, a group memory library and a memory buffer pool; interaction between the intelligent agent and the environment and between the intelligent agent and other intelligent agents is established, and the intelligent agent continuously updates a learning mechanism of a decision strategy in continuous learning; a memory and learning collaborative decision-making mechanism with double functions of history and experience is introduced in the decision-making process of the intelligent agent; a cyclic feedback mechanism between decision memory and learning of the agent is established, a memory credibility evaluation mechanism is provided, the memory credibility is dynamically evaluated, self-adaptive updating of the memory is realized, and effective support is provided in the decision process, so that the modeling quality of the individual Agent is improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and intelligent agent systems, and in particular relates to an intelligent agent decision-making model based on memory-learning collaboration and its application. Background Art

[0002] In the early stages of artificial social modeling, the core modeling method was agent-based modeling and simulation, which is also the main research method of complex adaptive system theory. The construction of individual agent models has always been the focus of research and the foundation of model construction. In the early stages of research, the basic theoretical framework of agents was the BDI (Belief-Desire-Intention) model, which defined the agent's beliefs, desires, and intentions. Figure 3 These elements became the classic paradigm for early individual agent modeling techniques, driving the evolution from rule-based agents to learning agents. With continuous technological advances, particularly breakthroughs in artificial intelligence and machine learning, individual agent models have gradually integrated more advanced algorithms to simulate more complex and realistic social phenomena. These modeling methods have not only played an important role in theoretical research but have also been widely applied in key areas such as public policy making, urban planning, economic forecasting, and environmental management. By simulating underlying social dynamics and their possible outcomes, these models provide decision makers with deep insights and scientific support, laying the foundation for solving complex real-world problems. In recent years, with the development of large language models (LLMs), a growing number of studies have begun to use LLMs as core controllers to construct individual agent models, enabling key human intelligence capabilities such as contextual learning, continuous learning, and complex reasoning. Artificial societies based on LLMs exhibit characteristics highly similar to real social systems at multiple levels, including organizational collaboration, rational competition, information dissemination, and group emergence.

[0003] Current research on individual agent modeling techniques has incorporated memory mechanisms as key modules, aiming to optimize agents' adaptability in dynamic environments. However, while some studies have considered the role of memory in agent modeling, existing modeling approaches are often simplistic and focus on the construction of individual or group memory, with little consideration of the relationships between individual and group memory. In particular, systematic research on how to extract useful experience from individual memory and form group memory from this experience remains significantly lacking. Current research generally views memory models as static storage mechanisms, lacking dynamic updating mechanisms. Static memory models typically simply store information without considering how to adjust and update memory content based on environmental changes and behavioral feedback. This results in excessively complex memory information and a lack of real-time validity and accuracy. As the environment changes and agent behavior evolves, static memory often fails to fully reflect the latest experience and knowledge, thus impacting the quality of the agent's decision-making.

[0004] Furthermore, existing memory modeling methods fail to adequately address the issues of information retrieval efficiency and quality. Due to the lack of efficient retrieval mechanisms, information stored in memory is often not effectively classified and optimized, leading agents to over-reliance on historical experience or excessive information clutter during decision-making. This not only reduces the efficiency of the memory system but can also cause agents to make outdated or inappropriate decisions in the current environment, thereby impacting overall system performance. While some existing research has begun to explore the dynamic updating of memory mechanisms and the interaction between individuals and groups, these studies are typically limited to specific scenarios or types of agent models, lacking systematic analysis and optimization for a wider range of application scenarios. Therefore, designing memory models that can be continuously updated and optimized in dynamic environments has become a key optimization direction for improving individual agent modeling capabilities and enhancing system adaptability. Summary of the Invention

[0005] In response to the difficulties in the prior art, the present invention provides a solution to the problems in the prior art. The present invention proposes a memory-learning collaborative agent decision-making model. By introducing a memory buffer pool, individual memory information is collected, and this information is continuously extracted and integrated during the system evolution process to construct group memory. In response to the transformation of individual memory into group memory, the present invention further introduces value evaluation, and conducts a comprehensive evaluation based on the value error and rarity of the memory to ensure that the information stored in the group memory has a higher decision-making value. In addition, in response to the shortcomings of the memory and learning collaborative decision-making algorithm in current research, the present invention proposes a memory credibility evaluation mechanism, which realizes the adaptive update of memory by dynamically evaluating the memory credibility, and provides effective support in the decision-making process, thereby improving the quality of individual agent modeling. The specific technical solution of the present invention is as follows: A method for an intelligent agent decision-making model based on memory-learning collaboration, comprising the following steps:

[0006] Step S1: construct an agent feature attribute module, wherein the feature attribute module includes static attribute features and dynamic attributes of each agent;

[0007] Step S2: Construct an agent behavior set module, which is used to select individual agent behaviors and organizational behaviors:

[0008] Step S3: Construct an agent behavior pattern module. The agent behavior pattern module combines its own state with the perceived information and refers to its accumulated memory set to generate the next action plan under the guidance of the behavior rules according to the following formula and thus updates its own state;

[0009] R×S t ×Y t ×M t →A t ×S t

[0010] Step S4: Constructing an interaction mechanism between agents, including:

[0011] The memory mechanism is composed of individual memory banks, group memory banks, and memory buffer pools. It establishes a learning mechanism that allows the agent to interact with the environment and other agents, and continuously update its decision-making strategy during continuous learning.

[0012] Based on the decision-making process of the intelligent agent, a memory and learning collaborative decision-making mechanism with the dual effects of "history + experience" is introduced;

[0013] Establish a circular feedback mechanism between the agent's decision memory and learning.

[0014] Furthermore, the memory mechanism establishment process includes:

[0015] Construct an individual memory bank: The individual memory bank stores the memory information experienced by the individual; wherein: the individual memory bank stores the memory information unique to a single agent, including the experience and knowledge accumulated in the process of interacting with the environment and other agents. The storage process of individual memory is described as follows:

[0016]

[0017] in, is a new memory item, defined as follows:

[0018]

[0019] The information in the memory item includes the memory type type, the local information observed by the individual in the previous time step Actions executed at the current time step and the observed local information at the current time step

[0020] Constructing a memory buffer pool: The information in the memory buffer pool is the individual memory information in each time step, and the memory items are screened and integrated into the group memory set under appropriate conditions; the storage process of the buffer pool is described as follows;

[0021] Constructing a group memory set: The group memory set is used to store information shared by all agents in the entire system. Its specific evaluation function is as follows:

[0022]

[0023] in:

[0024] ·δ t is the value error of the memory item, which is used to measure the impact of the decision in the memory item on the overall strategy; |δ t A larger | indicates that the memory item has a greater impact on the strategy, regardless of whether it is a positive experience or a negative experience; δ t The specific calculation formula is as follows:

[0025] δ t =γV(s t+1 )-V(s t )

[0026] Among them, V is the state value function, which is used to measure the value of the current state; γ is the discount factor, which is used to determine the importance of future value in the current decision.

[0027] ·R(m i) is a rarity evaluation index for a memory item, which is used to measure whether there is similar experience in the current system. If there is no similar experience in the group memory set, then the memory item is worth storing regardless of its value error. The calculation formula for rarity is as follows:

[0028]

[0029] ·θ value and θ rare are the value error threshold and the rarity threshold, respectively.

[0030] Furthermore, a memory and learning collaborative decision-making mechanism process is established, including:

[0031] Extract individual set-based memory items according to time steps and input them into the memory buffer pool;

[0032] The memory buffer pool is integrated with the group memory set through value evaluation as the global information of the experience of multiple agents at different time steps;

[0033] The empirical global information is used to construct the agent behavior decision model according to the following formula:

[0034]

[0035] Among them, C memory is the credibility of memory experience, which is used to measure whether the experience in memory is applicable to the current decision; C memory The calculation is based on three core elements:

[0036]

[0037] in:

[0038] ·S env Indicates environmental similarity: the degree of match between the current state and historical memory (range 0-1);

[0039] S success Represents the positive return ratio: the positive return ratio of the same decision in history (range 0-1);

[0040] It is expressed as the degree of memory decay, and λ is the decay factor;

[0041] ω1, ω2, and ω3 are weight parameters, and need to satisfy ω1+ω2+ω3=1.

[0042] Furthermore, a loop feedback mechanism process between the agent's decision memory and learning is established; including:

[0043] Memory update mechanism: After each agent makes a decision, the results of its decision and its interaction with the environment will be stored in the agent's individual memory bank. At the same time, if the decision has an impact on group behavior, the corresponding high-quality decision experience will be extracted and stored in the group memory bank.

[0044] Learning mechanism adjustment: Whenever the agent updates its decision-making strategy through reinforcement learning or other learning algorithms, it makes adjustments based on historical experience information in the memory bank. When the agent is learning, it first retrieves historical decision-making experiences similar to the current situation from the memory bank and adjusts its learning strategy based on these experiences.

[0045] Feedback integration mechanism: The behavioral adjustment results after each learning of the intelligent agent will be used as feedback information and entered into the memory bank, especially the individual memory bank and the group memory bank, to form a cycle of decision-making and experience; the immediate decision-making experience in the memory buffer pool will be evaluated for its effectiveness after each round of learning. If the experience can promote the improvement of the model, it will be added to the long-term memory bank; throughout the process, memory and learning are updated in parallel. The learning mechanism makes decisions based on the current memory, while the memory bank continuously updates the stored content according to the learning results.

[0046] Dynamic adaptation and adjustment: Over time, the agent's learning and memory will continue to iterate and optimize, forming an adaptive feedback system. Whenever the agent makes a decision in a new environment or situation, historical learning and memory will provide it with effective decision-making support and will be continuously updated and corrected through feedback.

[0047] Furthermore, the agent decision model is:

[0048] Agent= <R,S t ,E t ,Y t ,A t ,M t ,N>

[0049] Among them, R represents the static characteristics of the agent; S t Represents the dynamic characteristics of the intelligent agent; E t It is a collection of external events perceived by the agent. These events may come from the environment or other agents. After being input as observation information, they may affect the agent's behavior and decision-making; Y t It is the decision-making mechanism adopted by the intelligent agent when receiving external sensory information and stimulation or interacting with other intelligent agents; t It is the set of behaviors of the agent, including the behaviors taken by the agent under external stimuli and the behaviors generated spontaneously; M t is the memory collection accumulated by the agent; N is the constraints imposed on the agent, including environmental constraints and target constraints.

[0050] This invention innovatively optimizes the individual agent model used in artificial social modeling, focusing on the introduction of a memory mechanism that enables each agent to fully utilize its historical experience during decision-making. This improvement not only enhances the agent's adaptability but also significantly accelerates the learning process. To verify the effectiveness of this model, we conducted a detailed comparative analysis with existing memory models. We constructed multiple agents based on different learning mechanisms and conducted systematic simulation experiments. The experimental results demonstrate that agents incorporating a memory mechanism significantly improve decision accuracy and strategy stability when handling complex scenarios. Furthermore, comparisons with existing models reveal that the optimized model proposed in this invention performs superiorly across multiple key metrics. In particular, in terms of model realism and accuracy, the optimized model better reflects the complex behavior of real-world systems. By incorporating a memory mechanism, agents can not only make decisions based on real-time information but also flexibly adjust their strategies to cope with changing environments. Simulation results are also more consistent with the behavior patterns of real-world agents. Therefore, this improvement not only enhances the model's intelligence but also provides a more reliable theoretical basis and practical guidance for further research and application in artificial social modeling.

[0051] Beneficial effects

[0052] This invention optimizes the decision-making process of individual agents by synergizing memory and learning mechanisms. Individual behavior is the result of the synergistic effect of memory and learning, and this behavior not only updates the information in the memory set but also further optimizes the individual's decision-making. Through these two feedback mechanisms, individuals can not only utilize learning strategies to improve their short-term decision-making capabilities, but also leverage memory mechanisms to continuously optimize their long-term decision-making framework, improving overall decision-making performance. Therefore, this invention significantly improves the adaptability and decision-making efficiency of individual agents in complex environments, providing effective support for autonomous optimization and improvement of long-term decision-making frameworks in dynamically changing environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Implementation Flowchart

[0054] Figure 2 Overall structure of the individual agent model based on memory learning and collaboration

[0055] Figure 3 Memory storage module structure diagram

[0056] Figure 4 Diagram of the Agent decision-making process of memory and learning collaboration

[0057] Figure 5Comparison of the average daily income of imitation learning agents under different memory models

[0058] Figure 6 Comparison of average daily returns of reinforcement learning agents under different memory models

[0059] Figure 7 Comparison of the average daily revenue of the large language model agent under different memory models

[0060] Figure 8 Comparison of Agent's average daily completed orders and effective working time under different decision models

[0061] Figure 9 System visualization platform DETAILED DESCRIPTION

[0062] In order to more clearly illustrate the operation process of the present invention and deepen the understanding of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and examples. The embodiments are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0063] The implementation flow chart of the present invention is as follows Figure 1 As shown, the implementation process of the intelligent agent decision model based on memory-learning collaboration proposed in the present invention is provided, which specifically includes the following steps:

[0064] Step S1: Define the model subject: The model subject is the core research object in the entire model construction process. Its type and characteristics directly affect the system's behavior and decision-making process. In this invention, the subject can be a single individual (such as a human, animal, robot, vehicle, etc.) or a more complex social organization (such as a business, government, social group, etc.);

[0065] Step S2 defines the model hierarchy: the model subjects are divided into different levels. In this invention, three levels are defined: individual level, organizational level, and social level. Each level has a different evolution mechanism: 1) Individual level: focuses on the individual behavior and decision-making mechanism of each agent, simulating the decision-making process and goal achievement of a single agent; 2) Organizational level: focuses on the collective behavior and collaboration between individual agents, studying how to achieve certain global goals through group cooperation or competition; 3) Social level: involves more macro-level social structure and policy formulation, studying the impact of social organizations and institutions on individual and group behavior;

[0066] Step S3 defines the characteristics and behaviors of the Agent: Define the static and dynamic attributes of each Agent. At the same time, determine the specific behavioral rules of the Agent in the model, including how to make decisions, how to interact, how to learn or adapt to the environment, etc. These rules are usually expressed through mathematical models, heuristic methods, learning algorithms, etc. The overall structure of the individual Agent model based on memory-learning collaboration proposed in this invention is as follows: Figure 2 As shown in Figure 2. In this structure, the continuous flow of information organically links the various parts together to form a unified whole. The specific expression of the Agent model is defined as:

[0067] Agent= <R,S t ,E t ,Y t ,A t ,M t ,N>

[0068] Among them, R represents the static characteristics of the agent, which will not change over time, such as movement speed, gender, etc.; S t Represents the dynamic characteristics of the agent, which will change over time, such as the role and age of the agent; E t It is a collection of external events perceived by the agent. These events may come from the environment or other agents. After being input as observation information, they may affect the agent's behavior and decision-making; Y t It is the decision-making mechanism adopted by the intelligent agent when receiving external sensory information and stimulation or interacting with other intelligent agents; t It is the set of behaviors of the agent, including the behaviors taken by the agent under external stimuli and the behaviors generated spontaneously; M t It is the memory collection accumulated by the agent, including long-term memory and short-term memory; N is the constraint imposed on the agent, including environmental constraints and target constraints.

[0069] Step S4 establishes the interaction mechanism between agents: determines how agents interact locally, which may be through direct communication, cooperation, competition, sharing of resources, etc.

[0070] Step S5: Design a memory mechanism: Design a memory library for the Agent. In this invention, the individual memory library, group memory library, and memory buffer pool are innovatively introduced. The individual memory library stores the memory information experienced by the individual, the memory buffer pool stores the individual memory information within a certain time interval during the model operation, and the group memory library stores the high-quality memory information of the group. These three memory storage sets work together to form the memory storage framework of the Agent system, where Figure 3The memory storage module structure diagram is shown in Figure 1, which describes the relationship between individual memory sets, group memory sets, and memory buffer pools.

[0071] Furthermore, step S5 is specifically as follows:

[0072] Step S51 constructs an individual memory set: This stores the unique memory information of a single agent, including the experience and knowledge accumulated during its interaction with the environment and other agents. This memory typically involves the agent's recent events, environmental conditions, decision-making processes, and their execution results.

[0073] Step S52 constructs a collective memory collection: This collection stores information shared by all agents in the system, known as a collective knowledge base. Compared to individual memory, collective memory focuses more on collaboration and knowledge integration between agents, encompassing the experiences, rules, and patterns contributed by different agents. This sharing mechanism promotes the dissemination and reuse of knowledge, improving the overall intelligence of the agent group.

[0074] Step S53 constructs a memory buffer pool: This pool is primarily used to temporarily store key information experienced by the agent over a short period of time and, under appropriate conditions, filter and integrate memory items into the collective memory set. Because the agent system generates a large amount of short-term memory during operation, and this information may not always have long-term value, the memory buffer pool plays a role in information screening and management.

[0075] Step S54: Transform memory buffer pool information into group memory set: Extracting memory from the memory buffer pool set is a key step in group experience dissemination. The memory evaluation method of the present invention comprehensively considers the value error and rarity of memory. Its specific evaluation function is as follows:

[0076]

[0077] Where: t is the value error of the memory item, which is used to measure the impact of the decision in the memory item on the overall strategy. i ) is the rarity evaluation index of the memory item, which is used to measure whether there is similar experience in the current system, θ value and θ rare are the value error threshold and the rarity threshold, respectively.

[0078] Step S6: Design a learning mechanism. This step reiterates the core role of the learning mechanism in this invention. Through continuous interaction with the environment and other agents, agents continuously update their decision-making strategies through continuous learning. This involves not only individual learning but also the incorporation of collective intelligence at the group level to ensure that each agent can collectively acquire information and update its strategy.

[0079] Step S7: Establish memory and learning collaborative decision-making mechanism: Memory and learning mechanism are coordinated, and the dual role of "history + experience" is introduced in the decision-making process of the agent. The memory and learning collaborative decision-making process is as follows: Figure 4 For example, when an agent faces a new situation, it first extracts similar historical decisions from the memory bank and then adjusts the decision through the learning process. As a global information source, it aggregates the experiences of multiple agents at different time steps to provide a global reference. The specific mathematical description of the decision-making process is as follows:

[0080]

[0081] Among them, C memory The credibility of a memory experience is an important indicator for measuring whether the stored memory is applicable to the current decision. The credibility assessment mainly combines the memory decay index, historical positive return ratio, and environmental similarity.

[0082] Step S8: Establishing a cyclic feedback mechanism between memory and learning: To improve the model's decision-making capabilities and quality, the present invention establishes a cyclic feedback mechanism between memory and learning. Memory can assist the learning mechanism in making decisions, and the memory information stored in the decision-making process, in turn, influences the evolution of the learning mechanism.

[0083] Furthermore, step S7 is specifically as follows:

[0084] Step S81: Memory Update Mechanism: Each time an agent completes a decision, the outcome and its interaction with the environment are stored in the agent's individual memory. Simultaneously, if the decision impacts group behavior, the corresponding high-quality decision experience is extracted and stored in the group memory. Furthermore, any immediate experience that significantly impacts the system (such as special events or critical decisions) is stored in the memory buffer for quick access.

[0085] Step S82: Learning Mechanism Adjustment: Whenever the agent updates its decision-making strategy through reinforcement learning or other learning algorithms, adjustments are made based on historical experience information in its memory bank. When learning, the agent first retrieves historical decision-making experiences similar to the current situation from its memory bank and adjusts its learning strategy based on these experiences. The learning algorithm adjusts the priority of the agent's behavior based on the updated decision-making strategy and re-evaluates the effectiveness of the new strategy after each interaction.

[0086] Step S83: Feedback Integration Mechanism: Each time the agent learns, its behavioral adjustments serve as feedback and are entered into the memory pool, specifically the individual and group memory pools, forming a cycle of decision-making and experience. The immediate decision-making experience in the memory buffer pool is evaluated for effectiveness after each round of learning. If the experience contributes to model improvement, it is added to the long-term memory pool. Throughout this process, memory and learning are updated in parallel. The learning mechanism makes decisions based on current memory, while the memory pool continuously updates its stored content based on the learning results.

[0087] Step S84: Dynamic Adaptation and Adjustment: Over time, the agent's learning and memory will continuously iterate and optimize, forming an adaptive feedback system. Whenever the agent makes a decision in a new environment or situation, its historical learning and memory will provide effective decision support, continuously updated and revised through feedback. This dynamic feedback loop enables the agent to maintain high decision-making efficiency in an ever-evolving environment, avoiding performance degradation caused by outdated memory or over-reliance on immediate feedback.

[0088] This invention innovatively optimizes the individual agent model used in artificial social modeling, focusing on the introduction of a memory mechanism that enables each agent to fully utilize its historical experience during decision-making. This improvement not only enhances the agent's adaptability but also significantly accelerates the learning process. To verify the effectiveness of this model, we conducted a detailed comparative analysis with existing memory models. We constructed multiple agents based on different learning mechanisms and conducted systematic simulation experiments. The experimental results demonstrate that agents incorporating a memory mechanism significantly improve decision accuracy and strategy stability when handling complex scenarios. Furthermore, comparisons with existing models reveal that the optimized model proposed in this invention performs superiorly across multiple key metrics. In particular, in terms of model realism and accuracy, the optimized model better reflects the complex behavior of real-world systems. By incorporating a memory mechanism, agents can not only make decisions based on real-time information but also flexibly adjust their strategies to cope with changing environments. Simulation results are also more consistent with the behavior patterns of real-world agents. Therefore, this improvement not only enhances the model's intelligence but also provides a more reliable theoretical basis and practical guidance for further research and application in artificial social modeling.

[0089] The system proposed in this invention is verified by constructing an instant delivery system in a smart city. The experimental results are shown in Figure 5-8 Experimental results show that compared with existing memoryless modeling methods, single-agent memory modeling methods, and group-agent memory modeling methods, the agent constructed by the model proposed in this invention exhibits better decision-making quality and adaptability in systems with different decision-making algorithms. This verifies the effectiveness of the individual agent model based on the memory-learning synergy mechanism proposed in this invention in improving the modeling quality of the individual layer in artificial social modeling and realizing anthropomorphic features.

[0090] In addition, the present invention also integrates the constructed system with a visual modeling tool, so that the behavior pattern of the agent in the system evolution process can be intuitively presented, and the emergence phenomenon from the micro-agent decision-making to the macro-system can be clearly observed. Figure 9 This not only enhances the transparency of system operation but also facilitates in-depth exploration of the interaction mechanisms and evolutionary patterns of multi-agent systems. This system has a wide range of applications, extending beyond just instant delivery systems to include autonomous driving, robot collaboration, and intelligent traffic scheduling.

[0091] Although the present invention has been described above, the present invention is not limited to the above-mentioned specific embodiments. The above-mentioned specific embodiments are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can make many variations without departing from the purpose of the present invention, and these are all protected by the present invention.

Claims

1. An application of an intelligent agent decision model based on memory-learning collaboration, characterized in that: The steps include: Step S1: construct an agent feature attribute module, wherein the feature attribute module includes static attribute features and dynamic attributes of each agent; Step S2: Construct an agent behavior set module, which is used to select individual agent behaviors and organizational behaviors: Step S3: Construct an agent behavior pattern module. The agent behavior pattern module combines its own state with the perceived information and refers to its accumulated memory set to generate the next action plan under the guidance of the behavior rules according to the following formula and thus updates its own state; R×S t ×Y t ×M t →A t ×S t Step S4: Constructing an interaction mechanism between agents, including: The memory mechanism is composed of individual memory banks, group memory banks, and memory buffer pools. It establishes a learning mechanism that allows the agent to interact with the environment and other agents, and continuously update its decision-making strategy during continuous learning. Based on the decision-making process of the intelligent agent, a memory and learning collaborative decision-making mechanism with the dual effects of "history + experience" is introduced; Establish a circular feedback mechanism between the agent's decision memory and learning.

2. The application of the intelligent agent decision model based on memory-learning collaboration according to claim 1, characterized in that: The memory mechanism establishment process includes: Construct an individual memory bank: The individual memory bank stores the memory information experienced by the individual; wherein: the individual memory bank stores the memory information unique to a single agent, including the experience and knowledge accumulated in the process of interacting with the environment and other agents. The storage process of individual memory is described as follows: in, is a new memory item, defined as follows: The information in the memory item includes the memory type type, the local information observed by the individual in the previous time step Actions executed at the current time step and the observed local information at the current time step Constructing a memory buffer pool: The information in the memory buffer pool is the individual memory information in each time step, and the memory items are screened and integrated into the group memory set under appropriate conditions; the storage process of the buffer pool is described as follows; Constructing a group memory set: The group memory set is used to store information shared by all agents in the entire system. Its specific evaluation function is as follows: in: ·δ t is the value error of the memory item, which is used to measure the impact of the decision in the memory item on the overall strategy; |δ t A larger | indicates that the memory item has a greater impact on the strategy, regardless of whether it is a positive experience or a negative experience; δ t The specific calculation formula is as follows: δ t =γV(s t+1 )-V(s t ) Among them, V is the state value function, which is used to measure the value of the current state; γ is the discount factor, which is used to determine the importance of future value in the current decision. ·R(m i ) is a rarity evaluation index for a memory item, which is used to measure whether there is similar experience in the current system. If there is no similar experience in the group memory set, then the memory item is worth storing regardless of its value error. The calculation formula for rarity is as follows: ·θ value and θ rare are the value error threshold and the rarity threshold, respectively.

3. The application of the intelligent agent decision model based on memory-learning collaboration according to claim 1, characterized in that: The process of establishing a collaborative decision-making mechanism for memory and learning includes: Extract individual set-based memory items according to time steps and input them into the memory buffer pool; The memory buffer pool is integrated with the group memory set through value evaluation as the global information of the experience of multiple agents at different time steps; The empirical global information is used to construct the agent behavior decision model according to the following formula: Among them, C memory is the credibility of memory experience, which is used to measure whether the experience in memory is applicable to the current decision; C memory The calculation is based on three core elements: in: ·S env Indicates environmental similarity: the degree of match between the current state and historical memory (range 0-1); S success Represents the positive return ratio: the positive return ratio of the same decision in history (range 0-1); It is expressed as the degree of memory decay, and λ is the decay factor; ω1, ω2, and ω3 are weight parameters, and need to satisfy ω1+ω2+ω3=1.

4. The application of the intelligent agent decision model based on memory-learning collaboration according to claim 1, characterized in that: Establishing a feedback loop between the agent's decision memory and learning; including: Memory update mechanism: After each agent makes a decision, the results of its decision and its interaction with the environment will be stored in the agent's individual memory bank. At the same time, if the decision has an impact on group behavior, the corresponding high-quality decision experience will be extracted and stored in the group memory bank. Learning mechanism adjustment: Whenever the agent updates its decision-making strategy through reinforcement learning or other learning algorithms, it makes adjustments based on historical experience information in the memory bank. When the agent is learning, it first retrieves historical decision-making experiences similar to the current situation from the memory bank and adjusts its learning strategy based on these experiences. Feedback integration mechanism: The behavioral adjustment results after each learning of the intelligent agent will be used as feedback information and entered into the memory bank, especially the individual memory bank and the group memory bank, to form a cycle of decision-making and experience; the immediate decision-making experience in the memory buffer pool will be evaluated for its effectiveness after each round of learning. If the experience can promote the improvement of the model, it will be added to the long-term memory bank; throughout the process, memory and learning are updated in parallel. The learning mechanism makes decisions based on the current memory, while the memory bank continuously updates the stored content according to the learning results. Dynamic adaptation and adjustment: Over time, the agent's learning and memory will continue to iterate and optimize, forming an adaptive feedback system. Whenever the agent makes a decision in a new environment or situation, historical learning and memory will provide it with effective decision-making support and will be continuously updated and corrected through feedback.

5. An intelligent agent decision model based on memory-learning collaboration, characterized in that: The intelligent agent decision model is constructed by applying any one of claims 1 to 4, wherein the intelligent agent decision model is: Agent=<R,S t ,E t ,Y t ,A t ,M t ,N> Among them, R represents the static characteristics of the agent; S t Represents the dynamic characteristics of the intelligent agent; E t It is a collection of external events perceived by the agent. These events may come from the environment or other agents. After being input as observation information, they may affect the agent's behavior and decision-making; Y t It is the decision-making mechanism adopted by the intelligent agent when receiving external sensory information and stimulation or interacting with other intelligent agents; t It is the set of behaviors of the agent, including the behaviors taken by the agent under external stimuli and the behaviors generated spontaneously; M t is the memory collection accumulated by the agent; N is the constraints imposed on the agent, including environmental constraints and target constraints.

Images