An Autonomous Decision-Making Method for AI Intelligent Agent Systems Based on Deep Learning

By synchronizing and aligning timestamps of multi-source heterogeneous environmental data and extracting multi-scale convolutional features, combined with hierarchical cognitive memory and dual value network evaluation, an AI agent autonomous decision-making closed loop was constructed, solving the problems of inconsistent state representation and difficulties in cross-modal fusion, and achieving stable and accurate decision-making in complex environments.

CN123070165APending Publication Date: 2026-11-10BEIJING KANGLIAN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611233609.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-14
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing AI agent autonomous decision-making methods suffer from problems such as inconsistent state representation, difficulty in cross-modal fusion, complex temporal synchronization, and decision failure when facing complex multi-source heterogeneous environments, making it difficult to maintain stability and accuracy in dynamic environments.

Method used

By employing timestamp synchronization alignment and multi-channel tensor mapping of multi-source heterogeneous environmental data, combined with multi-scale convolutional feature extraction and self-attention weighted encoding, state representation enhancement is achieved through a hierarchical cognitive memory module, and the value of the strategy is evaluated using a dual value network. A closed-loop feedback optimization mechanism is constructed for parameter updates.

Benefits of technology

It achieves standardized processing of multi-source heterogeneous environmental data, improves the consistency of state representation and the stability of policy generation, enhances the accuracy of decision-making and the robustness of the system, and strengthens the autonomous decision-making ability of AI agents in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123070165A_ABST
    Figure CN123070165A_ABST
Patent Text Reader

Abstract

This invention discloses an autonomous decision-making method for an AI agent system based on deep learning, comprising the following steps: constructing an AI agent autonomous decision-making system; collecting multi-source heterogeneous environmental data and synchronizing and aligning it to construct an original state tensor sequence; performing multi-scale convolutional feature extraction and self-attention weighted encoding to generate a high-dimensional state embedding vector; generating an enhanced state representation vector; obtaining a target execution policy; performing joint state-action evaluation of the target execution policy based on a dual-value network structure, outputting the corresponding policy value vector, and determining the optimal action instruction based on the policy value ranking result; executing the optimal action instruction, obtaining environmental feedback information, and constructing a state transition record set; generating perturbation samples through an adversarial policy perturbation mechanism, performing joint parameter updates to generate a target decision parameter set, thereby realizing autonomous decision-making of the AI ​​agent in a dynamic and complex environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and autonomous decision-making of intelligent agents, and in particular to an autonomous decision-making method for AI intelligent agent systems based on deep learning. Background Technology

[0002] With the development of artificial intelligence technology, AI agents are widely used in scenarios such as robot control, intelligent manufacturing, intelligent operation and maintenance, intelligent interaction, and complex process scheduling. The core objective of autonomous decision-making by AI agents is to form executable action instructions based on the perception of the environment, thereby completing the task objective, under conditions of constantly changing external environments, complex information sources, and time-dependent information. In existing technologies, autonomous decision-making by AI agents typically relies on a basic process of "environmental perception—state representation—policy generation—action execution," where the quality of state representation directly affects the reliability and stability of subsequent policy generation.

[0003] Existing technical solutions mainly include rule-based decision-making methods, supervised learning-based policy fitting methods, and reinforcement learning-based policy optimization methods. Rule-based decision-making methods rely on manually constructing rule bases and state machines, which are prone to rule explosion in complex environments, and the cost of handling conflicts between rules is high. Supervised learning-based policy fitting methods usually learn the mapping relationship from observation to action through historical samples, but they are highly dependent on the sample coverage and are prone to decision failure when the environment distribution changes. Reinforcement learning-based policy optimization methods can improve policy performance through interactive feedback, but in practical applications, they still face engineering and algorithmic problems such as high-dimensional environmental input, multi-source heterogeneity, difficulties in cross-modal fusion, and complex temporal synchronization. Summary of the Invention

[0004] One objective of this invention is to propose an autonomous decision-making method for AI intelligent agent systems based on deep learning. This invention fully utilizes multi-source heterogeneous environmental data acquisition and timestamp synchronization alignment, construction of original state tensor sequences from multi-channel tensor mapping, multi-scale convolutional feature extraction and self-attention weighted encoding of cross-modal state encoding, and three-channel gating update and memory fusion of hierarchical cognitive memory to achieve autonomous decision-making closed-loop processing from environmental perception to policy generation to action value assessment and closed-loop feedback optimization. It has the advantages of strong consistency of state representation, strong cross-modal feature extraction capability, stable memory fusion, high decision accuracy, and strong system robustness.

[0005] A meteorological data transmission method for a cloud-based collaborative meteorological anemometer according to an embodiment of the present invention includes the following steps: Construct an AI agent autonomous decision-making system, which includes an environment perception module, a cross-modal state encoding module, a hierarchical cognitive memory module, a meta-policy generation module, an action value assessment module, and a closed-loop feedback optimization module. Multi-source heterogeneous environmental data is collected by the environmental perception module and synchronized according to timestamps to construct the original state tensor sequence; The original state tensor sequence is processed by multi-scale convolutional feature extraction and self-attention weighted encoding through a cross-modal state encoding module to generate a high-dimensional state embedding vector. The high-dimensional state embedding vector is input into the hierarchical cognitive memory module to generate an enhanced state representation vector; The enhanced state representation vector is input into the meta-policy generation module to obtain the target execution policy; The target execution strategy is input into the action value assessment module. The action value assessment module performs a state-action joint assessment of the target execution strategy based on a dual value network structure, outputs the corresponding strategy value vector, and determines the optimal action instruction based on the strategy value ranking result. Execute the optimal action command and obtain environmental feedback information through the closed-loop feedback optimization module to construct a state transition record set; Based on the state transition record set, perturbation samples are generated through an adversarial strategy perturbation mechanism. Joint parameter updates are performed on the meta-policy generation module and the action value assessment module to generate the target decision parameter set.

[0006] Optionally, the functional modules and interface specifications of the AI ​​agent system are defined on the distributed computing platform. An environmental perception module is deployed to access and collect multi-source heterogeneous environmental data. A cross-modal state encoding module is deployed to perform unified tensor quantization representation and feature encoding on synchronized multi-source data. A hierarchical cognitive memory module is deployed to perform short-term memory writing, long-term semantic memory retrieval, and context-related fusion on the state embedding vector. A meta-policy generation module is deployed to generate candidate policies and output the target execution policy. An action value assessment module is deployed to perform state-action value estimation and output the optimal action instruction. A closed-loop feedback optimization module is deployed to record state transition data and update the execution parameters of the policy network and value network, and establish data flow paths and parameter update paths between modules.

[0007] Optionally, the construction of the original state tensor sequence specifically includes: Multi-source heterogeneous environmental data is collected through the environmental perception module. The multi-source heterogeneous environmental data includes sensor observation data, system operation data, interaction and task data, and historical decision data. The start time, end time, and preset sampling interval of the decision cycle are determined, and a unified time axis is constructed based on the start time, the end time, and the preset sampling interval. Multi-source heterogeneous environmental data are synchronized and aligned according to a unified timeline to generate aligned observation vectors; For each time step in the unified time axis, the aligned observation vectors of each data source within the same time step are concatenated in a fixed order to obtain the unified feature vector of the time step. Set the number of channels and the number of feature slots per channel for the multi-channel tensor mapping. For each time step in the unified time axis, construct the original state tensor of the time step from the unified feature vector of the time step using the multi-channel tensor mapping method. The original state tensors are arranged and combined in chronological order to obtain the sequence of original state tensors.

[0008] Optionally, the generation of the high-dimensional state embedding vector specifically includes: The original state tensor sequence is input into the cross-modal state encoding module. Convolution operations are performed on the original state tensor using multiple convolution kernel sizes to obtain multi-scale convolution results with corresponding kernel sizes. The multi-scale convolution results are then added to the corresponding biases and input into a nonlinear activation function to obtain multi-scale convolution features with corresponding kernel sizes. By concatenating the multi-scale convolutional features corresponding to the same original state tensor along the feature dimension, the fused convolutional features corresponding to the original state tensor are obtained. The fused convolutional features are flattened to obtain a sequence representation, and the sequence representation is linearly mapped to obtain an encoded input. Perform self-attention weighted encoding on the encoded input to obtain the attention output; The attention output is aggregated to obtain the high-dimensional state embedding vector corresponding to the original state tensor.

[0009] Optionally, the generation of the enhanced state representation vector specifically includes: The high-dimensional state embedding vector is input into the hierarchical cognitive memory module, which includes a short-term memory unit, a long-term semantic memory unit, and a context-associated memory unit. The short-term memory state of the previous time step is obtained from the short-term memory unit, the long-term semantic memory state of the previous time step is obtained from the long-term semantic memory unit, and the context-associated memory state of the previous time step is obtained from the context-associated memory unit. A three-channel gating update mechanism is used to generate short-term gating vectors, long-term gating vectors, and context-based gating vectors, respectively. The short-term memory state, long-term semantic memory state, and context-associated memory state are updated separately through a three-channel gating update mechanism. The high-dimensional state embedding vector, the updated short-term memory state, the updated long-term semantic memory state, and the updated context-related memory state are concatenated in a fixed order to form a fusion input vector. A fusion linear mapping is performed on the fusion input vector and a fusion bias is superimposed to obtain an enhanced state representation vector.

[0010] Optionally, obtaining the target execution strategy specifically includes: The enhanced state representation vector is input into the meta-policy generation module. The first path of the meta-policy generation module is used to perform forward computation of the first path policy network on the enhanced state representation vector, and the action score vector of the exploratory candidate policy is output. The second path of the meta-policy generation module is used to perform forward computation of the second path policy network on the enhanced state representation vector, and the action score vector of the exploitation candidate policy is output. Softmax normalization is performed on the action score vectors of the exploratory candidate strategy and the action probability vectors of the exploitation candidate strategy, respectively, to obtain the action probability vectors of the exploratory candidate strategy and the action probability vectors of the exploitation candidate strategy. A set of candidate policies is constructed based on the action probability vectors of exploratory candidate policies and the action probability vectors of exploitative candidate policies. The policy entropy and policy confidence corresponding to each action probability vector in the set of candidate policies are calculated respectively. The target execution strategy is determined from the set of candidate strategies through a strategy competition selection mechanism.

[0011] Optionally, determining the optimal action instruction specifically includes: The target execution strategy is input into the action value assessment module to establish an action embedding matrix and generate a strategy action embedding vector based on the target execution strategy. Construct a joint state-action input vector; Parallel forward propagation computation is performed on the joint state-action input vector through a dual value network structure, which includes a first value network and a second value network. The first action value vector is output through the first value network; The second action value vector is output through the second value network; Generate a strategy value vector based on the first action value vector and the second action value vector; The optimal action instruction is determined based on the strategy value vector.

[0012] Optionally, the construction of the state transition record set specifically includes: Send the optimal action instruction to the execution interface to drive the execution unit to perform the corresponding action; After the execution unit completes the action execution, the environmental perception module collects multi-source heterogeneous environmental data after the action execution, and extracts environmental state data after the action execution from the multi-source heterogeneous environmental data as the environmental state data for the next time step. The closed-loop feedback optimization module obtains environmental feedback information, which includes the reward value and the environmental state data of the next time step. The environmental state data of the next time step is sequentially input into the environmental perception module, the cross-modal state coding module, and the hierarchical cognitive memory module to generate the enhanced state representation vector of the next time step, and obtain the enhanced state representation vector of the current time step, the optimal action instruction of the current time step, and the reward value of the current time step. A set of state transition records is generated based on the enhanced state representation vector of the current time step, the optimal action instruction of the current time step, the reward value of the current time step, and the enhanced state representation vector of the next time step.

[0013] Optionally, the generation of the target decision parameter set specifically includes: The enhanced state representation vectors of each time step in the state transition record set are input into the meta-policy generation module to obtain the action probability vector of the target execution policy for the corresponding time step. Based on the action probability vector, a perturbation vector of the same dimension is constructed. The perturbation vector and the action probability vector are added element by element to obtain the intermediate perturbation probability vector. The intermediate perturbation probability vector is non-negatively truncated, and the negative components are set to zero. The non-negatively truncated intermediate perturbation probability vector is normalized to obtain the perturbation action probability vector. The enhanced state representation vector, the perturbation action probability vector, the optimal action instruction, the reward value, and the enhanced state representation vector of the next time step are combined to form a perturbation sample. The advantage estimate is calculated using an adaptive advantage estimation update mechanism; The meta-policy generation module and the action value assessment module are jointly updated to generate a target decision parameter set. The joint parameter update includes constructing a policy update target based on the state transition record set and perturbation samples. The policy update target includes a policy loss term and a perturbation consistency term. A value update target is constructed based on the state transition record set. Gradient updates are performed on the network parameters of the meta-policy generation module and the action value assessment module based on the policy update target and the value update target to obtain updated network parameters of the meta-policy generation module and the action value assessment module. The updated network parameters of the meta-policy generation module and the updated network parameters of the action value assessment module are then combined to form the target decision parameter set.

[0014] The beneficial effects of this invention are: This invention constructs an AI agent autonomous decision-making system comprising an environmental perception module, a cross-modal state encoding module, a hierarchical cognitive memory module, a meta-policy generation module, an action value assessment module, and a closed-loop feedback optimization module. This forms a complete closed-loop structure from environmental data collection to policy optimization and updating. Compared to the existing technologies where each functional module is relatively independent and the data flow is discontinuous, this invention achieves unified data structure and seamless processing flow between modules at the system architecture level. This enables multi-source heterogeneous environmental data to undergo standardization, serialization, and structuring before entering the decision core, thereby significantly improving the overall system's operational stability and scalability.

[0015] In terms of environmental data processing, this invention constructs a unified timeline and synchronizes timestamps for multi-source heterogeneous environmental data, and builds an original state tensor sequence. This enables data from different sampling frequencies and data formats to be structurally mapped on the same time scale, avoiding information loss and inconsistent state representation caused by time misalignment or simple splicing in existing technologies. By forming a unified-dimensional original state tensor sequence through multi-channel tensor mapping, a stable and repeatable input structure is provided for subsequent cross-modal state encoding, enhancing the consistency between model training and online inference.

[0016] In terms of state representation capabilities, this invention generates high-dimensional state embedding vectors through multi-scale convolutional feature extraction and self-attention weighted encoding. This enables the model to extract both local fine-grained features and model global dependencies, effectively overcoming the limitations of single-scale feature extraction or simple concatenation in existing technologies. Simultaneously, a hierarchical cognitive memory module performs memory fusion on the high-dimensional state embedding vectors. Utilizing a three-channel gating update mechanism—short-term memory, long-term semantic memory, and context-related memory—the current state can be controlled and fused with historical information at different time scales, thus forming an enhanced state representation vector. This structure avoids the information decay and noise accumulation problems inherent in single-memory structures, improving the sensitivity and continuity of state representation to changes in historical task context and situation.

[0017] In terms of strategy generation and evaluation, this invention constructs a target execution strategy through a meta-policy generation module and performs joint state-action evaluation of the target execution strategy using a dual-value network structure in the action value evaluation module. The minimum policy value output by the dual networks is used to generate a policy value vector, effectively reducing the overestimation problem caused by single-network estimation bias and improving the reliability of strategy evaluation. In the closed-loop feedback optimization stage, this invention provides structured storage of the decision-making process through a set of state transition records and performs joint parameter updates using an adversarial policy perturbation mechanism and an adaptive advantage estimation update mechanism. This makes the strategy robust to perturbation samples during optimization, while also improving the stability of gradient estimation and sample utilization efficiency.

[0018] In summary, this invention outperforms existing technologies in terms of unified modeling of multi-source heterogeneous environmental data, enhanced cross-modal state representation, stable fusion of hierarchical memory, and robustness of policy evaluation and parameter updates. It achieves more complete state representation, more stable policy generation, more accurate decision results, and greater adaptability to environmental disturbances, thereby significantly improving the continuous autonomous decision-making ability of AI agents in complex dynamic environments. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of an autonomous decision-making method for an AI intelligent agent system based on deep learning, as proposed in this invention. Figure 2 This is a schematic diagram illustrating the construction of an enhanced state representation vector for an autonomous decision-making method for an AI intelligent agent system based on deep learning, as proposed in this invention. Figure 3 This is a schematic diagram illustrating the construction of the optimal action instruction for an AI intelligent agent system autonomous decision-making method based on deep learning proposed in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figures 1-3 A method for autonomous decision-making in an AI intelligent agent system based on deep learning includes the following steps: Construct an AI agent autonomous decision-making system, which includes an environmental perception module, a cross-modal state encoding module, a hierarchical cognitive memory module, a meta-policy generation module, an action value assessment module, and a closed-loop feedback optimization module. Multi-source heterogeneous environmental data is collected by the environmental perception module and synchronized and aligned according to timestamps to construct the original state tensor sequence; The original state tensor sequence is processed by multi-scale convolutional feature extraction and self-attention weighted encoding through a cross-modal state encoding module to generate a high-dimensional state embedding vector. The high-dimensional state embedding vector is input into the hierarchical cognitive memory module to generate an enhanced state representation vector; The enhanced state representation vector is input into the meta-policy generation module to obtain the target execution policy; The target execution strategy is input into the action value assessment module. The action value assessment module performs a joint state-action assessment of the target execution strategy based on a dual value network structure, outputs the corresponding strategy value vector, and determines the optimal action instruction based on the strategy value ranking result. Execute the optimal action command and obtain environmental feedback information through the closed-loop feedback optimization module to construct a set of state transition records; Based on the state transition record set, perturbation samples are generated through an adversarial strategy perturbation mechanism, and joint parameter updates are performed on the meta-policy generation module and the action value assessment module to generate the target decision parameter set.

[0022] In this embodiment, the functional modules and interface specifications of the AI ​​agent system are defined on a distributed computing platform. An environmental perception module is deployed to access and collect multi-source heterogeneous environmental data. A cross-modal state encoding module is deployed to perform unified tensor quantization representation and feature encoding on synchronized multi-source data. A hierarchical cognitive memory module is deployed to perform short-term memory writing, long-term semantic memory retrieval, and context-related fusion on state embedding vectors. A meta-policy generation module is deployed to generate candidate policies and output target execution policies. An action value assessment module is deployed to perform state-action value estimation and output the optimal action instruction. A closed-loop feedback optimization module is deployed to record state transition data and update the execution parameters of the policy network and value network, and establish data flow paths and parameter update paths between modules.

[0023] In this embodiment, the construction of the original state tensor sequence specifically includes: The environmental perception module collects multi-source heterogeneous environmental data, including sensor observation data, system operation data, interaction and task data, and historical decision data. The data acquisition process is as follows: within a preset sampling period, the environmental perception module establishes data connections with the sensor interface, system interface, and interaction interface respectively. It reads sensor observation data from the sensor interface, system operation data from the system interface, interaction and task data from the interaction interface, and historical decision data from the local storage unit or cache queue. It encapsulates all types of data in a structured manner using a unified data frame encapsulation format, writes a data source identifier, data type identifier, and timestamp field for each piece of data, and writes the encapsulated data into the data acquisition buffer to form multi-source heterogeneous environmental data. Determine the start time, end time, and preset sampling interval of the decision cycle, and construct a unified timeline based on the start time, end time, and preset sampling interval; The process for determining the start and end times of the decision cycle is as follows: After the environmental perception module completes the collection of multi-source heterogeneous environmental data, it reads the timestamp field in the encapsulated data frames of each data source, calculates the minimum value of the timestamps of all data frames within the decision cycle as the start time of the decision cycle, and calculates the maximum value of the timestamps of all data frames within the decision cycle as the end time of the decision cycle; when there is a decision trigger signal output by the system controller, the trigger timestamp corresponding to the decision trigger signal is used as the start time of the decision cycle, and the start time is added to the preset decision window length to obtain the end time of the decision cycle; The construction process is as follows: taking the start time as the starting point of the first time step of the unified time axis, the subsequent time steps are generated by increasing at equal intervals according to the preset sampling interval, and in the process of generation, all time steps less than the termination time are arranged in chronological order to form a unified time axis. Multi-source heterogeneous environmental data are synchronized and aligned according to a unified timeline to generate an aligned observation vector; The generation process is as follows: For each time step in the unified timeline, all data with timestamps less than or equal to the time step are filtered from the multi-source heterogeneous environment data according to the data source, and the data with the largest timestamp in the filtering results is selected as the valid data corresponding to the time step; the data content in the valid data is converted into a fixed-dimensional feature vector corresponding to the data source. The conversion process involves establishing a feature field set and field order consistent with the data source for each data source, extracting and quantifying the data content in the valid data according to the feature field set, converting discrete fields in the feature field set into numerical fields using an index mapping method, and directly taking the numerical values ​​of continuous fields in the feature field set and normalizing the values ​​according to the field units; when any feature field is missing in the valid data, the corresponding field position is filled with a zero value; the values ​​of each field are arranged in the field order and concatenated to obtain the fixed-dimensional feature vector corresponding to the data source. For each time step in the unified time axis, the aligned observation vectors of each data source within the same time step are concatenated in a fixed order to obtain the unified feature vector of the time step. Set the number of channels and the number of feature slots per channel in the multi-channel tensor mapping. The product of the number of channels and the number of feature slots per channel is greater than the dimension of the unified feature vector. For each time step in the unified time axis, the unified feature vector of the time step is constructed into the original state tensor of the time step through the multi-channel tensor mapping method. The original state tensor is a two-dimensional tensor containing multiple channels and each channel contains multiple feature slots. The construction process is as follows: The number of channels and the number of feature slots per channel of the two-dimensional tensor are determined based on the dimension of the unified feature vector. The product of the number of channels and the number of feature slots per channel is greater than the dimension of the unified feature vector. For each time step in the unified time axis, a two-dimensional tensor with a size of the number of channels × the number of feature slots per channel is created as the original state tensor of the time step, and all feature slots of the original state tensor are initialized to zero. The unified feature vector of the time step is written into the feature slots of the original state tensor in ascending order of component number. The writing order is to write from the first slot to the last slot according to the feature slot of the current channel. After the current channel is full, the writing continues to the next channel until all components of the unified feature vector are written, thus obtaining the original state tensor corresponding to the time step. Arrange and combine the original state tensors in chronological order to obtain the sequence of original state tensors; The specific implementation method of the combination is as follows: for each time step in the unified time axis, the original state tensor corresponding to the time step is associated with the timestamp of the time step, and all the original state tensors are sorted in ascending order of timestamp. The sorted original state tensors are written into the same storage structure to form an ordered set, thus obtaining the original state tensor sequence.

[0024] In this embodiment, the generation of the high-dimensional state embedding vector specifically includes: The original state tensor sequence is input into the cross-modal state encoding module. Convolution operations are performed on the original state tensor using multiple convolution kernel sizes to obtain multi-scale convolution results with corresponding kernel sizes. The multi-scale convolution results are then added to the corresponding biases and input into the nonlinear activation function to obtain multi-scale convolution features with corresponding kernel sizes. By concatenating the multi-scale convolutional features corresponding to the same original state tensor along the feature dimension, the fused convolutional features corresponding to the original state tensor are obtained. The fused convolutional features are flattened to obtain a sequence representation, and the sequence representation is linearly mapped to obtain the encoded input. The specific implementation of the flattening process is as follows: For the fused convolutional features corresponding to each original state tensor, read the arrangement structure in the spatial dimension and feature dimension, extract the feature vectors of each position in a fixed traversal order, and connect the feature vectors of each position end by end in the extraction order to form a one-dimensional vector sequence; divide the one-dimensional vector sequence into multiple position vectors according to position, and each position vector corresponds to a position feature in the fused convolutional features, to obtain a sequence representation composed of multiple position vectors in order; The specific implementation of the linear mapping process is as follows: After obtaining the sequence representation with fused convolutional features, construct the weight matrix and bias vector of the linear mapping. The number of rows in the weight matrix is ​​equal to the feature dimension of the encoding input, and the number of columns is equal to the feature dimension of a single position vector in the sequence representation. Perform matrix multiplication on each position vector in the sequence representation, and add the matrix multiplication result to the bias vector to obtain the mapped position vector corresponding to that position vector. Arrange all the mapped position vectors in the original order in the sequence representation to obtain the encoding input composed of the mapped position vectors. Perform self-attention weighted encoding on the encoded input to obtain the attention output; The self-attention weighted encoding process includes: performing query mapping, key mapping, and value mapping on the encoding input to obtain query representation, key representation, and value representation respectively; performing matrix multiplication on the transpose of the query representation and the key representation to obtain the relevance matrix, and performing scaling on the relevance matrix; performing row-normalized exponential operation on the scaled relevance matrix to obtain the attention weight matrix; and performing matrix multiplication on the attention weight matrix and the value representation to obtain the attention output. The attention output is aggregated to obtain the high-dimensional state embedding vector corresponding to the original state tensor. The aggregation process includes: summing the position vectors in the attention output and dividing by the number of position vectors to obtain the average feature vector, then performing an embedding linear mapping on the average feature vector and superimposing the embedding bias to obtain a high-dimensional state embedding vector.

[0025] In this embodiment, the generation of the enhanced state representation vector specifically includes: The high-dimensional state embedding vector is input into the hierarchical cognitive memory module, which includes short-term memory units, long-term semantic memory units, and context-associated memory units. The short-term memory state of the previous time step is obtained from the short-term memory unit, the long-term semantic memory state of the previous time step is obtained from the long-term semantic memory unit, and the context-associated memory state of the previous time step is obtained from the context-associated memory unit. The acquisition process involves setting up memory storage areas in the hierarchical cognitive memory module that correspond one-to-one with time steps on a unified time axis for short-term memory units, long-term semantic memory units, and context-associated memory units. At the end of each time step, the updated short-term memory state, long-term semantic memory state, and context-associated memory state are written into the current time step index position of the corresponding memory storage area. When entering the current time step for memory fusion, the hierarchical cognitive memory module reads the content stored at the index position immediately preceding the current time step index in the memory storage area according to the current time step index, and uses it as the short-term memory state, long-term semantic memory state, and context-associated memory state of the previous time step, respectively. A three-channel gating update mechanism is used to generate short-term gating vectors, long-term gating vectors, and context-based gating vectors, respectively. The short-term gating vector is obtained by performing gating linear mappings on the high-dimensional state embedding vector and the short-term memory state of the previous time step, summing the results, and then superimposing the short-term gating bias to obtain the short-term gating intermediate vector. Then, a sigmoid operation is performed on each element of the short-term gating intermediate vector. The long-term gating vector is obtained by performing gating linear mappings on the high-dimensional state embedding vector and the long-term semantic memory state of the previous time step, summing the results, and then superimposing the long-term gating bias to obtain the long-term gating intermediate vector. Then, a sigmoid operation is performed on each element of the long-term gating intermediate vector. The context gating vector is obtained by performing gating linear mappings on the high-dimensional state embedding vector and the context-associated memory state of the previous time step, summing the results, and then superimposing the context gating bias to obtain the context gating intermediate vector. Then, a sigmoid operation is performed on each element of the context gating intermediate vector. The short-term memory state, long-term semantic memory state, and context-associated memory state are updated separately through a three-channel gating update mechanism. The update of the short-term memory state includes: multiplying the short-term gating vector element-wise with the high-dimensional state embedding vector to obtain the first update term; multiplying the element-wise complement of the short-term gating vector element-wise with the short-term memory state of the previous time step to obtain the second update term; and adding the first update term and the second update term element-wise to obtain the short-term memory state of the current time step. The update of the long-term semantic memory state includes: multiplying the long-term gate vector with the high-dimensional state embedding vector element by element to obtain the first update term; multiplying the element-wise complement of the long-term gate vector with the long-term semantic memory state of the previous time step element by element to obtain the second update term; and adding the first update term and the second update term element by element to obtain the long-term semantic memory state of the current time step. The update of the context-associated memory state includes: multiplying the context-gated vector with the high-dimensional state embedding vector element by element to obtain the first update term; multiplying the element-by-element complement of the context-gated vector with the context-associated memory state of the previous time step element by element to obtain the second update term; and adding the first update term and the second update term element by element to obtain the context-associated memory state of the current time step. The high-dimensional state embedding vector, the updated short-term memory state, the updated long-term semantic memory state, and the updated context-related memory state are concatenated in a fixed order to form a fusion input vector. A fusion linear mapping is performed on the fusion input vector and a fusion bias is superimposed to obtain an enhanced state representation vector.

[0026] In this embodiment, obtaining the target execution strategy specifically includes: The enhanced state representation vector is input into the meta-policy generation module. The first path of the meta-policy generation module is used to perform forward computation of the first path policy network on the enhanced state representation vector, and the action score vector of the exploratory candidate policy is output. The forward computation process is as follows: a linear transformation is performed on the enhanced state representation vector and the layer weights of the first path policy network, and the corresponding layer biases are superimposed. The result of the linear transformation is then input into the activation function to obtain the output of that layer, and the output of that layer is used as the input of the next layer until the final output layer. The number of output nodes of the final output layer is consistent with the number of actions in the action space. Each output component generated by the output layer corresponds to the action score of each action in the action space. The action score vector of the exploratory candidate policy is composed of each output component. The second path of the meta-policy generation module is used to perform forward computation of the second path policy network on the enhanced state representation vector, and the action score vector of the exploitation candidate policy is output. The forward computation process is as follows: a linear transformation is performed on the enhanced state representation vector and the layer weights of the second path policy network, and the corresponding layer biases are superimposed. The result of the linear transformation is then input into the activation function to obtain the output of that layer, and the output of that layer is used as the input of the next layer until the final output layer. The number of output nodes of the final output layer is consistent with the number of actions in the action space. Each output component generated by the final output layer corresponds to the action score of each action in the action space. The action score vector of the utilization candidate policy is composed of each output component. Softmax normalization is performed on the action score vectors of the exploratory candidate strategy and the action probability vectors of the exploitation candidate strategy, respectively, to obtain the action probability vectors of the exploratory candidate strategy and the action probability vectors of the exploitation candidate strategy. The Softmax normalization process includes: performing an exponential operation on the action score of each dimension in the action score vector, and dividing the exponential value of the action score by the sum of the exponential values ​​of all action scores in the action score vector to obtain the action probability of the corresponding action in that dimension. A set of candidate policies is constructed based on the action probability vectors of exploratory candidate policies and the action probability vectors of exploitative candidate policies. The policy entropy and policy confidence corresponding to each action probability vector in the set of candidate policies are calculated respectively. The calculation of policy entropy includes: taking the natural logarithm of the action probability in each dimension of the action probability vector and multiplying it by the action probability; summing the product of all dimensions and taking the negative value; the calculation of policy confidence includes: taking the maximum action probability in the action probability vector as the policy confidence. The target execution strategy is determined from the set of candidate strategies through a strategy competition selection mechanism; The strategy competition selection mechanism includes: calculating the competition score for each action probability vector in the candidate strategy set; the competition score is obtained by weighted summation of the normalized result of the strategy entropy and the strategy confidence; the normalized result of the strategy entropy is the strategy entropy divided by the natural logarithm of the number of actions in the action space; and selecting the action probability vector with the largest competition score as the target execution strategy.

[0027] In this embodiment, determining the optimal action instruction specifically includes: The target execution strategy is input into the action value assessment module to establish an action embedding matrix. Each row of the action embedding matrix corresponds to the action embedding vector of an action in the action space. The strategy action embedding vector is generated according to the target execution strategy. The generation of the strategy action embedding vector includes: for each action in the action space, multiplying the action probability of the action with the action embedding vector corresponding to the action and summing the product results of all actions to obtain the strategy action embedding vector. Constructing the joint state-action input vector involves concatenating the enhanced state representation vector and the policy action embedding vector in sequence to obtain the joint state-action input vector. Parallel forward propagation computation is performed on the joint state-action input vector through a dual value network structure, which includes a first value network and a second value network. The first action value vector is output through the first value network. The first value network is a feedforward network composed of multiple fully connected networks. The first value network includes an input layer, two hidden layers and an output layer. The input layer receives the state-action joint input vector. The hidden layer performs linear transformation, superposition bias and activation function operations in sequence. The output layer outputs the first action value vector. Each dimension of the first action value vector corresponds to the first action value of each action in the action space. The second action value vector is output through the second value network. The second value network is a feedforward network composed of multiple fully connected networks. The second value network includes an input layer, two hidden layers and an output layer. The input layer receives the state-action joint input vector. The hidden layer performs linear transformation, superposition bias and activation function operations in sequence. The output layer outputs the second action value vector. Each dimension of the second action value vector corresponds to the second action value of each action in the action space. Generate a strategy value vector based on the first action value vector and the second action value vector; The generation of the strategy value vector includes: for each action in the action space, obtaining the first action value in the first action value vector and the second action value in the second action value vector, and taking the minimum value between the first action value and the second action value as the strategy value of the action, and arranging the strategy values ​​of each action in the order of the actions to form a strategy value vector. The optimal action instruction is determined based on the strategy value vector. The determination of the optimal action instruction includes selecting the action corresponding to the strategy value with the largest value in the strategy value vector as the optimal action instruction.

[0028] In this embodiment, the construction of the state transition record set specifically includes: Send the optimal action instruction to the execution interface to drive the execution unit to perform the corresponding action; After the execution unit completes the action execution, the environmental perception module collects multi-source heterogeneous environmental data after the action execution, and extracts environmental state data after the action execution from the multi-source heterogeneous environmental data as the environmental state data for the next time step. The closed-loop feedback optimization module obtains environmental feedback information, including the reward value and the environmental state data of the next time step. The reward value is obtained by the output of the reward function, and the input of the reward function includes the environmental state data of the current time step, the optimal action instruction, and the environmental state data of the next time step. The environmental state data of the next time step is sequentially input into the environmental perception module, the cross-modal state coding module, and the hierarchical cognitive memory module to generate the enhanced state representation vector of the next time step, and obtain the enhanced state representation vector of the current time step, the optimal action instruction of the current time step, and the reward value of the current time step. A set of state transition records is generated based on the enhanced state representation vector of the current time step, the optimal action instruction of the current time step, the reward value of the current time step, and the enhanced state representation vector of the next time step. The generation process includes: combining the enhanced state representation vector, optimal action instruction, reward value, and the enhanced state representation vector of the next time step into a state transition record according to a fixed field order; and arranging and combining all state transition records generated in each time step within the same decision cycle in chronological order to obtain a set of state transition records.

[0029] In this embodiment, the generation of the target decision parameter set specifically includes: The enhanced state representation vectors of each time step in the state transition record set are input into the meta-policy generation module to obtain the action probability vector of the target execution policy for the corresponding time step. Based on the action probability vector, a perturbation vector of the same dimension is constructed. The perturbation vector and the action probability vector are added element by element to obtain the intermediate perturbation probability vector. The intermediate perturbation probability vector is non-negatively truncated, and the negative components are set to zero. The intermediate perturbation probability vector after non-negative truncation is normalized to obtain the perturbation action probability vector. The enhanced state representation vector, the perturbation action probability vector, the optimal action instruction, the reward value, and the enhanced state representation vector of the next time step are combined to form the perturbation sample. The advantage estimate is calculated using an adaptive advantage estimation update mechanism; The calculation includes: inputting the enhanced state representation vector and the target execution strategy into the action value assessment module to obtain the strategy value vector; calculating the state value based on the action probability vector and the strategy value vector, where the state value is the sum of the product of the action probability vector and the strategy value vector according to the corresponding action dimensions; setting discount factors and decay factors; calculating the temporal difference residual for each time step in the state transition record set, where the temporal difference residual is obtained by combining the reward value of the current time step, the state value of the next time step, and the state value of the current time step according to the discount factor; calculating the advantage estimate for any time step, where the advantage estimate is obtained by weighted summation of the temporal difference residuals of the time step and its multiple time steps in chronological order, and the weighting coefficient is determined by the power of the product of the discount factor and the decay factor. The meta-policy generation module and the action value assessment module are jointly updated to generate a target decision parameter set. The joint parameter update includes constructing a policy update target based on the state transition record set and perturbation samples. The policy update target includes a policy loss term and a perturbation consistency term. The policy loss term is obtained by averaging the product of the logarithm of the action probability corresponding to the optimal action instruction and the corresponding advantage estimate over all time steps. The perturbation consistency term is obtained by averaging the relative entropy between the perturbation action probability vector and the action probability vector over all time steps and multiplying it by a non-negative coefficient. A value update target is constructed based on the state transition record set. The value update target is obtained by averaging the squared difference between the discounted reward and the state value at the current time step over all time steps. The discounted reward is obtained by weighting the reward values ​​of multiple subsequent time steps from the current time step according to the power of the discount factor. Based on the policy update target and the value update target, gradient updates are performed on the network parameters of the meta-policy generation module and the action value assessment module to obtain the updated network parameters of the meta-policy generation module and the action value assessment module. The updated network parameters of the meta-policy generation module and the updated network parameters of the action value assessment module are then combined to form the target decision parameter set. The update process is as follows: The updated state transition record set and perturbation samples are grouped into training batches according to time steps. These training batches are then input into the meta-policy generation module and the action value evaluation module for forward propagation calculations to obtain the output values ​​corresponding to the policy update target and the value update target. The gradients of the policy update target and the value update target with respect to the network parameters of the meta-policy generation module and the action value evaluation module are calculated respectively, and the gradients are pruned to ensure the gradient norm is less than a set threshold. An adaptive moment estimation method is used to iteratively update the network parameters. This involves calculating the first-order moment estimate and the second-order moment estimate for each network parameter, performing bias correction, and updating the network parameters according to the learning rate.

[0030] Example 1: This example uses a large-scale warehouse center as an application scenario for multi-robot collaborative picking and replenishment. The warehouse center contains dense shelving, aisles, and intersections, with dynamic obstacles including other robots, manual forklifts, and temporarily stacked pallets. Each robot, after receiving a picking task, needs to travel from its starting position to the target location, confirm its arrival, and then proceed to the outbound exit or transfer area. In this scenario, environmental information sources are multi-source and heterogeneous: the robot's forward-facing camera and depth camera provide sensor observation data; odometers and motor current provide system operation data; target location, priority, and aisle occupancy information issued by the task management system constitute interaction and task data; and the robot's optimal action command and reward value from the previous round of decision-making constitute historical decision data. Common shortcomings in existing technologies are amplified in this scenario: inconsistent timestamps from different data sources lead to misaligned state representations; simple concatenation makes the state representation insensitive to information such as key intersection conflicts and channel congestion; without a hierarchical cognitive memory module, the robot struggles to maintain consistent references to "recently occurred avoidance events" and "current task constraints" at consecutive intersections, easily leading to strategy jitter; when using a single network for value assessment, overestimation of action value is prone to occur in congested scenarios, causing the robot to choose aggressive actions, resulting in increased emergency stops and detours, thus affecting overall efficiency and safety. The key technical problem to be addressed in this embodiment is: in a warehouse environment where multi-source heterogeneous data is constantly changing and time-dependent, how to form a consistent, stable, and decision-making-usable state representation, and on this basis, improve the stability of strategy generation and value assessment, so that the optimal action command can maintain a high success rate and a low risk event incidence rate under congestion and dynamic obstacle conditions.

[0031] When applying this invention in this warehousing scenario, an AI-powered autonomous decision-making system is deployed on the robot, maintaining continuous data flow across the environmental perception module, cross-modal state encoding module, hierarchical cognitive memory module, meta-policy generation module, action value assessment module, and closed-loop feedback optimization module. The environmental perception module reads sensor observation data from cameras, depth cameras, and ranging devices, as well as system operation data from the robot controller, interaction and task data from the task management system, and historical decision data from local cache. All collected data retains its own timestamp field, and timestamp synchronization and alignment are achieved through a unified timeline. Multi-source data within the same time step are converted into aligned observation vectors, and then the original state tensor sequence is constructed through multi-channel tensor mapping. The cross-modal state encoding module performs multi-scale convolutional feature extraction and self-attention weighted encoding processing on the original state tensor sequence, outputting a high-dimensional state embedding vector. This allows local information such as shelf edges, aisle boundaries, and relative obstacle positions, as well as global information such as task priorities and congestion trends, to be captured in the same representation space. The hierarchical cognitive memory module receives a high-dimensional state embedding vector and retrieves the corresponding short-term memory state, long-term semantic memory state, and context-associated memory state from the short-term memory unit, long-term semantic memory unit, and context-associated memory unit from the previous time step. After memory fusion through a three-channel gating update mechanism, it outputs an enhanced state representation vector, enabling the robot to maintain a stable reference to information such as "the cost of the most recent avoidance," "the remaining distance of the current task," and "changes in channel occupancy" when traversing multiple intersections. The meta-policy generation module takes the enhanced state representation vector as input and outputs a target execution policy through a dual-path policy network structure. The first path tends to generate exploratory candidate policies that cover more actions, while the second path tends to generate exploitative candidate policies that focus on high-reward actions. The target execution policy is then obtained through a policy competition selection mechanism. The action value assessment module receives the target execution policy and performs a joint state-action assessment. The dual value network structure outputs action value vectors and generates a policy value vector, thereby determining the optimal action instruction. After executing the optimal action command, the closed-loop feedback optimization module obtains environmental feedback information and generates a set of state transition records. These records are used to subsequently update the joint parameters through an adversarial strategy perturbation mechanism and an adaptive advantage estimation update mechanism, forming a target decision parameter set. This allows the system to maintain decision stability when faced with disturbances such as sudden forklift entry, temporary road closures, and dense robot encounters.

[0032] To provide verifiable data demonstrating the beneficial effects, this embodiment conducted a comparative test in the same warehouse area. The test area included two main roads, four intersections, and six shelving aisles. Twelve mobile robots were deployed and operated continuously for seven days, completing a total of 18,000 picking and replenishment tasks. The comparison method used a "baseline system," which also uses a deep network to output action probabilities, but does not construct the original state tensor sequence; it only directly concatenates the features from each data source. Its state encoding uses a single-scale convolutional or single-layer encoding structure, lacking self-attention weighted encoding processing. Its memory structure only retains a short-term cache and does not include a hierarchical structure of short-term memory units, long-term semantic memory units, and context-related memory units. Its value assessment uses a single value network to output action value vectors. Both systems operated under the same task allocation strategy, traffic rules, and safe speed limits, recording metrics such as task completion rate, average task completion time, number of emergency stops per unit task, number of collision alarms per unit task, average path deviation rate, and online inference time. The number of emergency stops was based on logs of emergency braking triggered by the execution unit; the number of collision alarms was based on logs of nearest-distance alarms triggered by safety sensors; and the inference time was based on the time interval from receiving the aligned observation vector to outputting the optimal action command, as recorded by the robot. Test results show that the method of this invention maintains a high completion rate and significantly reduces emergency stops and collision alarms even during congested periods (10:00-12:00 and 14:00-16:00 daily). The reason is that the timestamp synchronization alignment and the original state tensor sequence enable multi-source data to be consistently expressed at the same time step. The cross-modal state coding module strengthens the joint modeling of key local risks and global congestion trends. The hierarchical cognitive memory module improves the stable reference to the continuous context. The dual value network structure reduces the probability of aggressive action selection caused by overestimation of a single network, thus forming a more stable balance between efficiency and safety.

[0033] Table 1 Comparison of Autonomous Decision-Making Performance in Warehouse Picking Scenarios

[0034] As can be seen from Table 1, which shows the task completion rate and average task completion time, the method of this invention achieves a more stable task closed loop under the same task scale and operational constraints. The task completion rate increased from 96.8% to 99.2%, corresponding to a decrease in the number of failed tasks out of 18,000 tasks from approximately 576 to approximately 144. The main reasons for failure were "timeout cancellation due to channel congestion" and "inability to restore the target direction after multiple avoidances." The method of this invention, due to its enhanced state representation vector, provides a more stable expression of congestion trends and task constraints, enabling rapid restoration of the main target direction even after multiple avoidances, thus significantly reducing timeout cancellations. The decrease in average task completion time also reflects that the optimal action command involves fewer invalid trials and repeated corrections when switching between intersections and channels.

[0035] Safety-related indicators show that the method of this invention effectively reduces the triggering frequency of risk events in warehouse environments with dense dynamic obstacles. The number of emergency stops decreased from 31.6 per thousand orders to 18.2, and the number of collision alarms decreased from 6.3 per thousand orders to 3.1, indicating that the system tends to choose controllable action sequences and reduce sudden braking and proximity alarms in congested and oncoming traffic situations. Combined with the decrease in the average path deviation rate, it can be inferred that the system can make more reasonable detour and return decisions when encountering temporary obstacles, rather than getting stuck in localized areas and causing large deviations due to repeated avoidance.

[0036] From an engineering usability perspective, the online inference time increased from 43 milliseconds to 51 milliseconds, increasing computational overhead from the cross-modal state encoding module and the hierarchical cognitive memory module. However, this time is still within the allowable range of the mobile robot's control cycle, and the benefits in task completion rate, efficiency, and safety significantly outweigh the impact of the additional computational overhead. Under the actual operational constraints of the warehouse center, this "acceptable increase in inference time" in exchange for "higher completion rate and lower risk events" makes the system more suitable for long-term continuous operation and intensive multi-robot collaboration scenarios.

[0037] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for autonomous decision-making in an AI intelligent agent system based on deep learning, characterized in that, Includes the following steps: Construct an AI agent autonomous decision-making system, which includes an environment perception module, a cross-modal state encoding module, a hierarchical cognitive memory module, a meta-policy generation module, an action value assessment module, and a closed-loop feedback optimization module. Multi-source heterogeneous environmental data is collected by the environmental perception module and synchronized according to timestamps to construct the original state tensor sequence; The original state tensor sequence is processed by multi-scale convolutional feature extraction and self-attention weighted encoding through a cross-modal state encoding module to generate a high-dimensional state embedding vector. The high-dimensional state embedding vector is input into the hierarchical cognitive memory module to generate an enhanced state representation vector; The enhanced state representation vector is input into the meta-policy generation module to obtain the target execution policy; The target execution strategy is input into the action value assessment module. The action value assessment module performs a state-action joint assessment of the target execution strategy based on a dual value network structure, outputs the corresponding strategy value vector, and determines the optimal action instruction based on the strategy value ranking result. Execute the optimal action command and obtain environmental feedback information through the closed-loop feedback optimization module to construct a state transition record set; Based on the state transition record set, perturbation samples are generated through an adversarial strategy perturbation mechanism. Joint parameter updates are performed on the meta-policy generation module and the action value assessment module to generate the target decision parameter set.

2. The autonomous decision-making method for an AI intelligent agent system based on deep learning according to claim 1, characterized in that, On a distributed computing platform, the functional modules and interface specifications of the AI ​​agent system are defined. An environmental perception module is deployed to access and collect multi-source heterogeneous environmental data. A cross-modal state encoding module is deployed to perform unified tensor quantization representation and feature encoding on synchronized multi-source data. A hierarchical cognitive memory module is deployed to perform short-term memory writing, long-term semantic memory retrieval, and context-related fusion on state embedding vectors. A meta-policy generation module is deployed to generate candidate policies and output target execution policies. An action value assessment module is deployed to perform state-action value estimation and output the optimal action instruction. A closed-loop feedback optimization module is deployed to record state transition data and update the execution parameters of the policy network and value network. Data flow paths and parameter update paths between modules are also established.

3. The autonomous decision-making method for an AI intelligent agent system based on deep learning according to claim 1, characterized in that, The construction of the original state tensor sequence specifically includes: Multi-source heterogeneous environmental data is collected through the environmental perception module. The multi-source heterogeneous environmental data includes sensor observation data, system operation data, interaction and task data, and historical decision data. The start time, end time, and preset sampling interval of the decision cycle are determined, and a unified time axis is constructed based on the start time, the end time, and the preset sampling interval. Multi-source heterogeneous environmental data are synchronized and aligned according to a unified timeline to generate aligned observation vectors; For each time step in the unified time axis, the aligned observation vectors of each data source within the same time step are concatenated in a fixed order to obtain the unified feature vector of the time step. Set the number of channels and the number of feature slots per channel for the multi-channel tensor mapping. For each time step in the unified time axis, construct the original state tensor of the time step from the unified feature vector of the time step using the multi-channel tensor mapping method. The original state tensors are arranged and combined in chronological order to obtain the sequence of original state tensors.

4. The autonomous decision-making method for an AI intelligent agent system based on deep learning according to claim 1, characterized in that, The generation of the high-dimensional state embedding vector specifically includes: The original state tensor sequence is input into the cross-modal state encoding module. Convolution operations are performed on the original state tensor using multiple convolution kernel sizes to obtain multi-scale convolution results with corresponding kernel sizes. The multi-scale convolution results are then added to the corresponding biases and input into a nonlinear activation function to obtain multi-scale convolution features with corresponding kernel sizes. By concatenating the multi-scale convolutional features corresponding to the same original state tensor along the feature dimension, the fused convolutional features corresponding to the original state tensor are obtained. The fused convolutional features are flattened to obtain a sequence representation, and the sequence representation is linearly mapped to obtain an encoded input. Perform self-attention weighted encoding on the encoded input to obtain the attention output; The attention output is aggregated to obtain the high-dimensional state embedding vector corresponding to the original state tensor.

5. The autonomous decision-making method for an AI intelligent agent system based on deep learning according to claim 1, characterized in that, The generation of the enhanced state representation vector specifically includes: The high-dimensional state embedding vector is input into the hierarchical cognitive memory module, which includes a short-term memory unit, a long-term semantic memory unit, and a context-associated memory unit. The short-term memory state of the previous time step is obtained from the short-term memory unit, the long-term semantic memory state of the previous time step is obtained from the long-term semantic memory unit, and the context-associated memory state of the previous time step is obtained from the context-associated memory unit. A three-channel gating update mechanism is used to generate short-term gating vectors, long-term gating vectors, and context-based gating vectors, respectively. The short-term memory state, long-term semantic memory state, and context-associated memory state are updated separately through a three-channel gating update mechanism. The high-dimensional state embedding vector, the updated short-term memory state, the updated long-term semantic memory state, and the updated context-related memory state are concatenated in a fixed order to form a fusion input vector. A fusion linear mapping is performed on the fusion input vector and a fusion bias is superimposed to obtain an enhanced state representation vector.

6. The autonomous decision-making method for an AI intelligent agent system based on deep learning according to claim 1, characterized in that, The specific steps to obtain the target execution strategy include: The enhanced state representation vector is input into the meta-policy generation module. The first path of the meta-policy generation module is used to perform forward computation of the first path policy network on the enhanced state representation vector, and the action score vector of the exploratory candidate policy is output. The second path of the meta-policy generation module is used to perform forward computation of the second path policy network on the enhanced state representation vector, and the action score vector of the exploitation candidate policy is output. Softmax normalization is performed on the action score vectors of the exploratory candidate strategy and the action probability vectors of the exploitation candidate strategy, respectively, to obtain the action probability vectors of the exploratory candidate strategy and the action probability vectors of the exploitation candidate strategy. A set of candidate policies is constructed based on the action probability vectors of exploratory candidate policies and the action probability vectors of exploitative candidate policies. The policy entropy and policy confidence corresponding to each action probability vector in the set of candidate policies are calculated respectively. The target execution strategy is determined from the set of candidate strategies through a strategy competition selection mechanism.

7. The autonomous decision-making method for an AI intelligent agent system based on deep learning according to claim 1, characterized in that, The determination of the optimal action instruction specifically includes: The target execution strategy is input into the action value assessment module to establish an action embedding matrix and generate a strategy action embedding vector based on the target execution strategy. Construct a joint state-action input vector; Parallel forward propagation computation is performed on the joint state-action input vector through a dual value network structure, which includes a first value network and a second value network. The first action value vector is output through the first value network; The second action value vector is output through the second value network; Generate a strategy value vector based on the first action value vector and the second action value vector; The optimal action instruction is determined based on the strategy value vector.

8. The autonomous decision-making method for an AI intelligent agent system based on deep learning according to claim 1, characterized in that, The construction of the state transition record set specifically includes: Send the optimal action instruction to the execution interface to drive the execution unit to perform the corresponding action; After the execution unit completes the action execution, the environmental perception module collects multi-source heterogeneous environmental data after the action execution, and extracts environmental state data after the action execution from the multi-source heterogeneous environmental data as the environmental state data for the next time step. Environmental feedback information is obtained through a closed-loop feedback optimization module. The environmental feedback information includes the reward value and the environmental state data of the next time step. The environmental state data of the next time step is sequentially input into the environmental perception module, the cross-modal state coding module, and the hierarchical cognitive memory module to generate the enhanced state representation vector of the next time step, and to obtain the enhanced state representation vector of the current time step, the optimal action instruction of the current time step, and the reward value of the current time step. A set of state transition records is generated based on the enhanced state representation vector of the current time step, the optimal action instruction of the current time step, the reward value of the current time step, and the enhanced state representation vector of the next time step.

9. The autonomous decision-making method for an AI intelligent agent system based on deep learning according to claim 1, characterized in that, The generation of the target decision parameter set specifically includes: The enhanced state representation vectors of each time step in the state transition record set are input into the meta-policy generation module to obtain the action probability vector of the target execution policy for the corresponding time step. Based on the action probability vector, a perturbation vector of the same dimension is constructed. The perturbation vector and the action probability vector are added element by element to obtain the intermediate perturbation probability vector. The intermediate perturbation probability vector is non-negatively truncated, and the negative components are set to zero. The non-negatively truncated intermediate perturbation probability vector is normalized to obtain the perturbation action probability vector. The enhanced state representation vector, the perturbation action probability vector, the optimal action instruction, the reward value, and the enhanced state representation vector of the next time step are combined to form a perturbation sample. The advantage estimate is calculated using an adaptive advantage estimation update mechanism; The meta-policy generation module and the action value assessment module are jointly updated to generate a target decision parameter set. The joint parameter update includes constructing a policy update target based on the state transition record set and perturbation samples. The policy update target includes a policy loss term and a perturbation consistency term. A value update target is constructed based on the state transition record set. Gradient updates are performed on the network parameters of the meta-policy generation module and the action value assessment module based on the policy update target and the value update target to obtain updated network parameters of the meta-policy generation module and the action value assessment module. The updated network parameters of the meta-policy generation module and the updated network parameters of the action value assessment module are then combined to form the target decision parameter set.