Adaptive reinforcement learning inference migration method based on causal structure and latent variable

Abstract

The invention relates to the field of artificial intelligence and computer science, in particular to a causal structure and latent variable-based adaptive reinforcement learning reasoning migration method, which comprises the following steps of: constructing a causal world model fused with multi-modal observation and a decoupling latent variable space; establishing a hierarchical inference engine comprising an intuition layer, a conventional layer and a planning layer; pre-training a quick response and judicial planning dual-mode strategy and generating an interpretable fuzzy rule base; performing calculation level coarse tuning based on task identification and causal complexity; evaluating the real-time state criticality through an adaptive neural fuzzy system and dynamically switching a decision mode; after the action is executed, the threshold and the rule are subjected to closed-loop optimization, and cross-environment efficient migration is realized by utilizing a causal modularization characteristic. According to the technical scheme, consumption of computing resources is remarkably reduced on the premise that decision precision and safety are guaranteed, and the response speed and cross-scene adaptive capacity of a system on edge equipment are improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer science, specifically to an adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables. Background Technology

[0002] With the widespread application of deep reinforcement learning (DRL) in intelligent agent decision-making systems, its deployment in the real physical world has become a key path towards general artificial intelligence. However, current mainstream DRL methods face severe challenges when migrating from simulation environments to real-world scenarios. The core bottleneck lies in the imbalance between weak model generalization ability and online inference resource consumption. Traditional model-free or model-based DRL frameworks generally rely on statistical correlations between observations and actions for policy learning, lacking explicit modeling of the inherent causal mechanisms of the environment. This makes the policy highly susceptible to interference from non-causal factors (such as illumination, texture, and other surface features), leading to performance collapse even under minor environmental perturbations. At the same time, the ever-expanding model size in order to improve decision accuracy requires full computation for each inference, making it impossible to dynamically adjust cognitive resources according to task complexity. This results in a contradictory situation where low energy efficiency on edge devices and insufficient response in critical scenarios coexist.

[0003] Among these, the adaptive reasoning transfer method based on causal structure and latent variables aims to overcome the aforementioned limitations. By constructing a decoupled and modular Causal World Model (CWM), it achieves the learning of invariant environmental laws and efficient reasoning scheduling. The core of this approach lies in mapping high-dimensional heterogeneous observations to a low-dimensional causal latent variable space, and on this basis, establishing a hierarchical reasoning engine to enable the agent to possess dynamic decision-making capabilities similar to the human "intuition-deep thinking" dual system. However, current technologies have not yet effectively integrated the synergistic mechanism among causal discovery, latent variable decoupling, and adaptive computation, making it difficult to simultaneously achieve generalization, sample efficiency, and real-time performance.

[0004] In existing technologies, while static model compression can reduce average computational overhead, it sacrifices peak performance in complex scenarios. Traditional adaptive inference relies on shallow input features to determine computational requirements, lacking a deep understanding of the crucial nature of decision-making. Even though some studies have attempted to introduce causal models to enhance generalization, they have not fully utilized the modular structure of causal graphs and the dynamic values ​​of latent variables to guide fine-tuning of inference granularity. Especially in real-world scenarios driven by multi-source heterogeneous perceptual data, how to achieve coarse-grained tuning of computational levels and fine-tuning of mode switching online based on offline-built causal knowledge bases remains a pressing technical challenge. Therefore, there is an urgent need for an adaptive reinforcement learning inference transfer method that can deeply integrate causal structure priors and dynamic feedback of latent variables to achieve a highly generalized, low-latency, and robust intelligent decision-making system. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables. This method involves establishing a hierarchical reasoning engine comprising an intuition layer, a conventional layer, and a planning layer; pre-training a dual-mode strategy of rapid response and deliberate planning to generate an interpretable fuzzy rule base; performing coarse-tuning of the computational hierarchy based on task recognition and causal complexity; evaluating the criticality of real-time states and dynamically switching decision modes through an adaptive neurofuzzy system; optimizing thresholds and rules in a closed-loop manner after action execution; and leveraging the modularity of causality to achieve efficient cross-environment transfer.

[0006] To achieve the above objectives, this invention provides an adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables, comprising: Step 1: Construct a causal world model and initialize a knowledge base. Construct a high-dimensional observation space through multimodal data preprocessing, learn the latent variable space based on decoupling and causal representation, and use time-series causal graphs to discover and construct a complete causal world model using modular dynamic models. Step 2: Construct a hierarchical and multi-level reasoning engine for causal mechanisms. Quantify the importance of causal mechanisms in multiple dimensions, and define and implement three computational layers: a basic intuition layer, a key routine layer, and a complete planning layer. Step 3: Pre-train the dual-mode strategy and generate a fuzzy rule base to build a dual-mode decision system with a fast-response strategy and a prudent planning module, and automatically generate an interpretable fuzzy rule base through a data-driven approach; Step 4: Perform coarse adjustment of the computational hierarchy based on the static causal structure. Calculate the causal complexity index and determine the basic computational hierarchy by dynamically activating the subgraph of the causal world model through task identification and causal world model. Step 5: Dynamically switch modes based on dynamic latent variables, perform basic calculations and real-time evaluations, calculate the real-time state criticality through an adaptive neurofuzzy inference system, and make the final calculation mode decision based on dynamic thresholds. Step 6: Execute actions and perform closed-loop optimization and cross-environment migration, record performance data and perform closed-loop adaptive optimization, and realize cross-environment policy migration and adaptation.

[0007] Preferably, the multimodal data preprocessing in step 1 includes processing visual data from multiple vehicle cameras, spatial geometric data from LiDAR, dynamic and inertial data from inertial measurement units and global positioning systems, agent ontological state data, and historical interaction data. All sensor data are fused into a unified high-frequency timestamp sequence through a spatiotemporal alignment and synchronization method based on factor graph optimization, and the existing delays and drifts are compensated.

[0008] Preferably, the multi-sensor fusion and representation adopts a deep fusion architecture, using an attention mechanism to dynamically assign weights to the features of different sensor modalities. Image data is processed by a convolutional neural network to extract visual features, and point cloud data is processed by a point cloud processing network to extract geometric features. These features are projected into a shared feature space and deeply interacted and fused through a cross-attention module to finally generate a unified high-dimensional observation vector.

[0009] Preferably, the serialization and historical information embedding stacks the observation vectors of four consecutive frames to form an input tensor containing short-term historical dynamics, which serves as the final input for subsequent processing, ensuring complete capture of dynamic information.

[0010] Preferably, the latent variable space learning based on decoupling and causal representation employs an improved variational autoencoder. The encoder is a deep convolutional network that receives the serialized observation tensor and outputs the Gaussian parameters of the latent variable distribution. The decoder is a deconvolutional network that receives latent variables sampled from the latent variable distribution and attempts to reconstruct the original observations.

[0011] Preferably, the loss function includes reconstruction loss to ensure information integrity, KL divergence to constrain the latent variable distribution to approximate the standard normal distribution prior, and total correlation penalty to penalize the mutual information between various dimensions of latent variables through clever sampling and density ratio estimation techniques, thereby achieving full decoupling of the latent variable space.

[0012] Preferably, the training strategy adopts an annealing strategy, which gradually increases the values ​​of relevant hyperparameters in the early stage of training, so that the model first learns to reconstruct and grasp the main information, and then gradually learns to decouple, thereby achieving stable convergence of the training process.

[0013] Preferably, the time-series causal graph discovery employs an improved conditional independence test algorithm. This algorithm can effectively handle complex nonlinear relationships between latent variables, avoid the curse of dimensionality in high-dimensional latent variables, and is specifically designed to address the autocorrelation and unobserved common parent node problems that are common in time series data.

[0014] Preferably, the modular implementation of the dynamic model directly uses the discovered causal graph as the static adjacency matrix of the graph neural network. The update function of the latent variable corresponding to each node is designed as an independent neural network module, whose input is only the state of the causal parent node of that node, which perfectly reflects the modular characteristics of the causal mechanism.

[0015] Preferably, the uncertainty quantification adopts a deep ensemble method, training five graph neural network models with identical structures but different random initializations. During inference, multiple different predictions are obtained for the same input. The mean of these predictions is calculated as the final prediction, and the variance is used as a measure of cognitive uncertainty.

[0016] Preferably, the multi-dimensional quantification of the importance of the causal mechanism in step 2 is achieved through graph theory centrality analysis, including a weighted fusion of out-degree centrality, betweenness centrality, and PageRank score. Out-degree centrality reflects the number of downstream variables directly affected by a node, betweenness centrality reflects the pivotal role of a node in the causal chain, and PageRank score recursively defines the importance of a node.

[0017] Preferably, the weighted fusion is performed using a preset formula, wherein each centrality index is normalized and then multiplied by its corresponding weight and summed. The weights are set based on domain knowledge or automatically optimized through meta-learning to obtain a more robust and comprehensive importance ranking than a single index.

[0018] Preferably, the basic intuition layer in the definition of the computing layer includes the top 10% of nodes in terms of importance, which are mostly the agent's own state and the most direct environmental inputs, providing the fastest and most basic environmental state snapshots, with extremely low computing latency of less than 1 millisecond.

[0019] Preferably, the key conventional layer contains the top 60% of nodes in terms of importance, encompassing all the causal mechanisms required to handle the vast majority of conventional driving scenarios, including lane keeping, following other vehicles, and simple interactions with nearby vehicles, serving as the main computing layer used by default in the daily operation of the system.

[0020] Preferably, the complete planning layer contains 100% of the nodes and mechanisms, including all the knowledge of the causal world model, including modeling of secondary but critical factors such as severe weather, abnormal behavior, and long-term energy consumption, and is activated only when the system determines that the current state is extremely critical, complex, or highly uncertain.

[0021] Preferably, the engineering implementation in the graph neural network achieves hierarchical computation through a learnable gating mechanism or a simple mask. During inference, the system only activates the computation graph of the corresponding node according to the selected level, thereby saving computing resources at the hardware level.

[0022] Preferably, the fast-response strategy in step 3 adopts a small multilayer perceptron architecture, which contains 3 layers with 256 neurons each. Its input is designed as the latent variables of the key conventional layers to ensure low latency between receiving information and calculating output.

[0023] Preferably, the training algorithm combines model prediction and value learning, simulating only a short horizon of 5 steps forward in the causal world model, and then using a value network trained together with the policy network to estimate the long-term value of the state after the 5th step, which greatly reduces the computational load of online planning.

[0024] Preferably, the prudent planning module employs an improved optimization algorithm, which is a sampling-based black-box optimization algorithm that does not require the reward function to be smooth and differentiable. It is very suitable for searching for the optimal action in complex causal world model simulation environments where local optima may exist.

[0025] Preferably, the process deep analysis includes initializing and defining a high-dimensional Gaussian distribution to represent the search space for the next 15 steps of action sequence, sampling 500 candidate action sequences from this distribution, performing a full high-fidelity forward simulation on each sequence using a causal world model with a full planning layer, and obtaining multiple future trajectories and corresponding cumulative rewards.

[0026] Preferably, the elite selection and update selects the elite sequence with the highest reward of 10%, calculates the mean and variance of these elite sequences and uses them to update the search distribution. After repeating this process 3 times, the mean of the updated distribution is output as the optimal action sequence, and the first action of the sequence is executed.

[0027] Preferably, the automated generation of the fuzzy rule base adopts a fully data-driven approach, paying special attention to moments when the strategy makes mistakes or hesitates during the training of the fast-response strategy. These moments usually correspond to high Bellman errors, and their states are key states that require higher-level thinking.

[0028] Preferably, the fuzzy C-means clustering and the interpretability of the rules are used to perform fuzzy C-means clustering on the collected key state data, and output the membership matrix of each data point to each cluster center. Each cluster center is the prototype of the key state.

[0029] Preferably, the rule generation automatically generates highly interpretable fuzzy rules by analyzing the vector values ​​of cluster centers. For example, when the distance to the vehicle in front is very close and the uncertainty of predicting the intention of the vehicle in front is very high, the state is of extremely high importance. The shape and size of the cluster are used to determine the mathematical definition of the fuzzy linguistic variables.

[0030] Preferably, in step 4, the task identification and dynamic activation of the causal world model subgraph adopt a hybrid identification mechanism, including explicit identification from the navigation system and implicit identification from a lightweight scene classifier trained on the latent variables of the causal world model.

[0031] Preferably, once a task is identified, the system extracts the subgraph most relevant to the task from the global causal graph, which is accomplished by analyzing the set of latent variables activated by each task in advance in an offline phase. For example, the latent variables related to the high activation of the parking task and the precise distance perception of low-speed control.

[0032] Preferably, the causal complexity index includes average depth reflecting the average length of the causal chain in the task, average width reflecting the number of causal factors that need to be considered in parallel at each time step, and the longest causal chain reflecting the worst-case complexity of the task.

[0033] Preferably, the weights are not fixed, but are optimized on a dataset containing multiple tasks through meta-learning. The optimization goal is to make the causal complexity metric best predict the average computational resources required to achieve the same decision performance on different tasks.

[0034] Preferably, the threshold for determining the basic computing level is determined through extensive benchmark testing during the offline evaluation phase. The testing objective is to find the causal complexity index boundary point that can effectively distinguish tasks. The decision logic is a simple table lookup logic, and the computational overhead is negligible.

[0035] Preferably, in step 5, the basic calculation and real-time evaluation system calls the inference engine of the calculation level determined in step 4 to obtain the current latent variables and uncertainty measures, providing basic data support for subsequent refined decision-making.

[0036] Preferably, the adaptive neural fuzzy inference system directly maps the fuzzy inference system to a five-layer feedforward neural network. The first fuzzification layer calculates the degree to which the input variable belongs to a certain fuzzy set. The second rule layer calculates the trigger strength of each rule. The third normalization layer normalizes the trigger strength. The fourth defuzzification layer calculates the rule output based on the rule conclusion part. The fifth output layer performs a weighted summation of all rule outputs to obtain the real-time state key scalar value.

[0037] Preferably, the parameters of the adaptive neural fuzzy inference system, including the membership function parameters and the conclusion parameters, can be trained and fine-tuned end-to-end using the standard gradient descent algorithm, enabling it to continuously adapt to environmental changes.

[0038] Preferably, the dynamic threshold represents the system's current risk preference or level of alertness. It is not a static value but is dynamically adjusted based on recent performance feedback. The system uses an online learning algorithm to balance and explore switching modes under different real-time critical values ​​and to utilize the current optimal threshold strategy.

[0039] Preferably, the decision logic indicates that the current state is known, conventional, and predictable when the real-time state criticality is less than the dynamic threshold, and the system has a high degree of confidence that the rapid response strategy is sufficient to cope with it. When the real-time state criticality is greater than or equal to the dynamic threshold, it indicates that the current state triggers a rule about danger, unknown, and high uncertainty.

[0040] Preferably, when the computation upgrade is triggered, the system immediately discards the current calculation results of the base layer and instead calls the complete layer to re-evaluate the state to obtain the most comprehensive latent variables. Then, this high-fidelity state is sent to the prudent planning module to ensure that decisions are made based on the most complete and accurate information at critical moments.

[0041] Preferably, in step 6, the performance data recording system maintains a dedicated adaptive optimization experience pool, which records richer data tuples than the traditional reinforcement learning experience pool, including metadata such as causal complexity indicators, real-time state criticality, computational latency, and resource load.

[0042] Preferably, the online update algorithm for the dynamic threshold aims to maximize the long-term comprehensive utility function, which can be modeled as a contextual gambling problem. The context is the current real-time state key value, and the action is to choose between a quick reaction or prudent planning. The dynamic threshold is dynamically adjusted through an online learning algorithm.

[0043] Preferably, the asynchronous fine-tuning of the fuzzy rule base is performed when the system is idle or at a very low frequency. It utilizes the hybrid learning algorithm of the adaptive neural fuzzy inference system, uses data from the adaptive optimization experience pool to fine-tune the nonlinear parameters of the membership function through gradient descent, and updates the linear parameters of the rule conclusion part through least squares analysis.

[0044] Preferably, the theoretical basis for the cross-environment strategy transfer and adaptation is that the causal world model has a high combinatorial generalization ability because the causal representation of learning decoupled is learned. When an independent mechanism of the environment changes, ideally it will only affect the corresponding module in the causal world model.

[0045] Preferably, the environmental change detection and attribution includes the observation that the continuous increase in the reconstruction error of the encoder indicates a change in the sensor model or the appearance of the environment, and the mechanism change that the prediction error of the dynamic model for one or more latent variables is continuously high, indicating a change in the intrinsic dynamics of the environment. The system automatically locates the causal mechanism of the change by analyzing which node has the largest prediction error.

[0046] Preferably, the adaptation process includes fine-tuning the encoder by using a small amount of data collected in the new environment to perform end-to-end fine-tuning of the encoder to adapt it to new visual features; fine-tuning specific causal modules by freezing all other modules in the graph neural network and only fine-tuning the node update functions related to the change mechanism; and fine-tuning the policy network by keeping the semantics of the latent variable space stable, which only requires very brief fine-tuning on the new dynamic model to quickly master the skills in the new environment.

[0047] Compared with the closest existing technology, the present invention has the following advantages: By constructing a decoupled causal world model and a hierarchical inference engine, explicit modeling of the inherent causal mechanism of the environment is achieved, effectively overcoming the sensitivity of traditional methods to non-causal factors. Through a dual-mode strategy and an adaptive computation scheduling mechanism, the computational resource consumption is significantly reduced while ensuring decision-making accuracy. The modular design based on causal structure enables the system to have strong cross-environment migration capabilities, requiring only fine-tuning of specific modules to adapt to new environments. Real-time state criticality assessment and dynamic threshold adjustment mechanisms ensure the robustness and security of the system in complex scenarios. The entire method achieves high energy efficiency and fast response on edge devices, solving the core bottleneck problem of deploying intelligent decision-making systems in the real physical world. Attached Figure Description

[0048] Figure 1 This is a flowchart of the adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables provided by the present invention; Figure 2 This is a schematic diagram illustrating the principle framework of the adaptive reinforcement learning inference transfer method based on causal structure and latent variables provided by this invention. Detailed Implementation

[0049] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Example 1: This invention provides an adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables, such as... Figure 1 As shown, it includes: The overall technical architecture of this invention includes a causal world model construction module, a multi-level inference engine, a dual-mode strategy system, an adaptive computation scheduling unit, and a closed-loop optimization and transfer adaptation module. The entire system takes multi-source heterogeneous sensing data as input and, through the synergistic effect of causal structure and latent variables, achieves highly generalized, low-latency, and robust intelligent decision-making from simulation to reality.

[0052] In the aforementioned adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables, step (1) involves constructing a causal world model and initializing a knowledge base. A high-dimensional observation space is built through multimodal data preprocessing. The latent variable space is learned based on decoupling and causal representation. A complete causal world model is constructed using temporal causal graph discovery and modular dynamic models. Specifically, the multimodal data preprocessing handles visual data from multiple vehicle cameras, spatial geometric data from lidar, dynamic and inertial data from inertial measurement units and global positioning systems, agent ontology state data, and historical interaction data. All sensor data are fused into a unified high-frequency timestamp sequence using a factor graph-based spatiotemporal alignment and synchronization method, and the existing delays and drifts are compensated. This spatiotemporal alignment process employs a sliding window optimization strategy with a window length set to 200 milliseconds to ensure sub-millisecond time synchronization accuracy even in high-speed dynamic scenarios. Building upon this foundation, multi-sensor fusion and representation adopt a unified deep fusion architecture. An attention mechanism dynamically assigns weights to features from different sensor modalities. Image data is processed through a convolutional neural network to extract visual features, while point cloud data is processed through a point cloud processing network to extract geometric features. These features are projected into a shared feature space and deeply fused through a cross-attention module, ultimately generating a unified high-dimensional observation vector. This high-dimensional observation vector has 1024 dimensions, with each dimension carrying specific semantic information, such as lane curvature, relative speed of adjacent vehicles, and yaw rate of the vehicle itself.

[0053] Furthermore, serialization and historical information embedding stack the observation vectors of four consecutive frames to form an input tensor containing short-term historical dynamics, which serves as the final input for subsequent processing, ensuring complete capture of dynamic information. This input tensor has a shape of [4, 1024], with a time dimension of 4 and a feature dimension of 1024. Based on this input tensor, the system performs latent variable space learning based on decoupling and causal representation. This learning process employs an improved variational autoencoder. The encoder is a deep convolutional network that receives the serialized observation tensor and outputs the Gaussian parameters of the latent variable distribution. The decoder is a deconvolutional network that receives latent variables sampled from the latent variable distribution and attempts to reconstruct the original observations. The latent variable space is set to 64 dimensions to achieve effective compression of high-dimensional observations with minimal information loss. The loss function includes a reconstruction loss to ensure information integrity, a KL divergence constraint that the latent variable distribution approximates a standard normal distribution prior, and a total correlation penalty that penalizes the mutual information between different dimensions of the latent variables through clever sampling and density ratio estimation techniques, achieving sufficient decoupling of the latent variable space. The reconstruction loss is measured by mean squared error, and the KL divergence coefficient is initially set to 0.01, then linearly annealed to 0.1 during training. The total correlation penalty coefficient is initially set to 0, then linearly increased to 1.0 after the 5000th iteration. The training strategy employs an annealing approach, gradually increasing the values ​​of relevant hyperparameters in the early stages of training. This allows the model to first learn to reconstruct and capture the main information, and then gradually learn to decouple, achieving stable convergence during the training process.

[0054] After constructing the latent variable space, the system performs time-series causal graph discovery. This process employs an improved conditional independence test algorithm, which effectively handles complex nonlinear relationships between latent variables, avoids the curse of dimensionality in high-dimensional latent variables, and is specifically designed to address the autocorrelation and unobserved common parent node problems commonly found in time series data. The algorithm constructs the conditional independence test statistic based on the kernelized Hilbert-Schmidt independence criterion (HSIC) and introduces a sliding window mechanism to handle long-sequence dependencies. The final output causal graph is a directed acyclic graph (DAG), with nodes corresponding to 64 latent variables and edges representing direct causal effects between variables. Subsequently, the modular implementation of the dynamic model directly uses the discovered causal graph as the static adjacency matrix of the graph neural network. The update function for each node's latent variable is designed as an independent neural network module, whose input is only the state of the causal parent node of that node, perfectly reflecting the modularity of the causal mechanism. Each update function module adopts a gated recurrent unit (GRU) architecture with a hidden layer dimension of 32. Uncertainty quantification employs a deep ensemble approach, training five graph neural network models with identical structures but different random initializations. During inference, these models generate multiple different predictions for the same input. The mean of these predictions is calculated as the final prediction, and the variance is used as a measure of cognitive uncertainty. This uncertainty measure will serve as one of the key inputs for subsequent adaptive computational decision-making.

[0055] In the above adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables, step (2) constructs a hierarchical and multi-level reasoning engine for causal mechanisms. It quantifies the importance of causal mechanisms in multiple dimensions and defines and implements three computational layers: a basic intuition layer, a key routine layer, and a complete planning layer. Specifically, the multi-dimensional quantification of the importance of causal mechanisms is achieved through graph theory centrality analysis, including a weighted fusion of out-degree centrality, betweenness centrality, and PageRank score. Out-degree centrality reflects the number of downstream variables directly influenced by a node, betweenness centrality reflects the pivotal role of a node in the causal chain, and PageRank score recursively defines node importance. The weighted fusion is performed using a preset formula, where each centrality index is normalized and then multiplied by its corresponding weight. The weights are set based on domain knowledge or automatically optimized through meta-learning, resulting in a more robust and comprehensive importance ranking than a single index. The weighted fusion formula is as follows: Importance(vi)=w1*Norm(OutDegree(vi))+w2*Norm(Betweenness(vi))+w3*Norm(PageRank(vi))Importance(vi)=w1*Norm(OutDegree(vi))+w2*Norm(Betweenness(vi))+w3*Norm(PageRank(vi)) Where vi represents the i-th latent variable node, w1, w2, w3 are the corresponding weights, and w1+w2+w3=1. Norm() represents the normalization operation, which scales each index to the [0,1] interval.

[0056] Based on importance ranking, the computational layers are clearly defined: the basic intuition layer contains the top 10% of nodes, i.e., the 6 most important nodes (64 * 10% ≈ 6), mainly representing the agent's state and the most direct environmental inputs, providing the fastest and most basic snapshot of the environmental state with extremely low computational latency of less than 1 millisecond; the critical routine layer contains the top 60% of nodes, i.e., the 38 most important nodes (64 * 60% ≈ 38), encompassing all causal mechanisms required to handle most routine driving scenarios, including lane keeping, following other vehicles, and simple interactions with nearby vehicles, serving as the default main computational layer used in daily system operation; the complete planning layer contains 100% of the nodes and mechanisms, including all knowledge of the causal world model, including modeling of secondary but critical factors such as severe weather, abnormal behavior, and long-term energy consumption, and is activated only when the system determines that the current state is extremely critical, complex, or highly uncertain. In engineering implementation, hierarchical computation is achieved through learnable gating mechanisms or simple masks in the graph neural network implementation. During inference, the system only activates the computation graph of the corresponding node based on the selected layer, achieving computational resource savings at the hardware level. For example, when a key regular layer is selected, the system generates a 64-bit binary mask vector, where the first 38 bits are 1s and the rest are 0s. This mask is used to mask the computation of irrelevant nodes during the message passing phase of the graph neural network.

[0057] In the above adaptive reinforcement learning inference transfer method based on causal structure and latent variables, step (3) involves pre-training a dual-mode policy and generating a fuzzy rule base to construct a dual-mode decision system consisting of a fast-response policy and a prudent planning module. An interpretable fuzzy rule base is automatically generated through a data-driven approach. Specifically, the fast-response policy employs a small multilayer perceptron architecture, containing 3 layers with 256 neurons each. Its input is designed as the latent variables of the key conventional layers, ensuring low latency between information reception and output computation. The output of this policy is an action probability distribution, with inference latency controlled within 2 milliseconds. The training algorithm combines model prediction and value learning, simulating only a short horizon of 5 steps forward in the causal world model. Then, a value network trained jointly with the policy network is used to estimate the long-term value of the state after the 5th step, significantly reducing the computational load of online planning. The value network also employs a multilayer perceptron architecture, sharing some underlying features with the policy network.

[0058] The prudent planning module employs an improved optimization algorithm, a sampling-based black-box optimization algorithm that does not require the reward function to be smooth or differentiable. This makes it well-suited for searching optimal actions in complex causal world model simulation environments where local optima may exist. The process involves initializing a high-dimensional Gaussian distribution to represent the search space for the next 15 action sequences. From this distribution, 500 candidate action sequences are sampled. For each sequence, a full high-fidelity forward simulation is performed using a causal world model with a complete planning layer, yielding multiple future trajectories and corresponding cumulative rewards. Elite selection and updating involves selecting the top 10% of elite sequences (50 sequences), calculating their mean and variance, and using them to update the search distribution. This process is repeated three times, and the mean of the updated distribution is output as the optimal action sequence. The first action of this sequence is then executed. The entire prudent planning process has a computational latency of approximately 200 milliseconds and is only invoked when necessary.

[0059] The automated generation of the fuzzy rule base adopts a fully data-driven approach, paying particular attention to moments when the strategy errs or hesitates during the training of fast-response strategies. These moments typically correspond to high Bellman errors, and their states are critical states requiring higher-level consideration. The system collects latent variable vectors from these critical states, forming a critical state dataset. Fuzzy C-means clustering and rule interpretability are applied to the collected critical state data, outputting a membership matrix of each data point to each cluster center. Each cluster center represents the prototype of a critical state. The number of clusters, C, is determined offline using the silhouette coefficient method, typically between 5 and 10. Rule generation automatically generates highly interpretable fuzzy rules by analyzing the vector values ​​of the cluster centers. For example, when the distance to the vehicle ahead is very close and the uncertainty of predicting the vehicle's intention is high, the state is extremely critical. The shape and size of the clusters are used to determine the mathematical definitions of fuzzy linguistic variables, such as the membership function parameters of fuzzy sets like "very close" and "very high."

[0060] In the above adaptive reinforcement learning inference transfer method based on causal structure and latent variables, step (4) involves coarsely adjusting the computational hierarchy based on the static causal structure, calculating the causal complexity index, and determining the basic computational hierarchy through task identification and dynamic activation of the causal world model subgraph. Specifically, task identification and dynamic activation of the causal world model subgraph employ a hybrid identification mechanism, including explicit identification from the navigation system and implicit identification from a lightweight scene classifier trained on the latent variables of the causal world model. Explicit identification directly reads the task instructions issued by the navigation system, such as "highway cruise", "city road following", and "automatic parking"; implicit identification analyzes the current latent variables in real time through a two-layer fully connected network and outputs the scene category probability. The two identification results are fused through weighted voting, and the weights are dynamically adjusted according to the confidence level. Once a task is identified, the system extracts the subgraph most relevant to the task from the global causal graph, which is completed by analyzing the set of latent variables activated by each task in advance in the offline stage. For example, the parking task is highly activated and the latent variables are related to the precise distance perception of low-speed control.

[0061] The causal complexity index includes average depth, reflecting the average length of the causal chain in the task; average width, reflecting the number of causal factors that need to be considered in parallel at each time step; and longest causal chain, reflecting the worst-case complexity of the task. These three indices are calculated by graph traversal of the task subgraph. The weights are not fixed, but are optimized on a dataset containing multiple tasks through meta-learning. The optimization goal is to make the causal complexity index best predict the average computational resources required to achieve the same decision performance on different tasks. The threshold for determining the basic computational layer is determined through a large number of benchmark tests during the offline evaluation phase. The test goal is to find the causal complexity index boundary that can effectively distinguish tasks. The decision logic is a simple table lookup logic, and the computational overhead is negligible. For example, if the causal complexity index is below the threshold T1, the basic intuition layer is selected; if it is between T1 and T2, the key routine layer is selected; if it is above T2, the complete planning layer is initially recommended, but the final decision is left to the dynamic evaluation in step (5).

[0062] In the above adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables, step (5) involves dynamic mode switching based on dynamic latent variables, performing basic calculations and real-time evaluation, calculating the real-time state criticality through the adaptive neural fuzzy reasoning system, and making the final calculation mode decision based on dynamic thresholds. Specifically, the system for performing basic calculations and real-time evaluation calls the reasoning engine of the calculation layer determined in step (4) to obtain the current latent variables and uncertainty measure, providing basic data support for subsequent refined decision-making. The adaptive neural fuzzy reasoning system directly maps the fuzzy reasoning system to a five-layer feedforward neural network. The first fuzzification layer calculates the degree to which the input variable belongs to a certain fuzzy set, the second rule layer calculates the trigger strength of each rule, the third normalization layer normalizes the trigger strength, the fourth defuzzification layer calculates the rule output based on the rule conclusion part, and the fifth output layer performs a weighted summation of all rule outputs to obtain the real-time state criticality scalar value. The input of the system includes key latent variables, uncertainty measure, and causal complexity index in step (4), and the output is the real-time state criticality value in the interval [0,1].

[0063] The parameters of the adaptive neurofuzzy inference system, including membership function parameters and conclusion parameters, can be trained and fine-tuned end-to-end using the standard gradient descent algorithm, enabling it to continuously adapt to environmental changes. The dynamic threshold represents the system's current risk preference or alertness level; it is not a static value but dynamically adjusted based on recent performance feedback. An online learning algorithm balances exploration and attempts to switch modes under different real-time state criticality values, while adhering to the currently optimal threshold strategy. The decision logic states that when the real-time state criticality is less than the dynamic threshold, it indicates that the current state is known, conventional, and predictable, and the system has high confidence that a rapid response strategy is sufficient. When the real-time state criticality is greater than or equal to the dynamic threshold, it indicates that the current state triggers rules regarding dangerous, unknown, and highly uncertain conditions. When an upgrade is triggered, the system immediately discards the current base layer's calculation results and instead calls the complete layer to re-estimate the state, obtaining the most comprehensive latent variables. This high-fidelity state is then sent to the prudent planning module, ensuring that decisions at critical moments are based on the most complete and accurate information. The entire switching process is completed within 10 milliseconds, ensuring the system's real-time responsiveness.

[0064] In the above adaptive reinforcement learning inference transfer method based on causal structure and latent variables, step (6) involves executing actions and performing closed-loop optimization and cross-environment transfer, recording performance data and performing closed-loop adaptive optimization, and simultaneously realizing cross-environment policy transfer and adaptation. Specifically, the performance data recording system maintains a dedicated adaptive optimization experience pool, and the recorded data tuples are richer than those in the traditional reinforcement learning experience pool, including metadata such as causal complexity indicators, real-time state criticality, computational latency, and resource load. The online update algorithm for dynamic thresholds aims to maximize the long-term comprehensive utility function, which can be modeled as a contextual gambling problem. The context is the current real-time state criticality value, and the action is to choose between a rapid response or prudent planning. The dynamic threshold is dynamically adjusted through an online learning algorithm. The asynchronous fine-tuning of the fuzzy rule base is performed when the system is idle or at a very low frequency. Using the hybrid learning algorithm of the adaptive neural fuzzy inference system, the nonlinear parameters of the membership function are fine-tuned through gradient descent using data in the adaptive optimization experience pool, and the linear parameters of the updated rule conclusion part are analyzed and updated through least squares.

[0065] The theoretical basis for cross-environment policy transfer and adaptation is the causal world model. Due to the high combinatorial generalization ability of the decoupled causal representations learned, when an independent mechanism in the environment changes, ideally only the corresponding module in the causal world model will be affected. Environmental change detection and attribution include observing changes: a continuously increasing encoder reconstruction error indicates a change in the sensor model or the appearance of the environment; a continuously high prediction error of the dynamic model for one or more latent variables indicates a change in the intrinsic dynamics of the environment. The system automatically locates the causal mechanism of the change by analyzing which node has the largest prediction error. The adaptation process includes fine-tuning the encoder using a small amount of data collected in the new environment to adapt it to new visual features; fine-tuning specific causal modules, freezing all other modules in the graph neural network and fine-tuning only the update functions of nodes related to the change mechanism; and fine-tuning the policy network. Because the semantics of the latent variable space remain stable, the policy network only needs very brief fine-tuning on the new dynamic model to quickly master the skills in the new environment. The entire transfer process can be completed within hundreds of interaction samples, significantly outperforming the transfer efficiency of traditional end-to-end policies.

[0066] like Figure 2 As shown, the core principle framework of the hierarchical causal mechanism and adaptive computation mode switching in this invention clearly demonstrates the complete decision-making flow from task identification, causal complexity calculation, basic level selection, to real-time state criticality assessment, dynamic threshold comparison, and finally, deciding whether to upgrade the computation. This framework achieves an organic combination of static structural priors and dynamic latent variable feedback, ensuring that the system optimizes the allocation of computing resources to the maximum extent while guaranteeing security.

[0067] In this embodiment, an adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables is implemented as follows: An autonomous vehicle is waiting to turn left at an intersection in the city. The traffic light is currently red, but vehicles are crossing the intersection from the opposite lane, and pedestrians are crossing the road at the same time. The system needs to make a left turn at the appropriate time while ensuring safety. This is a typical "unprotected left turn" scenario, involving the interaction of multiple causal mechanisms and exhibiting high decision-making complexity.

[0068] Step 1: Offline Construction of the Causal World Model (CWM) and Initialization of the Knowledge Base For multimodal data preprocessing and high-dimensional observation space construction, the system collected multimodal data from five cameras (front-view, rear-view, and surround-view), a 16-line LiDAR, an IMU (1000Hz), GPS (1Hz), and vehicle status data (speed, steering angle, throttle / brake). Factor graph optimization was used to align all sensor data to a 100Hz timestamp sequence, compensating for the 20ms delay of IMU data and the 15ms delay of LiDAR data. Visual features were extracted using ResNet-18 (256-dimensional), and point cloud features were processed using PointNet++ (256-dimensional). A unified observation vector (512-dimensional) was generated by fusing the data through a cross-attention mechanism. The observation vectors from four consecutive frames were stacked to form an input tensor (4×512) to capture short-term dynamics.

[0069] Latent variable space learning based on decoupling and causal representation was trained using a β-TCVAE model. Training parameters included β gradually annealed from 0.1 to 1.5 and γ gradually annealed from 0.01 to 0.5. Decoupled latent variables included z1 traffic light status (semantic relevance 0.92), z2 oncoming lane vehicle speed (0.88), z3 relative distance between the vehicle and oncoming vehicles (0.91), z4 pedestrian position and speed (0.85), z5 road surface adhesion (0.83), z6 vehicle steering angle (0.90), z7 vehicle speed (0.94), and z8 vehicle acceleration (0.89). The mutual information index decreased by an average of 43.2%, and the semantic interpretability score averaged 8.7 / 10, significantly higher than the 6.2 / 10 of traditional VAEs.

[0070] The temporal causal graph discovery and modular dynamic model construction utilize the PCMCI+ algorithm to identify key causal chains: traffic light status affects oncoming vehicle speed, oncoming vehicle speed affects relative distance, relative distance affects the vehicle's acceleration, road adhesion affects sideslip angle, and pedestrian position affects steering angle. In the modular GNN implementation, each node's update function depends only on its causal parent node, for example, z8^(t+1) = f8([z3^(t)], a_t). Five GNN models are trained for uncertainty quantification, and the prediction variance is calculated as the cognitive uncertainty U_ξ during inference.

[0071] Step Two: Building a Layered Causal Mechanism and a Multi-Level Inference Engine The importance of causal mechanisms was quantified in multiple dimensions based on centrality indicators. Out-degree showed that z1 was the highest traffic light status (5), followed by z3 relative distance (4); betweenness centrality was highest for z1 (0.45), followed by z3 (0.38); PageRank score was highest for z1 (0.22), followed by z3 (0.18). The weighted fusion score (weights w1=0.4, w2=0.3, w3=0.3) yielded scores of 0.9 for z1, 0.8 for z3, 0.6 for z5, 0.4 for z2, and 0.2 for z4.

[0072] The computational hierarchy is defined as follows: L1 (base layer) contains the top 10% of nodes (z1, z6, z7, etc.) with a latency of 0.8ms; L2 (regular layer) contains the top 60% of nodes with a latency of 8.5ms; and L3 (complete layer) contains all nodes with a latency of 95ms. Layer switching is achieved through dynamic masking using PyTorch Geometric, with a switching latency of less than 5ms. A preloading mechanism ensures that the L1 and L2 computation graphs are preloaded into the GPU cache.

[0073] Step 3: Dual-mode policy pre-training and fuzzy rule base generation In the dual-mode strategy, π_fast employs the TD-MPC algorithm, with inputs being L2 layer latent variables, H=5 steps, and a value network combining security and efficiency. π_slow employs the CEM algorithm, H=15 steps, K=500, p=10%, and multi-objective optimization encompassing security, efficiency, and comfort.

[0074] The fuzzy rule base is automatically generated using FCM clustering. 2000 high Bellman error state fragments were collected for critical states. After FCM clustering, three rules were generated: The first rule states that if the traffic light is red, the oncoming vehicle speed is high, the relative distance is near, and the pedestrian position is nearby, then the state criticality is extremely high. The second rule states that if the traffic light is red, the oncoming vehicle speed is medium, and the relative distance is medium, then the state criticality is high. The third rule states that if the traffic light is red, the oncoming vehicle speed is low, and the relative distance is far, then the state criticality is low. The rule base achieves a critical state recognition accuracy of 92.4% and an interpretability score of 8.5 / 10.

[0075] Step 4: Coarse-tuning of the computational hierarchy based on the static causal structure In terms of task recognition, the system explicitly identifies the "unprotected left turn" task through the navigation system and confirms it as a "left turn on an urban road" scenario through a scene classifier (small MLP). CWM subgraph activation extracts the task-related subgraph G_task, which contains nodes z1 to z7, excluding the z9 side slip angle node.

[0076] In the calculation of the Causal Complexity Index (CCI), the average depth D is 0.3, the average width W is 2.2, the longest causal chain L is 5, and the CCI = 0.4 × 0.3 + 0.3 × 2.2 + 0.3 × 5 = 2.28. Based on the thresholds T_low = 1.5 and T_high = 3.0, the CCI = 2.28 determines the basic computational level as L2.

[0077] Step 5: Dynamic mode switching based on dynamic latent variables In the basic calculation, the system calls the L2 layer inference engine to obtain latent variables z1=0 (red light), z2=22 (oncoming vehicle speed 22km / h), z3=12 (relative distance 12m), z4=0.8 (near pedestrian), z5=0.75 (road surface adhesion) and uncertainty U_t=0.6.

[0078] The ANFIS assessment inputs the above latent variables and calculates membership degrees: traffic light status (red) membership degree 1.0, oncoming vehicle speed (high) membership degree 0.7, relative distance (near) membership degree 0.6, pedestrian position (nearby) membership degree 0.8, and road surface adhesion (normal) membership degree 0.5. After calculating the rule trigger strength, RSC = 0.3864 is obtained. The dynamic threshold T_dynamic = 0.35; RSC ≥ T_dynamic triggers an upgrade calculation.

[0079] The upgrade process involves discarding the L2 layer results, calling the L3 layer for complete state estimation, inputting the results into π_slow for careful planning, outputting a 15-step action sequence, and executing the first action to start a 0.1° left turn. The upgrade latency is less than 12ms, representing an 83% reduction in latency compared to the original L3 inference.

[0080] Step Six: Action Execution, Closed-Loop Optimization, and Cross-Environment Migration During execution, the system performs the first action output by π_slow: initiating a 0.1° left turn, achieved through the vehicle control system. Empirical data includes Z_t^L3, U_t=0.6, a_t=0.1°, r_t=0.8, Z_{t+1}, CCI=2.28, RSC=0.3864, latency of 95ms, and resource load of 75%.

[0081] The online update of T_dynamic is implemented through a contextual gambling machine (Thompson Sampling). After the update, T_dynamic = 0.36, and the long-term utility is improved from -0.15 to -0.10. The fuzzy rule base is asynchronously fine-tuned when the system is idle, and the trigger strength weight of rule 1 is adjusted from 0.9 to 0.95.

[0082] In a cross-environment migration scenario, the system migrates from urban roads to rural dirt roads. Environmental change detection shows that the E_φ reconstruction error increased from 0.12 to 0.25, and the T_θ prediction error (z5 road surface adhesion) increased from 0.08 to 0.22. The adaptation process includes fine-tuning the encoder (using 200 dirt road samples, reducing the reconstruction error to 0.15), fine-tuning specific causal modules (z5 related nodes, reducing the prediction error to 0.10), and fine-tuning the policy network (500 steps, performance restored to 89.7%). After migration, the decision accuracy improved from 68.5% to 89.7%, requiring only 25 minutes and 200 samples, achieving a 12-fold improvement in adaptation speed compared to traditional methods.

[0083] System Operation Summary In the "unprotected left turn" scenario, the system constructs a CWM model offline, dividing the causal mechanism into three layers: L1, L2, and L3. Under normal conditions, layer L2 (8.5ms delay) is used. In high-risk conditions such as "red light + high oncoming vehicle speed + close relative distance + pedestrian nearby," an upgrade is triggered when RSC=0.3864 ≥ T_dynamic=0.35, switching to layer L3 and calling π_slow for careful planning to execute a safe left turn. After the action is executed, the system records experience data and dynamically updates T_dynamic and the fuzzy rule base. The system can quickly adapt to new environments (such as rural dirt roads), requiring only a small number of samples to fine-tune relevant modules.

[0084] In terms of system performance, the average inference latency has been reduced from 100ms in the original CWM to 8.5ms, the peak decision accuracy at critical moments remains above 95%, the utilization rate of computing resources in normal states has been improved by 90%, and the cross-environment adaptation time has been shortened from several hours to 25 minutes.

[0085] This example demonstrates how to solve the problem of low inference efficiency in CWM, and realize the transformation from "full computation" to "layered, on-demand, and intelligent invocation", ensuring that peak decision performance at critical moments is guaranteed while maximizing the utilization efficiency of computing resources under normal conditions.

[0086] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables, characterized in that, include: Step 1: Construct a causal world model and initialize a knowledge base. Construct a high-dimensional observation space through multimodal data preprocessing, learn the latent variable space based on decoupling and causal representation, and use time-series causal graphs to discover and construct a complete causal world model using modular dynamic models. Step 2: Construct a hierarchical and multi-level reasoning engine for causal mechanisms. Quantify the importance of causal mechanisms in multiple dimensions, and define and implement three computational layers: a basic intuition layer, a key routine layer, and a complete planning layer. Step 3: Pre-train the dual-mode strategy and generate a fuzzy rule base to build a dual-mode decision system with a fast-response strategy and a prudent planning module, and automatically generate an interpretable fuzzy rule base through a data-driven approach; Step 4: Perform coarse adjustment of the computational hierarchy based on the static causal structure. Calculate the causal complexity index and determine the basic computational hierarchy by dynamically activating the subgraph of the causal world model through task identification and causal world model. Step 5: Dynamically switch modes based on dynamic latent variables, perform basic calculations and real-time evaluations, calculate the real-time state criticality through an adaptive neurofuzzy inference system, and make the final calculation mode decision based on dynamic thresholds. Step 6: Execute actions and perform closed-loop optimization and cross-environment migration, record performance data and perform closed-loop adaptive optimization, and realize cross-environment policy migration and adaptation.

2. The adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables as described in claim 1, characterized in that: The multimodal data preprocessing includes processing visual data from multiple vehicle-mounted cameras, spatial geometric data from LiDAR, dynamic and inertial data from inertial measurement units and GPS, agent ontology state data, and historical interaction data. All sensor data are fused into a unified high-frequency timestamp sequence using a factor graph-based spatiotemporal alignment and synchronization method, and existing delays and drifts are compensated for. The high-dimensional observation space is generated through a deep fusion architecture. Image data is processed by a convolutional neural network to extract visual features, and point cloud data is processed by a point cloud processing network to extract geometric features. Both are projected into a shared feature space and fused through a cross-attention module to form a unified high-dimensional observation vector. The observation vectors of four consecutive frames are stacked to form the input tensor.

3. The adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables as described in claim 2, characterized in that: The latent variable space is learned through an improved variational autoencoder. The encoder is a deep convolutional network that receives the input tensor and outputs the Gaussian parameters of the latent variable distribution. The decoder is a deconvolutional network that receives the sampled latent variables and reconstructs the original observations. The loss function includes reconstruction loss, KL divergence constraint term and total correlation penalty term. The training employs an annealing strategy, gradually increasing the total correlation penalty coefficient and KL divergence coefficient in the early stages of training, so that the model first completes information reconstruction and then achieves latent variable decoupling.

4. The adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables as described in claim 1, characterized in that: The temporal causal graph is identified through an improved conditional independence test algorithm. This algorithm is based on the kernelized Hilbert-Schmidt independence criterion and introduces a sliding window mechanism, outputting a directed acyclic graph as the causal structure. The modular dynamic model uses this causal graph as the static adjacency matrix of the graph neural network. The update function of each latent variable node is an independent neural network module, which only receives the state of its causal parent node as input. Uncertainty quantification adopts a deep ensemble method, training five graph neural network models with the same structure but different initializations, using prediction variance as a measure of cognitive uncertainty.

5. The adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables as described in claim 1, characterized in that: The importance of the causal mechanism is quantified by a weighted fusion of out-degree centrality, betweenness centrality, and PageRank score, with each index normalized and then summed according to its weight. The basic intuition layer contains the top 10% of latent variable nodes in terms of importance, the key routine layer contains the top 60% of nodes, and the complete planning layer contains all nodes. Hierarchical computation is implemented in the graph neural network through binary masks or learnable gating mechanisms, activating only the nodes of the corresponding layer to compute the graph.

6. The adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables as described in claim 1, characterized in that: The rapid response strategy employs a multilayer perceptron with 256 neurons per layer (3 layers). The input is the latent variables of the key conventional layers, and the output is the action probability distribution. Its training combines short-field model prediction and value estimation. After 5 forward simulations in a causal world model, the subsequent long-term value is estimated by the co-trained value network. The prudent planning module uses a sampling-based black-box optimization algorithm. It initializes a Gaussian distribution to represent the action sequence of the next 15 steps, samples 500 candidate sequences, and selects the elite sequences with the highest rewards (10%) after simulation by the causal world model of the complete planning layer to update the search distribution. After 3 iterations, it outputs the first action of the optimal action sequence.

7. The adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables as described in claim 6, characterized in that: The fuzzy rule base is generated by collecting key state data corresponding to high Bellman errors during the training of fast-response strategies; fuzzy C-means clustering is performed on the key state data to obtain cluster centers as key state prototypes; fuzzy rules are automatically generated based on the vector values ​​of the cluster centers, and the cluster shape and size are used to define the membership function parameters of fuzzy linguistic variables.

8. The adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables as described in claim 1, characterized in that: The task identification adopts a hybrid mechanism, including explicit task instructions from the navigation system and implicit identification results from a lightweight scene classifier based on latent variables, which are fused by confidence weighting; the task subgraph is extracted from the global causal graph, and the set of latent variables activated by the corresponding task is pre-analyzed and determined in the offline stage; the causal complexity index includes average depth, average width, and longest causal chain, which are calculated by graph traversal. The basic computational level is determined based on a lookup logic that uses causal complexity indicators and preset thresholds.

9. The adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables as described in claim 1, characterized in that: The adaptive neural fuzzy reasoning system is mapped to a five-layer feedforward neural network, consisting of a fuzzification layer, a rule layer, a normalization layer, a defuzzification layer, and an output layer. The inputs include key latent variables, uncertainty measures, and causal complexity indices, and the output is a real-time state key scalar in the interval [0,1]. Its membership function parameters and conclusion parameters are trained end-to-end through gradient descent. The dynamic threshold is adjusted online based on recent performance feedback, and is modeled as a contextual gambling machine problem, where the context is the real-time state key value and the action is the selection of the calculation mode.

10. The adaptive reinforcement learning reasoning transfer method based on causal structure and latent variables as described in claim 1, characterized in that: The closed-loop optimization experience pool records metadata including causal complexity indicators, real-time state criticality, computational latency, and resource load; the fuzzy rule base is asynchronously fine-tuned when the system is idle, the membership function nonlinear parameters are updated through gradient descent, and the rule conclusion linear parameters are updated analytically through least squares method. When migrating across environments, the encoder reconstruction error and dynamic model prediction error are used to detect environmental changes, locate the affected causal mechanism, and fine-tune the encoder, specific causal modules and policy network respectively. The policy network only needs to be briefly fine-tuned on the new dynamic model.

Images