Data center network congestion control method based on double driving of data model

Abstract

The application discloses a data center network congestion control method based on a data model double drive, which combines the adaptive ability of deep reinforcement learning and the fast response ability of traditional model mapping to solve the problems of slow response speed and insufficient adaptability of the prior art in a dynamic network environment. The method adopts a hierarchical architecture, including an optimization engine module, an adjustment control module and a statistical information module. The optimization engine module learns network state data based on a deep reinforcement learning algorithm, dynamically optimizes a congestion control strategy; the adjustment control module quickly executes the strategy by using a model mapping mechanism, realizing real-time adjustment of the transmission rate; and the statistical information module continuously collects network state information, ensuring that the system adapts to complex traffic changes. The application can effectively reduce communication delay, reduce queue backlog, improve throughput and enhance the stability of the data center network in a complex dynamic network environment, and is particularly suitable for high-load scenes such as AI large model training.

Description

Technical Field

[0001] This invention relates to the field of computer network technology, specifically to a network congestion control method and system suitable for data center networks, and in particular to a fusion congestion control algorithm that combines traditional model mapping and deep reinforcement learning (DRL) to optimize network resource utilization, improve throughput, reduce latency, and ensure network stability and performance for large-scale AI model training. Background Technology

[0002] With the rapid development of generative artificial intelligence technology, especially the widespread application of large-scale models such as ChatGPT, the research and development of large AI models has become a hot topic at the forefront of science and technology. This trend has driven the upgrade of data center computing facilities, but it has also brought new challenges. First, large-scale model training requires highly parallel computing and frequent data exchange. While this can accelerate computation, it can easily lead to network congestion, reducing resource utilization and training efficiency. At the same time, training urgently requires high bandwidth, but large-scale data transmission will further reduce bandwidth utilization. In addition, high data throughput and complexity pose challenges to network stability. Network congestion or packet loss will prolong task processing time and affect system reliability. Therefore, to solve the severe challenges faced by data center networks, an efficient congestion control strategy is urgently needed to optimize resource utilization, improve network performance, and support large-scale model training.

[0003] Traditional congestion control schemes (such as DCTCP) have laid the foundation for research on data center network transmission protocols, but their simple control strategies have significant limitations in optimizing queue length and improving throughput. Improved schemes, such as DCQCN, while enhancing adaptability to network conditions, still exhibit slow response times, high flow latency, and complex parameter tuning in dynamic environments, increasing the difficulty of practical deployment. Other schemes, such as Swift and HPCC, while demonstrating excellent performance, rely on complex hardware configurations, making them difficult to promote and apply in ordinary commercial equipment. Furthermore, these schemes employ fixed event-action mapping mechanisms, lacking flexibility and struggling to adapt to rapid changes in network conditions.

[0004] In recent years, data-driven congestion control schemes (such as Aurora and AuTO) have demonstrated strong potential through DRL technology. These schemes learn from empirical data in the network to achieve real-time optimization in dynamic environments, exhibiting excellent generalization and adaptability. Compared to traditional schemes, they reduce reliance on manual parameter tuning and can adapt to complex and changing network conditions. However, under unknown network conditions, these schemes may lead to over- or under-utilization of network resources due to improper resource allocation. Furthermore, DRL algorithms still face challenges in convergence speed and stability, especially in scenarios with high-frequency dynamic bandwidth changes, where their slow response speed may affect actual performance.

[0005] In the complex network environment of large-scale AI model training, congestion control in data center networks faces more stringent challenges, especially in scenarios with dynamic bandwidth changes and frequent flow arrivals and departures. Congestion control schemes in large-scale AI model training require two capabilities: dynamic response capability and network adaptability. The former requires the scheme to quickly adjust the transmission rate to cope with frequent network state changes; the latter requires the scheme to flexibly adjust its strategy according to changes in link and traffic patterns. Model-mapping-based congestion control schemes, with their simple signal-action mapping mechanism, possess good dynamic response capabilities, but lack adaptability in frequently changing network environments. In contrast, data learning-based schemes, especially those based on DRL technology, exhibit strong network adaptability and can flexibly respond to dynamic changes. However, their high computational overhead and slow response limit their application in high-frequency dynamic scenarios. Therefore, in the environment of large-scale AI model training, achieving a balance between dynamic response and network adaptability becomes a key challenge in designing efficient congestion control.

[0006] This invention provides a data center network congestion control scheme based on a dual-drive data model, combining the flexible adaptability of DRL with the rapid response capability of traditional model mapping. This method effectively resolves the contradiction between dynamic response and network adaptation in existing congestion control schemes, providing a highly efficient and flexible solution for data center networks, particularly suitable for high-load scenarios such as large-scale AI model training. Summary of the Invention

[0007] Purpose of the invention: This invention provides a network congestion control method and system based on dual-drive data models. By combining the adaptability of DRL technology with the fast response capability of traditional model mapping, it solves the problems of insufficient response speed and limited adaptability of existing technologies in dynamic network environments. This optimizes network resource utilization, improves the throughput and stability of data center networks, effectively reduces flow completion time and communication latency, and meets the application scenarios with high bandwidth and high performance requirements such as AI large model training.

[0008] Technical solution: A data center network congestion control method based on dual-drive data model, the method comprising: S1. Construct a system framework including an optimization engine module, a statistical information module, and a rate regulation control module; the optimization engine module optimizes congestion control sub-policies in real time through optimization algorithms, the statistical information module is responsible for collecting network operation data to support the decision-making process of the optimization engine and system monitoring, the optimization engine module and the statistical information module together constitute an RL agent, and the rate regulation control module is used to execute the sub-policies generated by the RL agent. S2. Construct a parameterized rate adjustment model for network endpoints, specifically including: S21. The convergence result is calculated using network parameters and queuing delay, and used as a reference for adjusting the rate. The calculation formula is as follows:

[0009] in It's about adjusting parameters, and This is used to indicate the status of total network bandwidth and transmission demand. It is the queuing delay, which is the result obtained after serializing the RTT feedback and processing it to minimize the RTT. The rate update cycle needs to be adaptively adjusted according to network requirements. Reflecting network demand, determined by link bandwidth and the number of connections, the update cycle is designed as follows:

[0010] in It is an adaptive byte counter threshold. K Indicates link bandwidth. T++ This indicates an update to the byte counter. To improve convergence speed and fairness, the adaptive byte counter is designed to be independent of the transmission rate, i.e.:

[0011] in It is an adaptive byte counter threshold. It is an adjustment parameter, and , This is the current transmission rate; S22. Design a mechanism to reduce the rate of adjustment in the model, as follows:

[0012] in This indicates the target transmission rate that needs to be adjusted. Indicates the current end-side transmission rate, parameter and parameters To adjust the parameters, and These parameters control the aggressiveness of rate reduction when congestion occurs; if no CNP is received within a specified time, an update is required. ,in It is a regulatory factor. It is a fixed value used for adjustment. .

[0013]

[0014] S23. Design a rate increase mechanism for the adjustment model, which includes a rapid recovery phase, an adaptive increase phase, and a super-increase phase. Fast recovery phase: When both the timer and byte counter are less than the threshold, i.e. and Perform a fast recovery operation, in which To adjust the parameters, and The design is as follows:

[0015] Adaptive increment phase: When a timer or byte counter exceeds a threshold, i.e. or An adaptive additive speed-up operation is performed; the adaptive speed-up step size is related to the current transmission rate and link bandwidth, but not to the number of streams carried, as designed below:

[0016] in It's the link capacity. and To adjust parameters and ; Over-increment phase: When both the timer and the byte counter are greater than the threshold, i.e. and To perform an over-increment operation, the design is as follows:

[0017] in, ,and The value is 100. , To adjust the parameters; S24. After determining the overall design of the control model, set the corresponding control parameters in the network-side rate parameterization rate control model to be dynamically adjustable, as follows:

[0018] S3. Based on the DRL algorithm, the dynamically adjustable parameters in S23 are optimized, and the congestion control problem in the network is modeled as a Markov decision model, including setting state variables and reward functions. The description of network conditions includes the introduction of switch buffer occupancy rate. To describe the degree of congestion, this metric is represented by the proportion of packets marked with ECN; S4. Based on the DRL algorithm, design and solve the state and reward functions in step S3. Select the near-end policy optimization as the RL controller. The PPO algorithm is based on the Markov decision process for policy optimization.

[0019] Furthermore, step S3 specifically includes the following processes: S31. Set the state variables required for the DRL algorithm. Stateful parameters include current network throughput, round-trip latency, and packet loss rate, as well as measurements of transmission rate, providing visualization of network and bandwidth availability, where transmission rate... ; S32. Define the reward function required for the DRL algorithm. This reward function is used to quantify the performance standard of the task, guide the agent to improve the generated sub-policy sequence during the training phase, and the RL agent collects rewards at each monitoring interval to interact with the network environment; The reward function is designed as follows:

[0020] in:

[0021] in, This indicates the penalty for packet loss. , and These are weighting coefficients.

[0022] Furthermore, the design of the reward function includes taking power into account. Power maximization can reflect the maximum throughput while minimizing network latency, and also takes into account the ratio of transmission rate changes and the occupancy of switch buffers.

[0023] Furthermore, the objective function of the PPO algorithm described in step S4 includes advantage estimation and calculation of probability ratios; The advantage function is the difference between the expected value of a given action and the expected value of all possible actions in the same state. In order to estimate the advantage function, PPO uses a neural network model to train the value state function, calculates the total reward from the start to the expected realization of a given state, and finally uses the value of the critic network to estimate the advantage using generalized advantage estimation. The probability ratio is calculated as follows: ,in It's an old strategy. This is the updated strategy; To avoid large-scale policy updates, the PPO algorithm trims the target to within the advantage estimation range, as defined below:

[0024] It is time The estimated advantage, It is a hyperparameter that controls the cutting range; The optimization function for training a neural network model includes the squared error loss function:

[0025] By combining actor loss, critic loss, and entropy, the RL proxy objective function is achieved:

[0026]

[0027] in and It is a coefficient.

[0028] Furthermore, in each iteration of the PPO algorithm, participants independently collect observation trajectory data, including state, action, and reward sequences; For a given trajectory, the GAE algorithm calculates the advantage estimate for each time required to update the policy; Finally, using the collected observation trajectory data and estimates, the surrogate function is maximized via stochastic gradient descent (SGD).

[0029] Beneficial Effects: This invention proposes a data model-driven congestion control method that combines deep reinforcement learning technology with traditional model mapping mechanisms, achieving a good balance between dynamic response and adaptive capabilities. Its fast signal-action mapping mechanism can adjust the transmission rate in real time, while DRL technology enhances adaptability to complex network conditions through dynamic learning. This method can significantly optimize bandwidth utilization, reduce communication latency and queue length, and improve network throughput. It is suitable for high-performance and dynamically changing network environments such as large-scale AI model training. Attached Figure Description

[0030] Figure 1 A schematic diagram illustrating the dual-drive concept of the data model; Figure 2 This is a framework for a data center network congestion control method based on a dual-drive data model. Figure 3 This is a schematic diagram of the data center network algorithm execution process based on a dual-drive data model. Detailed Implementation

[0031] To illustrate the technical solutions disclosed in this invention in detail, the invention will be further explained below with reference to the accompanying drawings and embodiments.

[0032] (1) System architecture and implementation process

[0033] The data center network congestion control system based on a dual-drive data model proposed in this invention mainly consists of three parts: a rate regulation control module, an optimization engine module, and a statistical information module. The functions of each module and their interrelationships are as follows: The statistics module is responsible for collecting network operation data in real time, including key indicators such as throughput, round-trip latency, packet loss rate, and switch buffer utilization. It then performs standardized preprocessing on the data. Finally, it periodically transmits the processed network status data to the optimization engine module to support subsequent decision-making and optimization.

[0034] The optimization engine module employs deep reinforcement learning algorithms to analyze the network data provided by the statistical information module. It models the network state using Markov decision processes and optimizes the congestion control strategy based on PPO (Problem-Oriented Process). Furthermore, it iteratively optimizes the congestion control sub-strategies by incorporating historical experience data to adapt to current network conditions.

[0035] Rate regulation and control module: Receives congestion control strategies from the optimization engine module and executes corresponding rate adjustments based on the current network status. The module employs a parameterized rate regulation model, comprising four stages: fast recovery, adaptive adjustment, over-increase, and rate reduction control. Overall, it dynamically adjusts the data stream transmission rate through ECN marking and RTT feedback to ensure efficient utilization of network resources.

[0036] The system workflow of this invention employs a closed-loop feedback control mechanism, achieving efficient network congestion control through a continuously iterative evaluation-execution-policy generation loop. First, the statistics module collects network performance data and transmits it to the optimization engine. The optimization engine analyzes this data and uses reinforcement learning to optimize congestion control sub-policies, generating new policies adapted to the current network state. Subsequently, the adjustment control model executes these new policies, maximizing network performance by adjusting the rate. The system continuously monitors changes in network state, ensuring that each decision is based on the latest data, thereby achieving more efficient network congestion control.

[0037] (2) Design and implementation of network end-side rate adjustment model

[0038] This invention calculates the convergence result using parameters and queuing delay, and uses this result as a reference for adjusting the rate.

[0039]

[0040] in It is to adjust the parameters and This is used to indicate the status of total network bandwidth and transmission demand. It is the queuing delay, which is the result obtained by the algorithm after processing the RTT feedback by serialization delay and minimum RTT.

[0041] The system timer every 55 seconds Add 1 when the accumulated value exceeds the threshold. When the default value is 5, different speed-up phases are triggered. The rate update cycle needs to be adaptively adjusted according to network requirements. Reflecting network demand, it is determined by link bandwidth and the number of connections. Therefore, the update cycle should be designed as follows:

[0042] in This is an adaptive byte counter threshold. Similarly, the byte counter is a fixed value of 150KB. The counter increments by 1 after every 150KB of packets received. When the accumulated value exceeds the threshold, different acceleration phases are triggered. To improve convergence speed and fairness, the adaptive byte counter should be designed to be independent of the transmission rate.

[0043] in It is an adaptive byte counter threshold. It is to adjust the parameters and , This is the current transmission rate.

[0044] Design a mechanism to reduce the model's rate: To accelerate convergence, when the reference rate is significantly lower than the current rate, the model will drastically slow down to avoid severe congestion and quickly adjust to the ideal rate, as designed below:

[0045] Where parameters and parameters To adjust parameters and These parameters control the aggressiveness of rate reduction when congestion occurs. If no CNP is received within a certain period, an update is required. .

[0046]

[0047] Design adjustment model rate increase mechanism: The rate increase is mainly divided into three stages: rapid recovery stage, adaptive increase stage, and super-increase stage.

[0048] The first phase is the fast recovery phase, which occurs when both the timer and the byte counter are below the threshold. and Perform a fast recovery operation, in which To adjust parameters and The design is as follows:

[0049] The second phase is the adaptive increment phase, which occurs when a timer or byte counter exceeds a threshold. or Perform an adaptive additive speed-up operation. The adaptive speed-up step size should be related to the current transmission rate and link bandwidth, but not to the number of streams carried, as designed below:

[0050] in It's the link capacity. and To adjust parameters and .

[0051] Finally, there is the super-increment phase, when both the timer and the byte counter are greater than the threshold, i.e. and To perform an over-increment operation, the design is as follows:

[0052] in, ,and The value is 100. . To adjust the parameters.

[0053] After determining the overall design of the control model, this invention further sets the corresponding control parameters in the model to be dynamically adjustable, denoted as:

[0054] Specifically, parameters Used to represent the state of total network bandwidth and transmission demand, to calculate reference rates, and to provide the direction and magnitude of rate adjustments, while ensuring that timer thresholds are reasonable. Parameters Adjust the byte counter threshold to ensure that the rate increase interval is independent of the transmission rate. Parameter Ensure the adaptive growth step size is related to the current transmission rate and link bandwidth. Parameters and Adjusting the aggressiveness of rate reduction during congestion in a timely manner can accelerate convergence. Parameters The conservative degree of rate increase is uniformly adjusted to prevent re-congestion and accelerate convergence. Ultimately, the parameter set... The model's sending behavior is determined and defined as the output of the RL agent. This invention customizes sub-policies to adapt to the current network conditions by controlling these parameter settings.

[0055] (3) Design and implementation of RL agent optimization engine

[0056] Determine the state variables required for the algorithm: The state information collected by the RL agent needs to comprehensively consider the current network conditions and flow state. Throughput ( and Round-trip delay () and ) and packet loss rate ( This reflects the overall performance of the network; combined with the transmission rate ( The measurement of ) can visualize network and bandwidth availability, among which In a typical network, these three factors are sufficient to describe the network condition. However, due to the large amount of bursty traffic in data center networks (DCNs), we additionally introduce switch buffer occupancy (...). The congestion level is described by the proportion of packets marked with ECN.

[0057] Define the reward function required by the algorithm: The RL agent needs to define a reward function to quantify the performance standard of the task and guide the agent to improve the generated sub-policy sequences during the training phase. The RL agent collects rewards at each monitoring interval to interact with the network environment. The reward function is designed as follows:

[0058] in:

[0059] In the above formula, the first part fully considers power. Based on this fundamental design, maximizing power can achieve maximum throughput while minimizing network latency. This indicates the penalty for packet loss. The second part considers the rate of change in transmission rate and the occupancy of the switch buffer. , and These are weighting coefficients.

[0060] (4) Select an optimization algorithm to optimize the parameters.

[0061] This invention requires a comprehensive consideration of both the responsiveness and convergence speed of the algorithm. Even the best-performing reinforcement learning algorithm is not suitable if it has slow convergence. The stability of the RL algorithm is equally critical, especially in environmentally sensitive situations. Since data centers may contain thousands of devices, a high-performance, low-resource, and easily implemented algorithm is needed. We chose Proximal Policy Optimization (PPO) as the RL controller due to its fast convergence, low sample complexity, and high performance. Furthermore, PPO is simple to implement, insensitive to hyperparameters, requires few samples, has few convergence steps, and does not require replay buffer memory. Its optimization function design ensures that the policy does not deviate too much after each update, ensuring the stability of the algorithm. These characteristics make PPO the optimal choice.

[0062] The objective function of PPO includes advantage estimation and probability ratio: ,in It's an old strategy. This is the updated policy. To avoid large-scale policy updates, the PPO (Predicted Advantage Program) trims the objective to within the advantage estimation range. It is defined as follows:

[0063] It is time The estimated advantage, These are hyperparameters that control the scope of clipping. The advantage function is the difference between the expected value of a given action and the expected values ​​of all possible actions in the same state. To estimate the advantage function, PPO uses a neural network model (critic) to train the value state function, calculates the total reward from the start to the expected achievement of a given state, and finally estimates the advantage using generalized advantage estimation (GAE) based on the value of the critic network. The optimization function for training the critic model includes a squared error loss function:

[0064] By combining actor loss, critic loss, and entropy, we can implement the surrogate objective function, where... and It is a coefficient.

[0065]

[0066]

[0067] In each iteration of the PPO algorithm, multiple participants independently collect observation trajectory data over several time steps (state, action, and reward sequences). Given a trajectory, the GAE algorithm calculates the advantage estimate for each time step required for policy updates. Finally, using the collected data and estimates, the surrogate function is maximized via stochastic gradient descent (SGD) (or a similar method).

[0068] Based on the above implementation process, those skilled in the art should understand that this invention combines the adaptive capabilities of deep reinforcement learning with the rapid response capabilities of traditional model mapping to solve the problems of slow response speed and insufficient adaptability of existing technologies in dynamic network environments. It can effectively reduce communication latency, decrease queue backlog, increase throughput, and enhance the stability of data center networks in complex dynamic network environments, and is particularly suitable for high-load scenarios such as AI large-scale model training. This method is flexible in structure, easy to deploy, and can be widely applied to various computing network environments requiring efficient congestion control.

Claims

1. A data center network congestion control method based on double driving of data model, characterized in that, The method includes: S1. Construct a system framework including an optimization engine module, a statistical information module, and a rate regulation control module; the optimization engine module optimizes congestion control sub-policies in real time through optimization algorithms, the statistical information module is responsible for collecting network operation data to support the decision-making process of the optimization engine and system monitoring, the optimization engine module and the statistical information module together constitute an RL agent, and the rate regulation control module is used to execute the sub-policies generated by the RL agent. S2. Construct a parameterized rate adjustment model for network endpoints, specifically including: S21. The convergence result is calculated using network parameters and queuing delay, and used as a reference for adjusting the rate. The calculation formula is as follows: wherein is a tuning parameter, and , representing the state of the network total bandwidth and transmission demand, is the queuing delay, which is the result after the serialization delay and minimum RTT processing on the RTT feedback; The rate update period needs to be self-adaptively adjusted according to network demand, and the parameters Reflecting network demand, determined by link bandwidth and connection number, the update period is designed as: wherein is an adaptive byte counter threshold, K denotes the link bandwidth, T++ denotes the update of the byte counter, for improving the convergence speed and fairness, the adaptive byte counter is designed to be independent of the transmission rate, i.e.: in It is an adaptive byte counter threshold. It is an adjustment parameter, and , This is the current transmission rate; S22. Design a mechanism to reduce the rate of adjustment in the model, as follows: in This indicates the target transmission rate that needs to be adjusted. Indicates the current end-side transmission rate, parameter and parameters To adjust the parameters, and These parameters control the aggressiveness of rate reduction when congestion occurs; if no CNP is received within a specified time, an update is required. ,but: in It is a regulatory factor. It is a fixed value used for adjustment. ; S23. Design a rate increase mechanism for the adjustment model, which includes a rapid recovery phase, an adaptive increase phase, and a super-increase phase. Fast recovery phase: When both the timer and byte counter are less than the threshold, i.e. and Perform a fast recovery operation, in which To adjust the parameters, and The design is as follows: Adaptive increment phase: When a timer or byte counter exceeds a threshold, i.e. or An adaptive additive speed-up operation is performed; the adaptive speed-up step size is related to the current transmission rate and link bandwidth, but not to the number of streams carried, as designed below: in It's the link capacity. and To adjust parameters and ; Over-increment phase: When both the timer and the byte counter are greater than the threshold, i.e. and To perform an over-increment operation, the design is as follows: in, ,and The value is 100. , To adjust the parameters; S24. After determining the overall design of the control model, set the corresponding control parameters in the network-side rate parameterization rate control model to be dynamically adjustable, i.e. The definition is expressed as: S3. Based on the DRL algorithm, the dynamically adjustable parameters in S23 are optimized, and the congestion control problem in the network is modeled as a Markov decision model, including setting state variables and reward functions. The description of network conditions includes the introduction of switch buffer occupancy rate. To describe the degree of congestion, the switch's buffer occupancy rate Represented by the proportion of packets marked with ECN; S4. Based on the DRL algorithm, design and solve the state and reward functions in step S3. Select the near-end policy optimization as the RL controller. The PPO algorithm is based on the Markov decision process for policy optimization.

2. The data center network congestion control method based on dual-drive data model according to claim 1, characterized in that, Step S3 specifically includes the following processes: S31. Set the state variables required for the DRL algorithm. Stateful parameters include current network throughput, round-trip latency, and packet loss rate, as well as measurements of transmission rate, providing visualization of network and bandwidth availability, where transmission rate... ; S32. Define the reward function required for the DRL algorithm. This reward function is used to quantify the performance standard of the task, guide the agent to improve the generated sub-policy sequence during the training phase, and the RL agent collects rewards at each monitoring interval to interact with the network environment; The reward function is designed as follows: in: in, This indicates the penalty for packet loss. , and These are weighting coefficients.

3. The data center network congestion control method based on dual-drive data model according to claim 2, characterized in that, The design of the reward function includes consideration of power. Power maximization can reflect the maximum throughput while minimizing network latency, and also takes into account the ratio of transmission rate changes and the occupancy of switch buffers.

4. The data center network congestion control method based on dual-drive data model according to claim 1, characterized in that, The objective function of the PPO algorithm described in step S4 includes advantage estimation and calculation of probability ratios; The advantage function is the difference between the expected value of a given action and the expected value of all possible actions in the same state. In order to estimate the advantage function, PPO uses a neural network model to train the value state function, calculates the total reward from the start to the expected realization of a given state, and finally uses the value of the critic network to estimate the advantage using generalized advantage estimation. The probability ratio is calculated as follows: ,in It's an old strategy. This is the updated strategy; To avoid large-scale policy updates, the PPO algorithm trims the target to within the advantage estimation range, as defined below: It is time The estimated advantage, It is a hyperparameter that controls the cutting range; The optimization function for training a neural network model includes the squared error loss function: By combining actor loss, critic loss, and entropy, the RL proxy objective function is achieved: in and It is a coefficient.

5. The data center network congestion control method based on dual-drive data model according to claim 4, characterized in that, In each iteration of the PPO algorithm, participants independently collect observation trajectory data, including state, action, and reward sequences. For a given trajectory, the GAE algorithm calculates the advantage estimate for each time required to update the policy; Finally, using the collected observation trajectory data and estimates, the surrogate function is maximized via stochastic gradient descent (SGD).

Images