Model-based adaptive optics dynamic modeling and control method and device

Abstract

The application discloses a kind of based on model's adaptive optics dynamic modeling and control method and device, belong to the cross field of adaptive optics and based on model's reinforcement learning, comprising the following steps: AO system state is defined as the combination of multimodal observation, and is encoded accordingly;Through cross attention mechanism, multimodal observation data is fused, and unified and rich dynamic information observation latent state is formed;Deep dynamic world model based on recurrent neural network is constructed, the model maintains hidden state, and infers discrete latent state, and the complete latent state representation of world model is jointly formed by the two;According to the complete latent state representation of world model, reconstruct the original observation and predict reward;In the world model learned, train reinforcement learning agent, learn optimal strategy through imaginary trajectory, and output AO system key parameters in real time.The application improves the adaptability, control accuracy and robustness of AO system in complex dynamic environment.

Description

Technical Field

[0001] This invention relates to the intersection of adaptive optics and model-based reinforcement learning, and more specifically, to a model-based adaptive optics dynamic modeling and control method and apparatus. Background Technology

[0002] Adaptive optics (AO) systems aim to improve the quality of optical imaging or laser transmission by compensating for wavefront distortion in real time. Existing technologies typically rely on experience-based control strategies, where control parameters (such as integral gain, sampling frequency, and exposure time) are often determined by prior knowledge or human experience, making it difficult to achieve optimal performance in dynamically changing and complex environments. For example, the intensity and characteristics of atmospheric turbulence are constantly changing, requiring AO systems to adjust their operating parameters in real time to maintain the best correction effect.

[0003] In recent years, reinforcement learning (RL) has shown great potential in complex decision-making and control problems and has been attempted to be applied to AO system control. However, applying RL to AO systems faces challenges such as the effective fusion of multimodal data (e.g., image and vector data), accurate modeling of system dynamics, and sensitive capture of transient changes. Existing methods still have shortcomings in handling heterogeneous sensor data and their temporal dynamics. For example, some methods may only fuse features from different modalities by simple splicing or fail to fully mine the temporal change information contained in vector data, resulting in inaccurate perception of system dynamics. In addition, their environmental models may lack the ability to make deep predictions of complex dynamics. These limitations restrict the control accuracy and robustness of AO systems in scenarios that require adaptive and intelligent adjustment of key system parameters. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a model-based adaptive optics dynamic modeling and control method and device, which can realize refined dynamic modeling and intelligent control of AO system in multimodal observation scenarios, and significantly improve the adaptability, control accuracy and robustness of AO system in complex dynamic environment.

[0005] The objective of this invention is achieved through the following solution: A model-based adaptive optics dynamic modeling and control method, comprising: Step 1, State data input and encoding: Considering the temporal variation of vector observations as part of the state representation, the state of the AO system is defined as a combination of multimodal observations and encoded accordingly, and the encoding includes encoding the temporal variation characteristics of vector observations; Step 2, Cross-attention fusion: Multimodal observation data are fused through a cross-attention mechanism to form a unified observation potential state rich in dynamic information; Step 3, Construction of the dynamic part of the world model based on recurrent neural networks: Construct a deep dynamic world model based on recurrent neural networks. This model maintains the hidden state and infers the discrete latent state. The two together form the complete latent state representation of the world model; and observation and reward reconstruction: Reconstruct the original observation and predicted reward based on the complete latent state representation of the world model. Step 4, Reinforcement learning agent training and parameter control: Train the reinforcement learning agent in the learned world model, learn the optimal policy by imagining the trajectory, and output the key parameters of the AO system in real time to achieve intelligent adaptive control.

[0006] Furthermore, in step 1, the combination of multimodal observations also includes image observations, and image observations and vector observations are independently encoded; wherein, image observations extract spatial features through a convolutional neural network (CNN) structure and map them into a latent image representation; vector observations are constructed by a vector time series encoder, which receives the vector observation sequence, learns the dynamic change pattern of the vector data through a self-attention mechanism, and outputs a latent sequence representing the dynamic information of the entire sequence.

[0007] Further, in step 1, the image observation includes a wavefront sensor image and a far-field image; the vector observation includes the values ​​of the wavefront slope and the corrector voltage.

[0008] Furthermore, in step 2, the fusion of multimodal observation data through the cross-attention mechanism specifically includes: By fusing image latent representations and vector time series latent sequences through a cross-attention mechanism, image latent representations are used as queries and vector time series latent sequences are used as keys and values. Through cross-attention computation, a deeply fused observation latent state containing dynamic information is generated.

[0009] Furthermore, in step 3, the construction of a deep dynamic world model based on a recurrent neural network, which maintains hidden states and infers discrete latent states, together forming a complete latent state representation of the world model, specifically includes: The world model predicts the hidden state using the recurrent neural network based on the hidden state, the latent state, and the action from the previous time step. The hidden state is used to predict the prior distribution of the latent state using a prior model. Simultaneously, the posterior distribution of the latent state is inferred using a posterior model by combining the observed latent state at the current time step. Finally, the complete latent state representation of the world model is composed of the hidden state and the latent state sampled from the posterior distribution, which is used to accurately model the temporal dynamics of the AO system.

[0010] Furthermore, in step 3, the reconstruction of the original observations and predicted rewards based on the complete latent state representation of the world model specifically includes the following sub-steps: Based on the complete latent state representation of the world model, the original observations are reconstructed through a decoder, and the corresponding reward values ​​are predicted, thereby completing the construction of the world model, enabling it to map from the latent state back to the observable space and predict the reward.

[0011] Furthermore, in step 3, the training process of the world model includes: Training is performed by minimizing the reconstruction loss and the KL divergence loss between the prior and posterior distributions, thereby ensuring accurate modeling of environmental dynamics and precise reconstruction of observations and rewards.

[0012] Furthermore, in step 4, training a reinforcement learning agent in the learned world model, learning the optimal policy through imagining trajectories, and outputting key parameters of the AO system in real time to achieve intelligent adaptive control specifically includes: The reinforcement learning agent trains its policy by simulating future trajectories. The agent's action space includes the adjustment of key control parameters of the AO system. It learns the optimal policy by maximizing long-term cumulative rewards and outputs the optimal control parameters in real time during actual deployment, thereby achieving intelligent adaptive control.

[0013] Furthermore, in step 4, the agent is specifically trained in the following manner: First, by exploratoryly interacting with the real AO system environment, raw observation, action execution, and reward data are collected to train the world model. Secondly, after the world model is fully trained, the agent learns efficiently in the latent space constructed by the world model. The agent transforms current or historical real observations into observational latent states, and combines the hidden and latent states of the world model as the starting point for planning. Then, using the world model, a series of action sequences are generated through the policy network, and the world model predicts the future hidden states, latent states and rewards caused by the action sequences in the latent space, forming an imagined trajectory. Finally, the agent's policy network and value network are updated based on the cumulative rewards obtained from the imagined trajectory in order to maximize long-term cumulative rewards.

[0014] A model-based adaptive optics dynamic modeling and control device includes a processor and a memory, wherein the memory stores a computer program that, when loaded by the processor, executes the method described in any of the preceding claims.

[0015] The beneficial effects of this invention include: (1) High-precision dynamic perception and modeling: This invention adopts a heterogeneous data independent encoding strategy and innovatively introduces multimodal fusion based on a cross-attention mechanism, which can effectively capture the dynamic changes of AO system vector data and deeply fuse them with image information, thereby achieving more comprehensive and accurate dynamic perception and modeling of the system state. This avoids the problem of insufficient state representation caused by simple splicing of heterogeneous data or lack of dynamic information in traditional methods, and significantly improves the richness and accuracy of state representation.

[0016] (2) Intelligent parameter adaptation and forward-looking control: Based on the learned deep dynamic world model, the reinforcement learning agent can autonomously learn and adjust key parameters of the AO system such as gain, sampling frequency, and exposure time in real time without human intervention or prior models. The world model's accurate prediction capability of system dynamics enables the agent to perform forward-looking planning, significantly improving the system's adaptability and performance in complex dynamic environments. Compared with traditional fixed parameter or simple feedback control, the method of this invention can more effectively cope with complex dynamic changes such as atmospheric turbulence.

[0017] (3) Strong versatility and scalability: The proposed multimodal data processing and fusion architecture, as well as the dynamic modeling paradigm based on the world model, are versatile and can be extended to other systems that need to process heterogeneous time series data and perform complex dynamic control, providing a new paradigm for intelligent decision-making and control in similar scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the steps of the method in an embodiment of the present invention. Detailed Implementation

[0020] All features disclosed in all embodiments of this specification, or steps in all methods or processes implied in the disclosure, may be combined and / or extended or replaced in any way, except for mutually exclusive features and / or steps.

[0021] The specific implementation process of this invention is as follows: In view of the problems in the background, the inventors of this invention believe that there is an urgent need for a model-based adaptive optics dynamic modeling and control method to solve the problems of inaccurate state perception caused by insufficient fusion of multimodal heterogeneous data, limited prediction ability caused by insufficient capture of dynamic changes in the system, and insufficient adaptive adjustment capability of AO system parameters in the prior art.

[0022] In a further inventive concept, addressing the shortcomings of current adaptive optics (AO) systems in dynamic environments, such as insufficient parameter adaptive adjustment capabilities, inadequate fusion of multimodal heterogeneous data, and inaccurate dynamic modeling, this invention aims to propose a model-based adaptive optics dynamic modeling and control method. This method achieves refined dynamic modeling and intelligent control of the AO system in multimodal observation scenarios through independent encoding of heterogeneous data, cross-attention fusion, collaboration between a deep dynamic world model and a reinforcement learning agent. First, the method structures the multimodal observation data (including image and vector data) of the AO system, paying particular attention to and explicitly encoding the temporal variation characteristics of the vector data, while simultaneously performing independent encoding. Second, a cross-attention mechanism is used to achieve deep fusion of image features and vector time-series dynamic features, forming a unified and informative observation potential state. Subsequently, a deep dynamic world model based on this latent state is constructed, and the complete latent state representation of the world model is used. By reconstructing observation and prediction rewards, and ultimately training a reinforcement learning agent, intelligent and adaptive control of key parameters of the AO system can be achieved.

[0023] In a preferred embodiment, a model-based adaptive optics dynamic modeling and control method is provided, applicable to the intelligent and adaptive adjustment of parameters in adaptive optics (AO) systems under dynamic and complex environments. This method, building upon existing reinforcement learning applications in AO systems, innovatively introduces heterogeneous data independent encoding, multimodal fusion based on a cross-attention mechanism, and a deep dynamic world model. Its main steps include: Step 1, State data input and encoding: Considering the temporal variation of vector observations as part of the state representation, the state of the AO system is defined as a combination of multimodal observations and encoded accordingly, and the encoding includes encoding the temporal variation characteristics of vector observations; Step 2, Cross-attention fusion: Multimodal observation data are fused through a cross-attention mechanism to form a unified observation potential state rich in dynamic information; Step 3, Construction of the dynamic part of the world model based on recurrent neural networks: Construct a deep dynamic world model based on recurrent neural networks. This model maintains the hidden state and infers the discrete latent state. The two together form the complete latent state representation of the world model; and observation and reward reconstruction: Reconstruct the original observation and predicted reward based on the complete latent state representation of the world model. Step 4, Reinforcement learning agent training and parameter control: Train the reinforcement learning agent in the learned world model, learn the optimal policy by imagining the trajectory, and output the key parameters of the AO system in real time to achieve intelligent adaptive control.

[0024] In another preferred embodiment, a model-based adaptive optics dynamic modeling and control method is provided, comprising the following steps: Step S1, State Data Input and Independent Encoding: The current state of the adaptive optics (AO) system is defined as a combination of multimodal observations, with particular attention paid to and the use of a Transformer structure to explicitly encode the temporal variation characteristics of vector observations. Image observations and vector time series are independently encoded; image observations include image observations... sum vector observation Among them, image observation Including wavefront sensor (WFS) images and far-field images, vector observations This includes the values ​​of wavefront slope and corrector voltage; to capture and explicitly utilize the dynamic changes in the vector data, the temporal changes in the vector observations are additionally considered as part of the state representation, i.e., using the vector observation sequence of the past K+1 time steps. Simultaneously, a convolutional neural network (CNN) structure is employed to specifically process image observations, extracting their spatial features and mapping them to latent image representations. For vector observations, a vector time series encoder is constructed to receive the vector observation sequence, learn the dynamic change patterns of the vector data through a self-attention mechanism, and output a latent sequence representing the dynamic information of the entire sequence. ; Step S2, Cross-Attention Fusion: The image latent representation and the vector time series latent sequence are deeply fused using a cross-attention mechanism to form a unified observation latent state rich in dynamic information. A cross-attention mechanism is introduced to integrate the image latent representation... As a query, vector time series latent sequence Using key and value, a deeply fused latent observation state containing dynamic information is generated through cross-attention computation. ; Step S3: Construction of the dynamic part of the world model based on recurrent neural networks: Construct a deep dynamic world model based on recurrent neural networks, which maintains a hidden state. And infer a potential state. Together, these two elements constitute the complete latent state representation of the world model; the world model is based on the hidden state of the previous time step. potential state and actions The hidden state is predicted by the recurrent neural network. , Predicting the prior distribution of latent states using a prior model Simultaneously, combined with the observed potential state at the current moment. The posterior distribution of the latent state is inferred through the posterior model. Ultimately, the hidden state and latent states sampled from the posterior distribution Together they form the complete latent state representation of the world model. , used for accurate modeling of the temporal dynamics of AO systems; Step S4, Observation and Reward Reconstruction: Reconstruct the original observations and predicted rewards based on the complete latent state representation of the world model. Specifically, reconstruct the original observations and predicted rewards based on the complete latent state representation of the world model. The original observations are reconstructed through a decoder, and the corresponding reward value is predicted, thereby completing the construction of the world model, enabling it to map from the latent state back to the observable space and predict the reward. Step S5: Reinforcement Learning Agent Training and Parameter Control: A reinforcement learning agent is trained within the learned world model. It learns the optimal policy by imagining trajectories and outputs key parameters of the AO system in real time, achieving intelligent adaptive control. Specifically, within the learned world model, the reinforcement learning agent trains its policy by simulating future trajectories. The agent's action space includes adjustments to the key control parameters of the AO system. It learns the optimal policy by maximizing long-term cumulative rewards and outputs the optimal control parameters in real time during actual deployment, achieving intelligent adaptive control.

[0025] The embodiments of the present invention do not rely on human experience or traditional fixed models. Through high-precision perception and refined dynamic modeling of multimodal heterogeneous data, the adaptability, control accuracy and robustness of the AO system in complex dynamic environments are significantly improved.

[0026] Based on the above embodiments, in other more specific alternative embodiments, a model-based adaptive optics dynamic modeling and control method is provided, including the following steps: SS1. State Data Input and Independent Encoding: This step aims to comprehensively acquire multimodal observation information from the AO system, with particular emphasis on capturing dynamic information. Specifically, at each time step t, multimodal observations from the adaptive optics system are received. .in The images include microlens array spot images from the wavefront sensor (WFS) and imaging quality images from the far-field camera, directly reflecting the current wavefront distortion and the imaging effect after system correction. This includes the values ​​of the wavefront slopes and DM voltages at the current moment, both of which are vectors. To more accurately capture the AO system's response to dynamic changes such as atmospheric turbulence, this embodiment innovatively focuses on and explicitly encodes the temporal changes in the vector observations, namely the differences in slope and voltage between the current and previous moments. These changes serve as supplementary features to the vector observations, collectively forming the complete vector input signal. Simultaneously, heterogeneous data is independently encoded: the Image Encoder employs a deep convolutional neural network structure, such as, but not limited to, variants of ResNet or EfficientNet, specifically for processing image observations. This CNN extracts spatial hierarchical features of an image through multiple layers of convolution, activation, and pooling operations, and ultimately maps them to a fixed-dimensional latent image representation. For vector observations, a vector time series encoder is constructed. This encoder receives the vector observation sequence from the past K+1 time steps. To capture the dynamic information of the vector data in depth, this encoder preferably adopts the structure of a Transformer encoder block. The vector at each time step... First, a linear projection layer maps the sequence to the embedding dimension of the Transformer, and positional encoding is added to preserve temporal order information. The Transformer encoder processes the sequence in parallel using a multi-head self-attention mechanism, effectively capturing the complex temporal dependencies between vectors at different time steps in the sequence, thereby learning the dynamic change patterns of the vector data. Its output is a latent sequence representing the dynamic information of the entire sequence. .

[0027] SS2. Cross-Attention Fusion: This step aims to effectively and intelligently fuse heterogeneous multimodal observation data from the AO system. This is for deep fusion of latent image representations. Sum vector time series latent sequence This embodiment innovatively introduces a cross-attention mechanism. In this layer, the image latent representation... The query Q is generated after passing through a linear layer. Vector time series latent sequence. The key K and value V are generated through two independent linear layers. A cross-attention mechanism calculates the similarity between Q and K, and uses these similarities as weights to perform a weighted summation of V. The calculation process is as follows:

[0028]

[0029] in, The dimension of the key. Through cross-attention, image information can actively "focus" on and extract the dynamic information most relevant to its own state from the vector time series, thereby generating a deeply fused observational latent state containing dynamic information. .Should The final observed latent state is then obtained through a small multilayer perceptron (MLP). .

[0030] SS3. Construction of the Dynamic Part of the World Model Based on Recurrent Neural Networks: This step aims to construct the core part of the world model responsible for predicting the dynamic evolution of states. This embodiment constructs a deep dynamic world model based on recurrent neural networks (RNNs) to learn and predict the temporal dynamic characteristics of the AO system. This world model maintains a continuous hidden state. And infer a potential state. Specifically, the world model consists of a recurrent state-space model (RSSM), at its core of which is a gated recurrent unit (GRU). The GRU is input with the hidden state from the previous time step. potential state and actions Output the current hidden state. The dynamic part of the world model also includes a prior model used to predict the distribution of potential states without relying on current observations. And combined with potential observations The posterior model is then used to more accurately infer the distribution of the latent states. Ultimately, due to the hidden state... and latent states sampled from the posterior distribution Together they form the complete latent state representation of the world model. This is used to accurately model the temporal dynamics of the AO system. The prior and posterior models are typically implemented using feedforward neural networks (e.g., multilayer perceptrons (MLPs) consisting of fully connected layers).

[0031] When training a world model, to ensure the learned latent space has good structure and predictive power, a Kullback-Leibler divergence is calculated between the latent state distribution output by the prior model and the latent state distribution inferred by the posterior model. Minimizing this Kullback-Leibler divergence as part of the loss function aims to encourage the posterior distribution to be as close as possible to the prior distribution. This allows the model to make reasonable latent state predictions even without current observation information and prevents the posterior distribution from becoming overly reliant on observations and ignoring the structure of the latent space. This contributes to learning a more robust and generalizable latent dynamic model.

[0032] SS4. Observation and Reward Reconstruction: This step aims to construct the part of the world model responsible for reconstructing observations and predicting rewards from the latent states, and to combine it with the dynamic part (SS3) to complete the training of the world model. This is based on the complete latent state representation of the world model. The original observations (including images) are reconstructed using a decoder. and vector data ), and predict the corresponding reward value. The world model (including the dynamic part defined in step S3 and the observation and reward part defined in this step) is trained by minimizing the reconstruction loss and the KL divergence loss between the prior and posterior distributions, thereby ensuring accurate modeling of environmental dynamics and precise reconstruction of observations and rewards. The loss function can be expressed as:

[0033] in, It is the probability of reconstructing the original observation from the latent state through the decoder.

[0034] SS5. Reinforcement Learning Agent Training and Parameter Control: This step aims to train an agent to achieve intelligent adaptive control of the AO system parameters based on the learned world model. In the model-based reinforcement learning framework, the training of the agent is closely related to the training of the world model, and the agent primarily learns by interacting with the learned world model. The specific training process is as follows: SS51. Data Collection and World Model Training: First, the agent collects raw observations through limited exploratory interactions with the real AO system environment. Execution of actions and receive rewards The data was used to train world models (SS3 and SS4) to accurately predict the dynamics of the environment. The training objective of the world models was to minimize the reconstruction loss and KL divergence loss to ensure that they could extract valid latent states from the observations. And predict future state transitions .

[0035] SS52. Trajectory Generation: Once the world model is sufficiently trained, the agent can use the model for efficient learning without frequent interaction with the real environment. Specifically, the agent learns from current or historical real observations. Transform into the observation potential state (Through the encoding and fusion process of steps SS1 and SS2), and combined with the hidden states of the world model. and potential state (i.e., complete latent state representation) This serves as the starting point for planning. Then, utilizing the learned world model, it is processed through a policy network (Actor). A series of action sequences are generated, and the world model predicts the future hidden states resulting from these action sequences within the latent space. Discrete latent states and rewards This has created a large number of imaginary trajectories. , where L is the length of the imagined trajectory.

[0036] SS53. Policy Network and Value Network Updates: Policy Network (PRN) is... Figure 1 The Actor shown is the agent's policy network. Responsible for representing the complete latent state based on the current world model. To predict the optimal action The training objective is to maximize the cumulative reward obtained on the imagined trajectory. This is typically achieved through policy gradient methods (such as PPO, SAC, etc.), which update the parameters of the policy network based on the reward signal in the imagined trajectory and the evaluation of the value network. The update of the policy network can be represented as:

[0037] in, It is the advantage function, used to measure the relative merits of taking a certain action in a specific state.

[0038] Value Network Figure 1 The Critic shown is the agent's value network. Responsible for evaluating the complete latent state representation in a given world model. The expected cumulative reward is the next possible outcome. The training objective of the value network is to minimize the error between its predicted and actual cumulative rewards. This is typically achieved through methods such as temporal difference (TD) learning or generalized advantage estimation (GAE). The update of the value network can be expressed as:

[0039] in From state Initial discount cumulative reward estimate.

[0040] SS54. Iterative Optimization: The parameters of the policy and value networks are updated using a gradient descent optimization algorithm. During training, data is periodically sampled from imagined trajectories generated by the world model to batch update the policy and value networks. The world model and the reinforcement learning agent learn together in a unified optimization loop. This iterative process allows the world model to continuously improve its understanding of the environment, while the agent uses a more accurate world model to learn better control policies.

[0041] After training, the agent can receive observation data from the AO system in real time and generate data through the encoder and fusion layer. And input world model update and The intelligent agent, based on the current... Output the optimal parameter adjustment action This enables intelligent and adaptive control of the AO system.

[0042] In another embodiment of the present invention, a model-based adaptive optics dynamic modeling and control device is also provided, including a processor and a memory, wherein the memory stores a computer program that executes any of the methods described above when the computer program is loaded by the processor.

[0043] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0044] According to other aspects of the present invention, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.

[0045] In another aspect, embodiments of the present invention also provide a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

Claims

1. A model-based adaptive optics dynamic modeling and control method, characterized in that, include: Step 1, State data input and encoding: Define the AO system state as a combination of multimodal observations, which include image observations and vector observations; Encoding the multimodal observations includes: encoding the vector observations and their temporal variations, wherein the temporal variation is the difference between the vector observations at the current time and the previous time, and the current vector observation and the temporal variation together constitute the vector input signal; wherein the vector observations include the values ​​of the wavefront slope and the corrector voltage; Step 2, Cross-attention fusion: Multimodal observation data are fused through a cross-attention mechanism to form a unified observation potential state rich in dynamic information; Step 3, Construction of the dynamic part of the deep dynamic world model based on recurrent neural networks: Construct a deep dynamic world model based on recurrent neural networks. This model maintains the hidden state and infers the discrete latent state. The two together constitute the complete latent state representation of the world model; and observation and reward reconstruction: Reconstruct the original observation and predicted reward based on the complete latent state representation of the world model; The deep dynamic world model refers to the environmental model constructed based on recurrent neural networks for predicting the state evolution of the AO system. Step 4, Reinforcement learning agent training and parameter control: Train the reinforcement learning agent in the learned world model, learn the optimal strategy by simulating future trajectories, and output the key parameters of the AO system in real time to achieve intelligent adaptive control; the simulated future trajectory refers to the future state-action-reward sequence generated by simulating action sequences within the deep dynamic world model.

2. The model-based adaptive optics dynamic modeling and control method according to claim 1, characterized in that, In step 1, the combination of multimodal observations also includes image observations, and image observations and vector observations are independently encoded; wherein, the image observations include wavefront sensor images and far-field images; the image observations extract spatial features through a convolutional neural network (CNN) structure and map them into latent image representations; the vector observations construct a vector time series encoder, receive the vector observation sequence, learn the dynamic change pattern of the vector data through a self-attention mechanism, and output a latent sequence representing the dynamic information of the entire sequence.

3. The model-based adaptive optics dynamic modeling and control method according to claim 2, characterized in that, In step 1, the image observation includes wavefront sensor images and far-field images; the vector observation includes the values ​​of wavefront slope and corrector voltage.

4. The model-based adaptive optics dynamic modeling and control method according to claim 2, characterized in that, In step 2, the fusion of multimodal observation data through the cross-attention mechanism specifically includes: By fusing image latent representations and vector time series latent sequences through a cross-attention mechanism, image latent representations are used as queries and vector time series latent sequences are used as keys and values. Through cross-attention computation, a deeply fused observation latent state containing dynamic information is generated.

5. The model-based adaptive optics dynamic modeling and control method according to claim 1, characterized in that, In step 3, the construction of a deep dynamic world model based on a recurrent neural network is described. This model maintains hidden states and infers discrete latent states, which together constitute the complete latent state representation of the world model. Specifically, this includes: The world model predicts the current hidden state using the recurrent neural network based on the hidden state, the potential state, and the action from the previous time step. The hidden state is used to predict the prior distribution of the potential state using a prior model. Simultaneously, the posterior distribution of the potential state is inferred using a posterior model, incorporating the observed potential state from the current time step. Finally, the hidden state and the potential states sampled from the posterior distribution together form the complete potential state representation of the world model, which is used to accurately model the temporal dynamics of the AO system.

6. The model-based adaptive optics dynamic modeling and control method according to claim 5, characterized in that, In step 3, the reconstruction of the original observations and predicted rewards based on the complete latent state representation of the world model specifically includes the following sub-steps: Based on the complete latent state representation of the world model, the original observations are reconstructed through a decoder, and the corresponding reward values ​​are predicted, thereby completing the construction of the world model, enabling it to map from the latent state back to the observable space and predict the reward.

7. The model-based adaptive optics dynamic modeling and control method according to claim 6, characterized in that, In step 3, the training process of the world model includes: Training is performed by minimizing the reconstruction loss and the KL divergence loss between the prior and posterior distributions, thereby ensuring accurate modeling of environmental dynamics and precise reconstruction of observations and rewards.

8. The model-based adaptive optics dynamic modeling and control method according to claim 1, characterized in that, In step 4, training a reinforcement learning agent in the learned world model, learning the optimal policy by simulating future trajectories, and outputting key parameters of the AO system in real time to achieve intelligent adaptive control specifically includes: The reinforcement learning agent trains its policy by simulating future trajectories. The agent's action space includes the adjustment of key control parameters of the AO system. It learns the optimal policy by maximizing long-term cumulative rewards and outputs the key control parameters of the AO system in real time during actual deployment, thereby achieving intelligent adaptive control.

9. The model-based adaptive optics dynamic modeling and control method according to claim 1, characterized in that, In step 4, the agent is trained in the following manner: First, by exploratoryly interacting with the real AO system environment, raw observation, action execution, and reward data are collected to train the world model. Secondly, after the world model is trained, the agent learns efficiently in the latent space constructed by the world model. The agent transforms current or historical real observations into observational latent states, and combines the hidden and latent states of the world model as the starting point for planning. Then, using the world model, a series of action sequences are generated through the policy network, and the world model predicts the future hidden states, latent states, and rewards caused by the action sequences in the latent space, forming a trajectory that simulates the future. Finally, the agent's policy network and value network are updated based on the cumulative rewards obtained from simulating future trajectories to maximize long-term cumulative rewards.

10. A model-based adaptive optics dynamic modeling and control device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when loaded by the processor, executes the method as described in any one of claims 1 to 9.

Images