A method, device, storage medium and electronic device for automatically adjusting parameters of an adaptive optical system based on reinforcement learning

Abstract

The embodiments of the present application provide a method, device, storage medium and electronic device for automatically adjusting the parameters of an adaptive optical system based on reinforcement learning, which relates to the fields of reinforcement learning and adaptive optics. The method includes the following steps: the AO system observes the target and collects relevant data; the cross-attention mechanism is used to extract the features of the wavefront sensor image and the far-field image; the extracted features are combined with the collected data to form a representation of the current system state; the policy network of reinforcement learning is used to predict the action parameters based on the current state; based on the action of the policy network and the current system state, the next state is predicted based on the action of the policy network and the current system state, and the corresponding reward value is calculated using the reward function; the complete interaction trajectory is collected to update the policy network and the environmental model. The technical solution of the present application can realize the automatic adjustment of the key parameters of the AO system in a dynamic environment, improve the imaging quality of the system under complex conditions, and significantly reduce the need for manual intervention.

Description

Technical Field

[0001] The present application relates to the fields of reinforcement learning and adaptive optics, and more specifically, to a method, device, storage medium, and electronic device for automatically adjusting parameters of an adaptive optics system based on reinforcement learning. Background Art

[0002] Adaptive optics (AO) technology utilizes optoelectronic devices to measure wavefront dynamic errors in real time, employs fast electronic systems for calculation and control, and uses active devices for real-time wavefront correction. This enables optical systems to automatically adapt to changing external conditions and maintain optimal operating conditions. It has important applications in high-resolution imaging observations and highly concentrated laser energy transmission. The core goal of adaptive optics systems is to overcome wavefront distortion caused by atmospheric turbulence, thereby improving the resolution and imaging quality of optical systems. They primarily consist of three core components: a wavefront sensor, a wavefront corrector, and a controller. The wavefront sensor measures wavefront distortion and provides wavefront slope information for subsequent correction. The wavefront corrector dynamically adjusts the surface shape or refractive index of optical components to correct wavefront distortion. The controller calculates the optimal correction signal based on the data provided by the wavefront sensor. Currently, wavefront sensors rely on manual control or pre-programmed rules. While these sensors can improve imaging quality, the interference from atmospheric turbulence is highly nonlinear, making it difficult to describe with precise mathematical models. Furthermore, they cannot adapt quickly to rapid changes in atmospheric conditions or target brightness.

[0003] To improve the imaging quality of adaptive optical systems, one of the main existing methods is to use deep learning technology. Traditional Shack-Hartmann sensors calculate the local slope of the wavefront by analyzing the position of the light spot on the microlens array. However, due to noise interference, the slope calculation may produce errors. To solve this problem, some methods propose to use convolutional neural networks (CNNs) to predict the slope from the wavefront image, or directly predict the phase difference of the wavefront. In addition, for phase recovery, existing technologies also achieve more efficient and accurate phase information reconstruction through the powerful nonlinear fitting capabilities of machine learning models.

[0004] Although these deep learning methods can significantly improve the performance of adaptive optics systems, they still have some limitations. First, deep learning methods are highly dependent on the quality of training data and require a large amount of labeled data for training, which places high demands on data acquisition. Especially in complex environments such as atmospheric turbulence, obtaining sufficient high-quality labeled data is a challenge. In addition, although these methods can achieve good results on the training set, they have poor generalization ability when faced with different environmental conditions (such as changes in turbulence intensity), often showing a significant performance degradation.

[0005] These issues have limited the widespread adoption of deep learning methods in practical applications. While they can improve the performance of AO systems under certain conditions, automated adjustments cannot fully replace manual intervention when the system's imaging quality is poor. In scenarios with strong atmospheric turbulence or rapidly varying target brightness, manually adjusting AO system parameters remains a common solution. However, due to the rapid pace of environmental change in these scenarios, manual adjustments often fail to respond in a timely manner and cannot fully adapt to rapidly changing external environmental conditions. Summary of the Invention

[0006] The present invention provides a method, apparatus, storage medium, and electronic device for automatically adjusting adaptive optics system parameters based on reinforcement learning. This approach addresses the problem of existing AO systems, where parameter adjustment relies on manual experience and suffers from slow response speeds. By introducing reinforcement learning techniques and designing corresponding adjustment mechanisms, the system achieves automatic optimization and adjustment of AO system parameters, improving imaging quality and system robustness in complex dynamic environments.

[0007] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0008] According to a first aspect of an embodiment of the present application, a method for automatically adjusting parameters of an adaptive optical system based on reinforcement learning is provided, comprising the steps of:

[0009] observing a target and collecting relevant data, wherein the relevant data includes a wavefront sensor image, a wavefront slope, a turbulence intensity, a wavefront corrector voltage, and a far-field image;

[0010] The features of the wavefront sensor image and far-field image are extracted using the cross-attention mechanism;

[0011] The extracted features are concatenated with the wavefront slope, turbulence intensity and wavefront corrector voltage to form a representation of the current system state;

[0012] A policy network using reinforcement learning predicts action parameters based on the current state;

[0013] Combined with the environment model, the next state is predicted based on the action of the policy network and the current system state, and the corresponding reward value is calculated using the reward function;

[0014] Collect interaction trajectories to update the policy network and environment model, thereby achieving continuous optimization.

[0015] In some embodiments of the present application, based on the aforementioned solution, the observation of the target and the collection of relevant data are achieved through the AO system, wherein the collected relevant data are used to comprehensively describe the current environment.

[0016] In some embodiments of the present application, based on the aforementioned scheme, the action parameters represent all parameters required for the AO system control process, and include the operating frame rate and operating gain of the wavefront sensor, and the exposure time, gain, and number of restoration modes of the imaging camera.

[0017] In some embodiments of the present application, based on the aforementioned solution, calculating the corresponding reward value using the reward function includes the following sub-steps:

[0018] The Strehl ratio is used as the reward function and is combined with the selected evaluation metric and defined in the selected calculation method.

[0019] In some embodiments of the present application, based on the aforementioned solution, the interaction trajectory includes state, action, and reward information.

[0020] In some embodiments of the present application, based on the above solution, the updating of the strategic network and environment model specifically includes the following sub-steps:

[0021] Update using offline data: Use offline datasets constructed using historical observation data to pre-train the policy network and environment model.

[0022] Use online updates: In practice, the policy network and environment model are dynamically updated based on the interaction trajectories collected in real time.

[0023] According to a second aspect of an embodiment of the present application, a device for automatically adjusting parameters of an adaptive optical system based on reinforcement learning is provided, comprising:

[0024] A data acquisition unit, configured to acquire relevant data, wherein the relevant data includes a wavefront sensor image, a wavefront slope, a turbulence intensity, a wavefront corrector voltage, and a far-field image;

[0025] A feature extraction unit for extracting features of the wavefront sensor image and the far-field image using a cross-attention mechanism;

[0026] a feature splicing unit, used to splice the extracted features with the wavefront slope, turbulence intensity and wavefront corrector voltage to form a current system state representation;

[0027] The action output unit is used to predict action parameters based on the current state using the reinforcement learning policy network;

[0028] The state prediction unit is used to combine the environment model, predict the next state based on the action of the policy network and the current system state, and calculate the corresponding reward value using the reward function;

[0029] The optimization unit is used to collect interaction trajectories to update the policy network and environment model, thereby achieving continuous optimization.

[0030] According to a third aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the method described in the first aspect above.

[0031] According to a fourth aspect of the embodiments of the present application, there is provided an electronic device, including: a memory and a processor;

[0032] The memory is used to store computer instructions;

[0033] The processor is configured to call the computer instructions stored in the memory so that the electronic device executes the method described in the first aspect above.

[0034] The beneficial effects of this application include:

[0035] The technical solution of this application collects data such as wavefront sensor images, wavefront slope, turbulence intensity, wavefront corrector voltage, and far-field images, and uses a cross-attention mechanism to extract image features to form an accurate representation of the system state. Based on this, in further inventive concepts, a reinforcement learning policy network can automatically predict and adjust AO system parameters, and continuously update the optimization strategy in combination with the environmental model and reward function, so that the system can continuously adapt to rapidly changing environments, significantly improving imaging quality under complex conditions and effectively reducing the need for manual intervention.

[0036] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, explaining the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0038] Figure 1 A schematic flow chart of a method for automatically adjusting parameters of an adaptive optical system based on reinforcement learning according to one embodiment of the present application is shown;

[0039] Figure 2 A schematic diagram illustrating a method for automatically adjusting parameters of an adaptive optical system based on reinforcement learning using specific modules according to an embodiment of the present application is shown;

[0040] Figure 3A block diagram of an apparatus for automatically adjusting parameters of an adaptive optical system based on reinforcement learning according to an embodiment of the present application is shown;

[0041] Figure 4 A structural block diagram of an electronic device according to an embodiment of the present application is shown.

[0042] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0044] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0045] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0046] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0047] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0048] Some implementation methods of the present application will be described in detail below with reference to the accompanying drawings.

[0049] To address the technical problems existing in the prior art, the present invention proposes a method for automatically adjusting the parameters of an adaptive optical system based on reinforcement learning. Specifically, to address the problem that manually adjusting the parameters of the AO system is unable to adapt to rapidly changing external environmental conditions, the present invention employs reinforcement learning to encode the sensor image, far-field image, slope, and other information of the AO system as states in a reinforcement learning environment. The reinforcement learning agent then outputs actions, including the operating frame rate, exposure time, camera gain, and number of restoration modes, thereby replacing manual adjustment of the AO system parameters and ensuring timely response to changes in the external environment.

[0050] See also Figure 1 , shows a flow chart of a method for automatically adjusting adaptive optical system parameters based on reinforcement learning according to an embodiment of the present application.

[0051] like Figure 1 As shown, a method for automatically adjusting parameters of an adaptive optical system based on reinforcement learning is presented, including steps S100 to S600.

[0052] In step S100 , the AO system observes the target and collects relevant data, including wavefront sensor images, slope, turbulence intensity, wavefront corrector voltage, and far-field images.

[0053] It can be understood that the target types of the observed targets are point sources, extended targets, including simulated KJ targets, solar grains, etc.

[0054] In some feasible embodiments, based on the aforementioned scheme, the AO system observes the target and collects relevant data, wherein the collected relevant data is used to comprehensively describe the current environment, including but not limited to wavefront sensor images, slope, turbulence intensity, wavefront corrector voltage and far-field images.

[0055] Continue to refer Figure 1 ,Step S200, extract features of the wavefront sensor image and the far-field image using the cross-attention mechanism.

[0056] It can be understood that in the embodiment of the present application, the cross-attention mechanism is used to extract the features of the wavefront sensor image and the far-field image, and the information of the wavefront sensor image and the far-field image can be fused to achieve more accurate modeling of the environmental state.

[0057] Continue to refer Figure 1 ,Step S300, the extracted features are spliced ​​with the wavefront slope, turbulence intensity and wavefront corrector voltage to form a representation of the current system state.

[0058] It should be noted that, in some embodiments, the extracted features are not necessarily spliced ​​with the wavefront slope, turbulence intensity and wavefront corrector voltage. Instead, other collected parameters that can reflect the system status are selected, which depends on the task requirements and specific implementation effects.

[0059] Continue to refer Figure 1 ,Step S400, uses the reinforcement learning policy network to predict a set of action parameters according to the current state, including the working frame rate and working gain of the wavefront sensor, the exposure time and gain of the imaging camera, and the number of restoration modes.

[0060] It can be understood that in the embodiment of the present application, the input of the policy network is the current state of the system, and the output is the action that should be taken at the moment.

[0061] It should be noted that, in some embodiments, the specific selected action parameters can be adjusted according to task requirements and are not limited to the working frame rate and working gain of the wavefront sensor, the exposure time, gain or number of restoration modes of the imaging camera.

[0062] Continue to refer Figure 1 ,Step S500,combining the environment model, predict the next state based on the action of the policy network and the current system state and calculate its corresponding reward value using the reward function.

[0063] In some embodiments of the present application, based on the aforementioned solution, combining the environment model, predicting the next state based on the actions of the policy network and the current system state, and calculating the corresponding reward value using the reward function includes:

[0064] Strehl ratio as a reward function.

[0065] It is defined in combination with other evaluation indicators, such as the signal-to-noise ratio score, using weighted or other calculation methods.

[0066] It can be understood that the environment model represents the system's state transition function P(s′|s,a). It can predict the next state corresponding to the state and action parameters without actually executing the operation, which can accelerate policy optimization while ensuring system security. The Strehl ratio is defined as the ratio of the peak light intensity of an actual optical system to the peak light intensity of an ideal diffraction-limited optical system and is used to measure the wavefront quality of an optical system.

[0067] It should be noted that in some embodiments, an environment model is not required; whether to use an environment model depends on the cost of actual interaction. When the cost of actual interaction is high, the environment model can be used for simulation; otherwise, the interaction can be directly based on the real environment. Furthermore, the reward function is not fixed. Its core role is to comprehensively evaluate the quality of actions to guide strategy optimization. In addition to the Strehl ratio, other evaluation metrics can also be combined, such as the signal-to-noise ratio score, and defined using weighted or other calculation methods.

[0068] Continue to refer Figure 1 ,Step S600, collects the complete interaction trajectory, including state, action and reward information, to update the policy network and environment model, thereby achieving continuous optimization.

[0069] It should be noted that, in the embodiment of the present application, by executing steps S100 to 500, the tuple (s t ,a t ,r t ,s t+1 ), where s t represents the state at time t, a t represents the action taken at time t, r t Represents the reward obtained at time t. By repeatedly executing steps S100 to S500 in chronological order, tuples at multiple moments can be continuously obtained, ultimately forming a complete interaction trajectory.

[0070] In some embodiments of the present application, based on the aforementioned solution, collecting complete interaction trajectories, including state, action, and reward information, to update the policy network and environment model, thereby achieving continuous optimization, includes:

[0071] Update using offline data: Use offline datasets constructed using historical observation data to pre-train the policy network and environment model.

[0072] Use online updates: In practice, the policy network and environment model are dynamically updated based on the interaction trajectories collected in real time.

[0073] In summary, the technical solution of this application proposes an adaptive control scheme for AO system parameters based on reinforcement learning. This scheme can automatically adjust the key parameters of the AO system in dynamic environments, improve the imaging quality of the system under complex conditions, and significantly reduce the need for manual intervention. At the same time, by combining the cross-attention mechanism with reinforcement learning, the present invention fully exploits the correlation between data, further improving the efficiency and accuracy of optimization.

[0074] Specifically, an example of implementing the method of the present application using a specific module is provided below.

[0075] See also Figure 2 , shows a schematic diagram of a method for automatically adjusting adaptive optical system parameters based on reinforcement learning using specific modules according to an embodiment of the present application.

[0076] like Figure 2 As shown, the collected relevant data is first state-encoded. The collected data includes but is not limited to wavefront sensor images, slopes, turbulence intensity, wavefront corrector voltages and far-field images. The state encoding module is used to encode the collected data to generate a state representation of the system. First, the wavefront sensor image and the far-field image are divided into small blocks (patches) to reduce the time complexity when calculating attention and ensure computational efficiency. Next, the cross-attention between the wavefront sensor image and the far-field image is calculated to extract their relevant feature information. Finally, the extracted image features are spliced ​​with non-image data such as slope, turbulence intensity, and wavefront corrector voltage to generate a state representation of the current environment, i.e., s t , for subsequent processing.

[0077] like Figure 2 As shown, the current state of the environment is s t There are three different uses. First, input it into the pre-set reward function to calculate the reward r of the current state t , used to evaluate the correction effect. Secondly, the state s t As a policy network module π(s t ) input, and get the output action a t , that is, the AO system parameters that need to be adjusted, including frame rate, gain, number of restoration modes, etc. Finally, the state s t With action a t The interaction and state update module, namely the environment model. The specific process can be expressed by the following formula:

[0078] s t+1 =f(s t ,a t )+∈

[0079] f(s t ,a t ) represents the environment model, whose input state s at time t t With action a t , predict the next state s t+1 . Where ∈ represents the random noise term, which is used to simulate the uncertainty in the real system.

[0080] Figure 2 The state s generated by the interaction t 、Action a t , reward r t and the next state s t+1A whole piece of data collected in continuous time is called a trajectory and used to update the policy network.

[0081] The following describes an embodiment of the device of the present application, which can be used to implement a method for automatically adjusting adaptive optical system parameters based on reinforcement learning in the above-mentioned embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0082] Reference Figure 3 As shown, according to an embodiment of the present application, a device 300 for automatically adjusting parameters of an adaptive optical system based on reinforcement learning includes: a data acquisition unit 301, a feature extraction unit 302, a feature splicing unit 303, an action output unit 304, a state prediction unit 305 and an optimization unit 306.

[0083] Among them, the data acquisition unit 301 is used to collect relevant data, including wavefront sensor images, slopes, turbulence intensity, wavefront corrector voltages and far-field images; the feature extraction unit 302 is used to extract features of the wavefront sensor images and far-field images using the cross-attention mechanism; the feature splicing unit 303 is used to splice the extracted features with the wavefront slope, turbulence intensity and wavefront corrector voltage to form a representation of the current system state; the action output unit 304 is used to use the reinforcement learning policy network to predict a set of action parameters based on the current state, including the working frame rate and working gain of the wavefront sensor, the exposure time and gain of the imaging camera, and the number of restoration modes; the state prediction unit 305 is used to combine the environment model, predict the next state based on the action of the policy network and the current system state, and calculate its corresponding reward value using the reward function; the optimization unit 306 is used to collect the complete interaction trajectory, including state, action and reward information, to update the policy network and the environment model, thereby achieving continuous optimization.

[0084] Reference Figure 4 As shown, an embodiment of the present application further provides an electronic device 400, comprising a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor. When the processor 420 executes the computer program 411, the steps of any one of the above-mentioned methods for automatically adjusting parameters of an adaptive optical system based on reinforcement learning are implemented.

[0085] Since the electronic device introduced in this embodiment is a device used to implement an apparatus for automatically adjusting the parameters of an adaptive optical system based on reinforcement learning in an embodiment of the present application, based on the method introduced in the embodiment of the present application, technical personnel in this field can understand the specific implementation of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present application will not be introduced in detail here. As long as the equipment used by technical personnel in this field to implement the method in the embodiment of the present application falls within the scope of protection to be protected by this application.

[0086] During the specific implementation process, when the computer program 411 is executed by the processor, any implementation method in the embodiments corresponding to the first aspect can be implemented.

[0087] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown.

[0088] It should be noted that Figure 5 The computer system 500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0089] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503, such as executing the method described in the above embodiment. Various programs and data required for system operation are also stored in the RAM 503. The CPU 501, ROM 502 and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0090] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. Removable media 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read therefrom can be installed into the storage section 508 as needed.

[0091] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 509, and / or installed from a removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the various functions defined in the system of the present application are executed.

[0092] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0094] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0095] As another aspect, the present application further provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for automatically adjusting adaptive optical system parameters based on reinforcement learning as described in the above embodiment.

[0096] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs, and when executed by the electronic device, the electronic device implements the method for automatically adjusting adaptive optical system parameters based on reinforcement learning described in the above embodiments.

[0097] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0098] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0099] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in this application. It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of this application is limited only by the appended claims.

Claims

1. A method for automatically adjusting parameters of an adaptive optical system based on reinforcement learning, characterized in that: Including steps: observing a target and collecting relevant data, wherein the relevant data includes a wavefront sensor image, a wavefront slope, a turbulence intensity, a wavefront corrector voltage, and a far-field image; The wavefront sensor image and far-field image are divided into small blocks, and the features of the wavefront sensor image and far-field image are extracted using the cross-attention mechanism; The extracted features are concatenated with the wavefront slope, turbulence intensity and wavefront corrector voltage to form a representation of the current system state; A policy network using reinforcement learning predicts action parameters based on the current state; Combined with the environment model, the next state is predicted based on the action of the policy network and the current system state, and the corresponding reward value is calculated using the reward function; Collect interaction trajectories to update the policy network and environment model, thereby achieving continuous optimization.

2. The method according to claim 1, characterized in that The observation of targets and collection of relevant data are achieved through the AO system, wherein the collected relevant data are used to comprehensively describe the current environment.

3. The method according to claim 2, characterized in that The action parameters represent all parameters required for the AO system control process, and include the operating frame rate and operating gain of the wavefront sensor, and the exposure time, gain, and number of restoration modes of the imaging camera.

4. The method according to claim 1, wherein Calculating the corresponding reward value using the reward function includes the following sub-steps: The Strehl ratio is used as the reward function and is combined with the selected evaluation metric and defined in the selected calculation method.

5. The method according to claim 4, characterized in that The interaction trajectory includes state, action and reward information.

6. The method according to claim 5, characterized in that The updating strategy network and environment model specifically includes the following sub-steps: Update using offline data: Use offline datasets constructed using historical observation data to pre-train the policy network and environment model; Use online updates: In practice, the policy network and environment model are dynamically updated based on the interaction trajectories collected in real time.

7. A device for automatically adjusting parameters of an adaptive optical system based on reinforcement learning, characterized in that: include: A data acquisition unit, configured to acquire relevant data, wherein the relevant data includes a wavefront sensor image, a wavefront slope, a turbulence intensity, a wavefront corrector voltage, and a far-field image; A feature extraction unit, configured to divide the wavefront sensor image and the far-field image into small blocks and extract features of the wavefront sensor image and the far-field image using a cross-attention mechanism; a feature splicing unit, used to splice the extracted features with the wavefront slope, turbulence intensity and wavefront corrector voltage to form a current system state representation; The action output unit is used to predict action parameters based on the current state using the reinforcement learning policy network; The state prediction unit is used to combine the environment model, predict the next state based on the action of the policy network and the current system state, and calculate the corresponding reward value using the reward function; The optimization unit is used to collect interaction trajectories to update the policy network and environment model, thereby achieving continuous optimization.

8. A computer-readable storage medium, characterized in that The storage medium stores computer instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: Including: memory and processor; The memory is used to store computer instructions; The processor is configured to call the computer instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 6.

Images