Anti-scattering electromagnetic topological structure dynamic optimization method and device based on deep reinforcement learning, and storage medium

Abstract

The invention relates to the technical field of artificial electromagnetic structures, and particularly discloses an anti-scattering electromagnetic topological structure dynamic optimization method and device based on deep reinforcement learning and a storage medium. The method comprises the following steps: carrying out digital representation on physical parameters of an electromagnetic topological structure to obtain an electromagnetic topological parameterized model; obtaining electromagnetic field real-time response data of the electromagnetic topology parameterized model; designing an action space and a reward mechanism; constructing a deep reinforcement learning network by constructing a near-end strategy optimization algorithm framework; implementing offline pre-training of the deep reinforcement learning network based on the electromagnetic field real-time response data; and after offline pre-training is completed, electromagnetic field real-time response data of the electromagnetic topology parameterized model to be optimized are accessed to an input interface of the deep reinforcement learning network, and dynamic optimization is executed.

Description

Technical Field

[0001] The present application relates to the technical field of artificial electromagnetic structures, and specifically to a method, device and storage medium for dynamic optimization of anti-scattering electromagnetic topological structures based on deep reinforcement learning. Background Art

[0002] With the rapid development of sixth-generation communication technology (6G) and terahertz imaging, artificial electromagnetic structures (such as metasurfaces and photonic crystals) are playing an increasingly important role in electromagnetic wave manipulation. Traditional electromagnetic topology optimization methods, which rely primarily on finite element simulation and empirical trial-and-error design, suffer from inability to adapt to scattering loss fluctuations caused by dynamic environments, low computational efficiency, and significant deviations between simulation optimization results and actual device performance. Summary of the Invention

[0003] The purpose of this application is to provide a dynamic optimization method, device and storage medium for anti-scattering electromagnetic topology structures based on deep reinforcement learning, so as to solve the problems that the electromagnetic topology optimization methods in the prior art mainly rely on finite element simulation and empirical trial and error design, and are unable to adapt to the scattering loss fluctuations caused by dynamic environments, have low computational efficiency, and have large deviations between simulation optimization results and actual processing device performance.

[0004] To achieve the above objectives, the present invention provides a method for dynamic optimization of scattering-resistant electromagnetic topology structures based on deep reinforcement learning, comprising:

[0005] Digitally characterizing the physical parameters of the electromagnetic topology structure to obtain an electromagnetic topology parameterized model, and obtaining electromagnetic field real-time response data of the electromagnetic topology parameterized model;

[0006] Design action space and reward mechanism;

[0007] Build a deep reinforcement learning network by building a proximal strategy optimization algorithm framework;

[0008] Implementing offline pre-training of the deep reinforcement learning network based on the real-time electromagnetic field response data;

[0009] After the offline pre-training is completed, the real-time electromagnetic field response data of the electromagnetic topology parameterized model to be optimized is connected to the input interface of the deep reinforcement learning network and dynamic optimization is performed.

[0010] Optionally, the digitally characterizing the physical parameters of the electromagnetic topology structure to obtain an electromagnetic topology parameterized model, and obtaining the electromagnetic field real-time response data of the electromagnetic topology parameterized model, specifically includes:

[0011] The shape, size, and spatial arrangement period of the metasurface unit are defined as multidimensional vectors, where the shape of the metasurface unit is quantified by geometric parameters, and the arrangement period is described by asymmetric angles and spacing parameters, thereby obtaining an electromagnetic topology parameterized model.

[0012] Acquiring and integrating electromagnetic field real-time response data from the electromagnetic topology parameterized model using electromagnetic simulation software, wherein the electromagnetic field real-time response data includes transmission efficiency, scattering loss value, and electric field distribution characteristics;

[0013] A state vector is constructed based on the real-time response data of the electromagnetic field.

[0014] Optionally, the design of the action space and reward mechanism specifically includes:

[0015] An executable action is defined as a set of discrete operation instructions, wherein each executable action corresponds to the adjustment of a physical parameter of an electromagnetic topological structure;

[0016] Establish reward function: R = 0.7Δη-0.2ΔL-0.1‖ΔT‖ 2 , where Δη is the change in transmission efficiency of the electromagnetic topology parameterized model, ΔL is the increment of scattering loss, ‖ΔT‖ 2 is the topology parameter mutation gradient penalty term.

[0017] Optionally, the building of a proximal strategy optimization algorithm framework specifically includes:

[0018] Design the Actor network and Critic network: The Actor network uses a three-layer fully connected neural network, the Critic network shares the input layer and hidden layer, and the output layer is the state value scalar;

[0019] Configure the experience replay buffer to store historical interaction data, and use priority replay as the sampling strategy.

[0020] Optionally, the offline pre-training of the deep reinforcement learning network based on the real-time electromagnetic field response data specifically includes:

[0021] Randomly generating a combination of physical parameters of an electromagnetic topological structure, inputting the combination of physical parameters of the electromagnetic topological structure into electromagnetic simulation software, obtaining corresponding real-time electromagnetic field response data from the electromagnetic topological parameterized model, forming an initial training data set based on the combination of physical parameters of the electromagnetic topological structure and the corresponding real-time electromagnetic field response data, and calculating its transmission efficiency and scattering loss through electromagnetic simulation;

[0022] Using the initial training data set and reward function, the Actor network and the Critic network are trained to a convergence standard, so that the Actor network output policy entropy is stabilized below 0.2 and the Critic network prediction error is less than 5%.

[0023] Optionally, the step of connecting the real-time electromagnetic field response data of the electromagnetic topology parameterized model to be optimized to an input interface of the deep reinforcement learning network and performing dynamic optimization specifically includes:

[0024] Obtain the current electromagnetic state of the electromagnetic topology parameterized model through a near-field scanning probe or simulation interface;

[0025] Preprocessing the current electromagnetic state and obtaining a corresponding state vector, wherein the preprocessing includes normalization operation and band-limited filtering;

[0026] Perform dynamic topology adjustments and anti-scattering optimization.

[0027] Optionally, performing dynamic topology adjustment and anti-scattering optimization specifically includes:

[0028] Inputting the state vector corresponding to the current electromagnetic state into the trained Actor network to obtain the output action instruction;

[0029] According to the action instruction, an adjustment operation of the physical parameters of the corresponding electromagnetic topological structure is selected from the discretized operation instruction set to drive the actuator to adjust the parameters of the metasurface unit;

[0030] The transmission efficiency of the adjusted electromagnetic topology parameterized model is monitored, and if the transmission efficiency is less than a preset efficiency threshold, a re-optimization process is triggered.

[0031] Optionally, it also includes:

[0032] Deploying the trained deep reinforcement learning network to an embedded hardware platform;

[0033] Specifically include:

[0034] The trained deep reinforcement learning network is lightweight processed and integrated into an FPGA or edge computing module, and connected to a topology adjustment actuator via the SPI / I2C protocol.

[0035] To achieve the above-mentioned purpose, the present application also provides a dynamic optimization device for anti-scattering electromagnetic topology structure based on deep reinforcement learning, comprising: a memory; and

[0036] A processor connected to the memory, wherein the processor is configured to execute the steps of the method described above.

[0037] To achieve the above objectives, the present application also provides a computer storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a machine.

[0038] The embodiments of the present application have the following advantages:

[0039] The deep reinforcement learning (DRL) framework is used to perceive electromagnetic field response data in real time. This is combined with a discrete operation instruction set to drive topology parameter adjustment. A multi-objective constraint mechanism for the reward function is introduced to improve transmission efficiency.

[0040] By adopting a lightweight DRL model and a priority experience replay strategy, the optimization speed is improved compared to the genetic algorithm under the same hardware conditions;

[0041] Discrete operation instruction sets are directly mapped to industrial control signals and are compatible with standard actuators such as MEMS drivers and piezoelectric ceramics, reducing system modification costs.

[0042] The state vector integrates the manufacturing error compensation mechanism to reduce the performance deviation caused by processing errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] To more clearly illustrate the embodiments of this application or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely illustrative, and those skilled in the art can, without inventive effort, derive other implementation drawings based on the provided drawings.

[0044] Figure 1 A flowchart of a method for dynamic optimization of scattering-resistant electromagnetic topology structures based on deep reinforcement learning provided in at least one embodiment of the present application;

[0045] Figure 2 A block diagram of a module for a dynamic optimization device for anti-scattering electromagnetic topology structures based on deep reinforcement learning, provided in at least one embodiment of the present application. DETAILED DESCRIPTION

[0046] The following specific embodiments illustrate the implementation of this application. Those familiar with the art can easily understand the other advantages and functions of this application from the content disclosed in this specification. Obviously, the described embodiments are part of the embodiments of this application, but not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0047] It should be noted that, in the claims and description of this application, the steps may be executed substantially in parallel or in reverse order under appropriate circumstances, depending on the functions involved.

[0048] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0049] An embodiment of the present application provides a method for dynamic optimization of anti-scattering electromagnetic topology structure based on deep reinforcement learning, referring to Figure 1 , Figure 1 A flowchart of a method for dynamic optimization of scattering-resistant electromagnetic topological structures based on deep reinforcement learning is provided in at least one embodiment of the present application. It should be understood that the method may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of the present application is not limited in this respect.

[0050] In step 101 , physical parameters of an electromagnetic topology structure are digitally represented to obtain an electromagnetic topology parameterized model, and real-time electromagnetic field response data of the electromagnetic topology parameterized model is obtained.

[0051] In some embodiments, the digital representation of the physical parameters of the electromagnetic topology structure to obtain an electromagnetic topology parameterized model and obtaining real-time electromagnetic field response data of the electromagnetic topology parameterized model specifically includes:

[0052] The shape, size, and spatial arrangement period of the metasurface unit are defined as multidimensional vectors. The shape of the metasurface unit is quantified by geometric parameters (such as the inner and outer diameters of the open ring and the metal line width), and the arrangement period is described by asymmetric angles and spacing parameters, thus obtaining an electromagnetic topology parameterized model.

[0053] Acquiring and integrating electromagnetic field real-time response data from the electromagnetic topology parameterized model using electromagnetic simulation software (e.g., ANSYS HFSS or CST Microwave Studio), wherein the electromagnetic field real-time response data includes transmission efficiency, scattering loss value, and electric field distribution characteristics;

[0054] Based on the real-time response data of the electromagnetic field, a state vector S=[s1,s2,…,s n ], where s i Represents the physical parameters of the i-th electromagnetic topological structure or the real-time response data of the electromagnetic field (for example, s1 represents the size of the metasurface unit, and s5 represents the transmission efficiency of the current frequency band), 1≤i≤n.

[0055] Specifically, the state vector consists of two parts: the physical parameters of the electromagnetic topology (such as unit size and arrangement period, etc., from the parameterized model) and the real-time response data of the electromagnetic field (such as transmission efficiency, scattering loss value, and field distribution characteristics, obtained through simulation or experimental measurement). For example, if the size of the current metasurface unit is 12μm, the arrangement angle is 30°, and the measured transmission efficiency is 85%, then the state vector is: S = [12, 30, 85].

[0056] At step 102 , the action space and reward mechanism are designed.

[0057] In some embodiments, the design of the action space and reward mechanism specifically includes:

[0058] Define executable actions as a discrete set of operation instructions A = {a1, a2, ..., a m}, where each executable action corresponds to the adjustment of a physical parameter of the electromagnetic topology (e.g., a1 represents increasing the size of the kth metasurface unit, a3 represents rotating the arrangement direction by 10°);

[0059] Establish reward function: R = 0.7Δη-0.2ΔL-0.1‖ΔT‖ 2 , where Δη is the change in transmission efficiency of the electromagnetic topology parameterized model, ΔL is the increment of scattering loss, ‖ΔT‖ 2 is the topology parameter mutation gradient penalty term.

[0060] In step 103, a deep reinforcement learning network is constructed by building a proximal policy optimization (PPO) algorithm framework.

[0061] In some embodiments, building a proximal strategy optimization algorithm framework specifically includes:

[0062] Design of the Actor and Critic networks: The Actor network uses a three-layer fully connected neural network (input layer with 256 nodes, hidden layer with 512 nodes, and output layer with action probability distribution). The Critic network shares the input layer and hidden layer, and the output layer is a state value scalar.

[0063] Configure the experience replay buffer to store historical interaction data, and use the Prioritized Experience Replay sampling strategy.

[0064] Specifically, the Actor Network and the Critic Network together form the core components of the PPO algorithm, which maximizes the cumulative reward through the interaction between the Actor and the Critic. Each action output by the Actor Network corresponds to a specific instruction in the discretized operation instruction set.

[0065] At step 104 , offline pre-training of the deep reinforcement learning network is performed based on the real-time electromagnetic field response data.

[0066] In some embodiments, the offline pre-training of the deep reinforcement learning network based on the real-time electromagnetic field response data specifically includes:

[0067] Randomly generating a combination of physical parameters of an electromagnetic topological structure, inputting the combination of physical parameters of the electromagnetic topological structure into electromagnetic simulation software, obtaining corresponding real-time electromagnetic field response data from the electromagnetic topological parameterized model, forming an initial training data set based on the combination of physical parameters of the electromagnetic topological structure and the corresponding real-time electromagnetic field response data, and calculating its transmission efficiency and scattering loss through electromagnetic simulation;

[0068] Using the initial training data set and reward function, the Actor network and the Critic network are trained to a convergence standard, so that the Actor network output policy entropy is stabilized below 0.2 and the Critic network prediction error is less than 5%.

[0069] At step 105 , after the offline pre-training is completed, the real-time electromagnetic field response data of the electromagnetic topology parameterized model to be optimized is connected to the input interface of the deep reinforcement learning network and dynamic optimization is performed.

[0070] In some embodiments, the step of connecting the real-time electromagnetic field response data of the electromagnetic topology parameterized model to be optimized to the input interface of the deep reinforcement learning network and performing dynamic optimization specifically includes:

[0071] Obtain the current electromagnetic state of the electromagnetic topology parameterized model through a near-field scanning probe or simulation interface;

[0072] Preprocessing the current electromagnetic state and obtaining the corresponding state vector. The preprocessing includes normalization (mapping the parameters to the interval [0, 1]) and band-limited filtering (the cutoff frequency is twice the operating frequency band).

[0073] Perform dynamic topology adjustments and anti-scattering optimization.

[0074] In some embodiments, performing dynamic topology adjustment and anti-scattering optimization specifically includes:

[0075] Inputting the state vector corresponding to the current electromagnetic state into the trained Actor network to obtain the output action instruction;

[0076] According to the action instruction, an adjustment operation of the physical parameters of the corresponding electromagnetic topological structure is selected from the discretized operation instruction set, and an actuator (such as a piezoelectric ceramic driver or a MEMS micromirror array) is driven to adjust the parameters of the metasurface unit (for example, if the model outputs action a3 (rotating the arrangement direction by 10°), the motor is controlled to rotate the metasurface unit to the target angle);

[0077] The transmission efficiency of the adjusted electromagnetic topology parameterized model is monitored, and if the transmission efficiency is less than a preset efficiency threshold, a re-optimization process is triggered.

[0078] In some embodiments, further comprising:

[0079] Deploy the trained deep reinforcement learning network to an embedded hardware platform.

[0080] In some embodiments, deploying the trained deep reinforcement learning network to an embedded hardware platform specifically includes: performing lightweight processing on the trained deep reinforcement learning network, specifically including quantizing it to an 8-bit integer (INT8) using a TensorFlow Lite framework, and compressing the model size to less than 1MB;

[0081] It is integrated into FPGA (such as Xilinx Zynq UltraScale+) or edge computing module (such as NVIDIA Jetson AGX Xavier), and connected to the topology adjustment actuator via SPI / I2C protocol.

[0082] Specifically, in the embedded hardware platform, the discretized instructions are converted into hardware control signals (such as PWM waves and digital IO signals) to ensure that the algorithm instructions are compatible with the physical interface of the actuator.

[0083] In some embodiments, it also includes: performing multi-scenario performance verification.

[0084] Specifically, laboratory tests: Measure the equivalent isotropic radiated power (EIRP) and sidelobe suppression ratio of the optimized structure in a microwave anechoic chamber, and compare the reduction in scattering loss before and after optimization;

[0085] Industrial environment testing: Verifying dynamic anti-scattering capabilities on a 6G base station prototype, recording the transmission efficiency improvement and algorithm response delay.

[0086] Figure 2 A block diagram of a device for dynamic optimization of scatter-resistant electromagnetic topology structures based on deep reinforcement learning, provided in at least one embodiment of the present application. The device includes:

[0087] A memory 201; and a processor 202 connected to the memory 201, the processor 202 being configured to: digitally characterize the physical parameters of the electromagnetic topology structure to obtain an electromagnetic topology parameterized model, and obtain real-time electromagnetic field response data of the electromagnetic topology parameterized model;

[0088] Design action space and reward mechanism;

[0089] Build a deep reinforcement learning network by building a proximal strategy optimization algorithm framework;

[0090] Implementing offline pre-training of the deep reinforcement learning network based on the real-time electromagnetic field response data;

[0091] After the offline pre-training is completed, the real-time electromagnetic field response data of the electromagnetic topology parameterized model to be optimized is connected to the input interface of the deep reinforcement learning network and dynamic optimization is performed.

[0092] In some embodiments, the processor 202 is further configured to: digitally represent the physical parameters of the electromagnetic topology structure to obtain an electromagnetic topology parameterized model, and obtain real-time electromagnetic field response data of the electromagnetic topology parameterized model, specifically including:

[0093] The shape, size, and spatial arrangement period of the metasurface unit are defined as multidimensional vectors, where the shape of the metasurface unit is quantified by geometric parameters, and the arrangement period is described by asymmetric angles and spacing parameters, thereby obtaining an electromagnetic topology parameterized model.

[0094] Acquiring and integrating electromagnetic field real-time response data from the electromagnetic topology parameterized model using electromagnetic simulation software, wherein the electromagnetic field real-time response data includes transmission efficiency, scattering loss value, and electric field distribution characteristics;

[0095] A state vector is constructed based on the real-time response data of the electromagnetic field.

[0096] In some embodiments, the processor 202 is further configured to: design an action space and a reward mechanism, specifically including:

[0097] An executable action is defined as a set of discrete operation instructions, wherein each executable action corresponds to the adjustment of a physical parameter of an electromagnetic topological structure;

[0098] Establish reward function: R = 0.7Δη-0.2ΔL-0.1‖ΔT‖ 2 , where Δη is the change in transmission efficiency of the electromagnetic topology parameterized model, ΔL is the increment of scattering loss, ‖ΔT‖ 2 is the topology parameter mutation gradient penalty term.

[0099] In some embodiments, the processor 202 is further configured to: build a proximal strategy optimization algorithm framework, specifically including:

[0100] Design the Actor network and Critic network: The Actor network uses a three-layer fully connected neural network, the Critic network shares the input layer and hidden layer, and the output layer is the state value scalar;

[0101] Configure the experience replay buffer to store historical interaction data, and use priority replay as the sampling strategy.

[0102] In some embodiments, the processor 202 is further configured to: implement offline pre-training of a deep reinforcement learning network based on real-time electromagnetic field response data, specifically including:

[0103] Randomly generating a combination of physical parameters of an electromagnetic topological structure, inputting the combination of physical parameters of the electromagnetic topological structure into electromagnetic simulation software, obtaining corresponding real-time electromagnetic field response data from the electromagnetic topological parameterized model, forming an initial training data set based on the combination of physical parameters of the electromagnetic topological structure and the corresponding real-time electromagnetic field response data, and calculating its transmission efficiency and scattering loss through electromagnetic simulation;

[0104] Using the initial training dataset and reward function, the Actor network and the Critic network are trained to convergence criteria.

[0105] In some embodiments, the processor 202 is further configured to: connect the real-time electromagnetic field response data of the electromagnetic topology parameterized model to be optimized to the input interface of the deep reinforcement learning network and perform dynamic optimization, specifically including:

[0106] Obtain the current electromagnetic state of the electromagnetic topology parameterized model through a near-field scanning probe or simulation interface;

[0107] Preprocessing the current electromagnetic state and obtaining a corresponding state vector, wherein the preprocessing includes normalization operation and band-limited filtering;

[0108] Perform dynamic topology adjustments and anti-scattering optimization.

[0109] In some embodiments, the processor 202 is further configured to: perform dynamic topology adjustment and anti-scattering optimization, specifically including:

[0110] Inputting the state vector corresponding to the current electromagnetic state into the trained Actor network to obtain the output action instruction;

[0111] According to the action instruction, an adjustment operation of the physical parameters of the corresponding electromagnetic topological structure is selected from the discretized operation instruction set to drive the actuator to adjust the parameters of the metasurface unit;

[0112] The transmission efficiency of the adjusted electromagnetic topology parameterized model is monitored, and if the transmission efficiency is less than a preset efficiency threshold, a re-optimization process is triggered.

[0113] In some embodiments, the processor 202 is further configured to:

[0114] Deploying the trained deep reinforcement learning network to an embedded hardware platform;

[0115] Specifically include:

[0116] The trained deep reinforcement learning network is lightweight processed and integrated into an FPGA or edge computing module, and connected to a topology adjustment actuator via the SPI / I2C protocol.

[0117] The specific implementation method is referred to the aforementioned method embodiment and will not be repeated here.

[0118] The present application may be a method, apparatus, system and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present application.

[0119] Computer-readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device.Computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device or any suitable combination thereof.More specific examples (non-exhaustive list) of computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove having instructions stored thereon, and any suitable combination thereof.Computer-readable storage medium used herein is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (for example, light pulses by fiber optic cables), or electrical signals transmitted by wires.

[0120] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0121] The computer program instructions for performing the operation of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data or source code or object code written in any combination of one or more programming languages, wherein the programming language includes object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or executed completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer by any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (such as by using an Internet service provider to connect to the Internet). In certain embodiments, by utilizing the state information of computer-readable program instructions to personalize electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs), the electronic circuits can execute computer-readable program instructions, thereby realizing various aspects of the present application.

[0122] Various aspects of the present application are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0123] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0124] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0125] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a special hardware-based system that performs the function or action of the specification, or can be implemented by a combination of special hardware and computer instructions.

[0126] Note that, unless otherwise directly stated, all features disclosed in this specification (including any accompanying claims, abstracts and drawings) may be replaced by alternative features for achieving the same, equivalent or similar purposes. Therefore, unless otherwise explicitly stated, each feature disclosed is only an example of a group of equivalent or similar features. Where used, further, preferably, further and more preferably are a simple starting point for elaborating another embodiment based on the aforementioned embodiment, and the content of the further, preferably, further or more preferably followed by the above embodiment is combined with the aforementioned embodiment as a complete composition of another embodiment. Several further, preferably, further or more preferably settings following the same embodiment can be arbitrarily combined to form another embodiment.

[0127] Although the present application has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications or improvements may be made based on the present application. Therefore, such modifications or improvements, which do not depart from the spirit of the present application, are within the scope of protection claimed in the present application.

Claims

1. A dynamic optimization method for anti-scattering electromagnetic topology structure based on deep reinforcement learning, characterized in that: include: Digitally characterizing the physical parameters of the electromagnetic topology structure to obtain an electromagnetic topology parameterized model, and obtaining electromagnetic field real-time response data of the electromagnetic topology parameterized model; Design action space and reward mechanism; Build a deep reinforcement learning network by building a proximal strategy optimization algorithm framework; Implementing offline pre-training of the deep reinforcement learning network based on the real-time electromagnetic field response data; After the offline pre-training is completed, the real-time electromagnetic field response data of the electromagnetic topology parameterized model to be optimized is connected to the input interface of the deep reinforcement learning network and dynamic optimization is performed.

2. The method for dynamic optimization of anti-scattering electromagnetic topology structure based on deep reinforcement learning according to claim 1, characterized in that: The method of digitally representing the physical parameters of the electromagnetic topology structure to obtain an electromagnetic topology parameterized model and obtaining the electromagnetic field real-time response data of the electromagnetic topology parameterized model specifically includes: The shape, size, and spatial arrangement period of the metasurface unit are defined as multidimensional vectors, where the shape of the metasurface unit is quantified by geometric parameters, and the arrangement period is described by asymmetric angles and spacing parameters, thereby obtaining an electromagnetic topology parameterized model. Acquiring and integrating electromagnetic field real-time response data from the electromagnetic topology parameterized model using electromagnetic simulation software, wherein the electromagnetic field real-time response data includes transmission efficiency, scattering loss value, and electric field distribution characteristics; A state vector is constructed based on the real-time response data of the electromagnetic field.

3. The method for dynamic optimization of anti-scattering electromagnetic topology structure based on deep reinforcement learning according to claim 2, characterized in that: The designed action space and reward mechanism specifically include: An executable action is defined as a set of discrete operation instructions, wherein each executable action corresponds to the adjustment of a physical parameter of an electromagnetic topological structure; Establish reward function: R = 0.7Δη-0.2ΔL-0.1‖ΔT‖ 2 , where Δη is the change in transmission efficiency of the electromagnetic topology parameterized model, ΔL is the increment of scattering loss, ‖ΔT‖ 2 is the topology parameter mutation gradient penalty term.

4. The method for dynamic optimization of anti-scattering electromagnetic topology structure based on deep reinforcement learning according to claim 3, characterized in that: The construction of the proximal strategy optimization algorithm framework specifically includes: Design the Actor network and Critic network: The Actor network uses a three-layer fully connected neural network, the Critic network shares the input layer and hidden layer, and the output layer is the state value scalar; Configure the experience replay buffer to store historical interaction data, and use priority replay as the sampling strategy.

5. The method for dynamic optimization of anti-scattering electromagnetic topology structure based on deep reinforcement learning according to claim 4, characterized in that: The offline pre-training of the deep reinforcement learning network based on the real-time electromagnetic field response data specifically includes: Randomly generating a combination of physical parameters of an electromagnetic topological structure, inputting the combination of physical parameters of the electromagnetic topological structure into electromagnetic simulation software, obtaining corresponding real-time electromagnetic field response data from the electromagnetic topological parameterized model, forming an initial training data set based on the combination of physical parameters of the electromagnetic topological structure and the corresponding real-time electromagnetic field response data, and calculating its transmission efficiency and scattering loss through electromagnetic simulation; Using the initial training dataset and reward function, the Actor network and the Critic network are trained to convergence criteria.

6. The method for dynamic optimization of anti-scattering electromagnetic topology structure based on deep reinforcement learning according to claim 5, characterized in that: The step of connecting the real-time electromagnetic field response data of the electromagnetic topology parameterized model to be optimized to the input interface of the deep reinforcement learning network and performing dynamic optimization specifically includes: Obtain the current electromagnetic state of the electromagnetic topology parameterized model through a near-field scanning probe or simulation interface; Preprocessing the current electromagnetic state and obtaining a corresponding state vector, wherein the preprocessing includes normalization operation and band-limited filtering; Perform dynamic topology adjustments and anti-scattering optimization.

7. The method for dynamic optimization of anti-scattering electromagnetic topology structure based on deep reinforcement learning according to claim 6, characterized in that: The performing of dynamic topology adjustment and anti-scattering optimization specifically includes: Inputting the state vector corresponding to the current electromagnetic state into the trained Actor network to obtain the output action instruction; According to the action instruction, an adjustment operation of the physical parameters of the corresponding electromagnetic topological structure is selected from the discretized operation instruction set to drive the actuator to adjust the parameters of the metasurface unit; The transmission efficiency of the adjusted electromagnetic topology parameterized model is monitored, and if the transmission efficiency is less than a preset efficiency threshold, a re-optimization process is triggered.

8. The method for dynamic optimization of anti-scattering electromagnetic topology structure based on deep reinforcement learning according to claim 1, characterized in that: Also includes: Deploying the trained deep reinforcement learning network to an embedded hardware platform; Specifically include: The trained deep reinforcement learning network is lightweight processed and integrated into an FPGA or edge computing module, and connected to a topology adjustment actuator via the SPI / I2C protocol.

9. A dynamic optimization device for anti-scattering electromagnetic topology structure based on deep reinforcement learning, characterized in that: include: Memory; as well as A processor connected to the memory, the processor being configured to perform the steps of the method according to any one of claims 1 to 8.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a machine, the steps of the method according to any one of claims 1 to 8 are implemented.

Images