Micro-nano structure optimization design method based on deep reinforcement learning

Abstract

The invention relates to the technical field of reinforcement learning, and further relates to a micro-nano structure optimization design method based on deep reinforcement learning. The method comprises the following steps: step 1, obtaining electron density distribution of a micro-nano structure; calculating the quantum tunneling current density according to the barrier height and the barrier width of the micro-nano structure; 2, mapping the quantum wave state function, the quantum stress field and the quantum tunneling current density to a state space of reinforcement learning to obtain a state vector of reinforcement learning; generating an optimization action based on the state vector; parameters of the micro-nano structure are updated according to the optimization action; calculating the structural performance of the micro-nano structure after the parameters of the micro-nano structure are updated; and 3, based on the change rate of the structural performance, establishing a convergence criterion based on Lyapunov stability, judging whether convergence occurs, if yes, terminating and finishing the optimization design, and if not, returning to the step 2. According to the method, the optimization precision and efficiency are remarkably improved, and meanwhile, the dynamic adjustment and global search capabilities are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of reinforcement learning, and particularly relates to a method for optimizing the design of micro-nano structures based on deep reinforcement learning. Background Art

[0002] With the rapid development of micro-nano technology, micro-nano structures are increasingly widely used in the fields of electronics, optics, energy, and biology. The design of micro-nano structures directly determines the performance of devices, such as the quantum tunneling efficiency of tunneling diodes, the optoelectronic conversion performance of quantum dot devices, and the transmission characteristics of heterojunction interfaces. However, the existing optimization design methods for micro-nano structures mostly rely on traditional physical modeling and numerical simulation. Although these methods have achieved performance optimization to a certain extent, there are still limitations. With the increasing application requirements, the parameter complexity, coupling characteristics, and diverse performance objectives in the design of micro-nano structures urgently require a more intelligent optimization means.

[0003] Currently, the design optimization of micro-nano structures mainly relies on the following methods: Traditional optimization methods such as gradient descent, genetic algorithms, and particle swarm optimization algorithms are widely used in the design of micro-nano structures. For example, in quantum tunneling devices, researchers usually construct a mathematical model of the barrier height and width, and gradually adjust the parameters through the gradient descent method based on the objective function (such as maximizing the tunneling current) to find the local optimal solution. Such methods have improved the design efficiency to a certain extent by introducing a parameter space search strategy. However, these methods are highly sensitive to the initial conditions and the form of the objective function, are prone to falling into local optima in high-dimensional parameter spaces, and lack the ability to handle multi-objective optimization. For example, when simultaneously optimizing the tunneling current and stress distribution, classical algorithms often struggle to find the optimal balance between the two. Finite element analysis is an important tool for designing micro-nano structures. By constructing mechanical, electrical, or thermal models of the structure, its performance can be simulated. For example, in the mechanical optimization of heterogeneous interfaces, the finite element method can be used to simulate the interface stress distribution and analyze its impact on the structural stability. However, finite element analysis is usually based on classical physical assumptions and ignores the significant quantum effects in micro-nano structures. For example, at the nanoscale, the electron transport properties are significantly affected by quantum tunneling, quantum interference, and surface effects, which are usually not fully reflected in the classical finite element framework. In addition, the computational complexity of finite element analysis is relatively high, making it difficult to adapt to large-scale optimization tasks that require frequent iterations. In recent years, the application of machine learning in the design of micro-nano structures has gradually increased. By constructing a high-throughput screening database of materials and training deep learning models, researchers can quickly predict the performance of micro-nano structures. For example, neural network-based models can be used to predict the thermal conductivity or electron mobility of materials. However, existing machine learning methods are usually limited to performance prediction and lack an effective optimization feedback mechanism. Most models require a large amount of data support during the training phase, but in actual optimization design, they cannot flexibly adjust the model parameters to meet new target requirements. In addition, the "black box" nature of machine learning models limits their application in multi-physical field coupling problems and makes it difficult to accurately reflect the quantum effects and thermodynamic properties in the design of micro-nano structures. Summary of the Invention

[0004] The main objective of the present invention is to provide a micro-nano structure optimization design method based on deep reinforcement learning, which significantly improves the optimization accuracy and efficiency, and at the same time has the ability of dynamic adjustment and global search, solves the problems that traditional methods are prone to falling into local optima in high-dimensional parameter spaces and have insufficient multi-objective optimization capabilities, and is widely applicable to performance improvement and design optimization in scenarios such as quantum devices, nano-composites, and heterogeneous interface structures.

[0005] To solve the above problems, the technical solution of the present invention is implemented as follows:

[0006] A micro-nano structure optimization design method based on deep reinforcement learning, the method includes:

[0007] Step 1: Obtain the electron density distribution of the micro-nano structure, construct the quantum state wave function of the micro-nano structure to describe the characteristics of the micro-nano structure at multiple scales; based on the quantum state wave function, calculate the quantum stress field of the micro-nano structure; according to the barrier height and barrier width of the micro-nano structure, calculate the quantum tunneling current density;

[0008] Step 2: Map the quantum wave state function, quantum stress field, and quantum tunneling current density to the state space of reinforcement learning to obtain the state vector of reinforcement learning; generate an optimized action based on the state vector; update the parameters of the micro-nano structure according to the optimized action; calculate the structural performance of the micro-nano structure after the parameters of the micro-nano structure are updated;

[0009] Step 3: Based on the rate of change of the structural performance, establish a convergence criterion based on Lyapunov stability to determine whether to converge. If so, terminate and complete the current optimization design. If not, return to Step 2.

[0010] Further, in Step 1, the methods for obtaining the electron density distribution include: using a transmission electron microscope to obtain the electron density distribution through high-resolution imaging technology; using a scanning tunneling microscope to measure the local electron state density and mapping the electron density distribution through the tunneling current; or performing an X-ray scattering experiment to make the X-ray scatter with the electrons in the material of the micro-nano structure, and restoring the electron density distribution through Fourier transform of the generated scattering pattern.

[0011] Further, in Step 1, the quantum state wave function of the micro-nano structure is constructed by the following formula:

[0012]

[0013] where, Ψ(r, t) represents the quantum state wave function of the micro-nano structure at time t and spatial position r; ρ(r) represents the electron density distribution of the micro-nano structure at spatial position r; E n represents the nth energy level in the quantum system; k B is the Boltzmann constant; T is the temperature; represents the probability distribution of the energy level E n under thermal equilibrium conditions; is the reduced value of the Planck constant; m * is the equivalent mass of the electron in the material of the micro-nano structure; ψ n (r, t) is the wave function representing the nth quantum state of the micro-nano structure at time t and spatial position r; is the Laplace operator; N is the number of energy levels.

[0014] Further, in Step 1, the quantum stress field of the micro-nano structure is calculated by the following formula based on the quantum state wave function:

[0015]

[0016] Among them, represents the component in the \(i\)-th row and \(j\)-th column at the spatial position \(r\); both \(i\) and \(j\) are indices in the spatial coordinate directions, and their values are all 1, 2, and 3, corresponding to the three directions of the \(X\)-axis, \(Y\)-axis, and \(Z\)-axis of the three-dimensional rectangular coordinate system respectively; when \(i = j\), represents the normal stress; when \(i\neq j\), represents the shear stress; \(\Psi\) * (\(r, t\)) is the conjugate of \(\Psi(r, t)\); \(x\) i represents the \(i\)-th spatial coordinate component in the micro-nano structure; \(x\) j represents the \(j\)-th spatial coordinate component in the micro-nano structure; \(x_1\) represents the \(X\)-axis coordinate component; \(x_2\) represents the \(Y\)-axis coordinate component; \(x_3\) represents the \(Z\)-axis coordinate component.

[0017] Furthermore, in step 1, according to the following formula, the quantum tunneling current density is calculated based on the barrier height and barrier width of the micro-nano structure:

[0018]

[0019] Among them, \(J\) quantum is the quantum tunneling current density; \(e\) is the electron charge; \(E\) F is the Fermi level; \(T(E)\) represents the transmission coefficient; \(f(\cdot)\) represents the Fermi-Dirac distribution function; \(\varPhi\) is the barrier height; \(d\) is the barrier width.

[0020] Furthermore, in step 2, the state vector of the reinforcement learning is calculated through the following formula:

[0021]

[0022] Among them, \(\Omega\) is the domain of the micro-nano structure; \(\lambda\) is the average interface thickness of the micro-nano structure; \(S\) t represents the state vector at time \(t\).

[0023] Furthermore, in step 2, let the model parameter of the reinforcement learning be \(\theta\), the learning rate be \(\beta\), and the policy function of the model parameter \(\theta\) be \(\pi\) θ ; the following formula is used to generate the optimized action based on the state vector:

[0024]

[0025] Among them, \(A\) t represents the optimized action at time \(t\); \(W\) is the weight matrix of the reinforcement learning; \(\sigma\) is the standard deviation of the Gaussian distribution; the model parameter \(\theta\) of the reinforcement learning satisfies the following update rule:

[0026]

[0027] Among them, F obj is the structural performance of the micro-nano structure; represents the gradient of the model parameter θ of the reinforcement learning; represents the gradient of the state vector; V is the value function; H(·) represents the system Hamiltonian.

[0028] Furthermore, the structural performance of the micro-nano structure is calculated using the following formula:

[0029]

[0030] Among them, ε ij represents the corresponding strain tensor; Y represents Young's modulus; γ represents the surface energy coefficient; κ represents the gradient energy coefficient; G s represents the surface Gibbs free energy; E g represents the energy gap width.

[0031] Furthermore, in step 2, the parameters of the micro-nano structure are a vector, and its elements include: Young's modulus, barrier height, barrier width, transmission coefficient, and average interface thickness.

[0032] Furthermore, in step 3, based on the rate of change of the structural performance, the convergence criterion based on Lyapunov stability is established through the following formula:

[0033]

[0034] Among them, R is the gas constant; ΔG is the gradient of the surface Gibbs free energy; when the convergence criterion Λ is less than the set convergence threshold ∈, convergence is judged.

[0035] The micro-nano structure optimization design method based on deep reinforcement learning of the present invention has the following beneficial effects:

[0036] The performance optimization of micro-nano structures usually involves multiple conflicting objectives. For example, increasing the tunneling current density may lead to an increase in stress concentration and a decrease in structural stability. The present invention comprehensively considers the dynamic changes of multiple performance objectives through the state vector in the reinforcement learning framework, and combines the policy network to generate optimization actions, thereby realizing the dynamic trade-off of multi-objective optimization. For example, when optimizing a tunneling transistor, the system can simultaneously focus on the increase in tunneling current and the maintenance of interface mechanical stability. Through the dynamic policy adjustment mechanism of reinforcement learning, the present invention can find the best balance between different objectives and avoid the performance bias problem caused by single-objective optimization in traditional methods.

[0037] In addition, the convergence criterion based on Lyapunov stability ensures the stability of the optimization process, enabling efficient implementation of multi-objective optimization in a complex parameter space. The optimization of micro-nano structures usually involves the coupling problem of multi-dimensional parameters (such as material properties, geometric parameters, and interface properties). Traditional methods are prone to falling into local optima in a high-dimensional parameter space. Through the deep policy network of reinforcement learning, the present invention uses high-dimensional physical quantities in the state vector as inputs and can quickly explore the complex parameter space. The gradient update rule in reinforcement learning combines the derivative information of the target performance and the dynamic changes of the quantum state wave function to guide the policy network to select the optimal adjustment direction in the parameter space, thus significantly improving the search efficiency. In addition, by introducing physical characteristics such as the transmission coefficient, the gradient of the electron density distribution, and the change in the interface free energy, the present invention can accurately capture the impact of minor adjustments on the performance in a high-dimensional parameter space, thereby avoiding large-scale blind searches. The reinforcement learning framework of the present invention can achieve dynamic adjustment of the optimization process through the state vector and the policy network. During the optimization process, the model parameters will be adaptively updated according to the current state and the change of the target performance. For example, when the optimization actions in a certain stage have no significant effect on the performance improvement, reinforcement learning can automatically adjust the output direction of the policy network and reallocate the optimization resources to avoid ineffective iterations. By combining the quantum stress field and the gradient of the tunneling current density in the state vector, the present invention can capture the performance change trend of the micro-nano structure at different stages in real time and generate targeted optimization actions. This intelligent optimization method significantly improves the efficiency, especially in the face of complex design problems, and can quickly respond to changes in performance requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 FIG. is a schematic flowchart of a method for optimizing the design of a micro-nano structure based on deep reinforcement learning provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims, and above-mentioned drawings of the present invention are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such used data may be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order different from that shown or described herein.

[0040] Example 1, refer to Figure 1 : A method for optimizing the design of a micro-nano structure based on deep reinforcement learning, the method comprising:

[0041] Step 1: Obtain the electron density distribution of the micro-nano structure, construct the quantum state wave function of the micro-nano structure to describe the characteristics of the micro-nano structure at multiple scales; based on the quantum state wave function, calculate the quantum stress field of the micro-nano structure; according to the barrier height and barrier width of the micro-nano structure, calculate the quantum tunneling current density;

[0042] The electron cloud distribution in the micro-nano structure exhibits complex quantum behaviors, which are determined by the geometric shape, material properties, and boundary conditions of the micro-nano structure. By introducing the wave function description of quantum mechanics, the probability amplitude can be used to reflect the distribution of electrons in a specific space. Electron density is the integral of the square of the wave function modulus, which can not only reflect the local characteristics of electrons in the micro-nano structure but also provide a basis for the calculation of the potential field distribution. Through the accurate acquisition of the density distribution, the system can determine the spatial distribution characteristics of the micro-nano structure at the electron level, thus laying a foundation for subsequent calculations. Next, by constructing the quantum state wave function, the dynamic characteristics of the micro-nano structure are further described. The wave function is the core description tool of the quantum system. It not only determines the spatial distribution of electrons but also controls the dynamic behavior of the quantum system. In the optimal design of micro-nano structures, the wave function needs to satisfy the Schrödinger equation and the boundary conditions related to the geometry of the micro-nano structure simultaneously. This process is actually a high-precision modeling of the potential fields inside and outside the micro-nano structure, transforming the micro-nano system from a macroscopic classical description to a microscopic quantum description. The significance of constructing the quantum state wave function lies in that it can capture both the local field effect and the quantum transition behavior of the micro-nano structure, which are important factors that cannot be ignored in the design of micro-nano structures.

[0043] On this basis, the calculation of the quantum stress field further reveals the mechanical properties of the micro-nano structure at the quantum scale. The existence of the stress field indicates that there is a mechanical response inside the micro-nano structure caused by the interaction between the electron density distribution and the potential field, and this response shows a complex tensor distribution at the quantum scale. The quantum stress field describes the mechanical effects generated by the interaction between microscopic particles, and its calculation depends on the coupling relationship between the gradient of the wave function and the potential field distribution. By accurately obtaining the stress field, the system can capture the changes in the mechanical properties of the micro-nano structure during the optimization process, such as the specific performance in optimization goals such as material strain and electronic structure adjustment. In addition, the calculation of the quantum tunneling current density takes into account the dynamic electron transport characteristics of the micro-nano structure. At the quantum scale, electrons can tunnel through the potential barrier, which is a unique quantum phenomenon in the micro-nano structure. The magnitude of the tunneling current density depends not only on the height and width of the potential barrier but also on various factors such as the shape of the wave function and the system temperature. By calculating the tunneling current density, the specific performance of the micro-nano structure in quantum transport performance can be clarified, which is particularly important for designing efficient electronic devices. The calculation of the tunneling current density is directly related to the transmission coefficient of electrons, and its physical meaning is to measure the probability of electrons crossing the potential barrier and to provide a quantitative performance index for the reinforcement learning state vector.

[0044] Step 2: Map the quantum wave state function, quantum stress field, and quantum tunneling current density to the state space of reinforcement learning to obtain the state vector of reinforcement learning; generate an optimized action based on the state vector; update the parameters of the micro-nano structure according to the optimized action; calculate the structural performance of the micro-nano structure after the parameter update of the micro-nano structure.

[0045] In this method, the quantum state wave function of the micro-nano structure describes the microscopic dynamic behavior of the system, while the quantum stress field and quantum tunneling current density provide the mechanical and electron transport characteristics of the system. By mapping these physical parameters to the state space of reinforcement learning, a high-dimensional state vector is formed, which can comprehensively reflect the performance of the micro-nano structure under the current parameter configuration. The state vector not only retains the high-precision characteristics of quantum mechanics calculations but also provides direct input data for reinforcement learning, enabling the entire optimization process to be based on the physical reality. This method of constructing the state space breaks through the dependence on low-dimensional features in traditional optimization and can better describe the complexity of micro-nano structures, especially in multi-scale and multi-parameter design problems. After the state space is constructed, the deep reinforcement learning algorithm comes into play, generating optimized actions through the policy network to guide the update of the micro-nano structure parameters. The design of the policy network is based on the neural network structure of deep learning, with the state vector as its input and the corresponding optimized action as its output. In the present invention, the optimized action usually manifests as an adjustment of the geometric parameters, material properties, or boundary conditions of the micro-nano structure. For example, the optimized action may determine how to change the barrier width or height of the micro-nano structure to improve the quantum tunneling efficiency; or optimize the stress distribution of the material to improve the overall stability. The policy network continuously learns the correspondence between historical states and actions and summarizes strategies that can improve the performance of the micro-nano structure. Compared with traditional optimization methods such as gradient descent or genetic algorithms, deep reinforcement learning can more efficiently explore the complex parameter space and avoid falling into local optima.

[0046] It should be noted that the generation of optimization actions does not solely depend on the input of the current state vector but is also influenced by the parameters of the policy network. These parameters are continuously updated through a reward mechanism during the process of reinforcement learning. The calculation of the reward is based on the changes in the performance of the micro-nano structure. Specifically, the magnitude of the change in the optimized performance compared to the pre-optimized performance constitutes the main basis for the reward. If the optimization action can significantly improve the quantum tunneling efficiency, reduce the stress concentration, or improve other performance metrics, the corresponding reward value is high; otherwise, the reward value is low or even negative. Through this positive feedback mechanism, the policy network can continuously correct the optimization actions it outputs, making the generated actions increasingly tend towards the global optimum. When the optimization action is generated, the system updates the parameters of the micro-nano structure according to the action and recalculates the structure performance. The core of this step lies in ensuring the practical feasibility of the optimization action and the accuracy of the effect evaluation. The parameter update is carried out based on physical models and engineering constraints, such as ensuring that the updated micro-nano structure still meets the actual manufacturing capabilities and application requirements. Subsequently, the quantum state wave function, quantum stress field, and quantum tunneling current density are recalculated through quantum mechanical calculation methods to evaluate the changes in the structure performance. This cyclic calculation provides real-time state updates for reinforcement learning, ensuring the dynamic nature and continuous improvement ability of the optimization process.

[0047] Step 3: Based on the rate of change of the structure performance, establish a convergence criterion based on Lyapunov stability to determine whether convergence has occurred. If so, terminate and complete the current optimization design; if not, return to Step 2.

[0048] In the context of micro-nano structure optimization, the design objectives can be quantified by one or more performance metrics, such as the density of quantum tunneling current, the uniformity of stress distribution in the structure, the energy conversion efficiency, etc. The optimization of these performance metrics can be regarded as the state evolution in the system's dynamic process, and the ultimate goal of optimization is to gradually evolve the system from the initial state to the target state. The construction of the Lyapunov function provides an effective means to evaluate whether the change of the system state converges towards the target state. By defining a scalar function that can reflect the performance gap of the system, namely the Lyapunov function, the complex state evolution process can be mapped to the change trend of a single scalar. When the value of the Lyapunov function gradually decreases and its derivative tends to be negative during the optimization process, it indicates that the system is gradually converging towards the design objective. In specific implementation, the present invention defines a Lyapunov function suitable for the optimization process according to the characteristics of the performance change of the micro-nano structure. This function usually takes the change rate of the performance metric or the distance from the target as the core and constructs a mathematical form that can comprehensively reflect the optimization effect. For example, for the optimization of the quantum tunneling current density, the Lyapunov function can be expressed as the cumulative square difference between the current current density and the target current density; for the optimization of the stress distribution, it can be expressed as the mean square error between the current stress distribution and the ideal stress distribution. Through this definition, the value of the Lyapunov function has a clear physical meaning during the optimization process. It can not only quantitatively evaluate the gap between the current design and the target design, but also provide immediate feedback on whether the optimization action is effective. Based on the derivative characteristics of the Lyapunov function, it is possible to further determine whether the optimization process is stable and convergent. The negative trend of the derivative is the key judgment basis for whether the optimization design is developing in the correct direction. When the derivative is negative, it means that the optimization action is reducing the design deviation, and this trend conforms to the optimization objective; on the contrary, if the derivative is positive, it may indicate that there are problems with the current optimization action and may even lead to the deviation of the optimization direction from the correct trajectory. To ensure the robustness of the optimization, in the present invention, the change trend of the Lyapunov function is further combined through the reward mechanism of reinforcement learning, that is, the effect of the optimization action is directly associated with the change of the Lyapunov function. If the optimization action can effectively reduce the value of the Lyapunov function, a positive reward is given; otherwise, a negative reward is given. In addition, the design based on the convergence criterion also needs to consider the actual engineering constraints and optimization costs. To balance the optimization effect and computational efficiency, the present invention determines whether to terminate the optimization by setting a threshold. When the value of the Lyapunov function reaches the preset convergence threshold, or its change rate is lower than a certain set minimum value, the system determines that the optimization has converged and terminates the subsequent iteration. This judgment method avoids unnecessary waste of computing resources and ensures that the optimization result has met the design objective. In the optimization design of micro-nano structures, this convergence mechanism is particularly important because the quantum characteristics of micro-nano structures usually require high-precision calculations, and unrestricted iteration will significantly increase the computational burden.Therefore, the reasonable design of the convergence criterion is directly related to the efficiency and cost of the entire optimization process.

[0049] Example 2: In step 1, the method for obtaining the electron density distribution includes: using a transmission electron microscope to obtain the electron density distribution through high-resolution imaging technology; using a scanning tunneling microscope to measure the local electron state density and mapping the electron density distribution through the tunneling current; or performing an X-ray scattering experiment, causing the X-ray to scatter with the electrons in the material of the micro-nano structure, and restoring the electron density distribution from the generated scattering pattern through Fourier transform.

[0050] Specifically, the first method is to obtain the electron density distribution by using a transmission electron microscope in combination with high-resolution imaging technology. The core of this technology lies in the fact that a transmission electron microscope can record the changes in phase and intensity during the interaction between the internal structure of the sample and electrons by passing a high-speed electron beam through the sample material. Through high-resolution imaging technology, the distribution characteristics of the electron cloud can be directly analyzed, and the electron density information of micro-nano structures can be obtained. This method is applicable to micro-nano materials with complex internal structures and can accurately capture the details of electron distribution with sub-nanometer spatial resolution. For example, for heterogeneous interfaces or orderly arranged nano-lattices, the imaging of a transmission electron microscope can clearly show the spatial variation of electron density, thus providing direct data support for the subsequent construction of the quantum state wave function. Since the signal acquisition of a transmission electron microscope is based on the coherence of electron waves, it has high sensitivity and high resolution to the electron distribution of the sample, making this method a powerful tool for studying the electron state distribution of micro-nano structures. The second method is to measure the local electron state density by using a scanning tunneling microscope, and thus map the electron density distribution through the tunneling current. The working principle of a scanning tunneling microscope is based on the quantum tunneling effect. When a sharp probe approaches the surface of the sample, due to the tunneling barrier formed between the probe and the sample, electrons can tunnel through the barrier, generating a tunneling current. The magnitude of this current is directly related to the local electron state density (LDOS) on the surface of the sample. By changing the probe position and recording the corresponding tunneling current, STM can reconstruct the electron density distribution of micro-nano structures with extremely high resolution. The advantage of this method lies in its ability to perform localized measurements, which can resolve the details of the electron state distribution on the surface of micro-nano structures or even around individual atoms. For example, when studying the electron transport properties of single-molecule devices, STM can reveal the electron state coupling between the molecule and the electrode, thus providing important information for constructing the quantum state wave function. In addition, by adjusting the probe voltage, STM can also obtain electron density data related to the energy distribution, which provides an effective technical means for studying the energy dependence of the electron state distribution. The third method is to restore the electron density distribution by using X-ray scattering experiments. The basic principle of X-ray scattering is that when X-rays irradiate the material of micro-nano structures, the electrons in the material will undergo elastic scattering with the X-rays, and the intensity and angle of the scattering carry information about the electron distribution. By collecting these scattering patterns and performing Fourier transforms on them, the electron density distribution inside the material can be reconstructed. This method is particularly suitable for studying periodic or quasi-periodic micro-nano structures, such as crystals, thin films, and multi-layer nano materials. Compared with TEM and STM, X-ray scattering has a larger penetration depth and can provide electron density information in three-dimensional space, thus enabling a comprehensive analysis of the entire micro-nano structure. Especially when studying micro-nano structures with complex geometries or multi-layer interfaces, X-ray scattering can reveal the overall characteristics of their internal electron distribution.Another advantage of this method is its support for non-destructive measurements, which is particularly suitable for studying sensitive samples or micro-nano structures in working conditions.

[0051] Example 3: In step 1, the quantum state wave function of the micro-nano structure is constructed by the following formula:

[0052]

[0053] where Ψ(r, t) represents the quantum state wave function of the micro-nano structure at time t and spatial position r; ρ(r) represents the electron density distribution of the micro-nano structure at spatial position r; E n represents the nth energy level in the quantum system; k B is the Boltzmann constant; T is the temperature; represents the probability distribution of the energy level E n under thermal equilibrium conditions; h is the reduced value of the Planck constant; m * is the effective mass of the electron in the material of the micro-nano structure; ψ n (r, t) is the wave function representing the nth quantum state of the micro-nano structure at time t and spatial position r; is the Laplace operator; N is the number of energy levels.

[0054] Specifically, the wave function Ψ(r, t) in the formula describes the overall quantum state of the micro-nano structure at time t and space r, which is the core concept of probability amplitude in quantum mechanics. By taking the square of the modulus of the wave function, the probability density distribution of electrons at a certain spatial position can be determined, which provides statistical support for the performance analysis of the micro-nano structure. The structure of the formula is divided into two parts, which respectively reflect the density distribution of electrons in the micro-nano structure and the contribution of each quantum energy level to the overall wave function. The first part is represented by the form of , which combines the spatial distribution of electrons in the micro-nano structure and the occupation probability of energy levels under thermodynamic equilibrium conditions. Here, ρ(r) is the electron density, which reflects the spatial distribution characteristics of the micro-nano structure under specific geometric shapes and material conditions. The electron distribution of the micro-nano structure is usually affected by local fields, geometric boundaries, and material properties. Therefore, the calculation of ρ(r) depends on experimental measurements or high-precision numerical simulations. And represents the energy level distribution under thermal equilibrium conditions, which describes how the quantum system distributes electrons to different energy levels according to probability at temperature T through statistical mechanics principles. Combining these two, by introducing the Laplace operator , the expansion and gradient changes of the electron density distribution in space can be further captured, thus unifying the local and diffusion characteristics of electron behavior. The second part of the formula describes the contribution of each energy level wave function to the system wave function through . This part reveals how the kinetic energy term of the micro-nano structure affects the quantum state distribution. ψn (r, t) represents the wave function of the nth energy level and is a detailed description of the spatial and temporal characteristics of the electron behavior in the micro-nano structure at a specific energy level. The Laplace operator is used here to characterize the spatial variation of the wave function, reflecting the influence of the coupling between energy levels and the electron kinetic energy distribution inside the micro-nano structure. And the coefficient in front comes from the kinetic energy operator in quantum mechanics, is the reduced value of Planck's constant, m * is the effective mass of the electron, reflecting the actual constraint of the micro-nano material on the electron motion. The physical meaning of this part of the formula lies in revealing how the quantum states in the micro-nano structure are restricted by the basic characteristics of the material, such as the band structure and crystal symmetry. This description method is particularly suitable for studying the quantum behavior of complex micro-nano structures, such as the layered transport characteristics of electrons in multi-layer heterojunctions. By combining the above two parts, this formula constructs a complete expression of the overall quantum state wave function of the micro-nano structure. It not only synthesizes the distribution of electron density, the thermodynamic energy level probability distribution, and the local behavior of the single quantum state wave function, but also depicts the details of the spatial variation through the Laplace operator. This multi-scale and multi-dimensional representation method significantly improves the accuracy and applicability of the wave function construction, enabling it to adapt to complex micro-nano structures and high-performance optimization requirements. In the framework of the present invention, the role of this wave function construction formula is to provide accurate quantum state input data for the state space of deep reinforcement learning. Reinforcement learning relies on the high-dimensional information of the state space to generate optimized actions, and the quantum state information contained in the formula is exactly the core characteristic of the performance of the micro-nano structure. Compared with the way of constructing the state space by simple parameterization methods in traditional optimization design, the present invention directly introduces the multi-dimensional characteristics of the micro-nano structure into the optimization process through precise calculation of the quantum state wave function, greatly improving the physical authenticity and optimization efficiency of the state space. In addition, another significant advantage of this formula is its combination with temperature. By introducing the energy level thermal equilibrium distribution term the formula can capture the quantum state changes of the micro-nano structure under different temperature conditions. This temperature sensitivity provides a wider applicability for the application scenarios of the present invention, especially in the design of micro-nano devices involving thermal management or thermal-induced performance optimization. For example, by adjusting the temperature T in the formula, the specific characteristics of enhanced quantum tunneling in the micro-nano structure under low-temperature conditions or the diffusion of electron distribution under high-temperature conditions can be analyzed, thus providing an additional parameter regulation dimension for optimization design.

[0055] Example 4: In step 1, based on the quantum state wave function, calculate the quantum stress field of the micro-nano structure through the following formula:

[0056]

[0057] where represents the component at the \(i\)-th row and \(j\)-th column in the spatial position \(r\); both \(i\) and \(j\) are indices in the spatial coordinate directions, taking values of 1, 2, and 3 respectively, corresponding to the three directions of the \(X\)-axis, \(Y\)-axis, and \(Z\)-axis in a three-dimensional rectangular coordinate system; when \(i = j\), represents the normal stress; when \(i\neq j\), represents the shear stress; Ψ * (r, t) is the conjugate of Ψ(r, t); x i represents the \(i\)-th spatial coordinate component in the micro-nano structure; x j represents the \(j\)-th spatial coordinate component in the micro-nano structure; \(x_1\) represents the \(X\)-axis coordinate component; \(x_2\) represents the \(Y\)-axis coordinate component; \(x_3\) represents the \(Z\)-axis coordinate component.

[0058] Specifically, the quantum stress field describes the quantum mechanical stress distribution of the micro-nano structure at a specific spatial position \(r\), which is a generalization of the stress tensor in classical mechanics to the field of quantum mechanics. The components of the stress field are represented by the second derivative which reflects the influence of the spatial variation of the wave function, indicating the interaction and coupling of the wave function in different coordinate directions. This description method directly maps the quantum state behavior of the micro-nano structure to its mechanical properties, and is particularly suitable for analyzing micro-scale structures that are significantly affected by quantum mechanics, such as nano-thin films, quantum dots, and heterojunctions. The core of the formula is a bilinear form calculated from the wave function Ψ(r, t) and its conjugate wave function Ψ * (r, t). The wave function Ψ(r, t) describes the micro-nano structure in a quantum state, and the square of its modulus provides the probability density of electrons, while the derivative operation further reveals the curvature characteristics of the wave function in spatial variation. When the curvature is large, it indicates a strong local mechanical response in the system; when the curvature is small, it reflects a weak stress state. The introduction of the conjugate wave function Ψ * (r, t) makes the formula symmetric, and this symmetry conforms to physical laws, that is, the description of the quantum state of the system should meet the requirements of real physical quantities.

[0059] In the formula, is a dimensional adjustment factor, which comes from the characteristics of the kinetic energy operator in quantum mechanics. Here, is the reduced value of Planck's constant, \(m\) * is the effective mass of the electron, which is used to reflect the effective inertia of the electron when moving in the micro-nano material. This factor converts the dimensionless characteristics of the wave function derivative into a dimension with practical mechanical significance, ensuring that the results of the formula can be directly compared with experimental measurements or classical mechanics results. The calculation process of quantum stress includes the superposition of each energy level, which is achieved through the summation symbol

[0060] Embodiment. Quantum states in micro-nano structures usually have multiple energy levels, and the wave function Ψ(r, t) of each energy level contributes to the total stress field. Through this superposition method, the formula captures the comprehensive mechanical behavior of micro-nano structures in the case of multi-level coupling, rather than being limited to the description of a single energy level. This method is particularly applicable to micro-nano materials with complex energy level distributions or multi-state transition characteristics, such as semiconductors with strong spin-orbit coupling effects or superconducting materials that exhibit quantum interference effects at low temperatures. In the optimal design of micro-nano structures, the calculation of the quantum stress field is of great significance. It can reveal local mechanical effects that cannot be captured by classical mechanics, such as the quantization characteristics of stress distribution at extremely small geometric sizes or the local field enhancement effect at material interfaces. This information is crucial for optimizing the performance of micro-nano structures. For example, when designing nano-devices, the stress concentration can be reduced by adjusting the geometric shape or material distribution, thereby improving the reliability and stability of the structure. And this adjustment requires the precise calculation of the quantum stress field as a support. In addition, the components of the quantum stress field provide multi-dimensional information. When i = j, the diagonal elements reflect the normal stress distribution of the system in a certain direction, that is, the tensile or compressive behavior of the material in this direction; when i ≠ j, the off-diagonal elements represent shear stress, revealing the slip or torsion characteristics inside the material. This comprehensive representation of the three-dimensional stress field provides rich data support for optimizing the mechanical properties of micro-nano structures. In the reinforcement learning framework, these stress components can be directly mapped as part of the state vector and used as the input of the optimization strategy, thus providing an accurate physical basis for action generation and parameter adjustment.

[0061] Example 5: In step 1, according to the following formula, calculate the quantum tunneling current density based on the barrier height and barrier width of the micro-nano structure:

[0062]

[0063] where J quantum is the quantum tunneling current density; e is the electron charge; E F is the Fermi level; T(E) represents the transmission coefficient; f(·) represents the Fermi-Dirac distribution function; Φ is the barrier height; d is the barrier width.

[0064] Specifically, the overall structure of the formula is centered on the transmission probability of quantum mechanics and combines the electron distribution law of statistical mechanics to calculate the tunneling current density. In the basic expression of the current density J quantum , the upper and lower limits E F -eV and E F of the integral define the energy range, describing how electrons near the Fermi level participate in tunneling transmission under the applied voltage V. The Fermi level E Fis a key parameter that determines the electron distribution, marking the boundary point of the occupancy probability of electrons in different energy states. Applying an external voltage V causes electrons to transition from high-energy states to low-energy states, thereby forming a net flow at both ends of the potential barrier. The magnitude of this net flow is determined by the difference f(E - eV) - f(E) of the Fermi-Dirac distribution, reflecting the statistical origin of the tunneling current. The expression for the transmission coefficient T(E) further reveals how quantum tunneling behavior is affected by the potential barrier height and width d. The transmission coefficient is given by the exponential decay characteristic of the wave function in the potential barrier region, in the specific form of This term originates from the solution of the wave equation and describes the penetration probability of the wave function in the potential barrier. In the exponential term, is the potential barrier factor for the decay of the wave function by the potential barrier, where m * is the effective mass of electrons in the material, used to reflect the inertial limitation of the material on the electron motion. The introduction of the effective mass is crucial because it directly links the electronic properties of micro-nano materials with the tunneling behavior. The potential barrier height Φ represents the potential energy that electrons need to overcome at the potential barrier. When the electron energy E approaches Φ, the actual influence of the potential barrier weakens, resulting in a sharp increase in the transmission coefficient. This explains the phenomenon that the tunneling current can be significantly enhanced by reducing the potential barrier height in experiments.

[0065] The formula also accumulates the tunneling currents at different energy states through an integration operation to form the total tunneling current density. This integration process reflects the statistical characteristics of the electron behavior in the micro-nano structure. The lower and upper limits of the integration determine that only those electrons close to the Fermi level will participate in tunneling, while electrons far from the Fermi level have negligible contribution to the current due to low state density or insufficient kinetic energy. Through this integration method, the formula can accurately capture how the quantum effects in the micro-nano structure accumulate to affect the tunneling behavior at different energy states. The Fermi-Dirac distribution f(E) is an indispensable part of the formula, revealing the statistical dynamic origin of the tunneling behavior. The form of the distribution function indicates that the distribution probability of electrons is related to the deviation of temperature T and energy E. When the temperature increases, the distribution probability of electrons in high-energy states increases, resulting in more electrons being able to participate in the tunneling process. This temperature dependence explains the trend of the tunneling current varying with temperature observed in experiments: at higher temperatures, the tunneling efficiency of electrons is higher and the current density increases. This dependence provides a theoretical basis for optimizing the performance of micro-nano structures by adjusting the temperature. In the deep reinforcement learning framework of the present invention, this formula provides extremely important input parameters for the state space. The optimization process of reinforcement learning needs to rely on high-dimensional and real physical data, and the quantum tunneling current density, as the core index of the micro-nano structure performance, directly affects the definition of the optimization goal and the training of the policy network. For example, when optimizing the design of a tunneling diode, reinforcement learning adjusts the potential barrier width d, height Φ, or material properties (such as the effective mass m *)Maximize J quantum Each parameter involved in the formula can be dynamically mapped into the state vector of reinforcement learning and used as input data to drive the policy network to generate optimized actions, realizing the efficient iteration of parameter adjustment. The calculation of traditional tunneling current often simplifies to a single barrier height or linear approximation. However, by introducing a non-linear form of the transmission coefficient and the energy distribution correction in the integration process, this formula achieves an accurate capture of tunneling behavior. This accuracy is not only of great theoretical significance but also provides direct guidance in practical applications. For example, in the design of quantum dot devices or low-power tunneling transistors, this formula can accurately predict the current density under different parameter combinations, providing a quantitative basis for design decisions.

[0066] Embodiment 6: In step 2, the state vector of reinforcement learning is calculated by the following formula:

[0067]

[0068] where Ω is the domain of the micro-nano structure; λ is the average interface thickness of the micro-nano structure; S t represents the state vector at time t.

[0069] Specifically, the state vector S in the formula t is the core input data of reinforcement learning, which describes the physical state of the system at time t. The state vector is calculated in integral form over the entire domain Ω of the micro-nano structure, aggregating the local physical properties in space into a global state description. The core idea of this method is to use the integral operation to capture the cumulative effect of quantum states at different positions, while highlighting those regions that have a greater impact on the overall performance through weighting factors. First, the main part of the formula is the tensor product of three quantum physical quantities This tensor product form organically combines the quantum state wave function, the quantum stress field, and the tunneling current density, reflecting their synergistic effect in the design of micro-nano structures. The quantum state wave function ψ(r, t) describes the dynamic characteristics of electron distribution, providing the basic quantum state information of the system; the quantum stress field reflects the mechanical response ability of the structure at the quantum mechanical level; the tunneling current density J quantum is the direct manifestation of electron transport performance. By combining these three physical quantities in tensor product form, the multi-scale performance characteristics of the micro-nano structure can be comprehensively reflected. For example, in the design of high-performance electronic devices, the state vector can simultaneously capture the complex coupling relationships of electron distribution, stress distribution, and transport characteristics, providing key high-dimensional data support for the decision-making of reinforcement learning. The weighting factor is another core part of the formula. It is obtained through the Laplacian operator of the electron density distribution And the average interface thickness λ is used to smooth the physical quantities in space. The physical significance of this part lies in emphasizing the influence of the interface region, because the interface is usually the region where the performance changes most significantly in micro-nano structures. The Laplace operator describes the curvature of the electron density distribution, and the density gradient change at the interface is often large. Therefore, through the weight factor, the formula can enhance the attention to the state characteristics of the interface region. At the same time, the parameter λ controls the distribution width of the weight factor, reflecting the size of the interface influence range. For micro-nano structures with complex geometries or multi-layer heterogeneous interfaces, the selection of λ is crucial, and it needs to balance local characteristics and global information to ensure that the state vector can accurately reflect the true state of the entire structure. The integral operation ∫ Ω ·dΩ is the key step for the formula to achieve global information aggregation. Through the integration over the structural domain Ω, the formula can transform the coupling effects of local quantum state wave functions, stress fields, and tunneling current densities into a global description. The advantage of this processing method is that it can simultaneously capture the detailed features at a small spatial scale and the overall performance of the macroscopic structure, thus providing comprehensive state information for the reinforcement learning algorithm. For example, when optimizing complex nano-devices, the state vector not only contains the distribution information of local electron states but also can reflect the comprehensive performance of the entire device in current transmission and mechanical stability.

[0070] Example 7: In step 2, let the model parameters of reinforcement learning be θ, the learning rate be β, and the policy function of the model parameter θ be π θ ; Use the following formula to generate an optimized action based on the state vector:

[0071]

[0072] where A t represents the optimized action at time t; W is the weight matrix of reinforcement learning; σ is the standard deviation of the Gaussian distribution;

[0073] In the formula, the state vector S t is the basis for generating the optimized action. It synthesizes various quantum physical quantities of the micro-nano structure, such as quantum state wave functions, quantum stress fields, and tunneling current densities. These physical quantities not only describe the current performance state of the micro-nano structure but also provide the high-dimensional information required in the optimization process. The optimized action A tIt is obtained by calculating the gradient of the state vector. This gradient-based generation method ensures that the optimization action can capture the local change trend of the current state and point to the direction that can improve the performance. In the reinforcement learning framework, this state-gradient-based optimization action generation is the core of the model's intelligence, which enables the optimization process to dynamically adjust the parameters of the micro-nano structure to achieve the performance optimization of specific goals. The weight matrix W in the formula reflects the contribution ratio of different state components to the optimization action. This weighting process enables the reinforcement learning model to adjust the focus according to the task requirements. For example, when optimizing a micro-nano device mainly for electron transport, it may be necessary to assign a higher weight to the tunneling current density J quantum component, while when optimizing the stress distribution, more attention needs to be paid to the components of the quantum stress field. By dynamically adjusting W, the formula provides flexibility for reinforcement learning, enabling it to adapt to different optimization goals and scenarios. The learning rate β plays a balancing role in the formula. It controls the influence amplitude of the state vector gradient on the optimization action, ensuring that the optimization process can respond to the performance changes and will not fall into an unstable state due to excessive adjustment. The setting of the learning rate directly affects the convergence speed of the optimization and the improvement effect of the final performance. For example, in the early iterations, a larger learning rate helps to quickly explore the global parameter space, while when approaching convergence, a smaller learning rate can refine the optimization results and improve the accuracy. The generation of the optimization action also includes the gradient correction term for the quantum tunneling current density. This term further refines the directionality of the optimization action through the gradient change of the tunneling current by the state vector. The quantum tunneling current density J quantum is the core index describing the electron transport performance of the micro-nano structure, and its change trend directly reflects the effectiveness of the optimization design. By introducing the gradient correction term, the formula can capture the influence of small parameter adjustments on the tunneling current and feedback this influence into the optimization action. The significance of this mechanism is that it can respond to the changes in the performance index in real time and provide dynamic guidance for the optimization process. For example, when the adjustment of the state vector causes a significant increase in the tunneling current, the gradient correction term will amplify this change trend, thus promoting the parameter adjustment in a favorable direction. The Gaussian weighting factor in the formula is the adjustment of the quantum stress field . The stress field is a key parameter in the performance optimization of micro-nano structures. Especially when it comes to mechanical properties or multi-physical field coupling, its distribution characteristics have an important impact on the structural stability. Through the weighting factor the formula can reduce the adverse effects of the stress concentration area on the optimization action. This weighting mechanism is based on the Gaussian distribution, assigning smaller weights to larger stress values, thereby weakening the factors that may cause structural failure in the generation of the optimization action. This design reflects the comprehensive consideration of safety and stability in the optimization design of the present invention and provides a more robust optimization method for reinforcement learning.

[0074] The model parameters θ of reinforcement learning satisfy the following update rules:

[0075]

[0076] where F obj is the structural performance of the micro-nano structure; represents the gradient of the model parameter θ of reinforcement learning; represents the gradient of the state vector; V is the value function; H(·) represents the system Hamiltonian.

[0077] Specifically, the first key part of the formula involves the combination of the policy function π θ and the state gradient. Through this term, the reinforcement learning model can use the current state information and the state gradient to update the policy function. Here, π θ is the policy function, which describes the probability distribution of the model generating optimized actions under specific parameters θ, while the state gradient reflects the direction and magnitude of the energy change of the system in the current state S t . By combining the policy function and the state gradient, the formula can ensure that the update direction of the model parameters is consistent with the direction of the system energy reduction. This physically driven update rule ensures the rationality of the optimization process in terms of physics. The Hamiltonian is one of the core physical quantities of the formula, which describes the total energy of the system in a specific state, including two parts: potential energy and kinetic energy. Here, S t is the current state vector, and is the gradient of the value function, which describes the response characteristics of the state vector to the optimization objective. The introduction of the Hamiltonian provides a physical basis for reinforcement learning. By calculating the gradient of the Hamiltonian, the formula can capture the optimal adjustment path of the current energy state of the system. This way not only makes the optimization process more in line with physical laws but also enhances the sensitivity of the policy update to state changes. For example, when optimizing the tunneling current density of a quantum dot structure, the Hamiltonian can accurately describe the energy change trend of electrons between potential barriers, thus guiding the reinforcement learning model to generate more efficient policy updates. The second part of the formula involves the second-order gradient correction of the optimization objective F obj . Through the reinforcement learning model can further enhance the non-linear sensitivity to the optimization objective during parameter updates. The optimization objective F objIt is a direct quantitative index for the performance of micro-nano structures, such as tunneling current density, stress distribution uniformity, or thermal management efficiency. In the actual optimization process, the optimization objectives often exhibit highly non-linear and multi-modal distributions, and simple gradient descent methods may not be able to quickly find the global optimal solution. Through second-order gradient correction, the formula can capture the curvature effect of parameter changes on the optimization objective, thus considering more complex non-linear relationships in parameter updates. This correction mechanism is particularly important when optimizing complex micro-nano structures. For example, when optimizing multi-layer quantum well structures, second-order gradient correction can more accurately identify the optimal parameter combination, thereby improving the optimization efficiency. In addition, the learning rate β in the formula plays a role in balancing the convergence speed and accuracy of the optimization process. A larger learning rate can accelerate the global exploration of parameters, while a smaller learning rate is suitable for the later refinement stage of the optimization process to ensure the accuracy and stability of the final result. By adjusting β, the reinforcement learning model can dynamically adapt to the requirements of different optimization stages, thus maintaining high efficiency and reliability throughout the optimization process.

[0078] Example 8: The structural performance of the micro-nano structure is calculated using the following formula:

[0079]

[0080] where ε ij represents the corresponding strain tensor; Y represents Young's modulus; γ represents the surface energy coefficient; k represents the gradient energy coefficient; G s represents the surface Gibbs free energy; E g represents the energy gap width.

[0081] Specifically, the first term of the formula is the policy gradient part in reinforcement learning, which describes how the model parameters θ are updated through the change direction of the state vector and the Hamiltonian of the system. The Hamiltonian is used in quantum mechanics to describe the total energy distribution of the system, and its form combines the state vector S t and its gradient here to reflect the energy distribution characteristics of the micro-nano structure performance in the current state space. Through this term, the formula incorporates the energy change of the physical system into the policy optimization process, ensuring that the parameter update direction of reinforcement learning is consistent with the optimal energy path of the physical system. This energy distribution-based update method can effectively avoid ineffective searches and make the optimization process more efficient. Specifically, the introduction of the Hamiltonian is to establish an energy-based optimization path in the state space, describing the energy gradient of the system in the state space, which directly reflects the impact of the current design parameters on the energy distribution. For example, when optimizing the electron transport performance of micro-nano structures, the Hamiltonian can be designed to be related to the tunneling current density J quantumAnd stress field distribution For the related energy function, through the gradient calculation of this function, reinforcement learning can capture how to adjust the parameters to reduce the system energy and thus improve the structural performance.

[0082] The policy function π in the formula θ is the core of reinforcement learning, which generates an optimized policy through the input of the state vector. This term is coupled with the gradient of the Hamiltonian in the formula, and through the calculation of the policy gradient , the model parameters θ are optimized to improve the quality of the policy. The update method of the policy function directly affects the ability of reinforcement learning to solve complex optimization problems. For example, in the design of micro-nano structures, the policy function needs to simultaneously consider the coupling effects of multiple performance objectives (such as stress distribution, tunneling current, and material stability). Through the gradient term in the formula, the policy function can dynamically adapt to the optimization requirements of these complex objectives. The second term of the formula is the second-order gradient correction term based on the performance objective, which is used to further enhance the sensitivity and convergence of the optimization process. The performance objective F here obj is the core evaluation index for the optimization of micro-nano structures, such as maximizing the tunneling efficiency, improving the stress distribution uniformity, or optimizing the thermal performance. By taking the second derivative of F objRegarding the second-order gradient calculation of model parameters, the formula captures the non-linear variation characteristics of the objective function in the parameter space. This non-linear variation is particularly significant in the optimization process of micro-nano structures because quantum effects and multi-physics field coupling usually result in the objective function showing complex high-order non-linear relationships. Through this correction term, the formula can guide the model parameters to quickly approach the optimal point of the target performance while avoiding being trapped in local optima. The learning rate β plays a regulatory role in the second-order gradient correction. By adjusting the magnitude of β, the stepping amplitude of the optimization process can be controlled to achieve a balance between exploration and convergence. In the early stage of optimization, a larger learning rate can help the algorithm quickly explore the global optimal solution, while when approaching convergence, a smaller learning rate helps to improve the optimization accuracy. In addition, the setting of β can also be dynamically adjusted to flexibly adapt to changes in the optimization objective and current performance, thereby enhancing the robustness of the algorithm. The overall design of this formula organically combines the quantum mechanical characteristics of physical systems with the intelligent optimization mechanism of reinforcement learning, providing theoretical support and technical implementation paths for the optimal design of micro-nano structures. Compared with traditional optimization methods, the innovation of this formula lies in the dynamic adjustment of model parameters through the dual effects of state gradient and policy gradient, and at the same time, combined with the gradient correction of the Hamiltonian and performance objectives, ensuring that the optimization process can quickly and accurately converge to the global optimal solution. This method is particularly suitable for the design of micro-nano structures with complex coupling characteristics and multi-objective optimization requirements. For example, when designing nano-scale tunneling transistors or quantum dot devices, through the gradient update rule in the formula, the reinforcement learning model can effectively capture the mutual relationship between different objectives, thereby generating the optimal design scheme.

[0083] Example 9: In step 2, the parameters of the micro-nano structure are a vector, and its elements include: Young's modulus, barrier height, barrier width, transmission coefficient, and average interface thickness.

[0084] Specifically, as an important part of the parameter vector, the Young's modulus reflects the mechanical properties of the micro-nano structure. The Young's modulus is a key indicator for measuring the stiffness of materials and describes the strain response ability of materials under stress. In micro-nano structures, the Young's modulus has a direct impact on their mechanical stability, deformation resistance, and interface strength. For example, when designing nano-films or multi-layer composite materials, optimizing the Young's modulus can regulate the elastic response characteristics of the materials, thereby improving their durability and reliability. In the reinforcement learning framework, as part of the state vector, the Young's modulus can adjust the material properties by optimizing actions, making the design more in line with the performance requirements. The barrier height and barrier width are the parts in the parameter vector that are directly related to the quantum mechanical properties. These two parameters determine the occurrence probability of electron tunneling behavior in micro-nano structures. The barrier height represents the energy barrier that electrons need to overcome, while the barrier width describes the spatial range of this barrier. By adjusting these two parameters, the magnitude of the tunneling current can be significantly affected. For example, when designing a tunneling diode, reducing the barrier height or decreasing the barrier width can increase the transmission coefficient of electrons, thereby improving the conductivity of the device. During the reinforcement learning optimization process, the barrier height and width can be dynamically adjusted through the feedback data in the state vector, enabling the optimization process to adapt to the performance requirements of different application scenarios in real time. The transmission coefficient is an important physical quantity that describes the probability of electrons passing through the barrier and is also a key element in the parameter vector. It directly reflects the transmission efficiency of micro-nano structures under the quantum tunneling phenomenon. The transmission coefficient is not only related to the barrier height and width but also affected by material properties (such as the effective mass) and the electron energy distribution. In the state vector of reinforcement learning, the dynamic change of the transmission coefficient provides an important optimization basis for policy generation. For example, when the state vector indicates a low transmission coefficient, reinforcement learning can generate optimized actions to adjust the barrier parameters, thereby increasing the transmission coefficient and making the design more efficient. The average interface thickness is an important element in the parameter vector that reflects the geometric characteristics. It describes the average thickness of the interface in micro-nano structures and has an important impact on the interface strength, stress distribution, and heat conduction performance of materials. Especially in multi-layer composite structures or heterojunctions, the average interface thickness directly affects the mechanical response and electron transport characteristics at the interface. For example, increasing the average interface thickness may help disperse stress concentration and enhance the mechanical stability of the structure; while reducing the average interface thickness may improve the cross-interface transmission efficiency of electrons. In the reinforcement learning optimization, as part of the state vector, the average interface thickness can be dynamically adjusted through optimized actions, thereby achieving a balance of different performance goals.

[0085] Example 10: In step 3, based on the change rate of the structural performance, the convergence criterion based on Lyapunov stability is established through the following formula:

[0086]

[0087] where R is the gas constant; ΔG is the gradient of the surface Gibbs free energy; when the convergence criterion Λ is less than the set convergence threshold ∈, convergence is determined.

[0088] Specifically, in the formula, the convergence criterion Λ contains multiple key terms that respectively describe the time evolution of the system performance and the coupling characteristics of the physical process. First, is the time derivative of the target performance F obj and is used to quantify the speed of performance change of the micro-nano structure under the action of the current optimization action. This term directly reflects the immediate effect of the optimization. For example, if the optimization action can significantly increase the quantum tunneling current density or improve the stress distribution uniformity, the performance derivative will show a large positive value. When the performance derivative gradually approaches zero, it indicates that the optimization process is approaching convergence. This dynamic performance index is the core part of the convergence criterion, ensuring that the optimization process is performance-driven. The second part of the formula, further introduces the influence of the dynamic change of the electron tunneling behavior on the convergence. Here, ρ(r) represents the electron density distribution, and the integral term describes the contribution of the tunneling current density in the entire micro-nano structure domain Ω. By calculating the time derivative of this integral term, the formula can capture the change trend of the tunneling current. If the distribution of the tunneling current tends to be stable within the structure domain, that is, its change rate gradually decreases, the value of the second part of the formula will approach zero, indicating that the electron transport behavior of the micro-nano structure has reached a stable state. The introduction of this part enables the convergence criterion to comprehensively consider the role of quantum effects in the optimization process, especially in those designs where electron transport is a key performance, this term is particularly important. In addition, the formula uses the exponential term The gradient of the surface Gibbs free energy, ΔG, is incorporated into the convergence judgment. The physical significance of this term lies in reflecting the impact of interface energy changes on the optimization of micro-nano structures. The Gibbs free energy is an important thermodynamic parameter describing the interface characteristics of materials, and its gradient ΔG reflects the rate of change of the interface energy in space. For example, in multi-layer heterogeneous interfaces or nanoparticle systems, the energy gradient at the interface may significantly affect the stability and electron transport characteristics of the system. Through an exponential decay form, the formula can suppress the contribution of high-energy gradient regions, ensuring that the optimization process pays more attention to those regions where the energy tends to be stable. When the rate of change of ΔG is small, it indicates that the interface characteristics of the system have approached stability, thus further supporting the convergence judgment. The product form in the formula synthesizes the combined effects of performance changes, tunneling behavior, and interface energy. By dynamically weighing the relative importance of these three, it provides a multi-dimensional physical basis for the convergence criterion. Compared with traditional convergence criteria that rely only on a single performance index or the change of the optimization step size, the innovation of this formula lies in its multi-physical field comprehensiveness, especially the deep integration of quantum effects and thermodynamic characteristics. For example, when designing quantum dot devices, the performance changes of the system are not only affected by the tunneling current but may also be significantly regulated by the interface energy distribution. This formula can capture these coupling effects simultaneously, making the convergence judgment more comprehensive and accurate. When the convergence criterion Λ is less than the set threshold ∈, the system determines that the optimization has reached the convergence state. The setting of this threshold needs to be combined with specific design goals and the characteristics of the physical system. In the initial optimization stage, the threshold can be relatively high to allow for larger performance fluctuations and exploration space; while approaching convergence, the threshold needs to be gradually reduced to improve the optimization accuracy. Through this dynamic adjustment, the optimization process can achieve a balance between efficiency and accuracy.

[0089] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A micro-nanostructure optimization design method based on deep reinforcement learning, characterized in that: The method comprises: Step 1: Obtain the electron density distribution of the micro-nano structure and construct the quantum state wave function of the micro-nano structure to describe the characteristics of the micro-nano structure at multiple scales; calculate the quantum stress field of the micro-nano structure based on the quantum state wave function; calculate the quantum tunneling current density based on the potential barrier height and potential barrier width of the micro-nano structure; Step 2: Map the quantum wave state function, quantum stress field and quantum tunneling current density to the state space of reinforcement learning to obtain the state vector of reinforcement learning; generate optimization actions based on the state vector; update the parameters of the micro-nano structure according to the optimization actions; calculate the structural performance of the micro-nano structure after the parameters of the micro-nano structure are updated; Step 3: Based on the rate of change of structural performance, establish a convergence criterion based on Lyapunov stability to determine whether it has converged. If so, terminate and complete this optimization design. If not, return to step 2.

2. The micro-nanostructure optimization design method based on deep reinforcement learning according to claim 1, characterized in that: In step 1, the method for obtaining the electron density distribution includes: using a transmission electron microscope to obtain the electron density distribution through high-resolution imaging technology; using a scanning tunneling microscope to measure the local electron state density, and mapping the electron density distribution through tunneling current; or performing an X-ray scattering experiment to cause X-rays to scatter electrons in the micro-nanostructure material, and the resulting scattering pattern is restored to the electron density distribution through Fourier transform.

3. The micro-nanostructure optimization design method based on deep reinforcement learning according to claim 2, characterized in that: In step 1, the quantum state wave function of the micro-nano structure is constructed by the following formula: Where Ψ(r, t) represents the quantum state wave function of the micro-nanostructure at time t and spatial position r; ρ(r) represents the electron density distribution of the micro-nanostructure at spatial position r; E n represents the nth energy level in the quantum system; k B is the Boltzmann constant; T is the temperature; Indicates energy level E n Probability distribution under thermal equilibrium conditions; is the reduced value of Planck's constant; m * is the equivalent mass of electrons in micro-nanostructured materials; ψ n (r, t) is the wave function representing the nth quantum state of the micro-nanostructure at time t and spatial position r; is the Laplace operator; N is the number of energy levels.

4. The micro-nanostructure optimization design method based on deep reinforcement learning according to claim 3, characterized in that: In step 1, the quantum stress field of the micro-nano structure is calculated based on the quantum state wave function using the following formula: in, Represents the component of the i-th row and j-th column at the spatial position r; i and j are indexes of the spatial coordinate direction, and their values ​​are 1, 2, and 3, corresponding to the X-axis, Y-axis, and Z-axis directions of the three-dimensional rectangular coordinate system, respectively; when i=j, represents normal stress; when i≠j, represents shear stress; * (r, t) is the conjugate of Ψ(r, t); x i represents the i-th spatial coordinate component in the micro-nano structure; x j represents the jth spatial coordinate component in the micro-nano structure; x1 represents the X-axis coordinate component; x2 represents the Y-axis coordinate component; x3 represents the Z-axis coordinate component.

5. The micro-nanostructure optimization design method based on deep reinforcement learning according to claim 4, characterized in that: In step 1, the quantum tunneling current density is calculated according to the barrier height and barrier width of the micro-nano structure by the following formula: Among them, J quantum is the quantum tunneling current density; e is the electron charge; E F is the Fermi level; T(E) is the transmission coefficient; f(·) is the Fermi-Dirac distribution function; Φ is the barrier height; d is the barrier width.

6. The micro-nanostructure optimization design method based on deep reinforcement learning according to claim 4, characterized in that: In step 2, the state vector of reinforcement learning is calculated by the following formula: Where Ω is the structural domain of the micro-nanostructure; λ is the average interface thickness of the micro-nanostructure; S t represents the state vector at time t.

7. The micro-nanostructure optimization design method based on deep reinforcement learning according to claim 5, characterized in that: In step 2, let the model parameter of reinforcement learning be θ, the learning rate be β, and the policy function of the model parameter θ be π θ ; Use the following formula to generate the optimized action based on the state vector: Among them, A t represents the optimization action at time t; W is the weight matrix of reinforcement learning; σ is the standard deviation of Gaussian distribution; the model parameter θ of reinforcement learning satisfies the following update rule: Among them, F obj The structural properties of micro-nano structures; Represents the gradient of the model parameter θ of reinforcement learning; represents the gradient of the state vector; V is the value function; H(·) represents the system Hamiltonian.

8. The micro-nanostructure optimization design method based on deep reinforcement learning according to claim 7, characterized in that: The structural properties of the micro-nano structure are calculated using the following formula: Among them, ε ij express The corresponding strain tensor; Y represents Young's modulus; γ represents the surface energy coefficient; κ represents the gradient energy coefficient; G s represents the surface Gibbs free energy; E g Represents the energy gap width.

9. The micro-nanostructure optimization design method based on deep reinforcement learning according to claim 8, characterized in that: In step 2, the parameter of the micro-nano structure is a vector, whose elements include: Young's modulus, barrier height, barrier width, transmission coefficient and average interface thickness.

10. The micro-nanostructure optimization design method based on deep reinforcement learning according to claim 9, characterized in that: In step 3, based on the rate of change of structural performance, the convergence criterion based on Lyapunov stability is established through the following formula: Where R is the gas constant; ΔG is the gradient of the surface Gibbs free energy; when the convergence criterion Λ is less than the set convergence threshold ∈, it is judged to be converged.

Images