A two-step reinforcement learning method for constructing an adiabatic environment to simulate crystal structures.

Abstract

This invention provides a two-step reinforcement learning method for constructing an adiabatic environment to simulate crystal structures, belonging to the fields of reinforcement learning and materials synthesis technology. Because this method simulates the process of preparing crystalline materials using a two-step method under adiabatic conditions, it searches for possible phase transition paths that the crystal may undergo during the experiment by approximating the relationship between the phase transition process of the crystal structure and the energy supply. Specifically, this method first obtains all possible structures of the crystal material undergoing phase transitions to construct the simulation environment. Then, it processes the specific information contained in all possible structural information, performs structural transformation, and formulates transformation rules to simulate the phase transition process of the crystal material. Therefore, this method can effectively assist experiments, reduce the number of experiments, lower costs, and has high economic value for the discipline of materials synthesis.

Description

Technical Field

[0001] This invention belongs to the field of reinforcement learning and materials synthesis technology, specifically providing a two-step method for constructing an adiabatic environment for reinforcement learning that simulates crystal structures. Background Technology

[0002] Crystals with different structures possess different properties. In the process of material synthesis, we need to prepare materials with crystal structures that have a certain optimal performance. The method is to transform an existing crystal structure into the target crystal structure through a phase transition process. This process requires verification through multiple experiments, but the cost of these experiments is too high.

[0003] An adiabatic environment generally refers to an ideal simulated environment, i.e., one that does not consider energy dissipation from interaction with the external environment. The two-step preparation process involves injecting raw materials in batches. First, a portion of the raw material is added, and intermediate structural terms are formed through a low-temperature isothermal and annealing process. Then, another portion of the raw material is added, and the process is slowly heated to maintain a constant temperature, finally cooling at a suitable rate. The reaction process is easy to control, but there is currently no simulation algorithm for the two-step synthesis of crystalline materials. Summary of the Invention

[0004] This invention is made to solve the above-mentioned problems, and aims to provide a two-step reinforcement learning adiabatic environment construction method for simulating crystal structure.

[0005] In a first aspect, the present invention provides a two-step reinforcement learning adiabatic environment construction method for simulating crystal structure, characterized by: acquiring all possible structures of a crystal material undergoing a phase transition process; processing the specific information contained in all possible structures, performing structural transformation, and defining transformation rules.

[0006] The method provided by this invention also has the following features: the transformation rules include a first transformation rule and a second transformation rule. The first transformation rule includes setting a transformation radius. The second transformation rule, based on the first transformation rule, includes randomly selecting a transformable structural point and determining the available energy value of the transformable structural point. The formula for calculating the available energy value of the transformable structural point is as follows: available energy value = energy supply value of the simulated environment + difference between the initial energy values ​​of the structure before and after the transformation.

[0007] The method provided by this invention also has the following feature: the initial energy supply value E0 of the simulated environment ranges from 0 to E0. max E max It represents the maximum difference in initial energy values ​​across all possible structures.

[0008] The method provided by this invention also has the following feature: wherein the initial energy supply value of the simulated environment...

[0009] The method provided by this invention also has the following characteristics: the process of preparing crystalline materials using a two-step method under simulated adiabatic conditions includes a first simulation stage, a second simulation stage, a third simulation stage, a fourth simulation stage, and a fifth simulation stage. In the first simulation stage, the energy supply value of the simulated environment is E1 = E0; in the second simulation stage, the energy supply value of the simulated environment is E2 < E0; and in the third simulation stage, the energy supply value of the simulated environment is E2 ≤ E3 < E0. max In the fourth simulation phase, the energy supply value of the simulated environment is E4 = E max In the fifth simulation phase, the energy supply value of the simulated environment, E5, is less than E. max The energy supply values ​​of the simulated environment in the second and fifth simulation phases decay in a linear manner.

[0010] The method provided by this invention also has the following feature: if the available energy value of the transformable structural point is positive, the transformation is successful; if the available energy value of the transformable structural point is negative, it remains unchanged, and other transformable structural points are searched within the set transformation radius.

[0011] The method provided by this invention also has the following feature: if the starting structure and the ending structure are known, the structure transformation approaches or reaches the ending structure by setting a reward function, wherein the reward function includes a reward function set according to the reinforcement learning environment; or the reward function includes a reference value for the reward based on the energy absorption value or energy release value during the phase transition process.

[0012] Secondly, the present invention provides a reinforcement learning adiabatic environment construction device for simulating a two-step method of crystal structure, characterized by comprising: a crystal material all possible structure acquisition unit, used to acquire all possible structures of the crystal material during the phase transition process; and a transformation rule formulation unit, used to process the specific information contained in all possible structures, perform structural transformation, and formulate transformation rules.

[0013] Thirdly, the present invention provides an interaction method for a reinforcement learning environment, characterized by: mapping all possible structures corresponding to the reinforcement learning environment into state representations and then interacting with a reinforcement learning agent; or encoding all possible structures corresponding to the reinforcement learning environment into quantum states and mapping them into state representations and then interacting with a reinforcement learning agent, wherein the reinforcement learning environment is constructed by the two-step reinforcement learning adiabatic environment construction method of simulating crystal structures of any of the above-mentioned methods.

[0014] The method provided by this invention also has the following features: the quantum state encoding includes: normalizing the high-dimensional data of each possible structure to obtain a normalized vector; processing the normalized vector to obtain the quantum state right vector; conjugating and transposing the quantum state right vector to obtain the corresponding quantum state left vector; and taking the outer product of the quantum state right vector and the quantum state left vector to obtain the structure density matrix obtained after quantum state encoding. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the overall process of building and executing a two-step method for simulating crystal structures in reinforcement learning adiabatic environments according to an embodiment of the present invention.

[0016] Figure 2 This is a first structural schematic diagram of a visualized two-dimensional plane according to an embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of the second structure of a visualized two-dimensional plane according to an embodiment of the present invention;

[0018] Figure 4 This is a schematic diagram of the third structure of a visualized two-dimensional plane according to an embodiment of the present invention;

[0019] Figure 5 This is a line graph illustrating the energy replenishment value of the simulated environment during the two-step heating process in an embodiment of the present invention.

[0020] Figure 6 This is a schematic diagram of the quantum encoding process in an embodiment of the present invention.

[0021] The role and effect of invention

[0022] The reinforcement learning adiabatic environment construction method for simulating crystal structures provided by the present invention searches for possible phase transition paths of crystals during experiments by approximately establishing the relationship between the phase transition process of crystal structure and the energy supply. Specifically, the method first obtains all possible structures of crystal materials undergoing phase transition processes to construct a simulation environment, and then processes the specific information contained in all possible structure information to perform structural transformation and formulate transformation rules, thereby simulating the phase transition process of crystal materials. Therefore, this method can effectively assist experiments, reduce the number of experiments and lower costs, and has high economic value for the discipline of materials synthesis.

[0023] The reinforcement learning adiabatic environment construction device for simulating crystal structure using a two-step method provided by the present invention obtains the intermediate structure of the changing process by simulating the phase transition process of crystal structure using computer simulation. The device includes a crystal all-possible structure acquisition unit for acquiring all possible structures of crystal material undergoing phase transition process; and a transformation rule formulation unit for processing the specific information contained in all possible structures, performing structural transformation and formulating transformation rules. Therefore, the device can simulate the phase transition process of crystal material structure, reduce experimental costs, and has high application prospects in the field of materials science.

[0024] According to the interaction method of the reinforcement learning environment provided by the present invention, since the method interacts and learns with the reinforcement learning agent built by the present invention, and encodes all possible structures constituting the reinforcement learning environment into quantum states, and then maps them into state representations and interacts with the reinforcement learning agent, this interaction method can use quantum circuits to replace classical neural networks as decision agents, thereby improving the decision efficiency of quantum reinforcement algorithms. Detailed Implementation

[0025] To make the technical means, creative features, objectives and effects of this invention easy to understand, the following describes in detail, with reference to embodiments and accompanying drawings, a method, apparatus and interaction method for constructing a two-step reinforcement learning adiabatic environment to simulate crystal structure provided by this invention.

[0026] <Example>

[0027] Figure 1 This is the overall flowchart of the reinforcement learning adiabatic environment construction and execution method of the two-step method for simulating crystal structure provided in the embodiments of the present invention.

[0028] like Figure 1 As shown, the method for building and executing a two-step reinforcement learning adiabatic environment (hereinafter referred to as reinforcement learning environment) to simulate crystal structure in this embodiment of the invention includes the following steps:

[0029] Step S1: Obtain all possible structures for the crystal to undergo a phase transition, as well as the atomic three-dimensional spatial coordinates and initial energy values ​​corresponding to each structure.

[0030] Step S2 involves reducing the dimensionality of the atomic three-dimensional spatial coordinates of each structure to obtain the corresponding dimensionality-reduced two-dimensional array, which is then visualized.

[0031] Step S3: Based on the dimensionality reduction coordinates of each structure corresponding to the two-dimensional data points in the visualized two-dimensional plane, set the range of the transformation radius, that is, set the first transformation rule.

[0032] Step S4: Based on the set range of transformation radius, randomly select a transformable structural point within the radius region of a certain structural point, determine the available energy value of the transformable structural point, that is, set the second transformation rule.

[0033] Step S5: Determine whether the available energy value of the transformable structure point is positive.

[0034] Step S6: If the available energy value of the transformable structural point is positive, the transformation is successful, and the system switches to the region of the transformable structural point and the transformation radius to continue structural transformation.

[0035] Step S7: If the available energy value of the transformable structure point is negative, then keep the structure unchanged and return to step S4.

[0036] Step S8: Determine whether the successfully transformed transformable structure point is the endpoint structure or whether the maximum number of transformations is met. If it is met, the structure transformation ends; if it is not met, return to step S4 until the transformable structure point is the endpoint structure or the maximum number of transformations is met.

[0037] The reinforcement learning environment built in this embodiment is used to simulate the possible structural phase transition paths of materials in the synthesis experiment. The overall idea of ​​building the environment is to generate all possible material structures with structural information and perform structural transformations within them, while defining the specific rules for the transformations.

[0038] Furthermore, this embodiment uses DFT (Density Functional Theory) calculations to obtain all structural information of a crystal cell, including at least atomic three-dimensional spatial coordinate information and initial energy value information. The three-dimensional spatial coordinate information of each cell structure is processed into a set of high-dimensional data. Then, each set of high-dimensional data is used to calculate a set of two-dimensional data using a manifold learning dimensionality reduction algorithm. The purpose of this process is to find a reasonable low-dimensional feature representation by maintaining a certain invariant feature quantity between the high-dimensional and low-dimensional data, which is beneficial for visualization while reducing computational requirements. This embodiment uses a manifold learning dimensionality reduction algorithm to reduce high-dimensional data to two-dimensional data. In other embodiments, existing dimensionality reduction algorithms can also be used for dimensionality reduction processing. The two-dimensional data points of all structures obtained after dimensionality reduction are saved as an array, with each data point representing a material structure. All data points are visualized on a two-dimensional planar graph. The maximum difference between the initial energy values ​​of all structure points is calculated as a reference value for energy replenishment in the simulated phase transition environment. The energy value replenished in each step of the simulated environment is adjusted according to the heating method used in the actual experimental material synthesis.

[0039] The properties of the simulation environment defined in this embodiment are as follows:

[0040] Furthermore, this embodiment specifies the relationship between points (two-dimensional data points): the closer two points are on the visualized two-dimensional plane, the higher their similarity.

[0041] Furthermore, this embodiment defines the meaning of transformation: transformation refers to the change of a crystal from one structure to another, corresponding to the phase transition process of materials in reality. This embodiment sets two transformation rules to realize the structural transformation, namely the first transformation rule and the second transformation rule.

[0042] See Figure 2 The first transformation rule is to set the transformation radius. In this embodiment, the range of the transformation radius is set based on the maximum difference of the dimensionality reduction coordinates corresponding to all possible structures. Figure 2 Let X1 and Xn be the minimum and maximum values ​​of the dimension-reduced coordinates among all possible structures, respectively. The range of the transformation radius is set based on the maximum difference X between them (not shown in the figure). Figures 2-4 Mid-radius The specific transformation radius can be set randomly according to the specific situation. This embodiment includes other transformation radii set based on the maximum difference X.

[0043] Furthermore, this embodiment specifies the variable range: the radius value r of the variable range is determined based on the size of the visualized two-dimensional planar diagram, the density of points on the diagram, and the material synthesis heating method. During the structural transformation process, the radius r scales up or down with the amount of heating energy replenishment. The scaling method of the r value is adjusted through multiple simulations according to the accuracy requirements.

[0044] Furthermore, in this embodiment, based on the first transformation rule, a second transformation rule is set to determine whether the structural transformation is successful. The second transformation rule set in this embodiment is to randomly select a transformable structural point and determine the available energy value of the transformable structural point. If the available energy value of the transformable structural point is positive, the transformation is successful; if the available energy value of the transformable structural point is negative, it remains unchanged, and other transformable structural points are searched within the set transformation radius.

[0045] Furthermore, the expression for the available energy value of any transformable structure point is as follows:

[0046] Available energy value = energy replenishment value of simulated environment + difference between initial structural energy value before and after transformation.

[0047] In this embodiment, the maximum difference between the initial energy values ​​of all possible structures is used as a reference for replenishing the energy of the simulated environment.

[0048] Specifically, this embodiment sets initial energy values ​​for all possible structural information from the outset, and also sets initial energy values ​​for the simulation environment. See [link to documentation]. Figure 2Using the first possible structure (initial possible structure 0) as the center and a predefined phase transition radius (r0) as the search radius, an arbitrary variable structure search is performed. If the randomly searched possible structure (the possible structure where 1 is located) is within the search radius r0, and the available energy value of the possible structure where 1 is located is positive, then the structure transformation is successful, and the structure where 1 is located is the first transformable structure point. If the available energy value of the randomly searched possible structure is negative, then the structure remains unchanged, and other transformable structure points are searched within the predefined transformation radius until the available energy value of the first transformable structure point is found to be positive. A reward is then set according to the predefined reward function to make the structure transformation approach or reach the endpoint structure. Figure 2 In the diagram, r0 and r1 are the search radii, and r0 = r1. 1 is the first transformable structure point, and 2 is the second transformable structure point obtained in the same way as 1.

[0049] Furthermore, the energy replenishment value for the simulated environment in this embodiment is shown below. Figure 5 The diagram shown is a broken line diagram, as follows: Figure 5 As shown, the simulated environmental energy supply value E i The range is 0 < E i ≤E max i = 0, 1, 2, 3, 4, 5, E max The maximum difference between the initial energy values ​​set for all possible structural information. The energy supply value of the simulation environment during the first simulation phase 0-T1. During the simulation phase 0-T1, the energy supply value E1 of the simulated environment remains constant. In the second simulation phase T1-T2, the energy supply value E2 < E1, showing a decreasing trend. In the third simulation phase T2-T3, the energy supply value E3 of the simulated environment slowly increases from the energy supply value E2 of the second simulation phase to E1. max In the fourth simulation phase T3-T4, the energy supply value of the simulated environment is E4 = E max The energy supply value E2 and E5 of the simulated environment remains unchanged during simulations T3-T4; in the fifth simulation stage T4-T5, the energy supply value E5 of the simulated environment is attenuated from the energy supply value E5 of the simulated environment in the fourth simulation stage. In this embodiment, the energy supply values ​​E2 and E5 of the simulated environment in the second simulation stage T1-T2 and the fifth simulation stage T4-T5 decay linearly. The decay method can be linear with the number of steps, or it can be decayed in a way that conforms to the specific experiment. The number of steps is determined according to the specific synthetic material. In other embodiments, other decay methods may also be used.

[0050] See Figure 3 , Figure 3In this example, the circle containing 0 represents the initial structure, and the circle containing 1 represents the first transformable structure point. In this embodiment, the initial structure 0 is used as the center, and a set transformation radius r0 (r0 is referenced) is used. Figure 2 The search radius is set to r0 for structure searching. The two possible structure points indicated by the dashed arrows in the diagram are within the transformation radius r0. Assuming the available energy values ​​of these two possible structure points are both negative, the initial structure 0 is kept unchanged, and the first transformable structure point is searched using the same transformation radius r0. If the available energy value of structure 1 is positive, then this structure point is the first transformable structure point. Subsequently, following the same method, the second transformable structure point is searched using the first transformable structure point (structure 1) as the center and the set phase transition radius r1 (r1 = r0) as the search radius. Based on the set reward function R, the search proceeds to or near the endpoint structure (not shown). In this embodiment... Figure 2 and Figure 3 A schematic diagram of the phase transition path that can be used as a known starting and ending structure.

[0051] Furthermore, this embodiment sets a reward function based on the phase transition environment. This reward function is used to evaluate the transformation of the crystal structure. A good transformation will give the reinforcement learning agent positive feedback, and a bad transformation will give the reinforcement learning agent negative feedback. Specifically, the quality of the transformation is defined according to the tendency of the crystal material to undergo phase transition in the experiment. In addition, the reinforcement learning algorithm influences how the agent chooses the transformation path through iteration. The transformation path selected by the algorithm that converges after multiple iterations often conforms to the process of phase transition of crystal materials in the real environment. The specific method of setting the reward function varies depending on whether there is a specific known starting structure and ending structure.

[0052] Furthermore, if the starting and ending structures are known, a reward function is set according to a typical reinforcement learning environment, and is divided into a first type of reward function and a second type of reward function. The first type of reward function sets the reward for a successful structure transformation to 1, and the reward for any intermediate transition structures to 0, to facilitate rapid algorithm iteration in finding a reasonable phase transition path. The second type of reward function can use the energy absorption or release value during the phase transition process as a reference value for reward setting. Although this embodiment lists the above two specific reward functions, the specific reward function settings can be set according to the phase transition environment. This embodiment also includes other reward functions in the prior art.

[0053] See Figure 4 If the starting and ending structures are unknown, since each round of structural transformation sets a maximum number of transformations, such as... Figure 4If possible structure 0 is any randomly selected possible structure that is not the starting structure, then a structure search is performed by transforming the radius r0. Assuming the structure search is performed in a counterclockwise direction, the available energy values ​​of the possible structures pointed to by the dashed arrows in the figure are all negative. If the possible structure where 1 is located has reached the preset maximum number of times (threshold) of this round of structure transformation, then regardless of whether the available energy value of structure 1 is negative, structure 1 is the ending structure, and this round of structure transformation terminates.

[0054] Furthermore, the phase transition environments simulated in this embodiment include absolutely free environments, adiabatic environments, and dissipative environments. The first type, the absolutely free environment, does not consider the case of insufficient phase transition energy, allowing the crystal to freely search for phase transition paths in the structural space, thus imposing the lowest degree of constraint on the transformation rules. The second type simulates an adiabatic environment, which only considers the interaction between the crystal and the energy supply source, constraining the structural transformation based on the total energy of both. The third type is a dissipative environment, which considers not only the relationship between the crystal and the energy supply source but also the energy dissipation generated by both in the environment, thus imposing the highest degree of constraint on the transformation rules.

[0055] On the other hand, this embodiment also provides an interaction method for a reinforcement learning environment, which maps all possible structures corresponding to the reinforcement learning environment built above to state representations, interacts and learns with the reinforcement learning agent, and updates the state representations based on the actions output by the reinforcement agent.

[0056] Furthermore, in this embodiment, each state(s) in quantum reinforcement learning is defined as a structural representation, and all structures constitute the state set of the simulated phase transition system. At a certain moment, a structure is obtained from the constructed reinforcement learning environment, which is mapped to an input state representation and input to the reinforcement learning agent. Based on the action output by the reinforcement learning agent, the crystal structure changes from one state to another, and the algorithm is iteratively updated based on the feedback obtained.

[0057] Furthermore, in this embodiment, high-dimensional data from the established reinforcement learning environment can be input into a dimensionality reduction algorithm for dimensionality reduction before being input into the reinforcement learning algorithm to interact with the reinforcement learning agent. However, due to the high-dimensional complexity of material structure data, the dimensionality reduction algorithm may not be able to fully express the similarity between different structures. In this case, it is necessary to directly perform computational processing on a large amount of high-dimensional data of structures. Direct computation on high-dimensional data requires enormous computing power, while quantum computers can meet the computing power requirements. Therefore, this embodiment also provides a quantum state encoding method for representing structural data.

[0058] Figure 6 This is a schematic diagram of the quantum encoding process according to an embodiment of the present invention.

[0059] See Figure 6 Quantum encoding includes the following steps:

[0060] Step S102: First, normalize the high-dimensional data of the straightened structure with length n to obtain a normalized vector.

[0061] Step S104: Transform the normalized vector into complex form, i.e., the right vector of the quantum state in n*1 dimensions.

[0062] Step S106: Take the conjugate transpose of the n*1 dimensional quantum state right vector to obtain the 1*n dimensional quantum state left vector.

[0063] Step S108: Perform an outer product of the n*1 dimensional quantum state right vector and the 1*n dimensional quantum state left vector to obtain an n*n dimensional quantum state density matrix, thus completing the quantum state encoding.

[0064] The role and effect of the embodiments

[0065] The reinforcement learning adiabatic environment construction method for simulating crystal structures provided in this embodiment searches for possible phase transition paths of crystals during experiments by approximately establishing the relationship between the phase transition process of crystal structure and the energy supply. Specifically, this method first obtains all possible structures of crystal materials undergoing phase transition processes to construct a simulation environment, and then processes the specific information contained in all possible structure information to perform structural transformation and formulate transformation rules, thereby simulating the phase transition process of crystal materials. Therefore, this method can effectively assist experiments, reduce the number of experiments and lower costs, and has high economic value for the discipline of materials synthesis.

[0066] Furthermore, this embodiment simulates the process of preparing crystal materials using a two-step method under adiabatic conditions. By simulating the replenishment value of environmental energy and the phase transition process of crystal structure at different stages, the method provided in this embodiment can be simulated according to the intermediate target items of crystal synthesis, and has the advantages of stronger simulability and simulation results that are more in line with experimental laws.

[0067] The foregoing description and accompanying drawings have provided typical embodiments of specific structures for specific implementations. The above-described invention presents preferred embodiments, but these are not intended to be limiting. Various changes and modifications will undoubtedly be apparent to those skilled in the art after reading the foregoing description. Therefore, the appended claims should be considered to cover all changes and modifications that encompass the true intent and scope of the invention. Any and all equivalent scope and content within the scope of the claims should be considered to still fall within the intent and scope of the invention.

Claims

1. A two-step method for constructing an adiabatic environment for reinforcement learning to simulate crystal structures, characterized in that, include: To obtain all possible structures of crystalline materials during phase transition processes; After processing the specific information contained in all possible structures, structural transformation is performed and transformation rules are defined. The transformation rules include a first transformation rule and a second transformation rule. The first transformation rule includes setting the transformation radius; The second transformation rule, based on the first transformation rule, includes randomly selecting a transformable structural point and determining that... Available energy value of the transformable structural point; the calculation formula for the available energy value of the transformable structural point is as follows: Available energy value = Energy supply value of the simulated environment + Difference between the initial energy values ​​of the structure before and after the transformation; The initial energy supply value E0 of the simulated environment has the following range: 0 < E 0 < E max E max The maximum difference in the initial energy values ​​of all possible structures; Wherein, the initial energy supply value of the simulated environment The process of preparing crystalline materials using a two-step method under simulated adiabatic conditions includes a first simulation stage, a second simulation stage, a third simulation stage, a fourth simulation stage, and a fifth simulation stage. In the first simulation phase, the energy supply value of the simulated environment is E1 = E0. In the second simulation phase, the energy supply value of the simulated environment is E2 < E0. In the third simulation phase, the energy supply value E3 of the simulated environment is in the range of E2 ≤ E3 < E max , In the fourth simulation phase, the energy supply value of the simulated environment is E4 = E max , In the fifth simulation phase, the energy supply value of the simulated environment is E5 < E max , The attenuation mode of the energy supply value of the simulated environment in the second simulation stage and the fifth simulation stage includes linear attenuation.

2. The method according to claim 1, characterized in that: in, If the available energy value of the transformable structural point is positive, the transformation is successful; If the available energy value of the transformable structure point is negative, it remains unchanged, and other transformable structure points are searched within the set transformation radius.

3. The method according to claim 1, characterized in that: If the starting structure and the ending structure are known, the structure transformation approaches or reaches the ending structure by setting a reward function. Wherein, the reward function includes a reward function set according to the reinforcement learning environment; or The reward function includes a reference value for the reward based on the energy absorbed or released during the phase transition process.

4. An interactive method for a reinforcement learning environment, characterized in that: After mapping all possible structures corresponding to the reinforcement learning environment into state representations, they interact with the reinforcement learning agent; or All possible structures corresponding to the reinforcement learning environment are encoded into quantum states, mapped to state representations, and then interacted with the reinforcement learning agent. The reinforcement learning environment is constructed by the two-step reinforcement learning adiabatic environment construction method based on the simulated crystal structure as described in any one of claims 1-3.

5. The method according to claim 4, Its features are: The quantum state encoding includes: Normalize the high-dimensional data for each possible structure to obtain a normalized vector; The normalized vector is processed to obtain the quantum state right vector; The corresponding left quantum state is obtained by conjugating and transposing the right quantum state vector. The structure density matrix obtained by encoding the quantum state is obtained by taking the outer product of the right vector of the quantum state and the left vector of the quantum state.

Images