Model optimization method, model, device, storage medium, and program product

Abstract

The application discloses a model optimization method, a model, a device, a storage medium and a program product, relates to the technical field of artificial intelligence, and comprises the following steps: inputting molecular structure data into a pre-trained molecular generation model to perform sampling, generating a molecular conformation, and recording old policy transition probabilities and sampling trajectories of each sampling time step; calculating atomic force information of the molecular conformation through a machine learning force field, determining a reward value according to the fact that the smaller the force is, the higher the reward value is; taking the molecular generation model as a to-be-optimized model, determining new policy transition probabilities based on the sampling trajectories; performing parameter updating on the to-be-optimized model according to the old policy transition probabilities, the new policy transition probabilities and the reward value, obtaining an updated molecular conformation generation model, taking the updated molecular conformation generation model as a molecular generation model of next iteration, repeating sampling, calculating a reward value and updating, and stopping until a preset iteration termination condition is met, so that a target generation model for generating initial molecular coordinates for structure optimization by the machine learning force field is obtained.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to model optimization methods, target generation models, model optimization devices, storage media, and program products. Background Technology

[0002] In recent years, machine learning force field methods have demonstrated high computational efficiency in molecular structure optimization, accelerating the optimization process to some extent compared to traditional quantum chemical methods. However, for large molecular systems with a large number of atoms, even with machine learning force field methods, structure optimization still requires considerable computation time, and there is still room for further improvement in overall efficiency. On the other hand, generative models based on deep learning, especially diffusion models, have made significant progress in generating three-dimensional molecular structures. However, their optimization objectives typically focus on distribution fitting and structural rationality, lacking direct constraints on the stable low-energy states of molecules. Therefore, how to combine generative models to directly obtain molecular structures close to stable low-energy states, thereby reducing the computational cost and iteration count of subsequent structure optimization, remains an unresolved issue with significant research and application value. Summary of the Invention

[0003] The main objective of this application is to provide a model optimization method, a target molecule generation model, a model optimization device, a storage medium, and a program product, aiming to solve the technical problem of accelerating the process of molecular structure optimization based on diffusion models and machine learning force fields.

[0004] To achieve the above objectives, this application proposes a model optimization method, the method comprising: Acquire pre-trained molecular generation models and molecular structure data; The molecular structure data is input into the molecular generation model for sampling to generate molecular conformations, and the old strategy transition probability and sampling trajectory of the intermediate state of the molecular structure at each sampling time step are recorded during the sampling process. The atomic force information of the molecular conformation is calculated by machine learning force field, and the reward value corresponding to the sampling process is determined based on the atomic force information. The smaller the atomic force information, the higher the reward value. The molecular generation model is used as the model to be optimized; based on the sampling trajectory, the new strategy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step is determined; Based on the old strategy transition probability, the new strategy transition probability, and the reward value, the model parameters of the model to be optimized are updated to obtain an updated molecular conformation generation model. The updated molecular conformation generation model is used as the molecular generation model for the next iteration, and the step of inputting the molecular structure data into the molecular generation model for sampling is performed until the preset iteration termination condition is met to obtain the target generation model. The target generation model is used to generate initial molecular coordinates for structural optimization by machine learning force field.

[0005] In one embodiment, the step of calculating the atomic force information of the molecular conformation through a machine learning force field and determining the reward value corresponding to the sampling process based on the atomic force information includes: The force information of each atom in the molecular conformation is compared with the preset target force information to obtain the force deviation information; The reward value for generating the molecular conformation by the molecular generation model is determined based on the force deviation information.

[0006] In one embodiment, the step of determining the new policy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step based on the sampling trajectory includes: Obtain the sampling time step of each intermediate state during the sampling process; Based on the intermediate states corresponding to the current sampling time step and the previous sampling time step in the sampling trajectory, the new strategy transition probability of each intermediate state at each sampling time step is determined.

[0007] In one embodiment, the step of updating the model parameters of the model to be optimized based on the old strategy transition probability, the new strategy transition probability, and the reward value to obtain the updated molecular conformation generation model includes: The sampling ratio is obtained based on the ratio of the transition probability of the new strategy to the transition probability of the old strategy; Based on the sampling ratio and the reward value, the loss function of the model to be optimized is constructed. Based on the loss function, the model parameters of the model to be optimized are updated to obtain the updated molecular conformation generation model.

[0008] In one embodiment, after the step of determining the reward value corresponding to the sampling process based on the atomic force information, the method further includes: In each iteration, based on the sampling trajectory, the new strategy transition probability of the model to be optimized under the current model parameters is determined; Based on the new strategy transition probability, the old strategy transition probability, and the reward value under the current model parameters, the current model parameters of the model to be optimized are updated until a preset number of updates is reached, and then the next iteration begins.

[0009] In one embodiment, the step of obtaining the pre-trained molecular generation model includes: Obtain the basic molecular generation model and molecular structure training set; The basic molecular generation model is pre-trained using the molecular structure training set to obtain the pre-trained molecular generation model.

[0010] Furthermore, to achieve the above objectives, this application also proposes a target generation model, which is used to generate corresponding molecular conformations based on the molecular structure data, wherein the molecular conformations are initial molecular coordinates for structural optimization by machine learning force fields.

[0011] Furthermore, to achieve the above objectives, this application also proposes a model optimization apparatus, which includes: The acquisition module is used to acquire pre-trained molecular generation models and molecular structure data; The sampling module is used to input the molecular structure data into the molecular generation model for sampling, generate molecular conformations, and record the old strategy transition probability and sampling trajectory of the intermediate state of the molecular structure at each sampling time step during the sampling process. The reward module is used to calculate the atomic force information of the molecular conformation through machine learning force field, and determine the reward value corresponding to the sampling process based on the atomic force information, wherein the smaller the atomic force information, the higher the reward value; The determination module is used to select the molecular generation model as the model to be optimized; and based on the sampling trajectory, determine the new strategy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step. An update module is used to update the model parameters of the model to be optimized based on the old strategy transition probability, the new strategy transition probability, and the reward value, to obtain an updated molecular conformation generation model; the updated molecular conformation generation model is used as the molecular generation model for the next iteration, and the step of inputting the molecular structure data into the molecular generation model for sampling is executed until a preset iteration termination condition is met to obtain a target generation model, wherein the target generation model is used to generate initial molecular coordinates for structural optimization by machine learning force field.

[0012] In addition, to achieve the above objectives, this application also proposes a model optimization device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the model optimization method as described above.

[0013] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the model optimization method described above.

[0014] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the model optimization method described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: By acquiring pre-trained molecular generation models and molecular structure data, the molecular structure data is input into the molecular generation model for sampling to generate molecular conformations. The old strategy transition probability and sampling trajectory of the intermediate state of the molecular structure at each sampling time step are recorded. Then, the atomic force information of the generated molecular conformation is calculated through machine learning force field calculation, and the reward value corresponding to the sampling process is determined according to the rule that the smaller the atomic force, the higher the reward value. This quantifies the physical constraint of molecular stability (i.e., the requirement of low-energy state with force approaching zero) into an optimizable reward signal. Next, the molecular generation model is used as the model to be optimized. Based on the sampling trajectory, the new strategy transition probability of the model to be optimized at each sampling time step corresponding to the intermediate state is determined. The model parameters of the model to be optimized are updated according to the old strategy transition probability, the new strategy transition probability, and the reward value. This allows the molecular generation model to preferentially generate low-energy molecular conformations during the update process. At the same time, the new and old strategy transition probabilities are used to prevent the update step from being too large and maintain the ability to generate reasonable molecular structures.

[0016] Through multiple iterations, the updated molecular conformation generation model is used as the molecular generation model for the next iteration. Sampling, reward calculation, and parameter updates are repeated until a preset iteration termination condition is met, resulting in the target generation model. This process allows the target generation model to intrinsically learn how to directly output initial molecular coordinates close to a stable low-energy state when generating molecular conformations. This avoids the multiple iterative force calculations and coordinate adjustments required to reach a stable state in traditional machine learning force field methods, and also overcomes the deficiency of existing diffusion models that only focus on structural rationality and lack low-energy state constraints.

[0017] Therefore, when the initial molecular coordinates output by the target generation model are used by the machine learning force field for subsequent structure optimization, since the initial molecular coordinates are close to a stable low-energy state, the number of further optimization iterations required is greatly reduced, and the overall computational cost is significantly reduced. This effectively solves the technical problems of high computational cost and numerous iterations in molecular structure optimization in the prior art, and significantly improves the overall efficiency of molecular structure optimization. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the model optimization method of this application. Figure 2 A simplified flowchart is provided for another embodiment of the model optimization method of this application; Figure 3 This is a schematic diagram of the module structure of the model optimization device in this application; Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the model optimization method of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] Generating low-energy and stable molecular conformations is a crucial task in molecular design and drug development, significantly impacting fields such as new drug discovery, materials science, and biotechnology. It can significantly improve the efficiency of molecular design and the accuracy of molecular property prediction. In recent years, deep learning-based generative models, particularly diffusion models and geometric deep neural networks, have demonstrated immense potential in generating three-dimensional molecular structures. These generative models employ different approaches to generate the desired molecular structures. For example, ConfGF (Conformational Generation Framework) generates stable molecular conformations through kinetics, effectively reducing noise and errors; GeoDiff (Geometric Diffusion Model) generates stable molecular conformations through a reverse diffusion process; DMCG (Direct Molecular Coordinate Generation) proposes a method for directly predicting atomic coordinates, achieving more accurate three-dimensional molecular structure generation through iterative optimization of atomic and bond information; and DiSCO (Diffusion with Schrödinger Bridge Optimization) introduces the concept of the Schrödinger bridge, combining it with nonlinear diffusion methods to optimize 3D molecular conformations, significantly improving the physicochemical stability of the generated molecules. Currently, generative models mainly focus on the chemical rationality of molecular structures, and most of them are text-based graph models. Therefore, in terms of reinforcement learning fine-tuning, the reward function of conventional techniques uses aesthetic indicators as rewards to make the generative model generate molecular structures in the direction of high rewards. However, there is still some gap between these techniques and the actual implementation of molecular generation and diffusion models (i.e., the molecular structures generated by conventional techniques are chemically rational, but are not physically stable in a low-energy state).

[0025] It should be noted that the executing entity in this embodiment can be a model optimization device, such as a computing server, high-performance computing cluster, or dedicated scientific computing workstation deployed locally or in the cloud. It can also be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone. Alternatively, it can be an embedded device, edge computing node, or artificial intelligence device capable of performing the aforementioned functions. The following description uses a model optimization device as an example to illustrate this embodiment and the subsequent embodiments.

[0026] In addition, it should be noted that the model optimization device can also be used to generate molecular conformations corresponding to molecular structure data through the target generation model it generates. These molecular conformations serve as the initial molecular coordinates for structure optimization using machine learning force fields. The optimized molecular conformations can be used in downstream drug design or molecular dynamics simulation scenarios.

[0027] It is understood that this application may also provide a molecular conformation generation device for further generating molecular conformations corresponding to molecular structure data through the target generation model generated therefrom.

[0028] Based on this, the embodiments of this application provide a model optimization method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the model optimization method of this application.

[0029] In this embodiment, the model optimization method includes steps S1 to S5: Step S1: Obtain the pre-trained molecular generation model and molecular structure data; In one feasible embodiment, the model optimization device acquires a molecular generation model that has been pre-trained using large-scale molecular structure data, as well as molecular structure data for training or fine-tuning.

[0030] Optionally, the pre-trained molecular generation model is a deep learning model based on the diffusion process, which has learned to progressively reconstruct molecular structures that conform to physicochemical laws from random noise.

[0031] Alternatively, molecular structure data can be datasets consisting of a large number of known, validated stable molecular conformations, such as the QM9 (Quantum Machine) dataset, which contains atomic coordinates and atom type information for a variety of organic molecules.

[0032] Understandably, model optimization devices acquire models or data for subsequent reinforcement learning fine-tuning processes, in order to gradually improve the model's ability to generate low-energy stable molecular conformations.

[0033] Step S2: Input the molecular structure data into the molecular generation model for sampling, generate molecular conformations, and record the old strategy transition probability and sampling trajectory of the intermediate state of the molecular structure at each sampling time step during the sampling process. In one feasible embodiment, the model optimization device inputs the acquired molecular structure data into a pre-trained molecular generation model to perform a sampling operation. During this sampling process, it records the old strategy transition probability of the intermediate state of the molecular structure corresponding to each sampling time step, as well as the sampling trajectory composed of the molecular coordinates of all sampling time steps arranged in sequence.

[0034] Optionally, the sampling process refers to the process by which the molecular generation model generates molecular conformations step by step from the initial noise distribution according to the reverse diffusion process.

[0035] Optionally, in each sampling time step, the molecular generation model produces an intermediate state of the molecular structure. The model optimization device records the probability of transitioning from the state of the previous time step to the state of the current time step; this probability is the old strategy transition probability. Simultaneously, the model optimization device also saves the state sequence of all time steps throughout the sampling process, called the sampling trajectory. For example, if the sampling process contains 100 sampling time steps, then the sampling trajectory is an array consisting of 100 sets of atomic coordinates connected sequentially.

[0036] Alternatively, molecular conformation is the arrangement of atomic coordinates of a molecule in three-dimensional space.

[0037] It is understandable that recording the transition probabilities of the old policy and the sampling trajectory is used to facilitate subsequent comparisons of policy changes and calculation of reinforcement learning loss.

[0038] Step S3: Calculate the atomic force information of the molecular conformation through machine learning force field, and determine the reward value corresponding to the sampling process based on the atomic force information. The smaller the atomic force information, the higher the reward value. In one feasible embodiment, the model optimization device uses a machine learning force field to perform atomic force calculations on the sampled molecular conformation and calculates the reward value obtained by the molecular generation model in performing this sampling process. The reward rule is that the closer the calculated atomic force information is to the preset target force information, the higher the reward value allocated by the model optimization device for this sampling process.

[0039] Alternatively, the machine learning force field is a model trained using machine learning methods that can quickly predict the forces acting on each atom in a molecule.

[0040] Optionally, the model optimization device determines the reward value corresponding to this sampling process based on the calculated atomic force information. The reward value is a scalar, and its design principle is: the smaller the atomic force information, the higher the reward value; when the molecular conformation is close to a stable low-energy state, the atomic force approaches zero, at which point the reward value is the largest.

[0041] Alternatively, the force information of atoms reflects the degree of imbalance in the interactions between atoms within a molecule; the smaller the force, the more stable the molecular structure.

[0042] Understandably, in this way, the model optimization device transforms the physical stability of molecular structures into quantifiable reward signals, thereby guiding the molecular generation model to optimize towards generating more stable conformations.

[0043] Understandably, the model optimization device, by implementing the model optimization method provided in this application, combines a pre-trained diffusion model with reinforcement learning post-training, enabling the trained target generation model to directly generate initial molecular coordinates close to a stable low-energy state. These initial molecular coordinates serve as the low-energy initial state for subsequent structure optimization. Subsequently, the model optimization device, combined with a machine learning force field, performs further iterative optimization based on this low-energy initial state. Since the initial molecular coordinates used for iteration are those of a stable low-energy initial state, the number of further optimization iterations required is significantly reduced, and the overall computational cost is significantly lowered. This effectively solves the technical problems of high computational cost and numerous iterations in molecular structure optimization in the prior art, and significantly improves the overall efficiency of molecular structure optimization.

[0044] Step S4: Use the molecular generation model as the model to be optimized; based on the sampling trajectory, determine the new strategy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step. In one embodiment, the model optimization device uses the currently used molecular generation model as the model to be optimized, and recalculates the new policy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step based on the sampling trajectory obtained from previous sampling records. In this process, the initial model parameters of the model to be optimized are set to be the same as the model parameters of the molecular generation model, which can be regarded as the initial model parameters of the model to be optimized being initialized with the model parameters of the molecular generation model in the first iteration.

[0045] Optionally, the model to be optimized refers to the model that is fine-tuned through reinforcement learning. The model parameters of the model to be optimized will be continuously adjusted and updated during the training process.

[0046] Optionally, the model to be optimized and the molecular generation model adopt the exact same neural network layer design, the same mathematical definition of the diffusion process, and the same input and output data format. This ensures that the model to be optimized and the molecular generation model are completely consistent in behavior at the beginning of the first training iteration.

[0047] Optionally, the new policy transition probability is the probability that the model to be optimized will transition to the current state given the previous state. Specifically, the model to be optimized uses its current model parameters to recalculate the probability value of the state transition process recorded in the sampling trajectory, which is the transition from the coordinates of the previous time step to the coordinates of the current time step. This recalculated probability value is the new policy transition probability.

[0048] Optionally, unlike the old strategy transition probability, which is calculated based on the model parameters of the molecular generation model at the time of sampling, the new strategy transition probability is re-evaluated based on the model parameters of the current model to be optimized.

[0049] Understandably, model optimization devices can measure the impact of parameter changes on decision-making behavior after sampling by comparing the ratio of the transition probabilities of the new and old strategies, thus stabilizing the training process.

[0050] Step S5: Based on the old strategy transition probability, the new strategy transition probability, and the reward value, update the model parameters of the model to be optimized to obtain the updated molecular conformation generation model; use the updated molecular conformation generation model as the molecular generation model for the next iteration, and perform the step of inputting molecular structure data into the molecular generation model for sampling until the preset iteration termination condition is met to obtain the target generation model, wherein the target generation model is used to generate the initial molecular coordinates for structural optimization by machine learning force field.

[0051] In one feasible embodiment, the model optimization device updates the model parameters of the model to be optimized based on the old policy transition probability, the new policy transition probability, and the reward value, thereby obtaining an updated molecular conformation generation model. Specifically, the model optimization device constructs a loss function based on the old policy transition probability, the new policy transition probability, and the reward value. The loss function is calculated by multiplying the reward value by the ratio of the old and new policy transition probabilities, and then minimizing the loss using a gradient descent method, thereby maximizing the reward. That is, the real-time model parameters of the model to be optimized are iteratively updated using the loss function to obtain updated model parameters; and the model with the updated model parameters is called the updated molecular conformation generation model.

[0052] Optionally, after completing this round of updates, the model optimization device will use the current updated molecular conformation generation model as the molecular generation model for the next iteration. It will then repeatedly execute the cycle of inputting molecular structure data into the updated molecular conformation generation model for sampling, reward calculation, and model parameter updates until a preset iteration termination condition is met (e.g., reaching a set number of training rounds or reward value convergence). Specifically, steps S2 to S5 are performed cyclically. In each cycle, the molecular generation model of the current round is used for sampling and reward calculation, and the probability is calculated and model parameters are updated using the model to be optimized in the current round. This cycle continues until the preset iteration termination condition is met.

[0053] Optionally, the updated molecular conformation generation model obtained after satisfying the preset iteration termination condition is used as the target generation model. This target generation model is used to generate the initial molecular coordinates for subsequent structure optimization using the machine learning force field.

[0054] Understandably, when the initial molecular coordinates output by the target generation model are used by the machine learning force field for subsequent structure optimization, since the initial molecular coordinates are close to a stable low-energy state, the number of further optimization iterations required is greatly reduced, and the overall computational cost is significantly reduced. This effectively solves the technical problems of high computational cost and numerous iterations in molecular structure optimization in the prior art, and significantly improves the overall efficiency of molecular structure optimization.

[0055] This embodiment provides a model optimization method. It acquires a pre-trained molecular generation model and molecular structure data, inputs the molecular structure data into the molecular generation model for sampling, generates molecular conformations, and records the old strategy transition probability and sampling trajectory of the intermediate state of the molecular structure at each sampling time step. Then, it calculates the atomic force information of the generated molecular conformation using machine learning force fields, and determines the reward value corresponding to the sampling process based on the rule that the smaller the atomic force, the higher the reward value. This quantifies the physical constraint of molecular stability (i.e., the requirement of low-energy states with forces approaching zero) into an optimizable reward signal. Next, it uses the molecular generation model as the model to be optimized, determines the new strategy transition probability of the model in the intermediate state corresponding to each sampling time step based on the sampling trajectory, and updates the model parameters according to the old strategy transition probability, the new strategy transition probability, and the reward value. This allows the molecular generation model to preferentially increase the probability of generating low-energy molecular conformations during the update process, while using the old and new strategy transition probabilities to prevent excessively large update steps and maintain the ability to generate reasonable molecular structures.

[0056] Through multiple iterations, the updated molecular conformation generation model is used as the molecular generation model for the next iteration. Sampling, reward calculation, and parameter updates are repeated until a preset iteration termination condition is met, resulting in the target generation model. This process allows the target generation model to intrinsically learn how to directly output initial molecular coordinates close to a stable low-energy state when generating molecular conformations. This avoids the multiple iterations of force calculations and coordinate adjustments required to reach a stable state in traditional machine learning force field methods, and overcomes the deficiency of existing diffusion models that only focus on structural rationality and lack low-energy state constraints. Therefore, when the initial molecular coordinates output by the target generation model are used for subsequent structure optimization by the machine learning force field, the number of further optimization iterations required is significantly reduced because these initial molecular coordinates are close to a stable low-energy state. This significantly reduces the overall computational cost, effectively solving the technical problems of high computational cost and numerous iterations in molecular structure optimization in existing technologies, and significantly improving the overall efficiency of molecular structure optimization.

[0057] Based on the above embodiments of this application, in another embodiment of this application, the same or similar content as the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, step S4, which calculates the atomic force information of the molecular conformation through machine learning force field and determines the reward value corresponding to the sampling process based on the atomic force information, includes: Step S41: Compare the atomic force information of each atom in the molecular conformation with the preset target force information to obtain the force deviation information; In one feasible embodiment, during the process of calculating the atomic force information of the molecular conformation through machine learning force field and determining the reward value corresponding to the sampling process based on the atomic force information, the model optimization device compares the atomic force information of each atom in the generated molecular conformation with the preset target force information to obtain force deviation information.

[0058] Optionally, the atomic force information refers to the force vector predicted by the machine learning force field for each atom in the molecular conformation, reflecting the magnitude and direction of the force experienced by that atom in the current molecular structure.

[0059] Optionally, the preset target force information is a pre-set ideal force reference value. For a molecular conformation in a stable low-energy state, the ideal force on each atom should be a zero vector, indicating that there is no force inside the molecule to drive further changes in the molecular structure.

[0060] Optionally, the model optimization device calculates the difference between the actual atomic force information and the preset target force information on an atom-by-atom basis, for example, by calculating the norm or sum of squares of the force vector difference, to obtain force deviation information. This force deviation information describes the degree to which the current molecular conformation deviates from a stable low-energy state: the smaller the force deviation information, the closer the molecular conformation is to the ideal state after structural optimization.

[0061] Step S42: Determine the reward value for the molecular conformation generated by the molecular generation model based on the force deviation information.

[0062] In one feasible embodiment, after obtaining the force deviation information, the model optimization device determines the reward value for the molecular generation model to generate the current molecular conformation based on the force deviation information. Specifically, the design principle of the reward value is that the smaller the force deviation information, the higher the reward value. That is, the closer the force on each atom in the molecular conformation is to the preset target force information (e.g., zero force state), the greater the reward obtained by the model.

[0063] Optionally, the reward function It can be set as: ; in, This represents an intermediate state where the force is zero. Let represent the intermediate state at any time step, t∈[1,T]. Represents the last time step The reward for the intermediate state, the latter part is the KL (Kullback-Leibler Divergence, relative entropy) term. As weight, Let L2 norm squared be the difference between the noise predicted by the model to be optimized and the noise predicted by the reference model at time step t, used to constrain the update step size and direction of the model to be optimized. This represents the noise variance at the current time step.

[0064] Specifically, the sum of squares of the L2 norms of all atomic force vectors (i.e., the sum of squares of the total force amplitude) is first calculated. Then, the negative value or the result of the negative monotonic transformation function is taken as the reward value, thus establishing the optimization objective—maximizing the reward is equivalent to minimizing the overall atomic forces on the molecular structure. This determined reward value serves as a key performance feedback signal for subsequent model parameter updates, guiding the model to evolve towards generating molecules with smaller atomic forces and more stable structures in future sampling.

[0065] Optionally, the reward value can enable the model to update parameters in the direction of reducing force bias during reinforcement learning fine-tuning.

[0066] Optionally, the model optimization device uses the reward value for subsequent policy gradient calculations to guide the molecular generation model to learn to generate molecular conformations that are closer to stable low-energy states.

[0067] In this embodiment, a physical stability optimization objective is established by quantifying atomic force deviations into reward values ​​and embedding them into a reinforcement learning fine-tuning framework. This enables the final target generation model to generate not only chemically plausible molecular structures but also physically stable molecular structures. In other words, the initial molecular coordinates output by the target generation model have been trained to approach a stable low-energy state through the aforementioned reward mechanism, and their atomic force information is already close to the preset target force information. Therefore, when these initial molecular coordinates are used for subsequent structure optimization by the machine learning force field, the number of further optimization iterations required is significantly reduced, and the overall computational cost is significantly lowered. This effectively solves the technical problems of high computational cost and numerous iterations in molecular structure optimization in existing technologies, significantly improving the overall efficiency of molecular structure optimization.

[0068] Based on the above embodiments of this application, in another embodiment of this application, the same or similar content as the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, step S3, the step of determining the new policy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step based on the sampling trajectory, includes: Step S31: Obtain the sampling time step of each intermediate state during the sampling process; In one feasible embodiment, in the process of determining the new strategy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step, the model optimization device obtains the specific sampling time step in which each intermediate state in the sampling trajectory is located during the sampling process.

[0069] Optionally, the sampling time step refers to the time sequence number experienced during the reverse diffusion process of gradually restoring the molecular conformation from the initial noise state, such as gradually backtracking from the last time step to the first time step, with each time step corresponding to an intermediate state of the molecular structure.

[0070] Optionally, the time step identifies the specific sequential position of the intermediate state during the reverse diffusion (i.e., denoising) from the initial pure noise (corresponding to time step 1) to the final generated molecular conformation (corresponding to time step T).

[0071] Understandably, when predicting denoising actions, molecular generation models depend not only on the current noise state but also on the current time step. Therefore, they need to adjust their predictive behavior according to different time steps. It is crucial to clearly define the time step number corresponding to each intermediate state so that the state transition probability from the previous time step to the current time step can be correctly calculated sequentially.

[0072] Step S32: Based on the intermediate states corresponding to the current sampling time step and the previous sampling time step in the sampling trajectory, determine the new strategy transition probability of each intermediate state at each sampling time step.

[0073] In one feasible embodiment, after determining the sampling time step in which each intermediate state is located, the model optimization device determines the new strategy transition probability of the model to be optimized from the previous state to the current state at each sampling time step based on the intermediate state corresponding to the current sampling time step and the intermediate state corresponding to the previous sampling time step in the sampling trajectory.

[0074] Specifically, the model optimization device substitutes the model parameters of the current model to be optimized into the conditional probability distribution function, inputs the intermediate state of the previous sampling time step as a condition, and calculates the probability value of generating the intermediate state of the current sampling time step under the condition. This probability value is the transition probability of the new strategy.

[0075] Optionally, the new strategy transition probability reflects the degree of preference of the model to be optimized for the molecular conformational evolution path under the current parameters, and corresponds to the old strategy transition probability recorded at sampling time.

[0076] In this embodiment, by acquiring the time-step information of each intermediate state and independently calculating the new policy transition probability for each sampled time step, the model optimization device can compare the policy changes before and after the model update step by step, thereby obtaining the importance sampling ratio. This importance sampling ratio can accurately evaluate the merits of the new policy without completely discarding the experience of the old policy, ensuring the stability of the model parameter update and avoiding training oscillations or crashes caused by excessively large single update steps, thus laying a data foundation for subsequent reinforcement learning fine-tuning. On this basis, the atomic force deviation is quantified into a reward value and embedded into the reinforcement learning fine-tuning framework, so that the initial molecular coordinates output by the final target generation model are trained to be close to a stable low-energy state. When this initial molecular coordinate is used for subsequent structure optimization by the machine learning force field, the number of further optimization iterations required is greatly reduced, and the overall computational cost is significantly reduced, thereby effectively solving the technical problems of high computational cost and numerous iterations in molecular structure optimization in the prior art, and significantly improving the overall efficiency of molecular structure optimization.

[0077] Based on the above embodiments of this application, in another embodiment of this application, the same or similar content as the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, step S5, which involves updating the model parameters of the model to be optimized according to the old strategy transition probability, the new strategy transition probability, and the reward value, to obtain the updated molecular conformation generation model, includes: Step S51: Obtain the sampling ratio based on the ratio of the new strategy transition probability to the old strategy transition probability; In one feasible embodiment, during the process of updating the model parameters of the model to be optimized based on the old strategy transition probability, the new strategy transition probability, and the reward value to obtain the updated molecular conformation generation model, the model optimization device calculates the ratio of the new strategy transition probability to the old strategy transition probability to obtain the sampling ratio.

[0078] Optionally, the new policy transition probability is the conditional probability of the current model to be optimized generating the intermediate state of the current sampling time step given the intermediate state of the previous sampling time step, while the old policy transition probability is the same conditional probability calculated by the molecular generation model at the sampling time.

[0079] Optionally, the model optimization device divides the new policy transition probability by the old policy transition probability, and the resulting ratio is the sampling ratio.

[0080] Optionally, this sampling ratio, also known as the importance sampling ratio in the reinforcement learning proximal policy optimization framework, is used to measure the magnitude of policy change before and after model parameter updates: when the sampling ratio is close to 1, it indicates that the difference between the old and new policies is not significant; when the sampling ratio deviates significantly from 1, it indicates that the policy has changed considerably. By introducing the sampling ratio, the model optimization device can use the trajectory sampled from the old policy to evaluate the expected reward of the new policy, thereby efficiently updating the model parameters without resampling.

[0081] Understandably, the purpose of calculating this sampling ratio is to effectively and stably estimate the expected return or performance of the new policy using the data sampled from the old policy in subsequent updates of model parameters. At the same time, the magnitude of this sampling ratio constrains the magnitude of policy (which can also be called model parameters) updates, preventing the new policy from deviating too far from the old policy in the next update and causing training failure.

[0082] Step S52: Based on the sampling ratio and reward value, construct the loss function of the model to be optimized; In one feasible embodiment, after obtaining the sampling ratio, the model optimization device combines the sampling ratio and the reward value to construct a loss function to guide the updating of the parameters of the model to be optimized.

[0083] Alternatively, the loss function Loss can be set as: ; Where t is the time step; The policy parameters (i.e. model parameters) of the model to be trained (i.e. the model to be optimized). These are the strategy parameters for the molecular generation model; This represents the intermediate state of the model to be optimized at time step t-1. Generate intermediate states at time step t The probability of the new strategy transition (i.e., the probability of the new strategy transition); This represents the intermediate state of the molecular generation model at time step t-1. Generate intermediate states at time step t The probability of transitioning to the old strategy (i.e., the probability of transitioning to the old strategy); The ratio of the old policy transition probability to the new policy transition probability is the importance sampling coefficient (i.e., the sampling ratio), which prevents the policy to be trained from being too different from the reference policy. The function (clipping function) is used to: if the sampling ratio is within... and Between, return the sampling ratio; if the sampling ratio is greater than Then return If the sampling ratio is less than Then return To prevent excessively large step sizes or excessively reduced action probabilities during strategy updates, and to ensure stable update magnitudes, This is a hyperparameter, for example, set to 0.2.

[0084] Specifically, the sampling ratio is multiplied by the corresponding reward value to obtain the basic target term. Simultaneously, to ensure training stability and prevent update oscillations caused by excessively large or small sampling ratios, the sampling ratio processed by the pruning function is also multiplied by the same reward value to obtain another pruning target term. The pruning function forcibly restricts sampling ratios exceeding the preset interval [1-ε, 1+ε] to the boundaries of this preset interval. Finally, the model optimization device takes the smaller value of these two target terms, sums or averages this value over all sampling time steps, and then takes a negative sign to construct the loss function of the model to be optimized. This loss function is designed to indirectly maximize the reward while minimizing its value, but at the same time strictly constrains the difference between the old and new policies, achieving stable and efficient policy improvement.

[0085] Step S53: Based on the loss function, update the model parameters of the model to be optimized to obtain the updated molecular conformation generation model.

[0086] In one feasible embodiment, after calculating the sampling ratio, the model optimization device updates the model parameters of the model to be optimized based on a predefined loss function, thereby obtaining an updated molecular conformation generation model. Specifically, the model optimization device constructs a loss function to prevent excessively large single update steps from causing training instability.

[0087] Optionally, the model optimization device minimizes the loss function using gradient descent, i.e., adjusts the model parameters of the model to be optimized along the direction of decreasing loss function, thereby indirectly achieving the goal of maximizing the reward value. After parameter updates, an updated molecular conformation generation model is obtained. This model is more inclined to generate molecular conformations with low atomic forces and high reward values ​​compared to the original model, i.e., closer to stable low-energy states. When the initial molecular coordinates output by the final target generation model are used for subsequent structure optimization by the machine learning force field, since these initial molecular coordinates have been trained to be close to stable low-energy states through the above parameter update process based on sampling ratio and loss function, the number of further optimization iterations required is greatly reduced, and the overall computational cost is significantly reduced. This effectively solves the technical problems of high computational cost and numerous iterations in molecular structure optimization in existing technologies, and significantly improves the overall efficiency of molecular structure optimization.

[0088] In one feasible implementation, after step S5, which involves determining the reward value corresponding to the sampling process based on the atomic force information, the method further includes: Step A54: In each iteration, based on the sampling trajectory, determine the new policy transition probability of the model to be optimized under the current model parameters; In one feasible embodiment, after determining the reward value corresponding to the sampling process based on the atomic force information, the model optimization device determines the new strategy transition probability of the model to be optimized under the current model parameters based on the sampling trajectory in each iteration.

[0089] Optionally, each iteration refers to a complete cycle in the reinforcement learning fine-tuning training, during which the model optimization device updates parameters using the same batch of sampled data.

[0090] Optionally, the sampling trajectory is a record of a series of intermediate molecular states and their transition paths sampled from the molecular generation model, including the molecular structural state at each sampling time step; the model to be optimized refers to the molecular conformation generation model currently undergoing parameter updates; the current model parameters refer to the specific values ​​of variables such as weights and biases of the model to be optimized at the current time; the new strategy transition probability is the conditional probability of transitioning from the intermediate state of the previous sampling time step in the sampling trajectory to the intermediate state of the current sampling time step under the current model parameter conditions.

[0091] Understandably, the model optimization device calculates the new strategy transition probability step by step by substituting the current model parameters into the conditional probability distribution function and combining it with the state information recorded in the sampling trajectory at each time step.

[0092] Step A54: Based on the new strategy transition probability, old strategy transition probability, and reward value under the current model parameters, update the current model parameters of the model to be optimized until the preset number of updates is reached, and then proceed to the next iteration.

[0093] In one feasible embodiment, after calculating the new strategy transition probability, the model optimization device updates the current model parameters of the model to be optimized based on the new strategy transition probability under the current model parameters, the previously recorded old strategy transition probability, and the reward value, and repeats this update process until a preset number of updates is reached, and then enters the next iteration.

[0094] Optionally, the old strategy transition probability is the transition probability from the previous state to the current state calculated by the original molecular generation model at the sampling time; the reward value is determined based on the atomic force information and is used to quantify the degree to which the molecular conformation approaches a stable low-energy state; the preset update number is a pre-set positive integer, representing the maximum number of times the model parameters are updated by gradient descent using the same batch of sampled data in the same iteration.

[0095] Optionally, the model optimization device calculates the ratio of the new policy transition probability to the old policy transition probability to obtain the sampling ratio, and then constructs a loss function by combining the reward value. The loss function is minimized by the gradient descent method, thereby adjusting the current model parameters so that the model is more inclined to generate stable low-energy molecular conformations with high rewards.

[0096] Understandably, the model optimization device repeatedly executes the parameter update process, accumulating an update count with each update. When the update count reaches a preset value, the current iteration stops, and the next iteration begins, re-executing the complete process of sampling, reward calculation, determining the new strategy transition probability, and multiple parameter updates. Through fine-tuning of multiple parameter updates in each iteration, the initial molecular coordinates output by the target generation model are trained to approach a stable low-energy state. When these initial molecular coordinates are used for subsequent structure optimization by the machine learning force field, the required number of further optimization iterations is significantly reduced, and the overall computational cost is significantly lowered. This effectively solves the technical problems of high computational cost and numerous iterations in existing molecular structure optimization technologies, significantly improving the overall efficiency of molecular structure optimization.

[0097] In one feasible implementation, after step S5, which determines the reward value corresponding to the sampling process based on the atomic force information, the method further includes: Step B54: Generate the test molecular structure based on the updated molecular conformation generation model, and obtain the atomic force information of the test molecular structure; In one feasible embodiment, after determining the reward value corresponding to the sampling process based on the atomic force information, the model optimization device generates a test molecular structure based on the updated molecular conformation generation model and obtains the atomic force information of the test molecular structure.

[0098] Optionally, the updated molecular conformation generation model is a molecular generation model obtained after one or more rounds of reinforcement learning to fine-tune the parameters. The updated molecular conformation generation model has learned to a certain extent the ability to generate low-energy stable molecular conformations. The test molecular structure refers to the set of new molecular three-dimensional coordinates generated by the model optimization device from random noise through the back diffusion process of the updated molecular conformation generation model, which is used to evaluate the generalization performance and generation quality of the model. The atomic force information refers to the force vector predicted by the machine learning force field on each atom in the test molecular structure, which reflects the degree to which the molecular structure deviates from the stable low-energy state.

[0099] Understandably, model optimization devices calculate the magnitude and direction of the force on each atom by inputting the test molecular structure into a machine learning force field, thereby evaluating stability.

[0100] Step B55: Determine the stability index of the tested molecular structure based on the atomic force information and the preset target force information; In one feasible embodiment, after obtaining the atomic force information of the test molecular structure, the model optimization device determines the stability index of the test molecular structure based on the atomic force information and the preset target force information.

[0101] Optionally, the preset target force information is a pre-set ideal force reference value. For a molecular conformation in a completely stable low-energy state, the ideal force on each atom should be a zero vector, indicating that there is no force inside the molecule to drive further structural changes. The stability index is a quantitative value used to measure how close the test molecule structure is to a stable low-energy state. It can usually be obtained by calculating the deviation between the atomic force information and the preset target force information, such as calculating the root mean square or the average of the absolute values ​​of the force vectors of all atoms. The smaller the deviation, the higher the stability index.

[0102] Understandably, the model optimization device compares the force information of atoms with the preset target force information atom by atom, summarizes the overall force deviation, and uses this force deviation as a stability index, thereby realizing a quantitative assessment of the physical stability of the generated molecular structure.

[0103] Step B56: Evaluate the updated molecular conformation generation model based on stability indices.

[0104] In one feasible embodiment, after determining the stability index of the test molecular structure, the model optimization device evaluates the updated molecular conformation generation model based on the stability index.

[0105] Optionally, the evaluation is to determine whether the updated molecular conformation generation model has achieved the expected generation performance, such as comparing the stability index of the current model with the index of the baseline model or historical models, and observing whether there is a significant improvement. The updated molecular conformation generation model is a model obtained after fine-tuning training, and its evaluation results will determine whether the model can become the final target generation model for use, or whether it needs to continue to undergo another round of fine-tuning training.

[0106] Understandably, model optimization equipment determines whether the model's ability to generate low-energy stable molecular conformations meets the requirements by analyzing the magnitude of stability indices and their distribution on different test samples.

[0107] In this embodiment, by calculating the importance sampling ratio and constructing a loss function, the update magnitude of the new strategy is strictly constrained while utilizing the old strategy's sampling data. This effectively prevents drastic oscillations during training and ensures the stability of the fine-tuning process. Based on this, model parameters are updated using gradient descent, and the model's ability to generate low-energy stable molecular conformations is evaluated based on stability metrics. This ensures that the pre-trained molecular generation diffusion model can be stably and efficiently guided towards generating better molecular structures. When the initial molecular coordinates output by the final target generation model are used for subsequent structure optimization by the machine learning force field, since these initial molecular coordinates have been verified as close to a stable low-energy state through the aforementioned evaluation process, the required number of further optimization iterations is significantly reduced, resulting in a significant decrease in overall computational cost. This effectively solves the technical problems of high computational cost and numerous iterations in molecular structure optimization in existing technologies, significantly improving the overall efficiency of molecular structure optimization.

[0108] Based on the above embodiments of this application, in another embodiment of this application, the same or similar content as the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, step S1, the step of obtaining the pre-trained molecular generation model, includes: Step S11: Obtain the basic molecular generation model and molecular structure training set; In one feasible embodiment, during the process of acquiring a pre-trained molecular generation model, the model optimization device first acquires the basic molecular generation model and the molecular structure training set.

[0109] Optionally, the basic molecular generation model is a diffusion model that has not been trained or has only initial random parameters. The basic molecular generation model has the basic framework to gradually recover the molecular structure from noise but lacks the actual ability to generate molecular conformations that conform to physicochemical laws. For example, the basic molecular generation model can be the Geodiff diffusion model.

[0110] Optionally, the molecular structure training set is a database consisting of a large number of known molecular three-dimensional structures and their related attribute information, such as the structure, energy and physicochemical properties of about 130,000 organic molecules contained in the QM9 dataset. This training set is used to provide the model with the samples needed to learn the distribution of real molecules.

[0111] Step S12: Pre-train the basic molecular generation model using the molecular structure training set to obtain the pre-trained molecular generation model.

[0112] In one feasible embodiment, after obtaining the basic molecular generation model and the molecular structure training set, the model optimization device pre-trains the basic molecular generation model using the molecular structure training set to obtain the pre-trained molecular generation model.

[0113] Optionally, pre-training is a process in which real molecular structure samples from a molecular structure training set are used to gradually adjust the parameters of the basic molecular generation model by minimizing the standard loss function of the diffusion model, so that the model learns to recover physicochemically plausible molecular conformations from noise.

[0114] Optionally, the model optimization device uses molecular samples from the molecular structure training set as the target output to perform multiple rounds of iterative training on the basic molecular generation model, so that the basic molecular generation model gradually masters the distribution law of molecular structure and finally obtains a pre-trained molecular generation model with preliminary generation ability.

[0115] Optionally, the pre-trained molecular generation model serves as the starting point for subsequent reinforcement learning fine-tuning, providing the model with initial generation capabilities.

[0116] In this embodiment, by pre-training the diffusion-based architecture using a large-scale molecular structure dataset, the initial molecular generation model is equipped with the ability to gradually reconstruct stable molecular conformations that conform to physicochemical laws from noise. This ensures that subsequent fine-tuning is based on a model with strong prior knowledge of molecular generation, rather than starting from scratch, thus improving the efficiency and success rate of the entire optimization process. Based on this, through further optimization in the subsequent fine-tuning stage, the initial molecular coordinates output by the final target generation model are trained to approach a stable low-energy state. When these initial molecular coordinates are used for subsequent structure optimization using a machine learning force field, the number of further optimization iterations required is significantly reduced, and the overall computational cost is significantly lowered. This effectively solves the technical problems of high computational cost and numerous iterations in molecular structure optimization in existing technologies, significantly improving the overall efficiency of molecular structure optimization.

[0117] In one feasible embodiment, the existing conventional molecular structure optimization method is as follows: First, data is prepared using a "molecular structure training set," then input into a "conventional diffusion model" for generation, resulting in a "chemically reasonable molecular structure." The conventional diffusion model evaluates and guides the generated results through a "reward function" (the rule being: the more chemically reasonable the molecular structure, the higher the reward value), making the molecular structure generated by the conventional diffusion model more chemically reasonable. Subsequently, physical-level evaluation and optimization are performed through "DFT calculation," ultimately outputting a "physically stable molecular structure." This process clearly illustrates the iterative optimization path of conventional techniques, from data preparation, initial generation based on the diffusion model, reward guidance based on chemical reasonableness, to achieving physical stability optimization through first-principles calculations (i.e., DFT). It is understandable that existing methods mainly focus on learning the molecular structure distribution in the training data to ensure the chemical reasonableness of the generated molecular structure, but it is not in a stable low-energy state. It still requires additional, computationally expensive classical structure optimization algorithms to optimize the generated molecular structure to reach a low-energy state.

[0118] For example, please refer to Figure 2 , Figure 2 A simplified flowchart of a model optimization method is provided, specifically: Step 1: Input the molecular structure data into the molecular generation model for sampling.

[0119] Step 2: The molecular generation model generates molecular conformations and records the sampling trajectory during the sampling process, as well as the old policy transition probability for each intermediate state during the sampling process.

[0120] Step 3: Use the molecular conformation generated by the molecular generation model as the initial molecular coordinates and input them into the machine learning force field to obtain a reward value to evaluate this sampling process (the reward rule is: the smaller the force on the atoms of the molecular conformation, the more stable the physics, and the higher the reward value).

[0121] Step 4: Use the molecular generation model as the model to be optimized. The initial model parameters of the model to be optimized are the model parameters of the molecular generation model. It should be noted that this step is the training phase, which is used to guide the molecular conformation output by the model to be optimized, so as to maintain chemical rationality and physical stability.

[0122] Step 5: Based on the same sampling trajectory recorded in Step 2, the model to be optimized calculates the new policy transition probability corresponding to each intermediate state of the sampling trajectory.

[0123] Step 6: Based on the old strategy transition probability of each intermediate state during the sampling process of the molecular generation model and the new strategy transition probability of the model to be optimized for each intermediate state in the same sampling trajectory, calculate the "sampling ratio (also known as the importance sampling ratio)".

[0124] Step 7: Calculate the "loss" that guides the update of the model to be optimized based on the sampling ratio and reward value.

[0125] Step 8: Calculate the gradient of the model parameters of the model to be optimized based on the loss.

[0126] Step 9: Update the model parameters of the model to be optimized based on the calculated gradient, thereby optimizing the "model to be optimized" into an improved "updated molecular conformation generation model".

[0127] The updated molecular conformation generation model is used as the molecular generation model for the next iteration, forming a closed loop of continuous optimization until the preset iteration termination condition is met. The latest updated molecular conformation generation model is then used as the target molecular generation model, and the molecular conformation generated by the target molecular generation model is used as the initial molecular coordinates, which are then input into the machine learning force field for structure optimization, thereby reducing the number of iterations.

[0128] The flowchart clearly illustrates the iterative optimization process of this method: a reward mechanism guides the molecular generation model, enabling it to gradually generate molecular structures with continuously improving physical stability while maintaining chemical rationality. During this iterative process, the reward mechanism continuously drives the model parameters to optimize towards generating more stable molecular structures. The final target generation model can directly output a molecular conformation that is close to a stable low-energy state. Subsequently, the molecular conformation output by this target generation model is used as the initial molecular coordinates for subsequent structure optimization by the machine learning force field. Since the initial molecular coordinates are already close to a stable low-energy state, the number of further optimization iterations required by the machine learning force field is significantly reduced, resulting in a significant reduction in overall computational cost. Therefore, the method of this application effectively solves the technical problems of high computational cost and numerous iterations in molecular structure optimization in existing technologies, significantly improving the overall efficiency of molecular structure optimization.

[0129] It should also be noted that, Figure 2 The order of the steps shown is for illustrative purposes only and is not mandatory. It can be adjusted based on actual circumstances.

[0130] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the model optimization method of this application. Any simple transformations based on this technical concept, such as the interaction and combination of various embodiments, are all within the protection scope of this application.

[0131] This application also proposes a target generation model, which is used to generate a corresponding molecular conformation based on the molecular structure data, wherein the molecular conformation is the initial molecular coordinates for structural optimization by a machine learning force field.

[0132] It should be noted that the target generation model here refers to the machine learning model that has been finally trained through the model optimization methods described in the foregoing embodiments and can be directly deployed and applied.

[0133] Optionally, this target generation model receives molecule-related input data (i.e., molecular structure data) and automatically outputs a three-dimensional molecular structure (i.e., molecular conformation) that conforms to chemical laws and physical stability requirements based on its internally learned complex mapping relationships. Optionally, the molecular conformation can be a molecular formula representing the initial or target state of the molecule, a random noise vector, or a coarse molecular conformation coordinate that needs further optimization. After processing such inputs, the target generation model can utilize its model parameters, fine-tuned through reinforcement learning, to perform efficient and accurate molecular structure generation or optimization tasks.

[0134] Understandably, unlike ordinary molecular generation and diffusion models, this target generation model undergoes multiple rounds of reinforcement learning fine-tuning based on pre-training, with the reward function aiming to bring the atomic forces close to zero. Therefore, the target generation model learns to implicitly optimize the molecular conformation during the generation process, directly outputting a molecular conformation that is close to a stable low-energy state. By feeding this molecular conformation as the initial coordinates into the machine learning force field, because the initial coordinates are already very close to a stable state, the machine learning force field only needs a few iterations to complete the final optimization, significantly reducing the overall computational cost.

[0135] Understandably, in downstream applications such as drug discovery and materials design, the molecular structures generated using this target generation model can significantly reduce or even avoid relying on computationally expensive traditional quantum chemical methods (such as density functional theory DFT calculations) for subsequent geometric structure optimization, thereby further improving the overall efficiency of molecular structure generation and optimization.

[0136] This application also provides a model optimization device; please refer to [reference needed]. Figure 3 The model optimization device includes: Module 10 is used to acquire pre-trained molecular generation models and molecular structure data; The sampling module 20 is used to input the molecular structure data into the molecular generation model for sampling, generate molecular conformations, and record the old strategy transition probability and sampling trajectory of the intermediate state of the molecular structure at each sampling time step during the sampling process. The reward module 30 is used to calculate the atomic force information of the molecular conformation through machine learning force field, and determine the reward value corresponding to the sampling process based on the atomic force information, wherein the smaller the atomic force information, the higher the reward value; The determination module 40 is used to use the molecular generation model as the model to be optimized; and based on the sampling trajectory, to determine the new strategy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step. The update module 50 is used to update the model parameters of the model to be optimized based on the old strategy transition probability, the new strategy transition probability, and the reward value, to obtain an updated molecular conformation generation model; the updated molecular conformation generation model is used as the molecular generation model for the next iteration, and the step of inputting the molecular structure data into the molecular generation model for sampling is executed until a preset iteration termination condition is met to obtain a target generation model, wherein the target generation model is used to generate initial molecular coordinates for structural optimization by machine learning force field.

[0137] The model optimization apparatus provided in this application, employing the model optimization method described in the above embodiments, can solve the technical problems of high computational cost and numerous iterations in existing molecular structure optimization. Compared with the prior art, the beneficial effects of the model optimization apparatus provided in this application are the same as those of the model optimization method provided in the above embodiments, and other technical features in the model optimization apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0138] This application provides a model optimization device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the model optimization method in the first embodiment described above.

[0139] The following is for reference. Figure 4 The diagram illustrates a structural schematic of a model optimization device suitable for implementing embodiments of this application. The model optimization device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The model optimization device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0140] like Figure 4As shown, the model optimization device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the model optimization device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the model optimization device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a model optimization device with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0141] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0142] The model optimization device provided in this application, employing the model optimization method described in the above embodiments, can solve the technical problems of high computational cost and numerous iterations in existing molecular structure optimization. Compared with the prior art, the beneficial effects of the model optimization device provided in this application are the same as those of the model optimization method provided in the above embodiments, and other technical features of this model optimization device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0143] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0144] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0145] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the model optimization method in the above embodiments.

[0146] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0147] The aforementioned computer-readable storage medium may be included in the model optimization device; or it may exist independently and not be assembled into the model optimization device.

[0148] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0150] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0151] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described model optimization method. This solves the technical problems of high computational cost and numerous iterations in existing molecular structure optimization methods. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the model optimization method provided in the above embodiments, and will not be elaborated upon here.

[0152] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the model optimization method described above.

[0153] The computer program product provided in this application can solve the technical problems of high computational cost and numerous iterations in existing molecular structure optimization methods. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the model optimization method provided in the above embodiments, and will not be repeated here.

[0154] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A model optimization method, characterized in that, The model optimization method includes: Acquire pre-trained molecular generation models and molecular structure data; The molecular structure data is input into the molecular generation model for sampling to generate molecular conformations, and the old strategy transition probability and sampling trajectory of the intermediate state of the molecular structure at each sampling time step are recorded during the sampling process. The atomic force information of the molecular conformation is calculated by machine learning force field, and the reward value corresponding to the sampling process is determined based on the atomic force information. The smaller the atomic force information, the higher the reward value. The molecular generation model is used as the model to be optimized; based on the sampling trajectory, the new strategy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step is determined; Based on the old strategy transition probability, the new strategy transition probability, and the reward value, the model parameters of the model to be optimized are updated to obtain an updated molecular conformation generation model. The updated molecular conformation generation model is used as the molecular generation model for the next iteration, and the step of inputting the molecular structure data into the molecular generation model for sampling is performed until the preset iteration termination condition is met to obtain the target generation model. The target generation model is used to generate initial molecular coordinates for structural optimization by machine learning force field.

2. The method as described in claim 1, characterized in that, The step of calculating the atomic force information of the molecular conformation through machine learning force field and determining the reward value corresponding to the sampling process based on the atomic force information includes: The force information of each atom in the molecular conformation is compared with the preset target force information to obtain the force deviation information; The reward value for generating the molecular conformation by the molecular generation model is determined based on the force deviation information.

3. The method as described in claim 1, characterized in that, The step of determining the new policy transition probability of the model to be optimized in the intermediate state corresponding to each sampling time step based on the sampling trajectory includes: Obtain the sampling time step of each intermediate state during the sampling process; Based on the intermediate states corresponding to the current sampling time step and the previous sampling time step in the sampling trajectory, the new strategy transition probability of each intermediate state at each sampling time step is determined.

4. The method as described in claim 1, characterized in that, The step of updating the model parameters of the model to be optimized based on the old strategy transition probability, the new strategy transition probability, and the reward value to obtain the updated molecular conformation generation model includes: The sampling ratio is obtained based on the ratio of the transition probability of the new strategy to the transition probability of the old strategy; Based on the sampling ratio and the reward value, the loss function of the model to be optimized is constructed. Based on the loss function, the model parameters of the model to be optimized are updated to obtain the updated molecular conformation generation model.

5. The method as described in claim 1, characterized in that, After the step of determining the reward value corresponding to the sampling process based on the atomic force information, the method further includes: In each iteration, based on the sampling trajectory, the new strategy transition probability of the model to be optimized under the current model parameters is determined; Based on the new strategy transition probability, the old strategy transition probability, and the reward value under the current model parameters, the current model parameters of the model to be optimized are updated until a preset number of updates is reached, and then the next iteration begins.

6. The method as described in claim 1, characterized in that, The steps for obtaining the pre-trained molecular generation model include: Obtain the basic molecular generation model and molecular structure training set; The basic molecular generation model is pre-trained using the molecular structure training set to obtain the pre-trained molecular generation model.

7. A target generation model as described in any one of claims 1 to 6, characterized in that, The target generation model is used to generate corresponding molecular conformations based on the molecular structure data, wherein the molecular conformations are the initial molecular coordinates for structural optimization by machine learning force fields.

8. A model optimization device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the model optimization method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the model optimization method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the model optimization method as described in any one of claims 1 to 6.

Images