A method for evaluating machine learning potential energy surface feature engineering effect

Abstract

A method for evaluating machine learning potential energy surface feature engineering effect, comprising: generating feature engineering test data sets: rotation test data set, translation test data set, permutation test data set, sensitivity test data set and consistent continuity test data set; using feature engineering to calculate the output of each system in more than one of the five special test data sets of the feature engineering test data set, to perform result statistics and evaluate the feature engineering; statistics of evaluation results between different data sets in the feature engineering test data set, and give the overall evaluation of the feature engineering. The application strictly distinguishes equivalent structures and nonequivalent structures by considering the symmetry of the structure itself, and requires that the equivalent structures and nonequivalent structures are reflected in the output of the feature engineering, so as to ensure that the output information of the selected feature engineering method does not interfere with the fitting of the neural network model. The application can greatly improve the training accuracy of the machine learning potential energy surface model and reduce the calculation cost of network optimization.

Description

TECHNICAL FIELD

[0001] The present application relates to a potential energy surface feature engineering evaluation method. In particular, it relates to a method for evaluating the effect of machine learning potential energy surface feature engineering. BACKGROUND

[0002] Potential energy surface (PES) is a theoretical model used in chemistry, physics and materials science to study reaction kinetics. The basic idea is to regard the state point of a certain reaction system (including temperature, pressure and concentration of substances, etc. parameters) as a function value of a multi-dimensional potential energy function, and describe the change of the potential energy function between different state points in a certain way, so as to deduce the kinetic process of the reaction.

[0003] Machine learning potential energy surface model (ML-PES model) refers to a machine learning model used to predict the properties of a molecular system. It learns the potential energy surface by using machine learning algorithms, allowing the prediction of the properties of the target structure. The advantage of this method is that it omits the process of solving the Schrödinger equation from first principles calculation, greatly accelerating the process of calculating the properties of the structure, allowing larger time scale and space scale simulations under limited computing power. Common ML-PES models include neural network models, support vector machine models and decision tree models, etc. They usually need a large amount of data for training, and the hyperparameters of the model need to be adjusted to make the model better fit the data.

[0004] ML-PES models have a wide range of applications, and are widely used in chemistry, physics and materials science, etc. For example, in the field of chemistry, ML-PES models can be used to simulate chemical reactions, helping chemical researchers to find better catalysts. In the fields of physics and materials science, ML-PES models can be used to study and predict the physical and chemical properties of materials, helping materials scientists to design new materials.

[0005] In practical applications, ML-PES models may also encounter some challenges. For example, a large amount of data is needed to train the ML-PES model, which puts higher requirements on the quality and quantity of data, and the data from first principles calculation consumes a lot of computing power; the hyperparameters of the model need to be adjusted to make the model better fit the data; complex machine learning algorithms need to be used to improve the accuracy of the model; the trade-off between calculation accuracy and calculation efficiency also exists in ML-PES models.

[0006] In future developments, the ML-PES model may continue to be inspired by computing power, machine learning techniques, and other fields, leading to improved accuracy and efficiency of the model. Additionally, the ML-PES model may be applied in new application areas, providing more assistance to us. Its emergence provides us with a new method to study chemical and material-related problems and offers new ideas and methods for our research in related fields.

[0007] In the process of building a machine learning potential energy surface model, feature engineering is a key step. It involves preprocessing, cleaning, and transforming raw data to extract useful information. This includes selecting meaningful features, transforming and combining features, etc. The goal is to convert raw data into a form suitable for machine learning models, making input data more effective to improve model accuracy and generalization ability. Higher quality features often contribute more to model performance.

[0008] Due to the long training time of neural network models, sensitivity to hyperparameters, and the significant interference of noise in the input on training, the efficiency of using a pre-trained model to evaluate the effectiveness of feature engineering is too low. Therefore, the effectiveness of feature engineering needs to be evaluated before training, so as to select the optimal feature engineering method. Common feature engineering evaluation methods include:

[0009] Univariate statistical analysis method: exploratory data analysis is performed on the value distribution of each feature to discover the distribution characteristics of each feature, and then decide whether to perform transformation or missing value filling.

[0010] Visualization method: use visualization tools such as histograms, box plots, scatter plots, etc. to visualize data and manually identify relationships between features.

[0011] Statistical test method: analyze the correlation between features, such as using Pearson correlation coefficient, Spearman correlation coefficient, etc. to analyze the correlation between features.

[0012] Filter-based feature selection method: calculate the correlation coefficient between features and target variables to determine the optimal feature set.

[0013] Wrapper feature selection method: use recursive feature elimination method, etc. to select the optimal feature set automatically during model training.

[0014] Information gain or information gain ratio method: calculate the influence of different features on the target variable and select features that have a greater impact on the target variable.

[0015] Decision tree-based feature selection methods: such as classification and regression tree algorithms, select the optimal feature set by calculating the contribution of different features to the decision tree.

[0016] Random forest-based feature selection methods: such as Gini coefficient or tree-based feature selection methods, select the optimal feature set by calculating the contribution of different features to the random forest.

[0017] Support vector machine-based feature selection methods: such as L1 regularization, select the optimal feature set by calculating the contribution of different features to the support vector machine.

[0018] Other commonly used feature engineering evaluation methods: such as principal component analysis-based feature selection methods, independent component analysis-based feature selection methods, factor analysis-based feature selection methods, etc.

[0019] However, the common general evaluation method still has limitations in establishing the potential energy surface model. Since the feature engineering method in the process of establishing the potential energy surface model is often obtained through a series of transformations according to the Cartesian coordinate system, it lacks clear physical meaning, and its feature engineering output has strong correlation. The method of reducing a part of the feature often degrades the overall feature extraction effect. In addition, in terms of chemical structure, researchers can easily judge whether two structures are equivalent according to prior knowledge, but the Cartesian coordinate system commonly used cannot describe this equivalence or symmetry. If the structure is equivalent but the feature engineering output is not equivalent, or the structure is not equivalent but the feature engineering output is equivalent, it will obviously cause disturbance to the learning of the neural network. On this basis, since the neural network fitting hopes that the objective function is as smooth as possible Potential energy surface, when the feature engineering appears abnormal transition even the whole potential energy surface appears a large number of sharp peaks, that is, local extreme value, when the structure has a slight disturbance, it also causes disturbance to the fitting of the neural network. Therefore, an evaluation method is needed to measure the symmetry of the feature engineering, its discrimination degree for non-equivalent structures, that is, sensitivity, and whether its change is smooth when the structure is disturbed, that is, consistent continuity. SUMMARY

[0020] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art and provide a method for evaluating the effect of machine learning potential energy surface feature engineering driven by physical and chemical prior knowledge.

[0021] The technical solution adopted by the present application is: a method for evaluating the effect of machine learning potential energy surface feature engineering, comprising the following steps:

[0022] 1) Generate feature engineering test data set

[0023] The generating feature engineering test data set includes respectively generating: rotation test data set, translation test data set, permutation test data set, sensitivity test data set and consistent continuity test data set 5 special test data sets, each test data set includes data of more than one system, and the data of each system includes atomic coordinate data and element data of the system;

[0024] 2) The output of each system in one or more of the five special test data sets in the feature engineering test data set is calculated by using feature engineering, the results are counted and the feature engineering is evaluated, which is performed in one or more of the following ways:

[0025] The first one is to verify the rotational symmetry of the feature engineering;

[0026] The second one is to verify the translation symmetry of the feature engineering;

[0027] The third one is to verify the permutation symmetry of the feature engineering;

[0028] The fourth one is to verify the sensitivity of the feature engineering;

[0029] The fifth one is to verify the consistent continuity of the feature engineering;

[0030] 3) The evaluation results between different data sets in the feature engineering test data set are counted, and the overall evaluation of the feature engineering is given; wherein:

[0031] The method of counting includes: the evaluation results in a single data set are counted by using distance addition method, the proportion of the number of systems meeting the requirements after intersection to the entire feature engineering test data set, the minimum calculation accuracy or parameter range method meeting all test items, or the overall variance or standard deviation method of single physical and chemical equivalent system series output, or the overall variance or standard deviation method of single physical and chemical non-equivalent system series output, or the item-by-item scoring method of all evaluation items;

[0032] The overall evaluation includes: one or more of the rotational symmetry, the translation symmetry, the permutation symmetry, the sensitivity and the consistent continuity are evaluated;

[0033] When different feature engineering gives the same evaluation, the verification enhancement method is used for further verification.

[0034] The method for evaluating the effect of machine learning potential energy surface feature engineering of the application strictly distinguishes equivalent structures and non-equivalent structures by considering the symmetry of the structure itself, and requires that the equivalent structures and non-equivalent structures are reflected in the output of the feature engineering. This test method ensures that the output information of the preferred feature engineering method does not interfere with the fitting of the neural network model. Specifically, the three types of symmetry of translation, rotation and permutation basically cover the possibility of equivalent transformation of the structure itself, so that the output of the feature engineering method that can pass the test is unique when the structure is physically and chemically equivalent, that is, the feature engineering is a mapping of the structure to the output of the feature engineering, thereby reducing the difficulty of training the neural network and omitting the need for data enhancement for equivalent transformation. The requirement of sensitivity emphasizes that the transformation from the structure to the output of the feature engineering is injective, and the structure is physically and chemically different when the output of the feature engineering is different, thereby avoiding the interference of the same feature engineering output corresponding to different structural properties on the training of the neural network. The requirement of uniform continuity is more to reduce the difficulty of training the neural network, so that the fitting target of the neural network is as smooth as possible, thereby avoiding the existence of peaks, i.e. local extreme values, to hinder the training. The method of the application is a bijection from the structure in the physical and chemical domain to the input domain of the neural network for the feature engineering method that can meet the test of symmetry, sensitivity and uniform continuity. At the same time, the change does not reduce the smoothness of the original mapping from the structure to the property, thereby allowing the neural network to better learn the original mapping, i.e. the potential energy surface itself, so it is a feature engineering scheme that is friendly to the training of the neural network. Obviously, the feature engineering method verified by the application can greatly improve the training accuracy of the machine learning potential energy surface model and reduce the computational cost of network optimization, so it is a great help in the development of the machine learning potential energy surface model. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a flowchart of the method for evaluating the effect of machine learning potential energy surface feature engineering of the application;

[0036] Figure 2 is the structure of the feature engineering test data set in the application. DETAILED DESCRIPTION

[0037] The method for evaluating the effect of machine learning potential energy surface feature engineering of the application will be described in detail below in combination with embodiments and drawings.

[0038] As shown in Figure 1 , the method for evaluating the effect of machine learning potential energy surface feature engineering of the application comprises the following steps:

[0039] 1) generating a feature engineering test data set

[0040] In the feature engineering test data set, as Figure 2As shown, including: rotation test data set, translation test data set, permutation test data set, sensitivity test data set and consistent continuity test data set 5 special test data sets, each test data set includes more than one system data, each system data includes atomic coordinates and elements of the system. As follows:

[0041] (1.1) generating rotation test data set data

[0042] For a specified system, take any point in the system coordinate space as the center, select any direction as the axis, rotate at any angle and set times, generate atomic coordinate data and element data of the system for each rotation, and save in the rotation test data set;

[0043] (1.2) generating translation test data set data

[0044] For a specified system, take any direction in the system coordinate space as the translation direction, translate at any distance and set times, generate atomic coordinate data and element data of the system for each translation, and save in the rotation test data set;

[0045] (1.3) generating permutation test data set data

[0046] For a specified system, adopt a sequence with the same length as the number of atoms in the system, and perform set times of disordering sorting on the number of atoms in the system to obtain set sequences, the obtained set sequences are all different, each time the disordering sorting updates the current atomic coordinate data and element data of the system to a new sequence, and the atomic coordinate data and element data obtained by each disordering sorting are saved in the permutation test data set;

[0047] If the specified system is a periodic system, adopt a sequence with the same length as the number of atoms in the original unit cell of the system, and perform set times of disordering sorting on the number of atoms in the original unit cell of the system to obtain set sequences, the obtained set sequences are all different, each time the disordering sorting updates the current atomic coordinate data and element data of the system to a new sequence, and the atomic coordinate data and element data obtained by each disordering sorting are saved in the permutation test data set;

[0048] (1.4) generating sensitivity test data set data

[0049] For the specified system, one or more of the following methods is used to generate a number of non-repeating atomic coordinate data and element data, and save them in the sensitivity test data set: molecular dynamics sampling, randomly generating system coordinates, or perturbing the system atomic coordinates; the difference between the coordinates of each two systems in the sensitivity test data set is greater than the error threshold of the numerical type used for the system coordinates;

[0050] (1.5) Generating data for consistent continuity test data set

[0051] For the specified system, perturbing the system atomic coordinates generates a number of non-repeating atomic coordinate data and element data, and saves them in the consistent continuity test data set; in the consistent continuity test data set, the difference between the coordinates of each two systems is less than the absolute value of the larger one of the two systems, and is greater than the error threshold of the numerical type used for the system coordinates;

[0052] 2) Calculate the output of each system in one or more of the five special test data sets of the feature engineering test data set using feature engineering, perform result statistics and evaluate the feature engineering, specifically using one or more of the following methods;

[0053] The first one is to verify the rotational symmetry of the feature engineering

[0054] (1) Use the rotation test data set to calculate the feature engineering output under different atomic coordinate data and element data respectively;

[0055] (2) Compare the feature engineering output between each two and statistically analyze the entire rotation test data set to give the rotational symmetry evaluation result of the feature engineering;

[0056] The method of comparison between each two includes one or more of the following four methods: difference method, ratio method, difference-to-original-system-output ratio method, and distance method in the measure space before and after rotation transformation of the feature engineering output;

[0057] The method of statistical analysis of the entire rotation test data set includes the following four methods: distance in the measure space of the feature engineering output of physically and chemically equivalent systems, or the proportion of the number of systems meeting the symmetry requirement in the entire rotation test data set, or the minimum calculation accuracy or parameter range required by all systems in the rotation test data set to meet the symmetry requirement, or the overall variance or standard deviation statistical value of the output of a single physically and chemically equivalent system series.

[0058] The rotational symmetry evaluation result is determined by the statistical result of the rotational symmetry, and there are the following four evaluation results:

[0059] (a) When the proportion of the number of systems that meet the rotational symmetry requirement in the feature engineering output values ​​of multiple systems is greater than or equal to the set proportion value of the entire rotational test dataset, the feature engineering is considered to meet the rotational symmetry requirement; otherwise, the feature engineering is considered not to meet the rotational symmetry requirement.

[0060] (b) If the number of systems that meet the rotational symmetry requirement in the characteristic engineering output values ​​of multiple systems is less than 100%, the calculation precision of the characteristic engineering is reduced, the characteristic engineering output values ​​of the multiple systems are recalculated, new output values ​​are obtained, and it is then determined whether the number of systems that meet the rotational symmetry requirement in the new output values ​​reaches 100%. This process continues until all characteristic engineering output values ​​of multiple systems meet the rotational symmetry requirement. The calculation precision used at this time is recorded as the highest calculation precision at which the tested characteristic engineering meets the rotational symmetry requirement. If the highest calculation precision is greater than or equal to the set calculation precision value, the characteristic engineering is considered to meet the rotational symmetry requirement; otherwise, the characteristic engineering is considered not to meet the rotational symmetry requirement.

[0061] (c) When the number of systems that meet the rotational symmetry requirement in the characteristic engineering output values ​​of multiple systems is less than 100%, the parameter range of the characteristic engineering is narrowed, the characteristic engineering output values ​​of the multiple systems are recalculated, and new output values ​​are obtained. Then, it is determined whether the number of systems that meet the rotational symmetry requirement in the new output values ​​reaches 100%, until all characteristic engineering output values ​​of multiple systems meet the rotational symmetry requirement. The parameter range used at this time is recorded as the maximum parameter range in which the tested characteristic engineering meets the rotational symmetry requirement. If the maximum parameter range is greater than or equal to the set parameter range, the characteristic engineering is considered to meet the rotational symmetry requirement; otherwise, the characteristic engineering is considered not to meet the rotational symmetry requirement.

[0062] In the above three evaluation results, satisfying the rotational symmetry requirement means that when the distance between any two pairs of multiple feature engineering output values ​​in the measurement space is less than or equal to a set distance value, it is considered to satisfy the rotational symmetry requirement; otherwise, it is considered not to satisfy the rotational symmetry requirement.

[0063] (d) When the overall variance or standard deviation of the characteristic engineering output values ​​of multiple systems is less than or equal to the set variance or standard deviation, the characteristic engineering is considered to meet the symmetry requirement; otherwise, the characteristic engineering is considered not to meet the symmetry requirement.

[0064] The second item: Verify the translational symmetry of feature engineering.

[0065] (1) Using the translation test dataset, calculate the feature engineering output for different atomic coordinate data and element data respectively;

[0066] (2) pairwise comparison of the feature engineering outputs and statistics of the entire translation test dataset, to give the translation symmetry evaluation results of the feature engineering;

[0067] The pairwise comparison method includes one or more of the following four methods: difference method, ratio method, ratio method of the difference to the original system output, and distance method in the measure space of the feature engineering outputs before and after the translation transformation.

[0068] The method of statistics of the entire translation test dataset includes the following four methods: distance method in the measure space of the feature engineering outputs of the physical and chemical equivalent systems, or the proportion method of the number of systems satisfying the symmetry requirement to the entire translation test dataset, or the minimum calculation accuracy or parameter range method required by the symmetry of all systems in the translation test dataset, or the overall variance or standard deviation statistical value method of the output of a single physical and chemical equivalent system series.

[0069] The translation symmetry evaluation results are determined by the statistical results, and there are the following four evaluation results:

[0070] (a) When the number of systems satisfying the translation symmetry requirement in the feature engineering output values of multiple system data accounts for more than or equal to a set proportion value in the entire translation test dataset, the feature engineering is considered to satisfy the translation symmetry requirement; otherwise, the feature engineering is considered to not satisfy the translation symmetry requirement.

[0071] (b) When the number of systems satisfying the translation symmetry requirement in the feature engineering output values of multiple system data does not reach 100%, the calculation accuracy of the feature engineering is reduced, the feature engineering output values of the atomic coordinate data and element data of multiple systems are recalculated to obtain new output values, and then it is determined whether the number of systems satisfying the translation symmetry requirement in the new output values reaches 100%. This process is repeated until the feature engineering output values of the atomic coordinate data and element data of multiple systems all satisfy the translation symmetry requirement. The calculation accuracy used at this time is recorded as the highest calculation accuracy of the feature engineering satisfying the translation symmetry requirement. When the highest calculation accuracy is greater than or equal to a set calculation accuracy value, the feature engineering is considered to satisfy the translation symmetry requirement; otherwise, the feature engineering is considered to not satisfy the translation symmetry requirement.

[0072] (c) When the number of systems satisfying translational symmetry requirement in the feature engineering output values of multiple system data does not reach 100%, the parameter range of feature engineering is reduced, the feature engineering output values of atomic coordinate data and element data of multiple systems are recalculated to obtain new output values, and then it is judged whether the number of systems satisfying translational symmetry requirement in the new output values reaches 100%, until the feature engineering output values of atomic coordinate data and element data of multiple systems all satisfy translational symmetry requirement, at this time, the parameter range adopted is recorded as the maximum parameter range of the feature engineering satisfying translational symmetry requirement, and when the maximum parameter range is greater than or equal to the set parameter range, it is considered that the feature engineering satisfies translational symmetry requirement, otherwise it is considered that the feature engineering does not satisfy translational symmetry requirement.

[0073] Among the above three evaluation results, the satisfaction of translational symmetry requirement means that when the distance between two feature engineering output values in the measure space is less than or equal to the set distance value, it is considered that the translational symmetry requirement is satisfied, otherwise it is considered that the translational symmetry requirement is not satisfied.

[0074] (d) When the overall variance statistical value or the standard deviation statistical value of the feature engineering output values of multiple system data is less than or equal to the set variance value or the set standard deviation value, it is considered that the feature engineering satisfies translational symmetry requirement, otherwise it is considered that the feature engineering does not satisfy translational symmetry requirement.

[0075] Third: Verify the replacement symmetry of feature engineering

[0076] (1) Using the replacement test data set, the feature engineering output under different atomic coordinate data and element data is calculated respectively;

[0077] (2) Comparing the feature engineering output between two and the statistics of the entire replacement test data set, the replacement symmetry evaluation result of the feature engineering is given;

[0078] The method of two-by-two comparison includes one or more of the following four methods: difference method, ratio method, difference-to-raw-system-output ratio method, and distance method in the measure space of feature engineering output before and after replacement transformation;

[0079] The method of statistics of the entire replacement test data set includes the distance method in the measure space of the feature engineering output of the physically and chemically equivalent system, or the proportion method of the number of systems satisfying symmetry requirement in the entire replacement test data set, or the minimum calculation accuracy or parameter range method required for all systems in the replacement test data set to satisfy symmetry, or the overall variance or standard deviation statistical value method of the output of a single physically and chemically equivalent system series.

[0080] The replacement symmetry evaluation result is determined by the statistical result of the replacement symmetry, and there are the following four evaluation results:

[0081] (a) When the proportion of the number of systems satisfying the requirement of permutation symmetry in the feature engineering output values of the plurality of system data is greater than or equal to the set proportion value, the feature engineering is considered to satisfy the requirement of permutation symmetry; otherwise, the feature engineering is considered to not satisfy the requirement of permutation symmetry.

[0082] (b) When the number of systems satisfying the requirement of permutation symmetry in the feature engineering output values of the plurality of system data does not reach 100%, the calculation precision of the feature engineering is reduced, the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems are recalculated to obtain new output values, and it is determined whether the number of systems satisfying the requirement of permutation symmetry in the new output values reaches 100%. This process is repeated until the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems all satisfy the requirement of permutation symmetry. At this time, the calculation precision used is recorded as the highest calculation precision of the feature engineering satisfying the requirement of permutation symmetry. When the highest calculation precision is greater than or equal to the set calculation precision value, the feature engineering is considered to satisfy the requirement of permutation symmetry; otherwise, the feature engineering is considered to not satisfy the requirement of permutation symmetry.

[0083] (c) When the number of systems satisfying the requirement of permutation symmetry in the feature engineering output values of the plurality of system data does not reach 100%, the parameter range of the feature engineering is reduced, the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems are recalculated to obtain new output values, and it is determined whether the number of systems satisfying the requirement of permutation symmetry in the new output values reaches 100%. This process is repeated until the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems all satisfy the requirement of permutation symmetry. At this time, the parameter range used is recorded as the maximum parameter range of the feature engineering satisfying the requirement of permutation symmetry. When the maximum parameter range is greater than or equal to the set parameter range, the feature engineering is considered to satisfy the requirement of permutation symmetry; otherwise, the feature engineering is considered to not satisfy the requirement of permutation symmetry.

[0084] In the above three evaluation results, the requirement of permutation symmetry is satisfied when the distance between the output values of the plurality of feature engineering in the measure space is less than or equal to the set distance value; otherwise, the requirement of permutation symmetry is not satisfied.

[0085] (d) When the overall variance statistical value or the standard deviation statistical value of the feature engineering output values of the plurality of system data is less than or equal to the set variance value or the set standard deviation value, the feature engineering is considered to satisfy the requirement of permutation symmetry; otherwise, the feature engineering is considered to not satisfy the requirement of permutation symmetry.

[0086] The fourth item is to verify the sensitivity of the feature engineering

[0087] (1) Using the sensitivity test dataset, calculate the feature engineering output under different atomic coordinate data and element data respectively;

[0088] (2) Compare the feature engineering outputs pairwise and statistically analyze the entire sensitivity test dataset to give the sensitivity evaluation results of the feature engineering;

[0089] The pairwise comparison method includes one or more of the following five methods: difference method, ratio method, ratio of difference to original system output method, distance in measure space between feature engineering outputs of different systems method, and whether the hash values of feature engineering outputs of different systems are completely consistent method.

[0090] The statistical analysis method of the entire sensitivity test dataset includes the following methods: distance in measure space between feature engineering outputs of physically and chemically non-equivalent systems, or the proportion of the number of systems meeting the distinguishability requirement to the entire sensitivity test dataset, or the minimum calculation accuracy or parameter range required for all systems in the sensitivity test dataset to meet the distinguishability requirement, or the overall variance or standard deviation statistical value of the output of a single physically and chemically non-equivalent system series, or various correlation coefficients between systems and corresponding feature engineering outputs.

[0091] The sensitivity evaluation results are determined by the sensitivity distinguishability in the statistical results, and there are the following five evaluation results:

[0092] (a) When the number of systems meeting the sensitivity distinguishability requirement in the feature engineering output values of multiple system data accounts for more than or equal to the set proportion of the entire sensitivity test dataset, the feature engineering is considered to meet the sensitivity distinguishability requirement; otherwise, the feature engineering is considered not to meet the sensitivity distinguishability requirement.

[0093] (b) When the number of systems meeting the sensitivity distinguishability requirement in the feature engineering output values of multiple system data does not reach 100%, the calculation accuracy of the feature engineering is improved, the feature engineering output values of the atomic coordinate data and element data of multiple systems are recalculated to obtain new output values, and then it is determined whether the number of systems meeting the sensitivity distinguishability requirement in the new output values reaches 100%. This process is repeated until the feature engineering output values of the atomic coordinate data and element data of multiple systems all meet the sensitivity distinguishability requirement. At this time, the calculation accuracy used is recorded as the minimum calculation accuracy required for the feature engineering to meet the sensitivity distinguishability requirement. When the minimum calculation accuracy is less than or equal to the set calculation accuracy value, the feature engineering is considered to meet the sensitivity distinguishability requirement; otherwise, the feature engineering is considered not to meet the sensitivity distinguishability requirement.

[0094] (c) When the number of systems meeting the sensitivity distinguishability requirement in the feature engineering output values of the plurality of system data does not reach 100%, then increase the parameter range of the feature engineering, recalculate the feature engineering output values of the atomic coordinate data and element data of the plurality of systems, obtain new output values, and then determine whether the number of systems meeting the sensitivity distinguishability requirement in the new output values reaches 100%, until the feature engineering output values of the atomic coordinate data and element data of the plurality of systems all meet the sensitivity distinguishability requirement, at which time the parameter range used is recorded as the minimum parameter range of the feature engineering meeting the sensitivity distinguishability requirement, and when the minimum parameter range is less than or equal to the set parameter range, it is considered that the feature engineering meets the sensitivity distinguishability requirement, otherwise it is considered that the feature engineering does not meet the sensitivity distinguishability requirement.

[0095] Among the above three evaluation results, the sensitivity distinguishability requirement is met when the distance between the output values of the plurality of feature engineering in the measure space is greater than or equal to the set distance value, otherwise it is considered that the sensitivity distinguishability requirement is not met.

[0096] (d) When the overall variance statistical value or the standard deviation statistical value of the feature engineering output values of the plurality of system data is greater than or equal to the set variance value or standard deviation value, it is considered that the feature engineering meets the sensitivity distinguishability requirement, otherwise it is considered that the feature engineering does not meet the sensitivity distinguishability requirement.

[0097] (e) When the system data contains other properties in addition to atomic coordinate data and element coordinate data, when the correlation coefficient between the feature engineering output values of the plurality of system data and the system properties is greater than or equal to the set coefficient value, it is considered that the feature engineering meets the sensitivity distinguishability requirement, otherwise it is considered that the feature engineering does not meet the sensitivity distinguishability requirement.

[0098] Fifth: Verify the consistent continuity of the feature engineering

[0099] (1) Use the consistent continuity verification data set to calculate the feature engineering output under different atomic coordinate data and element data respectively;

[0100] (2) Calculate the derivative or difference quotient between the feature engineering outputs and the entire consistent continuity verification data set to give the consistent continuity evaluation result of the feature engineering;

[0101] The statistical method for the whole consistent continuity test data set includes statistics of: the extreme value of the derivative or difference quotient, or the extreme value of the absolute value of the derivative or difference quotient, or the number or proportion of the derivative or difference quotient exceeding the set threshold, or the proportion of the number of systems meeting the consistent continuity requirement in the whole consistent continuity test data set, or the minimum calculation accuracy or parameter range required for all systems in the consistent continuity test data set to meet the consistent continuity requirement, or the overall variance or standard deviation statistics of the derivative or difference quotient.

[0102] The consistent continuity evaluation result is determined by the statistical result of the consistent continuity, and there are five evaluation results as follows:

[0103] (a) When the proportion of the number of systems meeting the consistent continuity requirement in the whole consistent continuity test data set is greater than or equal to the set proportion value in the feature engineering output value of the multiple system data, it is considered that the feature engineering meets the consistent continuity requirement, otherwise, it is considered that the feature engineering does not meet the consistent continuity requirement.

[0104] (b) When the number of systems meeting the consistent continuity requirement does not reach 100% in the feature engineering output value of the multiple system data, the calculation accuracy of the feature engineering is improved, the feature engineering output value of the atomic coordinate data and the element data of the multiple systems is recalculated to obtain new output value, and then it is judged whether the number of systems meeting the consistent continuity requirement reaches 100% in the new output value, until the feature engineering output value of the atomic coordinate data and the element data of the multiple systems all meet the consistent continuity requirement, at this time, the calculation accuracy used is recorded as the minimum calculation accuracy of the feature engineering meeting the consistent continuity requirement, and when the minimum calculation accuracy is less than or equal to the set calculation accuracy value, it is considered that the feature engineering meets the consistent continuity requirement, otherwise it is considered that the feature engineering does not meet the consistent continuity requirement.

[0105] (c) When the number of systems meeting the consistent continuity requirement does not reach 100% in the feature engineering output value of the multiple system data, the parameter range of the feature engineering is increased, the feature engineering output value of the atomic coordinate data and the element data of the multiple systems is recalculated to obtain new output value, and then it is judged whether the number of systems meeting the consistent continuity requirement reaches 100% in the new output value, until the feature engineering output value of the atomic coordinate data and the element data of the multiple systems all meet the consistent continuity requirement, at this time, the parameter range used is recorded as the minimum parameter range of the feature engineering meeting the consistent continuity requirement, and when the minimum parameter range is less than or equal to the set parameter range, it is considered that the feature engineering meets the consistent continuity requirement, otherwise it is considered that the feature engineering does not meet the consistent continuity requirement.

[0106] In the above three evaluation results, the requirement of consistent continuity is met when the extreme value of the derivative or the difference quotient between two of the characteristic engineering output values of the multiple system data is within the set extreme value range, or the extreme value of the absolute value of the derivative or the difference quotient between two of the characteristic engineering output values of the multiple system data is less than or equal to the set extreme value, and the characteristic engineering is considered to meet the requirement of consistent continuity, otherwise the characteristic engineering is considered to not meet the requirement of consistent continuity.

[0107] (d) When the overall variance statistical value or the standard deviation statistical value of the characteristic engineering output values of the multiple system data is greater than or equal to the set variance value or standard deviation value, the characteristic engineering is considered to meet the requirement of consistent continuity, otherwise the characteristic engineering is considered to not meet the requirement of consistent continuity.

[0108] (e) When the number of derivative or difference quotient between two of the characteristic engineering output values of the multiple system data that exceeds the set extreme value range is less than or equal to the set number, or the proportion of the number of derivative or difference quotient between two of the characteristic engineering output values of the multiple system data that exceeds the set extreme value range to the number of system data in the consistent continuity test data set is less than or equal to the set proportion, or the number of absolute value of derivative or difference quotient between two that exceeds the set extreme value is less than or equal to the set number, or the proportion of the number of absolute value of derivative or difference quotient between two that exceeds the set extreme value to the number of system data in the consistent continuity test data set is less than or equal to the set proportion, the characteristic engineering is considered to meet the requirement of consistent continuity, otherwise the characteristic engineering is considered to not meet the requirement of consistent continuity.

[0109] 3) Statistics of evaluation results between different data sets in the feature engineering test data set to give an overall evaluation of the feature engineering

[0110] The statistical method includes one or more of the following: distance addition method, proportion of the number of systems that meet the requirements after intersection to the entire feature engineering test data set, minimum calculation accuracy or parameter range that meets all test items, overall variance or standard deviation of a single series of physically and chemically equivalent systems, or overall variance or standard deviation of a single series of physically and chemically non-equivalent systems, or individual scoring method for all evaluation items;

[0111] The overall evaluation includes evaluation of one or more of rotational symmetry, translational symmetry, substitutional symmetry, sensitivity, and consistent continuity;

[0112] When different feature engineering gives the same evaluation, a verification enhancement method is used for further verification.

[0113] The verification enhancement method includes one or more of increasing the number of system in the feature engineering test data set, adjusting the generation parameters of the rotation test data set, the translation test data set, the permutation test data set, the sensitivity test data set and the consistent continuity test data set in the feature engineering test data set, increasing the number of evaluation items, and replacing the statistical method for single evaluation of feature engineering.

[0114] Examples are given below:

[0115] Embodiment 1

[0116] The embodiment provides an evaluation method of rotational symmetry, which can include the following steps as shown in Figure 1 The steps are as follows:

[0117] On the basis of the existing data set, for each structure, taking the origin of the spatial coordinate system defined by the structure as the center, rotating around the x / y / z axis every 5°, a series of physically and chemically equivalent structure groups are generated.

[0118] For each physically and chemically equivalent structure group, the feature engineering output of each structure is calculated, and the difference between the feature engineering calculation results of the generated structures is calculated, and the maximum absolute value in the difference value is taken as the evaluation result.

[0119] For the feature engineering result calculated with float64-bit precision, the absolute value of the maximum error obtained after rotation evaluation should not be greater than 1E-8 times the absolute value of the maximum feature engineering output. When the condition is met, the tested feature engineering method passes the rotational symmetry evaluation.

[0120] Embodiment 2

[0121] The embodiment provides an evaluation method of translational symmetry, which can include the following steps as shown in Figure 2 The steps are as follows:

[0122] On the basis of the existing data set, for each structure, taking the spatial coordinate system defined by the structure, respectively translating along the x / y / z axis-1, -0.5, 0, 0.5, A series of physically and chemically equivalent structure groups are generated.

[0123] For each physically and chemically equivalent structure group, the feature engineering output of each structure is calculated, and the difference between the feature engineering calculation results of the generated structures is calculated, and the maximum absolute value in the difference value is taken as the evaluation result.

[0124] For the feature engineering result calculated with float64-bit precision, the absolute value of the maximum error obtained after rotation evaluation should not be greater than 1E-12 times the absolute value of the maximum feature engineering output. When the condition is met, the tested feature engineering method passes the translational symmetry evaluation.

[0125] Embodiment 3

[0126] The embodiment of the present application provides an evaluation method of substitution symmetry, which can include the following steps as shown in the figure: Figure 2

[0127] On the basis of the existing data set, for each structure, a sequence of the same number of atoms in the structure cell is randomly generated, and the atom number is rearranged according to the sequence to generate a series of physically and chemically equivalent structure groups.

[0128] For each physically and chemically equivalent structure group, the feature engineering output of each structure is calculated, and the difference between the generated structure feature engineering calculation results is calculated, and the maximum absolute value in the difference value is taken as the evaluation result.

[0129] For the feature engineering result calculated with float64-bit precision, the absolute value of the maximum error after translation evaluation should not be greater than 1E-12 times the absolute value of the maximum feature engineering output. When the condition is met, the tested feature engineering method passes the substitution symmetry evaluation.

[0130] Embodiment 4

[0131] The embodiment of the present application provides an evaluation method of sensitivity, which can include the following steps as shown in the figure: Figure 2

[0132] On the basis of the existing data set, for each structure, one atom in the structure is randomly translated in one or more directions of x / y / z axis to generate a series of physically and chemically equivalent structure groups.

[0133] For each physically and chemically equivalent structure group, the feature engineering output of each structure is calculated, and the difference between the generated structure feature engineering calculation results is calculated, and the maximum absolute value in the difference value is taken as the evaluation result.

[0134] For the feature engineering result calculated with float64-bit precision, the absolute value of the maximum error after translation evaluation should not be less than 1E-5 times the absolute value of the maximum feature engineering output. When the condition is met, the tested feature engineering method passes the sensitivity evaluation.

[0135] Embodiment 5

[0136] The embodiment of the present application provides an evaluation method of consistent continuity, which can include the following steps as shown in the figure: Figure 2

[0137] ​​​On the basis of the existing data set, one atom in one or more directions in the x / y / z axis is randomly translated on each structural basis A series of physically and chemically inequivalent approximate structure groups are generated.

[0138] For each physically and chemically equivalent structure group, the feature engineering output of each structure is calculated, and the difference between the generated structure feature engineering calculation results is calculated, and the difference value is divided by the corresponding displacement offset to obtain the difference quotient of the feature engineering to the structure perturbation, and the absolute value of the largest value in the difference quotient is taken as the evaluation result.

[0139] For the feature engineering results calculated with float64-bit precision, the absolute value of the largest value of the difference quotient obtained by evaluating the perturbation should not be greater than 1E5 times the absolute value of the largest value of the feature engineering output. When the condition is met, the tested feature engineering method is consistent and continuous, and the evaluation passes.

[0140] The remaining matters of the present application are known technologies.

[0141] The above examples are only for illustrating the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and to implement it, and cannot limit the protection scope of the present application. Any equivalent changes or modifications made in accordance with the spirit and essence of the present application should be covered within the protection scope of the present application.

[0142] Although the present application has been illustrated by the description of various embodiments, and although these embodiments have been described in considerable detail, the applicant does not intend to limit or restrict the scope of the appended claims or in any way to the details of such embodiments. Additional advantages and modifications will be readily apparent to those skilled in the art. Therefore, the present application is not limited to the specific details, representative devices and methods, and the illustrative examples shown and described. Therefore, deviations from such details can be made without departing from the spirit or scope of the general inventive concept.

Claims

1. A method for evaluating the effect of machine learning potential energy surface feature engineering, characterized in that, Comprising the following steps: 1) generating a feature engineering test data set In the generating of the feature engineering test data set, five special test data sets, i.e., a rotation test data set, a translation test data set, a permutation test data set, a sensitivity test data set, and a consistency continuity test data set, are generated respectively, each of which includes data of more than one system, and each system data includes atomic coordinate data and element data of the system; 2) calculating the output of more than one special test data set in the five special test data sets of the feature engineering test data set, performing result statistics and evaluating the feature engineering, specifically in the following way: Firstly, verify the rotational symmetry of the feature engineering, including: (1) using the rotation test data set, calculating the feature engineering output under different atomic coordinate data and element data respectively; (2) comparing the feature engineering output two by two and statistically analyzing the entire rotation test data set to give the evaluation result of the rotational symmetry of the feature engineering; The two-by-two comparison method includes one or more of the following four methods: difference method, ratio method, difference-to-original-system-output ratio method, and distance method for calculating the feature engineering output in the measure space before and after rotation transformation; The statistical method for the entire rotation test data set includes the distance method for the feature engineering output of the physically and chemically equivalent system in the measure space, or the proportion method for the number of systems meeting the symmetry requirement in the entire rotation test data set, or the minimum calculation accuracy or parameter range method required for all systems in the rotation test data set to meet the symmetry, or the overall variance or standard deviation statistical value method for the series output of a single physically and chemically equivalent system; Secondly, verify the translational symmetry of the feature engineering, including: (1) using the translation test data set, calculating the feature engineering output under different atomic coordinate data and element data respectively; (2) comparing the feature engineering output two by two and statistically analyzing the entire translation test data set to give the evaluation result of the translational symmetry of the feature engineering; The two-by-two comparison method includes one or more of the following four methods: difference method, ratio method, difference-to-original-system-output ratio method, and distance method for calculating the feature engineering output in the measure space before and after translation transformation; The statistical method for the entire translation test data set includes the distance method for the feature engineering output of the physically and chemically equivalent system in the measure space, or the proportion method for the number of systems meeting the symmetry requirement in the entire translation test data set, or the minimum calculation accuracy or parameter range method required for all systems in the translation test data set to meet the symmetry, or the overall variance or standard deviation statistical value method for the series output of a single physically and chemically equivalent system; Thirdly, verify the permutation symmetry of the feature engineering, including: (1) using the permutation test data set, calculating the feature engineering output under different atomic coordinate data and element data respectively; (2) comparing the feature engineering output two by two and statistically analyzing the entire permutation test data set to give the evaluation result of the permutation symmetry of the feature engineering; The pairwise comparison method includes one or more of the following four methods: difference method, ratio method, difference-to-ratio method of the original system output, and distance method of the feature engineering output in the measure space before and after the permutation transformation; The statistical method for the entire permutation test data set includes: the distance method of the feature engineering output in the measure space of the physical and chemical equivalent system, or the proportion method of the number of systems meeting the symmetry requirement in the entire permutation test data set, or the minimum calculation accuracy or parameter range required by the symmetry of all systems in the permutation test data set, or the overall variance or standard deviation statistical value of the output of a single physical and chemical equivalent system series; The fourth item is to verify the sensitivity of the feature engineering, which includes: (1) Using the sensitivity test data set, respectively calculate the feature engineering output under different atomic coordinate data and element data; (2) Compare the feature engineering outputs pairwise and statistically analyze the entire sensitivity test data set to give the sensitivity evaluation results of the feature engineering; The pairwise comparison method includes one or more of the following five methods: difference method, ratio method, difference-to-ratio method of the original system output, distance method of the feature engineering output in the measure space between different systems, and hash value consistency method of the feature engineering output of different systems; The statistical method for the entire sensitivity test data set includes: the distance method of the feature engineering output in the measure space of the physical and chemical equivalent system, or the proportion method of the number of systems meeting the symmetry requirement in the entire sensitivity test data set, or the minimum calculation accuracy or parameter range required by the symmetry of all systems in the sensitivity test data set, or the overall variance or standard deviation statistical value of the output of a single physical and chemical equivalent system series, or various correlation coefficients between the system and the corresponding feature engineering output; The fifth item is to verify the consistency of the feature engineering, which includes: (1) Using the consistency test data set, respectively calculate the feature engineering output under different atomic coordinate data and element data; (2) Calculate the derivative or difference quotient of the feature engineering outputs pairwise and statistically analyze the entire consistency test data set to give the consistency evaluation results of the feature engineering; The statistical method for the entire consistency test data set includes: the extreme value of the derivative or difference quotient, the extreme value of the absolute value of the derivative or difference quotient, the number or proportion of the derivative or difference quotient exceeding the set threshold, the proportion of the number of systems meeting the consistency requirement in the entire consistency test data set, the minimum calculation accuracy or parameter range required by the consistency of all systems in the consistency test data set, or the overall variance or standard deviation statistical value of the derivative or difference quotient; 3) Statistically analyze the evaluation results of different data sets in the feature engineering test data set to give the overall evaluation of the feature engineering; wherein: The overall evaluation includes: evaluating the rotational symmetry, translational symmetry, permutation symmetry, sensitivity, and consistency; When different feature engineering gives the same evaluation, the verification enhancement method is used for further verification.

2. The method of claim 1, wherein, The data generated in step 1) for generating the rotation test data set is that, for a specified system, taking any point in the system coordinate space as the center and selecting any direction as the axis, rotating at any angle and for a set number of times, generating atomic coordinate data and element data of the system each time the rotation is performed once, and saving in the rotation test data set.

3. The method of claim 1, wherein, The data generated in step 1) for generating the translation test data set is that, for a specified system, taking any direction in the system coordinate space as the translation direction, translating by any distance and for a set number of times, generating atomic coordinate data and element data of the system each time the translation is performed once, and saving in the translation test data set.

4. The method of claim 1, wherein, The data generated in step 1) for generating the permutation test data set is that, for a specified system, using a sequence with the same length as the number of atoms in the system, performing a set number of times of disordering sorting on the number of atoms in the system to obtain a set number of sequences, and the set number of sequences are all different, each time the disordering sorting updates the sequence of the current atomic coordinate data and element data of the system to a new sequence, and the atomic coordinate data and element data obtained each time the disordering sorting is performed are saved in the permutation test data set. If the specified system is a periodic system, a sequence with the same length as the number of atoms in the original unit cell of the system is used to perform a set number of times of disordering sorting on the number of atoms in the original unit cell of the system to obtain a set number of sequences, and the set number of sequences are all different, each time the disordering sorting updates the sequence of the current atomic coordinate data and element data of the system to a new sequence, and the atomic coordinate data and element data obtained each time the disordering sorting is performed are saved in the permutation test data set.

5. The method of claim 1, wherein, The data generated in step 1) for generating the sensitivity test data set is that, for a specified system, using one or more of the following three methods: molecular dynamics sampling, randomly generating system coordinates, or perturbing system atomic coordinates, to generate a plurality of non-repeating atomic coordinate data and element data, and save in the sensitivity test data set. The difference in coordinates between each two systems in the sensitivity test data set is greater than the error threshold of the numerical type used for system coordinates.

6. The method of claim 1, wherein, The data generated in step 1) for generating the consistent continuity test data set is that, for a specified system, perturbing the atomic coordinates of the system to generate a plurality of non-repeating atomic coordinate data and element data, and saving in the consistent continuity test data set. In the consistent continuity test data set, the difference in coordinates between each two systems is less than the value with the larger absolute value between the two systems, and is greater than the error threshold of the numerical type used for system coordinates.

7. The method of claim 1, wherein, In step 2), the first item: verifying the rotational symmetry of feature engineering, the rotational symmetry evaluation result is selected from the following four evaluation results according to the high and low of the statistical result: (a) When the number of systems that meet the rotational symmetry requirement in the feature engineering output value of the data of a plurality of systems accounts for more than or equal to a set proportion of the entire rotation test data set, it is considered that the feature engineering meets the rotational symmetry requirement; otherwise, it is considered that the feature engineering does not meet the rotational symmetry requirement; (b) When the number of systems satisfying the rotational symmetry requirement in the feature engineering output values of the data of multiple systems does not reach 100%, the calculation accuracy of the feature engineering is reduced, the feature engineering output values of the data of multiple systems are recalculated to obtain new output values, and then it is judged whether the number of systems satisfying the rotational symmetry requirement in the new output values reaches 100%. Until the feature engineering output values of the data of multiple systems all satisfy the rotational symmetry requirement, the calculation accuracy used at this time is recorded as the highest calculation accuracy of the feature engineering satisfying the rotational symmetry requirement. When the highest calculation accuracy is greater than or equal to the set calculation accuracy value, it is considered that the feature engineering satisfies the rotational symmetry requirement; otherwise, it is considered that the feature engineering does not satisfy the rotational symmetry requirement; (c) When the number of systems satisfying the rotational symmetry requirement in the feature engineering output values of the data of multiple systems does not reach 100%, the parameter range of the feature engineering is reduced, the feature engineering output values of the data of multiple systems are recalculated to obtain new output values, and then it is judged whether the number of systems satisfying the rotational symmetry requirement in the new output values reaches 100%. Until the feature engineering output values of the data of multiple systems all satisfy the rotational symmetry requirement, the parameter range used at this time is recorded as the maximum parameter range of the feature engineering satisfying the rotational symmetry requirement. When the maximum parameter range is greater than or equal to the set parameter range, it is considered that the feature engineering satisfies the rotational symmetry requirement, otherwise, it is considered that the feature engineering does not satisfy the rotational symmetry requirement; In the above three evaluation results, the rotational symmetry requirement is satisfied, which means that when the distance between the output values of multiple feature engineering in the measure space is less than or equal to the set distance value, it is considered to satisfy the rotational symmetry requirement, otherwise it is considered not to satisfy the rotational symmetry requirement; (d) When the overall variance statistical value or the standard deviation statistical value of the feature engineering output values of the data of multiple systems is less than or equal to the set variance value or the standard deviation value, it is considered that the feature engineering satisfies the symmetry requirement, otherwise, it is considered that the feature engineering does not satisfy the symmetry requirement.

8. The method of claim 1, wherein, Step 2) The second item: verifying the translational symmetry of the feature engineering, the translational symmetry evaluation result is selected from the following four evaluation results according to the high and low of the statistical result: (a) When the number of systems satisfying the translational symmetry requirement in the feature engineering output values of the data of multiple systems accounts for more than or equal to the set proportion in the entire translational test data set, it is considered that the feature engineering satisfies the translational symmetry requirement; otherwise, it is considered that the feature engineering does not satisfy the translational symmetry requirement; (b) When the number of systems satisfying translational symmetry requirement in the feature engineering output values of multiple system data does not reach 100%, the calculation precision of the feature engineering is reduced, the feature engineering output values of atomic coordinate data and element data of multiple systems are recalculated to obtain new output values, and then it is judged whether the number of systems satisfying translational symmetry requirement in the new output values reaches 100%, until the feature engineering output values of atomic coordinate data and element data of multiple systems all satisfy translational symmetry requirement, at this time, the calculation precision adopted is recorded as the highest calculation precision of the feature engineering satisfying translational symmetry requirement, and when the highest calculation precision is greater than or equal to the set calculation precision value, it is considered that the feature engineering satisfies translational symmetry requirement; otherwise, it is considered that the feature engineering does not satisfy translational symmetry requirement; (c) When the number of systems satisfying translational symmetry requirement in the feature engineering output values of multiple system data does not reach 100%, the parameter range of the feature engineering is reduced, the feature engineering output values of atomic coordinate data and element data of multiple systems are recalculated to obtain new output values, and then it is judged whether the number of systems satisfying translational symmetry requirement in the new output values reaches 100%, until the feature engineering output values of atomic coordinate data and element data of multiple systems all satisfy translational symmetry requirement, at this time, the parameter range adopted is recorded as the maximum parameter range of the feature engineering satisfying translational symmetry requirement, and when the maximum parameter range is greater than or equal to the set parameter range, it is considered that the feature engineering satisfies translational symmetry requirement, otherwise, it is considered that the feature engineering does not satisfy translational symmetry requirement; In the above three evaluation results, the translational symmetry requirement is satisfied, which means that when the distance between two feature engineering output values in the measure space is less than or equal to the set distance value, it is considered that the translational symmetry requirement is satisfied, otherwise it is considered that the translational symmetry requirement is not satisfied; (d) When the overall variance statistical value or standard deviation statistical value of the feature engineering output values of multiple system data is less than or equal to the set variance value or standard deviation value, it is considered that the feature engineering satisfies translational symmetry requirement, otherwise, it is considered that the feature engineering does not satisfy translational symmetry requirement.

9. The method of claim 1, wherein, Step 2) The third item: verifying the substitution symmetry of the feature engineering, the substitution symmetry evaluation result is selected from the following four evaluation results according to the high and low of the statistical result: (a) When the number of systems satisfying substitution symmetry requirement in the feature engineering output values of multiple system data accounts for more than or equal to the set proportion value in the entire substitution test data set, it is considered that the feature engineering satisfies substitution symmetry requirement; otherwise, it is considered that the feature engineering does not satisfy substitution symmetry requirement; (b) When the number of systems satisfying the requirement of substitution symmetry in the feature engineering output values of the plurality of system data does not reach 100%, the calculation precision of the feature engineering is reduced, the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems are recalculated to obtain new output values, and then it is judged whether the number of systems satisfying the requirement of substitution symmetry in the new output values reaches 100%, until the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems all satisfy the requirement of substitution symmetry, at this time, the calculation precision adopted is recorded as the highest calculation precision of the feature engineering satisfying the requirement of substitution symmetry, and when the highest calculation precision is greater than or equal to the set calculation precision value, it is considered that the feature engineering satisfies the requirement of substitution symmetry; otherwise, it is considered that the feature engineering does not satisfy the requirement of substitution symmetry; (c) When the number of systems satisfying the requirement of substitution symmetry in the feature engineering output values of the plurality of system data does not reach 100%, the parameter range of the feature engineering is reduced, the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems are recalculated to obtain new output values, and then it is judged whether the number of systems satisfying the requirement of substitution symmetry in the new output values reaches 100%, until the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems all satisfy the requirement of substitution symmetry, at this time, the parameter range adopted is recorded as the maximum parameter range of the feature engineering satisfying the requirement of substitution symmetry, and when the maximum parameter range is greater than or equal to the set parameter range, it is considered that the feature engineering satisfies the requirement of substitution symmetry, otherwise, it is considered that the feature engineering does not satisfy the requirement of substitution symmetry; In the above three evaluation results, the requirement of substitution symmetry is that when the distance between the two feature engineering output values in the measure space is less than or equal to the set distance value, it is considered to satisfy the requirement of substitution symmetry, otherwise it is considered not to satisfy the requirement of substitution symmetry; (d) When the overall variance statistical value or the standard deviation statistical value of the feature engineering output values of the plurality of system data is less than or equal to the set variance value or the set standard deviation value, it is considered that the feature engineering satisfies the requirement of substitution symmetry, otherwise, it is considered that the feature engineering does not satisfy the requirement of substitution symmetry.

10. The method of claim 1, wherein, Step 2) The fourth item: verifying the sensitivity of the feature engineering, the sensitivity evaluation result is selected from one of the following five evaluation results according to the sensitivity distinguishability in the statistical result: (a) When the number of systems satisfying the requirement of sensitivity distinguishability in the feature engineering output values of the plurality of system data accounts for more than or equal to the set proportion value in the proportion of the entire sensitivity test data set, it is considered that the feature engineering satisfies the requirement of sensitivity distinguishability; otherwise, it is considered that the feature engineering does not satisfy the requirement of sensitivity distinguishability; (b) When the number of systems that meet the sensitivity distinguishability requirement in the feature engineering output values of the plurality of system data does not reach 100%, the calculation accuracy of the feature engineering is improved, the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems are recalculated to obtain new output values, and then it is judged whether the number of systems that meet the sensitivity distinguishability requirement in the new output values reaches 100%, until the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems all meet the sensitivity distinguishability requirement, at this time, the calculation accuracy adopted is recorded as the minimum calculation accuracy of the feature engineering that meets the sensitivity distinguishability requirement, and when the minimum calculation accuracy is less than or equal to the set calculation accuracy value, it is considered that the feature engineering meets the sensitivity distinguishability requirement, otherwise, it is considered that the feature engineering does not meet the sensitivity distinguishability requirement; (c) When the number of systems that meet the sensitivity distinguishability requirement in the feature engineering output values of the plurality of system data does not reach 100%, the parameter range of the feature engineering is increased, the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems are recalculated to obtain new output values, and then it is judged whether the number of systems that meet the sensitivity distinguishability requirement in the new output values reaches 100%, until the feature engineering output values of the atomic coordinate data and the element data of the plurality of systems all meet the sensitivity distinguishability requirement, at this time, the parameter range adopted is recorded as the minimum parameter range of the feature engineering that meets the sensitivity distinguishability requirement, and when the minimum parameter range is less than or equal to the set parameter range, it is considered that the feature engineering meets the sensitivity distinguishability requirement, otherwise, it is considered that the feature engineering does not meet the sensitivity distinguishability requirement; In the above three evaluation results, the sensitivity distinguishability requirement is met, that is, when the distance between the plurality of feature engineering output values in the measure space is greater than or equal to the set distance value, it is considered that the sensitivity distinguishability requirement is met, otherwise, it is considered that the sensitivity distinguishability requirement is not met; (d) When the overall variance statistical value or the standard deviation statistical value of the feature engineering output values of the plurality of system data is greater than or equal to the set variance value or the standard deviation value, it is considered that the feature engineering meets the sensitivity distinguishability requirement, otherwise, it is considered that the feature engineering does not meet the sensitivity distinguishability requirement; (e) When the system data contains other properties in addition to the atomic coordinate data and the element coordinate data, when the various correlation coefficients between the feature engineering output values of the plurality of system data and the system properties are greater than or equal to the set coefficient value, it is considered that the feature engineering meets the sensitivity distinguishability requirement, otherwise, it is considered that the feature engineering does not meet the sensitivity distinguishability requirement.

11. The method of claim 1, wherein, Step 2) The fifth item: verification of the consistent continuity of the feature engineering, the consistent continuity evaluation result is selected from one of the following five evaluation results according to the high and low consistency continuity in the statistical result: (a) When the number of systems that meet the consistent continuity requirement in the feature engineering output values of the plurality of system data accounts for more than or equal to the set proportion value in the proportion of the entire consistent continuity test data set, it is considered that the feature engineering meets the consistent continuity requirement, otherwise, it is considered that the feature engineering does not meet the consistent continuity requirement; (b) When the number of systems satisfying the consistent continuity requirement in the feature engineering output values of the multiple system data does not reach 100%, the calculation accuracy of the feature engineering is improved, the feature engineering output values of the atomic coordinate data and the element data of the multiple systems are recalculated to obtain new output values, and then it is judged whether the number of systems satisfying the consistent continuity requirement in the new output values reaches 100%. Until the feature engineering output values of the atomic coordinate data and the element data of the multiple systems all satisfy the consistent continuity requirement, the calculation accuracy used at this time is recorded as the minimum calculation accuracy of the feature engineering satisfying the consistent continuity requirement. When the minimum calculation accuracy is less than or equal to the set calculation accuracy value, it is considered that the feature engineering satisfies the consistent continuity requirement. Otherwise, it is considered that the feature engineering does not satisfy the consistent continuity requirement. (c) When the number of systems satisfying the consistent continuity requirement in the feature engineering output values of the multiple system data does not reach 100%, the parameter range of the feature engineering is increased, the feature engineering output values of the atomic coordinate data and the element data of the multiple systems are recalculated to obtain new output values, and then it is judged whether the number of systems satisfying the consistent continuity requirement in the new output values reaches 100%. Until the feature engineering output values of the atomic coordinate data and the element data of the multiple systems all satisfy the consistent continuity requirement, the parameter range used at this time is recorded as the minimum parameter range of the feature engineering satisfying the consistent continuity requirement. When the minimum parameter range is less than or equal to the set parameter range, it is considered that the feature engineering satisfies the consistent continuity requirement. Otherwise, it is considered that the feature engineering does not satisfy the consistent continuity requirement. In the above three evaluation results, the consistent continuity requirement is satisfied when the extreme value of the derivative or the difference quotient between each other of the feature engineering output values of the multiple system data is within the set extreme value range, or the extreme value of the absolute value of the derivative or the difference quotient between each other is less than or equal to the set extreme value. Otherwise, it is considered that the feature engineering does not satisfy the consistent continuity requirement. (d) When the overall variance statistical value or the standard deviation statistical value of the feature engineering output values of the multiple system data is greater than or equal to the set variance value or the set standard deviation value, it is considered that the feature engineering satisfies the consistent continuity requirement. Otherwise, it is considered that the feature engineering does not satisfy the consistent continuity requirement. (e) When the number of derivatives or difference quotients between each other exceeding the set extreme value range in the feature engineering output values of the multiple system data is less than or equal to the set number, or the proportion of the number of derivatives or difference quotients between each other exceeding the set extreme value range in the number of system data in the consistent continuity test data set is less than or equal to the set proportion, or the number of derivatives or difference quotients between each other exceeding the set extreme value is less than or equal to the set number, or the proportion of the number of derivatives or difference quotients between each other exceeding the set extreme value in the number of system data in the consistent continuity test data set is less than or equal to the set proportion, it is considered that the feature engineering satisfies the consistent continuity requirement. Otherwise, it is considered that the feature engineering does not satisfy the consistent continuity requirement.

12. The method of claim 1, wherein, Step 3) The verification enhancement method includes one or more of increasing the number of system in the feature engineering verification data set, adjusting the generation parameters of the rotation verification data set, the translation verification data set, the permutation verification data set, the sensitivity verification data set, and the consistent continuity verification data set in the feature engineering verification data set, increasing the number of evaluation items, and changing the statistical method for single evaluation of the feature engineering.

Images