Outlier-aware adaptive mixed-precision quantization method for interatomic potential energy surface models

Abstract

The present invention proposes an outlier-aware adaptive mixed-precision quantization method and system for an interatomic potential energy surface model. By using mixed-precision quantization with different data widths for coefficients of different-order terms, the outlier-aware quantization method proposed in the present invention can effectively improve the efficiency of quantization representation. The present invention also proposes a mechanism and method for systematically controlling the errors in polynomial representation caused by quantization, and proposes a method for adaptively selecting the optimal quantization precision for mixed-precision quantization of coefficients of different-order terms under the constraints of error control. By analyzing the data of the polynomial coefficient table of the interatomic potential energy model, the present invention adaptively provides an optimal mixed-precision quantization scheme and performs outlier-aware mixed-precision quantization operations on the polynomial coefficient table based on this scheme, thereby improving the inference scale and speed of the interatomic potential energy model.

Description

Technical Field

[0001] The present invention relates to the technical field of efficient reasoning and performance optimization of deep learning interatomic potential energy surface models, and proposes an outlier-aware adaptive mixed-precision quantization method based on the piecewise polynomial coefficient table of the Deep Potential atomic potential energy model. Background Art

[0002] The Deep Potential Model is an end-to-end deep learning model for fitting interatomic potential energy surfaces (PES). The PES describes the total potential energy of an atomic system. Under the Born-Oppenheimer adiabatic approximation, the total potential energy of a system, determined by its electronic state, is a function of the coordinates of all atomic nuclei within the system. In other words, the system's geometry determines its total potential energy. The functional relationship between the system's total potential energy and its geometry constitutes the system's PES.

[0003] For a specific system, the input of the Deep Potential model is the coordinate position of all atoms in the system, and the output is the total potential energy of the system under this geometric configuration. The training data of the Deep Potential model is generally obtained through first-principles calculations, and the system PES is fitted by fitting a sufficient number of geometric configurations and their corresponding potential energies. The Deep Potential model realizes the fitting of high-dimensional potential energy surfaces. High-dimensional potential energy fitting means that the neural network can fit the total potential energy of a specific system containing any number of atoms. Specifically, it can be achieved by decoupling the input of the neural network from the specific scale of the target system. This method is mainly based on two points: first, during the fitting process, the total potential energy of the system can be decomposed into each atom of the system, that is, the total potential energy of the system can be obtained by calculating the potential energy of each atom in the system and then adding them up; second, the potential energy of each atom is mainly affected by its neighboring atoms. Therefore, when calculating the atomic potential energy, only the configuration of the atom and its neighboring atoms can be considered. This assumption is also called the "myopia principle". Based on the above two points, the process of fitting the total potential energy of the system by the Deep Potential model is as follows:

[0004] First, set the cutoff radius parameter R c , used to select the neighbor atoms of each atom; secondly, traverse all atoms in the system, take the atom as the center, and select the neighbor atoms of each atom according to R c Determine the neighboring atoms of the atom and represent the geometric configuration of the central atom and its neighboring atoms as the local environment matrix of the central atom , contains the positional relationship information of the central atom and all its neighboring atoms. As the input of the neural network, the potential energy E of the central atom is obtained i ; Furthermore, the potential energy E of all atoms in the system i Sum up and get the total potential energy E of the system total The unit of potential energy is eV electron volts. By calculating the potential energy of the system and the potential energy of the atoms, the force acting on the atoms can be calculated using E / Δposition=Force, where E is the atomic potential energy and Δposition is the distance the atoms move. The obtained Force can be used to perform molecular dynamics simulations to calculate various physical properties and microscopic behaviors, such as macroscopic physical properties such as melting point and electrical conductivity; and microscopic behaviors such as the electrochemical reaction performance of battery electrodes and electrolytes.

[0005] In this process, the neural network uses the same network parameters for each atom in a specific target system. Therefore, the trained neural network can handle systems of any size with any number of atoms.

[0006] In the Deep Potential model, from Generate E i The network model is divided into two parts: EmbeddingNet and FittingNet. Since the geometric configuration of the central atom and its neighboring atoms has translation, rotation and exchange invariance, EmbeddingNet is designed to ensure the above three invariances in the Deep Potential model. EmbeddingNet converts the input central atom local environment matrix into Mapped to an embedding space that guarantees the above three invariants, the environment descriptor D of the central atom is generated i The Fitting Net uses the central atom’s environmental descriptor D i As input, fit the potential energy E of the central atom i .

[0007] This invention mainly focuses on a hybrid precision optimization method for the model compression algorithm of the Embedding Net in the Deep Potential Model (hereinafter referred to as the DP model). The overall structure of the DP model is as follows Figure 1 As shown, the design of EmbeddingNet and its model compression algorithm are introduced below.

[0008] The specific implementation of Embedding Net:

[0009] Embedding Net is responsible for representing the geometric configuration of the central atom and its neighboring atoms as a set of features that satisfy translation, rotation and exchange invariance. This set of features is called the local environment descriptor D of the central atom. i , Fitting Net is D i Fit E to the input i Generate local environment descriptor D according to the geometric configuration of atoms i The process can be divided into four steps, such as Figure 2 As shown: First, calculate the relative coordinates of the central atom and all neighboring atoms within the truncation radius, expressed as the relative coordinate set R i ; Secondly, R i Transformed into the local environment matrix of the central atom Afterwards, according to Generate the embedding matrix G of the central atom i ; Finally, by R i With G i Calculation of the local environment descriptor D that ensures physical symmetry i .

[0010] (1) Generate the relative coordinate set R of the central atom and neighboring atoms i

[0011] In a physical system with N atoms, the position of each atom in three-dimensional space is represented by the coordinate r i describe:

[0012]

[0013] The potential energy E of each atom i The relative coordinate set R of the atom and all neighboring atoms within the cutoff radius i Decide:

[0014]

[0015] r ij =r j -r i ={x ij ,y ij , z ij}

[0016] L Rc (i) Identify Figure 1 The central atom shown in a is R c is the index set of neighbor atoms within the cutoff radius. Figure 1 The blue rectangular box in a is the atomic system to be calculated for potential energy.

[0017] (2) Generate the local environment matrix of the central atom

[0018] Local environment matrix of the central atom By R i Transformed, is an N m ×4 matrix, where 4 represents a quadruple where N m is a preset value, representing the maximum length of the neighbor list of all atoms. i generate The process is: for each neighbor atom j of the central atom i, execute:

[0019]

[0020] The left side of the above formula is R i , on the right in,

[0021]

[0022] S(r ij ) is a continuously differentiable smooth function defined as follows:

[0023]

[0024] where r cs is the smooth cutoff parameter, s(r ij ) is used to make Each quaternion in is smooth within the truncation radius r c The boundary of the local region defined by r is reduced to 0. ij and r ji Represents the same meaning as r j and r i The distance between them. c =r c .

[0025] (3) Generate the embedding matrix G of the central atom i

[0026] Depend on Generate G i The steps are completed by the neural network G in Embedding Net, which is in the form of:

[0027]

[0028] where α i , α jDenote the chemical element types of atom i and atom j. That is, for each combination of element types of the atom pair composed of the central atom and any of its neighbors, a set of parameters of G corresponds. When the chemical element types are determined, the neural network G is abbreviated as:

[0029] G(s(r ji )) The input of the neural network G is the scalar s(r ij ), and the output is a vector of dimension M1. Its network structure is as Figure 1 shown in c. By sequentially inputting the first column of , that is, the column of s(r ij ) into the network G, an embedding matrix G i of dimension N x M1 can be obtained.

[0030] (4) Generate the local environment descriptor D of the central atom i

[0031] After obtaining the embedding matrix G i , first take the first M2 columns (M2 < M1) of G i , and call it G i2 . The original G i matrix is called G i1 . Generate D i according to the following formula:

[0032]

[0033] is a matrix of dimension M1x M2. The above formula ensures the translational, rotational, and exchange invariance of D i to the atomic system configuration. After flattening the matrix D i into a vector and inputting it into the Fitting Net, the potential energy E i of the central atom can be obtained by fitting, as shown in d of Figure 1 .

[0034] 1.3 Embedding Net model compression based on piecewise fifth-order polynomial table lookup

[0035] Observing the Embedding Net neural network structure shown in c of Figure 1 , its input is the scalar s(r ij ), and its output is the row vector G i corresponding to G i_j . Therefore, the Embedding Net is responsible for generating G i from R iThe neural network part can be regarded as a mapping function from n groups of scalars to n groups of M-dimensional vectors, where the value of M is usually 128, corresponding to a high-dimensional and very complex continuous function; corresponding to the fully connected structure of EmbeddingNet that expands layer by layer, when performing inference, its calculation process is implemented at the bottom layer through general matrix multiplication GEMM operations. The computational overhead of matrix multiplication increases rapidly as the network expands layer by layer, such as Figure 1 As shown in e.

[0036] The existing technology proposes an Embedding Net model compression algorithm based on piecewise fifth-order polynomial tabulation. Its main idea is: for an Embedding Net with one-dimensional input, within the value range of its input, for each dimension of the output vector, a piecewise polynomial is used to fit the trained network model; when performing inference, a table is looked up for each output dimension according to the segmented interval of the input variable value, and the corresponding polynomial coefficients are obtained and the polynomial fitting value of each output dimension is calculated to obtain the M-dimensional vector of the network output.

[0037] The theoretical basis for using polynomial representation to compress neural network models is the Weierstrass approximation theorem: assuming g is a continuous real-valued function defined on the real interval [a, b], for any ε > 0, there exists a polynomial f such that for all x in the interval [a, b], |f(x) - g(x)| < ε. A fifth-order piecewise polynomial was selected to fit the trained EmbeddingNet.

[0038] After model compression, the main computational operation of the Embedding Net of the Deep Potential model during inference is transformed from GEMM (General Matrix Multiplication) to parallel table lookup and polynomial calculation, reducing the total floating-point calculation of the model by more than 80%. i Generate D i The process and the data structures used are as follows Figure 3 shown.

[0039] In terms of model compression accuracy, when the table interval is When the Deep Potential model is compressed, the inference result error can reach the extreme accuracy level of double-precision floating point numbers (single-atom force error The accuracy of the compressed model is almost the same as that of the original model. When the model compression causes the single-atom force inference error to be The magnitude fully meets the accuracy requirements of the first-principles molecular dynamics simulation based on DFT (Density Functional Theory, density functional theory) calculation (single atom force error magnitude).

[0040] Embedding Net model compression based on piecewise fifth-order polynomial table can significantly reduce the computational complexity of the DeepPotential model and accelerate the model's inference execution speed. Performance analysis of the compressed Deep Potential model shows that the fifth-order polynomial coefficient table lookup after model compression is a memory bandwidth-limited operation. During the inference process, its memory access time accounts for more than 66% of the total computing time. In the Deep Potential model, to ensure the accuracy of scientific computing results, all networks and data structures use double-precision types (64-bit double) for storage and calculation. Similarly, in the piecewise fifth-order polynomial table model compression, the piecewise fifth-order polynomial coefficients are also stored and calculated using double precision.

[0041] Numerical analysis of the compressed Deep Potential model shows that the value ranges of coefficients of different orders of the piecewise fifth-order polynomial are significantly different. Under the premise of ensuring the output accuracy of Embedding Net, there is room to use appropriate low-precision data storage formats for the value ranges of coefficients of different orders. The existing compression algorithm for the piecewise fifth-order polynomial table model fails to conduct a detailed analysis of the accuracy requirements for storage and calculation of the polynomial coefficients after table, but instead uniformly uses double precision for storage and calculation. Since the fifth-order polynomial coefficient table lookup operation in the Deep Potential model inference stage is a memory bandwidth-limited operation, the uniform use of double precision for storage and calculation of polynomial coefficients results in a huge waste of effective memory bandwidth, limiting further improvement in model inference performance.

[0042] The main reason why the Embedding Net model compression scheme based on piecewise fifth-order polynomial table does not use a mixed precision scheme for polynomial coefficient storage is that before applying the table method model compression, the main performance bottleneck of Embedding Net inference lies in GEMM calculation and embedding matrix G. iThe complete storage requirements of the Embedding Net model are met, and its inference time is mainly spent on GEMM matrix multiplication operations. The entire inference process is dominated by computational constraints. Therefore, the main focus of the model compression algorithm design is to reduce the computational complexity and memory usage of the model. However, after applying the piecewise fifth-order polynomial table method model compression, the computational mode of the Embedding Net inference process has also changed accordingly, from the original GEMM matrix multiplication to a large number of parallel table lookups and outer product matrix multiplication operations. While significantly reducing the computational complexity and storage usage of the model inference, the entire inference process has also changed from being dominated by computational constraints to being dominated by memory bandwidth constraints. It can be said that it is the optimization of the computational bottleneck of the original inference process of Embedding Net by the piecewise fifth-order polynomial table method model compression that has created the need and possibility for new mixed-precision storage optimization of fifth-order polynomial coefficients. Summary of the Invention

[0043] The purpose of the present invention is to solve the problem that after the Deep Potential model is compressed by the piecewise fifth-order polynomial table method, the performance of the model inference stage is limited by the memory access bandwidth due to the use of double-precision storage of the polynomial coefficient table. A quantization representation method for outlier perception of the fifth-order polynomial coefficient table is proposed. Based on this method, a mixed-precision quantization algorithm for the Deep Potential model piecewise fifth-order polynomial coefficient table is implemented to independently quantize the coefficients of terms of different orders and perform error control on the total value of the polynomial.

[0044] When analyzing the performance of an EmbeddingNet model compressed using a piecewise fifth-order polynomial table, the inventors discovered that the reasoning process of EmbeddingNet shifted from being dominated by computational constraints before model compression to being dominated by memory bandwidth constraints after model compression. The most important memory access operation is accessing the fifth-order polynomial coefficient table after model compression. Therefore, the present invention proposes a method for compressing and storing the fifth-order polynomial coefficient table based on mixed precision, while ensuring the accuracy of the calculation results. This method will become a feasible path to further improve the reasoning performance of EmbeddingNet.

[0045] Quantization reduces the amount of data, reduces the memory bandwidth requirement and increases the computing speed by converting floating-point numbers into low-precision integer data representations, thereby achieving data compression. Quantization methods have been widely used in AI model compression. The present invention applies quantization technology to the mixed-precision compressed storage of the polynomial coefficient table of the Deep Potential model to reduce the memory usage and memory bandwidth of the Deep Potential model in the inference stage, and further improve the scale and speed of model inference. There are two main challenges in applying quantization technology to the compression of the piecewise polynomial coefficient table of the Deep Potential model. One is how to efficiently quantize the data distribution characteristics of the piecewise polynomial coefficients, and the other is how to systematically analyze and control the quantization error of the piecewise polynomial coefficient table. The inventors intend to apply quantization technology to the mixed-precision compressed storage of the polynomial coefficient table.

[0046] The present invention aims to solve the following technical problems:

[0047] Numerical analysis of the Deep Potential model after model compression shows that in the fifth-order polynomial coefficient table, the range and distribution of the value intervals of the coefficients of different order terms show obvious differences. For example, the value intervals of the coefficients of low-order terms (constant terms, first-order terms, second-order terms) are often narrow, and the distribution of the values ​​within the intervals is relatively even; while the value intervals of the coefficients of high-order terms (third-order terms, fourth-order terms, fifth-order terms) are very wide, and the distribution of the values ​​within the intervals shows the characteristics of local concentration and the coexistence of large-span outliers. The existence of large-span outliers causes a waste of quantization bit width and seriously reduces the efficiency of quantization representation. In order to perform efficient quantization representation for this data characteristic, the present invention proposes an outlier-aware mixed precision quantization method, which can effectively improve the efficiency of quantization representation and reduce the bit width required for quantization.

[0048] The second technical problem to be solved by the present invention is:

[0049] As a specialized AI model for high-precision numerical calculations in the field of quantum physics and chemistry, the Deep Potential model differs significantly from mainstream AI models. This is particularly evident in that its fitting target is the numerical calculation results of a set of scientific computing models. Both the target physical model itself and the deep learning model may have high local sensitivity in their input space, meaning that a slight change in the input may lead to a drastic change in the output. For the fifth-order polynomial table compression algorithm, the fitting error can be precisely controlled by the table interval, which is reflected in the final prediction of the atomic energy by the Deep Potential model. The error caused by the table can increase by 10 -14 eV / atom to 10 -5The eV / atom level is precisely controlled. In practice, the meter interval is generally set to the compromise between prediction accuracy and model size. The corresponding model's final prediction error increases by 10 -7 Below the eV / atom level.

[0050] Therefore, for Deep Potential models or similar AI models that fit scientific computing numerical results, when quantization technology is applied to the compressed storage of its piecewise polynomial coefficient table, the most important goal and technical challenge in terms of precision control is: after the polynomial coefficients are quantized, the error of the total value of each piecewise fifth-order polynomial restored by dequantization can be precisely controlled. This is because the error control of the total value of the piecewise polynomial is the output value of the Embedding Net (that is, the embedding matrix G of the central atom). i ) error control, which will directly affect the accuracy of the Deep Potential model output.

[0051] In order to ensure that the error caused by quantization of the piecewise polynomial coefficient table to the Deep Potential model is controllable, the present invention proposes a mechanism and method for systematically controlling the error of the polynomial representation caused by quantization, and proposes an adaptive selection method for the optimal quantization accuracy to achieve mixed-precision quantization of coefficients of different orders under the constraint of error control.

[0052] In view of the shortcomings of the existing technology, the present invention proposes an outlier-aware adaptive mixed-precision quantization method for an interatomic potential energy surface model, comprising: obtaining an atomic system to be fitted with atomic potential energy, the atomic system comprising multiple atoms, taking each atom as the central atom and combining it with a preset cutoff radius, generating a relative coordinate set R of the neighboring atoms of each central atom. i ; According to the positional relationship between the central atom and all its neighboring atoms, the local environment matrix of each central atom is constructed by As the input of the interatomic potential energy surface model, the potential energy E of each central atom is obtained by fitting i ;

[0053] Among them, the interatomic potential energy surface model includes two parts: Embedding Net and Fitting Net. Embedding Net will input Mapped to the embedding space, according to Generate the embedding matrix G of the central atom i ; Through R i With G i Calculation of the local environment descriptor D that ensures physical symmetry i ;Fitting Net is D iAs input, the potential energy E of the central atom is obtained by fitting i ;

[0054] Among them, according to R i generate The process is: for each neighbor atom j of the central atom i, execute:

[0055]

[0056] The left side of the above formula is R i , on the right x ji ,y ji , z ji is the relative coordinate of neighbor atom j relative to the central atom i, r ji Represents the distance between atoms i and j;

[0057]

[0058]

[0059] where r cs is the smooth cutoff parameter, r c is the cutoff radius;

[0060] The neural network G in Embedding Net completes the S(r ji ) Generate G i ,G(s(r ji )), select a multi-order piecewise polynomial to fit the neural network G, so as to transform the neural network G into a piecewise polynomial coefficient table of MxNxK dimensions, where M is the number of segments, N is the output dimension of the neural network G, and K is the total order of the multi-order piecewise polynomial; according to S(r ji ) in the segmented interval, the coefficient table of the segmented polynomial is looked up for each output dimension, and S(r ji ) and calculate the polynomial fitting value of each output dimension to get G i ;

[0061] It is characterized by:

[0062] The internal data of the MxNxK-dimensional piecewise polynomial coefficient table is a floating-point number. The piecewise polynomial coefficient table is quantized based on outlier perception mixed precision to convert the piecewise polynomial coefficient table into a quantized coefficient table with fixed-point numbers as the internal data. ji ) is dequantized in the segment interval where the quantization coefficient table is located, and S(r ji ) polynomial coefficients, the specific process includes:

[0063] Step 1: Divide the piecewise polynomial coefficient table into K groups of NxM dimensional data along the K dimension, which are distinguished by the polynomial order, and set the quantization bit width L0-L for each of the K groups of data. K-1 ;

[0064] Step 2: In each set of floating-point data in the K sets of data, the maximum value is Vmax, the minimum value is Vmin, the maximum value of non-outlier points is Nsup, and the minimum value of non-outlier points is Ninf. After sorting each set of floating-point data, data is retrieved in descending order from Vmax to Nsup, and data is retrieved in ascending order from Vmin to Ninf. Data within the interval [Vmin, Ninf) and (Nsup, Vmax] are outliers.

[0065] Step 3: The outlier points in the piecewise polynomial coefficient table are stored in an additional outlier array. In the quantization representation process, the outlier points are represented as indices pointing to the storage array where they are located. The non-outlier points in the piecewise polynomial coefficient table are quantized into fixed-point numbers to obtain the quantized coefficient table.

[0066] Step 4: According to the input data S(r ji ) Perform segmented search in M ​​dimensions, select the NxK dimensional data table corresponding to the segment of the quantization coefficient table input data, divide it into K groups of N metadata, perform the inverse quantization process for each group of data, and identify the index of the outlier in the inverse quantization process. According to the index, find the floating point value of the outlier from the outlier array, and obtain the NxK dimensional polynomial coefficient table after inverse quantization as the input data S(r ji ) of the polynomial coefficients.

[0067] The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model is characterized in that the determination of the outliers includes:

[0068] The number of outliers accounts for x%, and the width of the outlier interval is: (Vmax-Nsup)+(Ninf-Vmin), which is proportional to the value interval of the data set ((Vmax-Nsup)+(Ninf-Vmin)) / (Vmax-Vmin) is y%. Qexp=y% / (N*x%), where Qexp is the expected proportion of the average interval width of the outliers, indicating the overall degree of outliers.

[0069] In MxNxK dimensional data, for a set of coefficients of a specific order item in a specific NxK dimensional table, Qexp is calculated according to the preset x% and y%. Then, when a set of candidate outliers is found from this set of coefficients, the number of candidate outliers is j, and the proportion of their value interval is k. For this set of candidate outliers, the true average interval proportion Q = k / j is calculated. If the Q value is not less than Qexp, the set of candidate points is determined to be the outliers.

[0070] The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model is characterized in that step 3 comprises:

[0071] For the MxN dimensional data in the piecewise polynomial coefficient table, all outliers in the M groups of N metadata are stored in a floating-point type array k_order_outliers_store that meets the preset accuracy requirements in the order of the M groups. A data structure named k_order_outliers_desc is used to describe these M groups of outliers, including the M-dimensional sequence number and N-dimensional sequence number of the outliers to indicate the position of the outliers in the MxN dimensional original data, and the storage index base address sequence number to indicate the starting index sequence number of the outliers in a specific M-dimensional group of N-element original data in the k_order_outliers_store array, and the storage index sequence number to indicate the offset index sequence number of the outliers in the k_order_outliers_store array relative to the index base address sequence number where the outliers are located;

[0072] After the outliers in the MxN-dimensional original data are centrally stored in the form of a one-dimensional array, the k_order_outliers_desc data structure implements the secondary indexing of the storage position of the one-dimensional array elements by grouping + offset within the group through the storage index base address sequence number + storage index sequence number, where the maximum value of the storage index base address sequence number is the total number of outliers in the MxN-dimensional original data minus 1, and the maximum value of the storage index sequence number is the total number of outliers in each group of N-element original data minus 1; when the outliers in each group of N-element original data are quantized, the quantized integer value is used to map the position information of the outlier in the centralized storage.

[0073] The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model is characterized in that step 4 includes quantization and dequantization of non-outlier data, specifically including:

[0074] A portion of the integer representation space is divided to represent the storage index of outliers, and the remaining integer representation space is used to asymmetric quantize non-outliers;

[0075] For a one-dimensional floating-point data set consisting of a set of floating-point numbers, NR_O outliers are marked in the data set. Let the maximum value of all non-outlier data be Vmax and the minimum value be Vmin. Use integer numbers with a bit width of b to perform outlier-aware quantization on this data set. The quantization and dequantization methods of the non-outlier points are:

[0076] The bit width of the quantized representation is b, and the total number of values ​​that can be represented by a b-bit integer is 2 b , among all the values ​​that can be represented by a b-bit integer, NR_O values ​​are marked out to represent the indices of NR_O outliers, and the remaining 2 b -NR_O values ​​are used to quantify the value interval [Vmin, Vmax] of non-outlier data, slots = 2 b NR_O is the number of available quantization slots, step = (Vmax–Vmin) / (slots–1) is the quantization step size, and base = Vmin is the quantization base value;

[0077] For non-outlier data with a value of Vx, its quantization representation is: Ix = round_to_int((Vx–base) / step), where Ix is the integer quantization representation of Vx, and round_to_int() represents the operation of rounding the decimal to an integer;

[0078] For a non-outlier quantized representation with a value of Ix, the inverse quantization recovery formula is: VRx=base+Ix*step, where VRx is a real value recovered from the integer quantized representation Ix.

[0079] The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model is characterized in that step 4 includes quantization and dequantization of outlier data, specifically including:

[0080] The value range of the quantized b-bit integer is {0..slots-1}, which is used to represent non-outlier data. The value range is {slots,2 b -1} is used to represent outlier data; for the MxN dimensional coefficients in the piecewise polynomial coefficient table, the outlier-aware quantization process for each group of N-dimensional floating-point data is as follows:

[0081] Establish a metadata data structure oaq_meta_info for outlier-aware quantization and inverse quantization for the N-tuple floating-point data. oaq_meta_info is the metadata of the outlier-aware quantization and inverse quantization of the N-tuple data, including recording the number of available quantization slots for non-outlier points in the group of data, the quantization step size step for non-outlier quantization, the quantization base value base for non-outlier quantization, the storage index base address sequence number outliers_base_index of the outliers in the group of data in the centralized storage array, and the memory access base address outliers_base_index pointing to the k_order_outliers_store array for the outliers in the group of data.

[0082] The quantization representation bit width of the N-ary floating-point data is b, the number of outliers is NR_O, and the number of available quantization slots for non-outliers is recorded in the oaq_meta_info data structure of the N-ary floating-point data: slots=2. b -NR_O, sets outliers_base_index to the storage index base address of the outlier point in the k_order_outliers_desc data structure; sets outliers_base_ptr to the memory access address of the element with offset outliers_base_index in the k_order_outliers_store array; searches for the entry corresponding to the outlier point in the k_order_outliers_desc data structure based on the M-dimensional and N-dimensional indices of the outlier point in the MxN-dimensional original data, and records the storage index in the entry as cur_outliers_idx. The quantitative representation of the outlier point is: Ix = oaq_meta_info.slots + cur_outliers_idx;

[0083] For the MxN dimensional coefficients of a specific order term in the polynomial coefficient table, the outlier-aware dequantization process for a set of N-ary floating-point data is as follows:

[0084] Determine the outlier-aware quantization metadata data structure oaq_meta_info for the set of floating-point data. For a quantized representation with a value of Ix, if Ix >= oaq_meta_info.slots, the data is outlier data. Restore the floating-point representation of the outlier based on the outlier storage index mapped to Ix: VRx = outliers_base_ptr[Ix–oaq_meta_info.slots], where VRx is the real value of the outlier recovered from the integer quantized representation Ix.

[0085] The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model is characterized in that, during the quantization process, the relative error limit ER of the preset polynomial term value is used. TV Determine the absolute error limit of each coefficient in the polynomial coefficient table, and calculate the quantization bit width L0-L that meets the error limit requirements for each K group of NxM dimension coefficient data K-1 To perform adaptive quantization error control of the polynomial coefficient table, specifically including:

[0086] According to ER TV Set the absolute error limit Δ for the polynomial term values ​​of M NxK dimensional polynomial coefficient tables TV , for each NxK dimension coefficient table, TV max_min =min{TV max_i |i=0..N-1} is the reference, where TV max_i |i=0..N-1 is the maximum absolute value of the polynomial term corresponding to the K coefficients in each row of the table, let Δ TV =ER TV *TV max_min The absolute error limit of the polynomial term value corresponding to the NxK dimensional table is obtained by repeating the above operation for M NxK dimensional polynomial coefficient tables to obtain the absolute error limit Δ of the polynomial term value corresponding to each of the M NxK dimensional tables. TV_i |i=0..M-1;

[0087] Calculate the absolute error limit allowed by the quantization representation of MxK groups of N-element coefficient data respectively: Determine the one-dimensional scalar input interval [x min_i ,x max_i ]|i=0..M-1, double traverse M and K dimensions, calculate Δ CV_ij =Δ TV_i / (x max_i ) j |i=0..M-1, j=0..K-1; note Δ CV is the absolute error limit of a set of N-dimensional coefficient data in the NxK-dimensional polynomial coefficient table, Δ CV_ij The absolute error limits of the K groups of N-element coefficient data in the M NxK-dimensional tables corresponding to the MxNxK-dimensional piecewise polynomial coefficient table;

[0088] The MxNxK dimensional piecewise polynomial coefficient table is divided into MxK groups of N-ary coefficient data, and the number of outliers in each group of N-ary coefficient data is recorded after outlier determination. ij |i=0..M-1,j=0..K-1, and calculate the non-outlier value range R of each group of data V_ij =V max_ij –Vmin_ij |i=0..M-1, j=0..K-1, let Q_SLOTS_ ij =R V_ij / (2*Δ CV_ij )+NR_O ij |i=0..M-1,j=0..K-1, then Q_BITS_ ij =ceil(log2(Q_SLOTS_ ij ))|i=0..M-1,j=0..K-1; Q_SLOTS _ij 、Q_BITS_ ij The K groups of N-element coefficient data in the M NxK dimensional tables corresponding to the MxNxK dimensional piecewise polynomial coefficient table are within the relative error limit index ER TV , the minimum number of quantization slots and the minimum quantization bit width required for the outlier-aware quantization method under the common constraints of the outlier determination indicators x%, y%, and Qexp;

[0089] Let L j =max{Q_BITS_ ij |i=0..M-1}|j=0..K-1, the quantization bit width L0-L of each order coefficient of the MxNxK dimensional piecewise polynomial coefficient table can be obtained. K-1 .

[0090] The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model is characterized in that the relative error limit ER TV The calculation process includes:

[0091] For each N polynomial described by the NxK dimensional polynomial coefficient table, extract the maximum TV of the absolute value of all terms in each of the N polynomials. max , get N TV max , in N TV max Select the smallest TV max As TV max_min , according to the preset absolute error limit Δ of NxK items TV , through ER TV =Δ TV / TV max_min Calculate ER TV .

[0092] The present invention also proposes an outlier-aware adaptive mixed-precision quantization system for an interatomic potential energy surface model, comprising: obtaining an atomic system to be fitted with atomic potential energy, the atomic system comprising a plurality of atoms, taking each atom as the central atom and combining it with a preset cutoff radius, and generating a relative coordinate set R of the neighboring atoms of each central atom. i; According to the positional relationship between the central atom and all its neighboring atoms, the local environment matrix of each central atom is constructed by As the input of the interatomic potential energy surface model, the potential energy E of each central atom is obtained by fitting i ;

[0093] Among them, the interatomic potential energy surface model includes two parts: Embedding Net and Fitting Net. Embedding Net will input Mapped to the embedding space, according to Generate the embedding matrix G of the central atom i ; Through R i With G i Calculation of the local environment descriptor D that ensures physical symmetry i ;Fitting Net is D i As input, the potential energy E of the central atom is obtained by fitting i ;

[0094] Among them, according to R i generate The process is: for each neighbor atom j of the central atom i, execute:

[0095]

[0096] The left side of the above formula is R i , on the right x ji ,y ji , z ji is the relative coordinate of neighbor atom j relative to the central atom i, r ji Represents the distance between atoms i and j;

[0097]

[0098]

[0099] where r cs is the smooth cutoff parameter, r c is the cutoff radius;

[0100] The neural network G in EmbeddingNet is used to complete the ji ) Generate G i ,G(s(r ji )), select a multi-order piecewise polynomial to fit the neural network G, so as to transform the neural network G into a piecewise polynomial coefficient table of MxNxK dimensions, where M is the number of segments, N is the output dimension of the neural network G, and K is the total order of the multi-order piecewise polynomial; according to S(rji ) in the segmented interval, the coefficient table of the segmented polynomial is looked up for each output dimension, and S(r ji ) and calculate the polynomial fitting value of each output dimension to get G i ;

[0101] It is characterized by:

[0102] The internal data of the MxNxK-dimensional piecewise polynomial coefficient table is a floating-point number. The piecewise polynomial coefficient table is quantized based on outlier perception mixed precision to convert the piecewise polynomial coefficient table into a quantized coefficient table with fixed-point numbers as the internal data. ji ) is dequantized in the segment interval where the quantization coefficient table is located, and S(r ji ) polynomial coefficients, the specific process includes:

[0103] Module 1 is used to divide the piecewise polynomial coefficient table into K groups of NxM dimensional data along K dimensions, which are distinguished by the polynomial order, and set the quantization bit width L0-L for each of the K groups of data. K-1 ;

[0104] Module 2 is used to sort each group of floating-point data in the K groups of data, where the maximum value is Vmax, the minimum value is Vmin, the maximum value of non-outliers is Nsup, and the minimum value of non-outliers is Ninf. Then, after sorting each group of floating-point data, data is retrieved in descending order from Vmax to Nsup, and data is retrieved in ascending order from Vmin to Ninf. Data within the interval [Vmin, Ninf) and (Nsup, Vmax] are outliers.

[0105] Module 3 is configured to centrally store the outlier points in the piecewise polynomial coefficient table in an additional outlier array. During the quantization representation process, the outlier points are represented as indices pointing to the storage array in which they are located. The non-outlier points in the piecewise polynomial coefficient table are quantized into fixed-point numbers to obtain the quantized coefficient table.

[0106] Module 4 is used to calculate the input data S(r ji ) Perform segmented search in M ​​dimensions, select the NxK dimensional data table corresponding to the segment of the quantization coefficient table input data, divide it into K groups of N metadata, perform the inverse quantization process for each group of data, and identify the index of the outlier in the inverse quantization process. According to the index, find the floating point value of the outlier from the outlier array, and obtain the NxK dimensional polynomial coefficient table after inverse quantization as the input data S(r ji ) of the polynomial coefficients.

[0107] The present invention also proposes a server comprising the outlier-aware adaptive mixed-precision quantization device of the interatomic potential energy surface model.

[0108] The present invention also proposes a storage medium for storing a computer program for executing the outlier-aware adaptive mixed-precision quantization method of the interatomic potential energy surface model.

[0109] From the above scheme, it can be seen that the advantages of the present invention are:

[0110] By analyzing data from the Deep Potential model, the inventors discovered that its polynomial coefficient table exhibits data distribution characteristics that are significantly correlated with the polynomial order, and that large-span outliers exist in the distribution of high-order coefficients. By using mixed-precision quantization with different data widths for coefficients of different order terms, while simultaneously sensing and specially processing outliers during the quantization process, the outlier-aware quantization method proposed in the present invention can effectively improve the efficiency of quantization representation. Furthermore, the present invention proposes a mechanism and method for systematically controlling the error in polynomial representation caused by quantization, and proposes a method for adaptively selecting the optimal quantization accuracy for mixed-precision quantization of coefficients of different order terms under the constraint of error control. Through the above invention, it is possible to achieve, under a specified polynomial representation error level, that the algorithm adaptively provides the optimal mixed-precision quantization scheme by analyzing the data of the Deep Potential model polynomial coefficient table and performs outlier-aware mixed-precision quantization operations on the polynomial coefficient table based on this scheme, thereby achieving error-controllable and efficient compression of the Deep Potential model's piecewise polynomial coefficient table, thereby improving the inference scale and speed of the Deep Potential model. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] Figure 1 It is the overall structure diagram of the prior art DP model;

[0112] Figure 2 This is the specific implementation flow chart of EmbeddingNet;

[0113] Figure 3 The calculation process and data structure of EmbeddingNet after model compression

[0114] Figure 4 This is an example diagram of the distribution characteristics of the coefficients of each order of the fifth-order polynomial in the piecewise polynomial coefficient table;

[0115] Figure 5 It is the structure diagram of the outlier array;

[0116] Figure 6a It is a schematic diagram of the symmetric quantization mapping method;

[0117] Figure 6b Schematic diagram of asymmetric quantization mapping method;

[0118] Figure 7 A schematic diagram of the association between the quantization and dequantization source information data structures of metadata;

[0119] Figure 8 The original data, outlier-aware quantization parameters, outlier-aware quantization results, and inverse quantization recovery results of MxN dimensional data are shown in Figure 2. DETAILED DESCRIPTION

[0120] In order to achieve the above technical effects, the present invention includes the following key technical points:

[0121] (1) A mixed precision quantization strategy is proposed for the Deep Potential model's piecewise polynomial coefficient table, which independently quantizes coefficients of terms of different orders.

[0122] (2) A method for outlier-aware quantization is proposed, specifically including: proposing a method for representing the expected degree of outlier of candidate outliers in one-dimensional floating-point data and control parameters, a method for marking outliers in one-dimensional floating-point data, a method for storing and indexing outliers in the process of outlier-aware quantization, and a method for quantitative representation and dequantization recovery of outliers in the process of outlier-aware quantization.

[0123] (3) Based on the outlier-aware quantization method described in the above technical point (2), a quantization error control and adaptive quantization accuracy selection method for the Deep Potential model piecewise polynomial coefficient table is proposed, specifically including a method for representing the quantization error level of the Deep Potential model piecewise polynomial coefficient table and control parameters, based on the control parameters, a method for calculating the absolute quantization error limit of the Deep Potential model piecewise polynomial coefficient table in the outlier-aware mixed precision quantization process is proposed, and based on the absolute quantization error limit calculation method, a method for calculating the optimal quantization representation bit width of each order coefficient of the Deep Potential model piecewise polynomial coefficient table is proposed.

[0124] (4) Combining the strategies and methods proposed in the above technical points (1), (2), and (3), a hybrid precision quantization method based on outlier perception and adaptive polynomial coefficient table of Deep Potential model is formed.

[0125] To illustrate the above-mentioned features and effects of the present invention more clearly and easily, the following embodiments are specifically described below with reference to the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are for illustrative purposes only. The scope of protection of the present invention is not limited to the disclosed embodiments; the present invention is defined by the appended claims.

[0126] When the inventors conducted a performance analysis on the Embedding Net after compression of the piecewise fifth-order polynomial table method model, they found that the reasoning process of EmbeddingNet changed from being dominated by computational constraints before model compression to being dominated by memory bandwidth constraints after model compression, where the most important memory access operation was accessing the fifth-order polynomial coefficient table after model compression. The numerical analysis of the Embedding Net after model compression showed that in the fifth-order polynomial coefficient table, the range and distribution of the value intervals of the coefficients of different order terms showed obvious differences. For example, the value intervals of the coefficients of low-order terms (constant terms, first-order terms, second-order terms) are often narrow, and the distribution of the values ​​within the intervals is relatively even; while the value intervals of the coefficients of high-order terms (third-order terms, fourth-order terms, fifth-order terms) are very wide, and the distribution of the values ​​within the intervals shows the characteristics of local concentration and large-span outliers. Taking into account the memory access bottleneck of the fifth-order polynomial coefficient table and the numerical characteristics of the coefficient table itself, while ensuring the accuracy of the calculation results, reasonable compression representation and storage of the fifth-order polynomial coefficient table through mixed precision technology will become a feasible path to further improve the Embedding Net inference performance.

[0127] The present invention proposes an outlier-aware adaptive mixed-precision quantization method for the polynomial coefficient table of the piecewise polynomial table model compression algorithm of the deep learning interatomic potential energy surface fitting model represented by the Deep Potential model. The main technical contents of the present invention include: a mixed-precision quantization strategy for independent quantization of each order coefficient is proposed in view of the storage structure of the piecewise polynomial coefficient table and the difference in the numerical distribution of coefficients of different orders; an outlier-aware mixed-precision quantization scheme is proposed in view of the data distribution characteristics of local concentration of medium and high order coefficients and the coexistence of large-span outliers; and a parameterized control method for piecewise polynomial quantization error and an adaptive calculation method for the optimal quantization representation bit width of each order coefficient are proposed in view of the high-precision requirements of the Deep Potential model.

[0128] In order to clearly illustrate the technical solution of the invention, the main technical points involved are elaborated in detail below according to the logical relationship between the invention contents, including:

[0129] (1) Data organization format of piecewise polynomial coefficient table

[0130] (2) Analysis of data distribution characteristics of piecewise fifth-order polynomial coefficient table

[0131] (3) Mixed-precision quantization strategy for independent quantization of coefficients of different orders

[0132] (4) Outlier determination index for one-dimensional floating point data sets

[0133] (5) Outlier determination method and process for one-dimensional floating-point data sets

[0134] (6) Centralized storage and indexing method for outliers in piecewise polynomial coefficient tables

[0135] (7) Quantitative representation and dequantization recovery methods for non-outlier data

[0136] (8) Quantitative representation and dequantization recovery methods for outlier data

[0137] (9) Outlier-aware quantization and dequantization of coefficients of specific orders in polynomial coefficient tables

[0138] (10) Quantization error analysis of polynomial coefficient table and control index of error level

[0139] (11) Quantization error control of polynomial coefficient table and calculation of optimal quantization bit width of each order coefficient

[0140] (12) Overall process of outlier-aware adaptive mixed-precision quantization of polynomial coefficient tables

[0141] 1. Data organization format of piecewise polynomial coefficient table

[0142] In the Deep Potential model, the input dimension of Embedding Net is always 1, and the value of the input variable is greater than 0; the output dimension of Embedding Net is configurable, and its dimension is set to N. For the Deep Potential model compressed by the piecewise polynomial model, its polynomial parameter is a three-dimensional array of MxNxK. Among them, M is the number of segments, which is determined by the range of the one-dimensional input of Embedding Net and the table interval. It can be considered that the polynomial parameters are composed of M NxK polynomial parameter tables; N is the output dimension of Embedding Net, and the number of rows in each polynomial parameter table can be considered to be N; K is the total order of the polynomial, and the number of columns in each polynomial parameter table can be considered to be K. For example, for a fifth-order polynomial, K = 6 (including constant terms and coefficients of first-order to fifth-order terms). Taking the Deep Potential model for water at room temperature as an example, its Embedding Net uses piecewise fifth-order polynomial table compression to produce two typical configurations: M = 1360, N = 128, K = 6 and M = 1220, N = 100, K = 6.

[0143] 2. Data distribution characteristics analysis of piecewise fifth-order polynomial coefficient table

[0144] Analysis of the compressed EmbeddingNet model shows that in the piecewise fifth-order polynomial coefficient table, the range and distribution of the value intervals of the coefficients of different order terms show obvious differences. For example, the value intervals of the coefficients of low-order terms (constant terms, first-order terms, second-order terms) are often narrow, while the value intervals of the coefficients of high-order terms (third-order terms, fourth-order terms, fifth-order terms) are very wide; the data distribution of the constant term in the value interval is relatively even, and as the order increases, the distribution of the coefficients of the corresponding order terms in the value interval gradually shows the characteristics of single-peak local concentration and large-span outliers. Moreover, the higher the order, the larger the span of the outliers and the smaller the proportion of the value range of non-outliers in the value interval, that is, the distribution peak of non-outliers is more concentrated.

[0145] Taking the Deep Potential model of room temperature water system as an example, we take the configuration of M=1220, N=100, and K=6, and extract a typical NxK dimensional polynomial coefficient table from the trained model. We visualize the values ​​of the six groups of polynomial coefficients a0-a5 from the constant term to the fifth-order term in the table. The results are as follows: Figure 4 As shown, the vertical axis shows the distribution of data values:

[0146] exist Figure 4In the example, the fifth-order coefficient a5 contains 100 floating-point numbers, whose value range is -2.5x107 to 8.8x107, and the value interval width is 1.3x108. Sorting a5 in descending order, the 10 largest numbers are: 88486978.13, 3108.313473, 408.8899266, 320.8529633, 150.9282659, 39.08764671, 28.43472622, 14.3886035, 3.995161043, 3.042761612; sorting ... Sorted in ascending order, the 10 smallest numbers are: -25365283.58, -1246380.356, -213439.2181, -35574.8122, -16274.63617, -11060.59383, -2393.529237, -530.7475767, -119.6419544, -113.6695028. If the nine numbers 88486978.13, 3108.313473, -25365283.58, -1246380.356, -213439.2181, -35574.8122, -16274.63617, -11060.59383, and -2393.529237 are considered as outliers, then the range of non-outlier values ​​is 4.1x10 2 to -5.3x10 2 , the width of the value interval is reduced to 9.4x10 2 In the above example, after analyzing the data of a5 and removing 9% of the outliers, the data value range is reduced to the original 1.38x10 5 For the method of compressing floating-point numbers by integer quantization, after removing outliers, the binary bit width required for integer representation can be reduced by about log2(1.38x10 5 )≈17 bits. The above analysis shows that for the piecewise polynomial coefficient table processed by the present invention, by identifying and individually processing outliers in the coefficients of each order of the polynomial, the efficiency of quantization representation can be greatly improved, the data bit width required for quantization representation can be reduced, and the total storage capacity occupied by the polynomial coefficient table after quantization can be reduced.

[0147] 3. Mixed precision quantization strategy for independent quantization of coefficients of different orders

[0148] From the distribution characteristics of the coefficients of the fifth-order polynomial mentioned above, we can see that the coefficients of different order terms have obvious differences in the range and distribution of values. Therefore, the present invention proposes a mixed-precision quantization strategy that distinguishes the coefficients of different order terms and quantizes them separately. The main basis for this strategy is mainly based on two points:

[0149] 1) Data distribution characteristics of the polynomial coefficient table. The distribution characteristics of the polynomial coefficient table shown above show that there are obvious differences in the range and distribution of values ​​of coefficients of different order items. This difference shows a convergent trend in the M segments of the MxNxK dimensional polynomial coefficient table. Therefore, the MxNxK dimensional data is divided in K dimensions, and data with different distribution characteristics are distinguished and data with the same distribution characteristics are aggregated according to the order. For coefficients of items of the same order, aggregating data with the same distribution characteristics is conducive to improving the basic efficiency of quantitative representation; for coefficients of items of different orders, setting different quantization widths according to different distribution characteristics can improve the overall quantization efficiency through mixed precision quantization.

[0150] 2) Data layout characteristics of the polynomial coefficient table. For MxNxK-dimensional data, partitioning is done according to the order of the polynomial, and the coefficients of the same order are assigned the same quantization width. This ensures that the data layout after quantization remains a regular three-dimensional array, which helps improve subsequent computational efficiency.

[0151] The specific implementation method is as follows: for the trained piecewise K-order polynomial model compressed version of Deep Potential model, its M NxK-dimensional polynomial coefficient tables are split along the K dimension into K groups of NxM-dimensional data distinguished by polynomial order, and the quantization bit widths L0-L are set for these K groups of data respectively. K-1 . Taking the fifth-order polynomial as an example, let the column names of the Nx6-dimensional polynomial coefficient table be a0-a5, which correspond to the constant terms and the first-order to fifth-order coefficients of the N fifth-order polynomials in the table respectively. Then, for M such Nx6-dimensional coefficient tables, they are regarded as 6 groups of NxM coefficients, which correspond to the 0-5 order coefficients of the M tables respectively. The quantization bit widths L0-L5 are set for the coefficients of these 6 groups of different order terms for quantization representation. The specific values ​​of L0-L5 will be obtained based on the outlier-aware quantization method, the polynomial coefficient table quantization error control method and the polynomial coefficient table quantization bit width calculation method proposed in the present invention. The values ​​depend on the specific analysis of the quantization error of the polynomial coefficient table and the specific distribution of the coefficients of each order. The specific calculation method and process are described in detail in Technical Point 11. Since the specific values ​​of L0-L5 may be different, for the entire polynomial coefficient table, adaptive mixed precision quantization is performed on them.

[0152] 4. Outlier determination index for one-dimensional floating point data set

[0153] Outlier determination is based on the degree of outlier distribution of each sample point in a data set. This paper proposes a method for representing the overall expected degree of outlier distribution in scalar floating-point data. For ease of explanation, we first define a common outlier example and then demonstrate how to represent the degree of outlier distribution based on this example.

[0154] In a set of scalar floating-point data, let its maximum value be Vmax and its minimum value be Vmin. Assume that there are several outliers at the edge of the data distribution interval, while the maximum value of non-outliers is Nsup and the minimum value is Ninf. After sorting the data, we search for data in descending order from Vmax to Nsup, and in ascending order from Vmin to Ninf. Then, we have: Vmax ≥ Nsup ≥ Ninf ≥ Vmin. Data within the interval [Vmin, Ninf) and (Nsup, Vmax] are outliers.

[0155] Suppose the proportion of outliers in the total data sample is x%, the width of the value interval of the outliers is: (Vmax-Nsup)+(Ninf-Vmin), and the ratio of the value interval of the outliers to the value interval of the data set ((Vmax-Nsup)+(Ninf-Vmin)) / (Vmax-Vmin) is y%, that is, the width of the value interval of the outliers accounts for y%. Then we can use the two data of x% and y% to describe our expectations for the number and distribution of outliers. The specific description is: suppose the total number of samples in the data sample set is N. Under the above expectation, we set that there are at most x% outliers in the data set. Their degree of outliers is determined by the average ratio of the width of the value interval occupied by each outlier. Our minimum expectation for this ratio is Qexp=y% / (N*x%). We call the Qexp defined above the expected proportion of the average interval width of the outliers. Qexp represents the overall degree of outliers and can be used as a judgment indicator for outlier selection.

[0156] It's important to note that x%, y%, and Qexp only define our overall expectation of the severity of outliers. They don't necessarily mean we'll find x% of outliers in a data set, or that the interval width percentage of each outlier found in that data set will meet Qexp. The expected average interval width percentage of outliers isn't used directly to find outliers individually in the data. Instead, it's used to determine whether a group of outliers, after using an outlier-finding algorithm to find them, meets our desired severity.

[0157] In MxNxK-dimensional data, for example, with N = 100, for a set of coefficients for a specific order term in a specific NxK-dimensional table, we set x% = 10 and y% = 90%, then Qexp = 90% / (100*10%) = 9%. Then, when we use an algorithm to find a set of candidate outliers from this data, let the number of candidate points found be j, and their value interval percentage be k. For this set of candidate outliers, we calculate their true average interval percentage, Q = k / j. If Q is not less than 9%, the candidate points are considered outliers.

[0158] 5. Outlier determination method and process for one-dimensional floating point data sets

[0159] For a set of scalar floating-point data, the idea of ​​the outlier determination algorithm is as follows: calculate the distance between adjacent sample points of the data, which is called the gap. According to the order of gap from large to small, the data set is divided into two parts with the current gap as the natural separator, and the part with a small amount of data samples is regarded as a group of candidate outliers A, with Amax-Amin+gap as its value interval width. Calculate the average interval width proportion Q of this group of candidate outliers. If Q ≥ the pre-set Qexp, the group of candidate points is determined to be outliers and this group of points is deleted from the original data. The above operations are performed in order from large to small according to the gap. If the total number of outliers determined reaches the preset upper limit, the algorithm is terminated. The detailed steps of the above process are as follows:

[0160] a) Set the upper limit of the number of outliers Omax and the expected average interval proportion of outliers Qexp;

[0161] b) sorting the N scalar floating-point data to be processed in descending order, and recording the data numbers from 1 to N;

[0162] c) Calculate the distances between adjacent data points in descending order to obtain N-1 gaps. Bind each gap and the data point with the larger value between its two corresponding endpoints into a tuple based on the sorted original data sequence number (ranging from 1 to N-1);

[0163] d) Sort the gap tuples in descending order using the gap value as the key field;

[0164] e) For the sorted gap tuples, find the corresponding candidate outlier set in the sorted original data according to their original data sequence numbers in descending order;

[0165] f) If the number of current candidate outliers is greater than the currently acceptable number of new outliers, the processing of this gap is abandoned and the process returns to step e) to continue processing the next gap;

[0166] g) Calculate the Q value Qcur of the candidate outlier set relative to the non-outlier set in the original data. If Qcur ≥ Qexp, mark the candidate outlier as an outlier in the original data and update the current acceptable number of new outliers.

[0167] h) If the current acceptable number of new outliers is 0, terminate the algorithm;

[0168] i) If all gaps have been processed, terminate the algorithm;

[0169] j) Return to step e) and continue processing the next gap.

[0170] It should be emphasized that in step g), Qcur calculates the average interval width ratio of the current candidate outlier set relative to the non-outlier set (including the current candidate outlier) in the original data set. In other words, step g) is to determine the outliers in the current data set after removing the outliers marked according to the expectations of Qexp. The reason for doing this is to mark as many outliers as possible from the original data set under the constraint of Omax, so as to minimize the value interval width of the non-outlier data and reduce the bit width required for quantization representation.

[0171] The above process can be drawn into a program execution flow chart, called the outliers_detect() algorithm, whose input parameters are:

[0172] data: the original data of the outliers to be detected, the data format is 1xN dimensions;

[0173] data_num: the amount of original data of the outlier to be detected;

[0174] Omax: The upper limit of the acceptable number of outliers;

[0175] Qexp: The expected average interval proportion of outliers.

[0176] This algorithm identifies outliers in a set of N metadata. Its goal is to identify the set of outliers with the highest degree of outlier severity, subject to the dual constraints of the maximum number of outliers, Omax, and the degree of outlier severity, Qexp. The algorithm exits when Omax outliers have been found, or when no more outliers can be found.

[0177] Taking the data set {-1.5, 1.3, 2.7, -805.5, 5000.1, 4.5, 9.2, -13.1, 100.7, 5000.2} as an example, the structure of the sorted original data is shown in the figure below.

[0178]

[0179]

[0180] The structure of the sorted gap tuples is shown in the figure below.

[0181]

[0182] If the upper limit of the number of outliers Omax is set to 1, the number of outliers and the expected distribution are: x% = 10%, y% = 90%, then Qexp = 90% / (10x 10%) = 90%. Under this initial condition, the outlier detection algorithm is run on the above example data set, and the data that is finally marked as an outlier is -805.5; if the upper limit of the number of outliers Omax is set to 2, the number of outliers and the expected distribution are: x% = 20%, y% = 90%, Qexp = 90% / (10x 20%) = 45%, then data samples 5000.2 and 5000.1 will be marked as outliers as a group, while -805.5 will not be marked as an outlier; if the upper limit of the number of outliers Omax is set to 3, the number of outliers and the expected distribution are: x% = 30%, y% = 90%, Qexp = 90% / (10x 30%)=30%, other conditions remain unchanged, then the data finally marked as outliers will be 5000.2, 5000.1 and -805.5, among which 5000.2 and 5000.1 are outliers determined by the gap number 1, and -805.5 is an outlier determined by the gap number 2.

[0183] 6. Centralized Storage and Indexing Method for Outliers in Piecewise Polynomial Coefficient Tables

[0184] The basic principle of outlier-aware quantization is to centrally store outliers in an additional array as floating-point numbers that meet precision requirements. During quantization, outliers in the polynomial are represented as indices into the array where they are stored, with the data type of these indices being integers of the quantization bit width. During dequantization, the indices representing the outliers are identified, and their floating-point values ​​can be retrieved from the array where they are stored.

[0185] Based on the mixed-precision quantization strategy for independent quantization of coefficients of different order terms proposed in Technical Point 3 above, for MxN dimensional data of coefficients of a specific order term, the method described in Technical Point 5 is applied to M groups of N metadata to perform outlier determination. All outliers in the M groups of N metadata are then stored in the order of the M groups in an array of a specific floating-point type that meets the accuracy requirements. Here, k_order_outliers_store is used to represent this array. The floating-point format used should be selected based on the quantization accuracy and the numerical range of the outliers. Options include float32, float16, or bfloat16. The representation range and accuracy (relative error limit) of these floating-point types are shown in the following table:

[0186]

[0187] In order to implement the index processing of outliers during the quantization and dequantization process of outlier perception, we use a data structure called k_order_outliers_desc to describe these M groups of outliers. Take the following MxN dimensional data as an example, assuming M = 4, N = 8, the sample data is as follows:

[0188] 1.3 2.5 -16.0 2.8 12.9 -5.0 5000.1 -600.0 21533.8 0.1 0.2 0.5 -0.03 -0.98 0.1 0.12 -2.5 18.6 -26.3 12.2 0.98 1.53 0.02 300.3 -13.8 400.5 -202.2 3.3 3663829.9 2.2 3.9 7.2

[0189] Assume that the red data in the above table are outliers marked by the outlier determination algorithm, then the structures of k_order_outliers_store and k_order_outliers_desc are as follows Figure 5 shown.

[0190] k_order_outliers_desc records the descriptive information of all outliers in the MxN dimensional original data in the form of a table. Each row in the table represents an outlier. The meanings of the table column attributes are as follows:

[0191] a) The M-dimensional sequence number and the N-dimensional sequence number are used to indicate the location of the outlier in the MxN-dimensional original data;

[0192] b) The storage index base address number indicates the starting index number of the outlier point in a set of N-element original data of a specific M dimension in the k_order_outliers_store array;

[0193] c) The storage index number indicates the offset index number of the outlier point in the k_order_outliers_store array relative to the M-dimensional storage index base address number described in b).

[0194] After the outliers in the MxN-dimensional raw data are centrally stored in a one-dimensional array, the k_order_outliers_desc data structure implements a "group + within-group offset" secondary indexing of the storage locations of the one-dimensional array elements using "storage index base address number + storage index number." The maximum value of the "storage index base address number" is the total number of outliers in the MxN-dimensional raw data minus 1, while the maximum value of the "storage index number" is the total number of outliers in a specific set of N-tuple raw data minus 1. When quantizing outliers in a specific set of N-tuple raw data, the location information of the outliers in the centralized storage must be mapped using quantized integer values. This secondary indexing method ensures that the quantized representation of each outlier only maps to the "storage index number" with a smaller value range, while the "storage index base address number" with a larger value range is shared by all outliers in the group and can be stored separately in the outlier-aware quantized metadata of the data set described in Technical Point 8, eliminating the need for mapping it using quantized integer values. The purpose of this design is to localize the storage location information of the outliers in each group of N metadata that need to be mapped by quantized values, under the premise of centralized storage of outliers in MxN dimensional data. That is, the numerical range of the index numbers that need to be mapped is only related to N and has nothing to do with M, thereby ensuring that the bit width required for mapping the storage location of outliers will not become a bottleneck for optimizing the bit width of the outlier-aware quantization representation.

[0195] The k_order_outliers_desc data structure establishes a connection between the position of the outlier in the MxN two-dimensional array of original data and the storage location of its "group + group offset" in the one-dimensional k_order_outliers_store array. During the outlier-aware quantization process, the information in the k_order_outliers_desc data structure will be used to implement the quantization representation mapping of the outlier storage location and the dequantization value recovery of the outlier. This process is described in detail in Technical Point 8.

[0196] 7. Quantitative representation and dequantization recovery methods for non-outlier data

[0197] The basic principle of integer quantization is to map the floating point representation R of real numbers with a value range of [Rmin, Rmax] to the integer representation Z of integers with a value range of [Zmin, Zmax]. Integer quantization can be further divided into symmetric quantization and asymmetric quantization according to the implementation method. Symmetric quantization requires that 0 in the floating point representation be mapped to 0 in the integer representation, while asymmetric quantization does not make this requirement. f As an example of 8-bit quantization, the mapping between symmetric quantization and asymmetric quantization is as follows Figure 6a and 6b shown.

[0198] The outlier-aware quantization method proposed in the present invention is based on the asymmetric integer quantization method, and introduces the processing of outliers. The basic idea is: a part of the integer representation space is divided into storage indexes for representing outliers, and the remaining integer representation space is used for ordinary asymmetric quantization of non-outliers.

[0199] For a one-dimensional floating-point data set consisting of a set of floating-point numbers, NR_O outliers are marked in the data set using the method described in Technical Point 5. Let the maximum value of all non-outlier data be Vmax and the minimum value be Vmin. If an integer with a bit width of b is used to perform outlier-aware quantization on this data set, then the quantization and dequantization methods for the non-outlier points are:

[0200] a) Assume that the bit width of the quantized representation is b and the number of outliers is NR_O;

[0201] b) The total number of values ​​that can be represented by a b-digit integer is 2 b , among all the values ​​that can be represented by a b-bit integer, NR_O values ​​need to be marked out to represent the indexes of NR_O outliers (see technical point 8 for the specific representation method), and the remaining 2 b -NR_O values ​​can be used to quantify the value interval [Vmin, Vmax] of non-outlier data, we call slots = 2 b -NR_O is The number of available quantization slots;

[0202] c) Step = (Vmax – Vmin) / (slots – 1) is called Quantization step size;

[0203] d) base = Vmin is called Quantified base value;

[0204] e) For non-outlier data with a value of Vx, its quantization formula is: Ix = round_to_int((Vx – base) / step), where Ix is the integer quantization representation of Vx and round_to_int() represents the operation of rounding the decimal to an integer;

[0205] f) For the non-outlier quantized representation with a value of Ix, the dequantization recovery formula is: VRx = base + Ix * step, where VRx is the real value recovered from the integer quantized representation Ix;

[0206] In the above non-outlier quantization and dequantization methods, the number of available quantization slots for non-outlier points is 2 b -NR_O, the corresponding b-bit unsigned integer has a value range of 0 to 2 b -1-NR_O. Take data set X f={1.21, 2.53, 3.13, 4.47, 5.29, 6.99, 7.22, 8.65, 9.82, 10.31, 200.57, 2000.78}, let X f The data points with medians of 200.57 and 2000.78 are outliers, so NR_O = 2. When performing outlier-aware 8-bit integer quantization, the number of available quantization slots for non-outlier points is: 2 b -NR_O=2 8 -2=254, and the corresponding 8-bit integer value is 0 to 253. The 8-bit integer values ​​254 and 255 will be used to represent the storage index numbers of the two outliers 200.57 and 2000.78 described in Technical Point 6. The specific representation method is described in detail in Technical Point 8.

[0207] 8. Quantitative Representation and Dequantization Recovery Methods for Outlier Data

[0208] For a one-dimensional floating-point dataset consisting of a set of floating-point numbers, the outlier-aware quantization method employs the following approach to quantizing outlier data: Within the integer representation range, a range of length equal to the number of outliers is reserved. The values ​​within these ranges are used to represent the storage index of the outliers in the centralized storage structure described in Technical Point 6. By associating the integer quantization representation values ​​with the storage index of the outliers, during dequantization, the storage index of the outliers is calculated from the integer quantization representation values. This storage index is then used to retrieve the floating-point dequantized value of the outliers from the centralized storage structure.

[0209] A key to outlier-aware quantization is to distinguish outliers from non-outliers by the range of the quantized integer value. This distinction can be achieved by the number of available quantization slots described in Technical Point 7. In the present invention, the range of the quantized b-bit integer value {0..slots-1} is used to represent non-outlier data, and the range of {slots,2 b -1} is used to represent outlier data.

[0210] For the MxN dimensional coefficients of a specific order term in the polynomial coefficient table, the outlier-aware quantization process for a set of N-ary floating-point data is as follows:

[0211] a) Create the meta-information data structure oaq_meta_info for outlier-aware quantization and dequantization for this set of data. Its structure is shown in the figure below.

[0212]

[0213]

[0214] b) Suppose the quantization representation bit width of this group of data is b and the number of outliers is NR_O. Record the available quantization slot bits of non-outliers slots = 2 b - NR_O in the oaq_meta_info data structure of this group of data. Calculate the quantization step step and quantization base value base of non-outliers according to the formula described in Technical Point 7; Set outliers_base_index as the storage index base sequence number of the outliers of this group of data in the k_order_outliers_desc data structure described in Technical Point 6; Set outliers_base_ptr as the memory access address of the element with an offset of outliers_base_index in the k_order_outliers_store array described in Technical Point 6

[0215] c) Traverse N original data to be quantized. If the data is a non-outlier, perform the non-outlier quantization operation according to the process described in Technical Point 7; If the data is an outlier, perform step d);

[0216] d) Find the entry corresponding to this outlier in the k_order_outliers_desc data structure described in Technical Point 6 according to the M dimension serial number and N dimension serial number of the outlier in the MxN-dimensional original data. Denote the storage index serial number in the entry as cur_outliers_idx. Then the quantization representation of the outlier is: Ix = oaq_meta_info.slots + cur_outliers_idx;

[0217] For the MxN-dimensional coefficients of a specific order term in the polynomial coefficient table, the outlier-aware inverse quantization process for a group of N-element floating-point data is as follows:

[0218] a) Find the outlier-aware quantization meta-information data structure oaq_meta_info of this group of floating-point data;

[0219] b) For the quantization representation with a value of Ix, if Ix < oaq_meta_info.slots, this data is non-outlier data. Perform the non-outlier inverse quantization operation according to the process described in Technical Point 7 based on oaq_meta_info.step and oaq_meta_info.base; If Ix >= oaq_meta_info.slots, this data is outlier data. Perform step c);

[0220] c) Restore the floating-point representation of the outlier according to the outlier storage index mapped to Ix: VRx = outliers_base_ptr[Ix–oaq_meta_info.slots], where VRx is the real value of the outlier restored according to the integer quantization representation Ix.

[0221] 9. Outlier-aware quantization and dequantization of specific-order coefficients in polynomial coefficient tables

[0222] Based on the data organization format of the piecewise polynomial coefficient table described in Technical Point 1 and the mixed-precision quantization strategy for independent quantization of coefficients of items of different orders described in Technical Point 3, in the outlier-aware quantization process, it is necessary to use a unified quantization representation bit width for the MxN-dimensional data of a specific order item in the MxNxK-dimensional polynomial coefficient table, and based on this bit width, perform outlier-aware quantization processing and dequantization data recovery on M groups of N metadata respectively.

[0223] Here we use the MxN dimensional data shown in Technical Point 6 as an example. The dimensions of the original data are M=4, N=8. Assume that the integer quantization bit width to be performed is 8 bits. The quantization and dequantization process is demonstrated using the first set of N metadata {1.3, 2.5, -16.0, 2.8, 12.9, -5.0, 5000.1, -600.0} as an example. We present the relationship between the MxN dimensional original data, the storage and index structure of the outliers, and the quantization and dequantization source information data structure of the first set of N metadata with M=0 in Figure 7 middle.

[0224] For N metadata of group M=0, the quantization width b=8, the number of outliers NR_O=2, and the number of available quantization slots slots=2 8 –2=254, the maximum value Vmax of the non-outlier points in the data is 12.9, and the minimum value Vmin is -16.0. The quantization parameters of the non-outlier points are calculated according to the formula described in Technical Point 7: Quantization step step=(Vmax–Vmin) / (slots-1)=(12.9+16.0) / (254–1)=0.11422925, quantization base value base=Vmin=-16.0, and the above calculation results are recorded in the slots, step, and base members of oaq_meta_info; the outliers of this group of data have a storage index base address number of 0 in the centralized storage index data structure k_order_outliers_desc, as indicated by the red circle in the figure above. Correspondingly, the outliers_base_index in the oaq_meta_info data structure is set to 0, and outliers_base_ptr points to the 0th element of the k_order_outliers_store array.

[0225] After completing the above outlier-aware quantization parameter calculation, based on the parameters recorded in the oaq_meta_info data structure, and following the quantization and dequantization processes for non-outliers and outliers described in Technical Points 7 and 8, the calculation process and results of the quantization representation and dequantization recovery of this set of data are shown in the following figure:

[0226]

[0227] When quantizing the MxN-dimensional data of a specific order term in the MxNxK-dimensional polynomial coefficient table, the outlier-aware quantization operation is performed on the M groups of N metadata in the MxN-dimensional original data according to the above process. Taking the above-mentioned MxN-dimensional data as an example, the MxN-dimensional original data, the outlier-aware quantization parameters of the M groups of N metadata, the 8-bit outlier-aware quantization results of the MxN-dimensional original data, and the dequantization recovery results of the MxN-dimensional 8-bit outlier-aware quantized data are respectively as follows: Figure 8 shown.

[0228] It should be noted that the outlier-aware quantization operation of the MxNxK-dimensional piecewise polynomial coefficient table is performed in batches on K groups of MxN-dimensional data during the post-processing model compression phase after the Deep Potential model training is completed, as shown in the figure above. However, the dequantization operation is performed during the inference phase of the Deep Potential model. According to the input data of the Embedding Net, a segmented search is performed in the M dimension. After selecting the NxK-dimensional polynomial coefficient table corresponding to the segment of the input data, the K groups of N metadata in the NxK-dimensional table are dequantized and restored separately.

[0229] 10. Quantitative error analysis of polynomial coefficient tables and control indicators of error levels

[0230] For the Deep Potential model's MxNxK-dimensional piecewise polynomial coefficient table, the M dimension corresponds to the number of segments in which the Embedding Net's one-dimensional input x is tabled. For each segment of x, an NxK-dimensional polynomial coefficient table represents the fitting of the Embedding Net's N-dimensional output on that segment using a set of N K-1-order polynomials. For each segment of the NxK-dimensional polynomial coefficient table, we analyze the error in the polynomial fitting value caused by quantization. Assume that the K-1-order polynomial is in the following form:

[0231] y=a0+a1*x+a2*x 2 +…+ak*x k =y0+y1+y2+…+yk (1)

[0232] In the above formula, a0..ak are the polynomial coefficients, and y0..yk are the values ​​of the polynomial terms of each order. Due to the error introduced by quantization, we denote the dequantized recovery value of a0..ak as according to The calculated values ​​of y0..yk are recorded as The inverse quantized polynomial fitting value obtained by summing is recorded as Then we have:

[0233]

[0234] set up The absolute error limit is Since the present invention performs integer quantization on the polynomial coefficients a0..ak, the absolute error limit of its inverse quantization is They are 1 / 2 of the quantization step size of a0..ak, that is, is a constant and has nothing to do with the values ​​of a0..ak. When the input value x is determined, It is also a constant and its value is independent of the values ​​of a0..ak.

[0235] set up The absolute error limit is because yes The absolute value of the sum of may be close enough to 0, so The relative error limit It may be very large. Based on this, we can confirm that due to the floating-point addition operation in the polynomial fitting formula, under the premise of quantizing the coefficients of each order of the polynomial, It is not feasible to precisely control the relative error limit of The value of is the output of the Embedding Net and the input of the Fitting Net. Therefore, controlling its absolute error limit is still feasible and necessary. Therefore, controlling the absolute error limit of the polynomial fitting value is the main goal of the error control mechanism of the present invention. We have the following analysis and explanation of some of the prerequisite technical conditions and specific implementation strategies for this goal:

[0236] a) Control the absolute error limit of the polynomial fitting value by controlling the absolute error limit of each term of the polynomial;

[0237] b) In the Deep Potential model, the value range of the one-dimensional scalar input x of the Embedding Net is positive. The model compression algorithm of the piecewise polynomial fitting divides the input x of the Embedding Net into small segments [x min ,x max], since the power function of x is monotonically increasing on the interval [0,1], the polynomial term value is in the segment [x min ,x max The absolute error limit on the polynomial is equal to the absolute error limit of the polynomial coefficient multiplied by x max The corresponding power of

[0238] c) Absolute error limit Δ for polynomial coefficients TC This is equal to 1 / 2 of the quantization step size of the polynomial coefficients during the quantization phase. Therefore, in the quantization process of non-outlier points described in Technical Point 7, for a numerically determined polynomial coefficient table, the absolute error limit of the polynomial coefficients corresponding to the non-outlier points will be determined by the quantization bit width. Therefore, regulating the quantization bit width in combination with data distribution characteristics is a direct means of implementing quantization error control.

[0239] d) For outliers in the polynomial coefficient table, since their total number in the coefficient table is very small, in the centralized storage method for outliers described in Technical Point 6, outliers are stored using a floating-point format with sufficient precision. Therefore, all outliers are ignored in the quantization error control analysis of the present invention, and the values ​​of the outliers in subsequent items e) to l) are all treated as 0 for processing;

[0240] e) Since the fitted value of a polynomial is the sum of the values ​​of each term, the absolute error limit of the fitted value of the polynomial is the sum of the absolute error limits of each term. However, when the absolute error limits of the terms are not of the same magnitude, the error limit with the larger magnitude will dominate the absolute error limit of the polynomial fitted value. This can be seen as the quantization bit width allocated to the coefficients of the terms with the smaller magnitude is not being used to most efficiently reduce the absolute error limit of the polynomial fitted value. In other words, too much quantization bit width is allocated to the coefficients of the terms with the smaller magnitude.

[0241] f) Based on the above item e), the present invention determines a specific implementation strategy for controlling the absolute error limit of the polynomial fitting value: controlling the absolute error limit of each order term to the same level, and based on this, analyzing the quantization error of the polynomial coefficient table and calculating the quantization bit width;

[0242] g) For NxK-dimensional polynomial tables, based on Technical Point 7 and the aforementioned item c), the quantization process is to independently quantize each of the K groups of N elements. Therefore, the N polynomial coefficients of the same order have the same quantization step size and absolute error limit. Furthermore, the N polynomial coefficients of the same order are multiplied by the same power of the same input value x to obtain the term value. Therefore, the absolute error limits of the term values ​​of the N polynomials of the same order corresponding to the NxK-dimensional polynomial table are the same.

[0243] h) Based on the above items f) and g), it can be concluded that: under the premise of controlling the absolute error limits of the polynomial values ​​at the same level, for any specific NxK-dimensional polynomial coefficient table in the MxNxK-dimensional polynomial coefficient table, the absolute error limits of the NxK terms of the corresponding N polynomials should be the same, denoted by Δ TV ;

[0244] i) The present invention proposes a method of setting a uniform absolute error limit Δ for the NxK items corresponding to any specific NxK-dimensional polynomial coefficient table in an MxNxK-dimensional polynomial coefficient table by setting an empirical relative error limit. TV The method used to determine Δ TV The empirical relative error limit is marked as ER TV , the details of the method are as described in items j)-l) below;

[0245] j) Consider a single polynomial. The value of the polynomial is obtained by summing up the values ​​of each item. When the polynomial coefficients are quantized, the item value errors caused by the quantization will be included in the summation operation together with the item values. Due to the dominant influence of the participating items with larger absolute values ​​brought by the summation operation on the result, under the premise that the absolute error limits of each item value are the same, the ratio of this absolute error limit to the absolute value of the larger item value can more accurately reflect the quantitative proportional relationship between the absolute error limit of the item value and the absolute value of each item value. Therefore, when the relative error limit ER of the polynomial item value is used, TV To set a uniform absolute error limit Δ for each value of a single polynomial TV When the maximum absolute value of each polynomial is TV max , then we can let Δ TV =ER TV *TV max , and the term with the largest absolute value in a polynomial is called Dominant term of a polynomial .

[0246] k) Consider the N polynomials corresponding to the NxK dimensional polynomial coefficient table, and let the absolute value of the leading term of each polynomial be TV max_i , i=0..N-1. Since the N polynomials are independent of each other and have no direct operational relationship, TV Set a uniform Δ for NxK item values TV When using TV max_min =min{TV max_i |i=0..N-1} is the benchmark, let Δ TV =ER TV *TV max_min , then it can be guaranteed that for any polynomial i among the N polynomials: Δ TV / TV max_i ≤ERTV ;TV max_i Refers to the maximum absolute value of the K terms of a K-1 order polynomial y = a0 + a1*x + a2*x^2 + ... + aK*x^K;

[0247] l) Consider the MxNxK dimensional polynomial coefficient table, as described in Technical Points 3., 7., 9., and the aforementioned Items c) and g), where the MxN dimensional polynomial coefficients of any particular order item use a uniform quantization bit width, and the quantization step size and absolute error limit of each set of N data elements are the same, but the quantization step size and absolute error limit of each of the M sets of N data elements are not necessarily the same, so the absolute error limits of the corresponding M sets of N item values ​​are also different. This means that for the MxNxK dimensional polynomial coefficient table, we can use the method described in Item k) to use a uniform ER TV Set the absolute error limit Δ for each item value of the M polynomial groups corresponding to the M NxK dimensional polynomial coefficient tables TV_i |i=0..M-1, without setting the same absolute error limit Δ for the M polynomial groups described by the MxNxK dimensional polynomial coefficient table TV

[0248] 11. Quantization error control of polynomial coefficient table and calculation of optimal quantization bit width of each order coefficient

[0249] Based on the technical point 10. The quantitative error control target of the polynomial coefficient table and the empirical relative error limit index ER TV , the quantization error control method and process for the polynomial coefficient table are as follows:

[0250] a) Based on the process items a) and b) described in technical point 11, for the MxNxK dimensional piecewise polynomial coefficient table, obtain the quantization error control index ER TV , outlier determination indicators x%, y%, and Qexp

[0251] b) Based on the process described in Technical Point 10, calculate the absolute error limits Δ of the terms of the M polynomial groups corresponding to the M NxK-dimensional polynomial coefficient tables. TV_i |i=0..M-1;

[0252] c) Calculate the absolute error limits allowed for the quantization representation of MxK sets of N metadata respectively: Determine the one-dimensional scalar input interval [x min_i ,x max_i ]|i=0..M-1, double traverse M and K dimensions, based on the technical point 10. item b), calculate Δ CV_ij =Δ TV_i / (xmax_i ) j |i=0..M-1,j=0..K-1. Remember Δ CV is the absolute error limit of a set of N metadata in the NxK dimensional polynomial coefficient table, Δ CV_ij The absolute error limits of the K groups of N data in the M NxK dimensional tables corresponding to the MxNxK dimensional piecewise polynomial coefficient tables;

[0253] d) MxNxK dimensional piecewise polynomial coefficient table, which is divided into MxK groups of N metadata, and the outlier determination method described in technical point 5 is executed respectively, and the number of outliers NR_O of each group of N metadata is recorded. ij |i=0..M-1,j=0..K-1, and calculate the non-outlier value range R of each group of data V_ij =V max_ij –V min_ij |i=0..M-1, j=0..K-1, let Q_SLOTS_ ij =R V_ij / (2*Δ CV_ij )+NR_O ij |i=0..M-1,j=0..K-1, then Q_BITS_ ij =ceil(log2(Q_SLOTS_ ij ))|i=0..M-1, j=0..K-1. Q_BITS_ ij The empirical relative error limit index ER determined in the above item a) corresponds to the K groups of N data in the M NxK dimensional tables of the MxNxK dimensional piecewise polynomial coefficient table TV , outlier determination indicators x%, y%, and Qexp are jointly constrained to execute technical point 7 and technical point 9. The minimum quantization bit width required for the outlier-aware quantization method;

[0254] e) Based on the mixed precision quantization strategy for independent quantization of coefficients of different orders described in Technical Point 3, for an MxNxK-dimensional piecewise polynomial coefficient table, it is necessary to split it along the K dimension into K groups of NxM-dimensional data distinguished by polynomial order. The quantization bit widths L0-L0 are set for each of these K groups of data. K-1 Based on the above item d), let L j =max{Q_BITS_ ij |i=0..M-1}|j=0..K-1, the quantization bit width L0-L of each order coefficient of the MxNxK dimensional piecewise polynomial coefficient table can be obtained. K-1 The quantization bit width is the empirical relative error limit index ER determined in the above item a). TV, outlier judgment indicators x%, y% and Qexp, when executing the mixed precision quantization strategy for independent quantization of coefficients of different orders as described in technical point 3, the optimal quantization bit width of each order coefficient.

[0255] 12. Overall process of outlier-aware adaptive mixed-precision quantization of polynomial coefficient tables

[0256] The outlier-aware quantization operation of the MxNxK-dimensional piecewise polynomial coefficient table is performed in the post-processing model compression stage after the Deep Potential model training is completed. The overall process is as follows:

[0257] a) For the MxNxK dimensional piecewise polynomial coefficient table, the empirical relative error limit index ER described in Technical Point 10 is determined based on experience. TV Reference value of

[0258] b) For the MxNxK dimensional piecewise polynomial coefficient table, empirically determine the reference values ​​of the outlier determination indicators x%, y%, and Qexp described in Technical Point 4;

[0259] c) According to the process described in Technical Point 11, determine the optimal quantization bit width L0-L for each order coefficient when executing the mixed precision quantization strategy for independent quantization of coefficients of different order terms described in Technical Point 3. K-1 ;

[0260] d) In the process of technical point 11. described in item c) above, outlier marking of the MxNxK dimensional piecewise polynomial coefficient table is also implemented based on the outlier determination method and process described in technical point 5.;

[0261] e) For the MxNxK dimensional piecewise polynomial coefficient table, it is necessary to split it along the K dimension into K groups of NxM dimensional data distinguished by the polynomial order, with each group corresponding to the quantization bit width L0-L obtained in the above item c). K-1 According to the outlier-aware quantization process described in Technical Point 9, the outlier-aware quantization operations for the K groups of data are completed respectively;

[0262] f) The process of technical point 9 described in the aforementioned item e) includes the centralized storage and indexing of outliers described in technical point 6, the quantitative representation of non-outlier data described in technical point 7, and the quantitative representation of outlier data described in technical point 8.

[0263] The outlier-aware dequantization operation of the quantized MxNxK-dimensional piecewise polynomial coefficient table is performed during the inference phase of the DeepPotential model. Based on the input data of the Embedding Net, segmented search is performed in the M dimension. After selecting the NxK-dimensional data table corresponding to the segment of the input data, the dequantization operation is performed on this specific NxK-dimensional table. The overall process is as follows:

[0264] a) For a specific NxK-dimensional quantized data table, divide it into K groups of N-dimensional data, and execute the outlier-aware dequantization process described in Technical Point 9 on each group to obtain an NxK-dimensional polynomial coefficient table after dequantization recovery.

[0265] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the description and implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. An outlier-aware adaptive mixed-precision quantization method for an interatomic potential energy surface model, comprising: Get the atomic system to be fitted with the atomic potential energy, which includes multiple atoms. Take each atom as the central atom and combine it with the preset cutoff radius to generate the relative coordinate set R of the neighboring atoms of each central atom. i ; According to the positional relationship between the central atom and all its neighboring atoms, the local environment matrix of each central atom is constructed ,by As the input of the interatomic potential energy surface model, the potential energy E of each central atom is obtained by fitting i ; Among them, the interatomic potential energy surface model includes two parts: Embedding Net and Fitting Net. EmbeddingNet takes the input Mapped to the embedding space, according to Generate the embedding matrix G of the central atom i ; Through R i With G i Calculation of the local environment descriptor D that ensures physical symmetry i ;Fitting Net is D i As input, the potential energy E of the central atom is obtained by fitting i ; Among them, according to R i generate i The process is: for each neighbor atom j of the central atom i, execute: The left side of the above formula is R i , on the right , is the relative coordinate of neighbor atom j relative to the central atom i, r ji Represents the distance between atoms i and j; where r cs is the smooth cutoff parameter, r c is the cutoff radius; The neural network G in Embedding Net completes the S(r ji ) Generate G i , , select a multi-order piecewise polynomial to fit the neural network G, so as to transform the neural network G into a piecewise polynomial coefficient table of MxNxK dimensions, where M is the number of segments, N is the output dimension of the neural network G, and K is the total order of the multi-order piecewise polynomial; according to S(r ji ) in the segmented interval, the coefficient table of the segmented polynomial is looked up for each output dimension, and S(r ji ) and calculate the polynomial fitting value of each output dimension to get G i ; It is characterized in that The internal data of the MxNxK-dimensional piecewise polynomial coefficient table is a floating-point number. The piecewise polynomial coefficient table is quantized based on outlier perception mixed precision to convert the piecewise polynomial coefficient table into a quantized coefficient table with fixed-point numbers as the internal data. ji ) is dequantized in the segment interval where the quantization coefficient table is located, and S(r ji ) polynomial coefficients, the specific process includes: Step 1: Divide the piecewise polynomial coefficient table into K groups of NxM dimensional data along the K dimension, which are distinguished by the polynomial order, and set the quantization bit width L0-L for each of the K groups of data. K-1 ; Step 2: In each set of floating-point data in the K sets of data, the maximum value is Vmax, the minimum value is Vmin, the maximum value of non-outlier points is Nsup, and the minimum value of non-outlier points is Ninf. After sorting each set of floating-point data, data is retrieved in descending order from Vmax to Nsup, and data is retrieved in ascending order from Vmin to Ninf. Data within the interval [Vmin, Ninf) and (Nsup, Vmax] are outliers. Step 3: The outlier points in the piecewise polynomial coefficient table are centrally stored in an additional outlier array. In the quantization representation process, the outlier points are represented as indices pointing to the storage array where they are located. The non-outlier points in the piecewise polynomial coefficient table are quantized into fixed-point numbers to obtain the quantized coefficient table. Step 4: According to the input data S(r ji ) Perform segmented search in M ​​dimensions, select the NxK dimensional data table corresponding to the segment of the quantization coefficient table input data, divide it into K groups of N metadata, perform the inverse quantization process for each group of data, and identify the index of the outlier in the inverse quantization process. According to the index, find the floating point value of the outlier from the outlier array, and obtain the NxK dimensional polynomial coefficient table after inverse quantization as the input data S(r ji ) of the polynomial coefficients.

2. The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model according to claim 1, characterized in that: The determination of the outlier includes: The number of outliers accounts for x%, and the width of the outlier interval is: (Vmax-Nsup)+(Ninf-Vmin). The ratio of the outlier interval to the value interval of the data set ((Vmax-Nsup)+(Ninf-Vmin)) / (Vmax-Vmin) is y%. Qexp = y% / (N * x%), where Qexp is the expected proportion of the average interval width of the outliers, indicating the overall degree of outliers. In MxNxK dimensional data, for a set of coefficients of a specific order item in a specific NxK dimensional table, Qexp is calculated according to the preset x%, y%. Then, when a set of candidate outliers is found from this set of coefficients, the number of candidate outliers is j, and the proportion of their value interval is k. For this set of candidate outliers, the true average interval proportion Q=k / j is calculated. If the Q value is not less than Qexp, the set of candidate points is determined to be the outliers.

3. The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model according to claim 1, characterized in that: This step 3 includes: For the MxN dimensional data in the piecewise polynomial coefficient table, all outliers in the M groups of N metadata are stored in a floating-point type array k_order_outliers_store that meets the preset accuracy requirements in the order of the M groups. A data structure named k_order_outliers_desc is used to describe these M groups of outliers, including the M-dimensional sequence number and N-dimensional sequence number of the outliers to indicate the position of the outliers in the MxN dimensional original data, and the storage index base address sequence number to indicate the starting index sequence number of the outliers in a specific M-dimensional group of N-element original data in the k_order_outliers_store array, and the storage index sequence number to indicate the offset index sequence number of the outliers in the k_order_outliers_store array relative to the index base address sequence number where the outliers are located; After the outliers in the MxN-dimensional original data are centrally stored in the form of a one-dimensional array, the k_order_outliers_desc data structure implements the secondary indexing of the storage position of the one-dimensional array elements by grouping + offset within the group through the storage index base address sequence number + storage index sequence number, where the maximum value of the storage index base address sequence number is the total number of outliers in the MxN-dimensional original data minus 1, and the maximum value of the storage index sequence number is the total number of outliers in each group of N-element original data minus 1; when the outliers in each group of N-element original data are quantized, the quantized integer value is used to map the position information of the outlier in the centralized storage.

4. The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model according to claim 3, characterized in that: This step 4 includes quantization and dequantization of non-outlier data, specifically including: A portion of the integer representation space is divided to represent the storage index of outliers, and the remaining integer representation space is used to asymmetric quantize non-outliers; For a one-dimensional floating-point data set consisting of a set of floating-point numbers, NR_O outliers are marked in the data set. Let the maximum value of all non-outlier data be Vmax and the minimum value be Vmin. Use integer numbers with a bit width of b to perform outlier-aware quantization on this data set. The quantization and dequantization methods of the non-outlier points are: The bit width of the quantized representation is b, and the total number of values ​​that can be represented by a b-bit integer is 2 b , among all the values ​​that can be represented by a b-bit integer, NR_O values ​​are marked out to represent the indices of NR_O outliers, and the remaining 2 b -NR_O values ​​are used to quantify the value interval [Vmin, Vmax] of non-outlier data, slots = 2 b -NR_O is the number of available quantization slots, step = (Vmax – Vmin) / (slots – 1) is the quantization step size, and base = Vmin is the quantization base value; For non-outlier data with a value of Vx, its quantization representation is: Ix = round_to_int((Vx – base) / step), where Ix is the integer quantization representation of Vx, and round_to_int() represents the operation of rounding the decimal to an integer; For the non-outlier quantization representation with a value of Ix, the dequantization recovery formula is: VRx = base + Ix * step, where VRx is the real value recovered from the integer quantization representation Ix.

5. The outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model according to claim 4, characterized in that: This step 4 includes quantization and dequantization of outlier data, specifically including: The value range of the quantized b-bit integer is {0..slots-1}, which is used to represent non-outlier data. The value range is {slots, 2 b -1} is used to represent outlier data; for the MxN dimensional coefficients in the piecewise polynomial coefficient table, the outlier-aware quantization process for each group of N-dimensional floating-point data is as follows: Establish a metadata data structure oaq_meta_info for outlier-aware quantization and inverse quantization for the N-tuple floating-point data. oaq_meta_info is the metadata of the outlier-aware quantization and inverse quantization of the N-tuple data, including recording the number of available quantization slots for non-outlier points in the group of data, the quantization step size step for non-outlier quantization, the quantization base value base for non-outlier quantization, the storage index base address sequence number outliers_base_index of the outliers in the group of data in the centralized storage array, and the memory access base address outliers_base_index pointing to the k_order_outliers_store array for the outliers in the group of data. The quantization representation bit width of the N-ary floating-point data is b, the number of outliers is NR_O, and the number of available quantization slots for non-outliers is recorded in the oaq_meta_info data structure of the N-ary floating-point data: slots = 2 b -NR_O, sets outliers_base_index to the storage index base address sequence number of the outlier point in the k_order_outliers_desc data structure; sets outliers_base_ptr to the memory access address of the element with offset outliers_base_index in the k_order_outliers_store array; searches for the entry corresponding to the outlier point in the k_order_outliers_desc data structure according to the M-dimensional sequence number and the N-dimensional sequence number of the outlier point in the MxN-dimensional original data in the N-ary floating-point data, and records the storage index sequence number in the entry as cur_outliers_idx. The quantitative representation of the outlier point is: Ix = oaq_meta_info.slots + cur_outliers_idx; For the MxN dimensional coefficients of a specific order term in the polynomial coefficient table, the outlier-aware dequantization process for a set of N-ary floating-point data is as follows: Determine the outlier-aware quantization metadata data structure oaq_meta_info for this set of floating-point data. For a quantized representation with a value of Ix, if Ix >= oaq_meta_info.slots, the data is outlier data. Restore the floating-point representation of the outlier based on the outlier storage index mapped to Ix: VRx = outliers_base_ptr[Ix – oaq_meta_info.slots], where VRx is the real value of the outlier recovered from the integer quantized representation Ix.

6. The outlier-aware adaptive mixed-precision quantization method for an interatomic potential energy surface model according to claim 4, wherein: During the quantization process, the relative error limit ER of the preset polynomial term value is TV Determine the absolute error limit of each coefficient in the polynomial coefficient table, and calculate the quantization bit width L0-L that meets the error limit requirements for each K group of NxM dimension coefficient data K-1 To perform adaptive quantization error control of the polynomial coefficient table, specifically including: According to ER TV Set the absolute error limit Δ for the polynomial term values ​​of M NxK dimensional polynomial coefficient tables TV , for each NxK dimension coefficient table, TV max_min = min{TV max _ i | i=0..N-1} is the benchmark, where TV max _ i | i=0..N-1 is the maximum absolute value of the polynomial term corresponding to the K coefficients in each row of the table, let Δ TV = ER TV * TV max_min The absolute error limit of the polynomial term value corresponding to the NxK dimensional table is obtained by repeating the above operation for M NxK dimensional polynomial coefficient tables to obtain the absolute error limit Δ of the polynomial term value corresponding to each of the M NxK dimensional tables. TV_i | i = 0..M-1; Calculate the absolute error limit allowed by the quantization representation of MxK groups of N-element coefficient data respectively: Determine the one-dimensional scalar input interval [x min_i , x max_i ] | i = 0..M-1, double traverse M and K dimensions, calculate Δ CV_ij = Δ TV_i / (x max_i ) j | i = 0..M-1, j=0..K-1; note Δ CV is the absolute error limit of a set of N-dimensional coefficient data in the NxK-dimensional polynomial coefficient table, Δ CV_ij The absolute error limits of the K groups of N-element coefficient data in the M NxK-dimensional tables corresponding to the MxNxK-dimensional piecewise polynomial coefficient table; The MxNxK dimensional piecewise polynomial coefficient table is divided into MxK groups of N-ary coefficient data, and the number of outliers in each group of N-ary coefficient data is recorded after outlier determination. ij | i = 0..M-1, j = 0..K-1, and calculate the range of non-outlier points R for each group of data V_ij = V max_ij – V min_ij | i = 0..M-1, j=0..K-1, let Q_SLOTS_ ij = R V_ij / (2*Δ CV_ij ) + NR_O ij | i = 0..M-1, j=0..K-1, then Q_BITS_ ij = ceil(log2(Q_SLOTS_ ij ))|i = 0..M-1, j=0..K-1; Q_SLOTS _ij 、Q_BITS_ ij The K groups of N-element coefficient data in the M NxK dimensional tables corresponding to the MxNxK dimensional piecewise polynomial coefficient table are within the relative error limit index ER TV , the minimum number of quantization slots and the minimum quantization bit width required for the outlier-aware quantization method under the common constraints of the outlier determination indicators x%, y%, and Qexp; Let L j =max{Q_BITS_ ij | i = 0..M-1} | j = 0..K-1, we can get the quantization bit width L0-L of each order coefficient of the MxNxK dimensional piecewise polynomial coefficient table K-1 .

7. The outlier-aware adaptive mixed-precision quantization method for an interatomic potential energy surface model according to claim 6, wherein: The relative error limit ER TV The calculation process includes: For each N polynomial described by the NxK dimensional polynomial coefficient table, extract the maximum TV of the absolute value of all terms in each of the N polynomials. max , get N TV max , in N TV max Select the smallest TV max As TV max_min , according to the preset absolute error limit Δ of NxK items TV , ER TV =Δ TV / TV max_min .

8. An outlier-aware adaptive mixed-precision quantization system for an interatomic potential energy surface model, comprising: It is used to obtain the atomic system to be fitted with atomic potential energy. The atomic system includes multiple atoms. Each atom is taken as the central atom and combined with the preset cutoff radius to generate the relative coordinate set R of the neighboring atoms of each central atom. i ; According to the positional relationship between the central atom and all its neighboring atoms, the local environment matrix of each central atom is constructed i ,by i As the input of the interatomic potential energy surface model, the potential energy E of each central atom is obtained by fitting i ; Among them, the interatomic potential energy surface model includes two parts: Embedding Net and Fitting Net. EmbeddingNet takes the input i Mapped to the embedding space, according to i Generate the embedding matrix G of the central atom i ; Through R i With G i Calculation of the local environment descriptor D that ensures physical symmetry i ;Fitting Net is D i As input, the potential energy E of the central atom is obtained by fitting i ; Among them, according to R i generate i The process is: for each neighbor atom j of the central atom i, execute: The left side of the above formula is R i , on the right i , is the relative coordinate of neighbor atom j relative to the central atom i, r ji Represents the distance between atoms i and j; where r cs is the smooth cutoff parameter, r c is the cutoff radius; The neural network G in Embedding Net completes the S(r ji ) Generate G i , , select a multi-order piecewise polynomial to fit the neural network G, so as to transform the neural network G into a piecewise polynomial coefficient table of MxNxK dimensions, where M is the number of segments, N is the output dimension of the neural network G, and K is the total order of the multi-order piecewise polynomial; according to S(r ji ) in the segmented interval, the coefficient table of the segmented polynomial is looked up for each output dimension, and S(r ji ) and calculate the polynomial fitting value of each output dimension to get G i ; It is characterized in that The internal data of the MxNxK-dimensional piecewise polynomial coefficient table is a floating-point number. The piecewise polynomial coefficient table is quantized based on outlier perception mixed precision to convert the piecewise polynomial coefficient table into a quantized coefficient table with fixed-point numbers as the internal data. ji ) is dequantized in the segment interval where the quantization coefficient table is located, and S(r ji ), the specific process includes: Module 1 is used to divide the piecewise polynomial coefficient table into K groups of NxM dimensional data along K dimensions, which are distinguished by the polynomial order, and set the quantization bit width L0-L for each of the K groups of data. K-1 ; Module 2 is used to sort each group of floating-point data in the K groups of data, where the maximum value is Vmax, the minimum value is Vmin, the maximum value of non-outliers is Nsup, and the minimum value of non-outliers is Ninf. Then, after sorting each group of floating-point data, data is retrieved in descending order from Vmax to Nsup, and data is retrieved in ascending order from Vmin to Ninf. Data within the interval [Vmin, Ninf) and (Nsup, Vmax] are outliers. Module 3 is configured to centrally store the outlier points in the piecewise polynomial coefficient table in an additional outlier array. During the quantization process, the outlier points are represented as indices pointing to the storage array in which they are located. The non-outlier points in the piecewise polynomial coefficient table are quantized into fixed-point numbers to obtain the quantized coefficient table. Module 4 is used to calculate the input data S(r ji ) Perform segmented search in M ​​dimensions, select the NxK dimensional data table corresponding to the segment of the quantization coefficient table input data, divide it into K groups of N metadata, perform the inverse quantization process for each group of data, and identify the index of the outlier in the inverse quantization process. According to the index, find the floating point value of the outlier from the outlier array, and obtain the NxK dimensional polynomial coefficient table after inverse quantization as the input data S(r ji ) of the polynomial coefficients.

9. A server, characterized in that: An outlier-aware adaptive mixed-precision quantization device comprising the interatomic potential energy surface model described in claim 8.

10. A storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the outlier-aware adaptive mixed-precision quantization method for the interatomic potential energy surface model described in any one of claims 1 to 7.

Images