Atomic pair distribution function-oriented database construction method

Abstract

The invention relates to an atom pair distribution function-oriented database construction method, which comprises the following steps of: receiving actually-measured atom pair distribution function data. Gr, a corresponding selectable structure file. Cif and related original scattering data. Iq / . Sq / . Fq, analyzing experimental parameters from a PDF data file header. Gr, carrying out consistency check on the parameters, and determining whether the experimental parameters are consistent or not; the parameters passing the inspection, the corresponding PDF data. Gr, the related original scattering data. Iq / . Sq / . Fq and the standardized metadata are uniformly stored in a database; when the database receives a verification task, I (Q)-S (Q)-F (Q)-G (r) reprocessing is executed again according to the uploaded metadata to obtain a verified PDF, a quality score is generated according to cosine similarity, PDF data with the quality score lower than a threshold value are marked, and a difference positioning report is generated. Compared with the prior art, the method has the advantages that the data management and query efficiency is improved, the data difference source is favorably positioned, and the requirements of local structure analysis, artificial intelligence and other applications on high-quality PDF data are met.

Description

Technical Field

[0001] This invention relates to the field of structural characterization data technology, and in particular to a database construction method oriented towards the distribution function of atom pairs. Background Technology

[0002] With the continuous development of materials science research, researchers' demand for structural characterization data is increasing, especially atomic pair distribution function (PDF) data, which plays a crucial role in revealing local structures and analyzing disordered and amorphous material systems. Currently, atomic pair distribution function (PDF) data in materials science is mainly obtained through independent experiments, and researchers typically store the data scattered across academic literature, personal hard drives, or local storage systems on laboratory beamlines. While a few general-purpose materials database platforms offer centralized storage and download functions for some data, they are mostly limited to material properties, crystal structures, or macroscopic performance data, and have not yet provided systematic support for PDF experimental data.

[0003] However, these databases generally suffer from the following shortcomings: a lack of automatic extraction and standardization of parameters from experimental files, making it difficult to ensure the consistency of data entry; a lack of reprocessing and reproducibility verification mechanisms, making it difficult to confirm the reliability of uploaded data; a lack of efficient retrieval methods based on logical expressions, resulting in low data query efficiency; and a lack of end-to-end traceability mechanisms, making it difficult to verify the source and processing of data. These problems mean that PDF experimental data still exhibits inconsistency, poor reproducibility, and low retrieval efficiency, failing to meet the needs of high-throughput research in materials science and artificial intelligence applications. Furthermore, due to the lack of a unified data structure description and quality evaluation system in existing platforms, PDF data struggles to achieve comparability and quantitative quality assessment across laboratories and processing workflows. Summary of the Invention

[0004] The purpose of this invention is to address the problems of existing PDF experimental data, such as lack of a unified format, inability to automatically extract processing parameters, non-reproducibility of data, difficulty in quantifying and evaluating quality, and lack of efficient retrieval and traceability management. This invention proposes a database construction method oriented towards the distribution function of atom pairs.

[0005] The objective of this invention can be achieved through the following technical solutions: A database construction method oriented towards the distribution function of atomic pairs, the method comprising the following steps: The system receives measured atom pair distribution function data (.gr) and related raw scattering data (.iq, .sq, and .fq). Preferably, experimental parameters are analyzed from the atom pair distribution function data. If any parameters are missing, they are extracted from the related raw scattering data. The parameters are then checked for consistency. The parameters that pass the check, the corresponding atom pair distribution function data (.gr), the related raw scattering data (.iq, .sq, and .fq), and the standardized metadata are all stored in the database. When the database receives a verification task, it first performs verification PDF calculation based on the metadata to obtain the verification PDF. Then, it generates a quality score based on the cosine similarity between the verification atom pair distribution function and the user-uploaded atom pair distribution function. If the metadata with a quality score below a threshold is marked, a difference localization report is generated.

[0006] Furthermore, in the relevant raw scattering data .iq, .sq, and .fq, .iq represents the total scattering intensity data, .sq represents the structure function, and .fq represents the reduced structure function.

[0007] Furthermore, the specific steps for calculating and verifying the PDF are as follows: Acquire the total scattering data and calculate the verification structure function S(Q), the verification reduced structure function F(Q), and the verification atom pair distribution function G(r) in sequence.

[0008] Furthermore, the quality score is: in, Indicates quality score, This represents the i-th data point used to verify the distribution function of the atom pair. This represents the i-th data point of the atomic pair distribution function uploaded by the user, where n represents the total number of data points.

[0009] Furthermore, the method also includes: When the database receives a retrieval request, it parses the elements or chemical formulas in the retrieval request logical expression. If the logical expression contains operators, it performs logical operations according to preset priorities to filter candidate datasets and obtain retrieval results.

[0010] Furthermore, the search request may include one or more of the following: material category, data type, reciprocal space and real space range, temperature, pressure, or experimental facility.

[0011] Furthermore, the search results are presented in the form of tables or cards.

[0012] Furthermore, the method also includes: when a structure file containing crystal structure information is received as input and a structure fitting operation is triggered, selecting target atom pair distribution function data from the database; then calculating the theoretical atom pair distribution function based on the atomic coordinates and unit cell information obtained from the structure file parsing; and fitting the theoretical atom pair distribution function with the target atom pair distribution function data and outputting a weighted residual factor, and analyzing and evaluating the local structure model based on the weighted residual factor; The weighted residual factor is: Where, r i This represents the i-th specific distance from the reference atom. Represents the distribution function of atomic pairs. This represents the distance calculated based on the structural model parameter set P. The theoretical atom pair distribution function at that location, Structural model parameter set.

[0013] In another aspect, the present invention provides a database construction apparatus for an atom pair distribution function, comprising a memory, a processor, and a program stored in the memory, wherein the processor executes the program to implement the method described above.

[0014] In another aspect, the present invention provides a storage medium having a program stored thereon, which, when executed, implements the method described above.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention automatically extracts experimental parameters and performs unit standardization and interval consistency checks during the data receiving stage. It then verifies the reproducibility of the uploaded PDF data through a reprocessing flow of I(Q)→S(Q)→F(Q)→G(r). The system uses a cosine similarity formula to compare the consistency between the calculated data and the uploaded data, generating a quality score in the 0-1 interval to evaluate the consistency and completeness of the PDF data. Furthermore, this invention calculates a weighted residual factor Rw based on the difference between the experimental atom-pair distribution function and the atom-pair distribution function calculated by the structural model, thus providing a quantifiable evaluation index for structural fit and avoiding fitting bias caused by differences in data magnitude. The weighted residual factor reflects the degree of deviation between the structural model and the experimental PDF data and can serve as a reference for parameter adjustment during structural refinement. The calculation of the weighted residual factor is based on the real-space atom-pair distribution function, which can be used to locate the source of difference between the structural model and the experimental data, facilitating users to identify deviation locations and improve model parameters during structural refinement, thereby improving refinement efficiency. Attached Figure Description

[0016] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0017] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0018] This invention proposes a database construction method oriented towards the distribution function of atomic pairs, the method comprising the following steps: Receive the measured PDF data, parse the experimental parameters from the PDF data file, and perform consistency checks on the parameters; store the parameters that pass the check and the corresponding PDF data into the database.

[0019] When the database receives a verification task, it performs reprocessing calculations of I(Q)→S(Q)→F(Q)→G(r) sequentially based on the stored raw scattering data to generate a verification atom pair distribution function. Subsequently, it calculates the cosine similarity based on the difference between the verification atom pair distribution function and the user-uploaded atom pair distribution function, and generates a quality score. For data with a quality score below the threshold, the system performs anomaly marking and difference localization. When receiving an upload request containing atomic pair distribution functions / total scattering data / structure functions / reduced structure functions (.gr / .iq / .sq / .fq), the file is parsed and parameters such as maximum scattering momentum qmax, minimum scattering momentum qmin, maximum real space range rmax, minimum real space range rmin, real space data resolution rstep, experimental wavelength, experimental beamline information, experimental facility information, experimental temperature, and experimental pressure are automatically extracted. If the parameters pass the unit standardization and interval consistency check, the original file is written to object storage and the metadata is written to the database to generate a dataset identifier. If the parameter fails the validation, the validation result is returned and the data entry is paused until the missing item is filled.

[0020] The parameter parsing includes physical validity determination of the reciprocal space Q interval, real space r interval, and step size, as well as dataset fingerprint generation.

[0021] The data entry includes records, sources, timestamps, and traceable fingerprints, which are used for subsequent auditing and reproduction.

[0022] The above method corresponds to a database construction system for PDF data, including: Module for file reception and parameter extraction; Module for unit standardization and consistency verification; And modules for reprocessing and reproducibility verification; The system includes modules for logical retrieval and indexing, visualization, downloading and sharing, and API authentication and invocation. The logical retrieval and indexing module is configured to parse element / chemical formula logical expressions and perform retrieval.

[0023] The reprocessing and reproducibility verification module is configured to perform I(Q)→S(Q). 验证 →F(Q) 验证 →G(r) 验证 The algorithm calculates and generates a quality score based on the cosine similarity between the reprocessed result and the distribution function of the atomic pairs uploaded by the user. The score is between 0 and 1. The specific algorithm is as follows.

[0024] When the database receives a verification task, the reprocessing module reconstructs S(Q), F(Q), and G(r) from the original scattering data; If G(r) 验证 If the cosine similarity with the uploaded G(r) is not lower than a preset threshold, it is marked as passed and a verification report is generated; if it is lower than the threshold, the difference location is output and it is marked as needing review.

[0025] When a retrieval request is received, the logical expression of the element or chemical formula is parsed. If the logical expression contains operators such as "!", "-", ",", and "*", then logical operations are performed according to a preset priority to filter candidate datasets; If the search request includes conditions such as material category, data type, reciprocal space Q and real space r range, temperature, pressure, or experimental facilities, then a combined screening is performed based on the conditions. The system returns search results in table or card format, and visualizes G(r), I(Q), S(Q), and F(Q) curves within the cards. The visualization supports zooming and coordinate reading operations, thereby improving the readability and interactivity of the results.

[0026] A method for downloading and sharing PDF databases with an API interface, comprising: When a user triggers a download or share, the original / reprocessed file is packaged with machine-readable metadata. If academic sharing is selected, it will connect to a third-party archiving platform (such as Zenodo or CSTR) to generate a citationable DOI; and return a download link or DOI citation information.

[0027] The API provides entry points for uploading, retrieving, refining, and querying reports, and employs JSON Web Token and API Key authentication and rate limiting strategies.

[0028] In this embodiment, when the user provides a structure file to be fitted (including but not limited to files that can represent atomic coordinates and unit cell parameters, such as CIF, XYZ, STRU, RMC6F, extxyz, struct, com, mol, mol2, in, pw, xsf, pdb, cell, POSCAR, CONTCAR, cfg, sys, traj, xtd, xsd, xsf, sdf, res, and geom) and selects the corresponding target PDF data, the system obtains the PDF data according to the user's selection and calculates the theoretical atom pair distribution function based on the unit cell parameters and atomic coordinates obtained from parsing the structure file.

[0029] Subsequently, PDF fitting is performed on the theoretical atom pair distribution function and the target PDF data, and the weighted residual factor is calculated. .

[0030] The weighted residual factor is calculated as follows: After the fitting is complete, the system will return the fitted curve, Rw value, and relevant fitting parameters to the user to support local structure analysis and model evaluation.

[0031] The main steps of this invention are illustrated in the following flowchart. Figure 1 As shown.

[0032] Areas for improvement: 1) The platform implements a unified process for data uploading, verification, retrieval, analysis and sharing, enabling data management and subsequent processing functions to be completed within the same system.

[0033] 2) The system automatically extracts parameters when data is entered into the database and can verify the uploaded data through reprocessing algorithms. 3) The data retrieval module supports logical expressions and multi-condition filtering, and the results are presented in a graphical visualization format. 4) Connect to high-throughput data processing requirements via API Experimental parameters are automatically extracted and standardized during the data receiving stage. At the same time, a reprocessing algorithm is called to compare the results with the uploaded results, thereby verifying the reproducibility of data quality, improving the comparability of data records under different experimental conditions, and reducing the deviation of results caused by differences in processing methods.

[0034] By introducing logical expression retrieval syntax and multi-condition combination filtering into the database, users can flexibly perform complex searches based on elements, chemical formulas, experimental conditions, etc., thereby significantly improving data retrieval efficiency. This approach can achieve more efficient retrieval operations in large-scale datasets.

[0035] In the results display and access phase, the system provides interactive curve visualization, which can display G(r), I(Q), S(Q) and F(Q) curves; the system also provides RESTful API and Python SDK interfaces so that users can call database functions in external programs.

[0036] The platform is directly integrated with high-throughput processing to reduce manual operations during data import and processing.

[0037] In the data sharing phase, the platform can package raw data, data verification results, and metadata files, and can connect to third-party archiving platforms to generate DOIs; enabling standardized citation and academic sharing of datasets to support long-term data preservation and cross-platform use.

[0038] This invention enables PDF experimental data and its related parameters to be automatically extracted, standardized, and stored, and quality verified based on reprocessing calculations, thus making PDF data reproducible and comparable. Combined with traceable management and retrieval methods that support logical expressions, it can improve data management and query efficiency, facilitate the identification of the source of data differences, and meet the demand for high-quality PDF data in applications such as local structure analysis and artificial intelligence.

[0039] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A database construction method oriented towards the distribution function of atomic pairs, characterized in that, The method includes the following steps: The system receives the measured atom pair distribution function data (.gr) and related raw scattering data (.iq, .sq, and .fq), and parses the experimental parameters from the atom pair distribution function data. If any parameters are missing, they are extracted from the related raw scattering data. The parameters are then checked for consistency. The parameters that pass the check, the corresponding atom pair distribution function data (.gr), the related raw scattering data (.iq, .sq, and .fq), and the standardized metadata are all stored in the database. When the database receives a verification task, it first performs a verification atom pair distribution function calculation based on the metadata to obtain the verification atom pair distribution function. Then, it generates a quality score based on the cosine similarity between the verification atom pair distribution function and the user-uploaded atom pair distribution function. If the quality score is lower than the threshold, the metadata is marked and a difference location report is generated.

2. The database construction method oriented towards the distribution function of atomic pairs according to claim 1, characterized in that, In the relevant raw scattering data .iq, .sq, and .fq, .iq represents the total scattering intensity data, .sq represents the structure function, and .fq represents the reduced structure function.

3. The database construction method oriented towards the distribution function of atomic pairs according to claim 2, characterized in that, The specific steps for calculating and verifying the atom pair distribution function are as follows: Acquire the total scattering data and calculate the verification structure function S(Q), the verification reduced structure function F(Q), and the verification atom pair distribution function G(r) in sequence.

4. The database construction method oriented towards the atomic pair distribution function according to claim 3, characterized in that, The quality rating is: in, Indicates quality score, This represents the i-th data point used to verify the distribution function of the atom pair. This represents the i-th data point of the atomic pair distribution function uploaded by the user, where n represents the total number of data points.

5. A database construction method oriented towards atom pair distribution functions according to claim 1, characterized in that, The method further includes: When the database receives a retrieval request, it parses the elements or chemical formulas in the retrieval request logical expression. If the logical expression contains operators, it performs logical operations according to preset priorities to filter candidate datasets and obtain retrieval results.

6. A database construction method oriented towards the distribution function of atomic pairs according to claim 5, characterized in that, The search request may include one or more of the following: material category, data type, reciprocal space and real space range, temperature, pressure, or experimental facility.

7. A database construction method oriented towards the distribution function of atomic pairs according to claim 6, characterized in that, Search results are in table or card format.

8. A database construction method oriented towards the distribution function of atomic pairs according to claim 1, characterized in that, The method further includes: when a structure file containing crystal structure information is received and a structure fitting operation is triggered, selecting target atom pair distribution function data from the database; then calculating the theoretical atom pair distribution function based on the atomic coordinates and unit cell information obtained from the structure file parsing; and fitting the theoretical atom pair distribution function with the target atom pair distribution function data and outputting a weighted residual factor, and analyzing and evaluating the local structure model based on the weighted residual factor. The weighted residual factor is: Where, r i This represents the i-th specific distance from the reference atom. Represents the distribution function of atomic pairs. This represents the distance calculated based on the structural model parameter set P. The theoretical atom pair distribution function at that location, Structural model parameter set.

9. A database construction apparatus oriented towards an atom pair distribution function, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-8.

10. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the method as described in any one of claims 1-8.

Images