Tree structure encapsulation method for first principle calculation data

Abstract

This invention relates to data encapsulation, specifically to a tree-structured encapsulation method for first-principles calculation data. The method includes: creating a file and root group: creating an HDF5 file and establishing the necessary top-level groups in the root directory; encapsulating the physical system: under the `system` group, creating subgroups and datasets according to a predetermined structure to store crystal structure and atomic information; recording the calculation process: for each calculation, creating a subgroup named after the calculation type under the `calculation` group to store the calculation data; and linking workflows: under the `workflow` group, explicitly recording the logical relationships and data flow between different calculation steps by creating subgroups and defining attributes. The technical solution provided by this invention effectively overcomes the shortcomings of existing technologies, such as data fragmentation and loss of correlation, lack of self-description, and low data access efficiency.

Description

Technical Field

[0001] This invention relates to data encapsulation, and more specifically to a tree-structure encapsulation method for first-principles calculation data. Background Technology

[0002] First-principles calculations (such as those using software like VASP, Quantum ESPRESSO, and ABINIT) are the core research methods in computational materials science. These calculations typically generate complex, multi-dimensional, and interconnected data, including crystal structure information, electronic potential fields, k-point settings, intermediate energies / forces during the iteration process, and final physical quantities such as total energy, band structure, and density of states.

[0003] Currently, the storage and management of first-principles calculation data mainly face the following problems:

[0004] 1) Data fragmentation and loss of correlation: A complete study usually includes multiple computational steps (such as structural relaxation, static self-consistency, band structure calculation, etc.). Existing technologies (such as text or binary output files customized by various software, including VASP's OUTCAR and XML files) store the computational results corresponding to different computational steps in independent files. The semantic correlation and derivation relationship between these files (such as band structure calculation being based on the relaxed structure, etc.) are not explicitly and structurally recorded, making data traceability extremely difficult.

[0005] 2) Lack of self-description: Existing file formats often lack machine-readable, standardized metadata. Parsing these files heavily relies on volatile parsing scripts written by developers and human understanding of software documentation. The meaning of physical quantities and physical units are not clearly described, making data sharing and reuse prone to errors and hindering the reproducibility of scientific results.

[0006] 3) Inefficient data access: When extracting specific physical quantities (such as the total energy of all systems from 1,000 calculation results) from unstructured text files or complex custom binary files, it is necessary to write complex text parsing or binary reading programs, which is cumbersome and inefficient.

[0007] In the existing technology, although there are general scientific data formats (such as HDF5, NetCDF, etc.), they only serve as containers for data organization and do not define a mandatory, complete tree structure and metadata specification for the field of first-principles calculation that can record the meaning of physical quantities and the calculation workflow. Summary of the Invention

[0008] (a) Technical problems to be solved

[0009] To address the aforementioned shortcomings of existing technologies, this invention provides a tree-structured encapsulation method for first-principles calculation data, which can effectively overcome the deficiencies of existing technologies, such as data fragmentation and loss of correlation, lack of self-description, and low data access efficiency.

[0010] (II) Technical Solution

[0011] To achieve the above objectives, the present invention provides the following technical solution:

[0012] A tree-structured encapsulation method for first-principles calculation data includes the following steps:

[0013] S1. Create file and root group: Create an HDF5 file and create the necessary top-level group in the root directory;

[0014] S2. Encapsulating the physical system: Under the system group, subgroups and datasets are created according to a predetermined structure to store crystal structure and atomic information;

[0015] S3. Record the calculation process: For each calculation, create a subgroup named after the calculation type under the calculation group to store the calculation data;

[0016] S4. Link Workflow: Under the workflow group, create subgroups and define attributes to explicitly record the logical relationships and data flow between different calculation steps.

[0017] Preferably, in S1, the file and root group are created: an HDF5 file is created, and the necessary top-level groups are created in the root directory, including:

[0018] Create an HDF5 file and create the necessary top-level groups, including the system, calculation, and workflow groups, in the root directory.

[0019] Preferably, in S2, the physical system is encapsulated: under the system group, subgroups and datasets are created according to a predetermined structure to store crystal structure and atomic information, including:

[0020] Under the system group, subgroups and datasets are created according to a predefined structure to store crystal structure and atomic information, while the @units attribute is attached to all datasets with physical units.

[0021] Preferably, the calculation process is recorded in S3: For each calculation, a subgroup named after the calculation type is created under the calculation group to store the calculation data, including:

[0022] In the input subgroup, all input parameters are stored, and the @source_system attribute is used to point to the crystal structure on which this calculation is based;

[0023] The output subgroup stores all computation results and attaches metadata attributes describing the meaning and physical units of each dataset, including the @units attribute.

[0024] Preferably, in S4, the workflow is linked as follows: Under the workflow group, subgroups are created and attributes are defined to explicitly record the logical relationships and data flow between different computation steps, including:

[0025] Under the workflow group, you can explicitly record the logical relationships and data flow between different calculation steps by creating subgroups and defining attributes such as @input_structure, @output_structure, and @calculation.

[0026] (III) Beneficial Effects

[0027] Compared with the prior art, the tree structure encapsulation method for first-principles calculation data provided by the present invention has the following beneficial effects:

[0028] 1) Solve the problems of data traceability and technology reproducibility.

[0029] This invention employs a mandatory tree structure and explicitly records the logical relationships and data flow between different calculation steps through the @source_system attribute and the link attributes in the workflow group. This ensures that any researcher or tool that opens this file can clearly trace which relaxed structure the band structure calculation result was derived from, and which initial model the relaxed structure originated from. This greatly satisfies the reproducibility requirements of scientific research.

[0030] 2) Achieve efficient and semantic data retrieval

[0031] This invention combines self-describing metadata attributes (such as @units, @type, etc.) and standardized tree paths, enabling users or programs to quickly and accurately extract specific physical quantities from thousands of result files in batches through a unified path (such as / / calculation[@type='scf'] / output / final_energy, etc.) and clear semantics. This eliminates the need to write complex parsers for different software, significantly improves the efficiency of data retrieval script writing, and achieves data extraction speeds higher than text parsing methods.

[0032] 3) Promote the interoperability of cross-platform tools

[0033] This invention is based on the open standard HDF5 and defines a unified "data language" in the field, namely tree path and metadata specification. This ensures that any programming language (such as Python, C++, MATLAB, etc.) or scientific visualization tool that supports HDF5 can correctly interpret the file content as long as it follows the "data language". This breaks down the barriers between different computing software and post-processing tools and forms a unified data exchange ecosystem. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0035] Figure 1 This is a schematic diagram of the process of the present invention;

[0036] Figure 2 This is a schematic diagram of the hierarchical relationship of the tree structure inside the HDF5 file in this invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0038] The core of this invention lies in:

[0039] 1) Domain-specific tree structure schema: A set of mandatory HDF5 hierarchical structures was designed specifically for first-principles calculations, which encapsulates the calculation inputs, outputs, intermediate results and their derivation relationships in a tree structure, fundamentally solving the problems of data correlation loss and traceability;

[0040] 2) Self-describing metadata specification: Define the metadata attributes that must be attached to each dataset in the tree structure, clarify their meaning, physical unit and data source, realize the machine readability and unambiguous understanding of the data, and ensure the self-describing nature of the data;

[0041] 3) Computational workflow encapsulation capability: Through tree structure and metadata attributes, a multi-step computational workflow can be fully encapsulated in one or a set of related files, recording the complete scientific context from the initial structure to the final result.

[0042] Through the combined effect of the above-mentioned inventive points, this invention achieves structured storage, complete traceability, efficient retrieval, and cross-platform interoperability of first-principles computation data, directly solving the technical problems existing in the prior art.

[0043] The following describes the specific process of the tree structure encapsulation method for first-principles calculation data provided by this invention, using specific examples (e.g.) Figure 1 (as shown) and technical effects.

[0044] S1. Create Files and Root Groups: Create an HDF5 file and create the necessary top-level groups in the root directory, including:

[0045] Create an HDF5 file and create the necessary top-level groups, including the system, calculation, and workflow groups, in the root directory.

[0046] S2. Encapsulating the Physical System: Under the `system` group, subgroups and datasets are created according to a predetermined structure to store crystal structure and atomic information, including:

[0047] Under the system group, subgroups and datasets are created according to a predefined structure to store crystal structure and atomic information, while the @units attribute is attached to all datasets with physical units.

[0048] S3. Record the calculation process: For each calculation, create a subgroup named after the calculation type under the calculation group to store the calculation data, including:

[0049] In the input subgroup, all input parameters are stored, and the @source_system attribute is used to point to the crystal structure on which this calculation is based;

[0050] The output subgroup stores all computation results and attaches metadata attributes describing the meaning and physical units of each dataset, including the @units attribute.

[0051] S4. Linking Workflows: Under the `workflow` group, create subgroups and define attributes to explicitly record the logical relationships and data flow between different computational steps, including:

[0052] Under the workflow group, you can explicitly record the logical relationships and data flow between different calculation steps by creating subgroups and defining attributes such as @input_structure, @output_structure, and @calculation.

[0053] Figure 2 This is a schematic diagram of the hierarchical relationship of the tree structure within the HDF5 file in this invention. The overall principles and innovations include:

[0054] 1) Single-file full encapsulation: Structure, input, output, workflow, log, image, and user-defined dataset are all encapsulated in a single HDF5 file;

[0055] 2) Verifiable and traceable: Each object has a unique ID, timestamp, generator, software and hardware environment, random seed, and input source reference, enabling a reproducible experimental chain;

[0056] 3) Strongly typed and unit system: Each dataset has units, dimension descriptions, error or uncertainty (optional), and data quality labels;

[0057] 4) Workflow graph and referencing mechanism: The computation graph is constructed through typed references (internal object references, external file references), supporting multiple structures and multiple steps (SCF / Relax / Bands / Phonons / MD / NEB, etc.).

[0058] 5) Cross-code adaptation layer: Retain a common dictionary and allow code-specific extensions (VASP, QE, ABINIT, CASTEP, etc.) to avoid losing information due to the "least common denominator";

[0059] 6) Production-level data engineering: index tables, deduplication and fingerprinting, compression and partitioning, verification and signature, concurrency security (SWMR), version management and schema verification;

[0060] 7) Multimedia and Reports: Images, charts, log text, and HTML / Markdown reports can be embedded and referenced in the generation process;

[0061] 8) MATLAB v7.3 is user-friendly: all objects are HDF5 primitives, which MATLAB can load directly.

[0062] To better illustrate the technical solution of this application, two specific examples are provided below for detailed explanation.

[0063] Example 1: Encapsulating a structural relaxation calculation

[0064] Structural relaxation of silicon crystals was performed using KSSOLV software:

[0065] 1) Execute S1 and S2: Create an HDF5 file, create a dataset under / system / structure, and fill in the initial lattice constant and atomic coordinates of silicon;

[0066] 2) Execute S3: Under the / calculation / relax group:

[0067] Create a dataset under input / parameters, recording values ​​such as encut=20 and kpoints=[4,4,4].

[0068] Set the input / @source_system attribute to " / system / structure";

[0069] After the calculation is completed, the final_energy,forces, and the final_structure (relaxed coordinates) parsed from VASPOUTCAR are recorded in the output subgroup.

[0070] 3) Execute S4: Under the / workflow / relaxation group, set the @input_structure attribute to " / system / structure" and the @output_structure attribute to " / calculation / relax / output / final_structure".

[0071] Example 2: Encapsulating a workflow for relaxation followed by band structure calculation

[0072] Following Example 1, we will perform band structure calculations using the relaxed structure:

[0073] 1) Execute S3: Under the / calculation / bands group:

[0074] Set the input / @source_system attribute to " / calculation / relax / output / final_structure" (directly referencing the relaxation result);

[0075] Simultaneously record the input parameters specific to band structure calculation (such as the path to point k).

[0076] 2) Execute S4: Under the / workflow / band_structure group, set the @input_structure attribute to " / workflow / relaxation / @output_structure" and the @calculation attribute to " / calculation / bands".

[0077] The two examples above demonstrate how the present invention clearly encapsulates a typical, multi-step first-principles calculation workflow.

[0078] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tree-structure encapsulation method for first-principles calculation data, characterized in that: Includes the following steps: S1. Create file and root group: Create an HDF5 file and create the necessary top-level group in the root directory; S2. Encapsulating the physical system: Under the system group, subgroups and datasets are created according to a predetermined structure to store crystal structure and atomic information; S3. Record the calculation process: For each calculation, create a subgroup named after the calculation type under the calculation group to store the calculation data; S4. Link Workflow: Under the workflow group, create subgroups and define attributes to explicitly record the logical relationships and data flow between different calculation steps.

2. The tree structure encapsulation method for first-principles calculation data according to claim 1, characterized in that: Creating files and root groups in S1: Create an HDF5 file and create the necessary top-level groups in the root directory, including: Create an HDF5 file and create the necessary top-level groups, including the system, calculation, and workflow groups, in the root directory.

3. The tree structure encapsulation method for first-principles calculation data according to claim 1, characterized in that: In S2, the physical system is encapsulated: under the `system` group, subgroups and datasets are created according to a predetermined structure to store crystal structure and atomic information, including: Under the system group, subgroups and datasets are created according to a predefined structure to store crystal structure and atomic information, while the @units attribute is attached to all datasets with physical units.

4. The tree structure encapsulation method for first-principles calculation data according to claim 1, characterized in that: S3 records the calculation process: For each calculation, a subgroup named after the calculation type is created under the calculation group to store the calculation data, including: In the input subgroup, all input parameters are stored, and the @source_system attribute is used to point to the crystal structure on which this calculation is based; The output subgroup stores all computation results and attaches metadata attributes describing the meaning and physical units of each dataset, including the @units attribute.

5. The tree structure encapsulation method for first-principles calculation data according to claim 1, characterized in that: Linking workflows in S4: Under the workflow group, subgroups and attributes are created to explicitly record the logical relationships and data flow between different computation steps, including: Under the workflow group, you can explicitly record the logical relationships and data flow between different calculation steps by creating subgroups and defining attributes such as @input_structure, @output_structure, and @calculation.

Images