Generative AI-assisted material data template design method and system

Abstract

The invention discloses a generative AI-assisted material data template design method and system, and relates to the technical field of material data management and generative artificial intelligence. The method comprises the following steps: receiving uploaded multi-source heterogeneous original material data, and carrying out decoding and decompression verification to generate a structured file storage path; analyzing the original material data, preliminarily deducing the data type, and generating a formatted intermediate file; generating an initial type mapping table by using a dynamic type discrimination mechanism based on a data scale; inputting the initial type mapping table and a target database template example into a generative AI engine, generating an initial JSON template, and performing man-machine collaborative optimization; and performing multi-dimensional verification on the optimized JSON template, storing the template which passes the verification, and outputting a standardized semantic data entity which can be correctly analyzed by the target database. According to the method, automatic analysis, semantic mapping and interactive optimization of the multi-source heterogeneous material data can be realized.

Description

Technical Field

[0001] This invention relates to the fields of materials data management and generative artificial intelligence, and in particular to a generative AI-assisted method and system for designing materials data templates. Background Technology

[0002] With the development of high-throughput experimental and computational simulation technologies, the amount of material data, including experimental records, process descriptions, and performance test results, is increasing exponentially. To support subsequent data-driven materials research and engineering applications, this data needs to be converted using standardized templates and then imported into specialized databases.

[0003] Currently, template generation technology is mainly used in the standardized management of multi-source heterogeneous material data and the aggregation of data to target databases. However, material data faces the complexity of multi-source heterogeneity (diverse formats, complex semantics, and structural differences), and the format specifications of different target databases differ significantly, leading to difficulties in cross-database integration.

[0004] Existing technologies mainly include traditional manual techniques and partially automated technologies, which have the following shortcomings: Traditional manual techniques rely heavily on manual operation, which is cumbersome, prone to errors, time-consuming for each build, and has a high rate of field omissions or type labeling errors, resulting in low efficiency.

[0005] In existing automation technologies, rule-based tools cannot handle semantic ambiguity and complex structured data, and have poor flexibility; early machine learning models have limited accuracy and lack deep semantic understanding capabilities; natural language processing (NLP) technology is currently mostly limited to keyword matching, and has failed to achieve accurate parsing of complex semantics in the field of materials, and cannot achieve dynamic adaptation of multi-source data and multi-format databases.

[0006] The aforementioned defects result in low template reuse rates and maintenance costs that increase exponentially with the growth of the database size. Summary of the Invention

[0007] To address the aforementioned problems, the present invention aims to provide a generative AI-assisted material data template design method and system, solving the issues of low efficiency, insufficient handling of semantic ambiguity, high data type misjudgment rate, and poor cross-database compatibility in traditional manual template design. This invention integrates generative artificial intelligence technology with knowledge in the field of materials science to achieve automated parsing, semantic mapping, and interactive optimization of multi-source heterogeneous data, providing an efficient solution for the aggregation and standardized storage of materials science data.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, a generative AI-assisted material data template design method is provided, the method comprising the following steps: S1. Receive the uploaded multi-source heterogeneous raw material data, decode and decompress it for verification, and generate a structured file storage path; S2. Based on the structured file storage path, the original material data is parsed, the data type is preliminarily inferred, and a formatted intermediate file is generated; S3. Perform dynamic type determination on the formatted intermediate file based on the data size, and generate an initial type mapping table; S4. Input the initial type mapping table and the target database template example into the generative AI engine to generate the initial JSON template and perform human-machine collaborative optimization. S5. Based on the standardized storage interface of the dynamic container database, perform multi-dimensional verification on the JSON template optimized by human-machine collaboration, store the verified template, and output standardized semantic data entities that can be correctly parsed by the target database.

[0009] Optionally, step S1 specifically includes: Design a general file upload interface to receive multi-source heterogeneous raw material data uploaded by users, including Excel files and ZIP compressed packages; For ZIP archives, a preset function is used to process the filenames, decoding them into GBK encoding format to ensure that the files are correctly decompressed and a structured file storage path is generated.

[0010] Optionally, step S2 specifically includes: Dynamically decompress and batch extract files based on the structured file storage path; Load the Excel workbook using a table data processing library and extract the first row header field as key descriptive information; The domain ontology library is used to semantically annotate the fields, and the samples are taken column by column starting from the second row. The data type of each column is initially inferred through preset rules, and a CSV formatted intermediate file containing "table name, field name, and initial type" is output.

[0011] Optionally, the table data processing library includes the openpyxl library, and the preset rules include regular expression matching and file extension detection.

[0012] Optionally, in step S3, the dynamic type determination based on data scale specifically includes: Intelligent sampling strategy: Use single-row extraction when the number of rows is small; use random sampling when the number of rows is large. Type consistency determination: If the data types of the randomly sampled samples are consistent, the data type is determined; if they are inconsistent, they are marked as "unknown" and manual review is triggered. Multi-rule judgment: Numeric data types are judged using regular expression matching; image / file data types are judged based on URL features; candidate data types are judged based on data validation attributes. After dynamic type determination based on data size, an initial type mapping table is generated.

[0013] Optionally, step S4 specifically includes: Input the initial type mapping table and the target database template example into the generative AI engine; Leveraging the few-shot learning capabilities of large language models, semantic mapping, hierarchical structure definition, and unit standardization are automatically completed to generate an initial JSON template; Set up a human-computer collaborative verification interface that allows users to optimize the initial JSON template, specifically by modifying one or more of the following: field names, data types, units, and hierarchical relationships.

[0014] Optionally, step S5 specifically includes: Design a standardized storage interface for a dynamic container database, and perform multi-dimensional validation on the JSON template optimized by human-computer collaboration, including syntax validation and format validation; The verified templates are uploaded to the database system for storage. The templates support vector retrieval and keyword filtering. Output standardized semantic data entities that can be correctly parsed by the target database for subsequent querying and analysis.

[0015] On the other hand, a generative AI-assisted material data template design system is provided to implement the method described in any of the above embodiments, the system comprising: The data preprocessing module is used to receive uploaded multi-source heterogeneous raw material data, decode and decompress it for verification, and generate a structured file storage path. The preliminary inference module is used to parse the raw material data based on the structured file storage path, make preliminary inferences about the data type, and generate a formatted intermediate file; The dynamic discrimination module is used to perform dynamic type discrimination on the formatted intermediate file based on the data size and generate an initial type mapping table; The generative AI module is used to input the initial type mapping table and the target database template example into the generative AI engine, generate the initial JSON template, and perform human-machine collaborative optimization. The verification storage and standardized output module is used to perform multi-dimensional verification of JSON templates optimized by human-machine collaboration based on the standardized storage interface of the dynamic container database. It stores the templates that pass the verification and outputs standardized semantic data entities that can be correctly parsed by the target database.

[0016] On the other hand, an electronic device is provided, the electronic device comprising: processor; The memory stores computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the generative AI-assisted material data template design method described above.

[0017] On the other hand, a computer-readable storage medium is provided, wherein program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the steps of the generative AI-assisted material data template design method described above.

[0018] The beneficial effects of the technical solution provided by this invention include at least the following: (1) Improve the accuracy of data type identification: By introducing a generative AI-assisted dynamic type discrimination mechanism and intelligent sampling strategy, the accuracy of identification of complex data types has been significantly improved.

[0019] (2) Shorten template generation time and improve semantic mapping accuracy: The generative AI model that integrates few-sample learning can quickly learn the norm with only a few examples, greatly shortening the generation time and effectively solving the standardization problem of heterogeneous synonym fields.

[0020] (3) Ensure template format standardization and database compatibility: Combined with the standardized storage interface of dynamic container database, ensure that the generated templates conform to the specifications, improve the template reuse rate, reduce maintenance costs, and strongly support high-throughput material calculation and performance prediction. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a generative AI-assisted material data template design method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a generative AI-assisted material data template design system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0024] This invention provides a generative AI-assisted material data template design method, which can be implemented by an electronic device, such as a terminal or a server. Figure 1 As shown, the processing flow of this method may include the following steps:

[0025] S1. Receive the uploaded multi-source heterogeneous raw material data, decode and decompress it for verification, and generate a structured file storage path.

[0026] As an optional embodiment of the present invention, step S1 specifically includes: Design a general file upload interface to receive multi-source heterogeneous raw material data uploaded by users, including Excel files, ZIP compressed packages, etc. For ZIP archives, a preset function is used to process filenames (e.g., Chinese or English filenames), decoding the filenames into GBK encoding format to ensure that the files are correctly decompressed and a structured file storage path is generated.

[0027] S2. Based on the structured file storage path, the original material data is parsed, the data type is preliminarily inferred, and a formatted intermediate file is generated.

[0028] As an optional embodiment of the present invention, step S2 specifically includes: Dynamically decompress and batch extract files based on the structured file storage path; Load the Excel workbook using a table data processing library (such as openpyxl) and extract the first row header field as key descriptive information; The domain ontology library is used to semantically annotate the fields, and the samples are taken column by column starting from the second row. The data type of each column is initially inferred through preset rules (such as regular expression matching and extension detection), and a CSV formatted intermediate file containing "table name, field name, and initial type" is output.

[0029] S3. Perform dynamic type determination on the formatted intermediate file based on the data size, and generate an initial type mapping table.

[0030] As an optional embodiment of the present invention, the dynamic type determination based on data scale in step S3 specifically includes: Intelligent sampling strategy: When the number of rows is small, single-row extraction is used (e.g., extracting the second row); when the number of rows is large, random sampling is used. Type consistency determination: If the data types of the randomly sampled samples are consistent, the data type is determined; if they are inconsistent, they are marked as "unknown" and manual review is triggered. Multi-rule judgment: Numeric data types are judged using regular expression matching, image / file data types are judged based on URL features, and candidate data types are judged based on data validation attributes; For detailed procedures, please refer to Figure 1 As shown, the data size is first determined based on an intelligent sampling strategy. Small samples undergo rapid single-row determination, while large samples are randomly sampled. Next, type consistency is determined to arrive at a definitive data type. During this determination process, multiple rules are used to classify the data type. After this dynamic type determination based on data size, an initial type mapping table is generated.

[0031] S4. Input the initial type mapping table and the target database template example into the generative AI engine to generate the initial JSON template and perform human-machine collaborative optimization.

[0032] As an optional embodiment of the present invention, step S4 specifically includes: Input the initial type mapping table and the target database template example into the generative AI engine; Leveraging the few-shot learning capabilities of large language models, semantic mapping, hierarchical structure definition, and unit standardization are automatically completed to generate an initial JSON template; A human-computer collaborative verification interface is set up, allowing users to optimize the initial JSON template, specifically by modifying field names, data types, units, hierarchical relationships, etc.

[0033] S5. Based on the standardized storage interface of the dynamic container database, perform multi-dimensional verification on the JSON template optimized by human-machine collaboration, store the verified template, and output standardized semantic data entities that can be correctly parsed by the target database.

[0034] As an optional embodiment of the present invention, step S5 specifically includes: Design a standardized storage interface for a dynamic container database, and perform multi-dimensional validation on the JSON template optimized by human-computer collaboration, including syntax validation and format validation (recursive traversal of the JSON tree). The verified templates are uploaded to the database system for storage. The templates support vector retrieval and keyword filtering. Output standardized semantic data entities that can be correctly parsed by the target database for subsequent querying and analysis.

[0035] In this embodiment of the invention, intelligent sampling strategies (single-row extraction, random sampling) and multi-rule judgment logic (regular expression, URL feature, data verification attribute) are designed for different data volumes. By using a generative AI engine, combined with an initial type mapping table and a small number of target database template examples, the semantic mapping and structure generation process is automatically completed, and a complete closed loop of AI generation, manual correction, standardized interface verification and database storage is constructed to ensure the semantic consistency and format standardization of the template.

[0036] Accordingly, embodiments of the present invention also provide a generative AI-assisted material data template design system, such as... Figure 2 As shown, the system includes: The data preprocessing module 201 is used to receive uploaded multi-source heterogeneous raw material data, decode and decompress it for verification, and generate a structured file storage path. The preliminary inference module 202 is used to parse the raw material data based on the structured file storage path, make preliminary inferences about the data type, and generate a formatted intermediate file; The dynamic discrimination module 203 is used to perform dynamic type discrimination on the formatted intermediate file based on the data size and generate an initial type mapping table; Generative AI module 204 is used to input the initial type mapping table and the target database template example into the generative AI engine, generate the initial JSON template, and perform human-machine collaborative optimization. The verification storage and standardized output module 205 is used to perform multi-dimensional verification of the JSON template optimized by human-machine collaboration based on the standardized storage interface of the dynamic container database, store the verified template, and output standardized semantic data entities that can be correctly parsed by the target database.

[0037] For ease of explanation, Figure 2 Only the main components of the system are shown. The system of this embodiment can be used to perform... Figure 1 The technical solutions of the method embodiments shown are similar in principle and in effect, and will not be described again here.

[0038] In an exemplary embodiment, the present invention also provides an electronic device, the electronic device comprising: processor; The memory stores computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the generative AI-assisted material data template design method described above.

[0039] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 3As shown, the electronic device 300 may include a first processor 3001 and a memory 3002. Optionally, the electronic device 300 may also include a transceiver 3003. The first processor 3001, memory 3002, and transceiver 3003 may be connected via a communication bus. The memory 3002 stores computer-readable instructions, which, when executed by the first processor 3001, implement the steps of the generative AI-assisted material data template design method described above.

[0040] In a specific implementation, as one example, the first processor 3001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0041] In a specific implementation, as one example, the electronic device 300 may also include multiple processors, for example... Figure 3 The first processor 3001 and the second processor 3004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0042] The memory 3002 is used to store the software program that executes the present invention, and is controlled by the first processor 3001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0043] The transceiver 3003 is used to communicate with network devices or with terminal devices.

[0044] Optionally, the transceiver 3003 may include a receiver and a transmitter. The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0045] Optionally, the transceiver 3003 can be integrated with the first processor 3001 or exist independently and be coupled to the first processor 3001 through the interface circuit of the electronic device 300. This embodiment of the invention does not specifically limit this.

[0046] It should be noted that, Figure 3 The structure of the electronic device 300 shown is not intended to limit the electronic device. Actual electronic devices may include more or fewer components than shown, or combine certain components, or have different component arrangements. Furthermore, the technical effects of the electronic device 300 can be understood by referring to the technical effects of the above-described method embodiments, and will not be repeated here.

[0047] In an exemplary embodiment, the present invention also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the steps of the generative AI-assisted material data template design method described above. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0048] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0049] The use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0050] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0051] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0052] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0053] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0054] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0055] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A generative AI-assisted material data template design method, characterized in that, Includes the following steps: S1. Receive the uploaded multi-source heterogeneous raw material data, decode and decompress it for verification, and generate a structured file storage path; S2. Based on the structured file storage path, the original material data is parsed, the data type is preliminarily inferred, and a formatted intermediate file is generated; S3. Perform dynamic type determination on the formatted intermediate file based on the data size, and generate an initial type mapping table; S4. Input the initial type mapping table and the target database template example into the generative AI engine to generate the initial JSON template and perform human-machine collaborative optimization. S5. Based on the standardized storage interface of the dynamic container database, perform multi-dimensional verification on the JSON template optimized by human-machine collaboration, store the verified template, and output standardized semantic data entities that can be correctly parsed by the target database.

2. The generative AI-assisted material data template design method according to claim 1, characterized in that, Step S1 specifically includes: Design a general file upload interface to receive multi-source heterogeneous raw material data uploaded by users, including Excel files and ZIP compressed packages; For ZIP archives, a preset function is used to process the filenames, decoding them into GBK encoding format to ensure that the files are correctly decompressed and a structured file storage path is generated.

3. The generative AI-assisted material data template design method according to claim 1, characterized in that, Step S2 specifically includes: Dynamically decompress and batch extract files based on the structured file storage path; Load the Excel workbook using a table data processing library and extract the first row header field as key descriptive information; The domain ontology library is used to semantically annotate the fields, and the data is sampled column by column starting from the second row. The data type of each column is initially inferred through preset rules, and a CSV formatted intermediate file containing "table name, field name, and initial type" is output.

4. The generative AI-assisted material data template design method according to claim 3, characterized in that, The table data processing library includes the openpyxl library, and the preset rules include regular expression matching and file extension detection.

5. The generative AI-assisted material data template design method according to claim 1, characterized in that, In step S3, the dynamic type determination based on data scale specifically includes: Intelligent sampling strategy: Use single-row extraction when the number of rows is small; use random sampling when the number of rows is large. Type consistency determination: If the data types of the randomly sampled samples are consistent, the data type is determined; if they are inconsistent, they are marked as "unknown" and manual review is triggered. Multi-rule judgment: Numeric data types are judged using regular expression matching; image / file data types are judged based on URL features; candidate data types are judged based on data validation attributes. After dynamic type determination based on data size, an initial type mapping table is generated.

6. The generative AI-assisted material data template design method according to claim 1, characterized in that, Step S4 specifically includes: Input the initial type mapping table and the target database template example into the generative AI engine; Leveraging the few-shot learning capabilities of large language models, semantic mapping, hierarchical structure definition, and unit standardization are automatically completed to generate an initial JSON template; Set up a human-computer collaborative verification interface that allows users to optimize the initial JSON template, specifically by modifying one or more of the following: field names, data types, units, and hierarchical relationships.

7. The generative AI-assisted material data template design method according to claim 1, characterized in that, Step S5 specifically includes: Design a standardized storage interface for a dynamic container database, and perform multi-dimensional validation on the JSON template optimized by human-computer collaboration, including syntax validation and format validation; The verified templates are uploaded to the database system for storage. The templates support vector retrieval and keyword filtering. Output standardized semantic data entities that can be correctly parsed by the target database for subsequent querying and analysis.

8. A generative AI-assisted material data template design system, the system being used to implement the method as described in any one of claims 1 to 7, characterized in that, The system includes: The data preprocessing module is used to receive uploaded multi-source heterogeneous raw material data, decode and decompress it for verification, and generate a structured file storage path. The preliminary inference module is used to parse the raw material data based on the structured file storage path, make preliminary inferences about the data type, and generate a formatted intermediate file; The dynamic discrimination module is used to perform dynamic type discrimination on the formatted intermediate file based on the data size and generate an initial type mapping table; The generative AI module is used to input the initial type mapping table and the target database template example into the generative AI engine, generate the initial JSON template, and perform human-machine collaborative optimization. The verification storage and standardized output module is used to perform multi-dimensional verification of JSON templates optimized by human-machine collaboration based on the standardized storage interface of the dynamic container database. It stores the templates that pass the verification and outputs standardized semantic data entities that can be correctly parsed by the target database.

9. An electronic device, characterized in that, The electronic device includes: processor; A memory storing computer-readable instructions that, when loaded and executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images