A multi-source heterogeneous material data stream-oriented integrity identification method and device

Abstract

The application provides a kind of integrity identification method and device for multi-source heterogeneous material data flow, relating to artificial intelligence technical field.The method comprises: carrying out format determination and preprocessing to JSON format multi-source heterogeneous material data to obtain standardized material data;constructing large language model prompt word based on material data resource directory and JSON specification;analyzing and importance evaluating each parameter of material data through large language model, outputting weight value and priority label of each parameter and labeling material data, checking each item of four-dimension evaluation system for labeled material data, and analyzing importance and integrity of different types of material data through large language model to obtain integrity score and grading determination of material data.The application can realize full-process automation, standardization and intelligent processing, improve material data quality from the source, and guarantee material data security, reliable, efficient circulation and sharing.

Description

Technical Field

[0001] This invention relates to the fields of materials informatics, big data processing, and artificial intelligence, and in particular to a method and apparatus for integrity identification of multi-source heterogeneous material data flow. Background Technology

[0002] With the deep integration and rapid development of materials science, big data technology, and artificial intelligence technology, data resources in the global materials field are experiencing explosive growth. Materials data has become a core strategic resource for promoting new materials research and development, process optimization, performance improvement, and industrial upgrading. In practical application scenarios, materials data generally exhibits typical characteristics of being multi-source, batch-based, distributed, and non-standardized: data sources cover various channels such as scientific research papers, patent documents, experimental testing, production records, simulation calculations, equipment acquisition, instrument output, and manual entry; data structures show significant differences in field names, parameter units, expression methods, and classification systems; and data scale often reaches batch, large-scale, or even massive levels.

[0003] In the processes of material data collection, integration, publication, sharing, and circulation, data integrity is the core foundation for ensuring data usability, credibility, and reliability, directly determining whether the data can support applications such as scientific research, manufacturing, quality inspection, and simulation analysis. However, the integrity management of multi-source heterogeneous batch material data currently faces a series of serious technical bottlenecks, specifically as follows: First, there are shortcomings in JSON (JavaScript Object Notation) data parsing and model adaptation. Traditional data processing methods have not built a dedicated large language model hint word system for JSON structures. The model cannot accurately understand the semantics of materials science and the hierarchical structure of JSON, making it difficult to efficiently extract core information such as material composition, performance, process, and conditions. It also cannot achieve efficient interaction between batch JSON data and large language models, resulting in low processing efficiency, high error rate, and insufficient standardization, making it difficult to form a unified, standardized data foundation that can be used for integrity identification.

[0004] Second, there is a lack of intelligent assessment of the importance of material data parameters. Existing technologies do not utilize large language models to conduct professional analysis and integrity assessment of the parameters in JSON material data. The weights rely on manual setting, which is highly subjective, poorly adaptable to professional fields, and fails to meet the actual needs of materials research and engineering, resulting in distorted and unreliable integrity assessment results.

[0005] Third, integrity verification is primarily manual, lacking automated, intelligent, and quantitative methods. Currently, material data integrity testing generally relies on manual verification and inspection, which is not only slow and unable to meet the demands of batch data processing, but also prone to omissions, errors, and misjudgments. Furthermore, the lack of a scientific, systematic, and quantitative integrity evaluation index system makes it impossible to objectively score data integrity. The system also fails to automatically identify, prompt, and warn of missing items, resulting in a large amount of incomplete, non-standard, and unusable data directly entering the sharing and circulation process, seriously affecting the accuracy and reliability of research conclusions, production decisions, and testing results.

[0006] In summary, existing technologies cannot provide an integrated integrity solution for efficient analysis, automatic annotation, intelligent identification, and quantitative control of batch multi-source heterogeneous material data. This severely restricts the high-quality sharing and efficient application of material data resources, becoming a significant obstacle to the digital and intelligent development of materials science and engineering. Therefore, developing a method and system for identifying the integrity of material data circulation that is adapted to the characteristics of multi-source heterogeneity, supports batch processing, possesses automation and intelligence capabilities, and has a quantitative evaluation system has significant theoretical and engineering application value.

[0007] Currently, patent CN116861042A focuses on the integration and format verification of material database metadata, relying on rules to verify the integrity of metadata; patent CN120832621A targets joint quality detection of multimodal data, employing multimodal feature comparison. Both rely primarily on rule or feature matching, lacking professional prompt word adaptation for JSON, and parameter weights depend on manual setting. They lack professional semantic understanding of materials and a percentage-based quantitative scoring system, resulting in poor adaptability to multi-source heterogeneous batch data, low parsing efficiency, and the inability to automatically grade and warn, making it difficult to meet the intelligent identification needs for the integrity of material data circulation. Summary of the Invention

[0008] To address the technical problems in existing technologies, such as poor parsing and adaptation of multi-source heterogeneous data, high cost of manual annotation, reliance on manual integrity verification, lack of quantitative evaluation system, and absence of a fully integrated management and control mechanism, this invention provides a method and apparatus for integrity identification of multi-source heterogeneous material data circulation. The technical solution is as follows: On the one hand, an integrity identification method for multi-source heterogeneous material data flow is provided. This method is implemented by an integrity identification device for multi-source heterogeneous material data flow, and includes: S1. Obtain multi-source heterogeneous material data in JSON format, and perform format determination and preprocessing on the material data to obtain standardized material data.

[0009] S2. Construct large language model prompt words based on material data resource catalog and JSON specification.

[0010] S3. Input standardized material data and prompt words into the large language model. Analyze and evaluate the importance of various parameters of the material data through the large language model, and output the weight values ​​and priority labels of each parameter.

[0011] S4. Label the material data according to the weight value and priority label, and verify the labeled material data item by item according to the four-dimensional evaluation system. Analyze the importance and completeness of different types of material data through a large language model to obtain the integrity score and classification of the material data. The four-dimensional evaluation system includes the completeness of material data fields, the completeness of material data records, the logical completeness of material data, and the completeness of material data sources.

[0012] Optionally, S1 includes: S11. Obtain multi-source heterogeneous material data in JSON format, and determine the format of the material data to obtain the material data to be parsed.

[0013] S12. Perform hierarchical parsing, field validation, name unification, deduplication, unit unification, and anomaly detection preprocessing on the material data to be parsed to obtain standardized material data.

[0014] Optionally, S1 also includes: Data type identification is performed on standardized material data to determine the data type of each parameter in the material data; among which, data types include string type, numeric type, range type, candidate type, image type, file type, array type, table type, container type, and generator type.

[0015] Optionally, the constructive language model cue words in S2 include: Reusable and scalable prompt word templates are constructed for multiple material categories. The prompt word template includes field parsing instructions, parameter extraction rules, professional constraints, output format requirements, and validation rules.

[0016] Optionally, in S4, material data is labeled based on weight values ​​and priority labels. The labeled material data is then validated item by item using a four-dimensional evaluation system. Furthermore, a large language model is used to analyze the importance and completeness of different types of material data, resulting in a completeness score and classification of the material data, including: S41. Statistically analyze and normalize the weight values ​​of each parameter to obtain the data completeness of each parameter in all material data, and label the material data according to the data completeness of each parameter.

[0017] S42. The labeled material data is evaluated item by item using a four-dimensional evaluation system. The importance and completeness of different types of material data are analyzed through a large language model to obtain the completeness score and classification of the material data.

[0018] Optionally, the integrity scoring and grading of material data in S4 includes: The total score for the completeness of the material data (out of 100), the scores for each dimension in the four-dimensional evaluation system, details of missing items, reasons for failure, and supplementary suggestions.

[0019] Optionally, material data field integrity includes no missing required fields and reasonable filling of optional fields.

[0020] Material data record integrity includes complete individual data entries and no duplication or omission in batch data.

[0021] The logical integrity of materials data includes conformity with basic understanding of materials science.

[0022] The source of the material data is complete, including clear data sources.

[0023] On the other hand, an integrity identification device for data flow of multi-source heterogeneous materials is provided. This device is applied to the integrity identification method for data flow of multi-source heterogeneous materials, and the device includes: The data acquisition and preprocessing module is used to acquire multi-source heterogeneous material data in JSON format, and to perform format determination and preprocessing on the material data to obtain standardized material data.

[0024] The prompt word construction module is used to construct prompt words for large language models based on the material data resource catalog and JSON specifications.

[0025] The importance assessment module is used to input standardized material data and prompt words into a large language model. The large language model analyzes and assesses the importance of various parameters of the material data and outputs the weight values ​​and priority labels of each parameter.

[0026] The output module is used to label material data according to weight values ​​and priority labels, perform item-by-item verification of the labeled material data based on a four-dimensional evaluation system, and analyze the importance and completeness of different types of material data through a large language model to obtain the integrity score and classification of the material data. The four-dimensional evaluation system includes the completeness of material data fields, the completeness of material data records, the logical completeness of material data, and the completeness of material data sources.

[0027] Optionally, the data acquisition and preprocessing module is further used for: Obtain multi-source heterogeneous material data in JSON format, and determine the format of the material data to obtain the material data to be parsed.

[0028] Standardized material data is obtained by performing hierarchical parsing, field validation, name unification, deduplication, unit unification, and anomaly detection preprocessing on the material data to be parsed.

[0029] Optionally, the data acquisition and preprocessing module is further used for: Data type identification is performed on standardized material data to determine the data type of each parameter in the material data; among which, data types include string type, numeric type, range type, candidate type, image type, file type, array type, table type, container type, and generator type.

[0030] Optionally, the prompt word construction module is further used for: Reusable and scalable prompt word templates are constructed for multiple material categories. The prompt word template includes field parsing instructions, parameter extraction rules, professional constraints, output format requirements, and validation rules.

[0031] Optionally, the output module is further used for: The weight values ​​of each parameter are statistically analyzed and normalized to obtain the data completeness of each parameter in all material data. The material data are then labeled according to the data completeness of each parameter.

[0032] The labeled material data is evaluated item by item using a four-dimensional evaluation system. The importance and completeness of different types of material data are analyzed through a large language model to obtain the completeness score and classification of the material data.

[0033] Optionally, the output module is further used for: The total score for the completeness of the material data (out of 100), the scores for each dimension in the four-dimensional evaluation system, details of missing items, reasons for failure, and supplementary suggestions.

[0034] Optionally, material data field integrity includes no missing required fields and reasonable filling of optional fields.

[0035] Material data record integrity includes complete individual data entries and no duplication or omission in batch data.

[0036] The logical integrity of materials data includes conformity with basic understanding of materials science.

[0037] The source of the material data is complete, including clear data sources.

[0038] On the other hand, an integrity identification device for multi-source heterogeneous material data circulation is provided. The integrity identification device for multi-source heterogeneous material data circulation includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-described integrity identification methods for multi-source heterogeneous material data circulation is implemented.

[0039] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for integrity identification of multi-source heterogeneous material data flow.

[0040] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: It exhibits strong adaptive parsing capabilities for multi-source heterogeneous material data. It constructs a large-scale language model prompting word system for materials science, enabling efficient interaction between the model and data. It possesses batch processing capabilities, high parsing efficiency, and a high degree of standardization, solving the problems of poor adaptability, low efficiency, and inconsistency in traditional parsing methods.

[0041] The intelligent assessment of parameter importance is objective and scientific. A large language model automatically completes material parameter analysis and weight evaluation, replacing manual weight setting, significantly reducing labor costs, standardizing assessment criteria, and improving the professionalism and accuracy of comprehensive assessments.

[0042] Integrity identification is automated, intelligent, and quantifiable. A scientific and systematic quantitative evaluation system is established, employing a large language model to achieve deep semantic understanding and accurate missing information identification. It provides a percentage-based score and detailed prompts, enabling automatic judgment, hierarchical control, and automatic early warning, replacing manual verification and significantly improving verification efficiency and reliability.

[0043] Integrated automated management and control throughout the entire process. It forms an integrated process of JSON parsing, prompt word construction, parameter evaluation, integrity identification, and integrity assessment, with seamless data connection, supporting distributed batch processing, and adapting to multiple scenarios such as material data platforms, scientific research, and enterprises, to achieve quality control of material data throughout its entire lifecycle. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of an integrity identification method for data flow of multi-source heterogeneous materials provided by an embodiment of the present invention; Figure 2 This is a flowchart of the material data integrity assessment based on a large language model provided in an embodiment of the present invention; Figure 3 This is a block diagram of an integrity identification device for multi-source heterogeneous material data flow provided in an embodiment of the present invention; Figure 4This is a schematic diagram of the structure of an integrity identification device for multi-source heterogeneous material data flow provided in an embodiment of the present invention. Detailed Implementation

[0046] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0047] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0048] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0049] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0050] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0051] This invention provides an integrity identification method for data flow of multi-source heterogeneous materials. This method can be implemented by an integrity identification device for multi-source heterogeneous material data flow, which can be a terminal or a server. Figure 1 The flowchart shown is for an integrity identification method for multi-source heterogeneous material data flow. The processing flow of this method may include the following steps: S1. Obtain multi-source heterogeneous material data in JSON format, and perform format determination and preprocessing on the material data to obtain standardized material data.

[0052] In one feasible implementation, multi-source heterogeneous material data is imported in batches through a data interaction unit to complete JSON structure parsing, format compliance verification, and standardization processing.

[0053] Specifically, step S1 above is the standardization of JSON material data, which includes the following steps S11-S13: S11. Obtain multi-source heterogeneous material data in JSON format, and determine the format of the material data to obtain the material data to be parsed.

[0054] In one feasible implementation, a data parsing unit is used to receive JSON format material data, perform preliminary format determination on the raw data, and obtain data to be parsed that meets the processing conditions before material data standardization and integrity control can be carried out.

[0055] S12. Perform hierarchical parsing, field validation, name unification, deduplication, unit unification, and anomaly detection preprocessing on the material data to be parsed to obtain standardized material data.

[0056] In one feasible implementation, the present invention adopts a standardized processing approach for multi-source heterogeneous JSON data throughout the entire process. It performs hierarchical parsing, field validation, name unification, format compliance checks, deduplication, unit unification, and anomaly detection preprocessing on batch JSON material data, and finally outputs standardized JSON structured data, which is suitable for batch, distributed, and multi-source material data processing scenarios, and provides a unified data foundation for subsequent prompt word invocation and model analysis.

[0057] S13. Identify the data types of standardized material data to obtain the data types of various parameters of the material data; among them, the data types include string type, numerical type, range type, candidate type, image type, file type, array type, table type, container type and generator type.

[0058] In one feasible implementation, based on the material data resource catalog system and JSON data structure specifications, the characteristics of multi-source JSON material data are comprehensively sorted out, the material data is parsed and classified, and the system identifies the data type of each material parameter, including ten data types such as string, numerical, range, candidate, image, file, array, table, container, and generator.

[0059] Specifically, 1. String type: Text fields such as material name, testing personnel, and manufacturer, with algorithm checks for null values, illegal special characters, text length, and professional terminology.

[0060] 2. Numerical type: Single indicators such as density, hardness, tensile strength, etc. The algorithm verifies the positive and negative values, significant figures, unit matching, and extreme value anomalies.

[0061] 3. Range type: temperature range, thickness range, aging time range, algorithm verifies the relationship between upper and lower limits, no logical conflicts in the range, and uniform boundary units.

[0062] 4. Candidate type: Fixed enumeration fields such as material category, heat treatment method, and testing standard number, and the algorithm verifies whether the value conforms to common material knowledge.

[0063] 5. Image type: metallographic images, sample appearance images, scanning electron microscope images, algorithm to verify file link validity and image format.

[0064] 6. File type: Original test report, simulation source file, experimental record document, and completeness of algorithm verification file attachments.

[0065] 7. Array type: Multiple sets of data of the same type, suitable for scenarios such as sequence data and curve data. The algorithm verifies the array length, the uniform format of sub-elements, no empty arrays, and no missing sub-items.

[0066] 8. Table type: Supports nesting multiple control types to form a multi-row, multi-column table structure, suitable for describing structured data scenarios such as component tables and performance tables. The algorithm verifies the integrity of the table header, the filling of row and column data, and the consistency of row and column units.

[0067] 9. Container type: Used to merge multiple fields, suitable for complex data structure organization, and can be used for scenarios such as grouping fields and simplifying form structure to improve the clarity of data organization. The algorithm recursively parses the nested level and verifies the completeness of all inner sub-fields.

[0068] 10. Generator type: Different options are preset in the template design to correspond to different data forms. It is suitable for multi-form data entry scenarios. The algorithm verifies the consistency of data generation rules, the continuity of the sequence, and the absence of gaps or missing data.

[0069] S2. Construct large language model prompt words based on material data resource catalog and JSON specification.

[0070] Optionally, step S2 above is the large language model parameter analysis and importance assessment, which includes constructing large language model cue words, including: Reusable and scalable prompt word templates are constructed for multiple material categories. The prompt word template includes field parsing instructions, parameter extraction rules, professional constraints, output format requirements, and validation rules.

[0071] In one feasible implementation, based on the semantic and JSON structure characteristics of materials science, a special prompt word for a large language model is constructed according to the materials data resource catalog and JSON specification. This forms a standardized, reusable, and extensible prompt word template that includes field parsing instructions, parameter extraction rules, professional constraints, output format requirements, and verification rules. This enables accurate interaction between the large language model and materials data, providing a unified instruction basis for subsequent parameter analysis and integrity identification.

[0072] In this invention, the construction of prompt words for the large language model is the core foundation for achieving accurate model interaction. Its design fully conforms to the professional characteristics of the materials field and the JSON data structure, covering multiple categories such as metallic materials and polymer materials. Each category is configured with exclusive prompt word rules, field specifications, unit systems, and terminology sets, which can accurately guide the large language model to understand the professional information of JSON material data. At the same time, it supports dynamic expansion, and new materials and new data sources can be added as prompt words at any time to ensure the long-term applicability of the system.

[0073] This invention constructs a materials science prompting word engineering system. Based on the materials data resource catalog system and JSON data structure specifications, it creates standardized prompting word templates covering material type, composition, performance, process, and testing conditions. It adapts to materials science semantics and JSON hierarchical structure, supports dynamic expansion, rule updates, field expansion, and structure adaptation, and adapts to the interaction needs of multiple material types and multiple data sources, achieving accurate interaction between large language models and multi-source heterogeneous material data.

[0074] S3. Input standardized material data and prompt words into the large language model. Analyze and evaluate the importance of various parameters of the material data through the large language model, and output the weight values ​​and priority labels of each parameter.

[0075] Specifically, step S3 above includes the following steps S31-S33: S31. Input standardized JSON data and prompt words into the large language model so that it can analyze the parameters.

[0076] S32, the large language model, based on materials expertise, conducts semantic understanding and correlation analysis on parameters such as composition, performance, process, testing, and environment.

[0077] S33. The model automatically outputs parameter weight values, distinguishing between core parameters, important parameters, and general parameters, and generating weight bases that can be used for integrity calculations.

[0078] In one feasible implementation, standardized JSON material data and constructed prompts are input into a large language model. The large language model then performs intelligent analysis, professional interpretation, and importance classification assessment of various parameters in the material data. It outputs parameter importance weights and priority classifications according to core parameters, important parameters, and general parameters, forming an objective, professional, and quantifiable basis for parameter importance. This basis is directly used for subsequent weight calculation and missing item determination in the completeness quantitative evaluation system, thereby improving the professionalism and accuracy of the completeness assessment.

[0079] In the parameter evaluation stage, the intelligent analysis and importance classification of the large language model is an important innovation of this invention. It replaces manual weight setting, and the model automatically completes the parameter importance determination based on material expertise. Core parameters ensure the weight of basic information, important parameters enhance professional relevance, and general parameters are reasonably allocated weight. The combination of the three forms a scientific, objective, and professional weight system, providing a reliable basis for the quantitative evaluation of completeness.

[0080] This invention uses a large language model to intelligently assess the importance of parameters. Standardized JSON material data is input into the large language model, which automatically performs intelligent analysis and importance classification of material parameters, dividing them into three levels: core parameters, important parameters, and general parameters. It outputs objective weight values ​​and priority labels, replacing manual weight setting and providing a professional basis for completeness quantitative assessment.

[0081] S4. Label the material data according to the weight value and priority label, and verify the labeled material data item by item according to the four-dimensional evaluation system. Analyze the importance and completeness of different types of material data through a large language model to obtain the integrity score and classification of the material data. The four-dimensional evaluation system includes the completeness of material data fields, the completeness of material data records, the logical completeness of material data, and the completeness of material data sources.

[0082] One feasible implementation involves designing a unified data evaluation process, establishing a four-dimensional integrity quantitative evaluation system, and using a large language model to construct an integrity identification model to achieve data integrity judgment, quantitative scoring, hierarchical determination, and early warning.

[0083] Specifically, such as Figure 2 As shown, the evaluated data will undergo intelligent integrity identification and release control, and hierarchical processing will be performed based on the scoring results to achieve closed-loop management of material data, including the following steps S41-S42: S41. Statistically normalize the evaluation results of parameter weight values, and calculate the data completeness of each parameter in all data as the basis for subsequent evaluation of the large language model.

[0084] S42. Each of the four dimensions of the evaluation system is verified. For the completeness of material data fields, required fields must be complete; otherwise, the data is directly classified as incomplete. Optional fields must be reasonably set and conform to basic materials science knowledge. For the completeness of material data records, each record must be able to independently support secondary reuse, basic information must be complete, and related parameters must not be largely missing. For the logical completeness of material data, a large language model is used to determine whether the data content conforms to basic materials science understanding and whether the data range is reasonable. For the completeness of material data sources, the source of the data is analyzed to determine whether its origin is clear. A large language model is used to perform semantic understanding and logical judgment. Based on materials science knowledge, the large language model analyzes the importance and completeness of different types of data, and finally outputs a completeness score.

[0085] In one feasible implementation, the annotated data undergoes format verification, annotation verification, and integrity identification. A four-dimensional evaluation system is invoked to complete integrity scoring and grading. Based on the integrity identification results, complete data is stored in the database, and incomplete data is automatically alerted and returned for correction, thus achieving closed-loop management of the entire material data process.

[0086] Specifically, for real-world scenarios involving distributed nodes, multi-source data acquisition, and batch uploads, a four-dimensional quantitative evaluation system for data integrity is established: material data field integrity (no missing required fields, and reasonable filling of optional fields), material data record integrity (complete individual data entries, and no duplication / omission in batch data), material data logical integrity (conforming to basic materials science understanding), and material data source integrity (clear data origin). A materials science-specific integrity identification model is constructed using a large language model. Dedicated prompt word templates are designed, and standardized data and annotation information are input. The system performs deep semantic understanding, field-by-field verification, parameter logic comparison, missing information identification, and standard compliance judgment. It outputs a total integrity score (out of 100), scores for each dimension, details of missing items, reasons for failure, and supplementary suggestions. A three-level judgment is applied based on the score: ≥80 points indicates complete data, 60–80 points indicates basically complete data, and <60 points indicates incomplete data. Automatic warnings are issued for data below the threshold, achieving automatic integrity identification, control, and early warning.

[0087] Furthermore, to achieve consistency in integrity determination, this embodiment of the invention introduces an integrity scoring mechanism based on rules and semantic understanding, defined as follows:

[0088] In the above formula, Indicates the first Item verification dimension weight, Indicates the first Each item is scored, with the total score calculated on a 100-point scale. This score measures the completeness of the current data, thereby achieving a quality classification of the material data.

[0089] Furthermore, complete data is automatically entered into the database and sharing platform; basically complete data is manually reviewed before being entered into the database or returned for modification; and incomplete data is returned to the submitter for supplementation, correction, or resubmission.

[0090] In the integrity identification stage, the four-dimensional quantitative evaluation system comprehensively covers the core elements of material data integrity. The scoring mechanism is objective, clear, and quantifiable, the hierarchical judgment mechanism enables refined control, the early warning mechanism intercepts incomplete data at the source, and the introduction of a large language model enables the system to have deep semantic understanding capabilities, which can identify problems that traditional methods cannot identify, such as logical errors, missing associations, and non-standard expressions, significantly improving the identification accuracy and reliability.

[0091] This invention establishes a four-dimensional integrity quantitative evaluation system, which includes the integrity of material data fields, the integrity of material data records, the logical integrity of material data, and the integrity of material data sources. This system covers all elements of material data integrity and enables quantitative scoring and accurate verification.

[0092] By using large language model-driven intelligent integrity identification, a material-specific integrity identification model is constructed. This model enables a percentage-based quantitative integrity scoring system, a three-level judgment (complete / basically complete / incomplete), detailed prompts for missing items, and automatic early warnings. This creates a closed-loop process of automatic identification, hierarchical control, and early warning return.

[0093] This invention addresses the problems of inconsistent formats, inconsistencies in labeling, difficulty in determining completeness, and low batch processing efficiency in existing multi-source heterogeneous material data. It provides a method for standardized parsing, automated labeling, and intelligent integrity control of material data throughout the entire process. The core process includes three steps: prompt word construction, parameter importance assessment, and quantitative identification of integrity across four dimensions. The entire process is based on a large language model and standardized JSON data. Specifically, it involves batch importing multi-source heterogeneous material data in JSON format, generating standardized JSON data through standardized parsing; constructing prompt words using a materials-specific large language model; performing parameter analysis and importance assessment using the large language model; scoring, determining, and issuing warnings using an integrity identification model; and finally, storing and publishing complete data while returning incomplete data for supplementation. This method, combining prompt word engineering with intelligent analysis using a large language model, can efficiently process large-scale material data in JSON format, ensuring data quality through a comprehensive standardization and integrity control mechanism. Compared to existing manual processing and single-step verification methods, this invention is particularly suitable for batch data processing on large-scale material data platforms, providing standardized, efficient, and reliable support for material data management, sharing, and analysis.

[0094] In this embodiment of the invention, a method combining automated analysis and intelligent discrimination is employed to efficiently process multi-source, heterogeneous, and large-scale material data. Data quality is improved through standardized, automated, and intelligent management throughout the entire process. Compared to existing manual processing and decentralized management methods, this invention is particularly suitable for large-scale material data public service platforms, supporting the processing of large volumes of material data and providing standardized, reliable, and efficient data support for materials science research and engineering applications.

[0095] Figure 3 This is a block diagram illustrating an integrity identification device for data flow of multi-source heterogeneous materials, according to an exemplary embodiment. The device is used in an integrity identification method for data flow of multi-source heterogeneous materials. (Refer to...) Figure 3 The device includes a data acquisition and preprocessing module 310, a prompt word construction module 320, an importance assessment module 330, and an output module 340. Among them: The data acquisition and preprocessing module 310 is used to acquire multi-source heterogeneous material data in JSON format, and to perform format determination and preprocessing on the material data to obtain standardized material data.

[0096] The prompt word construction module 320 is used to construct prompt words for a large language model based on the material data resource catalog and JSON specification.

[0097] The importance assessment module 330 is used to input standardized material data and prompt words into the large language model. The large language model analyzes and assesses the importance of various parameters of the material data and outputs the weight values ​​and priority labels of each parameter.

[0098] The output module 340 is used to annotate material data according to weight values ​​and priority labels, perform item-by-item verification of the annotated material data using a four-dimensional evaluation system, and analyze the importance and completeness of different types of material data through a large language model to obtain the completeness score and classification of the material data. The four-dimensional evaluation system includes the completeness of material data fields, the completeness of material data records, the logical completeness of material data, and the completeness of material data sources.

[0099] In this embodiment of the invention, a method combining automated analysis and intelligent discrimination is employed to efficiently process multi-source, heterogeneous, and large-scale material data. Data quality is improved through standardized, automated, and intelligent management throughout the entire process. Compared to existing manual processing and decentralized management methods, this invention is particularly suitable for large-scale material data public service platforms, supporting the processing of large volumes of material data and providing standardized, reliable, and efficient data support for materials science research and engineering applications.

[0100] Figure 4This is a structural schematic diagram of an integrity identification device for multi-source heterogeneous material data flow provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the integrity identification device for multi-source heterogeneous material data flow can include the above-mentioned Figure 3 The illustrated integrity identification device for multi-source heterogeneous material data flow. Optionally, the integrity identification device 410 for multi-source heterogeneous material data flow may include a first processor 2001.

[0101] Optionally, the integrity identification device 410 for multi-source heterogeneous material data flow may also include a memory 2002 and a transceiver 2003.

[0102] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0103] The following is combined with Figure 4 The following is a detailed description of each component of the integrity identification device 410 for multi-source heterogeneous material data flow: The first processor 2001 is the control center of the integrity identification device 410 for multi-source heterogeneous material data flow. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0104] Optionally, the first processor 2001 can perform various functions of the integrity identification device 410 for multi-source heterogeneous material data flow by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0105] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0106] In a specific implementation, as one example, the integrity identification device 410 for multi-source heterogeneous material data flow may also include multiple processors, for example... Figure 4The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0107] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0108] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be accessed through the interface circuit of the integrity identification device 410 for multi-source heterogeneous material data flow (…). Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0109] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0110] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0111] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the integrity identification device 410 for data flow of multi-source heterogeneous materials. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0112] It should be noted that, Figure 4 The structure of the integrity identification device 410 for multi-source heterogeneous material data flow shown in the diagram does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0113] Furthermore, the technical effects of the integrity identification device 410 for multi-source heterogeneous material data flow can be referred to the technical effects of the integrity identification method for multi-source heterogeneous material data flow described in the above method embodiments, and will not be repeated here.

[0114] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0115] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0116] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0117] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0118] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0119] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0120] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0122] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0125] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for integrity identification of data flow in multi-source heterogeneous materials, characterized in that, The method includes: S1. Obtain multi-source heterogeneous material data in JSON format, and perform format determination and preprocessing on the material data to obtain standardized material data; S2. Constructing prompt words for a large language model based on the material data resource catalog and JSON specification; S3. Input standardized material data and prompt words into the large language model. Analyze and evaluate the importance of various parameters of the material data through the large language model, and output the weight values ​​and priority labels of each parameter. S4. Label the material data according to the weight value and priority label, and verify the labeled material data item by item according to the four-dimensional evaluation system. Analyze the importance and completeness of different types of material data through a large language model to obtain the integrity score and classification of the material data. The four-dimensional evaluation system includes the completeness of material data fields, the completeness of material data records, the logical completeness of material data, and the completeness of material data sources.

2. The integrity identification method for multi-source heterogeneous material data circulation according to claim 1, characterized in that, S1 includes: S11. Obtain multi-source heterogeneous material data in JSON format, and determine the format of the material data to obtain the material data to be parsed. S12. Perform hierarchical parsing, field validation, name unification, deduplication, unit unification, and anomaly detection preprocessing on the material data to be parsed to obtain standardized material data.

3. The integrity identification method for multi-source heterogeneous material data circulation according to claim 1, characterized in that, S1 further includes: Data type identification is performed on standardized material data to determine the data type of each parameter in the material data; among which, data types include string type, numeric type, range type, candidate type, image type, file type, array type, table type, container type, and generator type.

4. The integrity identification method for multi-source heterogeneous material data circulation according to claim 1, characterized in that, The construction of large language model prompts in S2 includes: Reusable and scalable prompt word templates are constructed for multiple material categories. The prompt word template includes field parsing instructions, parameter extraction rules, professional constraints, output format requirements, and validation rules.

5. The integrity identification method for multi-source heterogeneous material data circulation according to claim 1, characterized in that, In step S4, material data is labeled based on weight values ​​and priority labels. The labeled material data is then validated item by item using a four-dimensional evaluation system. Furthermore, a large language model is used to analyze the importance and completeness of different types of material data, resulting in a completeness score and classification of the material data, including: S41. Statistically analyze and normalize the weight values ​​of each parameter to obtain the data completeness of each parameter in all material data, and label the material data according to the data completeness of each parameter; S42. The labeled material data is evaluated item by item using a four-dimensional evaluation system. The importance and completeness of different types of material data are analyzed through a large language model to obtain the completeness score and classification of the material data.

6. The integrity identification method for multi-source heterogeneous material data circulation according to claim 1, characterized in that, The integrity scoring and grading of the material data in S4 includes: The total score for the completeness of the material data (out of 100), the scores for each dimension in the four-dimensional evaluation system, details of missing items, reasons for failure, and supplementary suggestions.

7. The integrity identification method for multi-source heterogeneous material data circulation according to claim 1, characterized in that, The integrity of the material data fields includes no missing required fields and reasonable filling of optional fields; Material data record integrity, including complete individual data entries and no duplication or omission in batch data; The logical integrity of materials data includes conformity with basic understanding of materials science; The source of the material data is complete, including clear data origin.

8. An integrity identification device for data flow of multi-source heterogeneous materials, wherein the integrity identification device for data flow of multi-source heterogeneous materials is used to implement the integrity identification method for data flow of multi-source heterogeneous materials as described in any one of claims 1-7, characterized in that, The device includes: The data acquisition and preprocessing module is used to acquire multi-source heterogeneous material data in JSON format, and to perform format determination and preprocessing on the material data to obtain standardized material data. The prompt word construction module is used to construct prompt words for a large language model based on the material data resource catalog and JSON specifications; The importance assessment module is used to input standardized material data and prompt words into a large language model. The large language model analyzes and assesses the importance of various parameters of the material data and outputs the weight values ​​and priority labels of each parameter. The output module is used to label material data according to weight values ​​and priority labels, perform item-by-item verification of the labeled material data based on a four-dimensional evaluation system, and analyze the importance and completeness of different types of material data through a large language model to obtain the integrity score and classification of the material data. The four-dimensional evaluation system includes the completeness of material data fields, the completeness of material data records, the logical completeness of material data, and the completeness of material data sources.

9. An integrity identification device for data flow of multi-source heterogeneous materials, characterized in that, The integrity identification device for multi-source heterogeneous material data circulation includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images