Large model training data construction method and system for steel industry

Abstract

This invention provides a method and system for constructing large-scale model training data for the steel industry, comprising the following steps: Step 1: Structural analysis of steel industry literature to extract process nodes, process parameters, and process relationships; Step 2: Construction of a process link graph based on the process nodes and process relationships; Step 3: Generation of multiple types of question-and-answer distillation templates using the process link graph, followed by distillation using a large language model to obtain candidate question-and-answer data; Step 4: Sequential verification of parameter validity, link causal consistency, semantic correctness, and expert feedback on the candidate question-and-answer data to form a dataset with multi-layered quality control. This invention uses a complete process link graph of the steel industry as its core, imposes path constraints on the distillation process to ensure that the generated question-and-answer data conforms to the real industrial process; and constructs a multi-layered data quality control system to significantly improve the industrial usability of distillation data.

Description

Technical Field

[0001] This invention relates to the field of large model technology, and more specifically, to a method and system for constructing training data for large models in the steel industry. Background Technology

[0002] Currently, the technologies closest to this invention are mainly focused on data distillation, knowledge distillation, and knowledge graph construction based on large language models. However, existing technologies still have significant shortcomings in industry process link modeling, distillation process constraints, and data quality control systems, which can be summarized into the following two categories.

[0003] The first category is data distillation and knowledge transfer methods based on large language models. For example, patent application CN120104633A, "Database Query Method, System, Device and Medium Based on Large Model Data Distillation," generates training samples from multi-source data using a large language model and employs a teacher-student model distillation mechanism to train a lightweight model to complete the mapping task from natural language to structured queries. The shortcomings of this type of technology are that the data distillation process mainly revolves around general semantics or task objectives, without modeling the process flow, causal links, or parameter dependencies of specific industrial sectors; the distilled data samples lack constraints on the consistency of cross-step process logic, making it difficult to guarantee the professional correctness of question-and-answer samples in complex industrial processes. Therefore, this type of method is more suitable for general database queries or general task distillation and is difficult to directly transfer to industrial fields with strong process and mechanism constraints, such as steel.

[0004] The second category is knowledge graph construction or recommendation methods based on knowledge distillation. For example, patent applications CN117313851B, "A Method and Apparatus for Constructing Knowledge Graphs Based on Large Language Model Knowledge Distillation," and CN119961519A, "A Method, Equipment, and Medium for Recommending Knowledge Graphs Based on Efficient Knowledge Distillation," utilize these technologies to generate structured knowledge through large models and then distill it into knowledge graphs or recommendation models for downstream knowledge reasoning or recommendation tasks. While this type of technology uses knowledge graphs, the objects of knowledge graph construction are mostly general entity relationships or business concepts, failing to model process-linked knowledge structures such as "raw materials—processes—equipment—parameters—quality indicators" in the steel industry. Furthermore, the distillation process primarily focuses on entity-relationship extraction, lacking explicit control over process sequence, causal constraints of processes, and upstream-downstream dependencies.

[0005] Patent application CN120353893A discloses a method for constructing a large model training dataset based on tabular data, which includes: step 1, data preprocessing; step 2, constructing an automated processing method and extraction template based on table header association; step 3, constructing a deformable template model; and step 4, binding question types and fields. However, this patent cannot completely solve the existing technical problems, nor can it meet the needs of this invention. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for constructing large-scale model training data for the steel industry.

[0007] The method for constructing large-scale model training data for the steel industry provided by the present invention includes: Step 1: Collect raw knowledge data related to the steel industry and preprocess it, including structured parsing and identifying chapter titles, paragraph levels, tables, formulas and process descriptions to obtain basic text data of the steel industry. Step 2: Use a large language model to identify process link elements in the basic text data of the steel industry, identify process nodes, and extract the sequence relationship, dependency relationship and causal relationship between the process nodes based on the process description, causal explanation and process constraint relationship in the text data, and then construct a knowledge graph of the process link of the steel industry. Step 3: Driven by the knowledge graph of the steel industry process link, a graph path constraint decoding strategy is used to generate question and answer data. When generating each question and answer, a path or a subgraph in the graph is first specified as the answer skeleton, and the question and answer are generated based on the nodes and relationships in the answer skeleton. Step 4: Perform parameter validity and interval constraint verification, process link consistency and causal logic verification, semantic consistency and model cross-checking on the question and answer data, as well as expert sampling and feedback feedback.

[0008] Preferably, step 1 includes: Using keyword rules and text classification models, management systems, organizational structure descriptions, advertising descriptions, or repetitive paragraphs that are irrelevant to specific processes are removed from the original knowledge data. The text is cleaned up in a unified manner, including headers and footers, table of contents, reference numbers, OCR-recognized incorrect characters, garbled characters, broken text, duplicate paragraphs, and duplicate descriptions across documents. A terminology mapping table for the steel industry was constructed to unify synonyms and aliases in the text, and the parameter units were uniformly converted to a standard unit system.

[0009] Preferably, step 2 includes: The basic text data of the steel industry is segmented into paragraphs, with each paragraph accompanied by a chapter level label. For tables and process description units, they are converted into text descriptions in key-value pair format. The large language model is invoked to identify process nodes, process step nodes, equipment nodes, and process parameter nodes from the text using predefined prompt word templates; Based on the prior knowledge base of the steel industry, the recognition results are corrected according to rules. If a node appears in different categories at the same time, disambiguation is performed based on the context suffix words. If the parameter node is not associated with any process, the association is completed based on the most recent process mentioned in the text. During the relationship extraction process, the upstream-downstream process relationship is constrained to conform to the predefined steelmaking process rules, and the parameter node is only associated with its own process or equipment node.

[0010] Preferably, step 3 includes: Based on the preset question and answer types, the corresponding structural subgraphs are sampled from the steel industry process link knowledge graph; For each question-and-answer type, a fixed question template is designed, and the node names in the subgraph are filled into the question template; The node attributes in the subgraph are used as constraint facts to prompt the large language model to generate an answer based on the constraint facts. After the model outputs, the answer is checked by rules to see if it contains new nodes outside the subgraph. If so, it is rejected and regenerated. For the same subgraph, randomly perturb the wording in the question template to generate multiple variant question-answer pairs.

[0011] Preferably, step 4 includes: The parameter validity and interval constraint verification includes: verifying the validity of process parameters involved in the question and answer based on the preset industry rule base and standard range; for parameters not predefined in the rule base, inputting the parameter name and context into the large language model, requiring it to output a common range and compare it with common knowledge in the steel industry, and marking it as illegal if it is within the preset abnormal range; The process flow consistency and causal logic verification includes: extracting all process nodes from the question and answer, constructing candidate sequences according to the mention order or explicit sequence words in the text; querying the main process chain in the knowledge graph to obtain a standard sequence; determining whether the candidate sequence is a subsequence of the standard sequence, and identifying causal assertions in the question and answer; querying whether there is a causal edge from one node to another or a causal path through an intermediate node in the knowledge graph; if it exists and the causal strength is not less than 0.5, it is judged as reasonable; if there is no causal edge but there is a strong correlation, it is marked as needing semantic verification; if they are completely unrelated, they are judged as unreasonable. The semantic consistency and model cross-calibration include: inputting question-answer pairs into two large language models with different architectures, requiring the models to output a score on whether the question-answer is semantically consistent and conforms to the knowledge of the steel industry; taking the average of the scores of the two models as the final score, and if the score difference is not less than 1.5, triggering a third model for arbitration; using special prompt words to guide the two models to independently detect logical contradictions or professional errors in the question-answer, and taking the intersection question as the confirmation question; for question-answers with scores lower than a preset threshold or with detected problems, using correction prompt words to require the model to output a corrected version; The expert sampling and feedback process includes: manually sampling and reviewing the data that has passed the verification; marking any problems found as parameter range errors, sequence errors, causal errors, non-standard terminology, or semantic ambiguity. Specifically, parameter-related problems are used to update the rule base for parameter validity and interval constraint verification; sequence / causal problems are used to correct the knowledge graph; semantic problems are used to add typical error cases marked by experts to the example of contradictory detection prompts for semantic consistency and model cross-verification; and terminology problems are used to update the terminology mapping table.

[0012] The large-scale model training data construction system for the steel industry provided by the present invention includes: Module M1: Collects raw knowledge data related to the steel industry and preprocesses it, including structured parsing and identification of chapter titles, paragraph levels, tables, formulas and process descriptions to obtain basic text data of the steel industry. Module M2: Uses a large language model to identify process link elements in the basic text data of the steel industry, identifies process nodes, and extracts the sequence relationship, dependency relationship and causal relationship between the process nodes based on the process description, causal explanation and process constraint relationship in the text data, and then constructs a knowledge graph of the steel industry process link. Module M3: Driven by the knowledge graph of the steel industry process link, it uses a graph path constraint decoding strategy to generate question and answer data. When generating each question and answer, it first specifies a path or a subgraph in the graph as the answer skeleton, and generates questions and answers based on the nodes and relationships in the answer skeleton. Module M4: Performs parameter validity and interval constraint verification, process link consistency and causal logic verification, semantic consistency and model cross-checking on the question and answer data, as well as expert sampling and feedback feedback.

[0013] Preferably, the module M1 includes: Using keyword rules and text classification models, management systems, organizational structure descriptions, advertising descriptions, or repetitive paragraphs that are irrelevant to specific processes are removed from the original knowledge data. The text is cleaned up in a unified manner, including headers and footers, table of contents, reference numbers, OCR-recognized incorrect characters, garbled characters, broken text, duplicate paragraphs, and duplicate descriptions across documents. A terminology mapping table for the steel industry was constructed to unify synonyms and aliases in the text, and the parameter units were uniformly converted to a standard unit system.

[0014] Preferably, the module M2 includes: The basic text data of the steel industry is segmented into paragraphs, with each paragraph accompanied by a chapter level label. For tables and process description units, they are converted into text descriptions in key-value pair format. The large language model is invoked to identify process nodes, process step nodes, equipment nodes, and process parameter nodes from the text using predefined prompt word templates; Based on the prior knowledge base of the steel industry, the recognition results are corrected according to rules. If a node appears in different categories at the same time, disambiguation is performed based on the context suffix words. If the parameter node is not associated with any process, the association is completed based on the most recent process mentioned in the text. During the relationship extraction process, the upstream-downstream process relationship is constrained to conform to the predefined steelmaking process rules, and the parameter node is only associated with its own process or equipment node.

[0015] Preferably, the module M3 includes: Based on the preset question and answer types, the corresponding structural subgraphs are sampled from the steel industry process link knowledge graph; For each question-and-answer type, a fixed question template is designed, and the node names in the subgraph are filled into the question template; The node attributes in the subgraph are used as constraint facts to prompt the large language model to generate an answer based on the constraint facts. After the model outputs, the answer is checked by rules to see if it contains new nodes outside the subgraph. If so, it is rejected and regenerated. For the same subgraph, randomly perturb the wording in the question template to generate multiple variant question-answer pairs.

[0016] Preferably, the module M4 includes: The parameter validity and interval constraint verification includes: verifying the validity of process parameters involved in the question and answer based on the preset industry rule base and standard range; for parameters not predefined in the rule base, inputting the parameter name and context into the large language model, requiring it to output a common range and compare it with common knowledge in the steel industry, and marking it as illegal if it is within the preset abnormal range; The process flow consistency and causal logic verification includes: extracting all process nodes from the question and answer, constructing candidate sequences according to the mention order or explicit sequence words in the text; querying the main process chain in the knowledge graph to obtain a standard sequence; determining whether the candidate sequence is a subsequence of the standard sequence, and identifying causal assertions in the question and answer; querying whether there is a causal edge from one node to another or a causal path through an intermediate node in the knowledge graph; if it exists and the causal strength is not less than 0.5, it is judged as reasonable; if there is no causal edge but there is a strong correlation, it is marked as needing semantic verification; if they are completely unrelated, they are judged as unreasonable. The semantic consistency and model cross-calibration include: inputting question-answer pairs into two large language models with different architectures, requiring the models to output a score on whether the question-answer is semantically consistent and conforms to the knowledge of the steel industry; taking the average of the scores of the two models as the final score, and if the score difference is not less than 1.5, triggering a third model for arbitration; using special prompt words to guide the two models to independently detect logical contradictions or professional errors in the question-answer, and taking the intersection question as the confirmation question; for question-answers with scores lower than a preset threshold or with detected problems, using correction prompt words to require the model to output a corrected version; The expert sampling and feedback process includes: manually sampling and reviewing the data that has passed the verification; marking any problems found as parameter range errors, sequence errors, causal errors, non-standard terminology, or semantic ambiguity. Specifically, parameter-related problems are used to update the rule base for parameter validity and interval constraint verification; sequence / causal problems are used to correct the knowledge graph; semantic problems are used to add typical error cases marked by experts to the example of contradictory detection prompts for semantic consistency and model cross-verification; and terminology problems are used to update the terminology mapping table.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) By constructing a knowledge graph of the steel industry process link, the process nodes, process parameter nodes and their sequential, dependent and causal relationships are organized in the form of a graph structure. In the distillation generation stage, a graph path constraint decoding strategy is introduced, which solves the problem that the existing distillation method cannot express the long process and strong dependent process knowledge of the steel industry, and ensures that the generated question and answer data conforms to the upstream and downstream process sequence and causal logic in the real industrial process. (2) By introducing a multi-layer quality verification system that includes parameter legality and interval constraint verification, process link consistency and causal logic verification, semantic consistency and model mutual verification, as well as expert sampling and feedback feedback, the question and answer data generated by distillation is screened and corrected step by step. This solves the problem that existing distillation methods lack an effective verification mechanism for industry parameters and process rules, and avoids situations where process parameters exceed reasonable ranges, logical relationships between parameters are incorrect, and explanations of abnormal working conditions do not conform to actual production experience. (3) By introducing a rule correction mechanism based on the prior knowledge base of the steel industry in the process link element identification process, the disambiguation of process and equipment nodes and the association completion of parameter nodes and processes are constrained, thereby improving the accuracy of process node and relationship extraction. (4) By establishing a problem labeling and closed-loop feedback mechanism in the expert sampling and feedback feedback process, the problems found by manual review are fed back to the rule base, knowledge graph, mutual proofreading prompt word examples and terminology mapping table, thus realizing the continuous optimization of the data construction process. Attached Figure Description

[0018] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of the method for constructing large model training data for the steel industry according to the present invention. Detailed Implementation

[0019] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0020] Example 1 like Figure 1 This invention provides a method for constructing large-scale model training data for the steel industry, comprising the following steps: Step 1: Collection and Structured Analysis of Original Literature Data in the Steel Industry; Collect original knowledge data related to the steel industry, including but not limited to: Textbooks and professional books for the steel industry; Industry standards, process specifications, and operating procedures; Academic journal articles and technical reports; Industry magazines, internal technical documents, etc.

[0021] The collected raw documents undergo preprocessing, specifically including: Use OCR or document parsing technology to restore the text of scanned documents; The text content is cleaned to remove irrelevant descriptions, redundant annotations, and non-technical content. Specific operations include: Using keyword rules and text classification models, management systems, organizational structure descriptions, advertising descriptions, or repetitive paragraphs unrelated to specific technological processes in the literature are removed. The text is cleaned up in a unified manner, including headers and footers, table of contents, reference numbers, OCR-recognized incorrect characters, garbled characters, broken text, duplicate paragraphs, and duplicate descriptions across documents. Construct a terminology mapping table for the steel industry to unify synonyms and aliases in the text. For example, unify the parameter names and unit conversions of "converter tapping temperature / tapping temperature" to a standard unit system. The electronic document is structured and parsed to identify chapter titles, paragraph levels, tables, formulas, and flowcharts, specifically including: A method combining rules and models is used to identify chapter levels and paragraph structures; Annotate relevant paragraphs in the process description, such as text containing semantic signals like "process," "step," "flow," "condition," and "impact." The tables and process description paragraphs are parsed independently to form structured text units that can be processed later.

[0022] Through the above processing, we obtain basic text data of the steel industry with clear structure and explicit semantics, which provides input for subsequent process link extraction.

[0023] Step 2: Identification of Process Link Elements and Construction of Knowledge Graph in the Steel Industry; Based on the structured text obtained in Step 1, a state-of-the-art (SOTA) large language model is used to automatically identify the process link elements in the steel industry. The specific extraction method is implemented as follows: Input preparation: Segment the structured text obtained in step 1 into paragraphs, with each paragraph accompanied by a chapter level label. For tables and process description units, convert them into text descriptions in key-value pair format.

[0024] Prompt word template design: Dedicated prompt word templates are designed for different element types. For example, when identifying process nodes, the template is: "Identify the names of processes in steel production (such as sintering, blast furnace, converter, continuous casting, hot rolling, etc.) from the following text and output them as a JSON array"; when identifying process parameter nodes, the template is: "Extract process parameters from the text, including parameter name, numerical range (if any), and unit. Output format: {parameter name, typical range, unit}".

[0025] Large Model Call and Output Parsing: Call the large model API, setting the temperature parameter to 0 to ensure output stability. Perform regular expression matching and JSON parsing on the model's returned results.

[0026] Post-processing rule correction: Based on the prior knowledge base of the steel industry, the recognition results are corrected according to rules. For example, if a node appears in both the "process" and "equipment" categories, disambiguation is performed based on the context (whether it is accompanied by equipment suffixes such as "furnace", "machine", "line"); if the parameter node is not associated with any process, the association is completed based on the most recent process mention in the text.

[0027] Identify key process nodes in the steel industry from text, including but not limited to: Process nodes (such as sintering, blast furnace, converter, continuous casting, hot rolling, etc.); Process steps (such as feeding, heating, blowing, cooling, etc.); Equipment nodes (blast furnace, converter, continuous casting machine, heating furnace, etc.); Process parameter nodes (such as temperature, composition, pressure, flow rate, etc.).

[0028] Based on the process descriptions, causal explanations, and process constraints in the text, the sequential relationships, dependencies, and causal relationships between process nodes are automatically extracted: Sequential relationship: indicates the order of operations or steps; Dependency: This indicates that the input of a certain process depends on the output of the upstream process. Causal relationship: indicating the impact of parameter changes and process adjustments on the results; Constraints: These represent the range limitations of process parameters or the preconditions for operation.

[0029] During the relationship extraction process, the following constraint mechanisms are introduced: the upstream-downstream process relationship must conform to the predefined steelmaking process rules; reverse order or cross-layer causal relationships that obviously violate industrial common sense are not allowed; parameter nodes can only be associated with their respective process or equipment nodes.

[0030] Construct a knowledge graph of the steel industry's process chain, wherein the graph includes at least: The process layer describes the macro-level production flow; The process step layer describes the internal logic of a single process step. The equipment and parameter layer describes the physical carrier and control variables; The process dependence and causality layer describes the influence path between parameters and results.

[0031] The constructed process link knowledge graph serves as a structured constraint and generation template for subsequent distillation question and answer generation.

[0032] Step 3: Question-and-answer distillation data generation based on the process link graph; Driven by the steel industry process link knowledge graph constructed in Step 2, a state-of-the-art (SOTA) large language model is used to perform graph-based distillation of industry knowledge, generating high-quality question-and-answer data of various types. The difference between the "graph-based distillation" of this invention and the ordinary "knowledge graph + large model distillation" is that: the ordinary method usually uses the knowledge graph as the context input to the large model for retrieval enhancement, and the model generates questions and answers freely; while this invention adopts the "graph path constraint decoding" strategy: when generating each question and answer, a path or a subgraph in the graph is first specified as the "answer skeleton", and then the large model is required to generate questions and answers only based on the nodes and relationships in the skeleton, and facts outside the skeleton are prohibited from being introduced.

[0033] The specific implementation steps include: 1. Path Sampling: Based on the preset question-and-answer type (definition type, process type, causal type, etc.), the corresponding structural subgraph is sampled from the knowledge graph. For example, when generating process type questions and answers, a continuous process path is sampled (such as "blast furnace → converter → continuous casting"); when generating causal type questions and answers, a parameter node and the process nodes it affects are sampled.

[0034] 2. Templated Question Construction: For each question-and-answer type, design a fixed question template and fill in the node names from the subgraph into the template. For example, a process-type template: "Please describe the main processes in the steel production process from node A to node B?" 3. Large Model Answer Generation: The node attributes in the subgraph (such as process definition and parameter range) are used as constraint facts, prompting the large model: "Generate the answer based on the following facts; no additional facts may be fabricated: [Fact List]". After the model outputs, rules are used to check whether the answer contains new nodes outside the subgraph; if so, it is rejected and a new node is generated.

[0035] 4. Multi-round distillation: For the same subgraph, the wording in the question template is randomly perturbed to generate multiple variant question-answer pairs, enhancing data diversity.

[0036] This algorithm ensures that the generated question-and-answer data is fully constrained by the graph, avoiding the illusion problem and factual deviation caused by the free play of the model in ordinary distillation.

[0037] High-quality question-and-answer data, specifically including: Generate definition-based question-and-answer data based on a single process node to describe process concepts, equipment functions, and parameter meanings; Based on the process sequence relationships in the graph, process-related question-and-answer data is generated to describe the process flow and process connection. Based on process dependence and causal relationships, causal question-and-answer data is generated to describe the impact of parameter changes and process adjustments on production results. Generate question-and-answer data for parameter range and anomaly analysis based on parameter nodes; Generate cross-process comprehensive analysis question and answer data based on multi-node paths.

[0038] The generated question-and-answer data is automatically generated by a large language model based on graph constraints, avoiding the distillation of unstructured and unconstrained free text.

[0039] Step 4: Multi-layer quality verification of distillation question-and-answer data; For the distillation question-and-answer data generated in Step 3, a multi-layer quality verification system is constructed to filter and correct the data at each level, specifically including: Step 4.1: Parameter validity and interval constraint verification; For the process parameters mentioned in the Q&A, legality verification is performed based on a pre-set industry rule base and standard range. The rule base is constructed based on: the process parameter ranges clearly defined in national standards (GB / T) and industry standards (YB / T), such as the hot blast temperature range specified in the "Blast Furnace Ironmaking Process Design Specification"; typical operating ranges given in authoritative textbooks and manuals, such as the converter endpoint temperature range recorded in "Iron and Steel Metallurgy"; and common thresholds in the actual production procedures of enterprises (obtained through statistical analysis of anonymized data provided by cooperating steel enterprises).

[0040] The specific process for automatically determining legal / illegal status includes: 1. Parameter name standardization: Using the terminology mapping table in step 1, the parameter names in the question and answer are mapped to standard names. 2. Numerical extraction: Regular expressions are used to match the combination of numbers and units in the question and answer, and the values ​​are uniformly converted to standard units (e.g., temperature is uniformly set to ℃). 3. Threshold comparison: The reasonable range [L, H] corresponding to the parameter is queried in the rule base. If the extracted value is within the range, it is marked as "legal"; if it exceeds the range but is within the extended tolerance range (e.g., ±10%), it is marked as "suspicious" and sent to step 4.3 for further judgment; if it completely exceeds the extended range, it is marked as "illegal" and the question and answer pair is removed.

[0041] For parameters not predefined in the rule base, a large model is used to assist in the judgment: input the parameter name and context into the large model, ask it to output the "common range" and compare it with common knowledge in the steel industry. If there is obvious abnormality, it is marked as illegal.

[0042] Step 4.2: Process chain consistency and causal logic verification; Based on a process chain knowledge graph, consistency checks are performed on multiple process nodes involved in the question-and-answer process to verify whether their sequential and causal relationships conform to the actual steel production flow. Specific verification logic and algorithms include: Sequence Relationship Verification: Extract all process nodes appearing in the question and answer, and construct a candidate sequence S according to the order of mention in the text or explicit sequence words (such as "first...then..."). Query the main process chain in the knowledge graph to obtain the standard sequence T (e.g., sintering → blast furnace → converter → continuous casting → hot rolling). Determine whether S is a subsequence of T (skipping intermediate processes is allowed, but reversal is not). For example, S=[blast furnace, converter, hot rolling] is a valid subsequence; S=[converter, blast furnace] is invalid. If S contains branch process nodes (such as refining), check its dependencies with preceding and following nodes: the refining node must appear after the converter and before continuous casting.

[0043] Causal logic verification: Identify causal assertions in question-and-answer sessions, formatted as "If / because A then / leads to B", where A and B are process events or parameter changes; query the knowledge graph to determine if there is a causal edge (or a causal path through intermediate nodes) from node A to node B. If it exists and the causal strength is ≥0.5, it is considered reasonable; if there is no causal edge but a strong correlation exists (such as parameter-result pairs belonging to the same process), it is marked as "requires semantic verification"; if it is completely unrelated, it is considered unreasonable and removed.

[0044] The process can be summarized as follows: input question and answer → entity extraction → sequential subsequence extraction → main chain subsequence matching → pass / fail; simultaneously perform causal assertion extraction → graph causal path query → strength threshold judgment → pass / suspicious / fail.

[0045] Step 4.3: Semantic consistency and model cross-calibration; We utilize a large language model to assess the semantic consistency and professional rationality of distilled question-and-answer content. Employing two large models with different architectures for cross-validation helps avoid the systematic biases inherent in a single model.

[0046] The judgment strategy includes consistency scoring: Question-answer pairs are input into two models, requiring each model to output whether the question-answer is semantically consistent and conforms to steel industry knowledge (1-5 points). The average of the two model scores is taken as the final score; if the difference is ≥1.5, a third model is triggered for arbitration. Contradiction detection: A dedicated prompt is designed: "Please analyze the following question-answer sentence by sentence, list any possible logical contradictions or professional errors, and output the problem list in JSON format." After independent detection by the two models, the intersection of the problems is taken as the confirmation problem. Correction mechanism: For question-answers with a score below 3.5 or detected problems, a "correction prompt" is used to require the model to output a corrected version, for example: "The original question-answer has the following problems: [Problem list], please generate the corrected question-answer." After correction, the system re-enters the verification process. Indicators: Output the "semantic consistency score (0-1)," "contradiction flag," and "number of corrections" for each question-answer. The passing standard is: a score ≥0.8 and a false contradiction flag.

[0047] Step 4.4: Expert spot checks and feedback feedback; The data that passes the aforementioned verification will undergo manual expert sampling review (sampling rate not less than 5%). Any issues discovered will be fed back to the rule base or the preset model verification program, forming a closed-loop optimization. The specific feedback mechanism is as follows: Issue labeling: Experts will label the question-answer pairs on the review interface. Labeling types include: parameter range error, order error, causal error, non-standard terminology, semantic ambiguity, etc.

[0048] The closed-loop feedback path is as follows: Parameter-related issues → Update the rule base in step 4.1: Add or correct the correct parameter range confirmed by experts to the industry rule base. Simultaneously, record the large-scale model judgment error patterns for this parameter to optimize parameter extraction prompts; Sequence / causality-related issues → Update the knowledge graph in step 2: Correct process sequence relationships or add missing causal edges. After the graph is updated, it triggers the regeneration of questions and answers for related paths in step 3; Semantic issues → Update the mutual verification strategy in step 4.3: Add typical error cases annotated by experts to the examples of "contradiction detection prompts" to form small-sample learning. Simultaneously adjust the model scoring threshold (e.g., increase the penalty weight for a certain type of error if it is frequently missed); Terminology-related issues → Update the terminology mapping table in step 1.

[0049] For each batch of expert feedback received (e.g., every 500 pieces of feedback), a regression test is automatically run to compare the false positive rate and false negative rate of the model validation program before and after optimization. If the false positive rate decreases by more than 10%, the new rule / new suggestion word version is deployed to the production environment.

[0050] Example 2 This invention provides a system for constructing large-scale model training data for the steel industry, comprising: Module M1: Collects raw knowledge data related to the steel industry and preprocesses it, including structured parsing and identification of chapter titles, paragraph levels, tables, formulas and process descriptions to obtain basic text data of the steel industry. Module M2: Uses a large language model to identify process link elements in the basic text data of the steel industry, identifies process nodes, and extracts the sequence relationship, dependency relationship and causal relationship between the process nodes based on the process description, causal explanation and process constraint relationship in the text data, and then constructs a knowledge graph of the steel industry process link. Module M3: Driven by the knowledge graph of the steel industry process link, it uses a graph path constraint decoding strategy to generate question and answer data. When generating each question and answer, it first specifies a path or a subgraph in the graph as the answer skeleton, and generates questions and answers based on the nodes and relationships in the answer skeleton. Module M4: Performs parameter validity and interval constraint verification, process link consistency and causal logic verification, semantic consistency and model cross-checking on the question and answer data, as well as expert sampling and feedback feedback.

[0051] The module M1 includes: Using keyword rules and text classification models, management systems, organizational structure descriptions, advertising descriptions, or repetitive paragraphs that are irrelevant to specific processes are removed from the original knowledge data. The text is cleaned up in a unified manner, including headers and footers, table of contents, reference numbers, OCR-recognized incorrect characters, garbled characters, broken text, duplicate paragraphs, and duplicate descriptions across documents. A terminology mapping table for the steel industry was constructed to unify synonyms and aliases in the text, and the parameter units were uniformly converted to a standard unit system.

[0052] The module M2 includes: The basic text data of the steel industry is segmented into paragraphs, with each paragraph accompanied by a chapter level label. For tables and process description units, they are converted into text descriptions in key-value pair format. The large language model is invoked to identify process nodes, process step nodes, equipment nodes, and process parameter nodes from the text using predefined prompt word templates; Based on the prior knowledge base of the steel industry, the recognition results are corrected according to rules. If a node appears in different categories at the same time, disambiguation is performed based on the context suffix words. If the parameter node is not associated with any process, the association is completed based on the most recent process mentioned in the text. During the relationship extraction process, the upstream-downstream process relationship is constrained to conform to the predefined steelmaking process rules, and the parameter node is only associated with its own process or equipment node.

[0053] The module M3 includes: Based on the preset question and answer types, the corresponding structural subgraphs are sampled from the steel industry process link knowledge graph; For each question-and-answer type, a fixed question template is designed, and the node names in the subgraph are filled into the question template; The node attributes in the subgraph are used as constraint facts to prompt the large language model to generate an answer based on the constraint facts. After the model outputs, the answer is checked by rules to see if it contains new nodes outside the subgraph. If so, it is rejected and regenerated. For the same subgraph, randomly perturb the wording in the question template to generate multiple variant question-answer pairs.

[0054] The module M4 includes: The parameter validity and interval constraint verification includes: verifying the validity of process parameters involved in the question and answer based on the preset industry rule base and standard range; for parameters not predefined in the rule base, inputting the parameter name and context into the large language model, requiring it to output a common range and compare it with common knowledge in the steel industry, and marking it as illegal if it is within the preset abnormal range; The process flow consistency and causal logic verification includes: extracting all process nodes from the question and answer, constructing candidate sequences according to the mention order or explicit sequence words in the text; querying the main process chain in the knowledge graph to obtain a standard sequence; determining whether the candidate sequence is a subsequence of the standard sequence, and identifying causal assertions in the question and answer; querying whether there is a causal edge from one node to another or a causal path through an intermediate node in the knowledge graph; if it exists and the causal strength is not less than 0.5, it is judged as reasonable; if there is no causal edge but there is a strong correlation, it is marked as needing semantic verification; if they are completely unrelated, they are judged as unreasonable. The semantic consistency and model cross-calibration include: inputting question-answer pairs into two large language models with different architectures, requiring the models to output a score on whether the question-answer is semantically consistent and conforms to the knowledge of the steel industry; taking the average of the scores of the two models as the final score, and if the score difference is not less than 1.5, triggering a third model for arbitration; using special prompt words to guide the two models to independently detect logical contradictions or professional errors in the question-answer, and taking the intersection question as the confirmation question; for question-answers with scores lower than a preset threshold or with detected problems, using correction prompt words to require the model to output a corrected version; The expert sampling and feedback process includes: manually sampling and reviewing the data that has passed the verification; marking any problems found as parameter range errors, sequence errors, causal errors, non-standard terminology, or semantic ambiguity. Specifically, parameter-related problems are used to update the rule base for parameter validity and interval constraint verification; sequence / causal problems are used to correct the knowledge graph; semantic problems are used to add typical error cases marked by experts to the example of contradictory detection prompts for semantic consistency and model cross-verification; and terminology problems are used to update the terminology mapping table.

[0055] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0056] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for constructing training data for a large model in the steel industry, characterized in that, include: Step 1: Collect raw knowledge data related to the steel industry and preprocess it, including structured parsing and identifying chapter titles, paragraph levels, tables, formulas and process descriptions to obtain basic text data of the steel industry. Step 2: Use a large language model to identify process link elements in the basic text data of the steel industry, identify process nodes, and extract the sequence relationship, dependency relationship and causal relationship between the process nodes based on the process description, causal explanation and process constraint relationship in the text data, and then construct a knowledge graph of the process link of the steel industry. Step 3: Driven by the knowledge graph of the steel industry process link, a graph path constraint decoding strategy is used to generate question and answer data. When generating each question and answer, a path or a subgraph in the graph is first specified as the answer skeleton, and the question and answer are generated based on the nodes and relationships in the answer skeleton. Step 4: Perform parameter validity and interval constraint verification, process link consistency and causal logic verification, semantic consistency and model cross-checking on the question and answer data, as well as expert sampling and feedback feedback.

2. The method for constructing large-scale model training data for the steel industry according to claim 1, characterized in that, Step 1 includes: Using keyword rules and text classification models, management systems, organizational structure descriptions, advertising descriptions, or repetitive paragraphs that are irrelevant to specific processes are removed from the original knowledge data. The text is cleaned up in a unified manner, including headers and footers, table of contents, reference numbers, OCR-recognized incorrect characters, garbled characters, broken text, duplicate paragraphs, and duplicate descriptions across documents. A terminology mapping table for the steel industry was constructed to unify synonyms and aliases in the text, and the parameter units were uniformly converted to a standard unit system.

3. The method for constructing large-scale model training data for the steel industry according to claim 1, characterized in that, Step 2 includes: The basic text data of the steel industry is segmented into paragraphs, with each paragraph accompanied by a chapter level label. For tables and process description units, they are converted into text descriptions in key-value pair format. The large language model is invoked to identify process nodes, process step nodes, equipment nodes, and process parameter nodes from the text using predefined prompt word templates; Based on the prior knowledge base of the steel industry, the recognition results are corrected according to rules. If a node appears in different categories at the same time, disambiguation is performed based on the context suffix words. If the parameter node is not associated with any process, the association is completed based on the most recent process mentioned in the text. During the relationship extraction process, the upstream-downstream process relationship is constrained to conform to the predefined steelmaking process rules, and the parameter node is only associated with its own process or equipment node.

4. The method for constructing large-scale model training data for the steel industry according to claim 1, characterized in that, Step 3 includes: Based on the preset question and answer types, the corresponding structural subgraphs are sampled from the steel industry process link knowledge graph; For each question-and-answer type, a fixed question template is designed, and the node names in the subgraph are filled into the question template; The node attributes in the subgraph are used as constraint facts to prompt the large language model to generate an answer based on the constraint facts. After the model outputs, the answer is checked by rules to see if it contains new nodes outside the subgraph. If so, it is rejected and regenerated. For the same subgraph, randomly perturb the wording in the question template to generate multiple variant question-answer pairs.

5. The method for constructing large-scale model training data for the steel industry according to claim 1, characterized in that, Step 4 includes: The parameter validity and interval constraint verification includes: verifying the validity of process parameters involved in the question and answer based on the preset industry rule base and standard range; for parameters not predefined in the rule base, inputting the parameter name and context into the large language model, requiring it to output a common range and compare it with common knowledge in the steel industry, and marking it as illegal if it is within the preset abnormal range; The process flow consistency and causal logic verification includes: extracting all process nodes from the question and answer, constructing candidate sequences according to the mention order or explicit sequence words in the text; querying the main process chain in the knowledge graph to obtain a standard sequence; determining whether the candidate sequence is a subsequence of the standard sequence, and identifying causal assertions in the question and answer; querying whether there is a causal edge from one node to another or a causal path through an intermediate node in the knowledge graph; if it exists and the causal strength is not less than 0.5, it is judged as reasonable; if there is no causal edge but there is a strong correlation, it is marked as needing semantic verification; if they are completely unrelated, they are judged as unreasonable. The semantic consistency and model cross-calibration include: inputting question-answer pairs into two large language models with different architectures, requiring the models to output a score on whether the question-answer is semantically consistent and conforms to the knowledge of the steel industry; taking the average of the scores of the two models as the final score, and if the score difference is not less than 1.5, triggering a third model for arbitration; using special prompt words to guide the two models to independently detect logical contradictions or professional errors in the question-answer, and taking the intersection question as the confirmation question; for question-answers with scores lower than a preset threshold or with detected problems, using correction prompt words to require the model to output a corrected version; The expert sampling and feedback process includes: manually sampling and reviewing the data that has passed the verification; marking any problems found as parameter range errors, sequence errors, causal errors, non-standard terminology, or semantic ambiguity. Specifically, parameter-related problems are used to update the rule base for parameter validity and interval constraint verification; sequence / causal problems are used to correct the knowledge graph; semantic problems are used to add typical error cases marked by experts to the example of contradictory detection prompts for semantic consistency and model cross-verification; and terminology problems are used to update the terminology mapping table.

6. A system for constructing large-scale model training data for the steel industry, characterized in that, include: Module M1: Collects raw knowledge data related to the steel industry and preprocesses it, including structured parsing and identification of chapter titles, paragraph levels, tables, formulas and process descriptions to obtain basic text data of the steel industry. Module M2: Uses a large language model to identify process link elements in the basic text data of the steel industry, identifies process nodes, and extracts the sequence relationship, dependency relationship and causal relationship between the process nodes based on the process description, causal explanation and process constraint relationship in the text data, and then constructs a knowledge graph of the steel industry process link. Module M3: Driven by the knowledge graph of the steel industry process link, it uses a graph path constraint decoding strategy to generate question and answer data. When generating each question and answer, it first specifies a path or a subgraph in the graph as the answer skeleton, and generates questions and answers based on the nodes and relationships in the answer skeleton. Module M4: Performs parameter validity and interval constraint verification, process link consistency and causal logic verification, semantic consistency and model cross-checking on the question and answer data, as well as expert sampling and feedback feedback.

7. The system for constructing large-scale model training data for the steel industry according to claim 6, characterized in that, The module M1 includes: Using keyword rules and text classification models, management systems, organizational structure descriptions, advertising descriptions, or repetitive paragraphs that are irrelevant to specific processes are removed from the original knowledge data. The text is cleaned up in a unified manner, including headers and footers, table of contents, reference numbers, OCR-recognized incorrect characters, garbled characters, broken text, duplicate paragraphs, and duplicate descriptions across documents. A terminology mapping table for the steel industry was constructed to unify synonyms and aliases in the text, and the parameter units were uniformly converted to a standard unit system.

8. The large-scale model training data construction system for the steel industry according to claim 6, characterized in that, The module M2 includes: The basic text data of the steel industry is segmented into paragraphs, with each paragraph accompanied by a chapter level label. For tables and process description units, they are converted into text descriptions in key-value pair format. The large language model is invoked to identify process nodes, process step nodes, equipment nodes, and process parameter nodes from the text using predefined prompt word templates; Based on the prior knowledge base of the steel industry, the recognition results are corrected according to rules. If a node appears in different categories at the same time, disambiguation is performed based on the context suffix words. If the parameter node is not associated with any process, the association is completed based on the most recent process mentioned in the text. During the relationship extraction process, the upstream-downstream process relationship is constrained to conform to the predefined steelmaking process rules, and the parameter node is only associated with its own process or equipment node.

9. The system for constructing large-scale model training data for the steel industry according to claim 6, characterized in that, The module M3 includes: Based on the preset question and answer types, the corresponding structural subgraphs are sampled from the steel industry process link knowledge graph; For each question-and-answer type, a fixed question template is designed, and the node names in the subgraph are filled into the question template; The node attributes in the subgraph are used as constraint facts to prompt the large language model to generate an answer based on the constraint facts. After the model outputs, the answer is checked by rules to see if it contains new nodes outside the subgraph. If so, it is rejected and regenerated. For the same subgraph, randomly perturb the wording in the question template to generate multiple variant question-answer pairs.

10. The system for constructing large-scale model training data for the steel industry according to claim 6, characterized in that, The module M4 includes: The parameter validity and interval constraint verification includes: verifying the validity of process parameters involved in the question and answer based on the preset industry rule base and standard range; for parameters not predefined in the rule base, inputting the parameter name and context into the large language model, requiring it to output a common range and compare it with common knowledge in the steel industry, and marking it as illegal if it is within the preset abnormal range; The process flow consistency and causal logic verification includes: extracting all process nodes from the question and answer, constructing candidate sequences according to the mention order or explicit sequence words in the text; querying the main process chain in the knowledge graph to obtain a standard sequence; determining whether the candidate sequence is a subsequence of the standard sequence, and identifying causal assertions in the question and answer; querying whether there is a causal edge from one node to another or a causal path through an intermediate node in the knowledge graph; if it exists and the causal strength is not less than 0.5, it is judged as reasonable; if there is no causal edge but there is a strong correlation, it is marked as needing semantic verification; if they are completely unrelated, they are judged as unreasonable. The semantic consistency and model cross-calibration include: inputting question-answer pairs into two large language models with different architectures, requiring the models to output a score on whether the question-answer is semantically consistent and conforms to the knowledge of the steel industry; taking the average of the scores of the two models as the final score, and if the score difference is not less than 1.5, triggering a third model for arbitration; using special prompt words to guide the two models to independently detect logical contradictions or professional errors in the question-answer, and taking the intersection question as the confirmation question; for question-answers with scores lower than a preset threshold or with detected problems, using correction prompt words to require the model to output a corrected version; The expert sampling and feedback process includes: manually sampling and reviewing the data that has passed the verification; marking any problems found as parameter range errors, sequence errors, causal errors, non-standard terminology, or semantic ambiguity. Specifically, parameter-related problems are used to update the rule base for parameter validity and interval constraint verification; sequence / causal problems are used to correct the knowledge graph; semantic problems are used to add typical error cases marked by experts to the example of contradictory detection prompts for semantic consistency and model cross-verification; and terminology problems are used to update the terminology mapping table.

Images