A method and system for extracting domain knowledge for special material safety standards

Abstract

The application relates to the technical field of knowledge extraction, in particular to a special material safety standard-oriented domain knowledge extraction method and system; the method comprises the following steps: collecting special material safety standard data, pre-processing the data, and classifying standard clauses through manual work; carrying out ontology design; extracting knowledge from structured and semi-structured data; extracting knowledge from unstructured data; fine-tuning and optimizing the preference of entity types; the special material safety standard-oriented domain knowledge extraction system comprises a data collection module, a classification module, an ontology design module, a first data knowledge extraction module, a second data knowledge extraction and preference fine-tuning module; through the above mode, the professional domain corpus annotation efficiency can be improved, the dependence on artificial resources is reduced, and the problems of local deployment difficulty, insufficient understanding of professional terms in the model professional field and insufficient knowledge extraction capacity are solved.

Description

Technical Field

[0001] This invention relates to the field of knowledge extraction technology, and in particular to a method and system for extracting domain knowledge for special material safety standards. Background Technology

[0002] Energetic materials (such as gunpowder, explosives, and pyrotechnics) have wide applications in high-risk fields such as national defense, aerospace, and mining due to their high-energy properties. However, their high-energy properties also bring safety hazards such as explosions, combustion, and leaks. To effectively prevent accidents and ensure personnel and environmental safety, countries have formulated strict safety standards, which cover multiple aspects from raw material control to production processes, transportation, storage, and protective measures during use. With the rapid development of information technology, knowledge graphs, as a structured semantic knowledge base, have shown great potential for efficient knowledge management because they can describe the relationships between things in the world. In the field of special materials safety standards, the construction of knowledge graphs can integrate scattered standard data into a unified framework, facilitating querying, analysis, and updates, thereby improving the effectiveness of standard implementation and regulatory efficiency.

[0003] Traditional knowledge graph construction methods rely on a large amount of manual annotation, which not only requires annotators to have strong relevant domain knowledge, but is also limited by tedious annotation work, resulting in low annotation efficiency and high cost. In the face of general large language models, it is difficult to apply in environments with limited computing resources and there is insufficient understanding of professional terms in specific domain scenarios.

[0004] In conclusion, it is essential to propose a domain knowledge extraction method and system for special material safety standards that improves the efficiency of professional domain corpus annotation, reduces reliance on manual resources, and alleviates difficulties in local deployment, insufficient understanding of professional terminology in the model's professional domain, and inadequate knowledge extraction capabilities. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for extracting domain knowledge for special material safety standards. This method can improve the efficiency of professional domain corpus annotation, reduce reliance on manual resources, and alleviate the difficulties of local deployment, insufficient understanding of professional terminology in the model domain, and insufficient knowledge extraction capabilities.

[0006] To achieve the above objectives, this invention employs a domain knowledge extraction method for special material safety standards, comprising the following steps:

[0007] Collect safety standard data for special materials, preprocess the data, and manually categorize the standard clauses;

[0008] Perform the ontology design;

[0009] Knowledge extraction from structured and semi-structured data;

[0010] Knowledge extraction from unstructured data;

[0011] Fine-tuning and optimization of preferences for entity types.

[0012] Among the steps involved in collecting safety standard data for special materials, preprocessing the data, and manually categorizing the standard clauses:

[0013] Documents were collected from national and industry standards and normative standards of the special materials safety standard system, covering terminology definitions, classification labels, sensitivity testing, production processes, storage, transportation and application safety procedures, totaling 130 documents;

[0014] Use OCR technology to process PDF files and extract text and table content from the document;

[0015] The document content is labeled and divided into structured, unstructured, and semi-structured regions according to the above description, and the content is extracted for each region.

[0016] Remove images, headers, footers, and watermarks; remove extra spaces and line breaks from text; correct misidentified text; create standard knowledge entries; fill, delete, and correct table data; handle missing and outlier values ​​in the data; and create structured, semi-structured, and unstructured data.

[0017] The documents were manually categorized into five categories: comprehensive safety management, design and construction safety, production and operation safety, storage and transportation safety, and testing and emergency response, thus constructing a systematic safety standard resource library.

[0018] Among the steps involved in manually categorizing documents and dividing standard clauses into five categories—comprehensive safety management, design and construction safety, production and operation safety, storage and transportation safety, and testing and emergency response—to build a systematic safety standard resource library:

[0019] The comprehensive safety management category covers the overall requirements for establishing a safety system, management responsibilities, safety assessments, and safety education. The design and construction safety category focuses on the safety design principles of plant layout, explosion-proof buildings, and explosion-proof facilities. The production and operation safety category covers the process flow, equipment requirements, personnel operating procedures, and work protection equipment for energetic materials during manufacturing. The storage and transportation safety category specifies the conditions, facility configuration, and warehousing process management for materials during storage, handling, and transportation. The testing and emergency response category specifies product performance testing, hazardous waste disposal, and accident response mechanisms to ensure rapid response and risk control in abnormal situations.

[0020] Among the steps in ontology design:

[0021] Based on the needs, seven categories of entity types are constructed, including energetic materials, standard specifications, professional terminology, laws and regulations, technical standards, safety requirements, and testing methods;

[0022] The design incorporates three main relation types: standard metadata relations, reference and dependency relations, and technical clause relations.

[0023] Among the steps involved in knowledge extraction from structured and semi-structured data:

[0024] Semi-structured data is extracted, and rule-based methods such as regular expressions and dictionary matching are used to extract the name, publication date, standard number, drafting unit, and drafter entities and their attribute information from the standard.

[0025] Extract structured data and transform it into standard triplet form based on the entity, attribute, and relation columns in the table.

[0026] Among the steps involved in knowledge extraction from unstructured data:

[0027] Based on the prompt word engineering and large language model, a customized fine-tuning dataset Ssem-Instruct for special material safety standards was constructed through manual verification. The initially extracted data was selected based on expert opinions to form a positive example fine-tuning dataset, while the unqualified data was improved to construct a negative example dataset.

[0028] We used Qwen2.5-7B as the base model, and optimized the model by combining the QLoRA fine-tuning technique with positive and negative example fine-tuning datasets. During the testing phase, we collected the data with extraction errors and included them in the secondary optimization and improvement dataset for further preference fine-tuning.

[0029] We constructed a large-scale triple extraction model, SsemLM, for the field of special materials safety standards. Based on this model and combined with the knowledge extraction optimization process, we carried out triple extraction tasks to form a knowledge graph for the field of special materials safety standards.

[0030] Among the steps in the fine-tuning and optimization of entity type preferences:

[0031] To address the issue of incorrect entity type identification, a third fine-tuning strategy is designed, incorporating DPO preference optimization to train the model to output the correct entity type.

[0032] By collecting data on errors in entity type reasoning in the model after the second fine-tuning, a preference fine-tuning dataset is constructed to optimize the model in a targeted manner.

[0033] This invention also provides a domain knowledge extraction system for special material safety standards, including a data collection module, a classification module, an ontology design module, a first data knowledge extraction module, a second data knowledge extraction and preference fine-tuning module; wherein:

[0034] The data collection module is used to collect safety standard data for special materials and preprocess the data;

[0035] The classification module is used to classify the standard clauses;

[0036] The ontology design module is used for ontology design;

[0037] The first data knowledge extraction module is used to extract knowledge from structured and semi-structured data;

[0038] The second data knowledge extraction is used for knowledge extraction from unstructured data;

[0039] The preference fine-tuning module is used to fine-tune and optimize the preferences for entity types.

[0040] This invention discloses a method and system for extracting domain knowledge for special material safety standards. The method comprises a data collection module, a classification module, an ontology design module, a first data knowledge extraction module, a second data knowledge extraction module, and a preference fine-tuning module, performing the following steps: collecting special material safety standard data; preprocessing the data and manually classifying the standard clauses; performing ontology design; extracting knowledge from structured and semi-structured data; extracting knowledge from unstructured data; and fine-tuning and optimizing entity types. In this approach, different paths are used for knowledge extraction from unstructured text and structured data. For text, the Qwen2.5-7B model is fine-tuned using fine-tuning technology to form the unstructured data triple extraction model SsemLM. For structured and semi-structured data, a rule-based method is used for triple extraction. This method improves the efficiency of professional domain corpus annotation, reduces reliance on manual resources, and alleviates difficulties in local deployment, insufficient understanding of professional domain terminology, and inadequate knowledge extraction capabilities. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of the steps of the domain knowledge extraction method for special material safety standards of the present invention.

[0043] Figure 2 This is a flowchart of steps S100 of the present invention.

[0044] Figure 3 This is a flowchart of steps S200 of the present invention.

[0045] Figure 4 This is a flowchart of steps S300 of the present invention.

[0046] Figure 5 This is a flowchart of steps S400 of the present invention.

[0047] Figure 6 This is a flowchart of steps S500 of the present invention.

[0048] Figure 7 This is the framework diagram for constructing a knowledge graph of energetic material safety standards according to the present invention.

[0049] Figure 8This is a framework diagram for constructing the energetic materials safety standard resource library of the present invention.

[0050] Figure 9 This is a classification diagram of the safety standards for energetic materials according to the present invention.

[0051] Figure 10 This is a flowchart of the fine-tuning data construction, model fine-tuning, and triplet extraction process of the present invention.

[0052] Figure 11 This is a flowchart of the two-stage fine-tuning process of the present invention.

[0053] Figure 12 This is a schematic diagram of the model fine-tuning structure of the present invention.

[0054] Figure 13 This is a schematic diagram of the domain knowledge extraction system for special material safety standards of the present invention.

[0055] 601 - Data Collection Module, 602 - Classification Module, 603 - Ontology Design Module, 604 - First Data Knowledge Extraction Module, 605 - Second Data Knowledge Extraction Module, 606 - Preference Fine-tuning Module. Detailed Implementation

[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0057] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0058] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0059] Please see Figures 1-12This invention provides a method for extracting domain knowledge for safety standards of special materials, comprising the following steps:

[0060] S100: Collect safety standard data for special materials, preprocess the data, and manually categorize the standard clauses.

[0061] In this embodiment, special material safety standard data is collected, the data is preprocessed, and the standard clauses are manually categorized. The specific process is as follows:

[0062] S101: Collect documents from national and industry standards for the safety standard system of special materials, covering terminology definitions, classification labels, sensitivity testing, production processes, storage, transportation and application safety procedures, totaling 130 documents;

[0063] S102: Use OCR technology to process PDF files and extract text and table content from the document;

[0064] S103: Label the document content, divide the document into structured, unstructured and semi-structured regions as described above, and extract content for each region;

[0065] S104: Remove images, headers, footers and watermarks, remove extra spaces and line breaks from text, correct misidentified text, form standard knowledge entries, fill in, delete and correct table data, handle missing and outlier values ​​in data, and form structured data, semi-structured data and unstructured data.

[0066] S105: Manually categorize the documents, dividing the standard clauses into five categories: comprehensive safety management, design and construction safety, production and operation safety, storage and transportation safety, and testing and emergency response, to build a systematic safety standard resource library.

[0067] In the above process, firstly, documents were collected from national and industry standards of the special materials safety standard system, covering terminology definitions, classification labels, sensitivity testing, production processes, storage, transportation, and application safety procedures, totaling 130 documents. Next, OCR technology was used to process the PDF files, extracting text and table content. Then, the document content was labeled, dividing the documents into structured, unstructured, and semi-structured areas as described above, and content was extracted for each area. Then, images, headers, footers, and watermarks were removed, redundant spaces and line breaks were removed from the text, and misidentified text was corrected to form standard knowledge entries. Table data was then filled, deleted, and corrected, and missing and outlier values ​​were handled, resulting in structured, semi-structured, and unstructured data. Finally, the documents were manually categorized, and the labels were... The standard provisions are divided into five categories: comprehensive safety management, design and construction safety, production and operation safety, storage and transportation safety, and testing and emergency response. This forms the basis of a systematic safety standard resource library. The comprehensive safety management category covers the overall requirements for establishing a comprehensive safety system, management responsibilities, safety assessments, and safety education. The design and construction safety category focuses on the safety design principles of plant layout, explosion-proof buildings, and explosion-proof facilities. The production and operation safety category covers the process flow, equipment requirements, personnel operating procedures, and protective equipment for energetic materials during manufacturing. The storage and transportation safety category specifies the conditions, facility configuration, and warehousing process management for materials during storage, handling, and transportation. The testing and emergency response category specifies product performance testing, hazardous waste disposal, and accident response mechanisms to ensure rapid response and risk control in abnormal situations.

[0068] S200: Perform the main body design.

[0069] In this embodiment, the ontology design is carried out, and the specific process is as follows:

[0070] S201: Construct seven categories of entity types based on requirements, including energetic materials, standard descriptions, technical terms, laws and regulations, technical standards, safety requirements, and testing methods;

[0071] S202: Design three main relation types, including standard metadata relations, reference and dependency relations, and technical clause relations.

[0072] In the above process, following the top-down construction principle of knowledge graphs, the entity-relationship-tail entity structure is clearly defined. Entity types are relatively fixed in energetic materials safety standards. Based on requirements, seven major entity types are constructed, including energetic materials, standard descriptions, technical terms, laws and regulations, technical standards, safety requirements, and testing methods. Three main relation term types are designed, including standard metadata relations, reference and dependency relations, and technical clause relations.

[0073] S300: Knowledge extraction from structured and semi-structured data.

[0074] In this embodiment, the specific process for extracting knowledge from structured and semi-structured data is as follows:

[0075] S301: Extract semi-structured data, using rule-based methods such as regular expressions and dictionary matching to extract the name, publication date, standard number, drafting unit, and drafter entities and their attribute information from the standard;

[0076] S302: Extract structured data and transform it into standard triplet form based on the entity, attribute, and relation columns in the table.

[0077] In the above process, the basic information of security standard documents, such as the document's standard number, drafting unit, drafter, and various dates, are clearly defined and belong to semi-structured data. For this type of rule-based natural language data, rule-based methods such as regular expressions and dictionary matching are used to extract the name, publication date, standard number, drafting unit, and drafter entities and their attribute information from the standard. The rule-based methods are shown in Table 1.

[0078] Entity type Rule matching Example Standard Number Regular expressions such as "AQ / T\d{4}-\d{4}" match standard number formats. AQ / T3030-2010 Standard Name Extracting from the pattern "《(.*?)》standard number is" Safety Production Training Outline and Assessment Standards for Safety Production Management Personnel in Hazardous Chemical Production Units Date Entities Regular expression matching the format "yyyy-mm-dd" 2010-09-06 Extraction of information from the responsible unit, drafting unit, and drafter The name of the organization following the key verbs "propose", "under the jurisdiction", "drafting unit", and "drafter" State Administration of Work Safety Standard Reference The standard number or name following "referenced standard" GB12463, General Technical Requirements for Transport Packaging of Dangerous Goods

[0079] Table 1

[0080] For structured Excel spreadsheets, the data is transformed into standard triples based on the entity, attribute, and relation columns in the table. Neither of the above two methods requires model training and can efficiently construct knowledge graphs for specific domains.

[0081] S400: Knowledge extraction from unstructured data.

[0082] In this embodiment, the specific process for extracting knowledge from unstructured data is as follows:

[0083] S401: Based on the prompt word engineering and large language model, and through manual verification, construct a customized fine-tuning dataset Ssem-Instruct for special material safety standards. The initially extracted data is selected based on expert opinions to form a positive example fine-tuning dataset, and the unqualified data is improved to construct a negative example dataset.

[0084] S402: Using Qwen2.5-7B as the base model, the model is optimized based on QLoRA fine-tuning technology combined with positive and negative example fine-tuning datasets. During the testing phase, data with extraction errors are collected and included in the secondary optimization and improvement dataset for further preference fine-tuning.

[0085] S403: Construct a large-scale triple extraction model SsemLM for the field of special materials safety standards, and perform triple extraction tasks based on this model and knowledge extraction optimization process to form a knowledge graph for the field of special materials safety standards.

[0086] In the above process, based on the prompt word engineering and large language model, a customized fine-tuning dataset Ssem-Instruct for special material safety standards was constructed through manual verification. The initially extracted data was selected based on expert opinions to form a positive example fine-tuning dataset, while the unqualified data was improved to construct a negative example dataset. Then, Qwen2.5-7B was used as the base large model, and the model was optimized based on QLoRA fine-tuning technology combined with the positive and negative example fine-tuning datasets. During the testing phase, the data with extraction errors were collected and included in the secondary optimization and improvement dataset for further preference fine-tuning. Finally, a triple extraction large model SsemLM for the field of special material safety standards was constructed. Based on this model and the knowledge extraction optimization process, the triple extraction task was carried out to form a knowledge graph for the field of special material safety standards.

[0087] As shown in Table 2, the knowledge graph entities and their descriptions are as follows:

[0088] entity Entity description Energetic materials Such as explosives, fireworks, waste energetic materials, ammunition, etc. Standard cataloging The standard document includes the standard number, drafting unit, drafter, and responsible unit. technical terms Technical terms mentioned in energetic materials safety standards include detonation velocity, explosion-proof chamber, and disposal. Laws and regulations The scope of application of the standard, local regulations, and scope of effect, etc. Technical Standards Technical requirements for energetic materials handling methods, processing equipment, and working environment. Safety requirements Safety requirements for the production, use, and transportation of energetic materials, etc. Materials Testing Testing instruments, material property values, and testing methods for energetic products.

[0089] Table 2

[0090] As shown in Table 3, the types and descriptions of relation terms in the knowledge graph are as follows:

[0091] Relationship type Relationship description Standard metadata relationships Describe the standard's own attributes and lifecycle, such as (standard number, publication date, implementation date, supersedes, responsible unit, drafting unit, main drafter, applicable to, etc.). References and Dependencies Describe the references and dependencies between entities (such as references, compliance with standards, reference to standards, conflict clauses, prohibited behaviors, compliance with specifications, etc.). Technical terms relationship Describe the technical requirements and equipment parameters (must be basic requirements, must be met, strictly prohibited, should be equipped, not allowed, recommended, should be adopted, must not exceed, must not be less than, applicable, not applicable, etc.).

[0092] Table 3

[0093] In the above approach, a large-scale knowledge extraction model, SsemLM, for the field of energetic materials safety standards was constructed by combining a two-step method. Based on this model, the automated extraction task of fine-grained knowledge units was realized, that is, identifying and extracting key information such as professional terms, technical standards, and safety requirements from unstructured text, providing support for the structured representation of domain knowledge. The model fine-tuning and triplet extraction process is as follows: Figure 10 As shown.

[0094] Furthermore, the construction and improvement of the positive and negative example dataset for energetic materials safety standards: Given the illusion problem commonly found in large language models, and considering that cue word engineering can limit the output content and effectively improve the accuracy of the generated content, the cue words are as follows:

[0095] You are an expert in named entity recognition and relation identification in the field of energetic materials safety standards. I will provide you with some text information from which you need to extract entities and relations, and output the results in JSON format: ["h":"head entity", "r":"relation word", "o":"tail entity"], where h and o represent entity type definitions.

[0096] Identify constraints:

[0097] 1. Entity recognition allows for the use of useful qualifiers or modifiers to make the semantics of entities clearer;

[0098] 2. Triples where no relationship exists between entities do not need to be provided;

[0099] 3. Modification of the original fields is not allowed.

[0100] Entity definition:

[0101] 1. Energetic materials: such as explosives, fireworks, waste energetic substances, ammunition, etc.;

[0102] 2. Standard bibliography: Standard document, standard number, drafting unit, drafter, responsible unit, etc.;

[0103] 3. Technical terms: Technical terms mentioned in the safety standards for energetic materials, such as detonation velocity, explosion-proof chamber, and disposal;

[0104] 4. Laws and regulations: Scope of application of the standard, local regulations, scope of effect, etc.

[0105] 5. Technical Standards: Technical requirements for energetic material handling methods, processing equipment, and working environment;

[0106] 6. Safety requirements: Safety requirements for the production, use, and transportation of energetic materials, etc.

[0107] 7. Materials testing: testing instruments, material property values, and testing methods for energetic products.

[0108] Relationship definition:

[0109] Used to describe the degree of mandatory or advisory nature of technical requirements and constraints in specifications or standards, covering mandatory provisions, prohibitions, recommended practices, and applicability statements (such as mandatory, basic requirements, must be met, strictly prohibited, should be equipped with, not allowed, recommended, processing requirements, must not exceed, must not be less than, applicable, not applicable, etc.).

[0110] JSON format:

[0111] Shaped like this. {"Technical Terminology":"Support Ring","r":"Processing Requirements","Technical Requirements":"The rupture disc can detach in a timely manner"}.

[0112] The prompt words specify the data requirements, including task requirements, format output requirements and recognition constraints, and limit the entity type and relation type. The entity definition and relation definition are the same as the ontology design in step S200. These contents can guide the model to generate triplet data in the field of energetic materials safety standards, and construct fine-tuning instructions based on this.

[0113] To ensure the accuracy and standardization of knowledge extraction, this paper sets constraints in the prompt words for identification. These restrictions are necessary in the field of energetic materials safety standards: energetic materials standards have complex terminology, and many core entities are themselves nested phrases, such as "automatic control ignition system" and "aluminothermic reaction propellant." Allowing the retention of entity nesting or modifiers that contribute to semantics prevents the construction of unfounded relationships; safety standard documents emphasize logical rigor and sufficient technical evidence. If there is no clear semantic connection between two entities, forcibly constructing triples may introduce pseudo-knowledge, thus ensuring the authenticity of the constructed knowledge graph. Furthermore, energetic materials terminology is generally fixed; arbitrary changes to terminology or field content may lead to inconsistencies with the original standard. Prohibiting changes to original fields helps maintain consistency between terminology and standard text, ensuring the authority of the extracted knowledge. First, a preliminary fine-tuning dataset is constructed using the DeepSeek-R1 model and prompt words to address resource scarcity and reduce the cost of manual annotation. Then, manual methods are used to ensure the correctness and consistency of the instructions. Then, valid instructions are extracted to form a positive example fine-tuning dataset. However, large language models may encounter issues with extraction, such as uniqueness (repeated extraction of the same triplet), accuracy (incorrect meaning of extracted triples, i.e., no or incorrect relationship between head and tail entities), and conciseness (extracted triples contain redundant content). These substandard data are then improved into negative example training data to enhance the model's ability to avoid errors in knowledge extraction during fine-tuning. The negative example instruction template is shown below:

[0114] "conversations": [

[0115] {

[0116] "from": "system",

[0117] "value": "You are an expert in named entity recognition and relation recognition in the field of energetic materials safety standards. I will provide you with some information from this article, analyze whether the answer is correct, and give the correct answer. The output should follow the JSON format ["h":"head entity", "r":"relation word", "o":"tail entity"], where h and o represent entity type definitions.

[0118] },

[0119] {

[0120] "from": "user",

[0121] "value": "Text: " <text>Incorrect triple: <"h":"head entity", "r":"relation word", "o":"tail entity">, Error description: <description>"

[0122] },

[0123] {

[0124] "from": "assistant",

[0125] "value": "Analysis: "<Error Type> The correct triple is: {"h":"head entity", "r":"relation word", "o":"tail entity"}

[0126] } ]

[0128] The task requirements and the output format (JSON) are given, along with a given text. <text>Error triples and error descriptions <description>The answer requires judging based on the given incorrect triples and providing a response. <errortype>Error types and correct triplets are defined. Incorrect triplets are the original triplets, and correct triplets are the expert-modified triplets. The improved algorithm is shown below:

[0129] Step 1. Initialize i = 1

[0130] Step 2. Repeat the process until i > n:

[0131] Step 2.1. Take the i-th instruction si=(x, y) from set S.

[0132] Step 2.2. Determine if it is a positive example

[0133] Step 2.3.if false:

[0134] Step2.4.(ef, rf, yf, tf)=g(i)

[0135] Step 2.5. Fill in the negative example learning prompt template 𝑝 to obtain the negative example instruction set o

[0136] Step 2.6. Add 𝑜 to the negative example dataset O

[0137] Step 2.7.i increments by 1

[0138] Step 3. Return to the reflection dataset O

[0139] The S set is the instruction set obtained after the initial extraction of the large model. It is used to determine whether an instruction set is a positive example; if not, the instruction set is improved into a negative example for fine-tuning. i The expert opinion includes, e f For incorrect entities, r f For the incorrect relationship, e f r f It includes two fields: the original content and the correct content. If an error occurs, the expert will provide the correct entity and relationship. f This is an incorrect description; the content is based on expert analysis and... f r f The analysis of the content, t f Error types are defined as three categories: entity errors, relational errors, and integrity errors. i The implementation logic is as follows:

[0140] Step 1. Triples<h,r,o> judge

[0141] Step 2. If an entity has an error or is missing.

[0142] Step3.tf = <Entity Error> or <Integrity Error>

[0143] Step4.ef = {"Original Entity": Original value, "Expert Entity": Expert-corrected value}

[0144] Step 5. If there is an error or missing relationship.

[0145] Step6.tf = <Relationship Error> or <Integrity Error>

[0146] Step7.rf = {"Original Relation": Original value, "Expert Relation": Expert-corrected value}

[0147] Step8.yf="Triplet"<h,r,o> The entity or relation in the text is incorrect. It should be: "+ef["expert entity"]+rf["expert relation"]".

[0148] Step 9. Return (ef, rf, yf, tf)

[0149] Then it is filled into the negative example prompt template 𝑝 to form a negative example fine-tuning set, which will be based on e during the backfilling process. f r f The original values ​​in p form incorrect triples, while the expert-corrected values ​​form correct triples. <errortype>Corresponding to t f , <description>Corresponding to y f Instead of wasting data from incorrectly constructed general-purpose models, Ssem-Instruct uses these data as negative example instructions. These negative example training instructions not only fine-tune the model to output correct triples but also prevent the model from generating incorrect triples. Through this patented improved positive and negative example dataset, Ssem-Instruct, the fine-tuned model can effectively adapt to the task of extracting knowledge from energetic materials safety standards and avoid erroneous extraction.

[0150] Furthermore, based on QLoRA, a two-stage fine-tuning process was implemented: optimizations were achieved in the fine-tuning workflow, specifically training the model using the Ssem-Instruct dataset (positive and negative examples built from a general large model), and further fine-tuning and training on substandard data during the model testing phase. This ensured the model achieved good results in the energetic materials safety standard knowledge extraction task. The fine-tuning process is as follows: Figure 12 As shown, there are three fine-tuning tasks in total. The first two fine-tuning tasks use positive and negative example datasets from Ssem-Instruct. Note that Ssem-Instruct fine-tuning needs to be done separately, with positive and negative examples. The positive example dataset generally contains triples that the model can easily identify. This part is to fully adapt the model to the knowledge extraction task; this is the first fine-tuning. After positive example fine-tuning, negative example fine-tuning is performed to update the parameters after positive example fine-tuning. Negative example fine-tuning is to optimize the model's error correction ability in the knowledge extraction task, so that the model can extract the correct triples even on complex text passages; this is the second fine-tuning. The entire fine-tuning process also implements a third fine-tuning. The knowledge graph constructed in this paper is a knowledge graph with entity categories, so correctly identifying entity types is particularly important. The initially constructed SsemLM model was tested on the energetic materials safety standard. The prediction results were judged by experts to determine whether the extracted entities conformed to the entity type. Unqualified entities were input into the improvement module to be improved into a preference fine-tuning dataset. The model was then fine-tuned for entity type preference using this dataset, and the parameters of the first two fine-tunings were updated to enable the model to accurately identify the correct type of the entity in the extracted triplet.

[0151] Furthermore, the positive and negative example fine-tuning strategy: In this stage of fine-tuning, only the QLoRA module is inserted into the linear layer of the self-attention layer, without changing other layers. The model fine-tuning structure diagram is shown below. Figure 11 As shown. Fine-tuning includes the following steps: First, insert a QLoRA module into the attention layer of the Qwen2.5-7b model to fine-tune the model structure. Then, use the trainable low-rank matrices A and B within the module to fine-tune the model, as shown in the following formula:

[0152]

[0153]

[0154] Where W represents the frozen base model weights. It is decomposed into matrices A and B to ensure that no too much information is lost, where the rows of A and B are... The same applies to column B. Similarly, during fine-tuning, only matrices A and B are modified, where r represents the rank of the low-rank matrix, controlling the number of parameters in the fine-tuning process. This paper chooses 8 r. Then, the original parameters W of the base model are frozen, and the model weights are quantized using FN4 technology. Quantizing the weight precision of the large base model to 4 bits reduces memory usage. After initial quantization, to further reduce memory overhead, QLoRA uses a double quantization strategy to quantize the first quantization result again, and employs a paging optimizer to dynamically load and unload parameters during training to avoid memory overflow. Throughout the fine-tuning process, the original model parameters remain unchanged, and the low-rank matrix parameters are updated through backpropagation. After positive example fine-tuning, the model is again fine-tuned using negative examples. The knowledge learned from the positive examples is extracted into the large model, and QLoRA technology is still used for negative example fine-tuning, updating the previous QLoRA parameters to serve as initial parameters for the second round of preference learning.

[0155] S500: Fine-tuning and optimization of entity type preferences.

[0156] In this embodiment, the preference for entity types is fine-tuned and optimized, and the specific process is as follows:

[0157] S501: To address the issue of incorrect entity type identification, a third fine-tuning strategy is designed, introducing DPO preference optimization to train the model to output the correct entity type;

[0158] S502: By collecting data on errors in entity type reasoning of the model after the second fine-tuning, a preference fine-tuning dataset is constructed to optimize the model in a targeted manner.

[0159] In the above process, to address the issue of incorrect entity type judgment, a third fine-tuning strategy is designed, which introduces DPO preference optimization to train the model to output the correct entity type. By collecting data on the errors in entity type reasoning that occurred in the model after the second fine-tuning, a preference fine-tuning dataset is constructed to optimize the model in a targeted manner.

[0160] Furthermore, a comparative experiment was conducted between the un-fine-tuned model and the fine-tuned model to evaluate the information extraction capability of the fine-tuned model. Given that the technical terms used in this paper may differ across standards, the evaluation method focuses on the essence of the information rather than precise character matching. Precision, recall, and F1 score were selected as model evaluation metrics to analyze the impact of QLoRA-based fine-tuning on the extraction of key information from large models in the field of energetic materials safety standards. The formulas are as follows:

[0161]

[0162]

[0163]

[0164] Precision (P) measures the proportion of true positives in the model's predictions, while recall (R) represents the proportion of positive samples identified by the model. The F1 score is the average of these two values, and the model is evaluated by considering both precision and recall. We selected 80% of the samples as the training set and 20% as the model's validation set for cross-validation. The untuned base model Qwen2.5-7B had a precision of 0.557, a recall of 0.655, and an F1 score of 0.602, reflecting that even the untuned Qwen2.5-7B model demonstrated a certain level of knowledge understanding and triple extraction capability in the triple extraction task. The finely tuned SsemLM model showed improved performance. Its precision was 0.875, indicating that 87.5% of the predicted positives were correct in the triple prediction task, demonstrating that SsemLM outperformed the base model Qwen2.5-7B in handling more complex triples. SsemLM achieved a recall of 0.818, indicating an improvement in the model's ability to find correct triples. SsemLM also achieved an F1 score of 0.845, an improvement over the base model Qwen2.5-7B.

[0165] Please see Figure 13 The present invention also provides a domain knowledge extraction system for special material safety standards, comprising a data collection module 601, a classification module 602, an ontology design module 603, a first data knowledge extraction module 604, a second data knowledge extraction module 605, and a preference fine-tuning module 606; wherein:

[0166] The data collection module 601 is used to collect safety standard data for special materials and preprocess the data;

[0167] The classification module 602 is used to classify the standard clauses;

[0168] The ontology design module 603 is used for ontology design;

[0169] The first data knowledge extraction module 604 is used for extracting knowledge from structured and semi-structured data;

[0170] The second data knowledge extraction module 605 is used for extracting knowledge from unstructured data;

[0171] The preference fine-tuning module 606 is used for fine-tuning and optimizing the preferences for entity types.

[0172] In this embodiment, the data collection module 601 collects special material safety standard data and preprocesses the data; the classification module 602 classifies the standard clauses; the ontology design module 603 performs ontology design; the first data knowledge extraction module 604 extracts knowledge from structured and semi-structured data; the second data knowledge extraction module 605 extracts knowledge from unstructured data; and the preference fine-tuning module 606 fine-tunes and optimizes the preferences for entity types.

[0173] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0174] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.< / description> < / errortype> < / errortype> < / description> < / text> < / description> < / text>

Claims

1. A method for domain knowledge extraction for special material safety standards, characterized in that, Comprising the following steps: Collecting special material safety standard data, pre-processing the data, and classifying the standard clauses by manual; collecting documents in the national standard and industry specification standard of special material safety standard system, covering term definition, classification label, sensitivity test, production process, storage and transportation, and application safety process, a total of 130 documents; Carrying out ontology design; Knowledge extraction of structured and semi-structured data; Extracting semi-structured data, using regular expressions, dictionary matching, and rule-based methods to extract the name, release date, standard number, drafting unit, and drafter entity and its attribute information in the standard; Extracting structured data, converting the data into the form of standard triples according to the entity, attribute, and relationship in the table; Knowledge extraction of unstructured data; Preference fine-tuning and optimization of entity type; In the step of knowledge extraction of unstructured data: According to the prompt word engineering combined with the large language model, the method of artificial inspection is used to build a customized fine-tuning data set Ssem-Instruct for special material safety standards, and the preliminary extracted data is screened according to expert opinions to form a positive example fine-tuning data set, and the unqualified data is improved to build a negative example data set; Using Qwen2.5-7B as the base model, based on QLoRA fine-tuning technology combined with positive and negative example fine-tuning data set to optimize the model, after positive example fine-tuning, negative example fine-tuning, updating the parameters after positive example fine-tuning, entity type preference fine-tuning of the model after negative example fine-tuning; using the trainable low-rank matrix to fine-tune the model; And in the test stage, collect the extracted error data into the secondary optimization improvement data set, and fine-tune again; Build a triple extraction large model SsemLM for the field of special material safety standards, and perform triple extraction task according to the model combined with knowledge extraction optimization process to form the knowledge graph of special material safety standards field.

2. The safety standards for specialty materials domain knowledge extraction method of claim 1, wherein, In the step of collecting special material safety standard data, pre-processing the data, and classifying the standard clauses by manual: Using OCR technology to process PDF files, extracting text content and table content from documents; Calibrating the document content, dividing the document into structured, unstructured and semi-structured areas, and extracting the content of each area; Removing pictures, headers, footers and watermarks, removing extra spaces and line breaks in text, correcting recognition errors, forming standard knowledge items, and filling, deleting and correcting table data, handling missing values and outliers in data, forming structured data, semi-structured data and unstructured data; Manual classification of documents, dividing standard clauses into five categories, including comprehensive safety management, design and construction safety, production and operation safety, storage and transportation safety, and detection and emergency disposal, and building a systematic safety standard resource library.

3. The safety standards for specialty materials domain knowledge extraction method of claim 2, wherein, In the manual classification of documents, the standard clauses are divided into five categories, including comprehensive safety management, design and construction safety, production and operation safety, storage and transportation safety, and detection and emergency disposal. The steps to build a systematic safety standard repository are as follows: The comprehensive safety management category involves the establishment of a safety system, management responsibilities, safety assessment, and overall requirements for safety education. The design and construction safety category focuses on the safety design principles of plant layout, building explosion prevention, and explosion relief facilities. The production and operation safety category covers the process flow, equipment requirements, personnel operation specifications, and work protection equipment for energetic materials in the manufacturing process. The storage and transportation safety category specifies the condition restrictions, facility configuration, and warehouse access process management for materials during storage, handling, and transportation. The detection and emergency disposal category specifies product performance testing, hazardous waste disposal, and accident response mechanisms to ensure rapid response and risk control in abnormal situations.

4. The safety standards for specialty materials domain knowledge extraction method of claim 1, wherein, In the ontology design step: Seven entity types are constructed based on requirements, including energetic materials, standard bibliographic information, professional terms, laws and regulations, technical standards, safety requirements, and test methods. Three main relationship word types are designed, including standard metadata relationships, reference and dependency relationships, and technical clause relationships.

5. The safety standards for specialty materials domain knowledge extraction method of claim 1, wherein, In the preference fine-tuning and optimization of entity types step: To address entity type judgment errors, a third fine-tuning strategy is designed, introducing DPO preference optimization to train the model to output correct entity types. By collecting data with errors in entity type reasoning after the second fine-tuning, a preference fine-tuning dataset is constructed to optimize the model.

6. A domain knowledge extraction system for special material safety standards, applied to the domain knowledge extraction method for special material safety standards according to claim 1, characterized in that, It includes data collection module, classification module, ontology design module, first data knowledge extraction module, second data knowledge extraction and preference fine-tuning module; among them: The data collection module is used to collect special material safety standard data and preprocess the data. The classification module is used to classify standard clauses. The ontology design module is used for ontology design. The first data knowledge extraction module is used for structured and semi-structured data knowledge extraction. The second data knowledge extraction module is used for unstructured data knowledge extraction. The preference fine-tuning module is used for preference fine-tuning and optimization of entity types.

Images