Entity identification and relation determination method and related equipment in technical supervision text

Abstract

The invention provides an entity identification and relation determination method in a technical supervision text and related equipment. The method comprises the following steps: acquiring the technical supervision text; inputting the technical supervision text into a pre-trained entity model, performing entity extraction on the technical supervision text through the entity model, and determining and outputting a plurality of entities in the technical supervision text and type information corresponding to each entity; inputting the plurality of entities, the type information corresponding to each entity and the technical supervision text into a pre-trained relation model, determining an initial relation sequence corresponding to each entity through the relation model, and optimizing each initial relation sequence by using a self-attention mechanism; and based on each optimized initial relation sequence, determining and outputting a target relation sequence corresponding to each entity through the relation model, thereby solving the technical problem that the relation between the entities in the technical supervision text is inaccurate in the prior art.

Description

Methods and related equipment for entity identification and relationship determination in technical supervision texts. Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and related equipment for identifying entities and determining relationships in technical supervision text. Background Technology

[0002] Technical supervision documents are a core component in ensuring standardized operations and improving intelligent management across industries. Their efficient implementation relies on the accurate identification and extraction of key entities (such as fault types, equipment parameters, and operating procedures) from unstructured text. Entity recognition and relationship extraction, as fundamental technologies for knowledge mining and knowledge graph construction in technical supervision, can extract structured entity information from complex text, providing data support for risk warning, quality control, and decision support. Insufficient entity recognition accuracy will lead to distorted technical supervision data and process failures, directly impacting industry safety and production efficiency.

[0003] The current development of entity semantic recognition technology, whether based on traditional deep learning models or emerging methods combining large language models with prompting engineering or external knowledge bases (such as RAG), faces unresolved bottlenecks. The core weakness of traditional deep learning models lies in their generalization ability and deep understanding of complex structures; while the large-model approach is constrained by the credibility deficiency of "illusion" and the dual challenges of high resource consumption and inference latency. Improving the model's transferability under few-sample conditions, enhancing its understanding of complex text structures and logical constraints, effectively eliminating the illusion of large models, overcoming the bottlenecks of computational resources and efficiency, and achieving low-cost and efficient integration of external knowledge have become the core challenges urgently needing breakthroughs in this field. These intertwined problems collectively constitute a major obstacle to achieving accurate entity relationship recognition technology. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a method and related equipment for identifying entities and determining relationships in technical supervision texts, so as to overcome all or part of the deficiencies in the prior art.

[0005] To achieve the above objectives, this application provides a method for identifying entities and determining relationships in technical supervision text, comprising: acquiring technical supervision text; inputting the technical supervision text into a pre-trained entity model, extracting entities from the technical supervision text using the entity model, identifying and outputting multiple entities in the technical supervision text and type information corresponding to each entity; inputting the multiple entities, type information corresponding to each entity, and the technical supervision text into a pre-trained relation model, determining an initial relation sequence corresponding to each entity using the relation model, and optimizing each initial relation sequence using a self-attention mechanism; and based on each optimized initial relation sequence, determining and outputting a target relation sequence corresponding to each entity using the relation model.

[0006] Optionally, determining the initial relation sequence corresponding to each entity through the relation model and optimizing each initial relation sequence using a self-attention mechanism includes: using the bidirectional encoder layer in the relation model to encode the multiple entities, the type information corresponding to each entity, and the technical supervision text respectively, to obtain multiple entity codes, the type information code corresponding to each entity, and the technical supervision text code; using the bidirectional gated recurrent unit layer in the relation model to determine the initial relation sequence corresponding to each entity code based on the entity code, the type information code corresponding to the entity code, and the technical supervision text code; and using the self-attention mechanism layer in the relation model to reallocate relation weights to the initial relation sequence corresponding to each entity code.

[0007] Optionally, the step of reallocating relation weights to the initial relation sequence corresponding to each entity encoding using the self-attention mechanism layer in the relation model includes: reallocating relation weights to the initial relation sequence corresponding to the entity encoding using the following formula: ,in, This is the optimized initial relation sequence. The first in the initial relation sequence corresponding to the entity encoding Hidden values ​​of individual relationships, The first in the initial relation sequence corresponding to the entity encoding The weight parameters corresponding to each sub-relationship The first in the initial relation sequence corresponding to the entity encoding Individual relationships, The number of all sub-relations in the initial relation sequence corresponding to the entity encoding.

[0008] Optionally, the step of extracting entities from the technical supervision text using the entity model, and determining and outputting multiple entities in the technical supervision text and the type information corresponding to each entity, includes: encoding the technical supervision text using the encoding layer in the entity model to obtain multiple concatenated codes; extracting features from each concatenated code using the bidirectional long short-term memory layer in the entity model to obtain the encoding features corresponding to each concatenated code; and determining and outputting at least one entity corresponding to each encoding feature and the type information corresponding to each entity using the output layer in the entity model.

[0009] Optionally, the step of encoding the technical supervision text using the encoding layer in the entity model to obtain multiple concatenated codes includes: encoding the technical supervision text by characters to obtain multiple character codes, and encoding the technical supervision text by words to obtain multiple word codes; determining multiple type codes corresponding to the multiple character codes and multiple word codes based on the multiple character codes, the multiple word codes, and the multiple type codes; determining multiple encoding sequences containing at least one character code, at least one word code, and at least one type code based on the multiple character codes, the multiple word codes, and the multiple type codes; and concatenating all codes in each encoding sequence using an attention mechanism to obtain the concatenated code corresponding to the encoding sequence.

[0010] Optionally, the step of using the output layer in the entity model to determine and output at least one entity corresponding to each encoded feature and type information corresponding to each entity includes: introducing a conditional random field into the output layer in the entity model, using the conditional random field to determine at least one entity corresponding to each encoded feature and type information corresponding to each entity, and outputting at least one entity corresponding to each encoded feature and type information corresponding to each entity.

[0011] Optionally, the training methods for the entity model and the relation model include: acquiring historical technology-supervised text; generating multiple pseudo-entities, type information corresponding to each pseudo-entity, and a relation sequence corresponding to each pseudo-entity based on the historical technology-supervised text using a pre-trained large language model; training the entity model based on the historical technology-supervised text, the multiple pseudo-entities, and the type information corresponding to each pseudo-entity; and training the relation model based on the historical technology-supervised text, the multiple pseudo-entities, the type information corresponding to each pseudo-entity, and the relation sequence corresponding to each pseudo-entity.

[0012] Optionally, before inputting the technical supervision text into a pre-trained entity model, the method includes: performing structured parsing of the technical supervision text.

[0013] Based on the same inventive concept, this application also provides an apparatus for identifying entities and determining relationships in technical supervision text, comprising: an acquisition module configured to acquire technical supervision text; an entity extraction module configured to input the technical supervision text into a pre-trained entity model, extract entities from the technical supervision text through the entity model, determine and output multiple entities in the technical supervision text and type information corresponding to each entity; a determination module configured to input the multiple entities, type information corresponding to each entity, and the technical supervision text into a pre-trained relation model, determine an initial relation sequence corresponding to each entity through the relation model, and optimize each initial relation sequence using a self-attention mechanism; and an output module configured to determine and output a target relation sequence corresponding to each entity based on each optimized initial relation sequence through the relation model.

[0014] Based on the same inventive concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0015] As can be seen from the above, the method and related equipment for entity identification and relationship determination in technical supervision text provided in this application include: The method involves acquiring the technical supervision text; inputting the technical supervision text into a pre-trained entity model; extracting entities from the technical supervision text using the entity model; identifying and outputting multiple entities and their corresponding type information in the technical supervision text; and using an entity model specifically designed for entity extraction to avoid interference from relationship extraction tasks, accurately focusing on boundary detection and type classification, thus ensuring the accuracy of the multiple entities and their corresponding type information output by the entity model. The method also involves inputting the multiple entities, their corresponding type information, and the technical supervision text into a pre-trained relation model; determining an initial relation sequence for each entity using the relation model; and optimizing each initial relation sequence using a self-attention mechanism. Multimodal information fusion and dynamic relation reasoning are used to improve the accuracy of relation extraction. Based on each optimized initial relation sequence, the relation model determines and outputs a target relation sequence for each entity, ensuring the accuracy of the relation extraction results. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 is a flowchart illustrating the method for identifying entities and determining relationships in technical supervision text according to an embodiment of this application; Figure 2 is a schematic diagram of the relation model structure according to an embodiment of this application; Figure 3 is a schematic diagram of the ERNIE masking method according to an embodiment of this application; Figure 4 is a schematic diagram of the GRU neuron structure according to an embodiment of this application; Figure 5 is a schematic diagram of the structure of the bidirectional gated recurrent unit layer according to an embodiment of this application; Figure 6 is a schematic diagram of the model architecture of the entity model according to an embodiment of this application; Figure 7 is a schematic diagram of the structure of the device for identifying entities and determining relationships in technical supervision text according to an embodiment of this application; Figure 8 is a schematic diagram of the hardware structure of the electronic device according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0019] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] As described in the background section, technical supervision-related texts are a core component in ensuring standardized operations and improving intelligent management across industries. Their efficient implementation relies on the accurate identification and extraction of key entities (such as fault types, equipment parameters, and operating procedures) from unstructured text. Entity recognition and relation extraction, as fundamental technologies for knowledge mining and knowledge graph construction in technical supervision, can extract structured entity information from complex texts, providing data support for risk warning, quality control, and decision support. Insufficient entity recognition accuracy will lead to distorted technical supervision data and process failures, directly impacting industry safety and production efficiency. Therefore, improving the adaptability and accuracy of entity recognition and relation extraction technologies across various fields is crucial for enhancing the scientific rigor and reliability of technical supervision.

[0021] Current entity relation extraction methods, primarily named entity recognition and relation extraction techniques, while achieving superficially high performance metrics in specific domains through complex modeling, reveal systemic shortcomings when applied to real-world industrial scenarios. The biggest contradiction lies in the model's generalization ability and its severe data dependency: existing methods overly rely on large-scale, manually labeled data specific to certain domains. This type of data is not only costly to acquire and limited in scale, but also fails to cover the widespread long-tail entity types and diverse linguistic expressions in the real world, resulting in inaccurate outputs from trained models. When models need to be transferred to new domains with significantly different data distributions, their performance often suffers unacceptable and severe degradation, highlighting the serious lack of small-sample adaptive learning mechanisms in current methods. Furthermore, existing solutions have inherent limitations in parsing complex text structures, lacking robustness in recognizing complex entity expressions and cross-reference scenarios. Cross-modal fusion methods are inefficient when handling mixed-format data. Crucially, they generally lack the ability to deeply model the logical constraints inherent in technical standards and cross-document reference chains, resulting in the inability to fully extract core semantic information and further contributing to inaccurate outputs from trained models.

[0022] Meanwhile, entity relation extraction methods based on prompting engineering and Large Language Models (LLM), while demonstrating flexibility and potential in zero-shot or few-shot learning, also present new and serious challenges. The most prominent problem is their inherent "illusion" phenomenon: when identifying relationships between entities, the model generates or introduces information not explicitly mentioned in the text, or even completely fabricated information, based on its internal knowledge. This significantly threatens the credibility of the results and their practical application value. Furthermore, deploying and applying such large language models places extremely high demands on server hardware, facing significant computational resource consumption and massive GPU memory usage, resulting in high deployment costs and noticeable inference latency. These issues collectively constitute the key bottleneck for their large-scale deployment in resource-constrained environments or real-time-critical application scenarios.

[0023] While techniques like Retrieval Augmentation (RAG), which introduce external knowledge bases, have been used to attempt to alleviate the illusion problem of large models, this introduces new complexities. Building and maintaining a large-scale, high-quality, and comprehensive domain knowledge base is itself a massive, time-consuming, and costly undertaking. Even more challenging are the numerous difficulties RAG mechanisms face in practical applications: the accuracy of the retrieval process is difficult to guarantee (potentially recalling irrelevant or erroneous fragments), the retrieved knowledge is incomplete, and there are difficulties in efficiently and conflict-free integrating the retrieved external knowledge into the model generation process. This is especially true when dealing with complex relationships or integrating contextual information from multiple distributed heterogeneous data sources, which can easily lead to information conflicts and semantic integration obstacles. This often increases the overall system complexity and incurs additional resource overhead.

[0024] In summary, the current development of entity semantic recognition technology, whether based on traditional deep learning models or emerging methods combining large language models with prompting engineering or external knowledge bases, faces unresolved bottlenecks. The core shortcomings of traditional methods lie in generalization ability and a lack of deep understanding of complex structures; while the large model approach is constrained by the credibility deficiency of "illusion" and the dual challenges of high resource consumption and inference latency. Improving the model's transferability under few-sample conditions, enhancing the understanding of complex text structures and logical constraints, effectively eliminating the illusion of large models, overcoming the bottlenecks of computational resources and efficiency, and achieving low-cost and efficient integration of external knowledge have become the core challenges that urgently need to be overcome in this field. These intertwined problems collectively constitute a major obstacle to achieving efficient (low-cost, low-latency), accurate (reliable, anti-illusion), and lightweight (low-resource consumption) entity relationship recognition technology.

[0025] In summary, existing methods still have significant limitations when addressing the needs of industry-specific technical supervision: First, they lack domain adaptability. General models struggle to handle the specificities of professional fields, such as the scarcity of labeled data, the abundance of technical terminology, and the heterogeneity of multi-source data in certain industries (coal mining, medicine, equipment manufacturing, etc.), which limits the accuracy of entity recognition. Second, they have weak entity feature capture capabilities. When faced with nested entities, long-span / long-text entities, and semantically ambiguous scenarios, existing models often suffer from inaccurate boundary localization and entity type confusion. Third, feature and modal fusion is insufficient. Single feature representations are insufficient to fully capture complex semantics, and challenges exist in multimodal scenarios (such as text-image entities) such as image-text misalignment and inefficient cross-modal feature fusion. Fourth, there is a contradiction between low resources and generalization ability. Traditional machine learning methods rely on manual feature engineering, while deep learning models are prone to overfitting in small-sample scenarios and have weak cross-domain generalization ability. These shortcomings make it difficult to accurately determine entities and relationships in text, thus restricting the deep application of entity recognition technology in technical supervision. A breakthrough is urgently needed by combining domain characteristics with technological innovation.

[0026] In view of this, this application proposes a method for identifying entities and determining relationships in technical supervision text. Referring to Figure 1, the method includes the following steps: Step 101, obtaining technical supervision text.

[0027] In this step, each industry has its own technical supervision texts, which serve to ensure the standardized operation of that industry. Identifying the entities and relationships within these texts is a fundamental technique for knowledge mining and knowledge graph construction in technical supervision, providing accurate data support for subsequent operations and improving efficiency. Therefore, accurate identification of entities and relationships within technical supervision texts is crucial. The first step is to obtain the technical supervision texts to be identified.

[0028] Step 102: Input the technical supervision text into a pre-trained entity model, extract entities from the technical supervision text using the entity model, and determine and output multiple entities in the technical supervision text and the type information corresponding to each entity.

[0029] In this step, entity recognition and relation extraction are tasks of varying difficulty. Entity recognition requires boundary detection and type classification, belonging to shallow semantic understanding, while relation extraction requires understanding deep semantic relationships between entities, involving cross-entity contextual reasoning and domain knowledge, belonging to deep semantic understanding. This phased processing decouples the shallow task (entity recognition) from the deep task (relation extraction), avoiding a single model handling sub-tasks of different difficulties simultaneously, reducing training complexity, and ensuring the accuracy of the model's output. First, the entities in the technical supervision text are identified. The technical supervision text is input into a pre-trained entity model, for example, an entity model using the GLiNER framework (Generalist and Lightweight Model for Named Entity Recognition) combined with a pre-trained language model. The entity model extracts entities from the technical supervision text, identifying and outputting multiple entities and their corresponding type information. The type information of an entity reflects the relationship between that entity and other entities. Entity models specifically designed for entity extraction avoid interference from relation extraction tasks, accurately focus on boundary detection and type classification, and ensure the accuracy of multiple entities and their corresponding type information in the technical supervision text output by the entity model.

[0030] To address the issue that the relationship types of entity pairs in standard text are often hidden within the text's semantic features, and that the relationship types are relatively definite, the entity relationship extraction task is transformed into a multi-classification task. The predetermined relationship types are embedded into entity labels, and sequence labeling is used to complete the entity relationship extraction task. Accurately extracting domain-specific key entity information from technically supervised text is essentially an entity model that combines deep contextual understanding with standard domain knowledge through adaptive sequence labeling. For example, this entity model adopts a BERT (Bidirectional Encoder Representations from Transformers) - BiLSTM (Bidirectional Long Short-Term Memory) - CRF (Conditional Random Fields) architecture: pre-trained BERT deeply understands the semantics of complex technical terms such as "weather resistance test"; BiLSTM captures the contextual dependencies of entities in the text sequence; and the CRF layer ensures the global consistency of the entity label sequence. This entity model is specifically trained to identify core entity types in technical standards, such as standard numbers with fixed formats but variations (e.g., GB 9706.1-2020), specific technical parameters (e.g., "temperature rise limit 75K"), specified test methods (e.g., "salt spray test 500h"), and important safety clauses (e.g., "prevention of accidental start-up"). When facing technical supervision tasks, this entity model specifically addresses challenges that general-purpose NER (Named Entity Recognition) models struggle with, such as highly structured references (complex standard numbers), numerous industry abbreviations (e.g., "UL", "CE" markings), and ambiguous situations requiring contextual judgment of entity boundaries (e.g., distinguishing whether "10mm" is a dimensional requirement or a measurement error). By pre-training with synthetic training data and fine-tuning with real standards, this entity model can accurately identify entities and support clause-level tracing (e.g., the composite entity "EN 61000-3-2:2019 Chapter 7"), significantly improving the accuracy and recall of information extraction. This entity model relies heavily on chapter location information provided by page layout analysis technology to ensure that only target entity types in specific chapters (such as "security requirements") are extracted. The identified entities and entity type information are used as the basic nodes of the downstream relationship extraction model to support the construction of the subsequent knowledge graph.

[0031] Step 103: Input the multiple entities, the type information corresponding to each entity, and the technical supervision text into the pre-trained relation model, determine the initial relation sequence corresponding to each entity through the relation model, and optimize each initial relation sequence using a self-attention mechanism.

[0032] In this step, after identifying the entities in the technical supervision text, the relationships between these entities are determined. Multiple entities, their corresponding type information, and the technical supervision text are input into a pre-trained relational model, for example, a GLiNER model (a named entity model based on bidirectional Transformers). The relational model processes the input data to determine the initial relational sequence for each entity, and optimizes each initial relational sequence using a self-attention mechanism. The attention mechanism automatically focuses on core trigger words (such as "applicable to"). Entity type information provides strong semantic constraints to the relational model (e.g., "equipment" entities are more likely to participate in "fault-repair" relationships than "cooperation-R&D"), while the original technical supervision text retains its contextual information. The self-attention mechanism, by capturing long-distance dependencies and contextual interactions between entities, can dynamically adjust relation weights (e.g., strengthening the "reduction" relationship between "ultra-high voltage transformer" and "line loss," and weakening irrelevant relationships), thereby solving the problem of misjudgment that may be caused by insufficient information or noise. Through multimodal information fusion and dynamic relational reasoning, the accuracy of relation extraction is improved.

[0033] The relational model aims to reveal the complex regulatory logic and inter-entity constraints hidden within technical oversight texts. Its core is a hybrid model that integrates deep semantic understanding with explicit industry rule guidance. Technically, it typically includes a semantic-based module for learning the latent semantic relationships between entity pairs (e.g., "insulating materials - dielectric strength testing") within the textual context, combined with a rule module. For example, the semantic module could be SPAN-BERT, a pre-trained language model improved upon BERT. By expanding the masking granularity (using continuous segment masks) and introducing the Span BoundaryObjective (SBO) training objective, SPAN-BERT significantly enhances its ability to understand long texts, particularly excelling in tasks such as question answering and referential resolution. This module directly incorporates industry expert knowledge and technical regulatory logic (e.g., "medical electrical equipment must include emergency power requirements" or "if the material is used for the enclosure, its flame retardancy rating must meet UL94 V-0") to jointly determine specific relationships between entities. It primarily extracts crucial relationship types from technical standards: constraint relationships describing conditional constraints between parameters (e.g., "operating temperature ≤ 40°C"), reference relationships reflecting inter-standard dependencies (e.g., "this standard references GB / T 2423.1-2008"), and test associations linking the testing process to the verification object (e.g., "method A is used to verify performance B"). In technical supervision applications, this relationship model effectively addresses challenges that traditional pure deep learning models struggle with—the resolution of low-frequency, complex, but crucial logical chains (e.g., "the equipment should stop operating when the ambient humidity > 80%) and the integration of fragmented information across chapters (e.g., supplementing the main text clauses in the appendix). Its advantage in introducing rule and logical term (should / should not, unless) resolution lies in reducing the requirement for massive amounts of labeled data, significantly improving the model's interpretability, and making it more aligned with the needs of highly regulated fields for transparency and rule compliance. This relational model directly receives entity and type information extracted from the entity model along with technical supervision text as input. It also works in conjunction with layout analysis technology to accurately locate the effective context of the extracted relations (such as analyzing the constraint relations of test parameters only within the "Test Conditions" section). Finally, it outputs accurate relational triples, which is a key step in realizing automated compliance auditing and building a complete technical standard knowledge graph.

[0034] For example, by introducing positional labels to address the problem of overlapping relationships in text, a weight-optimized relational model based on an attention mechanism (ERNIE-AWE) is proposed. The relational model structure is shown in Figure 2. The relational model architecture consists of four parts: a bidirectional encoder layer based on ERNIE, a bidirectional gated recurrent unit layer based on BiGRU, a self-attention mechanism layer based on self-attention, and an output layer based on CRF. Among them, the bidirectional gated recurrent unit layer and the self-attention mechanism layer can be regarded as the hidden layers of the relational model. By using the pre-trained model ERNIE, the input text is transformed into sentence vectors to capture the semantic information in the text. The output sentence vectors are input into the bidirectional gated recurrent unit layer, which captures long-distance dependencies in the sequence through the bidirectional gated recurrent unit (BiGRU) to extract deeper semantic features. Then, the self-attention mechanism (Attention(char)) is used to give higher weights to the entities in the triples to improve the accuracy of the entity relation extraction task. Finally, the CRF in the output layer completes the sequence labeling.

[0035] Step 104: Based on each optimized initial relation sequence, determine and output the target relation sequence corresponding to each entity through the relation model.

[0036] In this step, based on each optimized initial relation sequence, the target relation sequence corresponding to each entity is determined and output through the relation model. The relation model has a multi-layered structure, with the last layer being the output layer. A Conditional Random Field (CRF) is introduced into the output layer of the relation model, and the CRF is used to determine the target relation sequence corresponding to each entity. In the final stage of relation extraction, based on the initial relation sequence optimized by the self-attention mechanism, by introducing the CRF into the output layer of the relation model, the global dependencies and label constraints of the relationships between entities can be explicitly modeled, thereby improving the accuracy and consistency of the target relation sequence.

[0037] By combining the CRF global constraint relationship label sequence in the output layer, the problems of relationship overlap and semantic offset are effectively solved, ensuring high reliability of relationship prediction. Multiple relationship categories are embedded into BIO (Begin, Inside, Outside) labels, completing the multi-classification task of entity relationship extraction through sequence labeling. Conditional Random Fields can comprehensively consider the dependencies between labels, predict the label category at the current position by learning the correct label information, and avoid the occurrence of invalid sequence labels by adding constraints when outputting labels, thus outputting the globally optimal solution.

[0038] The Conditional Random Field algorithm flow is as follows: For each optimized initial relation sequence After training the other layers in the relational model, the predicted label sequence corresponding to the technical supervision text was obtained. The scoring function for the conditional random field in the output layer is: Formula 1, where, The score of the initial relation sequence. Representative sub-relationship Mapping to tags The probability of observation is defined by the observation probability matrix. The output score matrix of the feature extraction module represents the score of each character on all labels in the input technical supervision text; Indicates label arrive The transition probabilities are used to form a non-normalized transition probability matrix. , This represents the total number of sub-relations in the initial relation sequence.

[0039] Specifically, Conditional Random Fields (CRFs) can correct global constraints that local attention mechanisms might overlook by considering the transition probability of adjacent relationship labels (e.g., "release" is more likely to be followed by "product" rather than "time"). This is achieved by avoiding the same entity being labeled with both "R&D" and "production" in conflicting relationships. Simultaneously, its structured prediction capabilities can integrate contextual information, accurately handling complex relationships (e.g., chain relationships like "equipment-fault-cause-repair") in technically supervised texts. By combining the optimized initial relationship sequence with the global constraints of CRFs, it can output logically coherent target relationship sequences that conform to domain knowledge and have a lower error rate, ensuring the accuracy of relationship extraction results.

[0040] This application employs lightweight components (GLiNER, ERNIE), significantly reducing computational requirements and improving inference speed compared to large models with hundreds of billions of data points (such as LLaMA-2-70B). A deeply integrated dictionary verification mechanism for the power sector and CRF structured constraints effectively prevent model illusions (such as fabricated standard clauses), ensuring high reliability of the output results. Structured outputs (JSON / XMind / relational triples) seamlessly integrate with knowledge graph construction and decision-making systems, providing efficient, low-cost, and highly robust core support for intelligent applications in power technology supervision (such as fault diagnosis and compliance review). Its core advantages are high accuracy (F1>91%), low resource consumption (small sample training, 70% reduction in annotation costs), strong anti-interference capabilities (extremely low illusion rate), and ease of deployment.

[0041] The above scheme obtains technical supervision text. The technical supervision text is input into a pre-trained entity model, which extracts entities from the text, identifying and outputting multiple entities and their corresponding type information. This dedicated entity model avoids interference from relation extraction tasks, accurately focusing on boundary detection and type classification, ensuring the accuracy of the entities and their corresponding type information in the technical supervision text output by the entity model. The multiple entities, their corresponding type information, and the technical supervision text are then input into a pre-trained relation model. This model determines an initial relation sequence for each entity and optimizes it using a self-attention mechanism. Multimodal information fusion and dynamic relation reasoning improve the accuracy of relation extraction. Based on the optimized initial relation sequence, the relation model determines and outputs a target relation sequence for each entity, ensuring the accuracy of the relation extraction results.

[0042] In some embodiments, determining the initial relation sequence corresponding to each entity through the relation model and optimizing each initial relation sequence using a self-attention mechanism includes: using a bidirectional encoder layer in the relation model to encode the plurality of entities, the type information corresponding to each entity, and the technical supervision text respectively, to obtain multiple entity codes, type information codes, and technical supervision text codes corresponding to each entity; using a bidirectional gated recurrent unit layer in the relation model, for each entity code, based on the entity code, the type information code corresponding to the entity code, and the technical supervision text code, to determine the initial relation sequence corresponding to the entity code; and using a self-attention mechanism layer in the relation model to reallocate relation weights to the initial relation sequence corresponding to each entity code. In this embodiment, the entities, type information, and text are first independently encoded through a bidirectional encoder layer. The bidirectional encoder structure in the bidirectional encoder layer can simultaneously capture the semantics of the context, converting the entity name, type label, and original text into a high-dimensional vector representation. This step not only preserves the semantic features of the entity itself but also injects strong domain constraints through type encoding. Subsequently, the Bidirectional Gated Recurrent Unit (BiGRU) layer, with entity encoding at its core, integrates its corresponding type information encoding and technically supervised text encoding. Through a gating mechanism, it dynamically filters key information to generate an initial relation sequence. The BiGRU layer can model the temporal dependencies between entities and text, while simultaneously using type information to narrow the relation search space. Finally, the self-attention mechanism layer globally reassigns weights to the initial relation sequence. By calculating the similarity of relation vectors between entities, it strengthens relationships strongly correlated with the current entity and suppresses noisy relationships (such as erroneous associations caused by textual ambiguity), thereby outputting a more accurate target relation sequence.

[0043] For example, the bidirectional encoder structure is the ERNIE (Enhanced Representation through-Knowledge Integration) structure. ERNIE is a bidirectional encoder model based on the Transformer architecture, pre-trained using a large-scale Chinese corpus and encyclopedic knowledge graph. ERNIE differs from the BERT pre-trained model. BERT focuses on processing raw semantic signals during semantic representation, making it difficult to capture the complete semantic information of larger semantic units. ERNIE, based on the characteristic of Chinese using words as the main semantic representation, introduces three levels of masking strategies. By masking semantic units such as entities and phrases, the model can learn and express complete Chinese semantics more deeply, compensating for BERT's shortcomings in Chinese semantic understanding. Technically supervised text contains a large number of domain-specific semantic units. To obtain feature vectors with richer semantic information, the ERNIE pre-trained model is used for character-level encoding in the relational model.

[0044] ERNIE performs tag embedding, fragment embedding, and position embedding on the input data during word embedding. It uses entity masking, phrase masking, and character masking to learn more refined knowledge features such as entity attributes, thus strengthening the information representation of semantic units in Chinese text. Its masking method is shown in Figure 3. ERNIE converts the input data... Convert to sentence vectors , Indicates the first in the input text A single word, Indicates the text sequence number The character vector corresponding to each character, for example, a character vector with a dimension of 768, that is... .

[0045] To capture long-distance dependencies in standard text statements and address the issues of LSTM (Long Short-Term Memory) networks lacking some local semantic information and having slow computation speed, a variant of LSTM, GRU (Gated Recurrent Unit), is used in the bidirectional gated recurrent unit layer to determine the initial relation sequence corresponding to the entity encoding. The GRU neuron structure is shown in Figure 4: the GRU neuron structure has two gates: a reset gate... and the update gate The reset gate combines the current input with the past state, while the update gate determines whether the current input updates the past state. The GRU neuron structure effectively handles the storage, reset, and update of contextual information through these two gating units, solving the problem of long-distance dependencies. The calculation process is as follows: Formula 2 Formula 3 Formula 4 Formula 5, where, This represents the input word embedding vector. This indicates that the door is being reset at time step. state, To update the door in time step state, In the candidate hidden state, For time step The hidden layer state, Let's define the first weight matrix for the reset gate. To reset the second weight matrix of the gate, To update the first weight matrix corresponding to the gate, To update the weight matrix corresponding to the gate, This is the first weight matrix corresponding to the hidden layer. This is the second weight matrix corresponding to the hidden layer; This is the first bias vector. This is the second bias vector. The third bias vector is ⊙, which represents the Hadamard product of matrices. This represents the hidden layer state at time step t-1.

[0046] The expressions for σ (sigmoid function) and tanh (hyperbolic tangent function) are as follows: Formula 6, where, It is the natural logarithm. It is the natural logarithm function.

[0047] Considering that the unidirectional propagation characteristic of the GRU structure cannot fully utilize contextual information, a BiGRU structure is introduced, which combines forward GRU and backward GRU to obtain global features of the entire input sequence and improve the model's entity relation extraction capability. The structure of the bidirectional gated recurrent unit layer is shown in Figure 5.

[0048] Based on the GRU neuron computation flow described above, the hidden layer outputs during forward and backward propagation are calculated respectively, yielding the forward hidden layer output at time t. Backward hidden layer output The output results at each time step are concatenated into vectors to obtain the initial relation sequence corresponding to the entity encoding of the hidden layer output of BiGRU. : Formula 7, where, , ⊕ represents the vector concatenation operation.

[0049] In entity relation extraction tasks, different words in the input text have varying impacts on relation classification. The entity attributes and semantic information of the head and tail entities in triples are strongly correlated with relation classification. Therefore, the relation model further extracts text features by assigning differentiated probability weights to different word vectors, highlighting the influence of key text information on relation extraction results and improving the extraction performance. To emphasize the impact of entity semantics on relation label classification, a self-attention mechanism layer is introduced after the bidirectional gated recurrent unit layer.

[0050] In some embodiments, the step of reallocating relation weights to the initial relation sequence corresponding to each entity encoding using the self-attention mechanism layer in the relation model includes: reallocating relation weights to the initial relation sequence corresponding to the entity encoding using the following formula: ,in, This is the optimized initial relation sequence. The first in the initial relation sequence corresponding to the entity encoding Hidden values ​​of individual relationships, The first in the initial relation sequence corresponding to the entity encoding The weight parameters corresponding to each sub-relationship The first in the initial relation sequence corresponding to the entity encoding Individual relationships, The number of all sub-relations in the initial relation sequence corresponding to the entity encoding. In this embodiment, the traditional self-attention mechanism calculates weights using the softmax function, which requires exponential operations and normalization of relation scores at all positions, resulting in high computational complexity. This application directly generates weights through a linear combination of trigonometric functions, avoiding exponential operations and global normalization steps, significantly reducing computational complexity. Optimized initial relation sequence The final output is a sequence of relation weights, reflecting the relationships between entities and the strength of those relationships. The hidden feature representation of the t-th sub-relation in the initial relation sequence corresponding to the entity encoding carries the semantic information of the sub-relation. It is a learnable scaling factor used to adjust the magnitude of the trigonometric function's adjustment of relation weights, thereby controlling the sensitivity to periodic changes. It can dynamically adjust the periodic variation of trigonometric functions, enabling the model to adapt to the weight distribution patterns of different entity relationships. The identifier or index of the t-th sub-relation may participate in the calculation through embedding representation to distinguish different relation types. By replacing the exponential operation and global normalization of softmax with a linear combination of trigonometric functions, complex calculation steps are reduced, significantly improving processing speed and reducing resource consumption in long text or large-scale entity relation scenarios.

[0051] The input to the self-attention mechanism layer is the output vector o obtained after activation processing by the bidirectional gated recurrent unit layer. The calculation process for the multiple coefficients is shown in the following formula: Formula 8 Formula Nine Formula 10, where, This is the weight matrix. Here, is the bias vector, and is the attention vector determined through random initialization. The attention weights are obtained using sine and cosine functions. The final output of the word attention layer is the initial relation sequence. ,in: Formula 11, where, for - Natural numbers within the range.

[0052] In some embodiments, the step of extracting entities from the technical supervision text using the entity model, determining and outputting multiple entities in the technical supervision text and the type information corresponding to each entity, includes: encoding the technical supervision text using the encoding layer in the entity model to obtain multiple concatenated codes; extracting features from each concatenated code using the bidirectional long short-term memory layer in the entity model to obtain the encoding features corresponding to each concatenated code; and determining and outputting at least one entity corresponding to each encoding feature and the type information corresponding to each entity using the output layer in the entity model. In this embodiment, using the encoding layer of the entity model to perform multi-granular encoding and concatenation of the technical supervision text can integrate multi-dimensional semantic features such as character-level and word-level, enhancing the representation ability of professional terms. By extracting context-aware encoding features through the bidirectional long short-term memory layer, the semantic information of contextual dependencies in the technical text can be comprehensively captured. Through hierarchical feature extraction and bidirectional context modeling, the problems of complex terminology and strong long-distance dependencies in technical supervision text are effectively solved, significantly improving the accuracy of entity recognition and the consistency of type labeling. Finally, the output layer predicts entities and their types based on feature sequences, ensuring the accuracy of entity boundary recognition and improving the rationality of classification through type constraints.

[0053] Concatenation encoding The input bidirectional long short-term memory network layer further enhances semantic representation, captures long-distance dependencies in text, and effectively utilizes the information of the preceding and following positions of entities in the named entity recognition task to better distinguish entity boundaries and entity types. Bi-LSTM is a special type of LSTM network that uses an independent LSTM layer at both ends of the input sequence to learn both forward and backward sequence information simultaneously.

[0054] Concatenation encoding Inputting the Bi-LSTM for feature extraction yields a global feature vector. , =LSTM( ), Formula 12, =LSTM( Formula Thirteen Formula Fourteen, where, Represents a word vector. This represents the forward output of the hidden layer unit t. This represents the backward output of the hidden layer unit at time t. This is the first weight matrix. This is the second weight matrix. This is the bias vector.

[0055] The core challenge of entity extraction is the detection of complex boundaries in power terminology (e.g., "10kV oil-immersed transformer" needs to be fully identified as a single equipment entity). This application employs a lightweight GLiNER model combined with a power domain entity dictionary to optimize the entity extraction process. The model architecture of the entity model is shown in Figure 6, including an input layer, an encoding layer, a bidirectional long short-term memory (LSTM) layer, and an output layer. The encoding layer extracts the input data by characters and words, providing corresponding positional codes for subsequent processing. The bidirectional LSTM layer takes the structured standard clause text and the power entity dictionary as input. For example, inputting "This standard applies to 10kV and above oil-immersed transformers," the dictionary provides the terminology label for "10kV oil-immersed transformer." Based on the DeBERTa-v3-Large pre-trained model, a bidirectional LSTM layer is introduced to capture the contextual dependencies of long sequences (e.g., the semantic association between "10kV" and "oil-immersed transformer"), alleviating the problem of tokenization splitting complex entities. The bidirectional LSTM layer captures the long-distance semantic dependencies of power complex entities (e.g., "10kV oil-immersed transformer").

[0056] In some embodiments, the process of encoding the technical supervision text using the encoding layer in the entity model to obtain multiple concatenated codes includes: encoding the technical supervision text by characters to obtain multiple character codes; and encoding the technical supervision text by words to obtain multiple word codes; determining multiple type codes corresponding to the multiple character codes and multiple word codes based on the multiple character codes, the multiple word codes, and the multiple type codes; determining multiple encoding sequences containing at least one character code, at least one word code, and at least one type code based on the multiple character codes, the multiple word codes, and the multiple type codes; and concatenating all codes in each encoding sequence using an attention mechanism to obtain the concatenated code corresponding to the encoding sequence. In this embodiment, the technical supervision text is input as a character sequence in the BERT-based encoding layer. The word sequence length is The target corpus is segmented and labeled with part-of-speech tags using a word segmentation tool (such as Python's jieba library) to obtain a word sequence. and part-of-speech type sequence The word sequence length is And satisfy .

[0057] The text sequence was encoded using Chinese mapping vocabularies of BERT-base (basic bidirectional Transformer encoder model) and BERT-WWM (full-word mask bidirectional Transformer encoder model), respectively. Separators [CLS] and [SEP] were added at the beginning and end of sentences to obtain the input vector for the character feature extraction encoder. Input vector of the word feature extraction encoder The part-of-speech (POS) type sequences are encoded using a POS mapping table to obtain the input vector for the POS feature extraction encoder. .

[0058] The above input vector is semantically represented by three encoders: a character feature extraction encoder (Char Encoder) using a pre-trained BERT model. Perform representation and output hidden state vector. Word Encoder: Uses the pre-trained model BERT-WWM to extract word features. Perform representation and output hidden state vector. Part-of-speech feature extraction encoder (PoS Encoder): Represents Z using part-of-speech vectors pre-trained based on the CBOW model, and outputs the hidden state vector of the part-of-speech sequence. .

[0059] The corresponding formula is as follows: Formula 15 Formula Sixteen Formula 17, for example, both BERT and BERT-WWM are pre-trained models consisting of 12 layers of Transformer encoders stacked together. The latter uses whole word masking to pre-train the text, which enhances the understanding of Chinese phrases. Semantic representation is performed using part-of-speech vectors pre-trained based on the CBOW model.

[0060] This attention-based, embedded part-of-speech feature enhancement hidden layer aims to process and fuse the semantic features determined by the encoding layer to understand long-term dependencies and sentence-level semantic information in technically supervised texts, providing better input representations for sequence labeling tasks. For example, the technically supervised text is the safety standard text for underground space rescue equipment. The specific implementation process is as follows: First, the word hidden state vectors represented by the BERT pre-trained model are... Word hidden state vectors represented by the BERT-WWM pre-trained model And part-of-speech hidden state vectors pre-trained based on the CBOW model (Continuous Bag-of-Words, a neural network model for learning word vectors). The system employs an attention mechanism for focused fusion learning. The feature vectors, which integrate character, word, and part-of-speech information, are then input into the BiLSTM network to capture long-range dependencies.

[0061] Because the length of the word sequence obtained after standard text segmentation differs from the length of the character sequence ( By introducing an attention mechanism for feature fusion, the semantic alignment problem can be effectively solved. The hidden layer uses the attention mechanism to learn the hidden relationships among characters, words, and part-of-speech types, and outputs a concatenated code that integrates the above three types of information. The fusion process is divided into two stages: word feature fusion and part-of-speech feature fusion. The specific implementation is as follows: The objective function of the attention mechanism is: Formula 18 Formula 19 Formula 20 Formula 21, namely: Formula 22, where the output dimension and the embedding dimension are the same, both being... d=768. Output fused state feature vector These correspond to character vectors, word vectors, and part-of-speech vectors, respectively. Let be the first parameter mapping matrix of the attention layer. The second parameter mapping matrix of the attention layer. This is the mapping matrix for the third parameter of the attention layer. Similarly, the hidden state vector fused with part-of-speech features... for: Formula 23, where, Let be the first parameter matrix. The second parameter matrix, This is the third parameter matrix.

[0062] Finally, the semantic representation vector based on character features will be... Hidden state vectors with fused word features Hidden state vectors that integrate part-of-speech type features Vector concatenation yields a concatenated encoding that incorporates character encoding, word encoding, and part-of-speech tag encoding into a fused semantic representation. We designed a multi-source feature fusion mechanism (character / word / part-of-speech) to enhance boundary detection capabilities.

[0063] This method employs parallel character and word encoding techniques to extract multi-granular features from supervised text, preserving character-level details while capturing lexical-level context. Type encoding is introduced to inject prior knowledge of entity categories into the encoded sequence, enhancing the model's ability to perceive domain concepts. Finally, an attention mechanism dynamically concatenates different encodings, enabling the model to adaptively focus on key features and suppress redundant information. This multimodal encoding fusion significantly improves the model's ability to parse complex terms and nested entities in technical texts. Combined with type constraints and dynamic attention allocation, it improves both the accuracy of entity boundary recognition and the accuracy of entity type labeling.

[0064] In some embodiments, determining and outputting at least one entity corresponding to each encoded feature and type information corresponding to each entity using the output layer of the entity model includes: introducing a Conditional Random Field (CRF) into the output layer of the entity model, using the CRF to determine at least one entity corresponding to each encoded feature and type information corresponding to each entity, and outputting at least one entity corresponding to each encoded feature and type information corresponding to each entity. In this embodiment, introducing a CRF into the output layer of the entity model aims to optimize the limitations of local prediction through global sequence modeling. Since using softmax independently for label prediction may ignore the correlation constraints between labels, CRF, by modeling the label transition probability matrix, can coordinate the rationality of adjacent labels as a whole (e.g., ensuring that "B-device" is more likely to be followed by "I-device" rather than other types), thereby avoiding label conflicts and logical breaks in complex technical texts. CRF improves the coherence of sequence labeling through global normalization, enhances the accuracy of entity boundary recognition, and at the same time uses transition features to suppress unreasonable type combinations, making the output entity and its type information more consistent with the professional logic of the technical supervision field, significantly improving the reliability of structured information extraction. By using CRF to add label validity constraints, the accuracy and robustness of entity recognition are significantly improved.

[0065] While BERT-based input layers and part-of-speech tagging-enhanced hidden layers can extract effective semantic features from sentences, they cannot fully consider the dependencies between labels, potentially leading to the prediction of unreasonable labels. To address this, Conditional Random Fields (CRFs) are introduced to add constraints to the predicted labels, ensuring their validity and reducing the probability of prediction errors. For example, the next character immediately following a character labeled "B-ELE" cannot be labeled "B-SECI"; the first character of any entity cannot be labeled as "1-XXX," and so on.

[0066] The probability of generating a labeled sequence Y given input statement X is calculated based on the scoring function: Formula 24, in which... For the true labeled value, For all possible tag values.

[0067] To optimize the training process and obtain a smaller loss function value, the loss function is converted into its natural logarithm form, yielding the likelihood function of the predicted label sequence: Formula 25, in which, Scoring of the true labeled sequence Given the scores of all possible labeled sequences, the predicted label output sequence with the highest score after decoding is: Formula 26.

[0068] In some embodiments, the training methods for the entity model and the relation model include: acquiring historical technical supervision text; generating multiple pseudo-entities, type information corresponding to each pseudo-entity, and a relation sequence corresponding to each pseudo-entity using a pre-trained large language model based on the historical technical supervision text; training the entity model based on the historical technical supervision text, the multiple pseudo-entities, and the type information corresponding to each pseudo-entity; and training the relation model based on the historical technical supervision text, the multiple pseudo-entities, the type information corresponding to each pseudo-entity, and the relation sequence corresponding to each pseudo-entity. In this embodiment, multiple pseudo-entities, type information corresponding to each pseudo-entity, and relation sequence corresponding to each pseudo-entity are generated using a pre-trained large language model based on the historical technical supervision text. A domain dictionary is generated using the multiple pseudo-entities, the type information corresponding to each pseudo-entity, and the relation sequence corresponding to each pseudo-entity. The entity model and the relation model are trained based on the domain dictionary. This innovative approach integrates human experience with large model technology, using the large model to generate candidate terms and relation samples to assist in constructing a dictionary for the power industry, significantly improving dictionary coverage and semantic accuracy. High-quality training data is synthesized based on domain rules and large model generation capabilities. Combined with strict text normalization (e.g., "10KV" is unified as "10kV") and data cleaning process, the problems of ambiguity of professional terms and noise interference are effectively solved, and strong domain prior knowledge is injected into the model.

[0069] The synthetic training dataset technique addresses the core pain point of scarce high-quality labeled data in the field of technical standards. Utilizing a large language model, it automatically generates a large amount of synthetic data that conforms to the technical context and is labeled with pre-defined entities (such as standard numbers, material codes, and limit parameters) and relationships (such as constraints and test associations) based on the framework and linguistic features of real technical standards. Its core principle is to leverage the text understanding capabilities of the large language model, systematically covering rare entities (such as specific international standard numbers), complex relationships (such as "withstand voltage ≥1500V"), and ambiguous expressions (such as "should not be lower than" vs. "should not be lower than"), given the highly standardized nature and limited language patterns of standard texts. Compared to the high cost of manual annotation, which heavily relies on domain experts, and the difficulty in guaranteeing the accuracy of web crawler data, synthetic data technology can generate large-scale, high-quality training samples on demand, effectively overcoming the long-tail distribution problem and small sample dilemma in model training, and simulating marginal cases that are difficult to cover with real annotations. It provides the necessary data foundation for training technical standard-specific named entity recognition models and relation extraction models. At the same time, this technology can also combine the results of page layout analysis (such as table structure and chapter position information) to optimize the synthesis rules, generate data that is closer to the actual document structure, and improve the generalization ability of the model in practical applications.

[0070] A large number of pseudo-samples are synthesized using a large model to obtain the necessary training samples. Then, named entity and relation dictionaries are extracted using the large model. Finally, these samples are used to train the final entity extraction and relation extraction models, which serve as the final technical supervision and relation extraction models. This approach uses only the large model for generating pseudo-samples and building the dictionary, avoiding the problems caused by frequent use of the large model in subsequent applications. The pre-training strategy enhanced by the large model enables a deep understanding of Chinese semantic units.

[0071] Domain dictionaries are crucial for improving the accuracy of entity recognition and relation extraction. This application constructs an entity dictionary and a relation dictionary for the technical supervision domain. The dictionary construction can be achieved through large-scale model assistance and manual annotation. The dictionary consists of the following two categories: Entity dictionary: Collects four types of entities, including power equipment (e.g., "GIS combined electrical appliances" and "relay protection devices"), standards (e.g., "GB / T7252-2001" and "DL / T 596-2021"), fault types (e.g., "oil chromatography anomaly" and "insulation aging"), and supervision actions (e.g., "oil chromatography test" and "equipment maintenance"), containing more than 1200 terms. The semantic representation of the terms is optimized using word embedding models (e.g., the semantic similarity between "GIS combined electrical appliances" and "gas-insulated switchgear" reaches 0.92); Relation dictionary: Defines three core relations, including "standards apply to equipment", "faults correspond to standard clauses", and "supervision actions are based on standards". Each relation corresponds to several trigger words (e.g., "applies to", "corresponds to", and "based on"). For example, the trigger words for "standards apply to equipment" include "this standard applies to" and "applies to the following equipment".

[0072] A dynamic domain dictionary constraint mechanism is introduced: In the entity recognition stage, industry terminology is encoded into entity type constraint vectors and injected into the model, forcing the model output to conform to domain knowledge specifications; in the relation extraction stage, invalid associations are filtered out through predefined causal / subordination / equivalence relation templates. For example, in power standards, "rated current" can only establish a "parameter attribute" relationship with equipment entities such as "transformer" and "switchgear". Simultaneously, an innovative entity extraction model and relation extraction model are designed, ultimately obtaining the entity relations from the input text.

[0073] The dynamic optimization mechanism is highly flexible. In addition to trigger-driven real-time updates, it can also integrate large language model capabilities: when new industry terms (such as "all-solid-state battery") are added, LLM is invoked to generate definitions and associated entities (cathode material / energy density), which are then automatically injected into the domain dictionary after manual review. This solution still adheres to the core design principle—using structured knowledge templates (entity types / relationship definitions) as the framework and manual verification rules as quality assurance, adapting to different update frequency scenarios. While maintaining the core process unchanged: the language layer can adapt to LLM's multi-language capabilities (such as LLaMA's support for 200+ languages), switching terminology dictionaries through prompts: "parse the text using [Japanese], entity types must conform to the [JIS Industrial Standard] dictionary." The modal layer integrates image OCR and speech-to-text modules to construct a processing flow of "equipment manual image → text parsing → parameter relationship extraction." Even after expansion, it still connects to downstream systems through a unified structured output interface (standard JSON Schema), maintaining the technical solution's compatibility advantages.

[0074] This method is decoupled from deep learning model architecture and is compatible with traditional machine learning, deep learning, and large language models (LLM). For example, when using a large language model, the output format is constrained by designing prompt words: "Extract entities and relations from text [XXX], output {entity: type, relation: [(subject, relation, object)]} in JSON format, and must conform to [domain dictionary] terminology." When using other models, the domain dictionary is converted into entity type constraint vectors and input into the encoder. All types of models can reuse the structured output mechanism and domain knowledge anti-illusion layer of this patent to ensure that the generated results conform to industry standards. It should be noted that the entity extraction and relation recognition method of this invention is universal and can be widely adapted to various structured or semi-structured texts. For example, in medical scenarios, it can parse the contraindications and dosage relationships of ingredients in drug instructions. Although different text types have format differences (such as complex medical terminology), by adjusting the pre-parsing strategy or domain dictionary (such as integrating standard disease codes in medical scenarios), high-quality entity relation recognition can be achieved based on the same core process, verifying the universality of the method in cross-industry texts. This technology is applicable to any scenario requiring entity extraction and relationship recognition. Typical examples include: in drug regulation, analyzing FDA instructions to construct an "ingredient-contraindication" map, or in manufacturing, extracting "test item-tolerance threshold" association chains from ISO process documents. Such scenarios can reuse the technical framework of text parsing → dictionary injection → lightweight model extraction to achieve the structured accumulation of industry knowledge.

[0075] In some embodiments, before inputting the technical supervision text into a pre-trained entity model, the method includes: performing structured parsing of the technical supervision text. In this embodiment, technical supervision standards often exist in unstructured formats such as PDF and Word, containing complex structures such as chapters, clauses, tables, and formulas. In order to effectively extract the technical supervision text, structured parsing is performed on the technical supervision text before inputting it into the entity model. For example, layout analysis technology is used to perform structured parsing of the technical supervision text. For instance, the PP-StructureV3 model combined with OCR technology is used to achieve structured parsing of the standard text: First, the layout elements of the standard text, such as titles, clauses, tables, and formulas, are identified. For example, "5.1 Inspection of Factory-Outgoing Equipment" in GB / T 7252-2001 "Guidelines for Analysis and Judgment of Dissolved Gases in Transformer Oil" is parsed as "Chapter Number: 5.1, Clause Content: Inspection of Factory-Outgoing Equipment", and represented in a structured language (such as JSON). Then, the parameter tables in the standard (such as the "Temperature Rise Limit Table for Transformers of Different Voltage Levels") are processed using a table processing model to extract the table structure and content, transforming them into a structured representation (example: {"Voltage Level": ["10kV", "35kV"], "Temperature Rise Limit": ["60℃", "70℃"]}). Finally, unstructured data (such as images) are transformed into structured data, such as flowcharts being converted into XMind or other formal description languages ​​for further processing of the standard text. For data that cannot be converted into structured data, object detection models such as YOLO will be used to extract the data, which will then be appropriately saved for subsequent processing.

[0076] After obtaining the technical supervision text, the preprocessing flow of the technical supervision standard data (i.e., the technical supervision text) is as follows: layout analysis, conversion to structured data, text normalization, data cleaning, and domain dictionary construction.

[0077] By integrating the PP-StructureV3 (document parsing toolkit) model with OCR (Optical Character Recognition) technology, accurate parsing of complex document formats such as PDF / Word documents for power technology supervision standards is achieved. Unstructured content such as chapters, clauses, tables, and formulas are efficiently converted into computable structured data such as JSON. Specifically for image information such as flowcharts, XMind formal transformation or YOLO object detection is used to extract and save features, significantly improving the usability of industrial standard data and the efficiency of downstream task processing.

[0078] Page layout analysis technology is a core technology based on computer vision and deep learning. Its core principle is to segment and transform the physical layout of technical standard documents (such as PDFs and scanned documents) into structured logical semantic units. It accurately locates regions such as text paragraphs, tables, formulas, and charts in the document through a page layout analysis model, extracts text content in combination with OCR technology, and uses rule templates or learning mechanisms to identify the internal chapter hierarchical structure of the document (such as the "chapter - article - clause" system stipulated by ISO or GB / T standards). This technology has great application value in the field of technical supervision because it can efficiently process various technical standards with highly standardized structures, automatically identify and extract key parts such as the content and location information of "technical requirements", "test methods", and "normative appendices", avoiding the inefficiency and errors of manual processing of unstructured scanned documents. Compared with traditional manual annotation or simple rule scripts, page layout analysis technology can adapt to diverse layout styles of different national and industry standards, accurately handle complex scenarios such as mixed text and graphics, and nested formulas in tables, ensuring the integrity of parameter descriptions, such as accurately associating the "limit value" in a table with the corresponding "test conditions". It is the cornerstone of subsequent processing processes. The structured text blocks and location information output by it provide an accurate and denoised input source for named entity recognition and relationship extraction models, significantly improving the accuracy of subsequent information extraction. By converting the original document into a unified markup language through a page layout analysis model (text / table / formula), it breaks through the dependence of traditional methods on standardized text formats.

[0079] It should be noted that before inputting the technical supervision text into the entity model, data fusion and cleaning are performed on the technically supervised text after structured parsing. Technical supervision data consists of a large amount of heterogeneous format data, and the following steps are required to achieve fusion.

[0080] First, perform text normalization: unify term expressions (such as correcting "transformer" to "transformer", and "10KV" to "10kV"), and remove noise such as redundant spaces and page numbers. Then, clean the data by removing duplicate records, missing values, and handling contradictory data (such as when "transformer voltage level" appears as both "10kV" and "20kV", it is subject to expert judgment); finally, integrate the data after the above fusion and cleaning to obtain the final structured text.

[0081] The above series of processing processes, including the fusion and cleaning of multi - source heterogeneous supervision data, the construction of domain entity and relationship dictionaries, and the design of entity extraction models based on technical supervision rules and technical standard clauses, can be replaced by other effective data pre - processing processes.

[0082] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0083] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0084] Based on the same inventive concept, corresponding to any of the above-described embodiments, this application also provides a device for identifying entities and determining relationships in technical supervision text.

[0085] Referring to Figure 7, the device for identifying entities and determining relationships in the technical supervision text includes: an acquisition module 10, configured to acquire the technical supervision text.

[0086] The entity extraction module 20 is configured to input the technical supervision text into a pre-trained entity model, extract entities from the technical supervision text through the entity model, and determine and output multiple entities in the technical supervision text and the type information corresponding to each entity.

[0087] The determination module 30 is configured to input the plurality of entities, the type information corresponding to each entity, and the technical supervision text into a pre-trained relation model, determine the initial relation sequence corresponding to each entity through the relation model, and optimize each initial relation sequence using a self-attention mechanism.

[0088] The output module 40 is configured to determine and output the target relation sequence corresponding to each entity based on the optimized initial relation sequence through the relation model.

[0089] The aforementioned apparatus is used to acquire technical supervision text. This text is then input into a pre-trained entity model, which extracts entities from the text, identifying and outputting multiple entities and their corresponding type information. This dedicated entity model avoids interference from relation extraction tasks, accurately focusing on boundary detection and type classification, ensuring the accuracy of the entities and their corresponding type information output by the entity model. The multiple entities, their corresponding type information, and the technical supervision text are then input into a pre-trained relation model. This model determines an initial relation sequence for each entity and optimizes it using a self-attention mechanism. Multimodal information fusion and dynamic relation reasoning improve the accuracy of relation extraction. Based on the optimized initial relation sequence, the relation model determines and outputs a target relation sequence for each entity, ensuring the accuracy of the relation extraction results.

[0090] In some embodiments, the determining module 30 is further configured to: utilize the bidirectional encoder layer in the relation model to encode the plurality of entities, the type information corresponding to each entity, and the technical supervision text respectively, to obtain a plurality of entity codes, a type information code corresponding to each entity, and a technical supervision text code; utilize the bidirectional gated recurrent unit layer in the relation model to determine the initial relation sequence corresponding to each entity code based on the entity code, the type information code corresponding to the entity code, and the technical supervision text code; and utilize the self-attention mechanism layer in the relation model to reallocate relation weights to the initial relation sequence corresponding to each entity code.

[0091] In some embodiments, the determining module 30 is further configured to reallocate relation weights to the initial relation sequence corresponding to the entity encoding using the following formula: ,in, This is the optimized initial relation sequence. The first in the initial relation sequence corresponding to the entity encoding Hidden values ​​of individual relationships, The first in the initial relation sequence corresponding to the entity encoding The weight parameters corresponding to each sub-relationship The first in the initial relation sequence corresponding to the entity encoding Individual relationships, The number of all sub-relations in the initial relation sequence corresponding to the entity encoding.

[0092] In some embodiments, the entity extraction module 20 is further configured to encode the technical supervision text using the encoding layer in the entity model to obtain multiple concatenated codes; to extract features from each concatenated code using the bidirectional long short-term memory layer in the entity model to obtain the encoding features corresponding to each concatenated code; and to determine and output at least one entity corresponding to each encoding feature and type information corresponding to each entity using the output layer in the entity model.

[0093] In some embodiments, the entity extraction module 20 is further configured to perform character encoding on the technical supervision text to obtain multiple character codes, and to perform word encoding on the technical supervision text to obtain multiple word codes; based on the multiple character codes and multiple word codes, determine multiple type codes corresponding to the multiple character codes and multiple word codes; based on the multiple character codes, the multiple word codes, and the multiple type codes, determine multiple encoding sequences containing at least one character code, at least one word code, and at least one type code; for each encoding sequence, use an attention mechanism to concatenate all the codes in the encoding sequence to obtain the concatenated code corresponding to the encoding sequence.

[0094] In some embodiments, the entity extraction module 20 is further configured to introduce a conditional random field into the output layer of the entity model, use the conditional random field to determine at least one entity corresponding to each coding feature and type information corresponding to each entity, and output at least one entity corresponding to each coding feature and type information corresponding to each entity.

[0095] In some embodiments, a training module is further included, the training module being configured to: acquire historical technology-supervised text; generate multiple pseudo-entities, type information corresponding to each pseudo-entity, and a relation sequence corresponding to each pseudo-entity based on the historical technology-supervised text using a pre-trained large language model; train the entity model based on the historical technology-supervised text, the multiple pseudo-entities, and the type information corresponding to each pseudo-entity; and train the relation model based on the historical technology-supervised text, the multiple pseudo-entities, the type information corresponding to each pseudo-entity, and the relation sequence corresponding to each pseudo-entity.

[0096] In some embodiments, a parsing module is also included, which is configured to perform structured parsing of the technical supervision text before inputting the technical supervision text into a pre-trained entity model.

[0097] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0098] The apparatus described above is used to implement the method for identifying entities and determining relationships in the corresponding technical supervision text in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0099] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for identifying entities and determining relationships in technical supervision text as described in any of the above embodiments.

[0100] Figure 8 shows a more specific hardware structure diagram of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0101] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0102] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0103] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0104] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0105] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0106] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0107] The electronic device described above is used to implement the method for identifying entities and determining relationships in the corresponding technical supervision text in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0108] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method for identifying entities and determining relationships in technical supervision text as described in any of the above embodiments.

[0109] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0110] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the method for identifying entities and determining relationships in technical supervision text as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0111] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides a computer program product, including computer program instructions, which, when run on a computer, cause the computer to execute the method for identifying entities and determining relationships in technical supervision text as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0112] It should be noted that the embodiments of this application can be further described in the following ways: It is understood that before using the technical solutions of the various embodiments in this disclosure, the user will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and the user's authorization will be obtained.

[0113] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0114] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0115] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0116] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0117] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0118] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0119] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for identifying entities and determining relationships in technically supervised text, characterized in that, include: Obtain technical supervision text; The technical supervision text is input into a pre-trained entity model, and the entity model is used to extract entities from the technical supervision text to determine and output multiple entities in the technical supervision text and the type information corresponding to each entity. The multiple entities, the type information corresponding to each entity, and the technical supervision text are input into a pre-trained relation model. The relation model determines the initial relation sequence corresponding to each entity, and the self-attention mechanism is used to optimize each initial relation sequence. Based on each optimized initial relation sequence, the relation model determines and outputs the target relation sequence corresponding to each entity.

2. The method according to claim 1, characterized in that, The step of determining the initial relation sequence corresponding to each entity through the relation model and optimizing each initial relation sequence using a self-attention mechanism includes: using the bidirectional encoder layer in the relation model to encode the multiple entities, the type information corresponding to each entity, and the technical supervision text respectively, to obtain multiple entity codes, the type information code corresponding to each entity, and the technical supervision text code; using the bidirectional gated recurrent unit layer in the relation model to determine the initial relation sequence corresponding to each entity code based on the entity code, the type information code corresponding to the entity code, and the technical supervision text code; and using the self-attention mechanism layer in the relation model to reallocate relation weights to the initial relation sequence corresponding to each entity code.

3. The method according to claim 2, characterized in that, The step of reallocating relation weights to the initial relation sequence corresponding to each entity encoding using the self-attention mechanism layer in the relation model includes: reallocating relation weights to the initial relation sequence corresponding to the entity encoding using the following formula: ,in, This is the optimized initial relation sequence. The first in the initial relation sequence corresponding to the entity encoding Hidden values ​​of individual relationships, The first in the initial relation sequence corresponding to the entity encoding The weight parameters corresponding to each sub-relationship The first in the initial relation sequence corresponding to the entity encoding Individual relationships, The number of all sub-relations in the initial relation sequence corresponding to the entity encoding.

4. The method according to claim 1, characterized in that, The step of extracting entities from the technical supervision text using the entity model, and determining and outputting multiple entities in the technical supervision text and the type information corresponding to each entity, includes: encoding the technical supervision text using the encoding layer in the entity model to obtain multiple concatenated codes; extracting features from each concatenated code using the bidirectional long short-term memory layer in the entity model to obtain the encoding features corresponding to each concatenated code; and determining and outputting at least one entity corresponding to each encoding feature and the type information corresponding to each entity using the output layer in the entity model.

5. The method according to claim 4, characterized in that, The process of encoding the technical supervision text using the encoding layer in the entity model to obtain multiple concatenated codes includes: encoding the technical supervision text by characters to obtain multiple character codes, and encoding the technical supervision text by words to obtain multiple word codes; determining multiple type codes corresponding to the multiple character codes and multiple word codes based on the multiple character codes, the multiple word codes, and the multiple type codes; determining multiple encoding sequences containing at least one character code, at least one word code, and at least one type code based on the multiple character codes, the multiple word codes, and the multiple type codes; and concatenating all codes in each encoding sequence using an attention mechanism to obtain the concatenated code corresponding to the encoding sequence.

6. The method according to claim 4, characterized in that, The step of using the output layer in the entity model to determine and output at least one entity corresponding to each encoded feature and type information corresponding to each entity includes: introducing a conditional random field into the output layer in the entity model, using the conditional random field to determine at least one entity corresponding to each encoded feature and type information corresponding to each entity, and outputting at least one entity corresponding to each encoded feature and type information corresponding to each entity.

7. The method according to claim 1, characterized in that, The training methods for the entity model and the relation model include: acquiring historical technology-supervised text; generating multiple pseudo-entities, type information corresponding to each pseudo-entity, and a relation sequence corresponding to each pseudo-entity based on the historical technology-supervised text and a pre-trained large language model; training the entity model based on the historical technology-supervised text, the multiple pseudo-entities, and the type information corresponding to each pseudo-entity; and training the relation model based on the historical technology-supervised text, the multiple pseudo-entities, the type information corresponding to each pseudo-entity, and the relation sequence corresponding to each pseudo-entity.

8. The method according to claim 1, characterized in that, Before inputting the technical supervision text into a pre-trained entity model, the method includes: performing structured parsing of the technical supervision text.

9. A device for identifying entities and determining relationships in technically supervised text, characterized in that, include: The acquisition module is configured to acquire technical supervision text; The entity extraction module is configured to input the technical supervision text into a pre-trained entity model, extract entities from the technical supervision text through the entity model, determine and output multiple entities in the technical supervision text and the type information corresponding to each entity; The determination module is configured to input the plurality of entities, the type information corresponding to each entity, and the technical supervision text into a pre-trained relation model, determine the initial relation sequence corresponding to each entity through the relation model, and optimize each initial relation sequence using a self-attention mechanism; The output module is configured to determine and output the target relation sequence corresponding to each entity based on the optimized initial relation sequence and the relation model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 8.

Images