An industrial large model-based knowledge graph construction method

Abstract

The application relates to a knowledge graph construction method based on an industrial large model, and belongs to the technical field of knowledge graph construction. The method solves the problems of lack of field adaptability, insufficient entity representation distinguishing ability, error propagation, and difficulty in processing unstructured attribute standardization in the existing industrial knowledge graph construction mode. The method comprises the following steps: sequentially performing prompt learning-based industrial entity extraction and dynamic sample mining-based contrast learning representation enhancement on a target industrial field to obtain enhanced representation of industrial entities in the target industrial field; performing hierarchical relationship extraction on the enhanced representation of the industrial entities to obtain corresponding hierarchical relationship extraction results; performing multi-channel semantic-based industrial attribute value processing and learning optimization on unstructured industrial attribute values in the target industrial field to convert the unstructured industrial attribute values into structured industrial attribute values; and constructing a knowledge graph according to the enhanced representation of the industrial entities, the hierarchical relationship extraction results, and the structured industrial attribute values.

Description

Technical Field

[0001] This invention relates to the field of knowledge graph construction technology, and in particular to a method for constructing a knowledge graph based on a large industrial model. Background Technology

[0002] Industrial knowledge graphs are a key infrastructure for industrial intelligence, playing a vital role in industrial scenarios such as equipment maintenance, process improvement, and fault detection. Traditional knowledge graph construction methods mainly rely on rule matching and statistical machine learning techniques. While these methods have achieved certain results in general domains, they have shown significant shortcomings when dealing with complex industrial scenarios.

[0003] Early industrial knowledge extraction often employed dictionary- and rule-based methods, with experts manually developing extraction rules and pattern matching templates. While this approach offered strong interpretability, it required extensive domain knowledge and struggled to adapt to new industrial scenarios. With advancements in machine learning, extraction methods based on traditional machine learning models such as Conditional Random Fields (CRF) and Support Vector Machines (SVM) emerged. These methods reduced reliance on manual rules to some extent, but their effectiveness was limited when dealing with specialized terminology and complex sentence structures in industrial texts.

[0004] In recent years, deep learning technology has brought new progress to knowledge extraction. Sequence labeling models based on recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) can better capture the contextual information of text, but they still have shortcomings in handling long-distance dependencies in industrial texts. Subsequently, the Transformer architecture based on attention mechanisms, especially pre-trained language models such as BERT, has significantly improved text representation capabilities through large-scale unsupervised pre-training, providing new solutions for industrial knowledge extraction.

[0005] However, existing pre-trained models still face many challenges in industrial applications. First, the industrial domain contains a large number of technical terms and specific expressions, making it difficult for general-purpose pre-trained models to fully understand their semantics. Second, there is a lack of effective means for standardizing industrial attribute values, especially the normalization of numerical parameters and textual descriptions. Furthermore, traditional entity representation learning methods are insufficient in distinguishing subtle differences between industrial entities, resulting in low entity linking accuracy. Regarding relation extraction, pipelined methods suffer from error propagation problems, while existing joint extraction methods struggle to handle the complex overlapping relationships in the industrial domain.

[0006] Industrial knowledge graph technology is primarily applied in industrial sectors such as machinery manufacturing, aerospace, electronics and electrical engineering, and energy and chemical engineering. For example, in machinery manufacturing, it is used for equipment knowledge management, fault diagnosis, and maintenance decision support; in aerospace, it supports parts supply chain management and quality control; in electronics and electrical engineering, it is used for product design knowledge management and production process optimization; and in energy and chemical engineering, it is used for equipment safety monitoring and process parameter optimization. These application scenarios place high demands on the accuracy, efficiency, and domain adaptability of knowledge extraction.

[0007] The following are the main technical bottlenecks currently existing in the construction of industrial knowledge graphs: (1) Poor domain adaptability leads to insufficient entity recognition accuracy; Traditional methods have problems such as poor domain adaptability and scarce labeled data in industrial entity recognition, resulting in insufficient entity extraction accuracy. (2) Insufficient entity representation discrimination ability affects link accuracy; In terms of entity representation learning, traditional methods lack the ability to distinguish subtle differences, resulting in low entity link accuracy. (3) Error propagation problem in pipelined relation extraction; In terms of relation extraction, pipelined methods have error propagation problems and are difficult to handle complex relation overlap in industrial fields.

[0008] (4) There is a lack of effective standardized methods for unstructured attribute descriptions; in terms of attribute value processing, there is a lack of unified standardized methods for unstructured industrial attribute descriptions, and the existing rule base is difficult to cover complex industrial scenarios.

[0009] These problems severely restrict the large-scale application and practical effects of industrial knowledge graphs, and new technological breakthroughs are urgently needed. Summary of the Invention

[0010] Based on the above analysis, the embodiments of the present invention aim to provide a knowledge graph construction method based on industrial large models, in order to solve the problems of lack of domain adaptability, insufficient entity representation differentiation ability and error propagation, and difficulty in handling the standardization of unstructured attributes in existing industrial knowledge graph construction methods.

[0011] This invention provides a method for constructing a knowledge graph based on a large industrial model, the method comprising: The target industrial domain is sequentially subjected to cue-based learning-based industrial entity extraction and dynamic sample mining-based contrastive learning representation enhancement to obtain the enhanced representation of industrial entities in the target industrial domain. Hierarchical relationships are extracted from the enhanced representation of industrial entities to obtain the corresponding hierarchical relationship extraction results; Unstructured industrial attribute values ​​in the target industrial field are processed and optimized based on multi-channel semantics, and then converted into structured industrial attribute values. A knowledge graph is constructed based on the enhanced representation of industrial entities, the results of hierarchical relationship extraction, and the structured industrial attribute values.

[0012] Based on the above solution, the present invention also makes the following improvements: Furthermore, industrial entities are extracted in the following manner: A structured prompt template library adapted to the target industrial field is constructed. Industrial entities are extracted from each structured prompt template in the library based on prompt learning, and the industrial entity extraction results are obtained.

[0013] Furthermore, the process of extracting industrial entities based on prompting learning involves: Construct a structured prompt template library adapted to the target industrial field; each prompt template in the prompt template library adopts a structured form: [CLS][Domain Keywords][Text Fragment][MASK]; where [CLS] is the sequence start marker, and [MASK] is used to guide the model to predict the entity type; A trainable matrix is ​​introduced to dynamically generate soft cue vectors, which are concatenated with the embedded representation of text fragments in the cue template to form an enhanced input representation. Based on the enhanced input representation, entity type prediction is performed for the corresponding cue template. All prompt templates that have completed entity type prediction are deduplicated and calibrated to obtain the industrial entity extraction results.

[0014] Furthermore, the industrial entity extraction results are enhanced with representation based on sample mining and contrastive learning optimization. The following steps are performed: A training sample set is constructed using the results of industrial entity extraction; In each training batch, dynamic sample selection is performed on the training sample set. The selected samples are used as anchor entities, and positive and negative sample pairs are dynamically constructed for the anchor entities. The positive and negative samples are used to train the comparative learning optimization model. The trained contrastive learning model is optimized to output the enhanced representation of each industrial entity.

[0015] Furthermore, hierarchical relationships are extracted from the enhanced representation of industrial entities, and the following steps are performed: A multi-head self-attention mechanism is used to process the augmented representation of industrial entities to obtain the corresponding shared representation. Hierarchical decoding is performed on the shared representation corresponding to the enhanced representation of industrial entities to obtain the corresponding hierarchical relationship extraction results.

[0016] Furthermore, the process of processing and learning optimization of industrial attribute values ​​based on multi-channel semantics involves: The received unstructured industrial attribute values ​​are standardized and preprocessed using an industrial attribute value normalization rule base. The standardized preprocessed industrial attribute values ​​are subjected to dual-channel semantic parsing, and learning optimization is performed during the parsing process to obtain structured industrial attribute values.

[0017] Furthermore, the dual-channel semantic parsing includes a numerical attribute processing channel and a text attribute processing channel.

[0018] Furthermore, the numerical attribute processing channel is suitable for processing numerical parameters including temperature, pressure, and size; The numerical attribute processing channel sequentially performs pattern matching, feature extraction, standardization mapping, and destandardization output to obtain the corresponding structured industrial attribute values.

[0019] Furthermore, the text-based attribute processing channel is suitable for processing text-based attribute parameters, including material type and surface treatment. The text-based attribute processing channel sequentially performs candidate value set construction, semantic encoding, similarity calculation and matching to obtain the corresponding structured industrial attribute values.

[0020] Furthermore, the construction of the knowledge graph involves: A knowledge graph is constructed by taking each industrial entity augmented representation as a node, the structured industrial attribute value corresponding to the industrial entity augmented representation as the attribute value of the corresponding node, and the extraction result of the hierarchical relationship between the industrial entity augmented representations as the edge between the corresponding nodes.

[0021] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects: This invention addresses key technical challenges in the construction of industrial knowledge graphs by proposing a method and system for constructing knowledge graphs based on large industrial models as a solution.

[0022] The innovations of this invention are mainly reflected in the following four aspects: (1) An industrial entity extraction method based on prompting learning is proposed. By constructing a domain-adaptive prompting template library, adopting trainable continuous soft prompting technology, and combining it with label semantic perception loss optimization, domain-adaptive entity recognition without a large amount of labeled data is achieved. (2) Develop an entity representation enhancement method based on sample mining and contrastive learning optimization. Through dynamic sample selection mechanism and optimized contrastive loss function, the ability to distinguish entities and the accuracy of linking are improved. (3) A hierarchical entity relationship joint extraction algorithm is proposed. It adopts a multi-task learning architecture and a relationship-aware attention mechanism to achieve collaborative optimization of entity recognition and relationship classification, effectively avoiding the error propagation problem. (4) Design an industrial attribute value processing and learning optimization method based on multi-channel semantics. By organically integrating rule base and deep learning, a dual-channel semantic parsing mechanism is constructed to process numerical and textual attributes respectively, thereby achieving efficient standardization of industrial attribute values.

[0023] These innovations, through a multi-stage optimization strategy, achieve synergistic optimization of entity recognition, relation extraction, and attribute standardization, thereby improving the accuracy and efficiency of knowledge extraction in the industrial field.

[0024] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0025] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 A flowchart illustrating a knowledge graph construction method based on a large industrial model, provided in an embodiment of the present invention. Detailed Implementation

[0026] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0027] One specific embodiment of the present invention discloses a method for constructing a knowledge graph based on a large industrial model. This method integrates cue learning, contrastive learning, and hierarchical joint extraction techniques for constructing industrial knowledge graphs. Through a multi-stage optimization strategy, it achieves synergistic optimization of entity recognition, attribute standardization, and relation extraction, thereby improving the accuracy and efficiency of knowledge extraction in the industrial field. The flowchart of this method is as follows: Figure 1 As shown in the figure. The specific implementation method is explained below.

[0028] Step S1: Perform cue-based learning-based industrial entity extraction and dynamic sample mining-based contrastive learning representation enhancement on the target industrial domain in sequence to obtain the enhanced representation of industrial entities in the target industrial domain.

[0029] Step S11: Construct a structured prompt template library adapted to the target industrial field, and extract industrial entities from each structured prompt template in the structured prompt template library based on prompt learning to obtain the industrial entity extraction results.

[0030] In this embodiment, step S11 proposes an industrial entity extraction method based on prompt learning, aiming to efficiently and accurately identify key entities in text for industrial fields such as machinery manufacturing and aerospace. This method effectively improves the accuracy of industrial entity recognition by constructing structured prompt templates adapted to the industrial field, employing dynamic soft prompting technology, and optimizing label semantic perception loss. Using this method, high-precision entity extraction from industrial text can be achieved, providing reliable support for the construction of industrial knowledge graphs.

[0031] Step S111: Build a structured prompt template library adapted to the target industrial field.

[0032] In the specific implementation process, a structured prompt template library adapted to the characteristics of different industrial fields such as machinery manufacturing and aerospace is constructed. In the structured prompt template library constructed in this embodiment, each prompt template adopts a structured form: [CLS][Domain Keywords][Text Fragment][MASK]. Specifically, [CLS] is the sequence start marker, [Domain Keywords] is used to inject domain prior knowledge and can be dynamically selected according to the application scenario (such as "spindle system", "feed mechanism", "hydraulic device" etc. in the machinery manufacturing field); the system can automatically match the most relevant domain keywords based on the text content. For example, in the machine tool fault diagnosis scenario, domain keywords can include specific component types such as "spindle system", "feed mechanism", "hydraulic device". [Text Fragment] is the text fragment to be analyzed. [MASK] is a mask marker to guide the model to predict the entity type.

[0033] In the specific implementation process, the prompt template library can be constructed as follows: First, extract domain features from the original industrial text (such as maintenance reports, operation manuals, etc.) to identify the core technology fields involved in the original industrial text (such as "machine tools" and "cutting tools" in mechanical manufacturing, and "engine blades" and "landing gear" in aerospace), and match 3-5 of the most relevant domain keywords from a preset domain keyword library. Then, segment the original text into multiple text fragments according to semantic units (each fragment's length is controlled within 512 tokens). For each text fragment, combine it with the matched domain keywords to generate a prompt template in the form of "[CLS][domain keyword][text fragment][MASK]". For example, for a text fragment containing "abnormal increase in spindle bearing temperature", if the domain keyword is "spindle system", the generated prompt template is "[CLS] spindle system spindle bearing temperature abnormal increase [MASK]". To enhance template diversity, the text fragments will also undergo synonym replacement (e.g., replacing "elevate" with "rise" or "increase") and sentence transformation (e.g., converting active and passive sentences) to generate 2-3 variant templates, ensuring that the model can understand the entity context from different perspectives.

[0034] Step S112: Introduce a trainable matrix to dynamically generate soft cue vectors, and concatenate them with the embedded representation of the text fragment in the cue template to form an enhanced input representation. Based on the enhanced input representation, predict the entity type of the corresponding cue template.

[0035] This embodiment considers that traditional prompting methods rely on fixed text templates and adopts trainable continuous prompt vectors to enhance the model's adaptability. In this embodiment, a trainable matrix is ​​introduced based on the continuous soft prompting method. Where d is the hidden layer dimension and k is the number of cue tokens. Dynamically generating soft cue vectors using a trainable matrix effectively avoids the limitations of manually designed templates. The generated soft cue vectors are then embedded with the text fragments in each cue template. (where n is the length of the text segment) are concatenated to form an enhanced input representation. Then, based on this enhanced input representation, the entity type prediction for the corresponding prompt template can be completed.

[0036] Preferably, an entity type prediction model is used to predict the entity type of the enhanced input representation based on the corresponding prompt template. For example, the entity type prediction model can be constructed using a BERT encoder and an entity type prediction network connected sequentially.

[0037] The BERT encoder encodes the enhanced input representation corresponding to each cue template, and outputs a text representation of the fused domain cue information for that template. Therefore, in terms of the model architecture of this embodiment, a trainable parameter matrix is ​​added as a soft cue vector before the input layer of the BERT encoder. Preferably, the recommended dimension of this matrix is ​​the product of the hidden dimension 768 and the number of cue tokens 20, i.e., creating a 768×20 trainable matrix. In practice, the soft cue vector is concatenated with the embedding representation of the original text to form an enhanced input representation. The concatenated sequence is input into the BERT encoder and jointly encoded through multiple Transformer blocks to obtain a text representation that incorporates domain cue information. This step enables the model to combine domain prior knowledge, improving the specificity of entity recognition.

[0038] The text representation of the fused domain prompt information corresponding to each prompt template is input into the entity type prediction network to predict the entity type of the corresponding prompt template.

[0039] In the fine-tuning stage of the entity type prediction network in this embodiment, a label semantic perception loss can be introduced to predict entity types so that the predicted entity types are semantically aligned with the real labels. The specific explanation is as follows.

[0040] Type encoding: The encoder f(·) is used to map entity types (such as “machine tool parts”, “material properties”) to semantic vector space.

[0041] Loss Calculation: Calculate the prediction type With real type Cosine similarity loss: (1) This loss function ensures that the model not only predicts the correct category, but also learns the semantic associations between categories (such as the subordinate relationship between "bearing" and "machine tool parts"), thereby improving generalization ability.

[0042] For example, to optimize model training, a dedicated label semantic awareness loss function needs to be designed. First, a separate encoder (such as a lightweight BERT or Word2Vec model) is used to map entity type labels to the semantic space. For instance, labels for mechanical parts such as "bearing," "gear," and "shaft" are encoded as high-dimensional semantic vectors. During training, in addition to the standard cross-entropy loss, a cosine similarity loss between the predicted and true types in the semantic space is additionally calculated. The weight coefficient of this loss term is recommended to be set between 0.3 and 0.5 to balance the main task loss and the semantic constraint loss.

[0043] In practice, the model training adopts a phased strategy: First, pre-training is performed on a general industrial corpus with a learning rate of 2e-5 and a batch size of 32 for 5-10 epochs; then, fine-tuning is performed on domain-specific data, at which point the learning rate is reduced to 5e-6 and the batch size can be reduced to 16, and training continues for 3-5 epochs. The AdamW optimizer is used during training, along with a linear learning rate decay strategy.

[0044] In practical deployment, after receiving raw industrial text input, the system automatically selects the most relevant domain keywords based on the text content, constructs corresponding prompt templates, and then uses a trained model to predict entity types. The implementation process is based on the pre-trained language model BERT, achieving domain-adaptive entity recognition by introducing learnable prompt parameters and semantic-aware optimization objectives. The final output not only includes entity boundary and type information but also provides prediction confidence scores, facilitating quality control in the subsequent knowledge graph construction process. Throughout the implementation process, continuous monitoring of model performance and regular updates to the prompt template library and domain keyword dictionary are required to adapt to the ever-changing industrial terminology system.

[0045] Step S113: Deduplicate and calibrate all prompt templates that have completed entity type prediction to obtain industrial entity extraction results.

[0046] The specific implementation process of step S113 is described as follows: First, the entity extraction results from each prompt template are initially aggregated to form an initial entity set. For this set, a dual deduplication strategy based on entity boundaries and type is employed: for extraction results with completely overlapping text boundaries and consistent entity types, the entry with the highest confidence is retained; for entities with partially overlapping boundaries but the same type, the semantic similarity of the core words within the entity is calculated (using Word2Vec cosine similarity, with a threshold of 0.85). If the similarity exceeds the threshold, the entity with the longest boundary is merged. During the calibration phase, a domain rule base is introduced for entity type verification. The rule base contains common entity type constraints in the industrial field (e.g., "bearing" can only belong to the "parts" type, and "temperature" belongs to the "parameter index" type). For entities whose predicted types conflict with the rule base, a secondary judgment is made based on the context, and if necessary, expert confirmation is submitted through a manual review interface. For example, if an entity is predicted as "material" but the context explicitly describes it as "sensor model," the type correction mechanism in the rule base is triggered, adjusting the entity type to "equipment model." The final output of industrial entity extraction results includes the entity ID of each industrial entity, the content of the text fragment and text boundaries (start / end character positions), entity type and prediction confidence (must reach 0.7 or above), providing standardized input for subsequent entity representation enhancement.

[0047] Step S12: Perform representation enhancement on the industrial entity extraction results based on sample mining and comparative learning optimization to obtain the enhanced representation of industrial entities.

[0048] In this step, this embodiment proposes an entity representation enhancement method based on contrastive learning. By using a sample mining strategy and an optimized loss function design, the method improves the discriminative ability of entity representations and increases the accuracy of entity linking.

[0049] Specifically, in this embodiment, the following operations are performed: a training sample set is constructed based on the industrial entity extraction results; in each training batch, dynamic sample selection is performed on the training sample set, and the selected samples are used as anchor entities. Positive and negative sample pairs are dynamically constructed for the anchor entities, and the contrastive learning optimization model is trained using the positive and negative samples; the enhanced representation of each industrial entity is output using the trained contrastive learning optimization model.

[0050] The dynamic sample selection mechanism proposed in this embodiment can automatically filter the most challenging negative samples during training. Within each training batch, the selected samples are used as anchor entities. Positive and negative sample pairs are dynamically constructed for these anchor entities, and the entity encoder parameters are optimized through backpropagation, causing similar entities to cluster in the representation space. For example, for each anchor entity... Within the training batch, select entities of the same type but different individuals. As negative samples, a similarity threshold is set. The range of negative samples is carefully considered to ensure that the selected negative samples are neither too simple nor too difficult. Cosine similarity is used for similarity calculation, reflecting the relative positional relationship of entities in the representation space. This strategy avoids the model getting trapped in local optima and promotes the learning of more discriminative entity representations.

[0051] This embodiment optimizes the entity representation space by designing a loss function for contrastive learning in the contrastive learning optimization model. The core of the loss function is to maximize the positive sample pairs. Similarity, while minimizing the negative sample pair The similarity is calculated by setting a temperature coefficient τ to adjust the sharpness of the softmax distribution and balance the learning intensity of easy and difficult samples. The similarity function s(·) adopts standard cosine similarity calculation to ensure fair comparison of representations at different scales. During the calculation process, a weighted sum is performed on K negative samples to enhance the model's discriminative ability, thus yielding the loss function. , represented as: (2) Step S2: Extract hierarchical relationships from the enhanced representation of industrial entities to obtain the corresponding hierarchical relationship extraction results.

[0052] This step proposes a hierarchical entity relation joint extraction algorithm, which achieves collaborative optimization of entity recognition and relation classification through a multi-task learning architecture and a relation-aware attention mechanism.

[0053] Step S21: Use a multi-head self-attention mechanism to process the industrial entity augmentation representation to obtain the corresponding shared representation.

[0054] In this embodiment, a shared text representation foundation is constructed by designing a shared coding layer and employing a multi-head self-attention mechanism (MHSA): the input text is transformed through a learnable linear transformation matrix. , , Query, key, and value representations are generated separately, and then global dependencies of the text are calculated through an attention mechanism to form a shared representation. .

[0055] This shared coding layer captures both local features and long-range dependencies of the text, providing rich semantic representations for subsequent tasks.

[0056] Step S22: Perform hierarchical decoding on the shared representation corresponding to the enhanced representation of industrial entities to obtain the corresponding hierarchical relationship extraction results.

[0057] Specifically, in this step, the hierarchical task decomposition architecture includes an entity recognition layer and a relationship classification layer. The following hierarchical processing flow is adopted in the decoding phase.

[0058] (1) Entity recognition layer Based on shared representation It uses a bidirectional LSTM to capture the contextual information of the sequence and models the label transition probability to ensure the rationality of the output label sequence. Then it outputs the boundary position and type category of the entity.

[0059] (2) Relationship classification layer Constructing a 3D entity pair matrix , where n is the text length. Each entity pair It consists of three parts: head entity representation Tail entity representation Entity interaction features (Hadamardi), which can be expressed as This design explicitly models the interaction relationships between entities, enhancing the ability to classify relationships.

[0060] In this embodiment, a relation-aware attention mechanism is used for analysis: firstly, a specific attention weight matrix is ​​designed for each relation type. Then, relation-specific attention is calculated using the softmax function. ,Right now This mechanism enables the model to adaptively focus on key features for different relationship types. It can support parallel prediction of multiple relationship categories and handle complex relationship overlap situations.

[0061] The training process employs a multi-task learning framework, sharing underlying encoding parameters, and designs a joint loss function to balance entity recognition and relation classification tasks. Simultaneously, end-to-end training achieves co-optimization of the two tasks, and global optimization is performed during the decoding phase to ensure consistency between entity and relation predictions.

[0062] When implementing the hierarchical entity relationship joint extraction method, a multi-task learning framework needs to be constructed first. This framework adopts a structure combining a shared encoder and a task-specific decoder. The shared encoder uses a 12-layer Transformer architecture with a hidden layer dimension of 768, and the multi-head attention mechanism is recommended to have 12 heads. After word segmentation, the input text is converted into a vector representation through a learnable word embedding layer. The maximum sequence length is recommended to be set to 512 tokens to cover the longer technical descriptions in industrial documents.

[0063] Implementing the shared encoding layer requires configuring linear transformation matrices for queries, keys, and values. It is recommended that these matrices be set to 768×768 dimensions and initialized using the Xavier initialization method. Attention computation employs a scaled dot product attention mechanism; the outputs of attention heads are concatenated and then linearly transformed to form the final shared representation. The text representation output by this layer simultaneously contains local syntactic features and global semantic dependencies, providing a unified feature foundation for subsequent tasks.

[0064] The entity recognition layer employs a sequence labeling architecture combining bidirectional LSTM and CRF. A dimension of 256 is recommended for the LSTM hidden layers, and the sequence labeling uses the BIOES labeling system, including entity boundary and type information. When calculating label transition probabilities, a randomly initialized transition matrix is ​​used, and the optimal label sequence is decoded using the Viterbi algorithm. The output of the entity recognition layer includes the entity's start and end positions, and the entity type; this information is directly passed to the relation classification layer.

[0065] The implementation of the relation classification layer requires the construction of a three-dimensional entity pair matrix. For each entity pair in the text, the head entity representation, tail entity representation, and their element-wise product features are extracted, and these features are concatenated to form a joint representation of dimension 2304 (when using 768-dimensional vectors). The entity pair matrix is ​​constructed dynamically, calculating only the actual existing entity pairs to reduce computational overhead.

[0066] Implementing a relationship-aware attention mechanism requires creating a dedicated attention weight matrix for each relationship type. Taking 20 common relationship types in the industrial field as an example, 20 attention weight matrices with a dimension of 2304×1 are created. When calculating relationship-specific attention, each entity pair representation is linearly transformed and then normalized using a softmax function to obtain the attention distribution under different relationship types. This mechanism enables the model to adaptively focus on different feature dimensions for different relationship types such as "part-component" and "fault-phenomenon".

[0067] The joint training process employs a multi-task loss function. It is recommended to set the weight coefficient for the entity recognition task to 0.6 and the weight for the relationship classification task to 0.4. Optimization uses the AdamW optimizer with an initial learning rate of 2e-5, employing a linear decay strategy for 10 training epochs. The batch size is recommended to be 16 or 32 depending on the GPU memory configuration. Gradient accumulation is used during training, updating parameters every 4 batches to ensure training stability.

[0068] The decoding phase employs a global optimization strategy, using constraint fulfillment to ensure consistency between entity and relationship predictions. For example, when predicting a "containment" relationship, the head entity must be a device type, and the tail entity must be a part type. This constraint is implemented through post-processing rules, effectively improving the reliability of the prediction results. The final output is in JSON format, containing a list of entities and a list of relationships. Each relationship record includes the head entity ID, tail entity ID, relationship type, and confidence score.

[0069] Establish a regular evaluation mechanism during model deployment, using precision, recall, and F1 score to monitor model performance. It is recommended to test the latest industrial documents weekly, triggering model fine-tuning when performance drops by more than 3%. Support incremental learning capabilities, enabling the integration of new entity and relationship types without impacting existing performance.

[0070] This step achieves the following technical effects: deep collaboration between entity recognition and relationship classification, avoiding error propagation in pipelined methods, and effectively improving the recognition capability of complex relationships through a relationship-specific attention mechanism. This technology provides an efficient joint entity and relationship extraction solution for industrial knowledge graph construction and can be widely applied to intelligent industrial scenarios such as equipment maintenance and process optimization.

[0071] Step S3: Process and learn the unstructured industrial attribute values ​​in the target industrial field based on multi-channel semantics to convert them into structured industrial attribute values.

[0072] This step improves the accuracy and adaptability of attribute value standardization by combining a rule base with deep learning. Details are as follows.

[0073] Step S31: Standardize and preprocess the received unstructured industrial attribute values ​​using the industrial attribute value normalization rule base.

[0074] Preferably, in this embodiment, the industrial attribute value normalization rule base includes the following four conversion rules: Unit conversion rules: used to unify different units of measurement into standard units, such as converting "mm" to "meter" and "rpm" to "revolutions per minute". Enumeration value mapping rules: used to quantify qualitative descriptions into numerical levels, such as quantizing "high / medium / low" into "1 / 2 / 3"; Format standardization rules: used to unify special data formats such as date format and number format; Terminology substitution rules: used to unify synonyms and near-synonyms into standard industrial terms.

[0075] This industrial attribute value normalization rule base can be organized in a hierarchical structure, supporting dynamic expansion and priority adjustment of rules to ensure coverage of complex industrial data. In specific implementation, the industrial attribute value normalization rule base can be organized in a tree-like hierarchical structure: root node: stores general rules for the industrial field; child nodes: correspond to sub-fields such as machinery manufacturing, electronics and electrical engineering, and aerospace; leaf nodes: store specific conversion rules. Rule priority is controlled by weight coefficients, with a recommended value range of 0.8-1.2, and rules with higher weights are applied first. The rule base supports dynamic addition, deletion, modification, and query operations. Administrators can add new unit conversion rules (such as converting "r / min" to "revolutions / minutes"), enumerated value mapping rules (such as mapping "excellent / good / medium / poor" to "1 / 2 / 3 / 4"), or terminology replacement rules (such as unifying "screw" to "bolt") through a graphical interface.

[0076] Preferably, in this embodiment, the received unstructured industrial attribute values ​​are standardized and preprocessed using an industrial attribute value normalization rule library. Specifically, the following operations are performed: based on the target industrial field, the child nodes in the tree-like hierarchical structure of the industrial attribute value normalization rule library are located; based on the data type of each unstructured industrial attribute value, the applicable conversion rules are matched from the corresponding child nodes for conversion to obtain the standardized preprocessed industrial attribute values.

[0077] Step S32: Perform dual-channel semantic parsing on the standardized preprocessed industrial attribute values, and perform learning optimization during the parsing process to obtain structured industrial attribute values.

[0078] In the specific implementation process, this embodiment designs a numerical attribute processing channel and a text attribute processing channel, and performs data splitting based on the attribute type of the standardized preprocessed industrial attribute values.

[0079] (1) Numerical attribute processing channel The numerical attribute processing channel is suitable for processing numerical parameters such as temperature, pressure, and size. The specific implementation method is explained below.

[0080] 1) Pattern Matching: Pre-configure a regular expression pattern library. For size parameters, configure regular expressions to match the pattern "number + unit", such as "\d+(\.\d+)?\s(mmcm m)"; for speed parameters, design patterns to match "number + speed unit", such as "\d+\s(rpmr / min)".

[0081] 2) Feature extraction: Using predefined regular expression sets Pattern matching is performed, and feature vectors are obtained by extracting numerical and unit features. .

[0082] 3) Standardization mapping: through a linear transformation layer The extracted feature vectors are mapped to a standardized space; the output range is constrained using the sigmoid activation function σ to obtain the final standardized value. .

[0083] (3) For example, the extracted numerical features are processed by a linear transformation layer, the output dimension of which is set to 1, and the output value is constrained to the range of 0-1 using the Sigmoid activation function.

[0084] 4) Denormalized output: The denormalized module converts the normalized values ​​into actual engineering values, which are then used as the corresponding structured industrial attribute values.

[0085] In practice, it is recommended to set the batch size to 32, the learning rate to 0.001, and use the Adam optimizer to update the parameters.

[0086] (2) Text attribute processing channel The text-based attribute processing channel is suitable for processing text-based attribute parameters such as material type and surface treatment. The specific implementation method is explained below.

[0087] 1) Candidate value set construction: Establish a standard candidate value dictionary as a reference benchmark. For example, the material type candidate set includes standard terms such as "45 steel", "304 stainless steel", and "6061 aluminum alloy", with each term equipped with no less than 5 synonym variants.

[0088] 2) Semantic encoding: A contrastive learning model based on BERT is used to encode the input text and each candidate value word into a 768-dimensional semantic vector.

[0089] 3) Similarity calculation and matching: The semantic similarity between the input text and each candidate value is calculated using a contrastive learning model, and the candidate value with the highest similarity is selected as the corresponding structured industrial attribute value (i.e., the standardized result).

[0090] For example, during training, a triplet loss function is used, with a suggested positive sample similarity threshold of 0.85, a negative sample threshold of 0.4, and a marginal parameter of 0.2. In practice, the numerical attribute processing channel can be optimized through supervised learning. For the text attribute processing channel, a contrastive learning model is trained using Triplet Loss. Both attribute processing channels share the underlying semantic representation while maintaining independent predictions, and continuously update the rule base and model parameters through an online learning mechanism.

[0091] The system employs a dual-channel parallel processing mechanism: input text first passes through a rule preprocessing module, where the highest-priority matching rule completes the initial transformation; then, it is routed to the corresponding processing channel based on attribute type—text containing combinations of numbers and units enters the numerical channel, while descriptive text enters the text channel. Both channels share the underlying BERT encoder but maintain independent parameter optimization paths. Finally, the standardized results are output through a decision fusion module, and transformation logs are recorded for subsequent model optimization. The model update mechanism uses an online learning approach: labeled samples are automatically collected weekly, and incremental model training is triggered when the accumulated sample count reaches 1000. Rule base updates are implemented through a periodic review mechanism, with domain experts reviewing new terminology variations monthly. Approved rules are added to the rule base with an initial weight of 0.9. System deployment is recommended with a server environment containing 4 CPU cores and 8GB of memory to support concurrent processing of multiple standardization tasks.

[0092] The above method, through the organic combination of rules and learning, achieves efficient and standardized processing of industrial attribute values, maintaining the required accuracy while supporting adaptive expansion to complex industrial scenarios. Compared with traditional methods, this solution reduces the need for manual intervention and improves the overall efficiency of knowledge extraction.

[0093] Step S4: Construct a knowledge graph based on the enhanced representation of industrial entities, the results of hierarchical relationship extraction, and the structured industrial attribute values.

[0094] In this embodiment, a knowledge graph is constructed by taking each industrial entity augmented representation as a node, the structured industrial attribute value corresponding to the industrial entity augmented representation as the attribute value of the corresponding node, and the extraction result of the hierarchical relationship between the industrial entity augmented representations as the edge between the corresponding nodes.

[0095] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0096] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a knowledge graph based on a large industrial model, characterized in that, The method includes: The target industrial domain is sequentially subjected to cue-based learning-based industrial entity extraction and dynamic sample mining-based contrastive learning representation enhancement to obtain the enhanced representation of industrial entities in the target industrial domain. Hierarchical relationships are extracted from the enhanced representation of industrial entities to obtain the corresponding hierarchical relationship extraction results; Unstructured industrial attribute values ​​in the target industrial field are processed and optimized based on multi-channel semantics, and then converted into structured industrial attribute values. A knowledge graph is constructed based on the enhanced representation of industrial entities, the results of hierarchical relationship extraction, and the structured industrial attribute values.

2. The knowledge graph construction method based on a large industrial model according to claim 1, characterized in that, Industrial entities are extracted in the following manner: A structured prompt template library adapted to the target industrial field is constructed. Industrial entities are extracted from each structured prompt template in the library based on prompt learning, and the industrial entity extraction results are obtained.

3. The knowledge graph construction method based on a large industrial model according to claim 2, characterized in that, The process of extracting industrial entities based on prompting learning involves: Construct a structured prompt template library adapted to the target industrial field; each prompt template in the prompt template library adopts a structured form: [CLS][Domain Keywords][Text Fragment][MASK]; where [CLS] is the sequence start marker, and [MASK] is used to guide the model to predict the entity type; A trainable matrix is ​​introduced to dynamically generate soft cue vectors, which are concatenated with the embedded representation of text fragments in the cue template to form an enhanced input representation. Based on the enhanced input representation, entity type prediction is performed for the corresponding cue template. All prompt templates that have completed entity type prediction are deduplicated and calibrated to obtain the industrial entity extraction results.

4. The knowledge graph construction method based on a large industrial model according to claim 3, characterized in that, The industrial entity extraction results are subjected to representation enhancement based on sample mining and contrastive learning optimization. The following steps are performed: A training sample set is constructed using the results of industrial entity extraction; In each training batch, dynamic sample selection is performed on the training sample set. The selected samples are used as anchor entities, and positive and negative sample pairs are dynamically constructed for the anchor entities. The positive and negative samples are used to train the comparative learning optimization model. The trained contrastive learning model is optimized to output the enhanced representation of each industrial entity.

5. The knowledge graph construction method based on a large industrial model according to any one of claims 1-4, characterized in that, Hierarchical relationships are extracted from the enhanced representation of industrial entities, and the following steps are performed: A multi-head self-attention mechanism is used to process the augmented representation of industrial entities to obtain the corresponding shared representation. Hierarchical decoding is performed on the shared representation corresponding to the enhanced representation of industrial entities to obtain the corresponding hierarchical relationship extraction results.

6. The knowledge graph construction method based on a large industrial model according to claim 5, characterized in that, The process involves industrial attribute value processing and learning optimization based on multi-channel semantics, and the following steps are performed: The received unstructured industrial attribute values ​​are standardized and preprocessed using an industrial attribute value normalization rule base. The standardized preprocessed industrial attribute values ​​are subjected to dual-channel semantic parsing, and learning optimization is performed during the parsing process to obtain structured industrial attribute values.

7. The knowledge graph construction method based on a large industrial model according to claim 6, characterized in that, The dual-channel semantic parsing includes a numerical attribute processing channel and a text attribute processing channel.

8. The knowledge graph construction method based on a large industrial model according to claim 7, characterized in that, The numerical attribute processing channel is suitable for processing numerical parameters including temperature, pressure, and size; The numerical attribute processing channel sequentially performs pattern matching, feature extraction, standardization mapping, and destandardization output to obtain the corresponding structured industrial attribute values.

9. The knowledge graph construction method based on a large industrial model according to claim 8, characterized in that, The text-based attribute processing channel is suitable for processing text-based attribute parameters, including material type and surface treatment. The text-based attribute processing channel sequentially performs candidate value set construction, semantic encoding, similarity calculation and matching to obtain the corresponding structured industrial attribute values.

10. The knowledge graph construction method based on a large industrial model according to claim 9, characterized in that, The construction of the knowledge graph involves: A knowledge graph is constructed by taking each industrial entity augmented representation as a node, the structured industrial attribute value corresponding to the industrial entity augmented representation as the attribute value of the corresponding node, and the extraction result of the hierarchical relationship between the industrial entity augmented representations as the edge between the corresponding nodes.

Images