Engineering machinery assembly entity identification method and system, storage medium and computer equipment

Abstract

The invention discloses an engineering machinery assembly entity identification method and system, a storage medium and computer equipment in the technical field of text processing. The method comprises the following steps: acquiring an engineering machinery assembly text to be identified; and inputting the engineering machinery assembly text into a trained entity recognition model, and determining an entity category corresponding to each word in the engineering machinery assembly text according to the output of the entity recognition model. According to the entity recognition method provided by the invention, entity category prediction can be carried out on each word in the engineering machinery assembly text under the condition of small samples, and the recognition precision of an entity recognition model is improved.

Description

Technical Field

[0001] This invention relates to the field of text processing technology, and in particular to a method, system, storage medium, and computer equipment for recognizing assembly entities of engineering machinery. Background Technology

[0002] Named entity recognition (NER) is one of the core tasks of natural language processing (NLP). Its main task is to identify different types of entities from large amounts of natural language text and label their types. Its goal is to efficiently and accurately identify the types of knowledge entities in text. This task is a fundamental task in NLP, preparing for downstream tasks such as syntactic analysis, building domain knowledge bases, and question answering. In the field of engineering machinery assembly, entity recognition is a crucial step in extracting assembly knowledge, but this field faces the following technical challenges:

[0003] (1) Scarcity of labeled data: There are no publicly available standard datasets in the field of engineering machinery assembly. High-quality labeled data needs to be produced by professional engineers, which is costly and small in scale, resulting in a serious decline in the performance of traditional NER models that rely on large-scale labeled data.

[0004] (2) High domain specialization: The text contains a large number of professional terms, and the existing general NER model has low word segmentation accuracy and insufficient entity boundary recognition accuracy for such terms;

[0005] (3) Weak cross-domain generalization ability: There are large differences in language style and entity type between the engineering machinery assembly field and the general field. Existing few-sample NER methods have low knowledge transfer efficiency in cross-domain scenarios and are difficult to adapt to the entity recognition needs of the assembly field.

[0006] (4) Insufficient collaboration among model modules: In existing models, each module is often optimized independently and lacks an effective collaboration mechanism, which makes it difficult to make full use of limited data information in small sample scenarios. The overall performance of the model is limited and cannot achieve efficient and accurate entity recognition.

[0007] To address the aforementioned issues, there is an urgent need for a technical solution that can achieve high-precision entity recognition in the field of engineering machinery assembly under small sample conditions. Summary of the Invention

[0008] The purpose of this invention is to provide a method, system, storage medium, and computer equipment for recognizing entities in engineering machinery assembly. This method can effectively improve the accuracy of entity recognition in engineering machinery assembly in small sample scenarios, while enhancing the generalization ability and knowledge transfer efficiency of the entity recognition model, thereby solving the problem of insufficient accuracy in entity boundary positioning and category discrimination.

[0009] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0010] In a first aspect, the present invention provides a method for identifying assembly entities of engineering machinery, the method comprising:

[0011] Obtain the assembly text of the engineering machinery to be identified;

[0012] The assembly text of the engineering machinery is input into the trained entity recognition model, and the entity category corresponding to each word in the assembly text of the engineering machinery is determined according to the output of the entity recognition model.

[0013] The entity recognition model includes a feature extraction layer, a feature fusion layer, a bidirectional LSTM network layer, a span layer, and a label layer; the training process of the entity recognition model includes:

[0014] A dataset was constructed based on the collected engineering machinery assembly text.

[0015] The dataset is preprocessed to obtain labeled text dataset and unlabeled text dataset;

[0016] The unlabeled text dataset is input into the BERT pre-trained model. Source domain features are extracted through the feature extraction layer, and the BERT pre-trained model is optimized by combining the cue learning loss function and the contrastive learning loss function to obtain the initial entity recognition model.

[0017] The labeled text dataset is input into the initial entity recognition model, and target domain features are extracted through the feature extraction layer; the target domain features are fused through the feature fusion layer to obtain fused semantic features.

[0018] The bidirectional LSTM network layer encodes the fused semantic features to obtain a context feature sequence;

[0019] Based on the context feature sequence, entity span features are generated through the span layer and label semantic features are generated through the label layer. Entity classification is performed by calculating the similarity between the entity span features and the label semantic features, and cross-entropy loss is calculated based on the classification results.

[0020] Based on the cross-entropy loss, the gradient descent algorithm is used to fine-tune the parameters of the initial entity recognition model to obtain the entity recognition model.

[0021] Furthermore, the preprocessing includes: labeling the text data of the dataset using the BIO annotation method, with annotation types including assembly parts, assembly tools, assembly processes, assembly problems, and design attributes.

[0022] Furthermore, the steps for optimizing the BERT pre-trained model by combining the cue learning loss function and the contrastive learning loss function include:

[0023] Construct mask filling template Combined with the engineering machinery assembly text T, an input sequence is generated. Calculate the learning loss. :

[0024]

[0025] In the formula, Indicates the first The true label of an entity; The model represents the first Predicted labels for each entity; Represents a given input sequence Under the condition that the Bert pre-trained model predicts the first The entity label is The predicted probability;

[0026] Construct a bootstrap template C to generate the input sequence. Construct positive and negative sample pairs and calculate the contrastive learning loss. :

[0027]

[0028] The loss component for each character sample is defined as follows:

[0029]

[0030] In the formula, The L2 norm of a vector is used to calculate... and and The Euclidean distance between them; The feature vector representing the anchor point sample; The feature vector representing a positive sample. The feature vector representing a negative sample; Indicates temperature parameter; This represents the set of all labels in the dataset; Represents a set The number of markers in the middle.

[0031] Furthermore, the step of fusing the target domain features through a feature fusion layer to obtain fused semantic features includes:

[0032] Constructing the dictionary tree D and training samples for the engineering machinery assembly domain Define the matching function Phrases This indicates that the training samples have been identified. Chinese characters The corresponding phrase;

[0033] Perform a nonlinear transformation on the set of phrases:

[0034]

[0035] In the formula, ( ) represents the nonlinear activation function tanh(); Character The corresponding embedding vector; , The weight matrix representing the nonlinear transformation of phrase features; , The bias vector representing the nonlinear transformation of phrase features; Characters obtained after nonlinear transformation The Deep feature representation of each word group;

[0036] Calculate attention score With gate signal :

[0037]

[0038]

[0039] In the formula, Character Context representation vector in a text sequence; Representation of deep features The constructed matrix vector Represents the attention weight matrix; The weight matrix representing the gated signal; The bias vector representing the gate signal; This represents the Sigmoid activation function;

[0040] Calculate attention score and gating signals Fusion weights :

[0041]

[0042] In the formula, Represents the fusion weight matrix. This represents the fusion bias vector. This indicates the attention score. With gate signal To splice;

[0043] Deep feature representation Weighted sum:

[0044]

[0045] In the formula, Character The number of corresponding phrases; Indicates fusion weight The first in Each component corresponds to the j-th word group. The weights;

[0046] Fusion semantic feature representation:

[0047] .

[0048] Furthermore, the bidirectional LSTM network layer obtains a context feature sequence by encoding the fused semantic features, the steps of which include:

[0049] The fused semantic features are input into the forward LSTM network of the bidirectional LSTM network layer to obtain a forward semantic relevance encoding vector:

[0050]

[0051] In the formula, Indicates the first The forward hidden state output vector at each time step Indicates the first The forward output gate control vector at each time step Indicates the first The forward cell state vector at each time step;

[0052] The fused semantic features are input into the inverse LSTM network of the bidirectional LSTM network layer to obtain the inverse semantic relevance encoding vector:

[0053]

[0054] In the formula, Indicates the first The backward hidden state output vector at each time step Indicates the first The backward output gate control vector at each time step Indicates the first The backward cell state vector at each time step; This represents element-wise multiplication.

[0055] The forward semantic relevance encoding vector and the backward semantic relevance encoding vector are added together to obtain the context feature sequence:

[0056]

[0057] In the formula, This represents the dimension of the hidden vectors in a single LSTM network.

[0058] Furthermore, the span layer includes a span generation submodule and an internal and external feature fusion submodule;

[0059] The span generation submodule is used to generate a text with a length of [missing information]. The span quantity Us is generated in the engineering machinery assembly text, in the following specific form:

[0060]

[0061] In the formula, Indicates the predefined maximum span length;

[0062] The internal and external feature fusion submodule performs feature fusion based on the span quantity Us, fusing external and internal features through a gating mechanism to obtain the span feature. The gating mechanism is based on the external features. With internal characteristics The calculation shows that the span feature Defined by the following formula:

[0063]

[0064]

[0065]

[0066] In the formula, The weight matrix represents the external fusion features. The deviation vector represents the external fusion features; Indicates vector concatenation; It represents the set of all labeled embedding vectors within the span; and This represents the learnable weight matrix. This represents the square root of the embedding dimension.

[0067] Furthermore, the step of generating label semantic features through the label layer includes:

[0068] Define a mapping function Λ to construct a set of tag words. ;

[0069] The initial entity recognition model is used to encode the tag words, and the tag semantic features are obtained through a pooling layer:

[0070]

[0071] In the formula, This represents the weight matrix of the pooling layer; This represents the bias vector of the pooling layer.

[0072] Secondly, the present invention provides an engineering machinery assembly entity identification system, comprising:

[0073] The acquisition unit is used to acquire the engineering machinery assembly text to be identified;

[0074] An entity recognition unit is used to input the engineering machinery assembly text into a trained entity recognition model and determine the entity category corresponding to each word in the engineering machinery assembly text based on the output of the entity recognition model.

[0075] The entity recognition model includes a feature extraction layer, a feature fusion layer, a bidirectional LSTM network layer, a span layer, and a label layer; the training process of the entity recognition model includes:

[0076] A dataset was constructed based on the collected engineering machinery assembly text.

[0077] The dataset is preprocessed to obtain labeled text dataset and unlabeled text dataset;

[0078] The unlabeled text dataset is input into the BERT pre-trained model. Source domain features are extracted through the feature extraction layer, and the BERT pre-trained model is optimized by combining the cue learning loss function and the contrastive learning loss function to obtain the initial entity recognition model.

[0079] The labeled text dataset is input into the initial entity recognition model, and target domain features are extracted through the feature extraction layer; the target domain features are fused through the feature fusion layer to obtain fused semantic features.

[0080] The bidirectional LSTM network layer encodes the fused semantic features to obtain a context feature sequence;

[0081] Based on the context feature sequence, entity span features are generated through the span layer and label semantic features are generated through the label layer. Entity classification is performed by calculating the similarity between the entity span features and the label semantic features, and cross-entropy loss is calculated based on the classification results.

[0082] Based on the cross-entropy loss, the gradient descent algorithm is used to fine-tune the parameters of the initial entity recognition model to obtain the entity recognition model.

[0083] Thirdly, the present invention provides a storage medium, characterized in that the storage medium stores instructions, which are loaded and executed by a processor to implement the engineering machinery assembly entity identification method as described in the first aspect.

[0084] Fourthly, the present invention provides a computer device comprising: a memory, a processor, and instructions stored in the memory, the instructions being loaded and executed by the processor to implement the engineering machinery assembly entity recognition method as described in the first aspect.

[0085] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0086] The invention discloses a method for recognizing entities in engineering machinery assembly. This method continuously optimizes the entity recognition model parameters. First, it obtains domain-specific feature representations through a feature extraction layer, achieving knowledge transfer between the general domain and the assembly domain. A training strategy combining cueing learning and contrastive learning is employed to pre-master entity boundary localization and category discrimination capabilities. Then, a feature fusion layer, a bidirectional LSTM network layer, a span layer, and a label layer are effectively integrated into the entity recognition model architecture, and gradient descent is used to fine-tune the model parameters. This method, through its entity recognition module, achieves accurate recognition of text entities in the engineering machinery assembly domain, improving the accuracy of entity boundary localization and category discrimination, and effectively enhancing the accuracy of entity recognition in the engineering machinery assembly domain under small sample conditions. Attached Figure Description

[0087] Figure 1 A flowchart illustrating the steps of the engineering machinery assembly entity identification method provided in this embodiment of the invention;

[0088] Figure 2 This is a schematic diagram of the structure of the engineering machinery assembly entity recognition model provided in an embodiment of the present invention;

[0089] Figure 3 A schematic diagram of the engineering machinery assembly knowledge ontology provided in an embodiment of the present invention. Detailed Implementation

[0090] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Example 1

[0091] like Figure 1 As shown, this embodiment provides a method for identifying assembly entities of engineering machinery, including...

[0092] Obtain the assembly text of the engineering machinery to be identified;

[0093] The engineering machinery assembly text is input into the trained entity recognition model, and the entity category corresponding to each word in the engineering machinery assembly text is determined based on the output of the entity recognition model.

[0094] The specific process of building and training the entity recognition model is described below:

[0095] I. Entity Recognition Model Construction

[0096] like Figure 1 As shown below, the model architecture of an entity recognition model provided by an embodiment of the present invention is as follows. The entity recognition model is constructed based on the BERT pre-trained model and mainly includes a feature extraction layer, a feature fusion layer, a bidirectional LSTM network layer (Bi-LSTM), a span representation layer, and a label representation layer.

[0097] The feature extraction layer comprises a source domain feature extraction subunit and a target domain feature extraction subunit, both sharing the underlying word embedding layer and some Transformer layer parameters. The source domain feature extraction subunit learns general language features and entity patterns through pre-training using unlabeled text data; the target domain feature extraction subunit learns assembly-related terminology features by inputting labeled text data from the engineering machinery assembly field and initializing it based on the shared parameters of the source domain feature extraction subunit.

[0098] The feature fusion layer is used to fuse features from the source domain and the target domain to obtain fused semantic features.

[0099] A bidirectional LSTM network layer is used to encode the fused semantic features to obtain a sequence of contextual features.

[0100] The span layer is used to generate entity span features based on the context feature sequence.

[0101] The label layer is used to generate label semantic features based on the context feature sequence.

[0102] II. Training of the Entity Recognition Model

[0103] Step 1: Construct a dataset based on the collected engineering machinery assembly text;

[0104] In this embodiment, based on the collected assembly text of construction machinery, an experimental dataset CMAD2025 in the field of construction machinery assembly is constructed. The main text data comes from the semi-structured data of construction machinery manufacturing enterprises, including design documents, assembly process documents and quality inspection reports. In addition, it also covers unstructured literature in professional academic databases, such as relevant literature on assembly optimization methods and new technologies, as well as practical experience texts collected from Internet forums and blogs.

[0105] This involves defining the text in the dataset, such as... Figure 3 As shown, five core entities are defined: Assembly Component (ABJ), Assembly Tool (AGJ), Assembly Process (AGY), Assembly Problem (AWT), and Design Attribute (DSX). Six relationships are also defined: Assemble, Parameter Relate, Processes Sequential, UseFor, Cause-Effect, and BelongTo. The BIO annotation system is adopted, labeling the starting position of entities as "B-XX" (e.g., B-ABJ), the middle or ending positions as "I-XX" (e.g., I-ABJ), and non-entity positions as "O". Annotations are completed using the LabelStudio tool.

[0106] Step 2: Preprocess the dataset to obtain labeled text dataset and unlabeled text dataset;

[0107] In this embodiment, the acquired engineering machinery assembly data is preprocessed. First, the data text is cleaned, such as by deleting useless characters, including newline characters and spaces.

[0108] The processed text data was labeled using the BIO annotation strategy, resulting in 3012 entities and 4602 relationships, forming the engineering machinery assembly experimental dataset. The dataset was divided into training, validation, and test sets in a 7:2:1 ratio. The training set includes 2108 entity categories, the validation set includes 602 entity categories, and the test set includes 302 entity categories, with no overlap in the entity categories across the training, validation, and test sets. This yields labeled and unlabeled text datasets for the engineering machinery assembly field.

[0109] Step 3: Input the unlabeled text dataset into the pre-trained model, extract source domain features through the feature extraction layer, and optimize the Bert pre-trained model by combining the cue learning loss function and the contrastive learning loss function to obtain the initial pre-trained model;

[0110] In this embodiment, the source domain features of the unlabeled text dataset are extracted by the source domain feature extraction subunit in the feature extraction module, using the cue learning loss function:

[0111] (1) Construct a mask filling template Combined with the engineering machinery assembly text T, which contains the [MASK] mark, such as using [MASK] to connect [MASK] with [MASK];

[0112] (2) Generate the input sequence Input BERT to predict the label word at the [MASK] position;

[0113] (3) Calculate the cue learning loss :

[0114]

[0115] In the formula, Indicates the first The true label of an entity; The model represents the first Predicted labels for each entity; Represents a given input sequence Under the condition that the Bert pre-trained model predicts the first The entity label is The predicted probability;

[0116] Using a contrastive learning loss function, the steps include:

[0117] (1) Construct the bootstrap template C;

[0118] (2) Generate the input sequence Construct positive and negative sample pairs and calculate the contrastive learning loss. :

[0119]

[0120] The loss component for each character sample is defined as follows:

[0121]

[0122] In the formula, The L2 norm of a vector is used to calculate... and and The Euclidean distance between them; The feature vector representing the anchor point sample; The feature vector representing a positive sample. The feature vector representing a negative sample; Indicates temperature parameter; This represents the set of all labels in the dataset; Represents a set The number of markers in the middle.

[0123] By jointly optimizing the cue-based learning loss function and the contrastive learning loss function, the BERT pre-trained model is optimized, ultimately obtaining an initial entity recognition model suitable for the field of engineering machinery assembly.

[0124] Step 4: Input the labeled text dataset into the initial entity recognition model, extract target domain features through the feature extraction layer; fuse the target domain features through the feature fusion layer to obtain fused semantic features;

[0125] In this embodiment, the target domain features are extracted from the labeled text dataset through the target domain feature extraction subunit in the feature extraction module.

[0126] The feature fusion layer fuses features from the target domain, and the steps are as follows:

[0127] (1) Constructing the dictionary tree D and training samples for the engineering machinery assembly domain. ;

[0128] (2) Define the matching function Phrases This indicates that the training samples have been identified. Chinese characters The corresponding phrase;

[0129] (3) Perform nonlinear transformation on the set of phrases:

[0130]

[0131] In the formula, ( ) represents the nonlinear activation function tanh(); Character The corresponding embedding vector; , The weight matrix representing the nonlinear transformation of phrase features; , The bias vector representing the nonlinear transformation of phrase features; Characters obtained after nonlinear transformation The Deep feature representation of each word group;

[0132] (2) Calculate attention score With gate signal :

[0133]

[0134]

[0135] In the formula, Character Context representation vector in a text sequence; Representation of deep features The constructed matrix vector Represents the attention weight matrix; The weight matrix representing the gated signal; The bias vector representing the gate signal; This represents the Sigmoid activation function;

[0136] (3) Calculate attention score and gating signals Fusion weights :

[0137]

[0138] In the formula, Represents the fusion weight matrix. This represents the fusion bias vector. This indicates the attention score. With gate signal To splice;

[0139] (4) Deep feature representation Weighted sum:

[0140]

[0141] In the formula, Character The number of corresponding phrases; Indicates fusion weight The first in Each component corresponds to the j-th word group. The weights;

[0142] (5) Fusion of semantic feature representation:

[0143] .

[0144] After obtaining the fused semantic features, the fused semantic features are input into a bidirectional LSTM network layer for encoding to obtain the context feature sequence. The steps are as follows:

[0145] (1) The fused semantic features are input into the forward LSTM network of the bidirectional LSTM network layer to obtain the forward semantic relevance encoding vector:

[0146]

[0147] In the formula, Indicates the first The forward hidden state output vector at each time step Indicates the first The forward output gate control vector at each time step Indicates the first The forward cell state vector at each time step;

[0148] (2) The fused semantic features are input into the inverse LSTM network of the bidirectional LSTM network layer to obtain the inverse semantic relevance encoding vector:

[0149]

[0150] In the formula, Indicates the first The backward hidden state output vector at each time step Indicates the first The backward output gate control vector at each time step Indicates the first The backward cell state vector at each time step; This represents element-wise multiplication.

[0151] The forward semantic relevance encoding vector and the backward semantic relevance encoding vector are added together to obtain the context feature sequence:

[0152]

[0153] In the formula, This represents the dimension of the hidden vectors in a single LSTM network.

[0154] Based on the context feature sequence, entity span features are generated through a span layer. The span layer includes a span generation submodule and an internal / external feature fusion submodule. The steps are as follows:

[0155] (1) The span generation submodule is used to generate a text with a length of 10 ... The span quantity Us is generated in the engineering machinery assembly text, in the following specific form:

[0156]

[0157] In the formula, Indicates the predefined maximum span length;

[0158] (2) The internal and external feature fusion submodule performs feature fusion based on the span quantity Us, and fuses external and internal features through a gating mechanism to obtain the span feature. The gating mechanism is based on the external features. With internal characteristics The calculation shows that the span feature Defined by the following formula:

[0159]

[0160]

[0161]

[0162] In the formula, The weight matrix represents the external fusion features. The deviation vector represents the external fusion features; Indicates vector concatenation; It represents the set of all labeled embedding vectors within the span; and This represents the learnable weight matrix. This represents the square root of the embedding dimension.

[0163] The steps for generating semantic features of tags through a tag layer are as follows:

[0164] (1) Define the mapping function Λ and construct the tag set. ;

[0165] (2) The initial entity recognition model is used to encode the tag words, and the tag semantic features are obtained through the pooling layer:

[0166]

[0167] In the formula, This represents the weight matrix of the pooling layer; This represents the bias vector of the pooling layer.

[0168] Entity classification is performed by calculating the similarity between the entity span feature and the label semantic feature, and the cross-entropy loss is calculated based on the classification result. Based on the cross-entropy loss, the gradient descent algorithm is used to fine-tune the parameters of the initial entity recognition model. These model parameters include those of the feature extraction layer, feature fusion layer, bidirectional LSTM network layer, span layer, and label layer, resulting in the initial entity recognition model. In this embodiment, the average F1 score is used as the model evaluation metric. The F1 score is the harmonic mean of precision and recall. The initial entity recognition model is evaluated using a validation set of engineering machinery assembly. Training stops when the average F1 score does not improve after five consecutive iterations, thus obtaining the trained entity recognition model. Example 2

[0169] This embodiment describes a system for recognizing assembly entities in engineering machinery, including:

[0170] The acquisition unit is used to acquire the engineering machinery assembly text to be identified;

[0171] An entity recognition unit is used to input the engineering machinery assembly text into a trained entity recognition model and determine the entity category corresponding to each word in the engineering machinery assembly text based on the output of the entity recognition model.

[0172] The entity recognition model includes a feature extraction layer, a feature fusion layer, a bidirectional LSTM network layer, a span layer, and a label layer; the training process of the entity recognition model includes:

[0173] A dataset was constructed based on the collected engineering machinery assembly text.

[0174] The dataset is preprocessed to obtain labeled text dataset and unlabeled text dataset;

[0175] The unlabeled text dataset is input into the BERT pre-trained model. Source domain features are extracted through the feature extraction layer, and the BERT pre-trained model is optimized by combining the cue learning loss function and the contrastive learning loss function to obtain the initial entity recognition model.

[0176] The labeled text dataset is input into the initial entity recognition model, and target domain features are extracted through the feature extraction layer; the target domain features are fused through the feature fusion layer to obtain fused semantic features.

[0177] The bidirectional LSTM network layer encodes the fused semantic features to obtain a context feature sequence;

[0178] Based on the context feature sequence, entity span features are generated through the span layer and label semantic features are generated through the label layer. Entity classification is performed by calculating the similarity between the entity span features and the label semantic features, and cross-entropy loss is calculated based on the classification results.

[0179] Based on the cross-entropy loss, the gradient descent algorithm is used to fine-tune the parameters of the initial entity recognition model to obtain the entity recognition model.

[0180] The training process of the entity recognition model, as well as the implementation of the text processing and recognition functions for engineering machinery assembly in practical applications, are detailed in Example 1. Example 3

[0181] This embodiment describes a storage medium storing instructions that are loaded and executed by a processor to implement the engineering machinery assembly entity recognition method as described in Embodiment 1. Example 4

[0182] This embodiment describes a computer device, which includes a memory, a processor, and instructions stored in the memory. The instructions are loaded and executed by the processor to implement the engineering machinery assembly entity recognition method as described in Embodiment 1. Example 5

[0183] The experimental configuration of the engineering machinery assembly entity recognition system in this implementation is based on Examples 1 to 4.

[0184] I. Experimental Environment Configuration and Dataset

[0185] Hardware environment: The experiment was conducted on a server equipped with an NVIDIA GeForce RTX 3090 GPU and an Intel Core i9-10900K CPU, with 64GB of DDR4 memory to ensure sufficient computing resources.

[0186] Software environment: PyTorch 1.12.0 deep learning framework, HuggingFaceTransformers 4.26.0 natural language processing library, and LabelStudio 1.7.0 annotation tool.

[0187] Key parameter settings for the model: The parameter settings for the entity recognition model are shown in Table 1:

[0188] Table 1 Entity Recognition Model Parameter Settings

[0189]

[0190] Comparison Datasets: In this embodiment, four public datasets and one self-built dataset are used to verify the performance of the entity recognition model. Specific information is shown in Table 2:

[0191] Table 2 Statistical information of the original dataset

[0192]

[0193] II. Comparative Experiments Between Different Models

[0194] To evaluate the performance of the model, a series of comparative experiments were designed to compare the current mainstream NER models with the model proposed in this paper.

[0195] Performance analysis under the same domain setting: In the Few-NERD dataset, the intra-domain experimental results for each class in the test set including 1-shot and 5-shot settings are shown in Tables 3 and 4. All experiments were performed on the same dataset, and the hardware environment and training protocol were kept consistent to ensure fairness.

[0196] Table 3. F1 scores (%) on the Few-Nerd-INTRA dataset

[0197]

[0198] Table 4. F1 scores (%) on the Few-Nerd-INTER dataset

[0199]

[0200] As shown in Tables 3 and 4, the entity recognition model of this invention outperforms the mainstream models provided in terms of average F1 score. Compared with the CONTaiNER model, the entity recognition model achieves an improvement of 0.78% and 1.95% in average F1 score on the Few-Nerd-INTRA and Few-Nerd-INTER datasets, respectively, indicating that the entity recognition model demonstrates strong competitiveness in small-sample named entity recognition tasks. Therefore, the entity recognition model exhibits excellent performance and efficiency, making it suitable for deployment in practical applications of engineering machinery assembly recognition tasks.

[0201] Performance analysis in cross-domain settings: The model was trained on the CMeEE dataset and tested on the entity recognition model on the CoNLL'03, WNUT17 and CMAD2025 test sets, as shown in Tables 5 and 6.

[0202] Table 5. Experimental results of domain migration settings (1-shot)

[0203]

[0204] Table 6. Experimental Results of Domain Migration Setup (5-shot)

[0205]

[0206] As can be seen from Tables 5 and 6, the entity recognition model exhibits the best performance in both the 1-shot and 5-shot settings for each category in the test set. This clearly demonstrates the model's ability to generalize across domain scenarios. Due to the large generalization gap between the training and testing distributions, cross-domain experiments are usually more challenging than intra-domain experiments.

[0207] Table 5 shows that, under the 1-shot setting, the entity recognition model of this invention achieved good performance on the three datasets CoNLL'03, WNUT17 and CMAD2025. Compared with other models, the entity recognition model achieved the best performance in all indicators and led other benchmark models with an average F1 score of 32.61%.

[0208] Table 6 shows that, under 5-shot settings, the entity recognition model of this invention achieves an average F1 score of 41.94%, significantly outperforming other baseline models. Particularly on CMAD2025, the entity recognition model achieves an average F1 score of 24.44%, approximately 2.2 points higher than the second-place CONTaiNER's average F1 score of 22.25%, indicating that the entity recognition model has better generalization ability on complex or emerging domain data.

[0209] III. Ablation Experiment Verification

[0210] To explore the effectiveness of individual structures, multiple ablation experiments were designed based on the Few-Nerd and CMAD2025 datasets. In 1-shot and 5-shot settings, improvements to individual structures were used to evaluate how these structures enhance model performance and how they interact to produce better results. Specifically: Exp1 represents removing the feature fusion layer, Exp2 represents removing cue learning pre-training, Exp3 represents removing contrastive learning pre-training, and Exp4 represents removing the dual encoder structure. See Tables 7 and 8 for details.

[0211] Table 7. F1 scores (%) of ablation experiments on the Few-Nerd dataset

[0212]

[0213] Table 8. F1 scores (%) of ablation experiments on the CMAD2025 dataset

[0214]

[0215] As shown in Tables 7 and 8, Exp1 removed the feature fusion layer. Since the input to the BERT training model only contains character features and cannot utilize word-level prior knowledge, the model struggles to capture the overall meaning of longer string entities, leading to a significant decrease in its ability to recognize complex terms and multi-character entities. Specifically, in the CMAD2025 dataset, the average F1 score of the model without the feature fusion layer decreased by approximately 14.24% compared to the entity recognition model in the 1-shot setting; and by approximately 13.87% in the 5-shot setting. This is mainly because text in the engineering machinery field lacks clear word segmentation boundaries, and dictionary information is crucial for recognizing long entities. Removing this information makes it difficult for the model to fully utilize word semantics, significantly weakening its generalization ability.

[0216] Exp2 removes cue-based pre-training, making it difficult for BERT-trained models to locate entity boundaries and classify them. The model cannot utilize label semantic information to assist prediction. On the 5-shot settings of the Few-Nerd-INTRA, Few-Nerd-INTER, and CMAD2025 datasets, the F1 score of this model decreased by approximately 9.8%, 8.43%, and 18.7% compared to the entity recognition model, respectively. It can be seen that under small sample conditions, it cannot effectively generalize using cue semantics, and the model's recognition performance degrades to varying degrees.

[0217] Removing the contrastive learning pre-training in Exp3 degrades the BERT-trained model's ability to cluster similar samples and distinguish between different types of samples. On the Few-Nerd-INTRA, Few-Nerd-INTER, and CMAD2025 datasets with 5-shot settings, the model's F1 score compared to the entity recognition model decreased by approximately 9.00%, 7.47%, and 16.45%, respectively. It is evident that without contrastive loss pre-training, the BERT-trained model struggles to learn similarity and difference representations between samples, leading to reduced separability of different types of entities in the vector space, thus impacting named entity recognition performance.

[0218] After removing the dual encoder structure in Exp4, it is evident that the average F1 score of the BERT-trained model is significantly lower than that of the entity recognition model. This is especially true when the number of samples is small, such as with only one labeled sample, as the source and target domains cannot share parameters or transfer knowledge to improve generalization ability, resulting in very limited learned features.

[0219] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying assembly entities of engineering machinery, characterized in that, The methods include: Obtain the assembly text of the engineering machinery to be identified; The assembly text of the engineering machinery is input into the trained entity recognition model, and the entity category corresponding to each word in the assembly text of the engineering machinery is determined according to the output of the entity recognition model. The entity recognition model includes a feature extraction layer, a feature fusion layer, a bidirectional LSTM network layer, a span layer, and a label layer; the training process of the entity recognition model includes: A dataset was constructed based on the collected engineering machinery assembly text. The dataset is preprocessed to obtain labeled text dataset and unlabeled text dataset; The unlabeled text dataset is input into the BERT pre-trained model. Source domain features are extracted through the feature extraction layer, and the BERT pre-trained model is optimized by combining the cue learning loss function and the contrastive learning loss function to obtain the initial entity recognition model. The labeled text dataset is input into the initial entity recognition model, and target domain features are extracted through the feature extraction layer; the target domain features are fused through the feature fusion layer to obtain fused semantic features. The bidirectional LSTM network layer encodes the fused semantic features to obtain a context feature sequence; Based on the context feature sequence, entity span features are generated through the span layer and label semantic features are generated through the label layer. Entity classification is performed by calculating the similarity between the entity span features and the label semantic features, and cross-entropy loss is calculated based on the classification results. Based on the cross-entropy loss, the gradient descent algorithm is used to fine-tune the parameters of the initial entity recognition model to obtain the entity recognition model.

2. The method for identifying engineering machinery assembly entities according to claim 1, characterized in that, The preprocessing includes: labeling the text data of the dataset using the BIO annotation method, with annotation types including assembly parts, assembly tools, assembly processes, assembly problems, and design attributes.

3. The method for identifying engineering machinery assembly entities according to claim 1, characterized in that, The steps for optimizing the BERT pre-trained model by combining the cue learning loss function and the contrastive learning loss function include: Construct mask filling template Combined with the engineering machinery assembly text T, an input sequence is generated. Calculate the learning loss. : In the formula, Indicates the first The true label of an entity; The model represents the first Predicted labels for each entity; Represents a given input sequence Under the condition that the Bert pre-trained model predicts the first The entity label is The predicted probability; Construct a bootstrap template C to generate the input sequence. Construct positive and negative sample pairs and calculate the contrastive learning loss. : The loss component for each character sample is defined as follows: In the formula, The L2 norm of a vector is used to calculate... and and The Euclidean distance between them; The feature vector representing the anchor point sample; The feature vector representing a positive sample. The feature vector representing a negative sample; Indicates temperature parameter; This represents the set of all labels in the dataset; Represents a set The number of markers in the text.

4. The method for identifying engineering machinery assembly entities according to claim 3, characterized in that, The step of fusing the target domain features through a feature fusion layer to obtain fused semantic features includes: Constructing the dictionary tree D and training samples for the engineering machinery assembly domain Define the matching function Phrases This indicates that the training samples have been identified. Chinese characters The corresponding phrase; Perform a nonlinear transformation on the set of phrases: In the formula, ( ) represents the nonlinear activation function tanh(); Character The corresponding embedding vector; , The weight matrix representing the nonlinear transformation of phrase features; , The bias vector representing the nonlinear transformation of phrase features; Characters obtained after nonlinear transformation The Deep feature representation of each word group; Calculate attention score With gate signal : ) In the formula, Character Context representation vector in a text sequence; Representation of deep features The constructed matrix vector Represents the attention weight matrix; The weight matrix representing the gated signal; The bias vector representing the gate signal; This represents the Sigmoid activation function; Calculate attention score and gating signals Fusion weights : In the formula, Represents the fusion weight matrix. This represents the fusion bias vector. This indicates the attention score. With gate signal To splice; Deep feature representation Weighted sum: In the formula, Character The number of corresponding phrases; Indicates fusion weight The first in Each component corresponds to the j-th word group. The weights; Fusion semantic feature representation: 。 5. The method for identifying engineering machinery assembly entities according to claim 4, characterized in that, The bidirectional LSTM network layer obtains a context feature sequence by encoding the fused semantic features, and the steps include: The fused semantic features are input into the forward LSTM network of the bidirectional LSTM network layer to obtain a forward semantic relevance encoding vector: In the formula, Indicates the first The forward hidden state output vector at each time step Indicates the first The forward output gate control vector at each time step Indicates the first The forward cell state vector at each time step; The fused semantic features are input into the inverse LSTM network of the bidirectional LSTM network layer to obtain the inverse semantic relevance encoding vector: In the formula, Indicates the first The backward hidden state output vector at each time step Indicates the first The backward output gate control vector at each time step Indicates the first The backward cell state vector at each time step; This indicates element-wise multiplication. The forward semantic relevance encoding vector and the backward semantic relevance encoding vector are added together to obtain the context feature sequence: In the formula, This represents the dimension of the hidden vectors in a single LSTM network.

6. The method for identifying engineering machinery assembly entities according to claim 5, characterized in that, The span layer includes a span generation submodule and an internal and external feature fusion submodule; The span generation submodule is used to generate a text with a length of [missing information]. The span quantity Us is generated in the engineering machinery assembly text, in the following specific form: In the formula, Indicates the predefined maximum span length; The internal and external feature fusion submodule performs feature fusion based on the span quantity Us, fusing external and internal features through a gating mechanism to obtain the span feature. The gating mechanism is based on the external features. With internal characteristics The calculation shows that the span feature Defined by the following formula: In the formula, The weight matrix represents the external fusion features. The deviation vector represents the external fusion features; Indicates vector concatenation; It represents the set of all labeled embedding vectors within the span; and This represents the learnable weight matrix. This represents the square root of the embedding dimension.

7. The method for identifying engineering machinery assembly entities according to claim 6, characterized in that, The step of generating label semantic features through the label layer includes: Define a mapping function Λ to construct a set of tag words. ; The initial entity recognition model is used to encode the tag words, and the tag semantic features are obtained through a pooling layer: In the formula, This represents the weight matrix of the pooling layer; This represents the bias vector of the pooling layer.

8. A system for identifying assembly entities of engineering machinery, characterized in that, include: The acquisition unit is used to acquire the engineering machinery assembly text to be identified; An entity recognition unit is used to input the engineering machinery assembly text into a trained entity recognition model and determine the entity category corresponding to each word in the engineering machinery assembly text based on the output of the entity recognition model. The entity recognition model includes a feature extraction layer, a feature fusion layer, a bidirectional LSTM network layer, a span layer, and a label layer; the training process of the entity recognition model includes: A dataset was constructed based on the collected engineering machinery assembly text. The dataset is preprocessed to obtain labeled text dataset and unlabeled text dataset; The unlabeled text dataset is input into the BERT pre-trained model. Source domain features are extracted through the feature extraction layer, and the BERT pre-trained model is optimized by combining the cue learning loss function and the contrastive learning loss function to obtain the initial entity recognition model. The labeled text dataset is input into the initial entity recognition model, and target domain features are extracted through the feature extraction layer; the target domain features are fused through the feature fusion layer to obtain fused semantic features. The bidirectional LSTM network layer encodes the fused semantic features to obtain a context feature sequence; Based on the context feature sequence, entity span features are generated through the span layer and label semantic features are generated through the label layer. Entity classification is performed by calculating the similarity between the entity span features and the label semantic features, and cross-entropy loss is calculated based on the classification results. Based on the cross-entropy loss, the gradient descent algorithm is used to fine-tune the parameters of the initial entity recognition model to obtain the entity recognition model.

9. A storage medium, characterized in that, The storage medium stores instructions that are loaded and executed by a processor to implement the engineering machinery assembly entity identification method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, The computer device includes: a memory, a processor, and instructions stored in the memory, the instructions being loaded and executed by the processor to implement the engineering machinery assembly entity identification method as described in any one of claims 1 to 7.

Images