A method, device or computer equipment for constructing an assembly process knowledge graph

Abstract

The present invention provides a method, device or computer equipment for constructing an assembly process knowledge graph, which belongs to the technical field of knowledge graph construction, including: obtaining assembly process text record information; constructing an assembly process entity recognition model based on BERT‑CNN‑BiLSTM‑CRF, inputting the assembly process text record information into the assembly process entity recognition model, and outputting a global optimal sequence result; establishing a conceptual model layer APKGSchemaLayer ontology model of APKG according to the assembly process text record information; inputting the global optimal sequence result into the conceptual model layer APKGSchemaLayer ontology model, and establishing a knowledge graph APKG of the assembly process. The method can effectively solve the problem of unclear entity boundary recognition in the named entity recognition task, and greatly improve the named entity recognition performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge graph construction, and specifically relates to a method, device or computer equipment for constructing an assembly process knowledge graph. Background Art

[0002] As the consistency between parts manufacturing and design levels has significantly improved, the focus of product performance assurance has shifted from the "design link" at the front end of the product development process to the "manufacturing link" and then to the "assembly link" at the end. Therefore, assembly, as an important part of the product life cycle, is the main link in realizing product functions. Relevant research shows that about one-third of the human resources in the production process are engaged in activities related to product assembly, and the assembly cost accounts for 40%-50% of the total product manufacturing cost. Therefore, assembly technology occupies an important position in the manufacturing process of complex products such as aircraft and ships.

[0003] Manufacturing companies have accumulated a large amount of product assembly process data and management data during the production process, which carry a wealth of assembly knowledge. For a long time, the rich assembly process data and management data have not been fully utilized. 7 Assembly process planning is a complex decision-making activity. The assembly knowledge that is the basis of assembly process planning mainly comes from the knowledge and experience of process engineers. This traditional assembly process planning activity under human brain intelligence is not conducive to the sharing and reuse of the rich knowledge accumulated by enterprises. At the same time, assembly process planning is mainly based on unstructured text data, which contains the structure of natural language, so it is difficult to infer the expected information from the data. Summary of the invention

[0004] In order to solve the problem that traditional assembly process planning activities are not conducive to the sharing and reuse of the rich knowledge accumulated by enterprises, the present invention provides a method for constructing an assembly process knowledge graph, which integrates the accumulated historical assembly processes and is used to establish a knowledge base in the field of assembly process planning. According to the target assembly scenario, the historical assembly process planning records with the same assembly scenario can be timely inferred. This will help assembly technicians to refer to the assembly process planning of previous assembly scenarios. At the same time, it will be instructive for new assembly technicians with limited knowledge and experience, enabling them to quickly improve their business level and work practice skills.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for constructing an assembly process knowledge graph comprises the following steps:

[0007] Obtain assembly process text record information;

[0008] An assembly process entity recognition model based on BERT-CNN-BiLSTM-CRF is constructed. The assembly process text record information is input into the assembly process entity recognition model to output the global optimal sequence result. The BERT vector representation layer is used to identify the named entities of the assembly process planning information, the CNN-BiLSTM serial neural network layer is used to capture semantic features, and the CRF inference layer outputs the global optimal sequence result based on the optimal probability.

[0009] Establish the APKGSchemaLayer ontology model of the APKG conceptual model layer according to the assembly process text record information;

[0010] The global optimal sequence results are input into the conceptual schema layer APKGSchemaLayer ontology model to establish the knowledge graph APKG of the assembly process.

[0011] Preferably, the BERT of the assembly process entity recognition model encodes words in the assembly process text record information in a bidirectional manner using a Transformer encoder, calculates the relationship between words, and uses the calculated relationship to adjust the weights, extracts features in the text to represent the semantics of the word in the context, and outputs a context embedding vector for each word.

[0012] Preferably, the assembly process entity recognition model uses multiple convolution kernels CNN to perform convolution on the input vector, and obtains a sequential text vector by connecting the vectors of the constituent words, as shown in the following formula:

[0013]

[0014] Where T is the number of words that make up the text; Vectorx i Represents word x i The corresponding vector, i is 1 to T; in the order text vector Vector 1:T Use multiple convolution kernels with different sizes to obtain unigram, bigram, and trigram features of text, capturing the implicit relationship between several adjacent words;

[0015] During the mth convolution, from Vector xk Extend to Vector xk+p The p word vectors are used as input to generate the convolution result W of the segment. d <Vector k:k+p >; Add the activation function ReLU after the convolution layer, input the convolution result into the activation function, and obtain the features of the segment in the p-word information, as shown in the following formula:

[0016] Feature <p，m> =ReLU(W d <Vectork：k+p >+b d )

[0017] Among them, Vector k：k+p represents the embedding vector of p input text words, W d represents the learning of the weight matrix by back-propagation operation, b d Represents the learning of the bias vector by back-propagation.

[0018] Preferably, the assembly process entity recognition model uses a BiLSTM layer of a serial model, in which word information in the context is transmitted through two sub-networks in opposite directions; the forward LSTM unit is used to complete the front-to-back representation of the kth word in the sentence sequence. t→ , the backward LSTM unit is used to complete the word from back to front representation h t← ; The results of the two LSTM units are connected together at the same time as the final output of BiLSTM.

[0019] Preferably, the CRF layer of the assembly process entity recognition model performs comprehensive annotation on the annotation output of the implicit sentence, including two parts: the annotation mark of a single word to a single entity and the transfer mark of a single word to an adjacent word label; the sequence of input sentences is represented as x: = [x1, x2, ..., x T’ ], the label sequence is represented as: L: = [L1, L2, ..., L T’ ], after being processed by the CRF layer, a score matrix M of size T'×M is obtained, where T' represents the total number of imported words and M represents the number of label categories. The relevant scores are calculated through the model:

[0020]

[0021] Where P xi , L i Represents the word x i Related tags: i The probability of appearing in a sentence, A Li，Li+1 Indicates label L i With label L i+1 The probability of switching between

[0022] Then apply softmax to get the normalized probability L of label sequence generation:;

[0023]

[0024] In the formula, L: represents the true value of the label, L:′ represents the estimated value of the label, and L X Represents the total set of possible labels; the following maximum log-likelihood function is used to perform the model training task:

[0025] log P(L): x： |x:)=Score(x:,L: x： )-log(∑ L：′ exp(Score(x: , L: ′)))

[0026] Finally, the Viterbi algorithm is used to obtain the sequence with the best estimation accuracy among all sequences as the final entity recognition annotation result, and the optimal sentence-level annotation sequence is obtained based on Eq.

[0027]

[0028] Preferably, the knowledge graph APKG of the assembly process consists of a conceptual model layer and an entity data layer. The conceptual model layer is built on the entity data layer as a framework structure for storing abstract concepts, knowledge, rules and patterns, and the entity data layer is used to store specific data.

[0029] Preferably, the conceptual model layer includes an assembly structure module, an assembly process module and an assembly resource module.

[0030] The present invention also provides a knowledge-driven assembly process knowledge graph construction device, comprising:

[0031] Process information acquisition module, used to obtain assembly process text record information;

[0032] The process information processing module is used to build an assembly process entity recognition model based on BERT-CNN-BiLSTM-CRF, input the assembly process text record information into the assembly process entity recognition model, and output the global optimal sequence result; the BERT vector representation layer is used to identify the named entities of the assembly process planning information, the CNN-BiLSTM serial neural network layer is used to capture semantic features, and the CRF inference layer outputs the global optimal sequence result based on the optimal probability;

[0033] Conceptual schema layer building module, used to build the conceptual schema layer APKGSchemaLayer ontology model of APKG according to the assembly process text record information;

[0034] The knowledge graph construction module is used to input the global optimal sequence results into the conceptual schema layer APKGSchemaLayer ontology model to establish the knowledge graph APKG of the assembly process.

[0035] The present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-mentioned assembly process knowledge graph construction method.

[0036] The assembly process knowledge graph construction method provided by the present invention has the following beneficial effects:

[0037] The present invention applies the knowledge graph APKG as a new knowledge representation method to the field of assembly process planning. The conceptual model layer of APKG is constructed through a top-down approach as the framework of APKG. For the entity data required for APKG construction, the deep learning model BERT-CNN-BiLSTM-CRF is used for the named entity recognition task. The model can fully learn the global context and local features, and improve the entity recognition rate by fusing the global context and local features, effectively solving the problem of unclear entity boundary recognition in the named entity recognition task, and greatly improving the named entity recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 The overall framework of the assembly process knowledge graph construction method of embodiment 1 of the present invention;

[0040] Figure 2 Identify the basic structure of the model for the assembly process entity;

[0041] Figure 3 This is the structure diagram of the multi-head attention mechanism;

[0042] Figure 4 It is the basic structure for local feature extraction;

[0043] Figure 5 To extract the basic structure of global context features;

[0044] Figure 6 Build graphs for ontology data;

[0045] Figure 7 Comparative results of the entity recognition performance of the models;

[0046] Figure 8 The detailed APKG corresponding to a record of the assembly process. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0048] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the technical solutions of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0049] In addition, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified or limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In the description of the present invention, unless otherwise specified, "plurality" means two or more, which will not be described in detail here.

[0050] Assembly process is an important reference in the product assembly process. The existing assembly process design is too dependent on expert experience, and a large amount of existing assembly process information has not been shared and reused. The assembly process text is described in natural language, which contains many terms and complex relationships in the assembly field, making it difficult for computers to understand. Constructing a professional field knowledge graph of assembly process is an effective way to solve such problems. In order to achieve the effective reuse of assembly processes accumulated by manufacturing enterprises and solve the problem of effective mining and utilization of assembly process knowledge, the present invention proposes a knowledge-driven assembly process knowledge graph (APKG) construction method. In the implementation process of APKG, the construction of the upper conceptual model layer of APKG is constructed in a top-down manner. Named entity (named entity is a proper noun in a sentence) recognition is one of the key links in constructing APKG, and the instance data required for APKG is generated. The present invention adopts a deep learning method to identify the named entities of the assembly process text based on the existing assembly process data. Using the BERT-CNN-BiLSTM-CRF deep neural network, after a large amount of data training, BERT has a stronger ability to process unlabeled data. Compared with traditional models (word2vec, glove, etc.) that can only express vocabulary in a static way, the word vector represented by BERT contains not only its own information but also contextual information. Taking advantage of the unique advantages of CNN and LSTM, a hybrid model of CNN+Bi-LSTM is constructed. This model uses CNN to extract features from different positions in the sentence, thereby reducing the number of input features. Bi-LSTM is used to extract contextual information from the features obtained by CNN. CRF can improve the prediction accuracy of labels by learning the labeling rules of NER. The accuracy acc of the BERT-CNN-BiLSTM-CRF deep neural network in identifying the entities of assembly process text is 96.48%, which is 1.39% higher than the accuracy acc of the BERT-BiLSTM-CRF model. The results show that BERT-CNN-BiLSTM-CRF has good performance in named entity recognition of assembly process text, can effectively determine entity boundaries, and improve the accuracy of named entity recognition tasks. Finally, taking the assembly process records of a company's scanning device equipment as an example, the knowledge graph APKG was constructed to verify the effectiveness of the construction method.

[0051] Figure 1 The main framework of the method of the present invention is shown. The framework is guided by APKG and realizes the unified structured representation of assembly process information from the perspective of assembly process semantics. The framework consists of three parts: entity recognition of assembly process information, conceptual model layer (upper semantic framework) and construction of knowledge graph APKG. Each part is briefly described separately:

[0052] (1) Named entity recognition based on BERT assembly process planning information, the core is the CNN-BiLSTM neural network model, its functions can be divided into three layers: BERT vector representation layer is used to improve feature representation capabilities, CNN-BiLSTM serial neural network layer is used to capture semantic features and CRF inference layer outputs the global optimal sequence results based on the optimal probability. The specific structure is as follows Figure 2 shown.

[0053] (2) APKGSchemaLayer is an assembly process model established from the semantic perspective of assembly process. According to the inherent concepts and natural language structures contained in the assembly process records, the APKG SchemaLayer ontology model of the APKG conceptual model layer is established.

[0054] (3) APKG is built on the basis of the upper semantic framework APKG SchemaLayer and is an indispensable basic carrier between assembly structure class, assembly process class and assembly resource class.

[0055] Specifically, the method for constructing an assembly process knowledge graph provided by the present invention comprises the following steps:

[0056] Step 1: Named entity recognition based on BERT assembly process planning information. The core is the CNN-BiLSTM neural network model, which can be divided into three layers: the BERT vector representation layer is used to improve the feature representation capability, the CNN-BiLSTM serial neural network layer is used to capture semantic features, and the CRF inference layer outputs the global optimal sequence results based on the optimal probability. The specific structure is as follows Figure 2 shown.

[0057] There are many information words such as assembly professional vocabulary, punctuation marks, and conjunctions in the assembly process file, which leads to blurred boundaries of hidden entities. However, accurate identification of entities is the basis for the development of efficient knowledge graphs in the field of assembly process design. In this study, the bidirectional encoder representation from the Transformer (BERT) vector representation layer is used to improve the feature representation ability of hidden records. At the same time, the global context features can be well extracted using the bidirectional long short-term memory (BiLSTM) network, but the basic local features in the sentence may be ignored. CNN can extract local features through convolution operations, solving the problem of BiLSTM lacking local features. The basic features in the corpus are extracted through convolution operations to ensure the integrity of local features. Therefore, the present invention combines the BiLSTM model with the CNN model. At the same time, the conditional random field (CRF) is used to select the optimal probability as the global optimal sequence result. To this end, an assembly process entity recognition model based on BERT-CNN-BiLSTM-CRF is proposed, which can fully consider the characteristics of assembly process records in the assembly process field. Figure 2The basic structure of the model is shown.

[0058] (1) Vector representation of assembly process text records

[0059] BERT uses the Transformer encoder in a bidirectional manner to encode words (words are text in the text), calculate the relationship between words, and use the calculated relationship adjustment weights to extract important features in the text to represent the semantics of the word in the context. Therefore, the output is the context embedding vector of each word. Compared with existing pre-trained models, it captures truly meaningful contextual information and can infer the connection between text fragments. The core of Transformer is the self-attention mechanism, which obtains the relationship between words in the context by calculating the attention between each word in the input sequence, regardless of the distance between words. In this way, the present invention can understand the long-distance dependencies in the sequence. As shown in formula (1), three input word vector matrices are used to calculate attention: QueryVector (Q), Value Vector (V), and Key Vector (K). d k is the dimension of the input vector, and QK t It is used to calculate the relationship between input words.

[0060] The weight representation is obtained through softmax normalization, and the final output is the sum of the weights of all input vectors.

[0061]

[0062] where d k is the dimension of the input vector, d k The role of is to control the dot product of Q and K, not too large, so as to avoid the problem of gradient disappearance after softmax processing.

[0063] In addition, if Figure 3 As shown in Figure 1, Transformer uses multiheadattention, which calculates the attention h at different times from different perspectives using different weight matrices and connects them together.

[0064] MultiHead(Query,Key,Value)=Concat(h1,h2,…,h n ) (2)

[0065] Using the idea of ​​positional encoding, different encodings are added to the same character at different positions, solving the problem of polysemy of a character. Based on the above idea, the BERT embedding layer is selected to add position information to the characters, and its structure consists of token embedding, segment embedding, and position embedding. First is token embedding, which encodes words by adding special tokens to the inputs [CLS] and [SEP] at the beginning and end of each sentence respectively. Secondly, segment embedding is used to encode the sentence position. Thirdly, position embedding encodes the position of the word in the sequence. The positional embedding layer uses sine and cosine functions to encode the position information of the character into a feature matrix. The same word can express different semantics under the feature matrices at different positions, solving the problem of polysemy of the word.

[0066] (2) Extraction of local semantic features from assembly process text records

[0067] CNN has strong generalization ability and fast learning speed. Its basic structure is mainly composed of input layer, convolution layer, activation function, pooling layer, fully connected layer and output layer. Figure 4 As shown. CNN uses convolutional layers to extract low-level semantic features from the network's input text and reduce the dimensionality. A convolutional layer can have one or more convolution kernels. The more convolution kernels there are, the more feature information can be extracted. In the present invention, multiple convolution kernels are used to perform convolution on the input vector.

[0068] The sequential text vector is obtained by connecting the vectors of the constituent words, as shown in formula (3):

[0069]

[0070] In the formula, T is the number of words constituting the text; Represents word x i The corresponding vector, i is 1 to T; in the order text vector Vector 1:T Multiple convolutional kernels with different sizes are used to obtain unigram, bigram, and trigram features of text, aiming to capture the implicit relationship between several adjacent words.

[0071] During the mth convolution, from Vector xk Extend to Vector xk+p The p word vectors can be used as input to generate the convolution result W of the segment d <Vector k:k+p > Add the activation function ReLU after the convolution layer, input the convolution result into the activation function, and obtain the features of the segment in the p-word information, as shown in formula (4):

[0072] Feature <p，m> =ReLU(W d <Vectork：k+p >+b d ) (4)

[0073] Among them, Vector k:k+p represents the embedding vector of p input text words, W d represents the learning of the weight matrix by back-propagation operation, b d Indicates the learning of the bias vector by back propagation. In deep neural networks, using ReLU as an activation function converges faster than using Tanh as an activation function.

[0074] Typically, the input text is convolved to generate a feature map, and a pooling operation (downsampling) is performed to reduce the dimensionality of the feature map space and the amount of data that needs to be processed. The present invention adopts the most widely used maximum pooling layer architecture to design the pooling layer, and uses the maximum value of the features in the pooling area as the feature characterizing the area. Therefore, the main features in the original feature map can be retained, while the secondary features can be ignored.

[0075] (3) Entity tagging and sequence analysis

[0076] The network structure of recurrent neural network (RNN) has a strong advantage in processing sequence data. In theory, RNN is able to process information from long sequences, but in practice, the gradient becomes very large during the back propagation process, resulting in gradient explosion, or the gradient is close to zero, making the gradient disappear after many steps. The use of RNN is more prone to long-distance dependency problems, which will cause the model to be unable to effectively learn global context features, resulting in invalid entity recognition. LSTM is improved on the basis of the recurrent neural network (RNN) model, and consists of one or more units with forgetting and memory functions. It has been proven to be effective in solving gradient explosion, gradient disappearance and long-distance dependency. Since the LSTM model is able to learn previous features rather than future features, the LSTM model cannot process context data at the same time. The BiLSTM model extends the backward knowledge acquisition capability and then generates context information for a given input sequence through a neural network. The specific steps are to connect the derived vectors of the forward hidden layer and the backward hidden layer of the LSTM to generate a global vector. The LSTM unit consists of an input gate mechanism, a forget gate mechanism, and an output gate mechanism, which are used to operate on the preservation and forgetting of information. The forget gate mechanism controls the gradient through it, the input gate mechanism determines the important information available in the current state, and the output gate mechanism determines the state of the next hidden layer. The LSTM network structure is shown in the following equations (5) to (10).

[0077] f t =sigmoid(W f <x t ,h t-1 >+bf ) (5)

[0078] i t =sigmoid(W i <x t ,h t-1 >+b i ) (6)

[0079] o t =sigmoid(W o <x t ,h t-1 >+b o ) (7)

[0080] c t =tanh(W c <X t ,h t-1 >+b c ) (8)

[0081]

[0082]

[0083] Among them, W and b are the variables to be trained in the hidden layer; xt represents the input sequence at time t; it represents the input gate; ft represents the forget gate; ot represents the output gate; represents the point multiplication operation; ⊕ represents the addition operation. In addition, h t Represents the output of the LSTM unit at time t.

[0084] Figure 5 The BiLSTM network architecture for capturing context-critical intra-sentence semantic information is shown. The output of BERT is used to provide deep-level feature representations, such as lexical, semantic, and main sentence-level features. The output is then used as the input of a CNN+BiLSTM serial model to further extract contextual information. The present invention utilizes the BiLSTM layer of the serial model, in which word information in the context is passed through two sub-networks in opposite directions. The forward LSTM unit is used to complete the front-to-back representation ht→ of the kth word in the sentence sequence, and the backward LSTM unit is used to complete the back-to-front representation ht← of the word. The results of these two LSTM units are connected together at the same time as the final output of the BiLSTM.

[0085] The BiLSTM model has the ability to obtain information at a distance, and mainly focuses on obtaining word vectors with contextual information, while ignoring the dependencies between individual tags in the entity recognition task. To this extent, the key stage is to deal with the relationship between tags once the labeled data set is available. The present invention converts it into a sequence labeling problem by adding a CRF layer and a BiLSTM layer. The dependency of tags is solved by adding many mandatory constraints on the sequence output tags. The BOI (Beginning-outside-Inside) mechanism is used to label the corpus. The main advantage of CRF is that some limitations of the output tags that comply with the BOI labeling scheme are discovered, thereby ensuring the accuracy of the predicted sequence tags. The output of the BiLSTM model is probabilistically calculated by CRF, and the label of each character in the sequence text is calculated. Therefore, hidden entity labeling does not classify each phrase in an independent manner, but calculates the optimization probability as the global optimal sequence result, where the global optimal sequence result is what type of each word in a sentence is.

[0086] The CRF layer performs comprehensive annotation on the annotation output of the implicit sentence, taking into account two main components: the annotation mark of a single word to a single entity and the transfer mark of a single word to the adjacent word label. Let the present invention represent the sequence of input sentences as x:=[x1,x2,…,x T’ ], the label sequence is represented as: L: = [L1, L2, ..., L T’ ]. After processing by the CRF layer, a score matrix M of size T'×M is obtained, where T' represents the total number of imported words and M represents the number of tag categories. Therefore, the relevant scores can be calculated through the model.

[0087]

[0088] Where P xi ,L i Represents the word x i Related tags: i The probability of appearing in a sentence, A Li,Li+1 Indicates label L i With label L i+1 Then apply softmax to get the normalized probability L: of label sequence generation.

[0089]

[0090] In the formula, L: represents the true value of the label, L:′ represents the estimated value of the label, and Lx represents the total set of possible labels. The following maximum log-likelihood function is used to perform the model training task:

[0091] log P(L):x： |x:)=Score(x:,L: x： )-log(∑ L：′ exp(Score(x:,L:′))) (13)

[0092] Finally, the Viterbi algorithm is used to obtain the sequence with the best estimation accuracy among all sequences as the final entity recognition annotation result. Based on Eq. (14), the optimal sentence-level annotation sequence is obtained.

[0093]

[0094] Through the above steps, CRF can effectively verify the labels of the columns and improve the recognition accuracy. Therefore, CRF is used as the inference layer of the label position error. Thus, the text sequence x: imported into the model is predicted by the BERT-CNN-BiLSTM-CRF model to obtain the sequence L:*.

[0095] Step 2: APKGSchemaLayer is an assembly process model established from the semantic perspective of assembly process. Based on the inherent concepts and natural language structures contained in the assembly process records, the APKGSchemaLayer ontology model of the APKG conceptual model layer is established.

[0096] Step 3: Establish the assembly process knowledge graph APKG based on the upper semantic framework APKGSchemaLayer. APKG is an indispensable basic carrier between assembly structure class, assembly process class and assembly resource class.

[0097] (1) Basic Definition

[0098] Definition 1: Assembly Process Knowledge Graph (APKG).

[0099] The assembly process knowledge graph is a structured semantic knowledge base of assembly process, with the "entity-relationship-entity" ternary structure as the basic component of the assembly process knowledge graph. The APKG is decomposed into three vertically associated modules, each with its own inherent pattern. The first is the assembly structure information module (Definition 2), which consists of the CAD / MBD assembly structure model that needs to be reasoned. The second is the assembly process knowledge module, which includes expert knowledge based on the assembly CAD model and process personnel. The third is the assembly resource module based on structured assembly tool and equipment data. The entire KG will be integrated by a higher-level conceptual pattern layer that defines the links across subgraphs.

[0100] Definition 2: Assembly structure information model.

[0101] The assembly structure information model is an objective carrier for realizing functions. It is used to express the hierarchical semantic relationship and subordinate semantic relationship between the parts that make up the product, including isConsistedOfSubassembly, isConsistedOfPart, isPartOfSubassembly, and isPartOfAssembly. The assembly structure information model consists of geometric information and annotation information. The formula is as follows:

[0102] ASIM=Σ i＝1 G i ∪Σ j＝1 Ano j (15)

[0103] The geometric information Gi includes assembly features, positioning features and geometric surfaces, while the annotation information Anoj includes material properties, annotation information, positioning datum, assembly datum and other properties.

[0104] Definition 3: Assembly process information model APIM.

[0105] Combining the assembly process information contained in the assembly process document and the assembly CAD / MBD model, the assembly process information model is described as:

[0106] APIM={AssSeq,AssCon,AssOpe,FitType,TAssProced} (16)

[0107] Among them, AssSeq represents assembly sequence; AssCon represents assembly constraint relationship, including Contact, Orientation, PointAlignment, LineAlignment, PlaneAlignment, CoordinateSystemAlignment, Equidistance, Coaxial, Insertion; AssOpe represents assembly operation, including BeingPlacedOn, LoadingInto, Matching, Inserting, ScrewingIn; FitType represents fit type, including ClearanceFit, TransitionFit, InterferenceFit. TAssProced represents typical assembly procedure, including Conveying, Positioning, Connecting, Adjusting, Inspecting.

[0108] Definition 4: Assembly resource model.

[0109] The non-product assembly elements used to ensure the quality of parts assembly, limit the connection deformation of the assembly process, and ensure accuracy and interchangeability during the assembly process include three aspects of information: assembly resource structure ARStr, assembly resource process ARPro, and assembly resource management ARMan. The formula is as follows:

[0110] ARM={ARStr,ARPro,ARMan} (17)

[0111] Among them, the assembly resource structure information ARStr includes topology information, work area size information, structure configuration, outline size information, etc. The assembly resource process information ARPro includes the scope of use, positioning and clamping method, assembly accuracy, energy consumption, etc. The assembly resource management ARMan includes model information, equipment location, number, current status, etc.

[0112] (2) Construction of the knowledge graph APKG

[0113] As a method of representing knowledge in the form of a graph structure, the knowledge graph is essentially a semantic network of relationships between entities. The nodes in the knowledge graph represent physical objects or abstract concepts in the real world, and semantic relationships are established between different nodes through edges. The knowledge graph has rich semantic representation capabilities and a flexible knowledge / information representation structure, and is an effective repository for assembly process information / knowledge.

[0114] In order to construct the assembly process knowledge graph APKG, the assembly process information is represented in the form of triples, which depend on the assembly structure-assembly resources-assembly process in the assembly process. ARKG consists of a conceptual model layer and an entity data layer. The conceptual model layer is built on the entity data layer as a framework structure to store abstract concepts, knowledge, rules and patterns, and the entity data layer is used to store specific data. In order to ensure the correctness and consistency of the concepts of the model layer in APKG, the present invention adopts a top-down approach to classify the concepts and hierarchical relationships of the model layer. Then, the entity is added to the knowledge base under the predefined model.

[0115] Since the goal of APKG is to provide process designers with the assembly process knowledge required for the product assembly process, the assembly process knowledge must be related to the specific assembly process served and the assembly resources needed to assemble the product structure in the process. Therefore, the APKG conceptual model layer consists of three dimensions, namely assembly structure, assembly process, and assembly resources.

[0116] In the conceptual model layer of APKG, the assembly resource dimension covers the relevant concepts of assembly resource knowledge representation, such as multi-domain knowledge of assembly resource capabilities, dimension structure, materials, etc. The assembly resource dimension establishes three subclasses through inheritance, namely resource structure information, resource process information, and resource management information, which are the perspectives from which process designers search for assembly resource knowledge. For example, the concept of resource structure information is further defined through subconcepts such as structure topology information, structure material information, and structure configuration information. The assembly process dimension creates subclasses through inheritance operations, where the inheritance subclasses of typical assembly processes include conveying, positioning, connection, adjustment, and inspection. Concepts such as marking process, punching process, screwing process, and riveting process are subconcepts that further define the connection process category. Similarly, concepts such as adjustment and inspection create their own subconcepts through inheritance operations. In particular, the specific definition of the adjustment process is given to the flattening process, the final processing process, and the painting process; and the specific detection concept definition is given to the level measurement, static balance test, dynamic balance test, and rainfall test.

[0117] The associative relationships between concepts in the schema layer are crucial to APKG. A list of relationships corresponding to the three-dimensional categories is deployed. As shown in Table 1, the relationship name, domain, and scope are tabulated.

[0118] Table 1 Relationship in APKG

[0119]

[0120] A bottom-up approach is used for the knowledge instantiation process. On the basis of the conceptual model layer, the assembly process entities are instantiated to the entity data layer through automatic extraction and manual intervention of the assembly process entities to obtain the process information written by experts and process designers. The knowledge instantiation process is represented by a one-to-many mapping relationship [Cla:ins], where Cla is a concept type, ins is an instance of the concept Cla, and multiple instances of the same concept Cla are allowed. Then, entities with high confidence are added to the knowledge base to complete the construction of the assembly process knowledge base. The APKG conceptual model layer and entity data layer constructed by combining the above two methods are shown in Figure 1. Figure 6 shown.

[0121] Figure 6 The framework of the APKG model layer and the upper-level concepts on the left are given, including the assembly structure module, assembly process module and assembly resource module. Figure 6 On the right is the physical data layer of APKG, which is obtained through the analysis of the assembly process records of a company's scanning equipment.

[0122] Given that the graph structure is a set of semantically related concepts, the first task in building APKG is to generate a triple that describes the relationship between a pair of concepts, and represents the triple in the form of <entity, relationship, entity>. The mapping of the ontology model to the knowledge graph storage container Neo4j is performed in the form of triples, which supports the storage of knowledge and the filling of data.

[0123] Based on the same inventive concept, the present invention also improves a knowledge-driven assembly process knowledge graph construction device, including a process information acquisition module, a process information processing module, a concept model layer construction module and a knowledge graph construction module.

[0124] Specifically, the process information acquisition module is used to obtain the assembly process text record information. The process information processing module is used to build an assembly process entity recognition model based on BERT-CNN-BiLSTM-CRF, input the assembly process text record information into the assembly process entity recognition model, and output the global optimal sequence result; wherein the BERT vector representation layer is used to identify the named entities of the assembly process planning information, the CNN-BiLSTM serial neural network layer is used to capture semantic features, and the CRF inference layer outputs the global optimal sequence result based on the optimal probability. The concept pattern layer construction module is used to establish the concept pattern layer APKGSchemaLayer ontology model of APKG based on the assembly process text record information. The knowledge graph construction module is used to input the global optimal sequence result into the concept pattern layer APKGSchemaLayer ontology model to establish the knowledge graph APKG of the assembly process.

[0125] Each module in the above-mentioned knowledge-driven assembly process knowledge graph construction device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0126] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the assembly process knowledge graph construction method embodiment. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0127] Example 1

[0128] To verify the effectiveness of this method, an experimental analysis was conducted using the scanning device assembly process record of a company as an example. This invention is an experimental research work based on the Python programming language. The configuration of the experimental platform is I7-8750CPU (2.20GHz), GTX3090GPU, Python3.7.8, PyTorch1.13.0, CUDA11.1, and operating system Ubuntu20.04.

[0129] Based on the deep learning framework of the PyTorch platform, the accuracy (acc), precision (p), recall (r) and F1 score (f1Score) evaluation indicators are used to evaluate the recognition effect of the assembly process entity recognition model. In the assembly process text information of the scanning device equipment, the longest sentence consists of 240 characters, so the maximum sentence length (MAX_LEN) is set to 256 characters, and other related settings are shown in Table 2.

[0130] Table 2 Experimental parameters

[0131]

[0132] (1) Performance analysis of assembly process entity recognition model

[0133] By using the scanning device equipment assembly process text information annotated with the corpus, the BERT-CNN-BiLSTM-CRF-based entity recognition model and the BERT-BiLSTM-CRF-based entity recognition model adopted in the present invention are applied to the assembly process text information, and the effects of entity recognition in the assembly process information are compared and analyzed.

[0134] like Figure 7 As shown in the figure, the comparison effect of model entity recognition is shown. The BERT-CNN-BiLSTM-CRF entity recognition model adds a CNN on the basis of the BERT-BiLSTM-CRF model, which improves the model recognition performance. It is improved by about 8% in F1, about 11% in R, and about 5% in P. The accuracy acc of the BERT-CNN-BiLSTM-CRF model is 96.48%; the accuracy acc of the BERT-BiLSTM-CRF model is 95.09%. After adding CNN, the accuracy acc of entity recognition increased by 1.39%.

[0135] (2) Constructing the knowledge graph of assembly process APKG

[0136] According to the proposed knowledge graph construction method, the construction of APKG is realized. It takes assembly structure, assembly process and assembly resource as nodes to form assembly structure class, assembly process class and assembly resource class that can be stored in Neo4j. The focus is to use the BERT-CNN-BiLSTM-CRF neural network model to parse the entities involving assembly structure, assembly resource and assembly process in the assembly process document and perform the named entity recognition task.

[0137] The present invention takes a record in an assembly process file as an example, and stipulates that: "After the lead plate is installed on the protective lead box of the two horizontal and vertical detector arms and the collimator, the lead is melted so that the lead plate is firmly connected to the surrounding frame, and no gap is allowed. The deformation of the lead box shall not exceed 2mm." Such assembly process information is identified by named entity recognition, which identifies the entities: (1) AssemR: lead plate, (2) AssemS: detector arm, (3) AssemS: collimator, (4) AssemS: protective lead box, (5) AssemS: frame, (6) AssemP: lead box deformation, (7) AssemP: 2mm. The 7 entities identified correspond to the three categories of assembly structure, assembly process and assembly resources, and corresponding nodes and association relationships between nodes are created. For example, a hierarchical affiliation is established between the node AssemS: detector arm and the node AssemS: protective lead box. An assembly constraint relationship is established between the node 'AssemS: protective lead box' and the node 'AssemP: lead box deformation'. APKG details are shown as follows: Figure 8 shown.

[0138] The BERT-CNN-BiLSTM-CRF model extracts named entities in assembly process documents. The extraction results of some assembly process records are shown in Table 3. After obtaining the composition of the corresponding class in each assembly process record, the present invention stores the extracted entities in the form of nodes and establishes corresponding relationships. An assembly process document is converted from text form to a knowledge graph form and saved. The entire knowledge graph APKG is constructed in this way.

[0139] The assembly process records of the scanning device equipment are constructed in the form of a knowledge graph, which graphically represents the association between the knowledge in the assembly process. APKG can use one or more combinations of the three vertical association categories of assembly structure, assembly process, and assembly resources as search objects, and provide existing relevant assembly process records to assist assembly process designers in designing assembly processes for new assembly structures and provide knowledge support for the implementation of subsequent projects.

[0140] Table 3 Named entity extraction results of some assembly process records

[0141]

[0142] In summary, the present invention applies the knowledge graph APKG as a new knowledge representation method to the field of assembly process planning. The conceptual model layer of APKG is constructed by a top-down method as the framework of APKG. For the entity data required for APKG construction, the deep learning model BERT-CNN-BiLSTM-CRF is used to perform the named entity recognition task. The model can fully learn the global context and local features, and improve the entity recognition rate by fusing the global context and local features. By adding labels to the knowledge in the field of assembly process, the problem of unclear entity boundary recognition in the named entity recognition task is effectively solved, and the named entity recognition performance is greatly improved. Taking the assembly process record of a company's scanning device equipment as an example, the knowledge graph APKG is constructed. Using the BERT-CNN-BiLSTM-CRF deep neural network, the accuracy rate acc of identifying assembly process text entities is 96.48%, which is 1.39% higher than the accuracy rate acc of the BERT-BiLSTM-CRF model. The precision p is 81.93%, the recall rate r is 84.96%, and the F1 score f is 83.42%.

[0143] In practical applications, it is possible to consider integrating data from multiple sources and types into the assembly process knowledge graph APKG to automatically expand the capacity of the knowledge graph.

[0144] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] It should be noted that the above-described specific implementation methods can enable those skilled in the art to more fully understand the invention, but do not limit the invention in any way. Therefore, although the present specification and embodiments have described the invention in detail, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

[0149] The embodiments described above are only preferred specific implementation modes of the present invention, and the protection scope of the present invention is not limited thereto. Any simple changes or equivalent replacements of the technical solutions that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belong to the protection scope of the present invention.

Claims

1. A method for constructing an assembly process knowledge graph, characterized in that: The following steps are involved: Obtain assembly process text record information; An assembly process entity recognition model based on BERT-CNN-BiLSTM-CRF is constructed. The assembly process text record information is input into the assembly process entity recognition model to output the global optimal sequence result. The BERT vector representation layer is used to identify the named entities of the assembly process planning information, the CNN-BiLSTM serial neural network layer is used to capture semantic features, and the CRF inference layer outputs the global optimal sequence result based on the optimal probability. Establish the APKGSchemaLayer ontology model of the APKG conceptual model layer according to the assembly process text record information; The global optimal sequence results are input into the conceptual schema layer APKGSchemaLayer ontology model to establish the knowledge graph APKG of the assembly process; The BERT of the assembly process entity recognition model encodes words in the assembly process text record information in a bidirectional manner using a Transformer encoder, calculates the relationship between words, adjusts weights using the calculated relationship, extracts features in the text to represent the semantics of the word in the context, and outputs a context embedding vector for each word.

2. The method for constructing an assembly process knowledge graph according to claim 1, characterized in that: The assembly process entity recognition model uses multiple convolution kernels CNN to perform convolution on the input vector, and obtains a sequential text vector by connecting the vectors of the constituent words, as shown in the following formula: In the formula, T is the number of words constituting the text; Represents word x i The corresponding vector, i is 1 to T; in the order text vector Vector 1:T Use multiple convolution kernels with different sizes to obtain unigram, bigram, and trigram features of text, capturing the implicit relationship between several adjacent words; During the mth convolution, from Vector xk Extend to Vector xk+p The p word vectors are used as input to generate the convolution result W of the segment. d <Vectork:k+p> ; Add the activation function ReLU after the convolution layer, input the convolution result into the activation function, and obtain the features of this segment in the p-word information, as shown in the following formula: Feature <p,m> =ReLU(W d <Vector k：k+p >+b d ) Among them, Vector k:k+p represents the embedding vector of p input text words, W d represents the learning of the weight matrix by back-propagation operation, b d Represents the learning of the bias vector by back-propagation.

3. The method for constructing an assembly process knowledge graph according to claim 2, characterized in that: The assembly process entity recognition model utilizes the BiLSTM layer of the serial model, wherein word information in the context is transmitted through two sub-networks in opposite directions; the forward LSTM unit is used to complete the front-to-back representation ht→ of the kth word in the sentence sequence, and the backward LSTM unit is used to complete the back-to-front representation ht← of the word; the results of the two LSTM units are simultaneously connected together as the final output of the BiLSTM.

4. The method for constructing an assembly process knowledge graph according to claim 1, characterized in that: The CRF layer of the assembly process entity recognition model performs comprehensive annotation on the annotation output of implicit sentences, including two parts: the annotation mark of a single word to a single entity and the transfer mark of a single word to adjacent word labels; the sequence of input sentences is represented as x:=[x1,x2,…,x T’ ], the label sequence is represented as: L: = [L1, L2, ..., L T’ ], after being processed by the CRF layer, a score matrix M of size T'×M is obtained, where T' represents the total number of imported words and M represents the number of label categories. The relevant scores are calculated through the model: Where P xi ,L i Represents the word x i Related tags: i The probability of appearing in a sentence, A Li,Li+1 Indicates label L i With label L i+1 The probability of switching between Then apply softmax to get the normalized probability L: of label sequence generation; In the formula, L: represents the true value of the label, L:′ represents the estimated value of the label, and L x Represents the total set of possible labels; the following maximum log-likelihood function is used to perform the model training task: log P(L: x： |x:)=Score(x:,L: x： )-log(Σ L：′ exp(Score(x:,L:'))) Finally, the Viterbi algorithm is used to obtain the sequence with the best estimation accuracy among all sequences as the final entity recognition annotation result, and the optimal sentence-level annotation sequence is obtained based on Eq.

5. The assembly process knowledge graph construction method according to claim 1, characterized in that: The knowledge graph APKG of the assembly process consists of a conceptual model layer and an entity data layer. The conceptual model layer is built on the entity data layer as a framework structure for storing abstract concepts, knowledge, rules and patterns, and the entity data layer is used to store specific data.

6. The method for constructing an assembly process knowledge graph according to claim 5, characterized in that: The conceptual model layer includes an assembly structure module, an assembly process module and an assembly resource module.

7. A knowledge-driven assembly process knowledge graph construction device, characterized in that: include: Process information acquisition module, used to obtain assembly process text record information; The process information processing module is used to build an assembly process entity recognition model based on BERT-CNN-BiLSTM-CRF, input the assembly process text record information into the assembly process entity recognition model, and output the global optimal sequence result; the BERT vector representation layer is used to identify the named entities of the assembly process planning information, the CNN-BiLSTM serial neural network layer is used to capture semantic features, and the CRF inference layer outputs the global optimal sequence result based on the optimal probability; Conceptual schema layer building module, used to build the conceptual schema layer APKGSchemaLayer ontology model of APKG according to the assembly process text record information; The knowledge graph construction module is used to input the global optimal sequence results into the conceptual schema layer APKGSchemaLayer ontology model to establish the knowledge graph APKG of the assembly process.

8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

Images