Onboard electromechanical product parameter traceability relation extraction method based on PCNN (Pulse Coupled Neural Network) model

Abstract

An airborne electromechanical product parameter traceability relation extraction method based on a PCNN model is characterized by comprising the steps that S1, the PCNN model is adopted as a feature extractor, segmented processing is conducted on an input sequence, attention of the model to different semantic areas is enhanced, and segmented coding input is conducted through a segmented pooling strategy; s2, introducing a multi-head attention mechanism layer into the model, and capturing diversified features of an input sequence through parallel computing of a plurality of independent attention heads; and S3, designing a full connection layer as a relation classifier. According to the method, the requirement for extracting the text relation of the airborne aircraft product can be met, efficient metering support is provided for metering personnel, the traceability requirement of the product quantity value is guaranteed, and application of the knowledge graph in the field of parameter traceability is promoted.

Description

Technical Field

[0001] The present invention relates to a natural language processing technology for airborne electromechanical product data, in particular to a parameter relationship recognition method, specifically to a method for extracting airborne electromechanical product parameter traceability relationships based on a PCNN model. Background Art

[0002] Airborne electromechanical products are core components for realizing various functions of aircraft, and are widely used in many systems of various types of aircraft. The performance level of these products is not only a key indicator for measuring the overall performance of the aircraft, but also directly affects the operating efficiency, reliability, safety and maintenance cost of the aircraft. At present, product parameter traceability files are usually stored as unstructured text and extracted as parameter traceability chains through manual queries. The parameter traceability process is extremely dependent on the experience of metrologists, which is not only time-consuming, but also difficult to ensure high data accuracy, thus posing potential risks to the safe operation of aircraft. Due to the high degree of specialization of the data entities and relationship names of airborne electromechanical products, their semantic features are difficult to extract correctly. Existing models are difficult to accurately infer the relationship between entities, and are not suitable for relationship extraction in the field of airborne electromechanical products. They have poor generalization ability and low accuracy.

[0003] Therefore, it is necessary to improve the existing relationship extraction model to overcome the shortcomings of the existing technology and expand its application in the field of airborne mechatronics products. Summary of the invention

[0004] The purpose of the present invention is to address the problem that the highly specialized names of airborne mechatronic product data entities and relationships make it difficult to correctly extract their semantic features and that existing models find it difficult to accurately infer the relationships between entities. A method for extracting airborne mechatronic product relationships based on a PCNN model is invented to solve the problems raised in the above background technology.

[0005] The technical solution of the present invention is:

[0006] A method for extracting airborne electromechanical product relations based on a PCNN model is characterized by comprising the following steps:

[0007] S1: Design a relation extraction model AeroPCNN-MultiAtt for airborne mechatronic products. Use the PCNN model as a feature extractor. By segmenting the input sequence, the model's attention to different semantic regions is enhanced. A segment pooling strategy is used for segment encoding input.

[0008] S2: A multi-head attention mechanism layer is introduced into the model to capture the diverse features of the input sequence by computing multiple independent attention heads in parallel.

[0009] S3: Design a fully connected layer as a relation classifier to receive the feature vector output by the multi-head attention mechanism layer and further process it for the relation classification task.

[0010] The step S1 includes the following contents:

[0011] a1: The input layer of the model needs to map each word in the input sentence into a vector, which is mainly composed of two parts: the combined word vector and the word relative position vector. For a sentence S = {w 1 ,w 2 ,...,w n}, where each word w i Mapped from the word vector matrix to a real-valued vector e i :

[0012] e i =w i ·V i

[0013] Where: is the word vector of the i-th word, d w is the dimension of the word vector, V i is the one-hot representation of the input word.

[0014] a2: In order to make full use of the semantic information of the sentence, the position vector is concatenated into the word vector representation of the word. If the word vector dimension is d w , the position vector dimension is d p , then the final vector sequence is:

[0015] x={x 1 ,x 2 ,…,x n}

[0016] Where: x i ∈R n×d (d=d w +d p ×2).

[0017] a3: The main function of the convolutional layer is to extract local features from the input sequence. Assume that the input sequence is the word vector matrix x i ∈R d , the model mainly extracts features through one-dimensional convolution operation. The convolution operation can be expressed as:

[0018] c i =f(W·X i:i+k-1 +b)

[0019] Where: W∈R k×d is the convolution kernel, K is the width of the convolution kernel, Xi:i+k-1 represents the submatrix from the i-th word to the i+k-1-th word in the input matrix, b is the bias term, and f is the nonlinear activation function.

[0020] a4: AeroPCNN introduces a segmented pooling operation after the convolutional layer. The purpose of segmented pooling is to divide the output feature vector C of the convolutional layer into three segments according to the positions of the two target entities: the front segment (before the left entity), the middle segment (between the two entities), and the back segment (after the right entity). Specifically, assuming that the position index of the left entity is p 1 , the position index of the right entity is p 2 , then the convolution feature vector C can be divided into three parts:

[0021]

[0022]

[0023] Then perform the maximum pooling operation on each segment separately:

[0024] p 1 =max(C 前段 )

[0025] p 2 =max(C 中段 )

[0026] p 3 =max(C 后段 )

[0027] The result of the three-stage pooling p 1 ,p 2 ,p 3 Spliced ​​together to form an overall feature vector P∈R 3m , where m is the dimension of each pooling output. This feature vector contains the salient features of different parts of the sentence at the same time, which helps the model understand and classify the relationship between two entities in the sentence. The formula for the splicing operation is:

[0028] P = [p 1 ;p 2 ;p 3 ]

[0029] In the formula: ; represents the concatenation operation of the vector. The concatenated vector P will be used as the input of the subsequent multi-head attention mechanism layer.

[0030] The step S2 includes the following contents:

[0031] b1: The feature vector output by the AeroPCNN layer is P. First, we need to calculate the query matrix Q, key matrix K and value matrix V:

[0032] Q=PW Q

[0033] K=PW K

[0034] V=PW V

[0035] Formula: W Q , W K , W V is the weight matrix.

[0036] b2: The core of AeroPCNN-MultiAtt is to determine the contribution of each element in the final feature representation by calculating the correlation (i.e., attention weight) between each element in the sequence (here, the word vector) and other elements. The calculation formula of attention weight is:

[0037]

[0038] Where A is the attention weight matrix, which represents the correlation between each position in the sequence and other positions. T is the dot product of the query matrix and the key matrix, representing the similarity between the query and the key. It is a scaling factor used to prevent the dot product value from being too large, resulting in a smaller gradient output by the softmax function.

[0039] b3: After obtaining the attention weight matrix A, we use these weights to perform weighted summation on the value matrix to generate a new feature representation. The formula for weighted summation is:

[0040] P=AV

[0041] Among them, O is the new feature representation matrix, and V is the value matrix. Through this operation, the feature representation of each position not only contains its own information, but also integrates the information of other positions, which is particularly effective for processing long-distance dependencies and complex syntactic structures.

[0042] The advantage of the multi-head attention mechanism is that it can process multiple independent attention heads at the same time, each head analyzing the input data from a different perspective. Assuming we have h attention heads, the outputs of these heads will be spliced ​​together to form the final multi-head attention output. Specifically, we need to splice the output O of each attention head together:

[0043] O multi-head =[o (1) ;o (2) ;…;O (h) ]W O

[0044] Among them, multi-headis the concatenated multi-head attention output matrix, W O is a linear transformation matrix used to map the concatenated output back to the original feature dimension.

[0045] The step S3 includes the following contents:

[0046] c1: The fully connected layer receives the output feature vector O of the multi-head attention mechanism module multi-head , and further processed for relation classification tasks. The calculation formula of the fully connected layer is:

[0047] h=f(W f ·O multi-head +b f )

[0048] Where: W f is the weight matrix of the fully connected layer, b f is the bias term and f is the activation function.

[0049] c2: The output layer uses the softmax function to normalize the output of the fully connected layer to generate the probability distribution of the relationship classification. The formula of the softmax layer is:

[0050]

[0051] Where: W O and b o is the weight matrix and bias term of the output layer, Represents the predicted relationship type probability distribution.

[0052] The beneficial effects of the present invention are:

[0053] 1. In order to more effectively extract the relationship between entities in the product parameter traceability text, this paper proposes a relationship extraction model that combines the multi-head attention mechanism with the PCNN model. This model uses PCNN to segment the input sequence and enhance the attention to different semantic areas.

[0054] 2. In order to address the shortcomings of existing models in processing long text data, this paper introduces a multi-head attention mechanism to capture the diverse features of the input sequence and effectively extract the relationship between entities in long texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of the relationship extraction method of airborne mechatronic product parameter traceability of the present invention.

[0056] Figure 2 It is a structural diagram of the AeroPCNN-MultiAtt model of the present invention.

[0057] Figure 3It is a structural diagram of the AeroPCNN-MultiAtt model feature extractor of the present invention.

[0058] Figure 4 It is an experimental comparison diagram of the model of the present invention and other methods. DETAILED DESCRIPTION

[0059] The following will be combined with the embodiments of the present invention and the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] See also Figure 1-4 .

[0061] A method for extracting airborne electromechanical product relations based on the PCNN model, such as Figure 1 As shown, the following steps are included:

[0062] S1: Design a relation extraction model AeroPCNN-MultiAtt for the field of airborne mechatronic products. It uses the PCNN model as a feature extractor, segments the input sequence, enhances the model's attention to different semantic regions, and uses a segment pooling strategy for segment encoding input.

[0063] S2: A multi-head attention mechanism layer is introduced into the model to capture the diverse features of the input sequence by computing multiple independent attention heads in parallel.

[0064] S3: Design a fully connected layer as a relation classifier to receive the feature vector output by the multi-head attention mechanism layer and further process it for the relation classification task.

[0065] in:

[0066] Step S1: Design a relation extraction model AeroPCNN-MultiAtt for the field of airborne electromechanical products:

[0067] like Figure 2 As shown in the figure, the input layer of the model needs to map each word in the input sentence into a vector, which is mainly composed of two parts: the combined word vector and the word relative position vector. 1 ,w 2 ,...,w n}, where each word w i Mapped from the word vector matrix to a real-valued vector e i :

[0068]

[0069] Where: is the word vector of the i-th word, d w is the dimension of the word vector, V i is the one-hot representation of the input word.

[0070] In order to make full use of the semantic information of the sentence, the position vector is concatenated into the word vector representation of the word. w , the position vector dimension is d p , then the final vector sequence is:

[0071] x={x 1 ,x 2 ,…,x n}

[0072] Where: x i ∈R n×d (d=d w +d p ×2).

[0073] The main function of the convolutional layer is to extract local features from the input sequence. Assume that the input sequence is the word vector matrix x i ∈R d , the model mainly extracts features through one-dimensional convolution operation. The convolution operation can be expressed as:

[0074] c i =f(W·X i:i+k-1 +b)

[0075] Where: W∈R k×d is the convolution kernel, K is the width of the convolution kernel, X i:i+k-1 represents the submatrix from the i-th word to the i+k-1-th word in the input matrix, b is the bias term, and f is the nonlinear activation function.

[0076] AeroPCNN introduces a segmented pooling operation after the convolutional layer. The purpose of segmented pooling is to divide the output feature vector C of the convolutional layer into three segments according to the positions of the two target entities: the front segment (before the left entity), the middle segment (between the two entities), and the back segment (after the right entity). Specifically, assuming that the position index of the left entity is p 1 , the position index of the right entity is p 2 , then the convolution feature vector C can be divided into three parts:

[0077]

[0078]

[0079] Then perform the maximum pooling operation on each segment separately:

[0080] p 1 =max(C 前段 )

[0081] p 2 =max(C 中段 )

[0082] p 3 =max(C 后段 )

[0083] The result of the three-stage pooling p 1 ,p 2 ,p 3 Spliced ​​together to form an overall feature vector P∈R 3m , where m is the dimension of each pooling output. This feature vector contains the salient features of different parts of the sentence at the same time, which helps the model understand and classify the relationship between two entities in the sentence. The formula for the splicing operation is:

[0084] P = [p 1 ;p 2 ;p 3 ]

[0085] In the formula: ; represents the concatenation operation of the vector. The concatenated vector P will be used as the input of the subsequent multi-head attention mechanism layer.

[0086] Step S2: Design a multi-head attention mechanism layer to capture the diverse features of the input sequence:

[0087] The output feature vector of the AeroPCNN layer is P. First, we need to calculate the query matrix Q, key matrix K and value matrix V:

[0088] Q=PW Q

[0089] K=PW K

[0090] V=PW V

[0091] Where: W Q , W K , W V is the weight matrix.

[0092] The core of AeroPCNN-MultiAtt is to determine the contribution of each element in the final feature representation by calculating the correlation (i.e., attention weight) between each element in the sequence (here, the word vector) and other elements. The calculation formula of attention weight is:

[0093]

[0094] Where: A is the attention weight matrix, which represents the correlation between each position in the sequence and other positions. T is the dot product of the query matrix and the key matrix, representing the similarity between the query and the key. It is a scaling factor used to prevent the dot product value from being too large, resulting in a smaller gradient output by the softmax function.

[0095] After obtaining the attention weight matrix A, we use these weights to perform weighted summation on the value matrix to generate a new feature representation. The formula for weighted summation is:

[0096] O=AV

[0097] Among them, O is the new feature representation matrix, and V is the value matrix. Through this operation, the feature representation of each position not only contains its own information, but also integrates the information of other positions, which is particularly effective for processing long-distance dependencies and complex syntactic structures.

[0098] The advantage of the multi-head attention mechanism is that it can process multiple independent attention heads at the same time, each head analyzing the input data from a different perspective. Assuming we have h attention heads, the outputs of these heads will be spliced ​​together to form the final multi-head attention output. Specifically, we need to splice the output O of each attention head together:

[0099] O multi-head =[O (1) ;O (2) ;…;O (h) ]W O

[0100] Among them, multi-head is the concatenated multi-head attention output matrix, W O is a linear transformation matrix used to map the concatenated output back to the original feature dimension.

[0101] Step S3: Design a fully connected layer to receive the feature vector output by the multi-head attention mechanism layer:

[0102] like Figure 3 As shown, the fully connected layer receives the output feature vector O of the multi-head attention mechanism module multi-head , and further processed for relation classification tasks. The calculation formula of the fully connected layer is:

[0103] h=f(W f ·O multi-head +b f )

[0104] Where: W f is the weight matrix of the fully connected layer, bf is the bias term and f is the activation function.

[0105] The output layer uses the softmax function to normalize the output of the fully connected layer to generate the probability distribution of relationship classification. The formula of the softmax layer is:

[0106]

[0107] Where: W O and b o is the weight matrix and bias term of the output layer, Represents the predicted relationship type probability distribution.

[0108] Verification results:

[0109] In order to verify the superiority of the method of the present invention, mainstream models in natural language processing are selected for comparison, and the relationship extraction task is completed based on CNN, BiLSTM, BiGRU, and RNN models. Figure 4 It can be seen that the proposed AeroPCNN-MultiAtt model has better results in relation extraction in the field of airborne mechatronic products than the other four models. Compared with the AeroPCNN-MultiAtt model, the other four models are not efficient enough when processing data with local features as the main feature. Relation extraction tasks usually need to focus on key information near entities. The AeroPCNN-MultiAtt model directly focuses on these key areas through its convolutional layer, avoiding unnecessary full sequence traversal, thereby not only improving processing efficiency, but also enhancing the accuracy of the model in feature extraction.

[0110] In the description of the present invention, it is necessary to understand that the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0111] In the present invention, unless otherwise clearly stipulated and limited, for example, it can be a fixed connection, a detachable connection, or an integrated one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, for ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0112] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0113] The present invention does not involve parts that are the same as the prior art or can be implemented by using the prior art.

Claims

1. A method for extracting the traceability relationship of airborne electromechanical product parameters based on a PCNN model, characterized in that: The following steps are involved: S1: Design a relation extraction model AeroPCNN-MultiAtt for airborne mechatronic products. It uses the PCNN model as a feature extractor, segments the input sequence, enhances the model's attention to different semantic regions, and uses a segment pooling strategy for segment encoding input. S2: A multi-head attention mechanism layer is introduced into the model to capture the diverse features of the input sequence by computing multiple independent attention heads in parallel. S3: Design a fully connected layer as a relation classifier to receive the feature vector output by the multi-head attention mechanism layer and further process it for the relation classification task.

2. The method according to claim 1, characterized in that: The step S1 includes the following contents: a1: The input layer of the model needs to map each word in the input sentence into a vector, which is mainly composed of two parts: the combined word vector and the word relative position vector; for a sentence of length n, S = {w1, w2, ..., w n }, where each word w i Mapped from the word vector matrix to a real-valued vector e i : e i jw i ·V i Where: is the word vector of the i-th word, d w is the dimension of the word vector, V i is the one-hot representation of the input word; a2: In order to make full use of the semantic information of the sentence, the position vector is concatenated into the word vector representation of the word. If the word vector dimension is d w , the position vector dimension is d p , then the final vector sequence is: x={x1,x2,…,x n } Where: x i ∈R n×d (d=d w +d p ×2); a3: The main function of the convolutional layer is to extract local features from the input sequence; assuming that the input sequence is the word vector matrix x i ∈R d , the model mainly extracts features through one-dimensional convolution operation; the convolution operation is expressed as: c i =f(W·X i:i+k-1 +b) Where: W∈R k×d is the convolution kernel, K is the width of the convolution kernel, X i:i+k-1 represents the submatrix from the i-th word to the i+k-1-th word in the input matrix, b is the bias term, and f is the nonlinear activation function; a4: AeroPCNN introduces a segmented pooling operation after the convolutional layer. The purpose of segmented pooling is to divide the output feature vector C of the convolutional layer into three segments according to the positions of the two target entities: the front segment before the left entity, the middle segment between the two entities, and the back segment after the right entity. Assuming that the position index of the left entity is p1 and the position index of the right entity is p2, the convolution feature vector C is divided into three parts: Then perform the maximum pooling operation on each segment separately: p1=max(C 前段 ) p2=max(C 中段 ) p3=max(C 后段 ) The three pooled results p1, p2, p3 are concatenated to form an overall feature vector P∈R 3m , where m is the dimension of each pooling output; This feature vector contains the salient features of different parts of the sentence at the same time, which helps the model understand and classify the relationship between two entities in the sentence; the formula for the concatenation operation is: P = [p1; p2; p3] In the formula: ; represents the concatenation operation of the vector; the concatenated vector P will be used as the input of the subsequent multi-head attention mechanism layer.

3. The method according to claim 1, characterized in that: The step S2 includes the following contents: b1: The feature vector output by the AeroPCNN layer is P. First, we need to calculate the query matrix Q, key matrix K and value matrix V: Q=PW Q K=PW K V=PW V Where: W Q , W K , W V is the weight matrix; b2: The core of AeroPCNN-MultiAtt is to determine the contribution of each element in the final feature representation by calculating the correlation between each word vector element and other elements in the sequence, namely the attention weight; the calculation formula of the attention weight is: Where A is the attention weight matrix, which represents the correlation between each position in the sequence and other positions. T is the dot product of the query matrix and the key matrix, representing the similarity between the query and the key; It is a scaling factor used to prevent the dot product value from being too large, resulting in a smaller gradient output by the softmax function; b3: After obtaining the attention weight matrix A, these weights are used to perform weighted summation on the value matrix to generate a new feature representation; the formula for weighted summation is: O=AV Among them, O is the new feature representation matrix, and V is the value matrix. Through this operation, the feature representation of each position not only contains its own information, but also integrates the information of other positions, which is particularly effective for processing long-distance dependencies and complex syntactic structures. The advantage of the multi-head attention mechanism is that it can process multiple independent attention heads at the same time, each head analyzing the input data from a different perspective; assuming there are h attention heads, the outputs of these heads will be spliced ​​together to form the final multi-head attention output; and the output O of each attention head needs to be spliced ​​together: O multi-head =[O (1) ;Oh (2) ;…;Oh (h) ]W O Among them, multi-head is the concatenated multi-head attention output matrix, W O is a linear transformation matrix used to map the concatenated output back to the original feature dimension.

4. The method according to claim 1, characterized in that: The step S3 includes the following contents: c1: The fully connected layer receives the output feature vector O of the multi-head attention mechanism module multi-head , and further processed for relationship classification tasks; the calculation formula of the fully connected layer is: h=f(W f ·O multi-head +b f ) Where: W f is the weight matrix of the fully connected layer, b f is the bias term, f is the activation function; c2: The output layer uses a function to normalize the output of the fully connected layer to generate a probability distribution of relationship classification; the formula of the softmax layer is: Where: W O and b o is the weight matrix and bias term of the output layer, Represents the predicted relationship type probability distribution.

Images