Forging press fault knowledge graph construction method of self-attention mechanism

Abstract

The invention relates to the technical field of forging press intelligent maintenance, and discloses a forging press fault knowledge graph construction method based on a self-attention mechanism, and the method comprises the steps: S1, data collection: collecting time sequence data of equipment operation and text data of equipment maintenance; s2, carrying out normalization processing on the time sequence data; s3, carrying out self-attention calculation on the collected data; and S4, constructing a forging press fault knowledge graph by using the multi-modal features, and aligning entity information of different data sources.

Description

Technical Field

[0001] The present application relates to the field of intelligent maintenance of forging presses, and specifically to a method for constructing a forging press fault knowledge graph based on a self-attention mechanism. Background Art

[0002] Because forging hydraulic presses often experience malfunctions during operation, such as hydraulic cylinder leakage, hydraulic pump wear, and hydraulic valve jamming, failure to promptly and quickly repair them can lead to the scrapping of workpieces. Routine maintenance of forging presses is plagued by issues such as manual and incomplete troubleshooting records, the creation of data silos such as massive amounts of time-series operating data, difficulty integrating data, time-consuming data calculations, and difficulties in data analysis. This makes it difficult for technicians to verify the effectiveness of current maintenance plans by analyzing fault data, and prevents them from quickly locating similar faults and analyzing their causes. Summary of the Invention

[0003] A method for constructing a forging press fault knowledge graph based on a self-attention mechanism includes the following steps:

[0004] S1: Data collection, including collecting time series data of equipment operation and text data of equipment maintenance;

[0005] S2: Normalize the time series data;

[0006] S3: Perform self-attention calculation on the collected data, including:

[0007] S31: Cross-modal fusion is achieved by improving the Transformer model, and text feature representation and temporal feature representation are input into the cross-modal attention module together;

[0008] S32: After cross-modal multi-head attention, residual connections and layer normalization are used to obtain a feature vector that integrates text modality and temporal modality information. This allows the improved Transformer model to learn information about text modality and temporal modality, achieving interactive fusion of information between text modality and temporal modality. After passing through a feedforward neural network, residual connections and layer normalization are again used to obtain a temporal feature vector that integrates text feature information.

[0009] S33: Perform maximum pooling and average pooling operations on the temporal feature vector after integrating the text feature information, and splice the results of maximum pooling and average pooling together, and use the splicing result as the output of cross-modal attention;

[0010] S34: Concatenate the high-level audio features and high-level video features within the modality with the output features of the cross-modal attention;

[0011] S35: A linear transformation layer is used to reduce the dimensionality of audio and video features to make them equal to the dimensions of text features. The three modal features of text, audio, and video are then concatenated together as the final multimodal feature representation.

[0012] S4: Use multimodal features to construct a knowledge graph of forging press faults and align entity information from different data sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flow chart of a method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to some embodiments of the present disclosure is shown;

[0014] Figure 2 A schematic diagram of the time-series-text cross-modal data processing principle based on self-attention according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0015] Figure 1 A flow chart of a method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to some embodiments of the present disclosure is shown.

[0016] like Figure 1 As shown, in some embodiments of the present disclosure, a method for constructing a forging press fault knowledge graph based on a self-attention mechanism may include step S1: data collection, including collecting time series data of equipment operation and text data of equipment maintenance.

[0017] In some embodiments of the present disclosure, the time series data includes data monitored by different types of sensors. For example, the time series data includes data from a hydraulic cylinder pressure sensor of a forging press, a hydraulic system liquid flow sensor, and the like.

[0018] In some embodiments of the present disclosure, the text data includes maintenance records of a forging press, parts replacement record data, and the like.

[0019] In some embodiments of the present disclosure, data can be collected by a data collection module. The data collection module can be used to pre-set the time series data collected by the collection device and the text data of the equipment maintenance.

[0020] The collected time series data is monitored by different types of sensors. Different sensors operate in different environments, and the measured values are generally associated with different units. To eliminate the range differences in the input data from different sensors, the values of all selected features are normalized to the range [0, 1] using the min-max method.

[0021] Therefore, if Figure 1As shown, in some embodiments of the present disclosure, the method for constructing a forging press fault knowledge graph based on the self-attention mechanism may also include step S2: normalizing the time series data.

[0022] In some embodiments of the present disclosure, normalization is performed on time series data:

[0023] (1)

[0024] Among them, x i Represents the original data of the i-th sequence, x max and x min Respectively represent the maximum and minimum values of the selected data, Represents the normalized result data of the i-th sequence.

[0025] Normalization is a method of mapping data to the range of [0, 1], which maps the value range of the original data to [0, 1] through linear transformation.

[0026] like Figure 1 As shown, in some embodiments of the present disclosure, the method for constructing a forging press fault knowledge graph based on the self-attention mechanism may also include step S3: performing self-attention calculation on the collected data.

[0027] Figure 2 A schematic diagram of the time-series-text cross-modal data processing principle based on self-attention according to some embodiments of the present disclosure is shown.

[0028] In some embodiments of the present disclosure, step S3 may include step S31: Figure 2 As shown in the figure, by improving the Transformer model to achieve cross-modal fusion, the text feature representation and time series feature representation Input the cross-modal attention module together, where L represents the text modality and A represents the temporal modality; As the main mode, provide Q (Query); As an auxiliary modality, K(Key) is provided.

[0029] In some embodiments of the present disclosure, a text-assisted temporal cross-modal attention representation is used:

[0030]

[0031] (2)

[0032] Among them, Q A Q, K generated by time series mode A L K is generated for the text mode L, T is the transposed mathematical symbol, , are the linear transformation weight matrices, is the corresponding dimension.

[0033] The derivation process of formula (2) includes:

[0034] Step 1: Linear Projection

[0035] First, the input features are projected into the query (Q), key (K), and value (V) space. The projection uses a learnable weight matrix:

[0036] Query (Q) comes from the time series modality (main modality):

[0037]

[0038] in Each row vector where is the query vector for position i.

[0039] Key (K) from text modal (auxiliary modal):

[0040]

[0041] in Each row vector where is the key vector for position j.

[0042] The value (V) comes from the text modal (auxiliary modal, same modal as K):

[0043]

[0044] in Each row vector where is the value vector at position j (usually ).

[0045] Since the text modality serves as the auxiliary modality, it provides K and V, while the time series modality serves as the main modality and provides Q. This allows time series queries (Q) to retrieve text values (V) based on text keys (K), thereby enhancing the time series representation with text information.

[0046] Step 2: Calculate Attention Scores

[0047] The attention score matrix S measures the similarity between the temporal query (Q) and the text key (K). It is calculated using a scaled dot-product to prevent the dot product value from being too large and causing the gradient to disappear:

[0048]

[0049] in ,element Represents a time series query at position i The text key with position j The score between:

[0050]

[0051] in is the dot product, is a scaling factor (used to stabilize training) to ensure that the score variance is about 1 and avoid softmax saturation;

[0052] Each row i of the matrix S corresponds to a time series position, which represents the similarity distribution of the time series query to all text keys.

[0053] Step 3: Calculate Attention Weights

[0054] Apply the softmax function to the attention score matrix S and calculate the attention weight matrix A row-wise. This ensures that the sum of the weights at each temporal position is 1:

[0055]

[0056] in, ,element Represents the normalized importance weight of text position j for a given temporal position i.

[0057]

[0058] The softmax function is applied along the row direction: for each fixed i;

[0059] Step 4: Compute the Output Representation

[0060] The attention output O is the weighted sum of the attention weight A and the text value V. The output sequence O incorporates the information of the text modality and serves as an enhancement of the temporal representation:

[0061]

[0062] in, , each vector is the output feature at position i:

[0063]

[0064] in is a vector of text values from the auxiliary modality.

[0065] To achieve cross-modal fusion of text and time series data, the cross-modal attention module proposed in this disclosure completes the overall calculation through the following function to obtain the cross-modal attention representation of text-assisted time series:

[0066]

[0067]

[0068] In some embodiments of the present disclosure, step S3 may further include step S32: after cross-modal multi-head attention, a feature vector of fused text modality and temporal modality information is obtained through residual connection and layer normalization, and the temporal modality information includes audio modality information and video modality information, so that the improved Transformer model can fully learn the information of text modality and temporal modality, realize the interactive fusion of information between text modality and temporal modality, realize fine-grained modality alignment through cross attention, and then after the feedforward neural network (forward propagation), the temporal feature vector after the fusion of text feature information is obtained through residual connection and layer normalization again. .

[0069] In some embodiments of the present disclosure, a feedforward neural network may include two linear layers.

[0070] In some embodiments of the present disclosure, step S3 may further include step S33: performing maximum pooling and average pooling operations on the temporal feature vector after fusing the text feature information, and splicing the results of the maximum pooling and average pooling together, and using the spliced result as the output of the cross-modal attention.

[0071] Because the pooling operation has the advantages of suppressing noise, reducing information redundancy and model calculation, and preventing overfitting, using combined pooling can obtain a richer feature layer. Maximum pooling is selected to capture local features at each moment, and average pooling is selected to make the model focus more on global features.

[0072] In some embodiments of the present disclosure, performing maximum pooling and average pooling operations and concatenating the results of the maximum pooling and the average pooling includes:

[0073] (3)

[0074] (4)

[0075] (5)

[0076] The derivation process is as follows:

[0077] Input feature definition

[0078] Audio feature sequence after fusing text features:

[0079] =

[0080] Max pooling operation

[0081] Global maximum pooling takes the maximum value element by element along the sequence dimension (time step dimension):

[0082]

[0083] The derivation process is as follows:

[0084] Audio feature sequence after fusing text features:

[0085] =

[0086] Global average pooling calculates the average value element by element along the sequence dimension (time step dimension):

[0087] Calculation process (cross-modal feature concatenation: combined pooling output):

[0088] Concatenation operation (Concat) concatenates two vectors along the feature dimension:

[0089]

[0090] Element-level representation:

[0091] set up:

[0092]

[0093] The splicing result is:

[0094] Similarly, the video feature vector can be obtained after integrating the text feature information.

[0095] In some embodiments of the present disclosure, step S3 may further include step S34: in order to obtain the fusion of single-modal internal information and inter-modal interactive information, the high-level audio features and high-level video features within the modality are spliced together with the features output by the cross-modal attention. The process is as follows:

[0096] (4)

[0097] S35: A linear transformation layer is used to reduce the dimensionality of the audio and video features to make them equal to the dimensions of the text features. The three modal features of text, audio, and video are then concatenated together as the final multimodal feature representation, as shown below:

[0098] (5)

[0099] Since maintenance knowledge is manually filled in by staff, it may lead to inconsistencies and lack of data normalization. In this paper, a fault knowledge graph of forging press is constructed to align the same entity information from different data sources to ensure the accuracy and consistency of the knowledge graph data, so as to quickly locate the cause of the fault and improve maintenance efficiency.

[0100] like Figure 1 As shown, in some embodiments of the present disclosure, the method for constructing a forging press fault knowledge graph based on the self-attention mechanism may also include step S4: constructing a forging press fault knowledge graph using multimodal features and aligning entity information from different data sources.

[0101] In some embodiments of the present disclosure, the multimodal features obtained in S25 are used to align entities using a method for calculating attribute similarity. A combination of the Jaccard coefficient and the Levenshtein minimum edit distance is used to determine entity equivalence to improve accuracy. The Jaccard coefficient is defined as the ratio of the intersection to the union of two groups. The larger the ratio, the more similar the two groups. The smaller the ratio, the greater the difference between the two groups, and the lower the similarity.

[0102] In some embodiments of the present disclosure, the Jaccard coefficient is defined as the ratio of the intersection to the union of two sets. The Jaccard coefficient is calculated as follows:

[0103] (6)

[0104] Among them, S and T represent text character sets. Indicates the number of characters shared by the two groups, Indicates the total number of characters in both groups.

[0105] In some embodiments of the present disclosure, the Levenshtein edit minimum edit distance calculation formula is:

[0106] (7)

[0107] in, Indicates inserting a letter into the string T. It means to delete letters in string S. When , no effect; same cost as the previous step , otherwise +1, finally Pick 、 and The smallest of the three.

[0108] In some embodiments of the present disclosure, a Neo4j graph database is used to construct a forging press fault knowledge graph.

[0109] In some embodiments of the present disclosure, Neo4j stores entity information aligned across different data sources, leveraging the natural extensibility of graph structures and employing an index-free query algorithm that traverses adjacent nodes. Neo4j's property graph model consists of "nodes" and "relationships." Nodes store entity information and are connected by "relationships," while property labels are stored in key-value pairs.

[0110] In some embodiments of the present disclosure, the Py2neo interface is used to call the Neo4j engine to interact with the Neo4j database. This allows us to efficiently retrieve and display results on the front end, enabling visual exploration and facilitating various operations such as adding, deleting, changing, and maintaining data.

[0111] The self-attention-based forging press fault knowledge graph construction method according to some embodiments of the present disclosure can process data from various sources and formats, such as time series data on forging press operation (e.g., hydraulic cylinder pressure, fluid flow in the hydraulic system, etc.) and text data on forging press maintenance (e.g., maintenance records, replacement records, etc.). By applying self-attention to information from different modalities (e.g., time series, text, images, audio, video), the improved Transformer model can capture cross-modal dependencies, enabling more accurate cross-modal data processing.

[0112] Since maintenance knowledge is manually filled in by staff, inconsistencies and lack of data normalization may occur. For example, "forging stall" and "forging overload" refer to the same phenomenon. According to the forging press fault knowledge graph construction method based on the self-attention mechanism in some embodiments of the present disclosure, a forging press fault knowledge graph is constructed, and the same entity information from different data sources is aligned to ensure the accuracy and consistency of the knowledge graph data, so as to quickly locate the cause of the fault and improve maintenance efficiency.

Claims

1. A method for constructing a forging press fault knowledge graph based on a self-attention mechanism, characterized in that: The following steps are involved: S1: Data collection, including collecting time series data of equipment operation and text data of equipment maintenance; S2: performing normalization processing on the time series data; S3: Perform self-attention calculation on the collected data, including: S31: Cross-modal fusion is achieved by improving the Transformer model, and text feature representation and temporal feature representation are input into the cross-modal attention module together; S32: After cross-modal multi-head attention, residual connections and layer normalization are used to obtain a feature vector that integrates text modality and temporal modality information. This allows the improved Transformer model to learn information about text modality and temporal modality, achieving interactive fusion of information between text modality and temporal modality. After passing through a feedforward neural network, residual connections and layer normalization are again used to obtain a temporal feature vector that integrates text feature information. S33: performing maximum pooling and average pooling operations on the temporal feature vector after fusing the text feature information, and splicing the results of the maximum pooling and the average pooling together, and using the splicing result as the output of the cross-modal attention; S34: Concatenate the high-level audio features and high-level video features within the modality with the output features of the cross-modal attention; S35: A linear transformation layer is used to reduce the dimensionality of audio and video features to make them equal to the dimensions of text features. The three modal features of text, audio, and video are then concatenated together as the final multimodal feature representation. S4: Use multimodal features to construct a knowledge graph of forging press faults and align entity information from different data sources.

2. The method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to claim 1, characterized in that: The temporal modal information includes audio modal information and video modal information.

3. The method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to claim 1, characterized in that: The time series data includes data monitored by different types of sensors.

4. The method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to claim 1, characterized in that: The text data includes maintenance records and component replacement record data of the forging press.

5. The method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to claim 1, characterized in that: Step S31 includes cross-modal attention representation of temporal sequences assisted by text.

6. The method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to claim 1, characterized in that: The feedforward neural network includes two linear layers.

7. The method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to claim 1, characterized in that: The multimodal features obtained by S35 are used to align entities using the method of calculating attribute similarity, and the equivalence of entities is determined by combining the Jaccard coefficient and the Levenshtein minimum edit distance.

8. The method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to claim 1, characterized in that: The Neo4j graph database is used to construct a knowledge graph of forging press failures.

9. The method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to claim 8, characterized in that: Neo4j stores entity information aligned with different data sources. Based on the natural extension characteristics of the graph structure, it uses an index-free query algorithm that traverses adjacent nodes. Neo4j's property graph model consists of "nodes" and "relationships." "Nodes" store entity information and are connected to each other by "relationships," while property labels are stored in key-value pairs.

10. The method for constructing a forging press fault knowledge graph based on a self-attention mechanism according to claim 9, characterized in that: Use the Py2neo interface to call the Neo4j engine to interact with the Neo4j database.

Images