A fastener detection data intelligent analysis method based on a knowledge graph

Abstract

This invention discloses an intelligent analysis method for fastener inspection data based on knowledge graphs, belonging to the field of industrial non-destructive testing technology. The method includes the following steps: S1, defining a two-layer structure of the knowledge graph; S2, extracting the node embedding tensor set for each time step; S3, constructing a preliminary knowledge graph tensor; S4, constructing and solving the optimization objective function to generate a directed acyclic graph mask; S5, updating the final knowledge graph tensor; S6, parsing rule correction instructions to add, delete, or modify first-order probabilistic logic rules and confidence weights; S7, updating and improving the STGCN model parameters through backpropagation. This method overcomes the limitations of traditional fastener quality inspection methods, such as isolated data sources, single analysis dimensions, and difficulty in capturing complex causes, providing an efficient, accurate, and interpretable solution for intelligent quality control and process optimization in fastener production.

Description

Technical Field

[0001] This invention relates to the field of industrial nondestructive testing technology, and in particular to an intelligent analysis method for fastener testing data based on knowledge graphs. Background Technology

[0002] With the profound transformation of modern manufacturing towards intelligent manufacturing, unprecedentedly high demands have been placed on the quality control of fundamental core components such as fasteners. In fields such as high-end equipment manufacturing, aerospace, and the automotive industry, the performance and reliability of fasteners are directly related to the safety and lifespan of the entire product. Therefore, achieving accurate prediction and quality traceability of production defects is a key link in ensuring the safety of the industrial chain. However, most current fastener quality inspection methods rely on traditional manual sampling, optical image screening, or process parameter monitoring based on simple thresholds. These methods are not only inefficient but also fail to detect potential and complex quality defects. While these traditional methods can meet basic factory standards, they lack the ability to deeply integrate and intelligently analyze data from the entire production cycle, making them unsuitable for the real-time control requirements of zero-defect quality and process optimization in an intelligent manufacturing environment.

[0003] The main limitations of traditional fastener quality inspection methods lie in the one-sidedness of inspection and the lag in prediction. Existing methods typically treat inspection processes in isolation or rely solely on a single data source (such as the final appearance image), resulting in an inability to capture potential quality risks caused by fluctuations in process parameters, batch differences in materials, or abnormal equipment conditions. When fastener production process data exhibits characteristics of multi-source heterogeneity, strong coupling, nonlinearity, and dynamic time-varying, the generalization ability and robustness of traditional inspection methods are severely limited. In particular, when faced with multimodal data from machine vision, sensors, text reports, and bills of materials, traditional single-dimensional analysis methods struggle to efficiently and accurately identify the complex causes of defects, making it difficult to detect and eradicate quality problems in a timely manner, seriously affecting product consistency and reliability.

[0004] Furthermore, traditional methods in quality analysis often overlook the complex interrelationships and causal logic among production factors, making it difficult to comprehensively utilize the potential dependencies between equipment, materials, processes, and testing data. For example, in a highly automated fastener production line, there are profound causal chains between upstream material characteristics, midstream process parameter settings, and downstream testing results. Traditional isolated analysis methods cannot effectively integrate this cross-process and cross-time correlation information, leading to difficulties in quality attribution and low prediction accuracy. Even when some methods employ conventional machine learning or deep learning models, they fail to fully explore the spatiotemporal evolution patterns and deep causal characteristics in production data, making it difficult to achieve efficient, accurate, and interpretable quality risk prediction and process optimization decisions.

[0005] Therefore, how to provide an intelligent analysis method for fastener inspection data based on knowledge graphs is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] This invention proposes an intelligent analysis method for fastener inspection data based on knowledge graphs. By combining an improved STGCN model with a two-layer knowledge graph structure, it can more accurately analyze and predict quality risks in the fastener production process. This method utilizes the heterogeneous graph attention layer of the improved STGCN model to automatically extract deep features from multi-source heterogeneous data such as visual, textual, and process parameters throughout the fastener production cycle. It also effectively integrates semantic relationships between different entities through a meta-path mechanism, significantly improving the comprehensiveness and accuracy of quality representation. By innovatively integrating a neural causal discovery layer and a causal-guided attention layer into the improved STGCN model, this method can autonomously learn the causal structure between variables from the data and generate a directed acyclic graph mask to guide attention calculations. This overcomes the shortcomings of traditional deep learning models that only focus on correlation and ignore causality, achieving more reliable and interpretable quality attribution and prediction. This method overcomes the limitations of traditional fastener quality inspection methods, such as isolated data sources, single analysis dimensions, and difficulty in capturing complex causes, providing an efficient, accurate, and interpretable solution for intelligent quality control and process optimization in fastener production.

[0007] A knowledge graph-based intelligent analysis method for fastener inspection data according to an embodiment of the present invention includes the following steps:

[0008] S1. Define an improved STGCN model, including a heterogeneous graph attention layer, a spatiotemporal sequence encoding layer, a neural causal discovery layer, and a causal guided attention layer; use the knowledge graph tensor generated by the improved STGCN model to form a vector space layer, use first-order probabilistic logic rules and confidence weights to form a probabilistic logic rule layer, and use the vector space layer and the probabilistic logic rule layer to construct a knowledge graph.

[0009] S2. Collect multi-source heterogeneous data, input it into the heterogeneous graph attention layer, extract visual and text features, construct a dynamic heterogeneous graph based on meta-path and meta-path level attention mechanism, and output the node embedding tensor set at each time step.

[0010] S3. Embed the nodes into a tensor set, input it into the spatiotemporal sequence encoding layer, add positional encoding and use the self-attention and feedforward network mechanism of the Transformer encoder to perform temporal modeling, and finally aggregate the outputs of all nodes to form a preliminary knowledge graph tensor.

[0011] S4. Input the preliminary knowledge graph tensor into the neural causal discovery layer, construct and solve the optimization objective function, and generate a directed acyclic graph mask;

[0012] S5. The acyclic graph mask is input into the causal guided attention layer and converted into an attention mask. This mask constrains the self-attention computation of the Transformer encoder, resulting in the final knowledge graph tensor.

[0013] S6. Receive rule correction instructions from user terminals, parse the rule correction instructions, and add, delete or modify the first-order probabilistic logic rules and confidence weights in the probabilistic logic rule layer according to the operation type.

[0014] S7. Calculate the rule loss term based on the first-order probabilistic logic rule, add it to the basic loss term to form the total loss function, and then update and improve the STGCN model parameters through backpropagation.

[0015] Optionally, S2 specifically includes:

[0016] S22. The input detection image is used as the input feature map. The residual network is used to learn features through cascaded residual units. Each residual unit achieves identity mapping through skip connections. The input feature map is added element-wise with the features on the main path after convolution, batch normalization and nonlinear activation transformation. Finally, the residual network is aggregated into a visual feature tensor through global average pooling at the end.

[0017] S23. Input text reports and bills of materials as input sequences. Contextual semantic encoders are used to perform contextual encoding through stacked Transformer encoders. Each encoder layer uses a multi-head self-attention mechanism to aggregate the contextual information of all labels in the sequence. At the end of the contextual semantic encoder, the hidden states of the corresponding classification labels are extracted as text feature tensors. The contextual semantic encoder refers to a neural network composed of multiple stacked Transformer encoder layers.

[0018] S24. Input process parameters as raw values, perform feature transformation through a normalization layer, and output a normalized numerical tensor; use global entity identifiers to index and aggregate all feature tensors of the same entity to generate an aligned entity feature tensor.

[0019] S25. Input the aligned entity feature tensor, map it to nodes and edges, and output a dynamic heterogeneous graph containing node type and edge relationship type; input the dynamic heterogeneous graph, aggregate neighborhood information using graph neural network, and output the updated node feature tensor.

[0020] S26. Input the node feature tensor and the predefined meta-path, collect the semantic neighbors of the center node along the meta-path instances, and output the set of neighbor nodes; calculate the attention weight based on the feature tensors of the center node and the neighbor nodes, perform weighted aggregation on the feature tensors of the neighbor nodes, and output the semantic-level embedding tensor.

[0021] S27. Input multiple semantic-level embedding tensors of the same node, map each semantic embedding into a query, key, and value triple through a meta-path-level attention network, calculate the attention score between paths through dot product interaction, normalize using Softmax and use it as the aggregation weight, and output the set of node embedding tensors at the current time step after weighted summation.

[0022] Optionally, S3 specifically includes:

[0023] S31. Input the set of embedded tensors of nodes and stack them according to time steps to form node temporal tensors;

[0024] S32. Add position encoding to the node temporal tensor to generate a position-aware temporal tensor;

[0025] S33. Map the location-aware temporal tensor to a query tensor, key tensor, and value tensor through a linear transformation;

[0026] S34. Calculate the product of the query tensor and the transpose of the key tensor to obtain the attention score tensor; scale and Softmax normalize the attention score tensor to generate the attention weight tensor.

[0027] S35. Use the attention weight tensor to perform a weighted summation on the value tensor and output the attention context tensor;

[0028] S36. After concatenating the attention context tensor and performing a linear transformation, add the residuals to the position-aware temporal tensor and perform layer normalization to output the attention normalized tensor.

[0029] S37. Input the attention normalization tensor into the feedforward neural network, add the residuals of its output to the attention normalization tensor and perform layer normalization, and output the feedforward normalization tensor.

[0030] S38. Aggregate the feedforward normalized tensors of all nodes to form a preliminary knowledge graph tensor.

[0031] Optionally, S4 specifically includes:

[0032] S41. Input the initial knowledge graph tensor and deconstruct it into multiple time slice tensors along the time step dimension according to the preset time step length; perform a flattening operation on each time slice tensor to generate the corresponding node feature vector; stack all node feature vectors in chronological order to form the observation data matrix.

[0033] S42. Set and initialize the weighted adjacency matrix;

[0034] S43. Using the observation data matrix and the weighted adjacency matrix, the reconstruction error term is calculated. The sparsity penalty term is calculated by summing the absolute values ​​of the elements of the weighted adjacency matrix. The objective function is obtained by adding the reconstruction error term and the sparsity penalty term.

[0035] S44. Calculate the element-wise product matrix of the weighted adjacency matrix; perform matrix exponentiation on the element-wise product matrix to generate an equivalent matrix; calculate the difference between the sum of the main diagonal elements of the equivalent matrix and the total number of nodes, as an equivalence constraint for the optimization objective function;

[0036] S45. Use the augmented Lagrange method to solve the optimization objective function with equivalence constraints, and iteratively update the weighted adjacency matrix until the preset convergence condition is met.

[0037] S46. Traverse each element of the converged weighted adjacency matrix and compare the element value with a preset threshold. If the element value is greater than the threshold, set the structure mask value at the corresponding position to 1; otherwise, set the structure mask value at the corresponding position to 0 and generate a binary adjacency matrix.

[0038] S47. Perform acyclic processing on the binary adjacency matrix to generate a directed acyclic graph mask.

[0039] Optionally, S5 specifically includes:

[0040] S51. Fill the forbidden connection positions in the directed acyclic graph mask with negative infinity values ​​to generate a binarized attention mask tensor.

[0041] S52. Calculate the product of the query matrix and the transpose of the key matrix of the Transformer encoder to generate the original attention score matrix;

[0042] S53. Add the original attention score matrix to the binary attention mask tensor element by element, and apply the softmax function to generate the attention weight matrix.

[0043] S54. Use the attention weight matrix to perform a weighted summation on the value matrix of the Transformer encoder, and output the final knowledge graph tensor.

[0044] Optionally, S6 specifically includes:

[0045] S61. Parse the rule correction instruction and identify the target operation type and target rule;

[0046] S62. If the target operation type is "add", then create a new first-order probabilistic logic rule and set the confidence weight.

[0047] S63. If the operation type is deletion, then remove the specified first-order probability logic rule;

[0048] S64. If the target operation type is modification, then update the content or confidence weight of the specified first-order probabilistic logic rule.

[0049] Optionally, S7 specifically includes:

[0050] S71. Generate rule instances from the knowledge graph tensor by matching the patterns of first-order probabilistic logic rules.

[0051] S72. For each rule instance, calculate the confidence of each fact triple in the instance using the embedding vectors of the corresponding entities and relations in the knowledge graph tensor.

[0052] S73. Based on the logical structure of the first-order probabilistic logic rule, aggregate the confidence of all fact triples within the instance to obtain the satisfaction score of the rule instance.

[0053] S74. Multiply the satisfaction score of the rule instance by the rule confidence weight to obtain the rule loss term;

[0054] S75. Using the cross-entropy loss function as the basic loss term of the improved STGCN model, the total loss function is obtained by adding the regular loss term to the basic loss term of the improved STGCN model.

[0055] S76. Update and improve the parameters of the STGCN model using the backpropagation algorithm based on the total loss function.

[0056] The beneficial effects of this invention are:

[0057] (1) This invention significantly improves the interpretability and causal inference capability of fastener quality prediction models by constructing a neural causal discovery layer. Although traditional deep learning models are powerful in feature extraction, the relationships they learn are often mixed with false causality, making it difficult to guide effective process optimization. To solve this problem, this invention constructs an optimization objective function that includes a reconstruction error term and a sparsity penalty term, and applies acyclic equivalence constraints to directly learn the causal structure between variables from the data. This process can effectively filter out strong non-causal correlations, identify key process paths and parameters that affect product quality, and thus elevate the model from "knowing what" to "knowing why," providing a solid technical foundation for accurate quality attribution and root cause analysis.

[0058] (2) This invention effectively solves the technical challenge of integrating discovered causal structures into deep learning models for accurate prediction by designing a causal-guided attention layer. Traditional models, when fusing multi-source heterogeneous data, often treat all possible connections equally, neglecting the information flow direction and dependency strength determined by causal mechanisms. The causal-guided attention layer directly constrains the self-attention computation process of the Transformer encoder by converting the directed acyclic graph mask generated by the neural causal discovery layer into an attention mask. This allows the model to strictly follow the discovered causal logic when performing time-series modeling, allowing only information flow in causal directions, thereby effectively suppressing noise interference introduced by non-causal associations. This method overcomes the limitation of traditional attention mechanisms lacking structural guidance when processing time-series data, significantly improving the accuracy and reliability of quality prediction, and providing interpretable and efficient decision support for intelligent quality control in fastener production. Attached Figure Description

[0059] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0060] Figure 1 This is an overall flowchart of a knowledge graph-based intelligent analysis method for fastener inspection data proposed in this invention.

[0061] Figure 2 This is a flowchart illustrating the working principle of the neural causal discovery layer of the improved STGCN model, which is proposed in this invention as an intelligent analysis method for fastener inspection data based on knowledge graphs. Detailed Implementation

[0062] The invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0063] refer to Figure 1 and Figure 2 A knowledge graph-based intelligent analysis method for fastener inspection data includes the following steps:

[0064] S1. Define an improved STGCN model, including a heterogeneous graph attention layer, a spatiotemporal sequence encoding layer, a neural causal discovery layer, and a causal guided attention layer; use the knowledge graph tensor generated by the improved STGCN model to form a vector space layer, use first-order probabilistic logic rules and confidence weights to form a probabilistic logic rule layer, and use the vector space layer and the probabilistic logic rule layer to construct a knowledge graph.

[0065] S2. Collect multi-source heterogeneous data, input it into the heterogeneous graph attention layer, extract visual and text features, construct a dynamic heterogeneous graph based on meta-path and meta-path level attention mechanism, and output the node embedding tensor set at each time step.

[0066] S3. Embed the nodes into a tensor set, input it into the spatiotemporal sequence encoding layer, add positional encoding and use the self-attention and feedforward network mechanism of the Transformer encoder to perform temporal modeling, and finally aggregate the outputs of all nodes to form a preliminary knowledge graph tensor.

[0067] S4. Input the preliminary knowledge graph tensor into the neural causal discovery layer, construct and solve the optimization objective function, and generate a directed acyclic graph mask;

[0068] S5. The acyclic graph mask is input into the causal guided attention layer and converted into an attention mask. This mask constrains the self-attention computation of the Transformer encoder, resulting in the final knowledge graph tensor.

[0069] S6. Receive rule correction instructions from user terminals, parse the rule correction instructions, and add, delete or modify the first-order probabilistic logic rules and confidence weights in the probabilistic logic rule layer according to the operation type.

[0070] S7. Calculate the rule loss term based on the first-order probabilistic logic rule, add it to the basic loss term to form the total loss function, and then update and improve the STGCN model parameters through backpropagation.

[0071] In this embodiment, S2 specifically includes:

[0072] S21. Collect multi-source heterogeneous data throughout the entire fastener production cycle. The multi-source heterogeneous data includes inspection images, process parameters, text reports, and bills of materials.

[0073] S22. The input detection image is used as the input feature map. The residual network is used to learn features through cascaded residual units. Each residual unit achieves identity mapping through skip connections. The input feature map is added element-wise with the features on the main path after convolution, batch normalization and nonlinear activation transformation. Finally, the residual network is aggregated into a visual feature tensor through global average pooling at the end.

[0074] S23. Input text reports and bills of materials as input sequences. Contextual semantic encoders are used to perform contextual encoding through stacked Transformer encoders. Each encoder layer uses a multi-head self-attention mechanism to aggregate the contextual information of all labels in the sequence. At the end of the contextual semantic encoder, the hidden states of the corresponding classification labels are extracted as text feature tensors. The contextual semantic encoder refers to a neural network composed of multiple stacked Transformer encoder layers.

[0075] S24. Input process parameters as raw values, perform feature transformation through a normalization layer, and output a normalized numerical tensor; use global entity identifiers to index and aggregate all feature tensors of the same entity to generate an aligned entity feature tensor.

[0076] S25. Input the aligned entity feature tensor, map it to nodes and edges, and output a dynamic heterogeneous graph containing node type and edge relationship type; input the dynamic heterogeneous graph, aggregate neighborhood information using graph neural network, and output the updated node feature tensor.

[0077] S26. Input the node feature tensor and the predefined meta-path, collect the semantic neighbors of the center node along the meta-path instances, and output the set of neighbor nodes; calculate the attention weight based on the feature tensors of the center node and the neighbor nodes, perform weighted aggregation on the feature tensors of the neighbor nodes, and output the semantic-level embedding tensor.

[0078] S27. Input multiple semantic-level embedding tensors of the same node, map each semantic embedding into a query, key, and value triple through a meta-path-level attention network, calculate the attention score between paths through dot product interaction, normalize using Softmax and use it as the aggregation weight, and output the set of node embedding tensors at the current time step after weighted summation.

[0079] This implementation method integrates multimodal information such as visual, textual, and numerical data, and utilizes a heterogeneous graph attention mechanism to deeply mine complex semantic relationships between entities. This generates high-quality node embeddings rich in contextual information for each entity, laying a solid data foundation for subsequent spatiotemporal and causal analysis. A residual network is used to extract visual feature tensors from the detected images, enabling automatic identification of fastener surface defects. A Transformer encoder parses text reports and bills of materials to generate text feature tensors containing contextual semantics. Process parameters are normalized to eliminate the influence of dimensions. Multimodal features are aligned and aggregated based on global entity identifiers to generate a unified entity feature tensor. This feature tensor is mapped to a dynamic heterogeneous graph, and a graph neural network is used to initially aggregate neighborhood information. Semantic neighbors are collected along predefined meta-paths and weighted aggregated through an attention mechanism to generate semantic-level embeddings. Finally, a meta-path-level attention network adaptively fuses semantic information from different paths, outputting the node embedding tensor set for the current time step.

[0080] In this embodiment, S3 specifically includes:

[0081] S31. Input the set of embedded tensors of nodes and stack them according to time steps to form node temporal tensors;

[0082] S32. Add position encoding to the node temporal tensor to generate a position-aware temporal tensor;

[0083] S33. Map the location-aware temporal tensor to a query tensor, key tensor, and value tensor through a linear transformation;

[0084] S34. Calculate the product of the query tensor and the transpose of the key tensor to obtain the attention score tensor; scale and Softmax normalize the attention score tensor to generate the attention weight tensor.

[0085] S35. Use the attention weight tensor to perform a weighted summation on the value tensor and output the attention context tensor;

[0086] S36. After concatenating the attention context tensor and performing a linear transformation, add the residuals to the position-aware temporal tensor and perform layer normalization to output the attention normalized tensor.

[0087] S37. Input the attention normalization tensor into the feedforward neural network, add the residuals of its output to the attention normalization tensor and perform layer normalization, and output the feedforward normalization tensor.

[0088] S38. Aggregate the feedforward normalized tensors of all nodes to form a preliminary knowledge graph tensor.

[0089] This implementation utilizes node embeddings aligned with a Transformer encoder for temporal modeling, effectively capturing the dynamic evolution patterns and short- and long-term dependencies of various entity states during fastener production. Ultimately, it aggregates these into a preliminary knowledge graph tensor rich in spatiotemporal information. The input node embedding tensor set is stacked by time step to form a node temporal tensor, constructing a data structure with a time dimension for temporal analysis. Position encoding is added to the node temporal tensor to generate a position-aware temporal tensor, enabling the model to distinguish node states at different time steps. The position-aware temporal tensor is linearly transformed into a query tensor, a key tensor, and a value tensor, preparing for self-attention computation. The product of the query tensor and the transpose of the key tensor is calculated to obtain the attention score tensor. This attention score tensor is scaled and Softmax normalized to generate an attention weight tensor, quantifying the degree of mutual influence between nodes at different time steps. The attention weight tensor is used to perform a weighted summation of the value tensor, outputting an attention context tensor, achieving adaptive aggregation of global temporal information. After concatenating the attention context tensors and performing a linear transformation, the residuals are added to the position-aware temporal tensor, and layer normalization is applied to output an attention-normalized tensor. This stabilizes the training process and integrates the original information. The attention-normalized tensor is then input into a feedforward neural network. Its output is added to the attention-normalized tensor, and layer normalization is applied to output a feedforward normalized tensor, further extracting the nonlinear combination of temporal features. Aggregating the feedforward normalized tensors of all nodes constructs a preliminary knowledge graph tensor, forming a structured representation of the entire system's comprehensive state in the current time slice.

[0090] In this embodiment, S4 specifically includes:

[0091] S41. Input the initial knowledge graph tensor and deconstruct it into multiple time slice tensors along the time step dimension according to the preset time step length; perform a flattening operation on each time slice tensor to generate the corresponding node feature vector; stack all node feature vectors in chronological order to form the observation data matrix.

[0092] S42. Set and initialize the weighted adjacency matrix;

[0093] S43. Using the observation data matrix and the weighted adjacency matrix, the reconstruction error term is calculated. The sparsity penalty term is calculated by summing the absolute values ​​of the elements of the weighted adjacency matrix. The objective function is obtained by adding the reconstruction error term and the sparsity penalty term.

[0094] S44. Calculate the element-wise product matrix of the weighted adjacency matrix; perform matrix exponentiation on the element-wise product matrix to generate an equivalent matrix; calculate the difference between the sum of the main diagonal elements of the equivalent matrix and the total number of nodes, as an equivalence constraint for the optimization objective function;

[0095] S45. Use the augmented Lagrange method to solve the optimization objective function with equivalence constraints, and iteratively update the weighted adjacency matrix until the preset convergence condition is met.

[0096] S46. Traverse each element of the converged weighted adjacency matrix and compare the element value with a preset threshold. If the element value is greater than the threshold, set the structure mask value at the corresponding position to 1; otherwise, set the structure mask value at the corresponding position to 0 and generate a binary adjacency matrix.

[0097] S47. Perform acyclic processing on the binary adjacency matrix to generate a directed acyclic graph mask.

[0098] This implementation autonomously learns the causal structure between variables from time-series data, extracting complex relationships into a clear directed acyclic graph mask, providing accurate topological basis for subsequent causal guidance analysis. The initial knowledge graph tensor is deconstructed and flattened along time steps to construct an observation data matrix, preparing structured input data for the causal discovery algorithm. A weighted adjacency matrix is ​​initialized as the representation of the causal structure to be optimized. Combining the observation data matrix and the weighted adjacency matrix, an optimization objective function including reconstruction error and sparsity penalty is calculated, aiming to learn an accurate and concise causal graph. By introducing matrix exponents and diagonal constraints, the discrete constraint of acyclicity is transformed into a continuous equivalent constraint, making the problem solvable using gradient methods. The augmented Lagrange method is used to iteratively optimize the objective function until convergence, obtaining the optimal weighted adjacency matrix, where the weights represent the causal strength between variables. The converged matrix is ​​binarized, converting continuous causal strength into explicit causal presence or absence, generating a binarized adjacency matrix. The binary adjacency matrix is ​​acyclic to generate a directed acyclic graph mask that conforms to causal logic.

[0099] In this embodiment, S5 specifically includes:

[0100] S51. Fill the forbidden connection positions in the directed acyclic graph mask with negative infinity values ​​to generate a binarized attention mask tensor.

[0101] S52. Calculate the product of the query matrix and the transpose of the key matrix of the Transformer encoder to generate the original attention score matrix;

[0102] S53. Add the original attention score matrix to the binary attention mask tensor element by element, and apply the softmax function to generate the attention weight matrix.

[0103] S54. Use the attention weight matrix to perform a weighted summation on the value matrix of the Transformer encoder, and output the final knowledge graph tensor.

[0104] This implementation transforms the directed acyclic graph mask generated by the neural causal discovery layer into an attention mask, thus imposing a hard constraint on the Transformer's self-attention mechanism. This ensures that the model strictly adheres to the discovered causal logic during temporal modeling, effectively suppressing interference from non-causal relationships and improving the prediction accuracy and interpretability of the knowledge graph tensor. The forbidden connection positions in the directed acyclic graph mask are filled with negative infinity to generate a binarized attention mask tensor, transforming the causal logic into numerical constraints recognizable by the attention mechanism. The product of the query matrix and the transpose of the key matrix of the Transformer encoder is calculated to generate the original attention score matrix, obtaining the initial association strength of all possible connections. The original attention score matrix is ​​added to the binarized attention mask tensor, and a softmax function is applied to generate an attention weight matrix, ensuring that the model assigns attention weights only to connections that conform to causal relationships. The value matrix is ​​weighted and summed using the attention weight matrix to output the final knowledge graph tensor, achieving information aggregation guided by causality and generating a more reliable knowledge representation.

[0105] In this embodiment, S6 specifically includes:

[0106] S61. Parse the rule correction instruction and identify the target operation type and target rule;

[0107] S62. If the target operation type is "add", then create a new first-order probabilistic logic rule and set the confidence weight.

[0108] S63. If the operation type is deletion, then remove the specified first-order probability logic rule;

[0109] S64. If the target operation type is modification, then update the content or confidence weight of the specified first-order probabilistic logic rule.

[0110] This implementation provides a human-computer interaction interface, allowing domain experts to dynamically modify the probabilistic logic rule layer based on prior knowledge. This achieves an organic combination of data-driven and knowledge-guided approaches, enhancing the model's flexibility and adaptability. The rule modification instructions are parsed to identify the target operation type and target rule, providing guidance for subsequent precise operations. If the target operation type is "add," a new first-order probabilistic logic rule is created and a confidence weight is set, integrating new expert knowledge into the model. If the operation type is "delete," the specified first-order probabilistic logic rule is removed, eliminating outdated or erroneous prior knowledge. If the target operation type is "modify," the content or confidence weight of the specified first-order probabilistic logic rule is updated, achieving fine-tuning of existing knowledge.

[0111] In this embodiment, S7 specifically includes:

[0112] S71. Generate rule instances from the knowledge graph tensor by matching the patterns of first-order probabilistic logic rules.

[0113] S72. For each rule instance, calculate the confidence of each fact triple in the instance using the embedding vectors of the corresponding entities and relations in the knowledge graph tensor.

[0114] S73. Based on the logical structure of the first-order probabilistic logic rule, aggregate the confidence of all fact triples within the instance to obtain the satisfaction score of the rule instance.

[0115] S74. Multiply the satisfaction score of the rule instance by the rule confidence weight to obtain the rule loss term;

[0116] S75. Using the cross-entropy loss function as the basic loss term of the improved STGCN model, the total loss function is obtained by adding the regular loss term to the basic loss term of the improved STGCN model.

[0117] S76. Update and improve the parameters of the STGCN model using the backpropagation algorithm based on the total loss function.

[0118] This implementation transforms symbolic probabilistic logic rules into differentiable rule loss terms and combines them with data-driven basic loss terms to construct a total loss function for neural symbolic co-optimization. This simultaneously injects data fitting ability and domain knowledge constraints into the model training process, improving the model's accuracy and interpretability. Rule instances are generated from the knowledge graph tensor based on the patterns of first-order probabilistic logic rules, associating abstract rules with specific graph data. For each rule instance, the confidence of each fact triple in the instance is calculated using the embedding vectors of corresponding entities and relations in the knowledge graph tensor, quantifying the degree to which the graph data supports the rule's premises and conclusions. Based on the logical structure of the first-order probabilistic logic rules, the confidence of all fact triples within the instance is aggregated to obtain the rule instance's satisfaction score, evaluating the overall validity of the rule instance in the current graph state. The rule loss term is obtained by multiplying the rule instance's satisfaction score by the rule confidence weight, transforming the symbolic logic satisfaction into a numerical loss that can participate in gradient calculation. Using the cross-entropy loss function as the basic loss term of the STGCN model, the total loss function is obtained by adding the rule loss term to the basic loss term of the STGCN model, thus achieving a unified optimization goal of data fitting and knowledge adherence. Based on the total loss function, the parameters of the STGCN model are updated and improved through the backpropagation algorithm, driving the model to continuously align its output with known logical rules while learning data features.

[0119] Example 1:

[0120] To verify the feasibility of this invention in the field of intelligent fastener quality inspection, the method of this invention was applied to the "Full-Process Quality Traceability and Intelligent Control System" (hereinafter referred to as "System A") of a high-end fastener manufacturing enterprise. In traditional fastener quality inspection, quality classification and defect analysis typically rely on manual sampling, rule-based threshold judgment, or a single machine learning model. These methods not only have limited detection accuracy but also cannot effectively integrate multi-source heterogeneous data such as visual, textual, and process data throughout the entire production cycle, making it difficult to uncover key causal paths affecting quality, leading to difficulties in quality attribution and delays in process optimization. To solve these problems, System A decided to adopt the knowledge graph-based intelligent analysis method for fastener inspection data proposed in this invention.

[0121] During implementation, System A first uses industrial cameras, high-precision sensors, and a production execution system to collect multi-source heterogeneous data in real time throughout the entire fastener production cycle. This includes images of surface defects, process parameters such as torque and temperature, text data such as quality inspection reports and operation logs, and bills of materials. The data is then cleaned, aligned, and standardized to form a high-quality dataset. Simultaneously, System A constructs an entity relationship graph based on product batches, equipment numbers, and material flow relationships, accurately representing the complex relationships between various elements in the fastener production process.

[0122] System A utilizes a heterogeneous graph attention layer, employing residual networks and Transformer encoders to extract visual and textual features respectively. It then constructs a dynamic heterogeneous graph based on meta-paths and meta-path-level attention mechanisms, achieving efficient fusion and semantic representation of multi-source heterogeneous information. Furthermore, a spatiotemporal sequence encoding layer models node embeddings temporally, capturing the dynamic evolution of production states over time. Subsequently, a neural causal discovery layer autonomously learns the causal structure between variables from temporal data, generating a directed acyclic graph mask. This causal guidance attention layer constrains subsequent temporal modeling processes, effectively improving the causality and accuracy of feature representation.

[0123] During training, this invention constructs a total loss function comprising rule-based and base loss terms for neural symbolic co-optimization. By transforming first-order probabilistic logic rules into differentiable rule-based loss terms and injecting prior knowledge from domain experts into the model training process, overfitting in sparse data scenarios is effectively avoided, improving the model's generalization performance and decision interpretability. Simultaneously, the system supports experts in dynamically modifying the probabilistic logic rules via an interactive terminal, enabling adaptive iterative optimization of the model.

[0124] During implementation, the technical team of System A discovered that, compared with traditional threshold rule methods and ordinary machine learning methods, the method of this invention significantly improves the accuracy of fastener defect detection and the reliability of quality attribution. Traditional methods cannot handle the complexity and causal relationships of multimodal data with fine precision, while the method of this invention effectively achieves accurate prediction and root cause localization of fastener quality through knowledge graphs and neural causal discovery technology.

[0125] To further verify the actual performance of the method of the present invention, System A conducted a detailed comparative test between the method of the present invention and the traditional method. The specific performance data is shown in Table 1:

[0126] Table 1. Performance Comparison of Fastener Quality Inspection Methods in System A

[0127] index Traditional methods Method of the present invention Increase Defect detection accuracy (%) 85.2 98.7 +13.5% Defect false alarm rate (%) 6.8 1.2 -5.6% Defect false negative rate (%) 8.0 0.1 -7.9% Time taken for a single detection and analysis (seconds) 120 35 -70.8% Accuracy rate of key process parameter positioning (%) 65.4 92.1 +26.7% Average time for quality attribution (minutes) 60 8 -86.7% Non-conforming rate (%) 2.5 0.8 -68.0% Operation, maintenance, and rework costs (ten thousand yuan / month) 40 22 -45.0% Average first-pass yield (%) 91.5 98.9 +7.4% Customer quality complaint rate (%) 1.2 0.3 -75.0%

[0128] As shown in Table 1, the performance of the fastener quality inspection system was comprehensively improved after applying the method of this invention. The defect detection accuracy increased from 85.2% using traditional methods to 98.7%, while the false alarm rate and false negative rate decreased by 5.6% and 7.9% respectively, significantly improving detection precision and effectively preventing the outflow of defective products. The analysis time for a single inspection decreased from 120 seconds using traditional methods to 35 seconds, greatly improving analysis efficiency. Furthermore, the accuracy of key process parameter positioning increased from 65.4% to 92.1%, and the average time for quality attribution decreased from 60 minutes to 8 minutes, providing rapid and accurate guidance for process optimization. The non-conforming product rate decreased from 2.5% per month to 0.8%, and the transportation, inspection, and rework costs decreased from 400,000 yuan per month to 220,000 yuan, resulting in a 45% cost saving. The first-pass yield also increased from 91.5% to 98.9%, significantly improving production efficiency and quality stability. The customer quality complaint rate also decreased significantly, from 1.2% to 0.3%.

[0129] Through the method of this invention, System A successfully achieved rapid and accurate detection and intelligent attribution of fastener quality, effectively reducing the risk of product defects, ensuring the stability and reliability of the production process, significantly improving the intelligence and automation level of fastener production, significantly reducing quality costs, enhancing the interpretability and robustness of system decisions, and providing strong technical support for the digital transformation of high-end manufacturing.

[0130] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A fastener detection data intelligent analysis method based on a knowledge graph, characterized in that, Comprise the following steps: S1, define the improved STGCN model, including heterogeneous graph attention layer, space-time sequence encoding layer, neural causal discovery layer, causal guide attention layer; the knowledge graph tensor generated by using the improved STGCN model constitutes a vector space layer, a first-order probabilistic logic rule and a confidence weight constitute a probabilistic logic rule layer, and the vector space layer and the probabilistic logic rule layer are used to build a knowledge graph; S2, collect multi-source heterogeneous data, input into the heterogeneous graph attention layer, extract visual and text features, build a dynamic heterogeneous graph based on the meta path and the meta path level attention mechanism, and output the node embedding tensor set of each time step; S3, input the node embedding tensor set into the space-time sequence encoding layer, add position encoding and use the self-attention and feedforward network mechanism of the Transformer encoder for time series modeling, finally aggregate the outputs of all nodes to form a preliminary knowledge graph tensor; S4, input the preliminary knowledge graph tensor into the neural causal discovery layer, build and solve the optimization objective function, and generate a directed acyclic graph mask; S5, input the directed acyclic graph mask into the causal guide attention layer, convert it into an attention mask, constrain the self-attention calculation of the Transformer encoder, and obtain the final knowledge graph tensor; S6, receive the rule modification instruction from the user terminal, parse the rule modification instruction, and add, delete or modify the first-order probabilistic logic rule and the confidence weight in the probabilistic logic rule layer according to the operation type; S7, calculate the rule loss term based on the first-order probabilistic logic rule, add it to the basic loss term to form a total loss function, and then update the improved STGCN model parameters through back propagation. 2.The knowledge graph-based fastener detection data intelligent analysis method of claim 1, wherein, The S2 specifically comprises: S21, collect multi-source heterogeneous data in the whole cycle of fastener production, and the multi-source heterogeneous data includes detection images, process parameters, text reports and bill of materials; S22, input the detection image as an input feature map, use the residual network to learn features through cascaded residual units, each residual unit realizes identity mapping through jump connection, adds the input feature map and the features on the main path which are transformed by convolution, batch normalization and nonlinear activation, and finally aggregates into a visual feature tensor through global average pooling at the end of the residual network; S23, input the text report and bill of materials as an input sequence, use the context semantic encoder to encode the context by stacking the Transformer encoder, each encoder layer uses the multi-head self-attention mechanism to aggregate the context information of all tokens in the sequence, and at the end of the context semantic encoder, the hidden state corresponding to the classification token is extracted as a text feature tensor; the context semantic encoder refers to a neural network stacked by multiple Transformer encoder layers; S24, input the process parameters as original numerical values, transform the features through the normalization layer, and output the normalized numerical value tensor; index and aggregate all feature tensors of the same entity using global entity identification to generate aligned entity feature tensors; S25. Input the aligned entity feature tensor, map it to nodes and edges, and output a dynamic heterogeneous graph containing node type and edge relationship type; input the dynamic heterogeneous graph, aggregate neighborhood information using graph neural network, and output the updated node feature tensor. S26. Input the node feature tensor and the predefined meta-path, collect the semantic neighbors of the center node along the meta-path instances, and output the set of neighbor nodes; calculate the attention weight based on the feature tensors of the center node and the neighbor nodes, perform weighted aggregation on the feature tensors of the neighbor nodes, and output the semantic-level embedding tensor. S27. Input multiple semantic-level embedding tensors of the same node, map each semantic embedding into a query, key, and value triple through a meta-path-level attention network, calculate the attention score between paths through dot product interaction, normalize using Softmax and use it as the aggregation weight, and output the set of node embedding tensors at the current time step after weighted summation. 3.The knowledge graph-based fastener detection data intelligent analysis method of claim 1, wherein, S3 specifically includes: S31. Input the set of embedded tensors of nodes and stack them according to time steps to form node temporal tensors; S32. Add position encoding to the node temporal tensor to generate a position-aware temporal tensor; S33. Map the location-aware temporal tensor to a query tensor, key tensor, and value tensor through a linear transformation; S34. Calculate the product of the query tensor and the transpose of the key tensor to obtain the attention score tensor; scale and Softmax normalize the attention score tensor to generate the attention weight tensor. S35. Use the attention weight tensor to perform a weighted summation on the value tensor and output the attention context tensor; S36. After concatenating the attention context tensor and performing a linear transformation, add the residuals to the position-aware temporal tensor and perform layer normalization to output the attention normalized tensor. S37. Input the attention normalization tensor into the feedforward neural network, add the residuals of its output to the attention normalization tensor and perform layer normalization, and output the feedforward normalization tensor. S38. Aggregate the feedforward normalized tensors of all nodes to form a preliminary knowledge graph tensor. 4.The knowledge graph-based fastener detection data intelligent analysis method of claim 1, wherein, S4 specifically includes: S41. Input the initial knowledge graph tensor and deconstruct it into multiple time slice tensors along the time step dimension according to the preset time step length; perform a flattening operation on each time slice tensor to generate the corresponding node feature vector; stack all node feature vectors in chronological order to form the observation data matrix. S42. Set and initialize the weighted adjacency matrix; S43. Using the observation data matrix and the weighted adjacency matrix, the reconstruction error term is calculated. The sparsity penalty term is calculated by summing the absolute values ​​of the elements of the weighted adjacency matrix. The objective function is obtained by adding the reconstruction error term and the sparsity penalty term. S44. Calculate the element-wise product matrix of the weighted adjacency matrix; perform matrix exponentiation on the element-wise product matrix to generate an equivalent matrix; calculate the difference between the sum of the main diagonal elements of the equivalent matrix and the total number of nodes, as an equivalence constraint for the optimization objective function; S45. Use the augmented Lagrange method to solve the optimization objective function with equivalence constraints, and iteratively update the weighted adjacency matrix until the preset convergence condition is met. S46. Traverse each element of the converged weighted adjacency matrix and compare the element value with a preset threshold. If the element value is greater than the threshold, set the structure mask value at the corresponding position to 1; otherwise, set the structure mask value at the corresponding position to 0 and generate a binary adjacency matrix. S47. Perform acyclic processing on the binary adjacency matrix to generate a directed acyclic graph mask.

5. The fastener detection data intelligent analysis method based on a knowledge graph according to claim 1, characterized in that, S5 specifically includes: S51. Fill the forbidden connection positions in the directed acyclic graph mask with negative infinity values ​​to generate a binarized attention mask tensor. S52. Calculate the product of the query matrix and the transpose of the key matrix of the Transformer encoder to generate the original attention score matrix; S53. Add the original attention score matrix to the binary attention mask tensor element by element, and apply the softmax function to generate the attention weight matrix. S54. Use the attention weight matrix to perform a weighted summation on the value matrix of the Transformer encoder, and output the final knowledge graph tensor.

6. The fastener detection data intelligent analysis method based on a knowledge graph according to claim 1, characterized in that, S6 includes the following steps: S61. Parse the rule correction instruction and identify the target operation type and target rule; S62. If the target operation type is "add", then create a new first-order probabilistic logic rule and set the confidence weight. S63. If the operation type is deletion, then remove the specified first-order probability logic rule; S64. If the target operation type is modification, then update the content or confidence weight of the specified first-order probabilistic logic rule.

7. The fastener detection data intelligent analysis method based on a knowledge graph according to claim 1, characterized in that, S7 includes the following steps: S71. Generate rule instances from the knowledge graph tensor by matching the patterns of first-order probabilistic logic rules. S72. For each rule instance, calculate the confidence of each fact triple in the instance using the embedding vectors of the corresponding entities and relations in the knowledge graph tensor. S73. Based on the logical structure of the first-order probabilistic logic rule, aggregate the confidence of all fact triples within the instance to obtain the satisfaction score of the rule instance. S74. Multiply the satisfaction score of the rule instance by the rule confidence weight to obtain the rule loss term; S75. Using the cross-entropy loss function as the basic loss term of the improved STGCN model, the total loss function is obtained by adding the regular loss term to the basic loss term of the improved STGCN model. S76. Update and improve the parameters of the STGCN model using the backpropagation algorithm based on the total loss function.

Images