An industrial defect detection and root cause tracing integrated method

Abstract

This application provides an integrated method for industrial defect detection and root cause tracing, relating to the field of industrial intelligent detection and quality control technology. The method includes: constructing a cross-stage structured dataset containing images, process parameters, and defect causes; training a multimodal large model based on this dataset using a course learning strategy, enabling it to simultaneously output defect location results and semantic descriptions of causes for input images; constructing a structured cause-process-control parameter causal graph to characterize causal relationships in the production process; and performing root cause reasoning based on the semantic descriptions of causes using a graph neural network to automatically identify the key processes and control parameters leading to the defects and output a complete root cause tracing path. This method achieves an integrated leap from defect perception to root cause recognition, solving the problems of disconnect between detection and root cause tracing, reliance on human experience, low efficiency, and poor interpretability in traditional methods.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial intelligent detection and quality control, and particularly relates to an industrial defect detection and root cause tracing integrated method. BACKGROUND

[0002] With the deep transformation of industrial manufacturing towards intelligence and digitization, higher requirements are put forward for real-time quality monitoring and accurate root cause analysis of the production process. Under this trend, visual detection technology based on deep learning has become the mainstream solution for industrial quality inspection, which can efficiently complete the identification and positioning of defects. Currently, this technology is evolving from single defect discovery to an integrated solution of detection, diagnosis and tracing, in order to locate the causes in the production process at the same time when the quality problem is found. This integrated technology path can significantly compress the time for quality abnormality investigation, and improve the transparency and controllability of the production process.

[0003] However, compared with dedicated and offline quality analysis systems, the integrated tracing solution based on real-time vision faces significant challenges: first, the visual model can only output the category and position of the defect, and cannot directly associate the defect cause, which forms a gap between perception and cognition; second, the relationship between defect cause and process parameters is usually complex and nonlinear, and the production processes are closely linked, so it is difficult for traditional methods to establish an end-to-end causal chain from defect phenomenon to process and then to key control parameters. These factors seriously restrict the diagnostic accuracy and practicality of the integrated tracing system.

[0004] To overcome the above challenges, the simple solution of connecting detection models and diagnosis rule bases in series has been difficult to meet the needs of complex industrial sites, and it is necessary to turn to the integrated modeling technology route driven by data and knowledge. The existing problems of the prior art mainly include:

[0005] 1) The problem of separation of defect detection and cause analysis:

[0006] Existing visual detection models can only output the category and position of the defect, but cannot provide its cause information at the same time. This leads to detection and analysis as two serial and human intervention-dependent links, and the root cause lies in the lack of computable association between the visual features of the defect and the process cause, which forms a gap between perception and cognition, and finally causes delay and low efficiency of quality diagnosis process, and poor consistency of conclusions.

[0007] 2) Difficulty in automatic tracing from defect cause to process parameter:

[0008] Even if the cause of the defect is described, due to the complex and implicit network relationship between the defect cause, production process and control parameters, the traditional method lacks a mechanism to map the semantic cause to the cause-process-parameter graph and perform automatic reasoning. The current mode highly depends on expert experience, which makes it difficult to achieve precise and rapid tracing from the cause to the specific control parameter, and the tracing result is difficult to directly translate into precise adjustment action. SUMMARY

[0009] The present application provides an industrial defect detection and root cause tracing integrated method, which aims to solve the two major problems of perception and cognition disconnection in existing industrial defect detection technology, and difficulty in automatic tracing of defect cause to process parameters. By training a single multi-modal model, the synchronous output of defect positioning and cause description is realized, so as to integrate the traditional serial detection-analysis process into collaborative reasoning action. Further, by constructing a structured cause-process-control parameter causal graph, automatic and interpretable tracing reasoning is performed using a graph neural network, and finally, end-to-end precise and rapid diagnosis from defect phenomenon to key process parameters is realized, and the automation and intelligent level and decision efficiency of industrial quality analysis are systematically improved.

[0010] The present application provides an industrial defect detection and root cause tracing integrated method, which includes:

[0011] A cross-link structured data set is constructed, which includes workpiece images, equipment operating parameters, process setting values collected from multiple process nodes, and associated defect positions, defect categories, defect cause text descriptions, process identifiers and control parameters;

[0012] Based on the cross-link structured data set, a multi-modal detection and cause analysis model is trained, the input of the multi-modal detection and cause analysis model is a workpiece image, and the output is a defect positioning result, including bounding box coordinates and defect category and cause semantic description;

[0013] A causal graph is constructed, which includes cause nodes, process nodes and parameter nodes, and directed edges between nodes defined based on process knowledge or historical data;

[0014] Based on the graph neural network, the cause semantic description output by the multi-modal detection and cause analysis model is mapped to the cause node in the causal graph, the causal propagation is simulated through the graph neural network, and the tracing path report is output.

[0015] Further, the cross-link structured data set is constructed by the following method:

[0016] Workpiece images, equipment operating parameters and process setting values are synchronously collected from K key process nodes of the production line;

[0017] The image is labeled with the location and category of defects to obtain a set of bounding box coordinates and a set of defect category labels, and then associated with the text description of the defect cause, process identification and control parameters.

[0018] The control parameters are normalized, and the workpiece image, bounding box coordinate set, defect category label set, text description of defect cause, process identifier, and normalized control parameters are integrated into a structured data record.

[0019] Furthermore, based on the aforementioned cross-stage structured dataset, a multimodal detection and causal analysis model is trained, including:

[0020] A multimodal large model that integrates visual and language capabilities is selected as the base network, which includes a text encoder and a language model;

[0021] Based on the aforementioned cross-stage structured dataset, a course learning strategy is employed to train the basic network, resulting in a multimodal detection and causal analysis model; wherein, the course learning strategy includes:

[0022] Defect perception stage: Fix the text encoder and language model parts, and fine-tune the visual encoder and visual language connector only using defect location and category labels;

[0023] Joint understanding phase: Unfreeze the text encoder and simultaneously optimize the defect localization loss and causal description generation loss.

[0024] Furthermore, the defect perception stage employs an improved detection loss function:

[0025] ;

[0026] In the formula, The total detection loss represents the overall detection task loss in the defect perception phase, and is a weighted sum of the classification loss, bounding box regression loss, and generalized intersection-union (GUC) loss. m The sample size refers to the total number of samples used in loss calculation within a batch. j The sample index indicates that the current calculation is for the [number]th [sample]. Loss per sample, L cls For category loss, For real category labels, To predict category labels, These are the weighting coefficients for the SmoothL1 loss. L reg For SmoothL1 loss, These are the coordinates of the actual bounding box. To predict the bounding box coordinates, The weighting coefficients for the generalized intersection-union loss are... L giou For generalized intersection and comparison of losses;

[0027] L cls The calculation formula is:

[0028] ;

[0029] In the formula, , Here, is a hyperparameter, and log is the natural logarithm, used in classification loss to calculate the logarithmic loss of the predicted probabilities. To predict class probabilities, the model refers to the probability of the first class. Each sample belongs to its true category. The predicted probability.

[0030] Furthermore, the total loss function for the joint understanding phase is:

[0031] ;

[0032] In the formula, L total For the total loss, L text For text loss, This refers to the multi-task weighting coefficient;

[0033] Text loss L text The calculation formula is:

[0034] ;

[0035] In the formula, For label smoothing factor, For vocabulary size, For sequence length, t For sequence index, v For the vocabulary index, y t For the first The real token at position n is a one-hot vector representing the token at position n in the sequence. The vector represents the distribution of the actual next word in the vocabulary at each position. Only the position corresponding to the actual word is 1, and the rest are 0. For the first The predicted probability distribution of each location, For the sequence number The position, for the first in the vocabulary list The predicted probability of each word.

[0036] Furthermore, a causal graph is constructed as follows:

[0037] Using the core keywords of various defect cause descriptions output by the multimodal detection and causal analysis model as causal nodes, it is represented as follows: , , and They are the first, the second, and the third, respectively. M One causal node;

[0038] Using specific steps in the production process as process nodes, it is represented as , , and They are the first, the second, and the third, respectively. L Each process node;

[0039] Using the key process variables that can be monitored and controlled in each process as parameter nodes, it is represented as follows: , , and They are the first, the second, and the third, respectively. K Each process node;

[0040] Determine the directed graph ,in For a set of nodes, Let be the set of edges. For edge weights, It is the space of real numbers;

[0041] The defined directed graph is used as a causal graph.

[0042] Furthermore, the edge weights are determined using the following formula:

[0043] ;

[0044] In the formula, Represents a node The set of parent nodes, For weighted thresholds, Conditional mutual information is used to measure the conditional mutual information of random variables. Under the condition that, the other two random variables and The degree of interdependence between them; X u Let be the random variable corresponding to node u, representing the random variable that node u represents in the graph model. X v Let v be the random variable corresponding to node v. xu For random variables A specific value of , which, when calculating probabilities and summing, represents the random variable of node u. A specific instance or observation, x v For random variables A specific value, x pa(v) For a set of random variables A specific combination of values, when calculating probabilities and summing, represents a set of specific instances or observations of all random variables in the set of parent nodes of node v. P ( x u , x v , x pa(v) ) represents the joint probability, indicating the random variable , and At the same time, take specific values ​​respectively , and The probability, For the conditional joint probability, given a set of parent nodes Values Under the condition that node u takes the value And the value of node v is The joint probability, For a given set of parent nodes Values Under the condition that node u takes the value The probability, For a given set of parent nodes Values Under the condition that node v takes the value The probability of.

[0045] Furthermore, based on graph neural networks, causal reasoning is performed, mapping the causal semantic descriptions output by the multimodal detection and causal analysis model to causal nodes in a causal graph. The graph neural network simulates causal propagation and outputs a causal path report, including:

[0046] The causal semantic description generated for new sample images based on the multimodal detection and causal analysis model is mapped to a certain causal node in the constructed causal map through keyword extraction and semantic matching.

[0047] Starting from the aforementioned causal node, a graph neural network is run to simulate causal propagation;

[0048] Identify critical paths based on node activation state or attention weights.

[0049] Furthermore, the formula for calculating the attention weights in the graph neural network is as follows:

[0050] ;

[0051] In the formula, It is attention weight. It is a shared weight matrix. It is a learnable attention vector, and LeakyReLU is the activation function. This represents a vector concatenation operation, where exp is an exponential function. k It is a neighbor node index. T It is the transpose operation of a vector. h v It is a node The feature vector represents the node in the graph neural network. The input feature representation in the current layer, h u It represents a node A neighbor node eigenvectors, h k It is a node Another neighboring node eigenvectors, It is a node The set of neighboring nodes;

[0052] Aggregate functions are defined as follows:

[0053] ;

[0054] In the formula, AGG attn for, u core node The neighboring nodes;

[0055] The node feature update formula is:

[0056] ;

[0057] In the formula, For nodes In the The feature vector of the layer, W is a non-linear activation function. (l) For the first The layer is a trainable weight matrix, and CONCAT is a vector concatenation operation. For nodes In the The input feature vector of the layer, For neighboring nodes In the The feature vector of the layer.

[0058] Furthermore, when identifying critical paths based on node activation states or attention weights, the final source tracing probability is calculated using the following formula:

[0059] ;

[0060] In the formula, Given a defect, the probability that a node is the root cause is used. Softmax is a function that normalizes any real-valued vector into a probability distribution. MLP is a two-layer fully connected network. For nodes In the The feature vector of the layer.

[0061] The integrated method for industrial defect detection and root cause analysis provided in this application has at least the following beneficial effects:

[0062] 1. This application achieves synchronous and integrated output of defect localization and causal semantic description by training a single multimodal model. It merges the traditionally fragmented and sequential "detection, analysis, and tracing" steps into a collaborative, automated reasoning process. This changes the sequential work mode reliant on manual intervention, shortening the timescale of quality diagnosis from hours or even days to minutes or seconds, resulting in an order-of-magnitude improvement in response speed. Simultaneously, this method significantly reduces the continuous reliance on the professional experience of senior engineers, alleviating labor costs and providing a technological foundation for achieving 24 / 7 automated quality monitoring and preliminary diagnosis.

[0063] 2. This application innovatively constructs a cause-process-control parameter causal graph and utilizes graph neural networks to simulate the propagation path of defects in the process chain. This mechanism fundamentally surpasses traditional analysis methods based on statistical correlation or static rules, enabling the reconstruction of the true causal chain of defect generation and precise location of specific abnormal processes and control parameters. This reasoning process is a white-box analysis, and its output includes a clear logical path and a visualized causal chain, making the source tracing conclusions logically clear and convincing. This greatly enhances the interpretability and reliability of the diagnostic results, as well as users' trust in the system's decision-making, providing direct and clear action basis for on-site control.

[0064] 3. After each source tracing diagnosis, this application generates and accumulates structured causal path data. This data is continuously incorporated into the system's knowledge base, forming valuable and reusable causal knowledge assets for the enterprise. This not only breaks down the black box of the production process, making quality control transparent and traceable, but more importantly, it provides a high-quality data foundation for subsequent process optimization, parameter tuning, risk prediction, and intelligent decision-making. Therefore, the system evolves from a tool for passively responding to defects into an intelligent platform capable of continuous learning, supporting forward process design and process improvement, and possessing long-term application value. Attached Figure Description

[0065] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0066] Figure 1 A flowchart illustrating an integrated method for industrial defect detection and root cause analysis provided in this application embodiment;

[0067] Figure 2 A flowchart illustrating the construction process of a cross-stage structured dataset provided in this application embodiment;

[0068] Figure 3 This is a flowchart of the multimodal detection and causal analysis model training process provided in the embodiments of this application;

[0069] Figure 4 A flowchart illustrating the causal graph construction process provided in this application embodiment;

[0070] Figure 5 A flowchart for generating a traceability path report is provided for embodiments of this application.

[0071] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0072] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0073] The collection, storage, use, processing, transmission, provision, and disclosure of relevant data and information in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0074] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0075] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0076] This application provides an integrated method for industrial defect detection and root cause analysis, such as... Figure 1 The diagram shows a flowchart of an integrated method for industrial defect detection and root cause analysis provided in this application. The continuous learning model hot update method for industrial product defect detection includes the following steps S1 to S4.

[0077] S1: Construct a cross-stage structured dataset.

[0078] In this embodiment, the cross-process structured dataset includes workpiece images, equipment operating parameters, process settings collected from multiple process nodes, as well as associated defect locations, defect categories, text descriptions of defect causes, process identifiers, and control parameters.

[0079] In some embodiments, such as Figure 2 As shown, a cross-stage structured dataset is constructed through the following steps S101-S103:

[0080] S101: From the production line K Simultaneously collect workpiece images, equipment operating parameters, and process settings at each key process node.

[0081] For example, workpiece images, equipment operating parameters, and process settings are simultaneously acquired from multiple key process nodes on the production line. The key process nodes include at least one of stamping, welding, and painting; the equipment operating parameters include at least one of current, pressure, and speed; and the process settings include at least one of temperature and flow rate. Assume there are a total of [number missing] processes on the production line. Each key process, and data collection for each process. Number of samples, total number of samples k is the index of the critical process.

[0082] S102: Label the image with the location and category of defects, obtain the set of bounding box coordinates and the set of defect category labels, and associate them with the text description of the cause of the defect, process identification and control parameters.

[0083] Step S102 is the step of implementing data association and annotation. For example, for defect annotation, for the acquired images, professionals or auxiliary annotation tools use bounding boxes to mark the location of defects and specify the defect category, such as scratches, dents, or stains. For cause association: combining the records of the production line manufacturing execution system and maintenance work orders, each defect image is associated with its analyzed and confirmed root cause, and described in text using process terminology, such as: "Boundary burrs caused by wear of stamping die".

[0084] S103: Normalize the control parameters by integrating the workpiece image, bounding box coordinate set, defect category label set, text description of defect cause, process identifier, and normalized control parameters into a structured data record.

[0085] The control parameters are normalized using z-score normalization:

[0086] ;

[0087] In the formula, and They represent the first The vector of mean and standard deviation of all process parameters for each process step. and These are the process parameters before and after normalization.

[0088] The final cross-stage structured dataset constructed in this embodiment is represented as follows:

[0089] ;

[0090] in, Indicates the first i Images of the workpiece from one sample. express The set of bounding box coordinates for each defect. x j , y j , w j and h j These are the coordinates of the center point of the bounding box, the width of the bounding box, and the height of the bounding box, respectively. m i For the first The number of defects in the nth sample, for the nth sample The nth sample (i.e., the nth sample) (Image of Zhang's workpiece) This represents the total number of defect instances detected in the image. Represents a set of defect category labels. , C Represents the total number of categories. A textual description indicating the cause of the defect. express The process parameter vector for each process step Indicates process identification.

[0091] S2: Based on a cross-stage structured dataset, a multimodal detection and causal analysis model is trained.

[0092] In this embodiment, a course-based learning strategy can be used for training. This strategy includes two phases: phase one is defect perception, and phase two is joint understanding. The input to the multimodal detection and causal analysis model is a workpiece image, and the output is the defect localization result, including the detection box coordinates and the defect category and causal semantic description.

[0093] In some embodiments, such as Figure 3 As shown, based on a cross-stage structured dataset, a multimodal detection and causal analysis model is trained through the following steps S201-S202:

[0094] S201: Select a multimodal large model that integrates visual and language capabilities as the base network. The base network includes a text encoder and a language model.

[0095] Step S201 is the model selection and initialization step. For example, a multimodal large model integrating visual and language capabilities is selected as the base network. In this embodiment, Qwen2.5-VL is selected as the base model, and pre-trained weights in general scenarios are used for initialization to inherit its powerful feature extraction and semantic understanding capabilities.

[0096] S202: Based on a cross-stage structured dataset, a course learning strategy is used to train the basic network to obtain a multimodal detection and causal analysis model; wherein, the course learning strategy includes two stages, namely stage one and stage two.

[0097] Phase 1, Defect Perception Phase: Fix the text encoder and language model parts, and fine-tune the visual encoder and visual-language connector using only defect location and category labels.

[0098] In this embodiment, during the defect perception stage, the text encoder and large language model of the model are first fixed. Only the defect locations and category labels of the images in the dataset are used to fine-tune the model's visual encoder and visual-language connector. The text input is an instruction issued by the operator, such as: "Please detect whether there are defects in the image and give the reason." The goal of this stage is to enable the model to initially grasp the visual feature distribution of the image and its preliminary alignment with language. An improved detection loss function is used in this stage.

[0099] ;

[0100] In the formula, The total detection loss represents the overall detection task loss in the defect perception phase, and is a weighted sum of the classification loss, bounding box regression loss, and generalized intersection-union (GUU) loss. m The sample size refers to the total number of samples used in loss calculation within a batch. j The sample index indicates that the current calculation is for the [number]th [sample]. Loss per sample, L cls For category loss, For real category labels, To predict category labels, These are the weighting coefficients for the Smooth L1 loss. L reg For Smooth L1 loss, These are the actual bounding box coordinates. To predict the bounding box coordinates, The weighting coefficients for the generalized intersection-union loss are... L giou To improve positioning accuracy by comparing generalized intersection and union losses.

[0101] Focal Loss is used to handle class imbalance:

[0102] ;

[0103] In the formula, , Here, is a hyperparameter, and log is the natural logarithm, used in classification loss to calculate the logarithmic loss of the predicted probabilities. To predict class probabilities, the model refers to the probability of the first class. Each sample belongs to its true category. The predicted probability.

[0104] For illustrative purposes only, in this embodiment, the hyperparameter is set to... , . Smooth L1 loss is used for bounding box regression. The weight coefficients are set to... , .

[0105] Phase Two, Joint Understanding Phase: Unfreeze the text encoder and simultaneously optimize the defect localization loss and causal description generation loss.

[0106] In this embodiment, during the joint understanding phase, the text encoder is unfrozen, and the text describing the causes of defects is used as the training target. At this point, the model's input is an image, and the optimization objectives include both accurate bounding box regression / classification loss and text generation loss for generating accurate causal descriptions. Through this phase, the model learns to associate visual defect patterns with semantic process terminology. The overall loss function is:

[0107] ;

[0108] In the formula, L total For the total loss, L text For text loss, This refers to the multi-task weighting coefficient;

[0109] The text generation loss uses labeled smoothed cross-entropy:

[0110] ;

[0111] In the formula, This is the label smoothing factor (in this embodiment, the factor can be 2). For vocabulary size, For sequence length, t For sequence index, v For the vocabulary index, y t For the first The real token at position n is a one-hot vector representing the token at position n in the sequence. The vector represents the distribution of the actual next word in the vocabulary at each position. Only the position corresponding to the actual word is 1, and the rest are 0. For the first The predicted probability distribution of each location, For the sequence number The position, for the first in the vocabulary list The predicted probability of each word.

[0112] As an example only, multi-task weight coefficients .

[0113] S3: Construct a causal graph.

[0114] In this embodiment, the causal graph includes causal nodes, process nodes, and parameter nodes, as well as directed edges between nodes defined based on process knowledge or historical data.

[0115] In some embodiments, such as Figure 4 As shown, based on the dataset collected in step S1, relevant domain knowledge is structured to provide a reasoning graph for automatic source tracing. A causal graph is constructed through the following steps S301-S305:

[0116] S301: The core keywords of various defect cause descriptions output by the multimodal detection and cause analysis model are used as cause nodes.

[0117] In this embodiment, the causal nodes are the core keywords derived from the various defect cause descriptions output by the model in step 2, such as "mold wear" and "uneven coating". The causal nodes are denoted as... , , and They are the first, the second, and the third, respectively. M There are 1 causal nodes, totaling 1 Each causal node.

[0118] S302: Use specific steps in the production process as process nodes.

[0119] In this embodiment, process nodes represent specific steps in the production process, such as "stamping," "welding," and "painting." Process nodes are denoted as... , , and They are the first, the second, and the third, respectively. L Each process node, totaling Each process node.

[0120] S303: Use key process variables that can be monitored and controlled in each process as parameter nodes.

[0121] In this embodiment, parameter nodes represent key process variables that can be monitored and controlled in each process step, such as "stamping pressure," "welding current," and "paint flow rate." Parameter nodes are denoted as... , , and They are the first, the second, and the third, respectively. K Each process node.

[0122] S304: Determine the directed graph ,in For a set of nodes, Let be the set of edges. For edge weights, It is the space of real numbers.

[0123] In step S304, the core lies in defining edge relationships, i.e., determining edge weights. Based on process expert knowledge or historical data statistical analysis, directed edges between nodes are defined. For example, a causal path is established: "Die wear -- (caused by) --> stamping -- (controlled by) --> stamping pressure". A directed graph is then defined. ,in For a set of nodes, Let be the set of edges. Edge weights are calculated using conditional mutual information.

[0124] ;

[0125] In the formula, Represents a node The set of parent nodes, For weighted thresholds, Conditional mutual information is used to measure the conditional mutual information of random variables. Given (the set of parent nodes of node v), the other two random variables and The degree of interdependence between them. X u Let be the random variable corresponding to node u, representing the random variable that node u represents in the graph model. X v Let v be the random variable corresponding to node v. x u For random variables A specific value of , which, when calculating probabilities and summing, represents the random variable of node u. A specific instance or observation, x v For random variables A specific value, x pa(v) For a set of random variables A specific combination of values, when calculating probabilities and summing, represents a set of specific instances or observations of all random variables in the set of parent nodes of node v. P ( x u , x v , x pa(v) ) represents the joint probability, which indicates the random variable (node ​​u) (node ​​v) and (The set of parent nodes of node v) simultaneously take specific values. , and The probability, For the conditional joint probability, given a set of parent nodes Values Under the condition that node u takes the value And the value of node v is The joint probability, For a given set of parent nodes Values Under the condition that node u takes the value The probability, For a given set of parent nodes Values Under the condition that node v takes the value The probability of.

[0126] S305: Treat a defined directed graph as a causal graph.

[0127] In this embodiment, the constructed graph is stored in a graph database such as Neo4j for subsequent inference calls.

[0128] S4: Based on graph neural networks, the source tracing reasoning is performed, and the cause semantic descriptions output by the multimodal detection and cause analysis model are mapped to cause nodes in the cause graph. The cause propagation is simulated through graph neural networks, and the source tracing path report is output.

[0129] In this embodiment, the traceability path report includes specific processes and control parameters.

[0130] In some embodiments, such as Figure 5 As shown, step S4 specifically includes the following steps S401-S403.

[0131] S401: The causal semantic description generated for new sample images based on the multimodal detection and causal analysis model is mapped to a certain causal node in the constructed causal graph through keyword extraction and semantic matching.

[0132] In this embodiment, the causal semantic description generated for the new sample image by the multimodal detection and causal analysis model in step S2 is mapped to a specific causal node in the causal graph constructed in step S3 through keyword extraction and semantic matching.

[0133] S402: Starting from the causal node, run a graph neural network to simulate causal propagation.

[0134] Step S402 is the graph neural network inference process. Using the causal node determined in step S401 as the starting signal, a graph neural network GraphSAGE or message passing algorithm is run on the graph. The algorithm simulates the propagation process of "causal influence" on the graph, calculating and updating the representations of all nodes in the graph. Specifically, this implementation uses the GraphSAGE algorithm with an attention mechanism. First, for the core node... With each of its neighboring nodes Calculate a differentiated attention weight To measure neighboring nodes For core nodes The importance of the (center point). The formula for calculating this weight is as follows:

[0135] ;

[0136] In the formula, It is a shared weight matrix. It is a learnable attention vector. LeakyReLU is the activation function, a variant of the Corrected Linear Unit (ReLU). Unlike ReLU, which directly sets negative values ​​to zero, LeakyReLU assigns a small non-zero slope to negative input values. This represents the vector concatenation operation, where exp is the exponential function expressed as the natural constant. Exponential operations with base 0. k It is the neighbor node index, used as the denominator in calculating the attention weights to traverse and sum the nodes. All neighboring nodes. It is the code of one of the neighboring nodes. T It is the transpose operation of a vector. h v It is a node The feature vector represents the node in the graph neural network. The input feature representation in the current layer, h u It represents a node A neighbor node eigenvectors, h k It is a node Another neighboring node eigenvectors, It is a node The set of neighboring nodes. The negative slope of LeakyReLU is set to 0.2.

[0137] The aggregation function AGG is defined as a weighted summation operation based on the attention mentioned above, and its formal expression is as follows:

[0138] ;

[0139] In the formula, AGG attn for, u core node The neighboring nodes;

[0140] The above function achieves selective focusing on neighbor information by assigning high weights to important neighbors.

[0141] Subsequently, the attention weights were calculated. Through aggregate functions It was used for information dissemination. The formula for updating the node features of a layer is:

[0142] ;

[0143] In the formula, For nodes In the The feature vector of the layer, The activation function is a non-linear activation function, here it is Sigmoid, W (l) For the first The layer is a trainable weight matrix, and CONCAT is a vector concatenation operation. For nodes In the The input feature vector of the layer, For neighboring nodes In the The feature vector of the layer.

[0144] This allows the model to automatically focus on neighboring nodes that are more important to the current tracing task when updating node representations, thereby achieving accurate tracing of the root cause.

[0145] S403: Identify critical paths based on node activation state or attention weights.

[0146] Step S403 is used to achieve path identification and output. Specifically, by analyzing the activation state or attention weight of each node after GNN inference, the upstream process node and control parameter node with the greatest influence starting from the causal node are automatically identified. The final formula for calculating the traceability probability is:

[0147] ;

[0148] Given a defect, softmax represents the probability that a node is the root cause. Softmax is the Softmax function, a function that normalizes any real-valued vector into a probability distribution. It ensures that all output values ​​are in the interval (0,1) and that the sum of all output values ​​is 1. For nodes In the The feature vector of the last layer (i.e., the final layer) is the high-level feature representation of the nodes obtained after multiple layers of message passing and aggregation, which is the final output of the graph neural network. It contains the node itself and its contextual information in the graph structure, and is used for the final root cause probability calculation. It is a two-layer fully connected network using the GeLU activation function.

[0149] Ultimately, this method can output a complete traceability report, clearly indicating the specific process where the root cause of the problem lies and the control parameters that need to be adjusted.

[0150] The integrated method for industrial defect detection and root cause analysis proposed in this application demonstrates transformative application value in smart manufacturing scenarios. The following is an illustration through a typical user scenario:

[0151] A quality issue involving "boundary microcracks" was detected in the stamping workshop of an automobile manufacturing plant. In the traditional model, while automated vision inspection systems can identify defective workpieces, they only provide basic information about the location of the defect, failing to answer the crucial question of why the defect occurred. Quality engineers need to spend hours or even days manually reviewing and analyzing historical data on dozens of process parameters, such as stamping pressure, die temperature, and feeding speed, attempting to find patterns through experience-based comparisons. This sequential analysis model, relying on expert experience, is not only inefficient but also frequently leads to diagnostic failures due to the complexity of factors and unclear correlations, resulting in continuous quality losses and wasted production capacity.

[0152] After applying this method, when the system detects boundary microcrack defects again, the multimodal large model immediately and simultaneously completes defect localization and cause analysis, displaying the defect location and the cause description of uneven material flow due to mold wear on the interface. This process integrates the two traditionally separate steps into one action, achieving real-time perception-cognition synergy.

[0153] More importantly, the system further uses this causal description as input to initiate graph neural network inference within a pre-constructed cause-process-control parameter causal graph. By simulating the propagation path of defects in the process chain, the system automatically identifies the complete causal chain of mold wear → stamping process → lower die clearance and generates a structured traceability report. Based on this report, on-site engineers can directly and precisely adjust the lower die clearance parameter.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An integrated method for industrial defect detection and root cause analysis, characterized in that, include: Construct a cross-process structured dataset, which includes workpiece images, equipment operating parameters, process settings collected from multiple process nodes, as well as associated defect locations, defect categories, defect cause text descriptions, process identifiers, and control parameters; Based on the cross-stage structured dataset, a multimodal detection and causal analysis model is trained. The input of the multimodal detection and causal analysis model is a workpiece image, and the output is the defect localization result, including the detection box coordinates and the defect category and causal semantic description. Construct a causal graph, which includes causal nodes, process nodes, and parameter nodes, as well as directed edges between nodes defined based on process knowledge or historical data; Based on graph neural networks, the causal semantic descriptions output by the multimodal detection and causal analysis model are mapped to causal nodes in the causal graph. The graph neural network simulates causal propagation and outputs a causal path report. Based on the aforementioned cross-stage structured dataset, a multimodal detection and causal analysis model is trained, including: A multimodal large model that integrates visual and language capabilities is selected as the base network, which includes a text encoder and a language model; Based on the aforementioned cross-stage structured dataset, a course learning strategy is employed to train the basic network, resulting in a multimodal detection and causal analysis model; wherein, the course learning strategy includes: Defect perception stage: Fix the text encoder and language model parts, and fine-tune the visual encoder and visual language connector only using defect location and category labels; Joint understanding phase: Unfreeze the text encoder and simultaneously optimize the defect localization loss and causal description generation loss; This approach leverages graph neural networks for source tracing reasoning, mapping the semantic descriptions of causes output by multimodal detection and causal analysis models to causal nodes in a causal graph. By simulating causal propagation through graph neural networks, it outputs a source tracing path report, including: The causal semantic description generated for new sample images based on the multimodal detection and causal analysis model is mapped to a certain causal node in the constructed causal map through keyword extraction and semantic matching. Starting from the aforementioned causal node, a graph neural network is run to simulate causal propagation; Identify critical paths based on node activation state or attention weights.

2. The integrated method for industrial defect detection and root cause analysis according to claim 1, characterized in that, Construct a cross-stage structured dataset using the following method: Workpiece images, equipment operating parameters, and process settings are collected synchronously from K key process nodes on the production line. The image is labeled with the location and category of defects to obtain a set of bounding box coordinates and a set of defect category labels, and then associated with the text description of the defect cause, process identification and control parameters. The control parameters are normalized, and the workpiece image, bounding box coordinate set, defect category label set, text description of defect cause, process identifier, and normalized control parameters are integrated into a structured data record.

3. The integrated method for industrial defect detection and root cause analysis according to claim 1, characterized in that, The defect perception stage employs an improved detection loss function: In the formula, The total detection loss represents the overall detection task loss in the defect perception phase, and is a weighted sum of the classification loss, bounding box regression loss, and generalized intersection-union (GUU) loss. m The sample size refers to the total number of samples used in loss calculation within a batch. j The sample index indicates that the current calculation is for the [number]th [sample]. Loss per sample, L cls For category loss, For real category labels, To predict category labels, These are the weighting coefficients for the SmoothL1 loss. L reg For SmoothL1 loss, These are the actual bounding box coordinates. To predict the bounding box coordinates, The weighting coefficients for the generalized intersection-union loss are... L giou For generalized intersection and comparison of losses; L cls The calculation formula is: In the formula, , Here, is a hyperparameter, and log is the natural logarithm, used in classification loss to calculate the logarithmic loss of the predicted probabilities. To predict class probabilities, the model refers to the probability of the first class. Each sample belongs to its true category. The predicted probability.

4. The integrated method for industrial defect detection and root cause tracing according to claim 3, characterized in that, The total loss function for the joint understanding phase is: In the formula, L total For the total loss, L text For text loss, This refers to the multi-task weighting coefficient; Text loss L text The calculation formula is: In the formula, For label smoothing factor, For vocabulary size, For sequence length, t For sequence index, v For the vocabulary index, y t For the first The real token at position n is a one-hot vector representing the token at position n in the sequence. The vector represents the distribution of the actual next word in the vocabulary at each position. Only the position corresponding to the actual word is 1, and the rest are 0. For the first The predicted probability distribution of each location, For the sequence number The position, for the first in the vocabulary list The predicted probability of each word.

5. The integrated method for industrial defect detection and root cause analysis according to claim 1, characterized in that, Construct a causal graph using the following method: Using the core keywords of various defect cause descriptions output by the multimodal detection and causal analysis model as causal nodes, it is represented as follows: , , and They are the first, the second, and the third, respectively. M One causal node; Using specific steps in the production process as process nodes, it is represented as , , and They are the first, the second, and the third, respectively. L Each process node; Using the key process variables that can be monitored and controlled in each process as parameter nodes, it is represented as follows: , , and They are the first, the second, and the third, respectively. K Each process node; Determine the directed graph ,in For a set of nodes, Let be the set of edges. For edge weights, It is the space of real numbers; The defined directed graph is used as a causal graph.

6. The integrated method for industrial defect detection and root cause analysis according to claim 5, characterized in that, The edge weights are determined using the following formula: In the formula, Represents a node The set of parent nodes, For weighted thresholds, Conditional mutual information is used to measure the conditional mutual information of random variables. Under the condition that, the other two random variables and The degree of interdependence between them; X u Let be the random variable corresponding to node u, representing the random variable that node u represents in the graph model. X v Let v be the random variable corresponding to node v. x u For random variables A specific value of , which, when calculating probabilities and summing, represents the random variable of node u. A specific instance or observation, x v For random variables A specific value, x pa(v) For a set of random variables A specific combination of values, when calculating probabilities and summing, represents a set of specific instances or observations of all random variables in the set of parent nodes of node v. P ( x u , x v , x pa(v) ) represents the joint probability, indicating the random variable , and At the same time, specific values ​​are taken respectively. , and The probability, For the conditional joint probability, given a set of parent nodes Values Under the condition that node u takes the value And the value of node v is The joint probability, For a given set of parent nodes Values Under the condition that node u takes the value The probability, For a given set of parent nodes Values Under the condition that node v takes the value The probability of.

7. The integrated method for industrial defect detection and root cause analysis according to claim 1, characterized in that, The formula for calculating the attention weights in the graph neural network is as follows: In the formula, It is attention weight. It is a shared weight matrix. It is a learnable attention vector, and LeakyReLU is the activation function. This represents a vector concatenation operation, where exp is an exponential function. k It is a neighbor node index. T It is the transpose operation of a vector. h v It is a node The feature vector represents the node in the graph neural network. The input feature representation in the current layer, h u It represents a node A neighbor node eigenvectors, h k It is a node Another neighboring node eigenvectors, It is a node The set of neighboring nodes; Aggregate functions are defined as follows: In the formula, AGG attn for, u core node The neighboring nodes; The node feature update formula is: In the formula, For nodes In the The feature vector of the layer, W is a non-linear activation function. (l) For the first The layer is a trainable weight matrix, and CONCAT is a vector concatenation operation. For nodes In the The input feature vector of the layer, For neighboring nodes In the The feature vector of the layer.

8. The integrated method for industrial defect detection and root cause tracing according to claim 1, characterized in that, When identifying critical paths based on node activation states or attention weights, the final source tracing probability is calculated using the following formula: In the formula, Given a defect, the probability that a node is the root cause is used. Softmax is a function that normalizes any real-valued vector into a probability distribution. MLP is a two-layer fully connected network. For nodes In the The feature vector of the layer.

Images