A Deep Learning-Based Intelligent Detection Method and System for Surface Defects in Automotive Parts

By combining multimodal image data and Bayesian causal graph models, the problem of misjudgment in the detection of surface defects in automotive parts has been solved, achieving efficient and accurate defect detection and adapting to the needs of multi-variety, small-batch production.

CN123118480APending Publication Date: 2026-11-17JINGZHOU YUDE MASCH MFG CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202611228904.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-13
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies for detecting surface defects in automotive parts suffer from low efficiency, high false detection rate, difficulty in adapting to the needs of multi-variety, small-batch production, and a tendency to misjudge process-related pseudo-defects as real defects.

Method used

A feature decoupling model is constructed using multimodal image data and generative adversarial networks. A Bayesian causal graph model is then used for feature extraction and defect inference. Texture and spectral features are extracted using multi-branch convolutional neural networks and lightweight convolutional networks. Finally, a hybrid detection model is constructed to distinguish between real defects and pseudo-defects caused by manufacturing processes.

Benefits of technology

It improves the accuracy of detecting minute defects in complex materials, reduces the error rejection rate of qualified parts, reduces the cost of manual re-inspection, adapts to the needs of multi-variety small-batch production, and reduces the cost of algorithm maintenance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118480A_ABST
    Figure CN123118480A_ABST
Patent Text Reader

Abstract

This invention discloses a deep learning-based intelligent detection method and system for surface defects in automotive parts, relating to the field of surface quality inspection technology. It includes: S1: Constructing a training dataset using multimodal industrial image data and multi-angle surface images, and training an initial hybrid detection model using a front-door intervention algorithm; S2: Using the training dataset as input to the final hybrid detection model, outputting multimodal features, and combining them with spatial location information to obtain a fused enhanced feature map; S3: Using the fused enhanced feature map as input to a Bayesian causal graph model, outputting the corresponding joint probability distribution. This invention avoids the problem of traditional models misjudging normal process traces as defects, reducing the erroneous rejection of qualified parts, lowering the cost of manual re-inspection, and reducing the risk of production line downtime.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of surface quality inspection technology, specifically to an intelligent detection method and system for surface defects in automotive parts based on deep learning. Background Technology

[0002] With the rapid development of the automotive industry towards high precision and automation, the surface quality of automotive parts directly determines the assembly accuracy, safety performance, and market compliance of the entire vehicle. In traditional production processes, surface defect detection mainly relies on manual visual inspection, where quality inspectors identify defects such as scratches, cracks, dents, or burrs with the naked eye under specific lighting conditions. However, this traditional method has significant technical bottlenecks: on the one hand, manual inspection is inefficient and difficult to match the production pace of modern high-speed automated production lines; on the other hand, inspection results are easily affected by subjective experience, visual fatigue, and emotional fluctuations, leading to high rates of missed and false detections, making it difficult to guarantee consistent product quality. Furthermore, defects in some small or hidden areas are often difficult for the human eye to detect, failing to meet the stringent process control requirements of quality management systems such as IATF 16949.

[0003] To replace manual inspection, traditional machine vision-based inspection technologies have been widely used. This method typically utilizes industrial cameras to acquire images and combines them with digital image processing algorithms such as edge detection, thresholding, and filtering to extract defect features. While traditional machine vision has improved inspection speed to some extent, its core limitation lies in its heavy reliance on manually designed feature operators. Automotive parts have diverse surface materials (e.g., highly reflective metals, translucent plastics) and complex, variable defect morphologies. Traditional algorithms exhibit poor robustness and insufficient generalization ability when facing complex textured backgrounds or fluctuating lighting. If a part model changes or the defect morphology undergoes even a slight alteration, the complex algorithm logic needs to be redesigned, resulting in high maintenance costs and difficulty in adapting to the flexible production needs of diverse product types and small batches.

[0004] Chinese invention patent application CN121982007A discloses a method, apparatus, device, and medium for defect detection of medical consumables, comprising: acquiring multimodal data of defect-free medical consumables as normal sample data; the multimodal data includes visible light images, infrared images, and three-dimensional point cloud information; adding interference to the normal sample data to generate pseudo-defect sample data; determining multimodal training data pairs based on the normal sample data and the pseudo-defect sample data; extracting features from the multimodal training data pairs to obtain multimodal feature information; aligning the multimodal feature information through comparative learning to obtain target feature information; inputting the target feature information into a defect detection model for self-supervised training, so as to use the trained defect detection model to detect defects in the medical consumables under test; and the defect detection model performing defect detection based on the difference between the target feature information and its reconstruction result. This solution can expand the defect detection coverage while reducing the dependence on defect annotation.

[0005] However, the above and similar technical solutions still have the following shortcomings: During the production process, automotive parts (such as gearbox housings, engine blocks, etc.) are very prone to being covered with oil stains, coolant residues, fingerprints, or shadows cast by lifting tools. Moreover, their grayscale distribution, texture features, and morphological features are highly similar to real scratches, dents, or rust defects. Therefore, when existing deep learning models distinguish between real defects and process-related pseudo-defects, they are very likely to misjudge normal interference as surface defects, which will lead to a large number of qualified parts being incorrectly rejected. This will not only increase unnecessary manual re-inspection costs and production downtime, but also reduce the overall efficiency of equipment and the reliability of intelligent manufacturing systems. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for intelligent detection of surface defects in automotive parts based on deep learning, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a deep learning-based intelligent detection method for surface defects in automotive parts, comprising:

[0008] S1: Feature decoupling model construction: A training dataset is constructed using multimodal industrial image data and multi-angle surface images of the parts to be detected, and the initial hybrid detection model is trained using the front-door intervention algorithm;

[0009] S2: Enhanced processing of feature extraction: The training dataset is used as the input of the final hybrid detection model, and the corresponding multimodal features are obtained from the output. The multimodal features are combined with spatial location information to obtain the corresponding fusion enhanced feature map.

[0010] S3: Defect reasoning of causal graphs: Using the fused and enhanced feature map as input to the Bayesian causal graph model, the output obtains the corresponding joint probability distribution.

[0011] Furthermore, the initial hybrid detection model is trained, including:

[0012] S1.1: Modal mapping: Collect multimodal industrial image data of the parts to be inspected, construct the corresponding training dataset, and at the same time construct pseudo-defect samples and counterfactual samples through generative adversarial network, and perform enhancement processing on the training dataset to obtain an enhanced training dataset;

[0013] S1.2: Model Construction: Based on the training dataset, set up the morphological texture branch and the spectral color branch, and combine the morphological texture branch, the spectral color branch and the causal structure learning network to construct the corresponding initial hybrid detection model;

[0014] S1.3: Intervention Training: Using the depth images in the enhanced training dataset, set corresponding mediating variables, and determine the correlation coefficient between each feature channel and the confusion variable by statistical correlation modeling based on the observed variables and confusion variables corresponding to the initial mixed detection model. At the same time, set corresponding training objectives based on the classification loss between the true defect probability and the defect prediction value and the correlation coefficient, and train the initial mixed detection model according to the training objectives to obtain the corresponding final mixed detection model.

[0015] Furthermore, the training dataset is augmented, including:

[0016] S1.1.1: Initial Dataset Construction: Through the set sensor network, multimodal industrial image data of the parts to be inspected is collected, including RGB grayscale images, polarized light images, depth images and multi-angle images. According to the identification ID of the parts to be inspected, the multimodal industrial image data is bound to the data. At the same time, according to the corresponding defect labels, corresponding training sample units are constructed, and the training sample units are combined to construct the corresponding initial training dataset.

[0017] S1.1.2: Dataset Augmentation: Using a generative adversarial network, construct pseudo-defect samples and counterfactual samples of the parts to be detected, and obtain the corresponding training sample units of the pseudo-defect samples and the training sample units of the counterfactual samples. At the same time, combine the training sample units of the pseudo-defect samples, the training sample units of the counterfactual samples and the initial training dataset to obtain the corresponding augmented training dataset.

[0018] Furthermore, the corresponding initial hybrid detection model is constructed, including:

[0019] S1.2.1: Variable definition: Based on the constituent elements corresponding to each training sample unit in the training dataset, set the observation variable, target variable, and confounding variable of the initial mixed detection model, and determine the causal relationship between the observation variable, target variable, and confounding variable through industrial field mechanisms;

[0020] S1.2.2: Feature extraction: Using a multi-branch convolutional neural network model, a lightweight convolutional network model, a physical-driven feature mapping model, and a frequency domain filtering model, corresponding morphological texture branches and spectral color branches are set to extract the corresponding texture geometric feature vectors and spectral material feature vectors.

[0021] S1.2.3: Path modeling: By setting the causal modeling layer, the texture geometric feature vector and spectral material feature vector are mapped to the corresponding causal representation space.

[0022] Furthermore, the multi-branch convolutional neural network model is configured with small-scale, medium-scale, and large-scale branches in parallel. Through the small-scale, medium-scale, and large-scale branches, corresponding edge gradient feature maps, local morphological change feature maps, and regional structural anomaly feature maps are output. At the same time, the edge gradient feature maps, local morphological change feature maps, and regional structural anomaly feature maps are compressed and stitched together to obtain a morphological texture preprocessed RGB grayscale image and a morphological texture preprocessed depth image, and the corresponding texture geometric feature vectors are extracted.

[0023] The lightweight convolutional network model, the physics-driven feature mapping model, and the frequency domain filtering model are used to obtain the corresponding RGB grayscale image and polarized light image after spectral color preprocessing. At the same time, the RGB grayscale image and polarized light image after spectral color preprocessing are spliced ​​and mapped to determine the corresponding spectral material feature vector.

[0024] Furthermore, the correlation coefficients of each feature channel are combined to construct a corresponding feature sparsity penalty term. Simultaneously, a logistic regression classification head is used to output the corresponding true defect probability. The classification loss between the predicted and actual defect probabilities is measured based on the true defect probability and the loss function. The classification loss and the feature sparsity penalty term are then combined to set the corresponding training objective, specifically:

[0025]

[0026] in: Let be the mathematical expectation of the set of random variables. The classification loss is the difference between the true defect label and the predicted defect probability. As a weighting factor, To mask the loss, For the target variable, To predict the probability of defects.

[0027] Furthermore, the corresponding fusion-enhanced feature maps are obtained, including:

[0028] S2.1: Multimodal extraction: Through the final hybrid detection model, the texture semantic feature map, gray-level gradient feature map, edge morphology feature map, and material saliency map are output. Based on the texture semantic feature map and depth image, the corresponding defect region segmentation result is obtained. Based on the RGB gray-level image, polarized light image, and defect region segmentation result, the corresponding spectral discrimination result is determined. At the same time, the texture semantic feature map, gray-level gradient feature map, edge morphology feature map, and material saliency map are feature aligned and weighted and fused to obtain the corresponding initial enhanced feature map.

[0029] S2.2: Attention Fusion: Based on the obtained texture semantic feature map and material saliency map, the corresponding morphological texture pooling features and spectral color pooling features are obtained, and the morphological texture pooling features and spectral color pooling features are concatenated to obtain the corresponding pooling aggregate features. At the same time, the pooling aggregate features are mapped through a convolutional mapping network model, and the mapped pooling aggregate features are combined with the initial enhanced feature map to obtain the final enhanced feature map.

[0030] Furthermore, the Bayesian causal graph model is connected to a multilayer perceptron classifier, and the multilayer perceptron classifier is configured with two defect classifications, including a real defect class and a process-related pseudo-defect class.

[0031] A deep learning-based intelligent detection system for surface defects in automotive parts uses any one of the aforementioned deep learning-based intelligent detection methods for surface defects in automotive parts.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] Firstly, this invention uses a front-door intervention algorithm to block the false association between process-related pseudo-defects and defect judgments, and combines a Bayesian causal graph model to output the joint probability distribution of real defects and process-related pseudo-defects. With the help of a two-layer classifier, it achieves accurate differentiation between the two types of samples, thereby avoiding the problem of traditional models misjudging normal process traces as defects, reducing the erroneous rejection of qualified parts, and lowering the cost of manual re-inspection and the risk of production line downtime.

[0034] Secondly, this invention acquires RGB grayscale, polarized light, depth, and multi-angle, multi-source data, and combines it with a dual-branch feature extraction network for morphology, texture, and spectral color. This not only covers the texture edge features of geometric defects such as scratches and pits, but also captures the spectral differences of material reflection and surface roughness. At the same time, it combines spatial location information for weighted fusion, which solves the detection blind spots under complex materials such as highly reflective metals and translucent plastics, and improves the detection rate of minute defects.

[0035] Thirdly, this invention constructs pseudo-defects and counterfactual samples through generative adversarial networks to enhance the training set, thereby covering diverse defect morphologies without requiring a large amount of real defect annotation data. At the same time, the design of dual-branch networks and causal modeling allows the model to adapt to new product categories without reconstructing the core logic as the part model changes. Only a few parameters need to be fine-tuned, thus meeting the rapid model changeover requirements of multi-variety, small-batch production lines and reducing algorithm maintenance costs. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the intelligent detection method for surface defects of automotive parts in this invention.

[0037] Figure 2 This is a schematic diagram illustrating the process of obtaining the enhanced training dataset in this invention;

[0038] Figure 3 This is a schematic diagram illustrating the construction process of the hybrid detection model in this invention;

[0039] Figure 4 This is a flowchart illustrating the enhanced feature extraction process of this invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] refer to Figure 1 This embodiment provides a deep learning-based intelligent detection method for surface defects in automotive parts. The method specifically includes the following steps:

[0042] Step S1: Feature Decoupling Model Construction. This involves acquiring multimodal industrial image data and multi-angle surface images of the parts to be inspected, constructing a training dataset containing both real defect samples and pseudo-defect samples. Simultaneously, by setting morphological texture and spectral color branches, a corresponding two-stream feature extraction network is constructed. This two-stream feature extraction network is then combined with a causal structure learning network to construct the initial hybrid detection model.

[0043] Furthermore, by using the front-door intervention algorithm, the initial hybrid detection model is trained based on the constructed training dataset to remove false correlations caused by process-related false defects in the initial hybrid detection model, thereby obtaining the corresponding final hybrid detection model.

[0044] Step S2: Enhancement of Feature Extraction. The training dataset from Step S1 is used as input to the final hybrid detection model, outputting corresponding texture features, spectral features, polarization saliency maps, and depth deformation features. Simultaneously, the output texture features, spectral features, polarization saliency maps, and depth deformation features are combined with corresponding spatial location information to obtain the corresponding fused enhanced feature map.

[0045] Step S3: Defect reasoning of the causal graph. This involves using the fused enhanced feature map obtained in Step S2 as input to a Bayesian causal graph model (such as a differentiable causal Bayesian network or a neural symbolic causal graph), and outputting a joint probability distribution including confidence levels. Specifically:

[0046]

[0047] in: For joint posterior probabilities, For causal conditional probability, Infer probability for the intermediary.

[0048] Furthermore, the Bayesian causal graph model in this embodiment is connected to a multilayer perceptron classifier, and the multilayer perceptron classifier in this embodiment is configured with two types of defect classification, including a true defect class and a process-related pseudo-defect class. That is to say, the corresponding defect category and defect location are obtained through the configured multilayer perceptron classifier.

[0049] This embodiment also provides a deep learning-based intelligent detection system for surface defects in automotive parts, which uses the aforementioned deep learning-based intelligent detection method for surface defects in automotive parts.

[0050] In this embodiment, a corresponding training dataset is constructed using multimodal industrial image data and multi-angle surface images of the parts to be detected. Simultaneously, a dual-stream feature extraction network and cross-modal contrastive learning are combined to construct an initial hybrid detection model. This initial hybrid detection model is then trained and updated using the constructed training dataset. (Reference) Figure 2 and Figure 3 This embodiment provides a model construction method for feature decoupling, which specifically includes the following steps:

[0051] Step S1.1: Modality Mapping. This involves acquiring multimodal industrial image data of the parts to be inspected using a sensor network, and constructing a corresponding training dataset. Simultaneously, a generative adversarial network (GAN) is used to construct corresponding pseudo-defect samples and counterfactual samples, which are then used to augment the training dataset. Details are as follows:

[0052] Step S1.1.1: Initial Dataset Construction. This involves acquiring multimodal industrial image data of the parts to be inspected through a sensor network set up on the automotive parts production line. Specifically, using industrial area array cameras, polarization cameras, laser profilometers, and multi-position cameras, RGB grayscale images, polarized light images, depth images, and multi-angle images of the parts to be inspected are acquired. Simultaneously, based on the identifier ID of the parts to be inspected, the RGB grayscale image, polarized light image, depth image, and multi-angle image corresponding to each part are data-bound to obtain the multimodal sample unit corresponding to each part, specifically:

[0053]

[0054] in: Let i be the multimodal sample unit of the i-th component to be tested. Let i be the RGB grayscale image of the i-th component to be detected. This is the polarized light image of the i-th component to be detected. Let i be the depth image of the i-th component to be detected. Let be the set of multi-angle images of the i-th component to be detected.

[0055] Furthermore, based on a preset number of multimodal sample units corresponding to the parts to be tested (which can be specifically set according to actual needs, and therefore not specifically described in this embodiment), all multimodal sample units are manually grouped according to the actual process state to determine the defect label corresponding to each multimodal sample unit, including real defect labels and pseudo-defect labels. Simultaneously, each multimodal sample unit and its corresponding real defect label and pseudo-defect label are combined to construct the corresponding training sample unit, specifically:

[0056]

[0057] in: Let i be the training sample unit for the i-th component to be detected. Let i be the multimodal sample unit of the i-th component to be tested. The label for the i-th component to be inspected is the actual defect label (i.e., when there is an actual physical defect, the corresponding actual defect label is set to 1; when there is no actual physical defect, the corresponding actual defect label is set to 0). The label for the i-th part to be inspected is a pseudo-defect label for manufacturing process (i.e., when a pseudo-defect for manufacturing process exists, the corresponding pseudo-defect label is set to 1; when no pseudo-defect for manufacturing process exists, the corresponding pseudo-defect label is set to 0).

[0058] In other words, based on the training sample units of each component to be detected, all training sample units are combined to construct the corresponding initial training dataset.

[0059] Step S1.1.2: Dataset Augmentation. This involves using a generative adversarial network to construct corresponding pseudo-defect samples (i.e., overlaying oil stains, coolant flow marks, fingerprints, or hanger shadows onto a real, defect-free image without altering the physical geometry of the component) and counterfactual samples (i.e., keeping the depth information of the component unchanged while altering only surface texture details or lighting direction and intensity). Training sample units corresponding to the pseudo-defect samples and counterfactual samples are then obtained.

[0060] Furthermore, based on the training sample units of the obtained pseudo-defect samples and the training sample units of the counterfactual samples, the training sample units of the pseudo-defect samples, the training sample units of the counterfactual samples, and the initial training dataset constructed in step S1.1.1 are combined to obtain the corresponding enhanced training dataset.

[0061] Step S1.2: Model Construction. Based on the multimodal sample units obtained in Step S1.1.1, corresponding morphological texture branches are set using the RGB grayscale and depth images from the multimodal sample units, and corresponding spectral color branches are set using the RGB grayscale and polarized light images from the multimodal sample units. Simultaneously, the set morphological texture and spectral color branches are combined with the causal structure learning network to construct the corresponding initial hybrid detection model. Specifically, as follows:

[0062] Step S1.2.1: Variable Definition. This involves defining the variables of the initial hybrid detection model based on the constituent elements of each training sample unit in the initial training dataset from step S1.1.1. Specifically, it involves setting the variable sizes of the initial hybrid detection model based on the multimodal sample units, real defect labels, and process-related pseudo-defect labels corresponding to the parts to be detected. In other words, it involves setting the observation variables in the initial hybrid detection model based on the multimodal sample units corresponding to the parts to be detected. It also involves setting the target variables in the initial hybrid detection model based on the real defect labels corresponding to the parts to be detected. Finally, it involves setting the confusion variables in the initial hybrid detection model based on the process-related pseudo-defect labels corresponding to the parts to be detected.

[0063] Furthermore, based on the observed variables, target variables, and confounding variables set in the initial mixed detection model, the causal relationships among the observed variables, target variables, and confounding variables are determined through industrial field mechanisms, specifically:

[0064] Observed variable → Target variable: This represents the inference of the corresponding true defect label through multimodal sample units.

[0065] Confounding variable → Observed variable: This means that the multimodal sample unit is changed by using pseudo-defects in the process.

[0066] Confounding variable ↛ Target variable: Indicates that pseudo-defects in the manufacturing process are not equivalent to real defect labels.

[0067] Step S1.2.2: Feature Extraction. This involves using a multi-branch convolutional neural network model and setting corresponding morphological texture branches. These branches extract features from the RGB grayscale and depth images of the multimodal sample units, resulting in edge gradient feature maps, local morphological change feature maps, and regional structural anomaly feature maps. Specifically, the multi-branch convolutional neural network model in this embodiment has three scale branches set in parallel: small-scale, medium-scale, and large-scale branches. The kernel size of the small-scale branch is set to 3*3, the medium-scale branch to 5*5, and the large-scale branch to 7*7. The receptive field size of each scale branch is set according to the size of the scale branch; that is, the receptive field is smallest for the small-scale branch and largest for the large-scale branch. Simultaneously, through the dilated spatial convolutional pooling pyramid module set in the morphological texture branch, the feature maps (i.e., edge gradient feature maps, local morphological change feature maps, and regional structural anomaly feature maps) obtained from the output of the small-scale, medium-scale, and large-scale branches are compressed and stitched together to obtain the corresponding RGB grayscale image and depth image after morphological texture preprocessing. Then, the corresponding texture geometric feature vectors are extracted from the RGB grayscale image and depth image after morphological texture preprocessing, specifically:

[0068]

[0069] in: For texture geometry feature vectors, Extracting and mapping morphological and textural features. This is an RGB grayscale image after morphological texture preprocessing. This is the depth image after preprocessing the morphological texture.

[0070] Furthermore, by employing a lightweight convolutional network model, a physics-driven feature mapping model, and a frequency domain filtering model, corresponding spectral color branches are set. These branches are used to extract features from the RGB grayscale and polarized light images in the multimodal sample units, thereby extracting the surface roughness features and material microstructure features corresponding to the RGB grayscale and polarized light images. Specifically, the lightweight convolutional network model in this embodiment includes a manual feature operator. This operator is used to convert the RGB grayscale image and obtain the corresponding illumination-invariant operator, determining the corresponding illumination-robust color invariant and material hue features. Simultaneously, the physics-driven feature mapping model extracts the degree of polarization and polarization angle of the polarized light image, determining the corresponding surface reflection characteristics and material uniformity features. It is worth noting that the formulas for obtaining the illumination-invariant operator, degree of polarization, and polarization angle in this embodiment are conventional formulas, and therefore are not specifically explained in this embodiment.

[0071] Furthermore, a two-dimensional Fourier transform is performed on the RGB grayscale image and polarized light image using a frequency domain filtering model to extract the corresponding high-frequency components, thereby determining the corresponding surface roughness features and material microstructure features. In other words, through a lightweight convolutional network model, a physics-driven feature mapping model, and a frequency domain filtering model, the corresponding spectral color preprocessed RGB grayscale image and spectral color preprocessed polarized light image are obtained. Simultaneously, the spectral color preprocessed RGB grayscale image and the spectral color preprocessed polarized light image are concatenated and mapped to determine the corresponding spectral material feature vector, specifically:

[0072]

[0073] in: For spectral material feature vectors, For spectral color feature extraction mapping, This is an RGB grayscale image after spectral color preprocessing. This is a polarized light image after spectral color preprocessing.

[0074] Step S1.2.3: Path Modeling. This involves obtaining the corresponding texture geometry feature vector and spectral material feature vector using the morphological texture branch and spectral color branch set in step S1.2.2, respectively. Simultaneously, through the established causal modeling layer, the obtained texture geometry feature vector and spectral material feature vector are mapped to the corresponding causal representation space. Specifically, based on the observed variables, target variables, and confounding variables set in step S1.2.1, and the causal relationships between these variables, a structural causal model embedding network is used to transform traditional statistical inference into causal inference, specifically:

[0075]

[0076] in: To observe variables, For intervention operators, For the target variable, As a mediating variable, It represents a probability distribution.

[0077] Step S1.3: Intervention Training. This involves extracting the corresponding depth images from the augmented training dataset obtained in Step S1.1.2, and then setting the corresponding mediating variables using these extracted depth images. Specifically:

[0078]

[0079] in: As a mediating variable, For geometric feature extraction functions, It is a depth image.

[0080] Furthermore, based on the observed variables and confounding variables set in step S1.2.1, the correlation coefficient between each feature channel and the confounding variable in the initial mixed detection model is obtained through statistical correlation modeling, specifically:

[0081]

[0082] in: Let be the correlation coefficient between the k-th feature channel and the confounding variable. For the confounding variables corresponding to the pseudo-defect labels of the manufacturing process, This represents the feature representation of the k-th feature channel. This is a correlation measurement function.

[0083] Furthermore, the correlation coefficients obtained from each feature channel are used to construct a corresponding feature sparsity penalty term. Simultaneously, through a set logistic regression classification head, the mediating variable is set as the input to the logistic regression classification head, and the corresponding true defect probability is obtained as the output. The predicted value is then measured based on the true defect probability and the loss output of the initial hybrid detection model to determine the classification loss between the predicted defect value and the true defect value. Specifically, based on the determined classification loss and the constructed feature sparsity penalty term, the corresponding training objective is set as follows:

[0084]

[0085] in: Let be the mathematical expectation of the set of random variables. The classification loss is the difference between the true defect label and the predicted defect probability. As a weighting factor, To mask the loss, To constrain sparsity, Let be the correlation coefficient between the k-th feature channel and the confounding variable. For the index of the feature channels, For the target variable, To predict the probability of defects, This is the loss function.

[0086] In other words, by setting the training objective, the initial hybrid detection model constructed in step S1.2 is trained to obtain the trained hybrid detection model, that is, the corresponding final hybrid detection model is obtained.

[0087] In this embodiment, the corresponding texture features, spectral features, polarization saliency map, and depth deformation features are extracted from the final hybrid detection model obtained in step S1.3, and combined with the corresponding spatial location information, a corresponding fusion enhancement feature map is obtained. (Reference) Figure 4 This embodiment provides an enhanced feature extraction processing method, which specifically includes the following steps:

[0088] Step S2.1: Multimodal Extraction. Based on the enhanced training dataset obtained in Step S1.1.2, the RGB grayscale image, polarized light image, and depth image corresponding to each training sample unit are determined. Simultaneously, the determined RGB grayscale image and depth image are used as inputs to the final hybrid detection model in Step S1.3. Through the morphological texture branch, the corresponding texture semantic feature map, grayscale gradient feature map, and edge morphological feature map are output. Through the spectral color branch, the corresponding material saliency map is output.

[0089] Furthermore, based on the determined depth image and the output texture semantic feature map, the height difference corresponding to the suspected defect region is determined. Then, by using a preset height threshold (which can be set according to actual needs, so it is not specifically described in this embodiment) and the determined height difference, the suspected defect region is classified to remove the suspected defect region with no height change, thereby obtaining the corresponding defect region division result.

[0090] Furthermore, based on the determined RGB grayscale image, polarized light image, and defect region segmentation results, the corresponding spectral discrimination results are determined using the HSV / Lab color modeling model and the Fourier spectrum analysis model. Simultaneously, the acquired texture semantic feature map, grayscale gradient feature map, edge morphology feature map, and material saliency map are feature-aligned, and combined with the acquired defect region segmentation results and spectral discrimination results, a weighted fusion is performed using a spatial attention fusion model to obtain the corresponding initial enhanced feature map.

[0091] It is worth noting that in this embodiment, the defect region segmentation result is obtained by setting a preset height threshold and height difference, and the spectral differentiation result is determined by using RGB grayscale image, polarized light image and defect region segmentation result. The feature alignment processing of feature map are all conventional technical means in this field, so they are not specifically described in this embodiment.

[0092] Step S2.2: Attention Fusion. Based on the texture semantic feature map and material saliency map obtained in Step S2.1, global average pooling is performed on the texture semantic feature map to obtain the corresponding morphological texture pooling features. Global max pooling is performed on the material saliency map to obtain the corresponding spectral color pooling features. Simultaneously, the obtained morphological texture pooling features and spectral color pooling features are concatenated to obtain the corresponding pooling aggregate features.

[0093] Furthermore, through a convolutional mapping network model, the obtained pooled aggregated features are mapped to a single-channel weight map (i.e., a spatial attention map), specifically as follows:

[0094]

[0095] in: This is a spatial attention map. This is a pooling aggregation feature. For convolution operations, This is the activation function.

[0096] Furthermore, based on the obtained spatial attention map, the initial enhanced feature map obtained in step S2.1 is processed to obtain the final enhanced feature map, specifically as follows:

[0097]

[0098] in: To ultimately enhance the feature map, For the initial enhanced feature map, This is a spatial attention map. This is the element-wise multiplication operator.

[0099] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended embodiments and their equivalents.

Claims

1. A deep learning-based intelligent detection method for surface defects of automobile parts, characterized in that, Including: S1: Feature decoupling model construction: A training dataset is constructed using multimodal industrial image data and multi-angle surface images of the parts to be detected, and the initial hybrid detection model is trained using the front-door intervention algorithm; S2: Enhanced processing of feature extraction: The training dataset is used as the input of the final hybrid detection model, and the corresponding multimodal features are obtained from the output. The multimodal features are combined with spatial location information to obtain the corresponding fusion enhanced feature map. S3: Defect reasoning of causal graph: The fused enhanced feature map is used as the input of the Bayesian causal graph model, and the corresponding joint probability distribution is obtained as the output.

2. The intelligent detection method of surface defects of automobile parts based on deep learning according to claim 1, characterized in that, The initial hybrid detection model is trained, including: S1.1: Modal mapping: Collect multimodal industrial image data of the parts to be inspected, construct the corresponding training dataset, and at the same time construct pseudo-defect samples and counterfactual samples through generative adversarial network, and perform enhancement processing on the training dataset to obtain an enhanced training dataset; S1.2: Model Construction: Based on the training dataset, set up the morphological texture branch and the spectral color branch, and combine the morphological texture branch, the spectral color branch and the causal structure learning network to construct the corresponding initial hybrid detection model; S1.3: Intervention Training: Using the depth images in the enhanced training dataset, set corresponding mediating variables, and determine the correlation coefficient between each feature channel and the confusion variable by statistical correlation modeling based on the observed variables and confusion variables corresponding to the initial mixed detection model. At the same time, set corresponding training objectives based on the classification loss between the true defect probability and the defect prediction value and the correlation coefficient, and train the initial mixed detection model according to the training objectives to obtain the corresponding final mixed detection model. 3.The intelligent detection method of surface defects of automobile parts based on deep learning according to claim 2, characterized in that, Augmentation processing of the training dataset includes: S1.1.1: Initial Dataset Construction: Through the set sensor network, multimodal industrial image data of the parts to be inspected is collected, including RGB grayscale images, polarized light images, depth images and multi-angle images. According to the identification ID of the parts to be inspected, the multimodal industrial image data is bound to the data. At the same time, according to the corresponding defect labels, corresponding training sample units are constructed, and the training sample units are combined to construct the corresponding initial training dataset. S1.1.2: Dataset Augmentation: Using a generative adversarial network, construct pseudo-defect samples and counterfactual samples of the parts to be detected, and obtain the corresponding training sample units of the pseudo-defect samples and the training sample units of the counterfactual samples. At the same time, combine the training sample units of the pseudo-defect samples, the training sample units of the counterfactual samples and the initial training dataset to obtain the corresponding augmented training dataset.

4. The intelligent detection method of surface defects of automobile parts based on deep learning according to claim 2, characterized in that, The corresponding initial hybrid detection model is constructed, including: S1.2.1: Variable definition: Based on the constituent elements corresponding to each training sample unit in the training dataset, set the observation variable, target variable, and confounding variable of the initial mixed detection model, and determine the causal relationship between the observation variable, target variable, and confounding variable through industrial field mechanisms; S1.2.2: Feature extraction: Using a multi-branch convolutional neural network model, a lightweight convolutional network model, a physical-driven feature mapping model, and a frequency domain filtering model, corresponding morphological texture branches and spectral color branches are set to extract the corresponding texture geometric feature vectors and spectral material feature vectors. S1.2.3: Path modeling: By setting the causal modeling layer, the texture geometric feature vector and spectral material feature vector are mapped to the corresponding causal representation space.

5. The intelligent detection method of surface defects of automobile parts based on deep learning according to claim 4, characterized in that, The multi-branch convolutional neural network model is configured with small-scale, medium-scale, and large-scale branches in parallel. Through the small-scale, medium-scale, and large-scale branches, the corresponding edge gradient feature maps, local morphological change feature maps, and regional structural anomaly feature maps are output. At the same time, the edge gradient feature maps, local morphological change feature maps, and regional structural anomaly feature maps are compressed and stitched together to obtain a morphological texture preprocessed RGB grayscale image and a morphological texture preprocessed depth image, and the corresponding texture geometric feature vectors are extracted. The lightweight convolutional network model, the physics-driven feature mapping model, and the frequency domain filtering model are used to obtain the corresponding RGB grayscale image and polarized light image after spectral color preprocessing. At the same time, the RGB grayscale image and polarized light image after spectral color preprocessing are spliced ​​and mapped to determine the corresponding spectral material feature vector.

6. The intelligent detection method of surface defects of automobile parts based on deep learning according to claim 2, characterized in that, The correlation coefficients of each feature channel are combined to construct a corresponding feature sparsity penalty term. Simultaneously, a logistic regression classification head is used to output the corresponding true defect probability. The classification loss between the predicted and actual defect probabilities is measured based on the true defect probability and the loss function. The classification loss and the feature sparsity penalty term are then combined to set the corresponding training objective, specifically: wherein: is a mathematical expectation of a set of random variables, is a classification loss between true defect labels and predicted defect probabilities, is a weighting coefficient, is a mask constraint loss, is a target variable, is a predicted defect probability.

7. The intelligent detection method of surface defects of automobile parts based on deep learning according to claim 1, characterized in that, The corresponding fusion-enhanced feature maps are obtained, including: S2.1: Multimodal extraction: Through the final hybrid detection model, the texture semantic feature map, gray-level gradient feature map, edge morphology feature map, and material saliency map are output. Based on the texture semantic feature map and depth image, the corresponding defect region segmentation result is obtained. Based on the RGB gray-level image, polarized light image, and defect region segmentation result, the corresponding spectral discrimination result is determined. At the same time, the texture semantic feature map, gray-level gradient feature map, edge morphology feature map, and material saliency map are feature aligned and weighted and fused to obtain the corresponding initial enhanced feature map. S2.2: Attention Fusion: Based on the obtained texture semantic feature map and material saliency map, the corresponding morphological texture pooling features and spectral color pooling features are obtained, and the morphological texture pooling features and spectral color pooling features are concatenated to obtain the corresponding pooling aggregate features. At the same time, the pooling aggregate features are mapped through a convolutional mapping network model, and the mapped pooling aggregate features are combined with the initial enhanced feature map to obtain the final enhanced feature map. 8.The intelligent detection method of surface defects of automobile parts based on deep learning according to claim 1, characterized in that, The Bayesian causal graph model is connected to a multilayer perceptron classifier, and the multilayer perceptron classifier is configured with two defect classifications, including real defects and pseudo-defects due to manufacturing processes.

9. A deep learning-based intelligent detection system for surface defects of automobile parts, characterized in that, The method for intelligent detection of surface defects of automotive parts based on deep learning, as described in any one of claims 1-8, was used.

Citation Information

Patent Citations

  • Defect detection method, device and equipment for medical consumables and medium

    CN121982007A