Skylight mounting plate welding quality detection method based on convolutional neural network
By using a detection method based on convolutional neural networks, combined with Sobel edge detection and region proposal networks, the efficiency and accuracy issues of welding quality inspection of sunroof mounting plates in welding workshops were solved, achieving high-precision automated inspection.
Patent Information
- Application Number
- CN202511197020.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-26
- Publication Date
- 2025-11-28
AI Technical Summary
In existing technologies, the welding quality inspection of sunroof mounting plates in welding workshops relies on manual visual inspection, which is inefficient and the accuracy of the inspection is difficult to guarantee. AI vision technology lacks a complete automated closed-loop inspection system, and the difficulty in collecting defect datasets leads to insufficient model training.
A detection method based on convolutional neural networks is adopted, which combines the Sobel edge detection operator and the region proposal network to extract local texture and edge features of the welded surface. Through few-shot learning and multi-level metric learning architecture, the geometric features of weld points and rivet nuts are identified. Attention mechanism is used to generate category prototypes for efficient defect identification.
It achieves high-precision welding quality inspection under limited training data conditions, improves inspection efficiency and accuracy, reduces false detection rate, and adapts to the diversity and changes in production environment.
Smart Images
Figure CN121032993A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of automobile detection, in particular to a sunroof mounting plate welding quality detection method based on a convolutional neural network. BACKGROUND
[0002] With the increasing requirements of vehicle manufacturers on production quality and efficiency, in the welding workshop, factors such as machining errors, positioning deviations, conveying errors and splicing errors of body parts are prone to cause welding position errors, and thus cause the increase of defective products.
[0003] The current sunroof mounting plate station in the welding workshop mainly relies on manual visual inspection and manual touch inspection, which is low in efficiency and is significantly affected by the experience and fatigue state of the personnel, and the detection accuracy of subtle welds is difficult to guarantee. In the prior art, AI vision technology based on convolutional neural networks has been gradually applied to feature extraction and model training, but it has not yet formed a complete automated closed-loop detection system suitable for workshop production with actual automobile manufacturing production environment elements. Moreover, the conventional sample learning of AI vision has the defect that the defect data set is difficult to collect, and the collection of defect data depends on the occasional defects in the production process, and the lack of learning samples leads to insufficient model training, which makes it difficult for AI vision to effectively solve the actual quality inspection problems in production. SUMMARY
[0004] The purpose of the present application is to provide a sunroof mounting plate welding quality detection method based on a convolutional neural network to solve the problems raised in the background art.
[0005] To solve the above technical problems, the present application provides the following technical solution: a sunroof mounting plate welding quality detection method based on a convolutional neural network,
[0006] comprising the following steps:
[0007] using an industrial camera to obtain picture data of the detected part and inputting the picture data into a model based on a convolutional neural network;
[0008] The convolutional neural network combines a Sobel edge detection operator to extract local texture and edge features of the welding surface, and calculates the similarity between the features based on cosine similarity, which is used to identify subtle texture changes on the welding surface.
[0009] The region proposal network is used to extract the shape of the weld and the geometric structure features of the rivet nut, which are used to identify size deviations.
[0010] Preferably, the step of extracting the shape of the weld and the geometric structure features of the rivet nut comprises:
[0011] Contour detection: Through the pre-processed X-ray image, the Canny edge detection algorithm is applied to extract the external contour features of the rivet nut, generating a two-dimensional contour representation containing edge points, and for the typical cylindrical structure of the rivet nut, the Hough transform is combined to detect the circular or elliptical contour, and the geometric parameters including the diameter and ellipticity are extracted;
[0012] Key point positioning: The region proposal network based on deep learning is used to locate the core area of the rivet nut in the X-ray image, and the candidate area frame is generated, and the local feature vector in the candidate area is extracted by the convolutional neural network, and the geometric structure details of the nut including the thread pitch and nut height are captured.
[0013] Preferably, it also includes similarity measurement:
[0014] The geometric feature vector of the rivet nut is mapped to a high-dimensional feature space, and the similarity between the detected piece and the standard template is measured by calculating the Euclidean distance between the feature vectors.
[0015] Preferably, the model output based on the convolutional neural network is classified into four categories of normal, shape deformation, size deviation and surface anomaly by a multi-classifier.
[0016] Preferably, during the training of the convolutional neural network-based model, for each defect type, K labeled samples are selected from the X-ray image data set, and the support set samples cover typical defect features and expand sample diversity.
[0017] The pre-trained convolutional neural network is used to extract features from the support set samples to generate high-dimensional feature vectors, and the feature extraction process combines a multi-level metric learning architecture to fuse pixel-level texture features, region-level geometric features and semantic-level defect pattern representations, ensuring the comprehensiveness of the feature vectors.
[0018] Preferably, the output end of the brake master cylinder is connected with a main hydraulic circuit electromagnetic valve, and the main hydraulic circuit electromagnetic valve is connected with a one-way valve between the calipers.
[0019] Preferably, the bypass control module includes a normally closed electromagnetic valve and a relay for controlling the normally closed electromagnetic valve, and the output end of the normally closed electromagnetic valve is connected to the calipers.
[0020] Preferably, the feature vectors of the support set samples are aggregated based on the attention mechanism to generate class prototypes, and the steps are as follows:
[0021] The feature vectors of each sample are scored using an attention network to generate representative weights, and the scoring is based on the cosine similarity between the sample features and the class center, and the higher the weight, the more representative the sample is of the class characteristics;
[0022] According to the representative weight, the feature vectors of the support set samples are weighted and summed to generate a class prototype representation, and the formula is:
[0023] [\mathbf{p}c=\sum{i=1}^K w_i\cdot\mathbf{x}i / \sum{i=1}^Kw_i];
[0024] Wherein, (\mathbf{p}_c) is the prototype of class c, (\mathbf{x}_i) is the feature vector of the ith support set sample, and (w_i) is the corresponding attention weight.
[0025] Preferably, the attention network adopts a two-layer MLP structure, the activation function is ReLU, the output layer uses Sigmoid function to generate normalized weights, and the distance measurement between the prototype and the query sample is optimized by combining the contrast loss during training.
[0026] Preferably, it also includes classifying the prototype according to the defect type:
[0027] For the input picture data of the tested piece, the feature vector is extracted by the same CNN, the Euclidean distance between it and each class prototype is calculated, and the classification is carried out based on the nearest neighbor principle, and the classification formula is: [\hat{y}=\arg\min_c|\mathbf{x}_q-\mathbf{p}_c|_2];
[0028] Wherein, (\mathbf{x}_q) is the feature vector of the query sample, (\mathbf{p}_c) is the prototype of class c, and (\hat{y}) is the predicted defect type.
[0029] A sunroof mounting plate welding quality detection system based on a convolutional neural network, characterized by: for executing the above-mentioned sunroof mounting plate welding quality detection method based on a convolutional neural network, comprising:
[0030] Visual acquisition system: industrial-grade CCD camera and image preprocessing module;
[0031] Algorithm system: algorithm training system and algorithm reasoning system;
[0032] Process running system: integrated visual control cabinet, system process controller, data management module and communication interface;
[0033] Mechanical execution mechanism: man-machine interaction unit, start button and positioning execution unit.
[0034] A storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to perform the above-mentioned method for detecting welding quality of a sunroof mounting plate based on a convolutional neural network.
[0035] Compared with the prior art, the beneficial effects of the present application are:
[0036] The present application adopts a template matching method based on small sample learning. The model first extracts key features in the input data and maps them to a unified feature space, and then performs similarity comparison with the help of a pre-constructed template library to realize fast defect type recognition. Through similarity threshold judgment and pattern matching, the system can effectively identify abnormal patterns and trigger corresponding quality control measures, realizing high-precision welding quality detection under limited training data conditions. BRIEF DESCRIPTION OF DRAWINGS
[0037] Figure 1 It is a schematic diagram of the overall process of the present application;
[0038] Figure 2 It is a schematic diagram of the detection system architecture of the present application;
[0039] Figure 3 It is a schematic diagram of the basic structure of the convolutional neural network applied in the present application;
[0040] Figure 4 It is a schematic diagram of the principle of the present application. DETAILED DESCRIPTION
[0041] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.
[0042] Please refer to Figures 1-4 A method for detecting welding quality of a sunroof mounting plate based on a convolutional neural network is shown in the present application, which uses small sample learning technology for algorithm model training, aiming to solve the problem of dependence on large-scale data sets of traditional deep learning models. Small sample learning can still achieve high classification accuracy under the condition of extremely limited training data. Through the use of existing knowledge and a small amount of samples for rapid learning, the technical difficulty of the lack of defect samples in the industrial scene is effectively solved.
[0043] Refer to Figure 4To improve the generalization ability of the model and reduce the risk of overfitting, the embodiment adopts targeted data augmentation and transfer learning strategies. Specifically, geometric transformation techniques (rotation, translation, scaling) are used to simulate the changes in the position of workpieces in actual production, and optical transformation techniques (brightness, contrast adjustment) are used to adapt to image features under different lighting conditions. At the same time, the transfer learning strategy makes full use of the pre-trained network weights, and through the progressive fine-tuning method, it avoids the occurrence of overfitting problems under small sample conditions.
[0044] Metric learning plays a core role, by learning the similarity metric function between samples, enabling the model to establish an effective distance concept in the feature space. For the quality evaluation needs of welds and rivet nuts, the embodiment designs a multi-level metric learning architecture: at the pixel level, learn the similarity of local texture and edge features, capture the subtle differences of the welding surface; at the regional level, capture the similarity of weld shape and rivet nut geometry, identify defects such as shape deformation; at the semantic level, understand the relevance between different defect types, and establish a high-level semantic representation of defect patterns.
[0045] Through the deep metric learning method based on contrastive loss (Contrastive Loss), the similarity distance metric in the high-dimensional feature space is constructed, and the classification performance of weld and rivet nut quality evaluation is optimized. The invention proposes a multi-level metric learning architecture, which specifically includes the following levels:
[0046] 1. Pixel-level feature learning: through convolutional neural network (CNN, such as ResNet50) combined with Sobel edge detection operator, extract local texture and edge features of the welding surface, calculate the similarity between features based on cosine similarity, and identify subtle texture changes (such as surface roughness or micro-cracks) on the welding surface.
[0047] 2. Regional-level feature learning: extract weld shape and rivet nut geometry features through region proposal network (RPN), measure shape similarity based on Euclidean distance, and detect defects such as shape deformation and size deviation.
[0048] 3. Semantic-level feature learning: model the semantic association between defect types (such as cracks, pores, and incomplete penetration) through fully connected layers and graph neural networks (GNN), establish a high-level defect pattern representation, and enhance the ability to distinguish different defect types.
[0049] This multi-level metric learning architecture realizes progressive learning from low-level local features to high-level semantic representation through layer-by-layer feature aggregation, significantly improves the defect detection accuracy under small sample conditions, and has higher robustness and generalization ability compared to traditional single feature extraction methods.
[0050] In this embodiment, the feature recognition of the pull nut is a core component of region-level metric learning, aiming to accurately extract and analyze its geometric structural features through deep learning methods to achieve efficient defect detection. To meet the quality evaluation needs of pull nuts during the welding process, the invention uses a feature extraction framework based on region proposal network (RPN), combined with geometric shape analysis and similarity measurement technology, to identify defects such as shape deformation, size deviation, and surface abnormalities. The specific methods are as follows:
[0051] 1. Geometric feature extraction:
[0052] Contour detection: Through the pre-processed X-ray image, the Canny edge detection algorithm is applied to extract the external contour features of the pull nut, generating a two-dimensional contour representation containing edge points. For the typical cylindrical structure of the pull nut, the Hough transform is combined to detect circular or elliptical contours, extracting key geometric parameters such as diameter and ellipticity.
[0053] Key point positioning: Use the region proposal network (RPN) based on deep learning to locate the core area of the pull nut in the X-ray image, generating a candidate region frame (Bounding Box). Extract the local feature vector within the candidate region through a convolutional neural network (CNN, such as ResNet50), capturing the geometric structure details of the nut (such as thread pitch and nut height).
[0054] Feature standardization: Normalize the extracted geometric features (such as normalizing to the [0, 1] interval) to ensure consistency of features under different scales or perspectives, adapting to the diversity of workpiece positions and lighting conditions in the production environment.
[0055] 2. Similarity measurement:
[0056] o Euclidean distance measurement: Map the geometric feature vector of the pull nut to a high-dimensional feature space, and measure the similarity between the sample and the standard template by calculating the Euclidean distance between the feature vectors. The standard template is generated from the support set samples (i.e., the feature representation of the defect-free pull nut) and stored in the template library.
[0057] Dynamic threshold adjustment: According to the noise level (such as the blurriness of the X-ray image) in the production environment, dynamically adjust the similarity threshold to distinguish between normal nuts and nuts with shape deformation, size deviation, or surface cracks. The threshold is determined through statistical analysis in the training stage, ensuring that the false detection rate is less than XX%.
[0058] 3. Defect identification:
[0059] Shape deformation detection: By comparing the geometric parameters (e.g. diameter, ovality) of the test sample with the deviation of the standard template, identify the shape deformation defects (e.g. non-circular deformation, stretch distortion) of the rivet nut.
[0060] Size deviation detection: Based on the RPN extracted nut height and thread pitch features, detect size deviation (e.g. nut height out of tolerance range ±0.1mm).
[0061] Surface anomaly detection: Combined with pixel-level texture feature analysis (e.g. edge intensity based on Sobel operator), identify defects such as small cracks or pits on the surface of the rivet nut.
[0062] Classification output: Through multi-classifier (e.g. Softmax layer), classify the defect type into four categories: normal, shape deformation, size deviation and surface anomaly, with a classification accuracy of XX% or more.
[0063] Through the combination of RPN and Hough transform, the geometric feature extraction accuracy is improved to XX%, significantly better than traditional detection methods based on manual features. Normalization processing and dynamic threshold adjustment make the system adapt to different lighting and viewing angle conditions, with a false detection rate of less than XX%. Optimization of template matching and similarity measurement algorithm makes the feature recognition and defect detection time of a single X-ray image controlled within a short time to meet the real-time production requirements.
[0064] The rivet nut feature recognition method of the present application, through regional level feature extraction and analysis in multi-level measurement learning architecture, seamlessly connects with pixel level and semantic level feature learning, and constructs a progressive defect detection system from local details to overall structure, significantly improving the welding quality evaluation performance under small sample conditions.
[0065] The design of the prototype network fully considers the characteristics of welding defects. For each defect type, the system constructs a prototype representation by aggregating the feature vectors of the support set samples. Unlike traditional simple feature averaging methods, this embodiment uses an intelligent aggregation strategy based on attention mechanism. During the prototype construction process, the system automatically identifies the representative contribution of each sample to the category, giving higher weights to more representative samples, thereby constructing a more robust and accurate category prototype.
[0066] In the present application, the design of the prototype network is optimized for the small sample characteristics of welding defect detection, aiming to achieve high-precision defect classification by constructing a robust category prototype. For each welding defect type (e.g. crack, porosity, incomplete penetration, rivet nut shape deformation), the system extracts feature vectors from the support set samples and generates a category prototype representation through an intelligent aggregation strategy based on attention mechanism.
[0067] The following is a specific implementation method:
[0068] 1. Support set construction and pre-processing:
[0069] Support set definition: For each defect type, the system selects K labeled samples (K-shot, K is usually 5 or 10) from the X-ray image dataset as the support set. The support set samples cover typical defect characteristics (such as the texture pattern of cracks, the circular outline of pores, and the geometric deformation of pull nut), and expand sample diversity through data augmentation (including geometric transformations such as rotation, translation, scaling, and optical transformations such as brightness, contrast adjustment).
[0070] Feature extraction: A pre-trained convolutional neural network (CNN, such as ResNet50) is used to extract features from the support set samples, generating high-dimensional feature vectors (such as 2048 dimensions). The feature extraction process combines multi-level metric learning architecture, fusing pixel-level texture features, region-level geometric features, and semantic-level defect pattern representation, ensuring the comprehensiveness of the feature vector.
[0071] Pre-processing optimization: The extracted feature vectors are normalized (such as L2 normalization) to eliminate scale differences, and reduced to a specified dimension (such as 128 dimensions) through principal component analysis (PCA) to reduce computational complexity and noise interference.
[0072] 2. Intelligent aggregation based on attention mechanism:
[0073] Limitations of traditional methods: Traditional prototype networks aggregate support set sample feature vectors through simple mean pooling (such as arithmetic mean) to generate class prototypes, but ignore the representative differences between samples, which may cause the prototype to be biased towards noisy samples or atypical samples.
[0074] Attention mechanism design: The invention proposes an intelligent aggregation strategy based on self-attention mechanism (Self-Attention), which dynamically allocates weights by calculating the contribution of each support set sample to the class prototype. The specific steps are as follows:
[0075] Representative scoring: Use an attention network (containing a multi-layer perceptron, MLP) to score the feature vector of each sample, generating a representative weight (range [0, 1]). The score is based on the cosine similarity between the sample features and the class center (initial mean vector), and the higher the weight, the more representative the sample is of the class characteristics.
[0076] Weighted aggregation: According to the representative weight, the feature vectors of the support set samples are weighted and summed to generate the class prototype representation. The formula is:
[0077] [\mathbf{p}c=\sum{i=1}^K w_i\cdot\mathbf{x}i / \sum{i=1}^Kw_i] where (\mathbf{p}_c) is the prototype of class c, (\mathbf{x}_i) is the feature vector of the i-th support set sample, and (w_i) is the corresponding attention weight.
[0078] Optimization of robustness: By introducing a regularization term (such as L2 regularization) to constrain the distribution of attention weights, prevent the weights from concentrating too much on a single sample, and ensure the robustness of the prototype representation.
[0079] Implementation details: The attention network adopts a two-layer MLP structure (hidden layer dimension such as 256), the activation function is ReLU, and the output layer uses the Sigmoid function to generate normalized weights. During training, combine the contrastive loss (Contrastive Loss) to optimize the distance measure between the prototype and the query sample.
[0080] 3. Prototype application and defect classification:
[0081] Prototype storage: Store the prototype representation of each defect type in the template library as a reference benchmark for subsequent defect classification.
[0082] Classification process: For the input test sample (query set), extract the feature vector through the same CNN, calculate the Euclidean distance between it and each class prototype, and classify based on the nearest neighbor principle. The classification formula is: [\hat{y}=\arg\min_c|\mathbf{x}_q-\mathbf{p}_c|_2] where (\mathbf{x}_q) is the feature vector of the query sample, (\mathbf{p}_c) is the prototype of class c, and (\hat{y}) is the predicted defect type.
[0083] Dynamic update: In the production environment, the system supports online update of the support set, and periodically recalculates the prototype representation.
[0084] Reference Figure 2 As shown, the embodiment also proposes a sunroof mounting plate welding quality detection system based on a convolutional neural network, characterized by: a sunroof mounting plate welding quality detection method based on the convolutional neural network described above, comprising: a visual acquisition system: an industrial-grade CCD camera and an image preprocessing module; an algorithm system: an algorithm training system and an algorithm reasoning system; a process running system: an integrated visual control cabinet, a system process controller, a data management module, and a communication interface; a mechanical execution mechanism: a man-machine interaction unit, a start button, and a positioning execution unit.
[0085] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.
Claims
1. A method for detecting the welding quality of a sunroof mounting plate based on a convolutional neural network, characterized in that: The method comprises the following steps: Obtain the picture data of the detected piece by using an industrial camera, and input the picture data into a model based on a convolutional neural network; The convolutional neural network combines a Sobel edge detection operator to extract local texture and edge features of the welding surface, and calculate the similarity between the features based on cosine similarity, which is used to identify the subtle texture changes of the welding surface; The region proposal network is used to extract the shape and geometric structure features of the rivet nut, which is used to identify the size deviation.
2. The method of claim 1, wherein the method is based on a convolutional neural network. The steps of extracting the shape and geometric structure features of the rivet nut include: Contour detection: Through the preprocessed X-ray image, the Canny edge detection algorithm is applied to extract the external contour features of the rivet nut, and a two-dimensional contour representation containing edge points is generated. For the typical cylindrical structure of the rivet nut, the Hough transform is combined to detect the circular or elliptical contour, and the geometric parameters including the diameter and ellipticity are extracted; Key point positioning: The region proposal network based on deep learning is used to locate the core area of the rivet nut in the X-ray image, and a candidate region frame is generated. The local feature vector in the candidate region is extracted by the convolutional neural network to capture the geometric structure details of the nut including the thread pitch and nut height. 3.The method of claim 1, wherein: Also includes similarity measurement: The geometric feature vector of the rivet nut is mapped to a high-dimensional feature space, and the similarity between the detected piece and the standard template is measured by calculating the Euclidean distance between the feature vectors.
4. The method of claim 3, wherein the method is based on a convolutional neural network. The model based on the convolutional neural network outputs through a multi-classifier to classify the defect types into four categories: normal, shape deformation, size deviation, and surface anomaly.
5. The method of claim 1, wherein: During the training of the model based on the convolutional neural network, for each defect type, K labeled samples are selected from the X-ray image dataset, and the support set samples cover typical defect features to expand the sample diversity; The pre-trained convolutional neural network is used to extract features from the support set samples to generate high-dimensional feature vectors. The feature extraction process combines a multi-level metric learning architecture to fuse pixel-level texture features, region-level geometric features, and semantic-level defect pattern representations to ensure the comprehensiveness of the feature vectors.
6. The method of claim 5, wherein the method is based on a convolutional neural network. The feature vectors of the support set samples are aggregated based on the attention mechanism to generate class prototypes, and the steps are as follows: The attention network is used to score the feature vectors of each sample to generate representative weights. The scoring is based on the cosine similarity between the sample features and the class center, and the higher the weight, the more representative the sample is of the class characteristics; According to the representative weight, the feature vectors of the support set samples are weighted and summed to generate the class prototype representation, and the formula is: [\mathbf{p}c=\sum{i=1}^K w_i\cdot\mathbf{x}i / \sum{i=1}^Kw_i]; Where (\mathbf{p}_c) is the prototype of class c, (\mathbf{x}_i) is the feature vector of the i-th support set sample, and (w_i) is the corresponding attention weight.
7. The method of claim 6, wherein the method is based on a convolutional neural network. The attention network adopts a two-layer MLP structure, the activation function is ReLU, and the output layer uses the Sigmoid function to generate normalized weights. During training, the contrast loss is combined to optimize the distance measurement between the prototype and the query sample.
8. The method of claim 6, wherein the method is based on a convolutional neural network. Also including the classification of prototypes according to defect types: For the input picture data of the tested piece, the feature vector is extracted by the same CNN, the Euclidean distance between it and each category prototype is calculated, and the classification is carried out based on the nearest neighbor principle, and the classification formula is: [\hat{y}=\arg\min_c|\mathbf{x}_q-\mathbf{p}_c|_2]; Wherein, (\mathbf{x}_q) is the feature vector of the query sample, (\mathbf{p}_c) is the prototype of category c, and (\hat{y}) is the predicted defect type. 9.A system for detecting welding quality of a sunroof mounting plate based on a convolutional neural network, characterized in that: A method for performing the welding quality detection of the sunroof mounting plate based on the convolutional neural network according to any one of claims 1-8, comprising: Visual acquisition system: industrial CCD camera and image preprocessing module; Algorithm system: algorithm training system and algorithm reasoning system; Process running system: integrated visual control cabinet, system process controller, data management module and communication interface; Mechanical execution mechanism: man-machine interaction unit, start button and positioning execution unit.
10. A storage medium containing computer-executable instructions, wherein: The computer executable instructions when executed by the computer processor are used to perform the welding quality detection of the sunroof mounting plate based on the convolutional neural network according to any one of claims 1-8.