Automatic identification method of multi-label spot welding electrode cap state based on deep learning
Through the multi-label spot welding electrode cap status automatic identification method based on deep learning, the problem of lack of standardization of electrode cap status detection in the prior art is solved, and the rapid and accurate automatic identification of electrode cap status is achieved, and the production efficiency and reliability are improved.
Patent Information
- Application Number
- CN202210158816.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-21
- Publication Date
- 2025-03-21
- Estimated Expiration
- 2042-02-21
AI Technical Summary
In the prior art, the status detection of spot welding electrode caps lacks a standardized method, resulting in the failure to replace or waste the electrode caps in time, affecting the quality of the solder joints.
The multi-label spot welding electrode cap status automatic recognition method based on deep learning is adopted. By collecting the original appearance of the electrode cap, cropping, scaling and splicing into detection images, the multi-label category of the electrode cap is automatically identified by using the convolutional neural network model.
Automatic recognition of electrode cap status is realized, and multiple labels and categories can be quickly and accurately identified, reducing manual intervention, and improving production efficiency and reliability.
Smart Images
Figure CN114511746B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image detection, and specifically provides a method for automatic recognition of the state of a multi-label spot welding electrode cap based on deep learning. Background Art
[0002] Resistance spot welding is an important welding method. During welding, a columnar electrode is used to form a weld between the contact surfaces of two overlapping workpieces. It is mainly used for thin plate components with a thickness of less than 4mm, stamping parts, and steel bar welding. It has the characteristics of high production efficiency, small deformation, high degree of automation, and rapid and concentrated release of energy. It is particularly suitable for automobile body-in-white production and assembly, aircraft fuselage welding, and other fields. During the spot welding process, the plate needs to be placed between the upper and lower electrode caps of the spot welding machine and clamped and energized, so that the contact surfaces of the two workpieces are partially melted to form a weld nugget. After the power is turned off, continuous pressure is applied so that the molten nugget solidifies under a given pressure to form a weld. Finally, the pressure is removed to complete the spot welding operation. Due to various reasons, when the spot welding machine welds the workpiece, the electrode cap will experience degradation such as wear and oxidation. When the electrode degradation is not large, the impact on the quality of the weld is not obvious, but when the degradation reaches a certain degree, it will seriously affect the quality of the weld (including surface appearance quality and internal weld nugget quality, etc.), and in severe cases, it will cause the workpiece to burn through and the cap to explode. Through on-site image data analysis, the five most common types of electrode caps with different appearance qualities can be identified: electrode caps that are too short and close to the warning line, electrode caps with end face oxidation due to high temperature oxidation, electrode caps with enlarged end faces due to high temperature and high pressure, electrode caps with tilted and deformed shape, and normal electrode caps. At present, in the actual production and manufacturing process, the state of electrode caps is generally detected by manual visual observation or regular inspection and replacement based on experience. There is no set of standardized methods for detecting the state of electrode caps, which can easily lead to electrode caps that should be replaced not being replaced in time, while some electrode caps in good condition are replaced because the number of spot welding has been reached, resulting in waste. The state of the electrode cap is related to the quality of the weld. If the state of the electrode cap can be identified in real time, and it can be repaired, adjusted, and replaced in time, it can ensure the smooth progress of the spot welding operation and reduce the probability of false welding and cap explosion. Summary of the invention
[0003] In view of this, the object of the present invention is to provide a multi-label spot welding electrode cap state automatic recognition method based on deep learning, which can automatically identify the multi-label categories of the appearance state of the spot welding electrode cap.
[0004] In order to achieve the above object, the present invention provides the following technical solutions:
[0005] A multi-label spot welding electrode cap state automatic recognition method based on deep learning includes the following steps:
[0006] 1) Collecting the original appearance images of the spot welding electrode cap: The original appearance images collected for the same spot welding electrode cap include the top view of the end face, the side view and the oblique views in two directions;
[0007] 2) Obtaining a detection image of the spot welding electrode cap: For the same spot welding electrode cap, the end face top view, side view and oblique views in two directions are respectively cropped and scaled to set sizes, and then spliced into one image to obtain a detection image of the spot welding electrode cap;
[0008] 3) Set the number of batch detection images: according to the total number of acquired detection images, set the number of detection images in each batch;
[0009] 4) Using a convolutional neural network model to automatically identify the appearance multi-label categories of the spot welding electrode cap, a detection image and a multi-label category matrix are obtained, wherein the appearance multi-label categories of the spot welding electrode cap include five categories: an electrode cap that is too short, an oxidized electrode cap, a tilted and deformed electrode cap, a normal electrode cap, and an end face enlarged electrode cap.
[0010] Further, in the step 4), the convolutional neural network model automatically identifies the multi-label category of the appearance of the spot welding electrode cap by extracting typical features of the image, wherein the typical image feature of the electrode cap that is too short is that the electrode cap is too short sideways, the typical image feature of the oxidized electrode cap is a large black oxide layer on the head of the electrode cap, the typical image feature of the tilted and deformed electrode cap is the obvious end face tilt deformation in the side view, and the typical image feature of the end face enlarged electrode cap is that the end face diameter is greater than the set threshold; the electrode cap excluding the above typical image features is a normal electrode cap.
[0011] Furthermore, the convolutional neural network model includes a data input layer, an Ind convolution module, an inception convolution unit, a global average pooling layer (AGP layer), a fully connected layer (FC layer) and an output layer, and the number of the inception convolution units is set to 5.
[0012] Furthermore, the Ind convolution module uses a combination of a 3*3 convolution kernel and a 1*1 convolution kernel to form a multi-scale convolution module, and connects different multi-scale convolution modules in parallel to more fully extract shallow features of the image.
[0013] Furthermore, the inception convolution unit includes 5 parallel convolutions and uses a 1*1 convolution kernel for dimensionality reduction; one of the parallel convolutions uses two 3*3 convolution kernels in series to reduce network parameters and widen the network width; the other parallel convolution uses three 3*3 convolution kernels in series to widen the convolution width; a maximum pooling process is performed after each inception convolution calculation is completed; the output layer uses a sigmoid classifier to classify each spot welding electrode cap into a label category or a combination of several label categories.
[0014] Furthermore, in the Ind convolution module and the inception convolution unit, each convolution layer uses the Rule function as a nonlinear activation function.
[0015] Furthermore, the training method of the convolutional neural network model comprises the following steps:
[0016] (1) Using the original appearance image of the spot welding electrode cap, and obtaining the spot welding electrode cap image training data after cropping, scaling and splicing the original appearance image of the spot welding electrode cap, and dividing the spot welding electrode cap image training data into a training set, a validation set and a test set;
[0017] (2) Use the Adam optimizer to train the convolutional neural network model; set early stopping to automatically stop training when the validation set loss no longer decreases, and finally save the model after testing it with the test set.
[0018] The beneficial effects of the present invention are:
[0019] The present invention provides a multi-label spot welding electrode cap state automatic recognition method based on deep learning. After acquiring the original appearance image of the spot welding electrode cap, the original image is cropped, scaled and spliced into a detection image of the spot welding electrode cap. The number of detection images in each batch is set according to the number of detection images of the spot welding electrode cap obtained. Finally, a convolutional neural network model is used to automatically identify the multi-label category of the appearance of the spot welding electrode cap, and the detection image and the multi-label category matrix are obtained, thereby obtaining the multi-label category corresponding to the spot welding electrode cap corresponding to the detection image. The method can replace manual eye recognition and save human resources. It has the characteristics of fast recognition speed, high accuracy, and the ability to recognize multiple labels and categories, which helps to improve manufacturing production efficiency and reliability and solves the demand for intelligent appearance detection of spot welding electrode caps in actual production processes. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following drawings for illustration:
[0021] Figure 1 It is a flow chart of the method for automatic identification of multi-label spot welding electrode cap status based on deep learning of the present invention;
[0022] Figure 2 The actual picture of the electrode cap that is too short; (a) is the original appearance image, and (b) is the detection image;
[0023] Figure 3 The actual picture of the oxidation electrode cap; (a) is the original appearance image, (b) is the detection image;
[0024] Figure 4 The actual picture of the tilted and deformed electrode cap; (a) is the original appearance image, and (b) is the detection image;
[0025] Figure 5The actual picture of a normal electrode cap; (a) is the original appearance image, and (b) is the detection image;
[0026] Figure 6 The actual picture of the end-face enlarged electrode cap; (a) is the original appearance image, (b) is the detection image;
[0027] Figure 7 It is a structural diagram of the convolutional neural network model;
[0028] Figure 8 It is a schematic diagram of the structure of the Ind convolution module;
[0029] Fig. 9 This is a schematic diagram of the structure of the inception convolution unit;
[0030] Fig.10 The multi-label category recognition results of the detection image; among them, (a) the recognition labels are too short and oxidized, and (b) the recognition labels are oxidized and tilted;
[0031] Fig.11 Training flowchart for the convolutional neural network model. DETAILED DESCRIPTION
[0032] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.
[0033] like Figure 1 FIG. 1 is a flowchart of a multi-label spot welding electrode cap state automatic identification method based on deep learning of the present invention. The multi-label spot welding electrode cap state automatic identification method based on deep learning of this embodiment includes the following steps:
[0034] 1) Collect the original appearance image of the spot welding electrode cap: The original appearance image collected for the same spot welding electrode cap includes the top view of the end face, the side view and the oblique view in two directions; the specific image data acquisition process depends on the on-site industrial camera, and multiple color industrial cameras are fixed to take images at predetermined angles, and the taken images are transmitted to a unified device, and the trained convolutional neural network model is also placed on this device. In this embodiment, the original appearance image is the image of the spot welding electrode cap taken at four different angles, and it is an RGB three-channel image.
[0035] 2) Obtaining the detection image of the spot welding electrode cap: For the same spot welding electrode cap, the collected end face top view, side view and oblique views in two directions are cropped and scaled to the set size, and then spliced into one image to obtain the detection image of the spot welding electrode cap. The size of the spliced detection image is adjusted to 230×230 to adapt to the neural network input.
[0036] 3) Set the number of batch detection images: Set the number of detection images in each batch according to the total number of detection images obtained.
[0037] 4) Use the convolutional neural network model to automatically identify the multi-label categories of the appearance of the spot welding electrode cap, and obtain the detection image and the multi-label category matrix. Specifically, the multi-label categories of the appearance of the spot welding electrode cap include five categories: too short electrode cap, oxidized electrode cap, tilted and deformed electrode cap, normal electrode cap and end face enlarged electrode cap. In this embodiment, the convolutional neural network model automatically identifies the multi-label categories of the appearance of the spot welding electrode cap by extracting typical image features, among which the typical image feature of the too short electrode cap is that the electrode cap is too short sideways, the typical image feature of the oxidized electrode cap is the large black oxide layer on the head of the electrode cap, the typical image feature of the tilted and deformed electrode cap is the obvious end face tilt deformation in the side view, and the typical image feature of the end face enlarged electrode cap is that the end face diameter is greater than the set threshold; the electrode caps excluding the above typical image features are normal electrode caps. Figure 2 As shown in the figure, it is a real picture of the electrode cap that is too short; among them, (a) is the original appearance image, and (b) is the detection image. Figure 3 As shown in the figure, it is a real picture of the oxidation electrode cap; (a) is the original appearance image, and (b) is the detection image. Figure 4 As shown in the figure, it is a real picture of the tilted and deformed electrode cap; (a) is the original appearance image, and (b) is the detection image. Figure 5 As shown in the figure, it is a real picture of a normal electrode cap; (a) is the original appearance image, and (b) is the detection image. Figure 6 As shown, it is a real picture of the end face enlarged electrode cap; among them, (a) is the original appearance image, and (b) is the detection image.
[0038] Further, if Figure 7 As shown, the convolutional neural network model of this embodiment includes a data input layer, an Ind convolution module, an inception convolution unit, a global average pooling layer (AGP layer), a fully connected layer (FC layer) and an output layer, and the number of the inception convolution units is set to 5. Figure 8 As shown in the figure, it is a schematic diagram of the structure of the Ind convolution module. The Ind convolution module of this embodiment forms a multi-scale convolution module by combining a 3*3 convolution kernel and a 1*1 convolution kernel. Its structure is different from that of the traditional convolution network. It avoids blindly increasing the network depth but broadens the network width and reduces parameters. At the same time, by connecting different multi-scale convolution modules in parallel, it ensures that the convolution layer can more fully extract the shallow features of the image. Fig. 9As shown in the figure, it is the structure diagram of the inception convolution unit. Each of the inception convolution units includes 5 parallel convolutions and uses a 1*1 convolution kernel for dimensionality reduction. The inception convolution unit refers to the GoogleNet network and has made some improvements. Specifically, one of the parallel convolutions replaces the 5*5 convolution kernel with two 3*3 convolution kernels in series to reduce network parameters and widen the network width; the other parallel convolution uses a set of convolutions consisting of 3 3*3 convolution kernels in series to widen the convolution width, realize feature extraction of image data under different convolution kernels, and deeply extract image features. After each inception convolution calculation is completed, a maximum pooling process is performed; the output layer uses a sigmoid classifier to classify each spot welding electrode cap into a label category or a combination of several label categories.
[0039] In the Ind convolution module and the inception convolution unit of this embodiment, each convolution layer uses the Rule function as a nonlinear activation function. In addition, this embodiment uses the cross entropy function as a loss function to construct a convolutional neural network model.
[0040] like Figure 8 As shown in the figure, it is a schematic diagram of the structure of the Ind convolution module, which forms a multi-scale convolution module by combining a 3*3 convolution kernel and a 1*1 convolution kernel to reduce parameters. Fig. 9 As shown in the figure, it is a schematic diagram of the structure of the inception convolution unit. Each time an inception convolution is completed, a maximum pooling (MaxPool) process is performed.
[0041] Specifically, the training method of the convolutional neural network model includes the following steps:
[0042] (1) Using the original image of the appearance of the spot welding electrode cap, and obtaining the spot welding electrode cap image training data after cropping, scaling and splicing the original image of the appearance of the spot welding electrode cap, the spot welding electrode cap image training data is divided into a training set, a validation set and a test set; specifically, this embodiment divides the spot welding electrode cap image training data into a training set, a validation set and a test set in a ratio of 6:2:2.
[0043] (2) Use the Adam optimizer to train the convolutional neural network model; set early stopping to automatically stop training when the validation set loss no longer decreases, and finally save the model after testing it with the test set.
[0044] Specifically, after the convolutional neural network model is trained, the performance of the network model, including recognition speed, accuracy, etc., can be tested using the detection image of a specific electrode cap. The model can be saved after the expected effect is achieved. When in use, the convolutional neural network model can be copied to the manufacturing site equipment to automatically identify the electrode cap status. Fig.10(a) shows the appearance of a short and oxidized electrode cap. Fig.10 (b) is an image of an oxidized, tilted, and deformed electrode cap. This means that the automatic recognition and classification of multi-label categories with five labels is realized. Therefore, the output result contains a label matrix corresponding to the recognized image, with 0 and 1 representing the test image without (0) or with (1) the label category. The output table after recognition is as follows: Figure 1 shown.
[0045] This embodiment is based on a deep learning-based multi-label spot welding electrode cap state automatic identification method. After obtaining the original appearance image of the spot welding electrode cap, it is cropped, scaled and spliced into a detection image of the spot welding electrode cap. The number of detection images in each batch is set according to the number of detection images of the spot welding electrode cap obtained. Finally, a convolutional neural network model is used to automatically identify the multi-label category of the appearance of the spot welding electrode cap, and a detection image and a multi-label category matrix are obtained, thereby obtaining the multi-label category corresponding to the spot welding electrode cap corresponding to the detection image. This method can replace manual eye recognition and save human resources. It also has the characteristics of fast recognition speed, high accuracy, and the ability to identify multiple labels and categories. It helps to improve manufacturing production efficiency and reliability, and solves the need for intelligent appearance detection of spot welding electrode caps in actual production processes.
[0046] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.
Claims
1. A multi-label spot welding electrode cap state automatic recognition method based on deep learning, characterized by: The steps include: 1) Collecting the original appearance images of the spot welding electrode cap: The original appearance images collected for the same spot welding electrode cap include the top view, side view and oblique views in two directions; 2) Obtaining the detection image of the spot welding electrode cap: For the same spot welding electrode cap, the end face top view, side view and oblique views in two directions are cropped and scaled to set sizes, and then spliced into one image to obtain the detection image of the spot welding electrode cap; 3) Set the number of test images in a batch: according to the total number of test images acquired, set the number of test images in each batch; 4) Using a convolutional neural network model to automatically identify the appearance multi-label categories of spot welding electrode caps, and obtaining a detection image and a multi-label category matrix, the appearance multi-label categories of the spot welding electrode caps include five categories: too short electrode caps, oxidized electrode caps, tilted and deformed electrode caps, normal electrode caps, and end face enlarged electrode caps; The convolutional neural network model includes a data input layer, an Ind convolution module, an inception convolution unit, a global average pooling layer (AGP layer), a fully connected layer (FC layer) and an output layer, and the number of the inception convolution units is set to 5; The Ind convolution module uses a combination of a 3*3 convolution kernel and a 1*1 convolution kernel to form a multi-scale convolution module, and connects different multi-scale convolution modules in parallel to more fully extract shallow features of the image. The inception convolution unit includes 5 parallel convolutions and uses a 1*1 convolution kernel for dimensionality reduction; one of the parallel convolutions uses two 3*3 convolution kernels in series to reduce network parameters and widen the network width; the other parallel convolution uses three 3*3 convolution kernels in series to widen the convolution width; a maximum pooling process is performed after each inception convolution calculation is completed; the output layer uses a sigmoid classifier to classify each spot welding electrode cap into a label category or a combination of several label categories.
2. According to the method for automatic identification of multi-label spot welding electrode cap status based on deep learning in claim 1, it is characterized by: In the step 4), the convolutional neural network model automatically identifies the appearance multi-label category of the spot welding electrode cap by extracting typical features of the image, wherein the typical image feature of the electrode cap that is too short is that the electrode cap is too short in side view, the typical image feature of the oxidized electrode cap is a large black oxide layer on the head of the electrode cap, the typical image feature of the tilted and deformed electrode cap is the obvious end face tilt deformation in the side view, and the typical image feature of the end face enlarged electrode cap is that the end face diameter is greater than the set threshold; the electrode cap excluding the above typical image features is a normal electrode cap.
3. According to claim 1, the multi-label spot welding electrode cap state automatic identification method based on deep learning is characterized by: In the Ind convolution module and the inception convolution unit, each convolution layer uses the Rule function as a nonlinear activation function.
4. The method for automatic identification of multi-label spot welding electrode cap status based on deep learning according to any one of claims 1 to 3, characterized in that: The training method of the convolutional neural network model comprises the following steps: (1) Using the original image of the appearance of the spot welding electrode cap, and obtaining the spot welding electrode cap image training data after cropping, scaling and splicing the original image of the appearance of the spot welding electrode cap, and dividing the spot welding electrode cap image training data into a training set, a validation set and a test set; (2) Use the Adam optimizer to train the convolutional neural network model; Set early stopping to automatically stop training when the validation set loss no longer decreases, and finally save the model after testing it with the test set.