A welding part surface two-dimensional code spraying code recovery method based on deep learning

Abstract

The application provides a kind of based on deep learning's welding piece surface two-dimensional code ink jet code recovery method.Belongs to industrial welding image processing technical field.Method includes following steps: step 1 data preparation: collect a batch of welding piece surface reflection two-dimensional code image data and corresponding clear two-dimensional code image data;Step 2 data preprocessing: through multi-scale data enhancement, expand the data amount and diversity of original data set obtained in step 1;Step 3 divides data set: the data set after step 2 preprocessing is divided into training set, validation set and test set;Step 4 training set once annotation;Step 5 establishes deep learning positioning model;Step 6 training set secondary annotation;Step 7 establishes deep learning recovery model;Step 8 verification and accuracy evaluation.The application is analyzed and handled to reflection two-dimensional code image by using deep learning algorithm, to realize the correct decoding of two-dimensional code by code scanner.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial welding image processing, and particularly relates to a welding part surface two-dimensional code spraying code recovery method based on deep learning. BACKGROUND

[0002] A two-dimensional code is a black-and-white pattern arranged according to certain distribution rules using a specific geometric figure, and is used to record and transmit data information. Compared with traditional one-dimensional bar codes, two-dimensional codes have many advantages, including large information capacity, high density, strong error correction capability, good security, and wide coding range. At the same time, two-dimensional codes also introduce check error correction codes, which have the functions of detecting errors and recovering deleted errors.

[0003] Two-dimensional codes have been widely used in many fields. In the business field, it is used for code scanning shopping, promotion activities, payment methods, etc. Two-dimensional codes can provide a convenient shopping experience and real-time transaction information. In the medical field, two-dimensional codes can be used for medical record management and drug traceability. By associating two-dimensional codes with patients' medical information and drug packaging, the medical information of patients and the production process of drugs can be tracked and managed throughout the process. In the industrial field, two-dimensional codes are applied to logistics management, production process tracking, equipment maintenance, etc. By associating two-dimensional code identifiers with goods or equipment, automated data collection and tracking can be achieved, improving production efficiency and management level.

[0004] In the industrial welding process, the identification of the welding part is very important for quality tracking and tracing. At present, one of the commonly used identification methods is to spray or laser mark a two-dimensional code on the surface of the welding part, which contains information such as production number, date, product name, welding process, welding personnel and equipment. With the help of a handheld bar code reader, the two-dimensional code information is read and entered into the background database. Based on the above information, the background database combines the parameter information in the welding production process to monitor and manage the welding production process and production traceability, and to determine the welding method and equipment process according to the data information.

[0005] However, in actual production, due to environmental light, too strong incident light caused by the auxiliary light source of the code scanner, uneven illumination, etc., the two-dimensional code may form a light spot or halo in the local area, resulting in low recognition of the two-dimensional code and incorrect decoding, which brings difficulties to subsequent quality tracking and tracing. In order to solve this problem, some methods based on image processing have been proposed to recover the light reflection two-dimensional code. Therefore, traditional methods usually need to rely on manual feature extraction and rule design, which cannot adapt to complex and variable actual scenes, and have high requirements for image quality. SUMMARY

[0006] To solve the above problems of the prior art, the present application provides a welding part surface two-dimensional code spraying code recovery method based on deep learning. The method analyzes and processes the reflective two-dimensional code image by using a deep learning algorithm to realize correct decoding of the two-dimensional code by the code scanner.

[0007] The specific technical solution is:

[0008] The welding part surface two-dimensional code spraying code recovery method based on deep learning comprises the following steps:

[0009] Step 1: data preparation

[0010] Collect a batch of two-dimensional code image data of the welding part surface reflection and corresponding clear two-dimensional code image data;

[0011] Step 2: data preprocessing

[0012] Through multi-scale data enhancement, the data amount and diversity of the original data set obtained in step 1 are expanded, which can adapt to different industrial environments and scenes;

[0013] Step 3: dividing the data set

[0014] The data set after step 2 preprocessing is divided into a training set, a validation set and a test set;

[0015] Step 4: one-time annotation of the training set

[0016] The training set is annotated to mark the positions of the light spots and halos;

[0017] Step 5: establishing a deep learning positioning model

[0018] The positioning model is used to locate the light spot and halo area; a target detection model YOLOv8 is selected, the training set with the light spot and halo positions annotated in step 4 is used to train the selected model to learn the features and position information of the light spot and halo, and after the training is completed, the best weight file of the trained YOLOv8 is saved to the local, and the weight model is used as a subsequent pre-training model;

[0019] Step 6: secondary annotation of the training set

[0020] The light spot and halo region positioned in step 5 are labeled again, and the two-dimensional code data information is labeled; first, the light spot and halo region are extracted and segmented into small blocks in point units, each small block is named in the form of "crop_x_y", wherein x represents the row number in the two-dimensional code, and y represents the column number in the two-dimensional code, and at the same time, the color of the small block is obtained according to the segmented small block in the same region of the original clear two-dimensional code, so that the small blocks are respectively stored in the "1" and "0" two folders, wherein the small blocks stored in "1" are white, and the small blocks stored in "0" are black, and the two-dimensional code illumination intensity corresponding to the small blocks stored in the two folders should be similar;

[0021] Step 7 establishes a deep learning recovery model:

[0022] A deep learning recovery model is established for the image data in the "1" and "0" two folders in step 6, and the recovery model is used to recover the two-dimensional code data; a MobileNet light model is selected, the recovery model is trained, and only the black and white small blocks containing the light spot and halo region are provided, and the training result is returned to the original image after the training is completed;

[0023] Step 8 verifies and evaluates the accuracy.

[0024] Further, in the method, the image in step 1 is collected from the field or obtained from an existing image source, the data set has sufficient samples, and contains images under different illumination intensities.

[0025] Further, in the method, step 2 specifically includes:

[0026] Step 2.1 image geometric transformation:

[0027] The image is scaled, cropped and rotated;

[0028] Step 2.2 image binarization;

[0029] Step 2.3 contrast enhancement:

[0030] The binarized image after step 2.2 is adjusted using histogram equalization and adaptive histogram equalization method to adjust the contrast of the image;

[0031] Step 2.4 sharpening

[0032] The enhanced image after step 2.3 is sharpened using a sharpening mask method to enhance the edges and details of the image;

[0033] Step 2.5 denoising

[0034] The median filter method is used to remove noise, and the pixel values in the window around the current pixel are sorted, and then the middle value is taken as the value of the current pixel.

[0035] Further, step 2.2 in the method specifically includes:

[0036] Step 2.2.1, using the cvtColor function in the OpenCV library to convert the color image after step 2.1 into a grayscale image;

[0037] Step 2.2.2, select an appropriate threshold to segment the grayscale image into target objects and background;

[0038] Step 2.2.3, using the threshold function in opencv to convert the grayscale image into a binary image;

[0039] Step 2.2.4, save the resulting binary image to the designated target location for subsequent processing and analysis.

[0040] Further, step 3 in the method is specifically:

[0041] 70% of the data as the training set, 15% as the validation set, and 15% as the test set.

[0042] Further, step 8 in the method specifically includes:

[0043] Step 8.1 model evaluation indicators:

[0044] Select accuracy, recall, precision, and F1 score evaluation indicators to measure the accuracy and performance of the model;

[0045] Step 8.2 model performance evaluation:

[0046] Use the validation set to evaluate the model and calculate the indicator values of the model on the validation set;

[0047] Step 8.3 error analysis:

[0048] Error analysis on the evaluation results;

[0049] Step 8.4 comparison experiment:

[0050] Comparison experiment with other positioning models to evaluate the relative performance of the model;

[0051] Step 8.5 hyperparameter tuning:

[0052] According to the evaluation results and error analysis, adjust the hyperparameters of the model, and use grid search and random search methods to determine the best hyperparameter combination;

[0053] Step 8.6 repeated verification:

[0054] Multiple verifications and evaluations are performed using different validation sets and randomly initialized models.

[0055] Further, the two-dimensional code in the method is generated by the libqrencode open source two-dimensional code generation library and adopts a dot matrix pattern form.

[0056] The beneficial effects of the above technical solutions of the present application are as follows:

[0057] The present application adopts double model recovery of reflective two-dimensional code, model 1 selects target detection model YOLOv8 for positioning the spot and halo area of the two-dimensional code on the surface of the welding part, and model 2 selects a lightweight convolutional neural network model MobileNet for recovering the data information of the spot and halo area of the two-dimensional code. For the two-dimensional code under a certain reflective intensity, when decoding by the scanner embedded with the deep learning algorithm in the present application, it is not necessary to adjust the light source, scanning angle, scanning distance and other parameters. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 The two-dimensional code recovery flowchart of the present application. DETAILED DESCRIPTION

[0059] In order to make the technical problems, technical solutions and advantages to be solved by the present application more clear, the following will be described in detail in combination with specific embodiments.

[0060] As shown in Figure 1 In the present application, the reflective two-dimensional code image is analyzed and processed by using the deep learning algorithm to realize the correct decoding of the two-dimensional code by the code scanner.

[0061] The two-dimensional code is generated by the libqrencode open source two-dimensional code generation library and adopts a dot matrix pattern form. This pattern can facilitate subsequent analysis and processing to extract the information contained in the two-dimensional code.

[0062] The method comprises:

[0063] Step 1 data preparation

[0064] Firstly, a batch of two-dimensional code image data of the surface reflection of the welding part and the corresponding clear two-dimensional code image data need to be collected. These images can be collected from the field or obtained from other existing images and the like. It is ensured that the data set has enough samples and contains images under different light intensities. The quality and clarity of the images are very important for subsequent analysis and processing. These images will be used as training data for training of the deep learning model;

[0065] Step 2 data preprocessing

[0066] Through multi-scale data enhancement, the data amount and diversity of the original data set obtained in step 1 are expanded and increased, which can adapt to different industrial environments and scenes, including:

[0067] Step 2.1 Image geometric transformation

[0068] Scale, crop, rotate, etc. the image. Ensure that the dataset has consistent dimensions and uniform format;

[0069] Step 2.2 Image binarization

[0070] Step 2.2.1, use the cvtColor function in the OpenCV library to convert the color image after step 2.1 to a grayscale image. This removes color information and highlights brightness and contrast in the image;

[0071] Step 2.2.2, select an appropriate threshold to segment the grayscale image into target objects and background. You can manually select the threshold, or use adaptive thresholding and other methods to select it;

[0072] Step 2.2.3, use the threshold function in opencv to convert the grayscale image to a binary image. If manual threshold selection is not accurate enough or the image has uneven lighting conditions, consider using the adaptiveThreshold function provided by OpenCV to implement adaptive thresholding. Adaptive thresholding divides the image into many small regions and selects the threshold according to the local characteristics of each region;

[0073] Step 2.2.4, save the resulting binary image to the specified target location for subsequent processing and analysis;

[0074] Step 2.3 Contrast enhancement

[0075] Use histogram equalization, adaptive histogram equalization, etc. on the binary image after step 2.2 to adjust the contrast of the image, making the difference between dark and light parts more obvious, achieving the purpose of enhancing the image. When the global contrast of the image is low and the brightness distribution is uneven, use histogram equalization to transform the pixel value distribution of the entire image, map the pixel value to the entire gray level range, stretch the pixel value range of the image, and make the difference between dark and light parts more obvious, enhancing the visual effect of the image. When there are large local contrast differences in the image, such as bright-dark boundaries or uneven lighting, use adaptive histogram equalization to divide the image into multiple small regions and perform histogram equalization on the pixels in each small region. The equalization operation of each small region is based on the pixel value distribution in that region, so it can better preserve image details and avoid over-enhancement;

[0076] Step 2.4 Sharpening

[0077] Unsharp Masking is a commonly used image enhancement technique that enhances the edge features of an image by applying a high-pass filter, making the image appear clearer. For a reflective two-dimensional code picture, this method can help highlight the edges and details of the two-dimensional code, improving the clarity and recognition rate of the two-dimensional code. Compared with the Laplacian sharpening, Unsharp Masking can effectively suppress the generation of noise while preserving image details, and is not prone to introduce excessive artifacts. Therefore, the Unsharp Masking method often achieves good results when processing reflective two-dimensional code pictures;

[0078] Step 2.5 denoising

[0079] For a reflective two-dimensional code picture, median filtering can be used to remove noise and make the image clearer. Median filtering is a commonly used nonlinear filtering method that sorts the pixel values within a window around the current pixel and then uses the median value as the value of the current pixel, thereby reducing noise. Compared with mean filtering and Gaussian filtering, median filtering preserves the edges and details of the image while removing noise, especially suitable for removing random noise such as salt and pepper noise. For a reflective two-dimensional code picture, due to the presence of noise caused by light or other interference, using median filtering can effectively reduce the impact of these noises on the two-dimensional code, thereby improving the recognition rate and readability of the two-dimensional code;

[0080] Step 3: Divide the dataset

[0081] Divide the dataset after step 2 preprocessing into training set, validation set and test set. Use 70% of the data as the training set, 15% as the validation set, and 15% as the test set;

[0082] Step 4: One-time annotation of training set

[0083] Annotate the training set to mark the positions of the light spots and halos;

[0084] Step 5: Establish deep learning model 1 (positioning model)

[0085] Model 1 is used to locate the light spot and halo area. Select the target detection model YOLOv8, use the training set with annotated light spot and halo positions in step 4 to train the selected model to learn the features and position information of the light spot and halo. After training, save the best weight file of YOLOv8 trained to the local, and use the weight model as the pre-trained model for subsequent training;

[0086] Step 6: Secondary annotation of training set

[0087] The light spot and halo region positioned in step 5 are labeled again, and the two-dimensional code data information is labeled. First, the light spot and halo region are extracted and divided into small blocks in point units, and each small block is named in the form of "crop_x_y", where x represents the row number in the two-dimensional code, and y represents the column number in the two-dimensional code, such as "crop_4_12" representing a black and white small block located in the fourth row and the twelfth column of the two-dimensional code. At the same time, according to the segmented small blocks, the same region of the original clear two-dimensional code is positioned to obtain the small block color, so that the small blocks are respectively stored in the "1" and "0" two folders, wherein the small blocks stored in "1" are white, and the small blocks stored in "0" are black. It should be noted that the two folders store small blocks corresponding to similar two-dimensional code illumination intensity;

[0088] Step 7: Establishing a deep learning model 2 (restoration model)

[0089] A deep learning model 2 is established for the image data in the "1" and "0" two folders in step 6, and the model 2 is used to restore two-dimensional code data. A MobileNet lightweight model is selected. MobileNet uses depthwise separable convolution to replace the conventional convolution, which can greatly reduce the parameter quantity and computational complexity of the model, and is easily integrated into various mobile and embedded devices, thereby realizing efficient inference on resource-constrained devices. In the case of low parameter quantity and computational complexity, it can still achieve precision performance comparable to traditional models such as VGG and ResNet. By using multiple layers of depthwise separable convolution layers, MobileNet can extract more complex feature information, thereby achieving better classification or detection effect. When training the model, only black and white small blocks containing light spots and halo regions are provided, and the training results are returned to the original image after the training is completed;

[0090] Step 8: Verification and accuracy evaluation

[0091] Step 8.1: Model evaluation index

[0092] Accuracy, recall rate, precision and F1 score are selected as evaluation indexes to measure the accuracy and performance of the model;

[0093] Step 8.2: Model performance evaluation

[0094] The model is evaluated using the validation set, and the index values of the model on the validation set are calculated. According to the evaluation results, the model can be further optimized or the parameters of the model can be adjusted;

[0095] Step 8.3: Error analysis

[0096] Perform error analysis on the evaluation results to identify the conditions under which the model is prone to errors and take corresponding improvement measures. For example, if the model performs poorly in localization under a certain halo shape, consider increasing the number of training samples with that shape.

[0097] Step 8.4 Comparative Experiment

[0098] Comparative experiments can be conducted with other positioning models to evaluate their relative performance. This helps identify the model's strengths and weaknesses and provides a reference for further improvements.

[0099] Step 8.5 Hyperparameter Tuning

[0100] Based on the evaluation results and error analysis, the model's hyperparameters can be adjusted to further improve model performance. Methods such as grid search and random search can be used to determine the optimal hyperparameter combination.

[0101] Step 8.6 Repeat verification

[0102] To confirm the stability and reliability of the model, multiple validation and evaluation tests can be performed using different validation sets and randomly initialized models. This yields more reliable evaluation results and avoids overfitting.

[0103] This invention develops a deep learning-based method for recovering QR code inkjet printing on the surface of welded parts, which can effectively recover reflective QR codes on the surface of welded parts and improve the ability to identify and track welded part information.

[0104] The above are preferred embodiments of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for recovering QR code inkjet printing on the surface of welded parts based on deep learning, characterized in that: Includes the following steps: Step 1: Data Preparation Collect a batch of QR code image data reflecting the surface of welded parts, as well as corresponding clear QR code image data; Step 2: Data Preprocessing By using multi-scale data augmentation, the amount and diversity of the original dataset obtained in step 1 are expanded and increased, making it adaptable to different industrial environments and scenarios. Step 3: Split the dataset: The dataset after preprocessing in step 2 is divided into a training set, a validation set, and a test set; Step 4: Label the training set once. The training set is labeled, marking the positions of light spots and halos; Step 5: Build a deep learning localization model: The localization model is used to locate the light spot and halo regions; the target detection model YOLOv8 is selected, and the selected model is trained using the training set with the light spot and halo positions marked in step 4 to learn the features and position information of the light spot and halo. After training, the best weight file trained by YOLOv8 is saved locally, and the weight model is used as the subsequent pre-trained model. Step 6: Secondary labeling of the training set: The light spot and halo areas located in step 5 are marked again, and the QR code data information is marked. First, the light spot and halo areas are extracted and divided into small blocks by points. Each small block is named in the form of "crop_x_y", where x represents the row number in the QR code and y represents the column number in the QR code. At the same time, the color of the small block is obtained by locating the same area of ​​the original clear QR code according to the segmented small blocks. The small blocks are then stored in two folders, "1" and "0". The small blocks stored in "1" are white and the small blocks stored in "0" are black. The light intensity of the QR codes corresponding to the small blocks stored in the two folders should be similar. Step 7: Build a deep learning recovery model: A deep learning restoration model is built for the image data in the "1" and "0" folders in step 6. The restoration model is used to restore the QR code data. The MobileNet lightweight model is selected to train the restoration model. A black and white patch containing only the light spot and halo area is provided. After the training is completed, the training result is returned to the original image. Step 8: Verification and accuracy assessment.

2. The method for restoring QR code inkjet printing on the surface of weldments based on deep learning as described in claim 1, characterized in that, In step 1 of the method, the images are collected from the field or obtained from existing image sources. The dataset has a sufficient number of samples and includes images under different lighting intensities.

3. The method for restoring QR code inkjet printing on the surface of weldments based on deep learning as described in claim 1, characterized in that, Step 2 in the method specifically includes: Step 2.1 Image geometric transformation: Perform scaling, cropping, and rotation operations on the image; Step 2.2 Image binarization; Step 2.3 Contrast Enhancement: The contrast of the binary image after step 2.2 is adjusted using histogram equalization and adaptive histogram equalization methods. Step 2.4 Sharpening The sharpening masking method is used to enhance the edges and details of the enhanced image after step 2.3; Step 2.5 Noise Reduction The median filtering method is used to remove noise by sorting the pixel values ​​within the window surrounding the pixel and then using the median value as the value of the current pixel.

4. The method for restoring QR code inkjet printing on weldment surface based on deep learning as described in claim 3, characterized in that, Step 2.2 of the method specifically includes: Step 2.2.1: Use the cvtColor function in the OpenCV library to convert the color image after step 2.1 into a grayscale image; Step 2.2.2: Select an appropriate threshold to segment the grayscale image into the target object and the background; Step 2.2.3: Use the threshold function in OpenCV to convert the grayscale image into a binary image; Step 2.2.4: Save the obtained binary image to the specified target location for subsequent processing and analysis.

5. The method for restoring QR code inkjet printing on the surface of weldments based on deep learning as described in claim 1, characterized in that, Step 3 in the method specifically involves: 70% of the data is used as the training set, 15% as the validation set, and 15% as the test set.

6. The method for restoring QR code inkjet printing on the surface of weldments based on deep learning as described in claim 1, characterized in that, Step 8 in the method specifically includes: Step 8.1 Model Evaluation Metrics: Choose accuracy, recall, precision, and F1 score as evaluation metrics to measure the accuracy and performance of the model; Step 8.2 Model Performance Evaluation: The model is evaluated using the validation set, and the metric values ​​of the model on the validation set are calculated. Step 8.3 Error Analysis: Perform error analysis on the evaluation results; Step 8.4 Comparative Experiment: Comparative experiments were conducted with other localization models to evaluate the relative performance of the model; Step 8.5 Hyperparameter Tuning: Based on the evaluation results and error analysis, the hyperparameters of the model are adjusted, and grid search and random search methods are used to determine the optimal combination of hyperparameters. Step 8.6 Repeat the verification: Validation and evaluation were performed multiple times, using different validation sets and randomly initialized models.

7. The method for restoring QR code inkjet printing on the surface of weldments based on deep learning as described in claim 1, characterized in that, In the method described, the QR code is generated by the libqrencode open-source QR code generation library and adopts a dot matrix pattern.

Images