Welding seam detection method, welding seam detection device, electronic device and storage medium
The weld inspection method combining data augmentation and deep learning solves the problem of insufficient robustness in weld inspection, achieves stable and accurate inspection in different environments, and improves the adaptability and accuracy of weld inspection.
Patent Information
- Application Number
- CN202511017826.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-23
- Publication Date
- 2025-10-28
Smart Images
Figure CN120852880A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing, and in particular to weld inspection methods, weld inspection devices, electronic devices, and storage media. Background Technology
[0002] In modern manufacturing, weld inspection is often required to assess welding quality and determine the presence of defects such as porosity, cracks, and undercut. Traditional manual inspection is inefficient and highly subjective, making it difficult to meet the needs of industrial production. Therefore, image processing techniques are employed to achieve automated weld inspection.
[0003] In related technologies, image processing techniques such as threshold segmentation and edge detection are commonly used to detect weld defects. However, these methods have poor adaptability to different environments, resulting in low detection robustness.
[0004] There is currently no effective solution to the problem of low robustness of related technologies for weld inspection. Summary of the Invention
[0005] This embodiment provides a weld inspection method, a weld inspection device, an electronic device, and a storage medium to address the problem of low robustness in weld inspection in related technologies.
[0006] Firstly, this embodiment provides a weld inspection method, including:
[0007] Acquire an image to be inspected; the image to be inspected contains the weld seam region to be inspected.
[0008] Based on the trained weld seam region extraction model, the weld seam region is extracted from the image to be detected to obtain the target weld seam region in the image to be detected; wherein, the weld seam region extraction model is trained based on the data-augmented weld seam image dataset; the data augmentation includes at least: illumination intensity enhancement and angle enhancement;
[0009] Image features are extracted from the target weld area to obtain target features;
[0010] The trained classifier is used to identify the target features and determine the weld category corresponding to the image to be detected.
[0011] In some embodiments, the training process of the weld area extraction model includes:
[0012] Obtain the raw weld seam image dataset;
[0013] The original weld image dataset is subjected to at least illumination intensity enhancement and angle enhancement to obtain an enhanced weld image dataset.
[0014] The augmented weld seam image dataset is labeled, and the initial detection and segmentation model is trained based on the labeling results and the augmented weld seam image dataset to obtain the trained weld seam region extraction model; wherein, the detection and segmentation model is a deep learning-driven computer vision model.
[0015] In some embodiments, the original weld image dataset is subjected to at least illumination intensity enhancement and angle enhancement to obtain an enhanced weld image dataset, including:
[0016] Based on randomly generated brightness and contrast gain factors, pixel-level illumination intensity adjustment is performed on the first image to obtain a dataset of weld seam images with different illumination intensities; and...
[0017] The second image is rotated at a random angle to obtain a dataset of weld seam images from different imaging perspectives.
[0018] The first image is an image from the original weld seam image dataset that has not undergone illumination intensity enhancement; the second image is an image from the original weld seam image dataset that has not undergone angle enhancement.
[0019] In some of these embodiments, the weld region extraction model is constructed based on a single forward propagation detection and segmentation model.
[0020] In some embodiments, the training strategy of the weld area extraction model includes at least one of the following: mixed precision training, mosaic data augmentation, and self-adversarial training.
[0021] In some embodiments, image feature extraction is performed on the target weld area to obtain target features, including:
[0022] Edge detection is performed on the target weld area to obtain weld edge features;
[0023] Texture features are extracted from the target weld area to obtain weld texture features;
[0024] Shape detection is performed on the target weld area to obtain weld shape features;
[0025] The target feature is obtained by fusing the weld edge features, the weld texture features, and the weld shape features.
[0026] In some embodiments, a trained classifier is used to identify the target features and determine the weld category corresponding to the image to be detected, including:
[0027] The target features are identified using a trained support vector machine to determine the weld category corresponding to the image to be detected.
[0028] Secondly, this embodiment provides a weld inspection device, including: an acquisition module, a region extraction module, a feature extraction module, and a recognition module; wherein:
[0029] The acquisition module is used to acquire the image to be detected; the image to be detected contains the weld seam area to be detected.
[0030] The region extraction module is used to extract the weld region from the image to be detected based on the trained weld region extraction model, thereby obtaining the target weld region in the image to be detected; wherein, the weld region extraction model is trained based on a data-enhanced weld image dataset; the data enhancement includes at least: illumination intensity enhancement and angle enhancement;
[0031] The feature extraction module is used to extract image features from the target weld area to obtain target features;
[0032] The recognition module is used to identify the target features using a trained classifier and determine the weld category corresponding to the image to be detected.
[0033] Thirdly, this embodiment provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the weld inspection method described in the first aspect above.
[0034] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the weld inspection method described in the first aspect above.
[0035] Compared with related technologies, this embodiment provides a weld detection method, a weld detection device, an electronic device, and a storage medium. The weld detection method involves acquiring an image to be detected; the image to be detected contains a weld region to be detected; based on a trained weld region extraction model, the weld region is extracted from the image to obtain the target weld region in the image; wherein the weld region extraction model is trained based on a data-augmented weld image dataset; data augmentation includes at least: illumination intensity enhancement and angle enhancement; image feature extraction is performed on the target weld region to obtain target features; and a trained classifier is used to identify the target features to determine the weld category corresponding to the image to be detected. This method enables stable and accurate weld detection under different lighting conditions and imaging perspectives, thereby improving the robustness of weld detection in different environments.
[0036] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description
[0037] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0038] Figure 1 This is a hardware structure block diagram of the terminal of the weld inspection method according to an embodiment of this application;
[0039] Figure 2 This is a flowchart of a weld inspection method according to an embodiment of this application;
[0040] Figure 3 This is a flowchart of a weld inspection method according to some embodiments of this application;
[0041] Figure 4 This is a structural block diagram of the weld inspection device according to an embodiment of this application. Detailed Implementation
[0042] To better understand the purpose, technical solution, and advantages of this application, the application is described and explained below in conjunction with the accompanying drawings and embodiments.
[0043] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.
[0044] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal of the weld inspection method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.
[0045] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the weld seam detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0046] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.
[0047] This embodiment provides a weld inspection method. Figure 2 This is a flowchart of the weld inspection method in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:
[0048] Step S210: Obtain the image to be inspected; the image to be inspected contains the weld area to be inspected.
[0049] This weld inspection method can be applied to weld inspection in automotive manufacturing, shipbuilding, and aerospace industries. The image to be inspected can be captured by an image acquisition device installed in the welding operation scene. For example, a camera can be used as the image acquisition device, installed at a specific location within the welding operation scene to capture a complete and clear image of the weld and its surrounding area. Camera parameters such as resolution and frame rate can be set according to actual conditions to ensure the quality of the acquired image meets welding inspection requirements. For instance, in automotive manufacturing, the camera resolution can be set to 1920×1080 pixels per inch (PPI) and the frame rate to 30 frames per second (fps) to meet the needs of real-time weld inspection. Furthermore, this welding operation scene can also be used for weld inspection in the aerospace industry, where higher precision is required; in such cases, the resolution can be increased to 4K or higher to obtain clearer image details.
[0050] Step S220: Based on the trained weld region extraction model, the weld region is extracted from the image to be detected to obtain the target weld region in the image to be detected; wherein, the weld region extraction model is trained based on the data-enhanced weld image dataset; the data enhancement includes at least: illumination intensity enhancement and angle enhancement.
[0051] The weld seam region extraction model can be a deep learning-driven computer vision model with image segmentation and object detection capabilities. For example, the weld seam region extraction model can be any of the following, or an adaptive improvement thereof: a single-stage real-time object detection model (YOLOv8), a masked region convolutional neural network (MaskR-CNN), a real-time instance segmentation model (YOLOACT++), a point rendering segmentation model (PointRend), etc. Alternatively, it can be a model based on a self-attention visual architecture (Transformer).
[0052] During the model training phase, the impact of different lighting and imaging conditions on weld detection can be considered. Data augmentation processing can be performed on the constructed weld image dataset based on factors such as illumination intensity and angle. Then, the initial detection and segmentation model is trained on the augmented weld image dataset to obtain the trained weld region extraction model. This trained weld region extraction model can adapt to different lighting conditions and shooting angles, thus achieving stable and accurate weld region extraction under various lighting conditions and imaging perspectives, improving the robustness and generalization ability of weld region extraction.
[0053] One approach is to adjust the illumination intensity of weld seam image samples to simulate welding scenarios under different lighting conditions. Without changing the original image content, a series of image samples with different illumination intensities can be generated as weld seam images with enhanced illumination intensity. This increases the diversity of samples, provides rich data support for model training, and improves the adaptability of the weld seam extraction model to different lighting environments.
[0054] Furthermore, by rotating the weld seam image samples and using image processing techniques to avoid image distortion during the rotation process, the imaging of weld seam images under different imaging perspectives can be simulated, thereby further enriching the sample data and enhancing the weld seam region extraction model's ability to identify and extract weld seams at different angles.
[0055] In addition, in some embodiments, noise, such as Gaussian noise or salt-and-pepper noise, can be added to the weld image samples for model training, thereby improving the weld region extraction model's resistance to different noises.
[0056] Step S230: Extract image features from the target weld area to obtain target features.
[0057] Based on the weld seam region extraction achieved in step S220, image features can be extracted from the target weld seam region segmented within the region to be detected using image processing techniques. Specifically, features that characterize the weld seam's shape, structure, texture, and regularity can be extracted from the target weld seam region to provide a basis for subsequent weld seam category identification and to determine whether weld defects exist. Understandably, when multiple image features are extracted, these features can be fused into a single feature vector to obtain the target features.
[0058] Step S240: Using the trained classifier, the target features are identified to determine the weld category corresponding to the image to be detected.
[0059] In this step, a trained classifier is used to identify and classify defects based on feature extraction. For example, this classifier could be a Support Vector Machine (SVM), Random Forest, or Gradient Boosting (XGBoost) algorithm. By processing the target features, the weld category to which the weld region in the image to be detected belongs is determined. Specifically, the identification of this weld category can be the identification of weld defects. For example, the trained classifier outputs the weld category to determine whether the weld contains defects such as porosity, cracks, or undercut.
[0060] By using classifiers for defect identification and classification, objective, standardized, and accurate criteria can be provided for welding quality assessment, thereby facilitating the timely detection of quality problems in the welding process and improving product safety and reliability.
[0061] Related technologies have limitations in adapting to complex environments. For example, they are prone to recognition failure and decreased detection accuracy when there are significant changes in lighting or imaging angle. This embodiment enhances the training data during the model training phase by adjusting dimensions such as lighting intensity and angle to generate image samples with different lighting intensities and angles. This increases sample diversity, enabling it to adapt to different lighting conditions and shooting angles, thereby improving the robustness and generalization ability of weld seam detection.
[0062] Furthermore, this embodiment provides a systematic weld inspection method, covering the entire process from image acquisition to final weld category identification and classification. Through a series of refined image processing and analysis, high-precision weld inspection can be achieved, thereby accurately identifying various weld defects and providing a more reliable and stable foundation for weld quality assessment in the manufacturing industry. From the overall weld inspection process perspective, a weld region extraction model adaptable to different lighting conditions and imaging angles is used to extract and segment the weld region from the currently acquired image to be inspected. Then, image feature extraction is performed based on this, finally achieving weld category identification. Therefore, this embodiment not only improves the accuracy and robustness of detection through image processing improvements in the aforementioned steps, but also optimizes the connection and data transmission between each step, thereby improving the efficiency and accuracy of the overall weld inspection process, and further enhancing the practical value of weld inspection and broadening its application prospects.
[0063] Through steps S210 to S240, an image to be detected is acquired; the image to be detected contains a weld seam region to be detected; based on the trained weld seam region extraction model, the weld seam region is extracted from the image to be detected to obtain the target weld seam region in the image to be detected; wherein, the weld seam region extraction model is trained based on a data-augmented weld seam image dataset; data augmentation includes at least: illumination intensity enhancement and angle enhancement; image features are extracted from the target weld seam region to obtain target features; using a trained classifier, the target features are identified to determine the weld seam category corresponding to the image to be detected. This enables stable and accurate weld seam detection under different lighting conditions and different imaging perspectives, thereby improving the robustness of weld seam detection in different environments.
[0064] In one embodiment, the training process of the weld area extraction model may include:
[0065] Obtain the original weld seam image dataset; perform at least illumination intensity enhancement and angle enhancement on the original weld seam image dataset to obtain the data-enhanced weld seam image dataset; perform data annotation on the data-enhanced weld seam image dataset, and train the initial detection and segmentation model based on the data annotation results and the data-enhanced weld seam image dataset to obtain the trained weld seam region extraction model; wherein, the detection and segmentation model is a computer vision model driven by deep learning.
[0066] Specifically, the process begins by collecting weld image samples covering different weld types and various weld defects to form an original weld image dataset. For example, this dataset could include pre-identified weld images with defects such as porosity, cracks, and undercut, as well as normal welds. Next, the original weld image dataset undergoes data augmentation processing, including enhancements to lighting intensity, angle, and noise, to obtain an augmented weld image dataset. Finally, based on this augmentation, the augmented weld image dataset is labeled.
[0067] During the data annotation process, each weld image sample in the augmented weld image dataset is annotated with information such as weld location, shape, and size, as well as information on the presence of weld defects. Specific annotation methods can be based on manual annotation or semi-automatic annotation. Open-source rectangular annotation tools (LabelImg) and multi-functional annotation tools (VGG Image Annotator) can be used to annotate the weld area to generate annotation files.
[0068] The generated annotation information can be saved in Extensible Markup Language (XML) file format. The annotation information can specifically include: weld boundary information (e.g., weld shape, location), category labels (e.g., whether it is a normal weld, whether porosity is present, whether cracks are present, whether undercut is present), and segmentation masks for the weld region. For example, when annotating a weld image sample with porosity defects, the image coordinates and size of the pores in the weld image sample can be recorded in the corresponding annotation file, and the category label can be set to "porosity," thus facilitating subsequent model training.
[0069] During model training, the initial detection and segmentation model is trained based on the data-augmented weld seam image dataset and the corresponding data annotation results (the annotation information mentioned above). During training, the detection and segmentation model can output the bounding box coordinates of the weld seam region and the weld seam list information, and then output a segmentation mask to accurately segment the weld seam region. The weld seam category information output by the detection and segmentation model can be used as the label data for training the classifier. Based on this, weld seam detection and segmentation can be achieved simultaneously, providing accurate regional basis for subsequent feature extraction and defect category identification.
[0070] In one embodiment, the original weld image dataset is subjected to at least illumination intensity enhancement and angle enhancement to obtain an enhanced weld image dataset, which may specifically include:
[0071] Based on randomly generated brightness gain factors and contrast gain factors, pixel-level illumination intensity adjustment is performed on the first image to obtain weld seam image datasets with different illumination intensities; and, based on random angles, image rotation processing is performed on the second image to obtain weld seam image datasets with different imaging perspectives; wherein, the first image is the original weld seam image dataset without illumination intensity enhancement; and the second image is the original weld seam image dataset without angle enhancement.
[0072] In this embodiment, the brightness and contrast of the first image can be randomly adjusted based on a light intensity adjustment algorithm to simulate welding scenarios under different lighting conditions. For example, a brightness gain factor and a contrast gain factor can be randomly generated based on a preset probability distribution. For instance, a brightness gain factor can be randomly generated within the numerical range [0.5, 1.5], and a contrast gain factor can be randomly generated within the numerical range [0.6, 1.4]. Then, pixel-level brightness adjustment and pixel-level contrast adjustment of the first image can be performed based on the brightness gain factor and the contrast gain factor, respectively. The brightness adjustment formula can be expressed as follows:
[0073] I_adjusted=I_original×gain+bias;
[0074] Where I_original represents the original pixel value in the first image, gain is the brightness gain factor, bias is the bias factor, and I_adjusted is the pixel value after brightness adjustment. The contrast adjustment method is similar and will not be described in detail here.
[0075] The luminance gain factor and bias factor can be calculated as follows:
[0076] gain = 1 + (random() - 0.5) × 1.0;
[0077] bias = (random() - 0.5) × 0.2;
[0078] Here, random() represents a random number within the interval [0,1].
[0079] The first image can be a weld image sample from the original weld image dataset that has neither undergone light intensity adjustment nor angle adjustment, or it can be a weld image sample that has undergone angle adjustment but not light intensity adjustment.
[0080] By augmenting the original weld seam image dataset with light intensity data, images under different lighting conditions can be generated without changing the original image content, increasing the diversity of samples and improving the model's adaptability to different lighting environments.
[0081] Understandably, the above is only a specific example of light intensity adjustment. The generation method and value range of the brightness gain factor can be adaptively adjusted according to the needs of the actual application scenario. This embodiment does not impose any specific limitations on this.
[0082] Furthermore, when performing angle data enhancement, the weld image samples can be processed based on an image rotation algorithm. The second image for angle adjustment here can be a weld image sample from the original weld image dataset that has neither undergone the aforementioned light intensity adjustment nor angle adjustment, or it can be a weld image sample that has undergone light intensity adjustment but not angle adjustment.
[0083] The rotation can be performed with the image center of the second image as the rotation center, and the rotation angle can be set to the range of [-30°, 30°]. During the rotation, a bilinear interpolation algorithm can be used to ensure the reasonableness of the pixel values after the image rotation, so as to avoid image distortion. Assuming the rotation angle is θ, the corresponding rotation matrix R is:
[0084] R=[cosθ,-sinθ;sinθ,cosθ];
[0085] For each pixel (x, y) in the second image, its coordinates after rotation based on the rotation matrix are (x', y'):
[0086] (x',y')=R×(x-cx,y-cy)+(cx,cy);
[0087] Where (cx, cy) are the coordinates of the image center.
[0088] Based on this, the coordinates of each pixel in the second image after rotation can be accurately obtained, thereby realizing the rotation operation of the second image. It is understood that the above angle enhancement process is merely an example, and those skilled in the art can use different value ranges and different rotation methods to achieve angle enhancement according to the needs of actual application scenarios. This embodiment does not impose specific limitations in this regard.
[0089] By performing angular data augmentation on the welding image dataset, it is possible to simulate the imaging of welds under different imaging perspectives, thereby further enriching the sample data and enhancing the model's ability to detect welds at different angles.
[0090] Therefore, by combining enhanced light intensity and enhanced angle, this embodiment can more comprehensively simulate the impact of different environmental conditions on weld images in weld inspection scenarios, thereby improving the robustness of weld inspection.
[0091] Furthermore, in one embodiment, the aforementioned weld region extraction model is constructed based on a single forward propagation detection and segmentation model. In this embodiment, YOLOv8 is introduced as the detection and segmentation model into the weld region extraction stage to achieve weld region detection and segmentation. YOLOv8 consists of a backbone network, a neck network, and a head network.
[0092] During weld region extraction, the backbone network, such as EfficientNet, extracts features from the input image to be detected. Specifically, it can extract features from the image through multi-layer convolution operations. The neck network, such as Cross-Stage Local Dense Connection Residual Module (CSP-DPR), further integrates and enhances the output of the backbone network. Specifically, it can fuse and enhance the features output by the backbone network. The head network outputs the bounding box coordinates, weld category information, and segmentation mask of the identified weld region to achieve accurate segmentation of the weld region.
[0093] After model training is complete, the image to be detected is input into the trained YOLOv8 model. The model first outputs the bounding box coordinates and weld category of the weld region in the image; then it outputs a segmentation mask to segment the weld region, thus obtaining the target weld region. Therefore, the YOLOv8 model can simultaneously detect and segment welds, exhibiting strong real-time performance, high detection accuracy, and high segmentation accuracy. Thus, implementing the YOLOv8 model to extract weld regions enables more accurate weld edge recognition and extraction, providing a more reliable regional basis for subsequent feature extraction and defect identification. For example, by inputting a captured image to be detected into the YOLOv8 model, the model can quickly and accurately locate the weld region and segment it from the background, thereby saving processing time and computational resources for subsequent feature extraction.
[0094] In related technologies, deep learning methods used for weld inspection often have limitations in terms of real-time performance and adaptability due to their high computational complexity. To address this, this embodiment employs the YOLOv8 model for image segmentation, which provides faster inference speed and higher detection accuracy for weld inspection, and achieves precise weld region extraction, thereby meeting the real-time detection requirements in weld inspection scenarios.
[0095] In one embodiment, the training strategy for the weld area extraction model includes at least one of the following: mixed precision training, mosaic data augmentation, and self-adversarial training.
[0096] Specifically, mixed-precision training is a technique that uses both single-precision (FP32) and half-precision (FP16) floating-point numbers during training to accelerate training and reduce memory usage. Specifically, during the forward and backward propagation of model training, half-precision floating-point numbers are used for model weights and gradients, while single-precision floating-point numbers are used when updating weights. For example, during forward propagation, the model's input data and parameters can be calculated using half-precision floating-point numbers, thus reducing memory usage and computation time. During backward propagation, gradients are calculated using half-precision floating-point numbers. When updating model weights, the calculated gradients are converted to single-precision floating-point numbers, thereby improving gradient precision and avoiding numerical instability issues.
[0097] Therefore, mixed-precision training can reduce model training time, which is especially suitable for training deep neural networks that require more parameters. Furthermore, the use of half-precision floating-point numbers in mixed-precision training can reduce memory usage, allowing larger models or larger batch sizes to be run even with limited hardware resources. In addition, using single-precision floating-point numbers during weight updates can avoid gradient explosion or vanishing problems, thereby improving the stability of model training.
[0098] Mosaic data augmentation is a data augmentation method that stitches multiple images together to form a new image. Specifically, when performing mosaic data augmentation, several images can be randomly selected from the training dataset, uniformly cropped into several small patches, and then stitched together to form a new image. This increases the diversity of the samples to simulate changes in images under different scenarios.
[0099] For example, four images can be randomly selected from a weld seam image dataset. These images are then cropped into small blocks of the same size, such as 256×256 pixels each. The cropped blocks are then stitched together to form a new image. During annotation, the bounding box coordinates and segmentation mask of the weld seam region are recalculated based on the stitched image, thus ensuring the accuracy of the labeling information.
[0100] Mosaic data augmentation provides richer scene and background information for new images generated from image stitching, thereby increasing sample diversity. Exposing the model to more diverse images during training also helps improve its adaptability to different scenes. Furthermore, simulating image variations under different lighting, angles, and background conditions can enhance the model's robustness.
[0101] Additionally, self-adversarial training is a technique that uses the idea of Generative Adversarial Networks (GANs) to enhance the robustness of a model. During training, a generator is used to generate adversarial examples (which can be input images with minor perturbations). These adversarial examples are then used to train the detector, improving its ability to recognize and handle such perturbations. In practice, a generator can be used to add minor perturbations to the input image to obtain adversarial examples. These generated adversarial examples, along with their unperturbated counterparts (the original samples), are then fed into the detector for training, enabling the detector to simultaneously identify weld seam regions in both the original and adversarial samples. An adversarial loss term can be added to the model's training loss function to ensure the detector can correctly handle adversarial examples. Based on self-adversarial training, the model's robustness to various perturbations can be improved, adapting to image variations in different environments and enhancing generalization ability. Therefore, the model can more accurately identify weld seam regions, reducing false positives and false negatives.
[0102] The loss function can include several components such as classification loss, localization loss, and segmentation loss. For classification loss, the cross-entropy loss function can be used; for localization loss, the complete intersection-union (CloU) loss function can be used; and for segmentation loss, the binary cross-entropy loss function can be used. The classification loss measures the difference between the model's output and the true label during training; the localization loss measures the model's ability to locate the weld seam region during training; and the segmentation loss measures the model's ability to extract regions.
[0103] In this way, the model can achieve more accurate, efficient and stable extraction and segmentation of weld areas on the images to be detected under different conditions, thereby improving the overall accuracy and robustness of weld detection.
[0104] In one embodiment, image feature extraction is performed on the target weld area to obtain target features, which may specifically include:
[0105] Edge detection is performed on the target weld area to obtain weld edge features; texture features are extracted from the target weld area to obtain weld texture features; shape detection is performed on the target weld area to obtain weld shape features; feature fusion is performed on the weld edge features, weld texture features, and weld shape features to obtain target features.
[0106] One approach is to extract weld edge features from the target weld area based on edge detection algorithms. For example, the Canny edge detection algorithm can be used for edge feature extraction. First, Gaussian filtering is applied to the target weld area, then the image gradient is calculated, followed by non-maximum suppression, double threshold detection, and edge connection to complete the extraction of weld edge features.
[0107] The extracted weld edge features can include information such as edge length, width, and direction to reflect the shape and structural characteristics of the weld, thus providing an accurate and reliable basis for subsequent weld category identification. Specifically, when performing edge detection based on the Canny operator, low and high thresholds can be automatically calculated based on the grayscale histogram and noise level in the target weld region. Then, based on the high and low threshold settings, hysteresis thresholding is used to filter out true edge pixels and suppress noise. For example, the high threshold can be determined based on Otsu's method, and the low threshold can be set to 0.4 times the high threshold, thereby achieving complete and accurate edge extraction. For an image region with a relatively uniform grayscale histogram, if the high threshold calculated by Otsu's method is 100, the corresponding low threshold is 40.
[0108] Understandably, the above-described edge feature extraction process and specific values are merely examples, and those skilled in the art can also choose other edge feature extraction methods to extract weld edge features according to the needs of actual application scenarios.
[0109] Texture feature extraction can be achieved using the Gray-Level Co-occurrence Matrix (GLCM). The GLCM is a two-dimensional matrix that describes the spatial relationships between the gray values of pixels in an image. In this embodiment, by calculating various statistical parameters of the GLCM, such as energy, contrast, homogeneity, and correlation, the texture information of the weld seam can be obtained. Specifically:
[0110] The calculation of energy can be performed as follows:
[0111] E=ΣΣ(p(i,j))²;
[0112] The contrast ratio can be calculated as follows:
[0113] C = ΣΣ|ij|²p(i,j);
[0114] The calculation of homogeneity can be performed as follows:
[0115] H=ΣΣp(i,j) / (1+|ij|);
[0116] The correlation can be calculated as follows:
[0117] R=[ΣΣ(i-μi)(j-μj)p(i,j)] / (σiσj);
[0118] Where p(i,j) is the probability of a pair of pixels with gray values i and j appearing in the gray-level co-occurrence matrix, μi and μj are the mean values of gray values i and j respectively, and σi and σj are the standard deviations of gray values i and j respectively.
[0119] Texture features extracted from the gray-level co-occurrence matrix (GLCM) can reflect information such as the roughness and texture directionality of the weld surface. Therefore, it helps in identifying minute defects in the weld. For example, when analyzing an image of a relatively rough weld surface, by calculating parameters such as the contrast and homogeneity of the GLCM, its texture features can be quantitatively described, providing a strong basis for defect identification.
[0120] For shape features, these can be obtained by acquiring the area, perimeter, and shape factor of the target weld region. For example, the area of the target weld region, Area, can be determined based on the following formula:
[0121] Area = ΣΣf(x,y);
[0122] Where f(x,y) represents the pixel value of the target weld area.
[0123] The perimeter of the target weld area can be determined based on the following formula:
[0124] Perimeter = ΣΣδ(x,y);
[0125] Where δ(x,y) represents the pixel value of the target weld edge.
[0126] Furthermore, the shape factor can be defined as:
[0127] Shape Factor=4π×Area / Perimeter²;
[0128] Shape factors can reflect the regularity of weld shape, thus providing accurate data for subsequent weld category identification. For example, if the weld shape in the target weld area is irregular, shape factors can be extracted to determine whether shape-level defects exist. Understandably, shape factors can also be extracted based on other types of shape parameters.
[0129] The extracted weld edge features, weld texture features, and weld shape features can be fused to construct a single feature vector. The dimension of this feature vector can be the sum of the number of each feature parameter (weld edge features, weld texture features, and weld shape features), with each parameter serving as a category within the feature vector. By integrating the multi-dimensional feature information of the weld, a richer and more comprehensive feature representation can be constructed, providing a comprehensive feature description for subsequent weld category identification. For example, the five parameters contained in the weld edge features, the four parameters contained in the weld texture features, and the three parameters contained in the weld shape features can be fused to construct a 12-dimensional feature vector, comprehensively describing the weld's feature information.
[0130] Compared to related technologies that rely on relatively single features for weld inspection, resulting in lower accuracy in weld defect identification, this embodiment employs a multi-feature extraction and fusion mechanism. This mechanism can more comprehensively describe the characteristic information of the weld, thereby improving the accuracy of weld category identification.
[0131] In another embodiment, using a trained classifier to identify target features and determine the weld type corresponding to the image to be detected may include:
[0132] The trained support vector machine is used to identify target features and determine the weld category corresponding to the image to be detected.
[0133] In this embodiment, a Support Vector Machine (SVM) is introduced for weld category identification. SVM is a classification method based on statistical learning theory. It finds an optimal hyperplane to distinguish data points belonging to different categories, maximizing the margin between the two classes. During the training of the SVM, a pre-constructed dataset labeled with weld categories can be used to train the SVM, determining its parameters (e.g., the type of kernel function such as linear kernel or RBF kernel), penalty function, etc., thereby enabling the SVM to accurately identify weld categories and determine whether welds contain defects based on the aforementioned target features.
[0134] This embodiment uses a support vector machine (SVM) as a classifier to identify weld types based on target features. This provides an automatic, quantitative, and more efficient weld type identification method, thus offering accurate classification criteria for welding quality assessment. For example, the image to be detected is input into the trained SVM. Based on the positional relationship between the target features of the image and the hyperplane, the SVM outputs the weld type information of the image, ultimately concluding that the weld has porosity defects, providing crucial classification information for subsequent quality control.
[0135] Figure 3 These are flowcharts of some embodiments of weld inspection methods, such as... Figure 3 As shown, the weld inspection method includes the following steps:
[0136] Step S301: Collect weld image samples containing different weld defects and normal welds as the original weld image dataset.
[0137] Step S302: Based on the enhancement of light intensity and angle, perform data augmentation on the original weld seam image dataset to obtain the data-enhanced weld seam image dataset.
[0138] Step S303: Data annotation is performed on the data-augmented weld seam image dataset, and the initial YOLOv8 model is trained based on the data annotation results and the data-augmented weld seam image dataset.
[0139] Step S304: During the training process, YOLOv8 is trained based on mixed precision training, mosaic data augmentation, and self-adversarial training techniques.
[0140] Step S305: Input the image to be detected into the trained YOLOv8 model to obtain the target weld area.
[0141] Step S306: Extract weld edge features, weld texture features, and weld shape features from the target weld area.
[0142] Step S307: The weld edge features, weld texture features, and weld shape features are fused into target features.
[0143] Step S308: Input the target features into the trained support vector machine for classification and recognition to obtain the weld category corresponding to the image to be detected.
[0144] Steps S301 to S308 above, based on enhancing the light intensity and angle of the training data, generate image samples with different light intensities and angles, increasing sample diversity and improving the robustness and generalization ability of the algorithm. Applying the YOLOv8 model to weld region extraction enables more accurate and efficient extraction of weld regions, providing reliable regional basis for subsequent feature extraction and defect identification, and meeting the real-time requirements of weld detection. Based on a multi-feature fusion feature extraction method, edge features, texture features, and shape features of the weld are fused to construct a feature vector, which can more comprehensively describe the feature information of the weld and improve the accuracy of defect identification. Weld category identification based on support vector machines can accurately determine whether there are defects in the weld, such as common welding defects like porosity, cracks, and undercut, providing an accurate basis for welding quality assessment. Overall, this achieves a fully optimized weld detection process with higher detection accuracy and efficiency.
[0145] This embodiment also provides a weld inspection device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0146] Figure 4 This is a structural block diagram of the weld inspection device 40 in this embodiment, as shown below. Figure 4 As shown, the weld inspection device 40 includes: an acquisition module 42, a region extraction module 44, a feature extraction module 46, and an identification module 48; wherein:
[0147] The acquisition module 42 is used to acquire the image to be detected; the image to be detected contains the weld seam region to be detected; the region extraction module 44 is used to extract the weld seam region from the image to be detected based on the trained weld seam region extraction model, to obtain the target weld seam region in the image to be detected; wherein, the weld seam region extraction model is trained based on the data-enhanced weld seam image dataset; the data enhancement includes at least: illumination intensity enhancement and angle enhancement; the feature extraction module 46 is used to extract image features from the target weld seam region to obtain target features; the recognition module 48 is used to use the trained classifier to recognize the target features and determine the weld seam category corresponding to the image to be detected.
[0148] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.
[0149] This embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.
[0150] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.
[0151] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:
[0152] S1, acquire the image to be inspected; the image to be inspected contains the weld area to be inspected;
[0153] S2, Based on the trained weld region extraction model, the weld region is extracted from the image to be detected to obtain the target weld region in the image to be detected; wherein, the weld region extraction model is trained based on the data-augmented weld image dataset; the data augmentation includes at least: illumination intensity enhancement and angle enhancement;
[0154] S3, extract image features from the target weld area to obtain target features;
[0155] S4. Using the trained classifier, the target features are identified to determine the weld category corresponding to the image to be detected.
[0156] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.
[0157] Furthermore, in conjunction with the weld inspection methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the weld inspection methods described in the above embodiments.
[0158] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.
[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0160] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.
[0161] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.
[0162] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.
Claims
1. A weld inspection method, characterized in that, include: Acquire an image to be inspected; the image to be inspected contains the weld seam region to be inspected. Based on the trained weld seam region extraction model, the weld seam region is extracted from the image to be detected to obtain the target weld seam region in the image to be detected; wherein, the weld seam region extraction model is trained based on the data-augmented weld seam image dataset; the data augmentation includes at least: illumination intensity enhancement and angle enhancement; Image features are extracted from the target weld area to obtain target features; The trained classifier is used to identify the target features and determine the weld category corresponding to the image to be detected.
2. The weld inspection method according to claim 1, characterized in that, The training process of the weld area extraction model includes: Obtain the raw weld seam image dataset; The original weld image dataset is subjected to at least illumination intensity enhancement and angle enhancement to obtain an enhanced weld image dataset. The augmented weld seam image dataset is labeled, and the initial detection and segmentation model is trained based on the labeling results and the augmented weld seam image dataset to obtain the trained weld seam region extraction model; wherein, the detection and segmentation model is a deep learning-driven computer vision model.
3. The weld inspection method according to claim 2, characterized in that, The original weld image dataset is subjected to at least illumination intensity enhancement and angle enhancement to obtain an enhanced weld image dataset, including: Based on randomly generated brightness and contrast gain factors, pixel-level illumination intensity adjustment is performed on the first image to obtain a dataset of weld seam images with different illumination intensities; and... The second image is rotated at a random angle to obtain a dataset of weld seam images from different imaging perspectives. The first image is an image from the original weld seam image dataset that has not undergone illumination intensity enhancement; the second image is an image from the original weld seam image dataset that has not undergone angle enhancement.
4. The weld inspection method according to claim 2, characterized in that, The weld seam region extraction model is constructed based on a single forward propagation detection and segmentation model.
5. The weld inspection method according to claim 4, characterized in that, The training strategy for the weld area extraction model includes at least one of the following: mixed precision training, mosaic data augmentation, and self-adversarial training.
6. The weld inspection method according to claim 1, characterized in that, Image feature extraction is performed on the target weld area to obtain target features, including: Edge detection is performed on the target weld area to obtain weld edge features; Texture features are extracted from the target weld area to obtain weld texture features; Shape detection is performed on the target weld area to obtain weld shape features; The target feature is obtained by fusing the weld edge features, the weld texture features, and the weld shape features.
7. The weld inspection method according to any one of claims 1 to 6, characterized in that, Using a trained classifier, the target features are identified to determine the weld category corresponding to the image to be detected, including: The target features are identified using a trained support vector machine to determine the weld category corresponding to the image to be detected.
8. A weld inspection device, characterized in that, include: The module comprises an acquisition module, a region extraction module, a feature extraction module, and a recognition module; among which: The acquisition module is used to acquire the image to be detected; the image to be detected contains the weld seam area to be detected. The region extraction module is used to extract the weld region from the image to be detected based on the trained weld region extraction model, thereby obtaining the target weld region in the image to be detected; wherein, the weld region extraction model is trained based on a data-enhanced weld image dataset; the data enhancement includes at least: illumination intensity enhancement and angle enhancement; The feature extraction module is used to extract image features from the target weld area to obtain target features; The recognition module is used to identify the target features using a trained classifier and determine the weld category corresponding to the image to be detected.
9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the weld inspection method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the weld inspection method according to any one of claims 1 to 7.