Method and device for detecting defects of a welded workpiece and storage medium

Abstract

The embodiment of the application provides a kind of welding workpiece defect detection method, device and storage medium, the method comprises: obtaining the welding image of welding workpiece and the preset welding position information of the welding workpiece;The welding image of the welding workpiece is input to the first detection model pre-trained, obtain the contour of the welding workpiece in the welding image that the first detection model outputs;The first detection model is pre-trained, for contour feature extraction to input welding image, output the model of the contour of welding workpiece in welding image;According to the contour of the welding workpiece that the first detection model outputs and the preset welding position information of the welding workpiece, determine whether the welding position of the welding workpiece is abnormal;If abnormal, it is determined that there is welding workpiece position degree offset defect.The defect detection efficiency of welding workpiece is improved.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and specifically to a defect detection method, apparatus and storage medium for welded workpieces. Background Technology

[0002] To improve product quality, it is typically necessary to inspect welded workpieces for defects such as incomplete welds, missing parts, and weld points climbing over the wall. However, the detection of defects like incomplete welds and weld points climbing over the wall should be based on the premise that the welded workpiece does not exhibit weld positional misalignment. If a welded workpiece has a weld positional misalignment, it indicates a deviation in the welded position. In this case, even if there are no other defects such as weld points climbing over the wall or incomplete welds, the workpiece is unusable due to the weld positional misalignment. Therefore, when inspecting welded workpieces for defects, it is usually necessary to first check for weld positional misalignment. Currently, the detection methods for weld positional misalignment and other welding defects typically employ traditional image algorithms and visual inspection. Traditional image algorithms suffer from insufficient robustness and have low detection efficiency due to their high correlation between the product and the image. Visual inspection requires manual intervention, has low detection efficiency, and is prone to missed detections. Summary of the Invention

[0003] In view of this, this application provides a method, apparatus and storage medium for detecting defects in welded workpieces, so as to solve the problem of low efficiency in detecting defects in welded workpieces in the prior art.

[0004] In a first aspect, embodiments of this application provide a method for detecting defects in welded workpieces, including:

[0005] Acquire welding images of the workpiece and preset welding position information of the workpiece;

[0006] The welding image of the welded workpiece is input into a pre-trained first detection model to obtain the contour of the welded workpiece in the welding image output by the first detection model; the first detection model is pre-trained and is used to extract contour features from the input welding image and output the model of the contour of the welded workpiece in the welding image.

[0007] Based on the outline of the welded workpiece and the preset welding position information of the welded workpiece output by the first detection model, determine whether the welding position of the welded workpiece is abnormal.

[0008] If an anomaly is found, it indicates that there is a positional deviation defect in the welded workpiece.

[0009] If there is no abnormality in the welding position of the workpiece, a target welding image is obtained; the target welding image is the welding image at the welding position of the workpiece.

[0010] The target welding image is input into a pre-trained second detection model to obtain the welding information of the welded workpiece in the target welding image output by the second detection model; the second detection model is pre-trained and is used to perform semantic segmentation on the input target welding image, extract multi-channel welding feature information, and output the welding information of the welded workpiece in the target welding image.

[0011] Based on the welding information of the welded workpiece, determine whether the welded workpiece has welding defects.

[0012] Preferably, determining whether the welding position of the welded workpiece is abnormal based on the outline of the welded workpiece output by the first detection model and the preset welding position information of the welded workpiece includes:

[0013] Based on the outline of the welded workpiece output by the first detection model, the welding position of the welded workpiece in the welding image is determined;

[0014] Based on the welding position of the workpiece in the welding image and the preset welding position information of the workpiece, calculate the interval distance between the welding position of the workpiece in the welding image and the preset welding position of the workpiece.

[0015] Obtain a preset distance threshold and determine whether the interval distance is greater than the preset distance threshold;

[0016] If the interval distance is greater than a preset distance threshold, the welding position of the workpiece is determined to be abnormal.

[0017] Preferably, the first detection model includes: a first encoder, a first decoder, and a first classifier;

[0018] The first encoder is used to extract contour feature information from the input welding image;

[0019] The first decoder is used to perform deconvolution and feature concatenation on the contour feature information extracted by the first encoder to obtain the first feature information;

[0020] The first classifier is used to classify each pixel in the welding image based on the first feature information to obtain the welding position information of the welded workpiece.

[0021] Preferably, the second detection model includes: a second encoder, a second decoder, and a second classifier;

[0022] The second encoder is used to perform semantic segmentation on the input target welding image and extract multi-channel welding feature information;

[0023] The second decoder is used to upsample and fuse the welding feature information of multiple channels to obtain the second feature information.

[0024] The second classifier is used to classify the second feature information to obtain the welding information of the welded workpiece.

[0025] Preferably, the welding information includes at least one of the following: weld point outline, first workpiece outline, second workpiece outline, weld point center, and defect point.

[0026] Preferably, determining whether there are welding defects in the welded workpiece based on the welding information of the welded workpiece includes:

[0027] When the welding information includes the center of the weld point, determine whether there are defects such as missing welds, missing parts, or missing weld points based on the center of the weld point; or,

[0028] When the welding information includes a first workpiece outline and a weld point outline, determine whether there is a defect of weld point misalignment based on the first workpiece outline and the weld point outline; or,

[0029] When the welding information includes a weld point outline and a second workpiece outline, determine whether a weld point climbing defect exists based on the second workpiece outline and the weld point outline; or,

[0030] When the welding information includes a first workpiece profile, the first workpiece angle is determined based on the first workpiece profile, and the presence of a defect in the first workpiece angle is determined based on the first workpiece angle; or,

[0031] When the welding information includes the contour of the second workpiece, it is detected whether the contour of the second workpiece conforms to a preset rule to determine whether the second workpiece has deformation defects; or,

[0032] When the welding information includes the center of the weld joint and the defect point, it is determined whether there is a gap defect based on the center of the weld joint and the defect point.

[0033] Preferably, determining the angle of the first workpiece based on the contour of the first workpiece includes:

[0034] Based on the first workpiece contour, determine the position of the first workpiece contour in the first coordinate system;

[0035] The angle of the first workpiece is determined based on the position of the first workpiece contour in the first coordinate system.

[0036] Preferably, detecting whether the contour of the second workpiece conforms to a preset rule includes:

[0037] Based on the second workpiece contour, determine the position of the second workpiece contour in the second coordinate system;

[0038] The shape corresponding to the second workpiece contour is determined based on the position of the second workpiece contour;

[0039] Determine whether the shape corresponding to the contour of the second workpiece conforms to the preset rules.

[0040] Preferably, acquiring the target welding image includes:

[0041] Based on the outline of the welded workpiece, the welding position area of ​​the welded workpiece in the welding image is cut to obtain the target welding image.

[0042] Preferably, the detection method further includes:

[0043] When it is determined that there is a welding defect in the welded workpiece, display information of the welding defect is generated, so as to display the concrete image or characteristic value of the welding defect according to the display information.

[0044] Secondly, embodiments of this application provide a defect detection device for welded workpieces, comprising:

[0045] A communicator for receiving a welding image of a workpiece from an image acquisition module and preset welding position information of the workpiece from an input device;

[0046] The processor, coupled to the communicator, is used to input the welding image of the welded workpiece into a pre-trained first detection model to obtain the contour of the welded workpiece in the welding image output by the first detection model; the first detection model is pre-trained and is used to extract contour features from the input welding image and output a model of the contour of the welded workpiece in the welding image.

[0047] Based on the outline of the welded workpiece and the preset welding position information of the welded workpiece output by the first detection model, determine whether the welding position of the welded workpiece is abnormal.

[0048] If an anomaly is found, it indicates that there is a positional deviation defect in the welded workpiece.

[0049] If there is no abnormality in the welding position of the workpiece, a target welding image is obtained; the target welding image is the welding image at the welding position of the workpiece.

[0050] The target welding image is input into a pre-trained second detection model to obtain the welding information of the welded workpiece in the target welding image output by the second detection model; the second detection model is pre-trained and is used to perform semantic segmentation on the input target welding image, extract multi-channel welding feature information, and output the welding information of the welded workpiece in the target welding image.

[0051] Based on the welding information of the welded workpiece, determine whether the welded workpiece has welding defects.

[0052] Preferably, the first detection model includes: a first encoder, a first decoder, and a first classifier; the first encoder is used to extract contour feature information from the input welding image; the first decoder is used to perform deconvolution and feature concatenation on the contour feature information extracted by the first encoder to obtain first feature information; the first classifier is used to classify each pixel in the welding image based on the first feature information to obtain the welding position information of the welded workpiece.

[0053] Preferably, the second detection model includes: a second encoder, a second decoder, and a second classifier; the second encoder is used to perform semantic segmentation on the input target welding image and extract multi-channel welding feature information; the second decoder is used to upsample and fuse the multi-channel welding feature information to obtain second feature information; the second classifier is used to classify the second feature information to obtain welding information of the welded workpiece.

[0054] The welding information includes at least one of the following: weld point outline, first workpiece outline, second workpiece outline, weld point center, and defect point.

[0055] Thirdly, embodiments of this application provide a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the method described in any of the first aspects above.

[0056] Using the solution provided in this application embodiment, the welding image of the welded workpiece can be input into a pre-trained first detection model, and the contour of the welded workpiece in the welding image can be output by the first detection model. Based on the contour of the welded workpiece output by the first detection model and the preset welding position information of the welded workpiece, it can be determined whether the welding position of the welded workpiece is offset. If offset is determined, it is determined that there is a welding workpiece position offset defect. For welded workpieces with no abnormal welding position, their corresponding welding image is further input into a second detection model. The second detection model extracts different welding feature information in the welding image, which can further realize the detection of different welding defects of the welded workpiece and improve the accuracy of welding workpiece defect detection. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 A schematic diagram of the structure of a defect detection device for welded workpieces provided in an embodiment of this application;

[0059] Figure 2a A schematic diagram illustrating a scenario for defect detection of a welded workpiece, provided in an embodiment of this application;

[0060] Figure 2b A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0061] Figure 2c A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0062] Figure 3 A schematic flowchart illustrating a defect detection method for welded workpieces provided in an embodiment of this application;

[0063] Figure 4 A flowchart illustrating a training method for a first detection model provided in an embodiment of this application;

[0064] Figure 5a A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0065] Figure 5b A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0066] Figure 5c A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0067] Figure 6 A schematic flowchart illustrating another defect detection method for welded workpieces provided in an embodiment of this application;

[0068] Figure 7a A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0069] Figure 7b A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0070] Figure 8 A flowchart illustrating a training method for a second detection model provided in an embodiment of this application;

[0071] Figure 9a A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0072] Figure 9b A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0073] Figure 9c A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0074] Figure 9d A schematic diagram illustrating another scenario for defect detection of a welded workpiece provided in an embodiment of this application;

[0075] Figure 10 This is a schematic diagram of another defect detection device for welded workpieces provided in an embodiment of this application. Detailed Implementation

[0076] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0077] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0078] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0079] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0080] To improve product quality, it is typically necessary to inspect welded workpieces for defects such as incomplete welds, missing parts, and weld points climbing over the wall. However, the detection of defects like incomplete welds and weld points climbing over the wall should be based on the premise that the welded workpiece does not exhibit weld positional misalignment. If a welded workpiece has a weld positional misalignment, it indicates a deviation in the welded position. In this case, even if there are no other defects such as weld points climbing over the wall or incomplete welds, the workpiece is unusable due to the weld positional misalignment. Therefore, when inspecting welded workpieces for defects, it is usually necessary to first check for weld positional misalignment. Currently, the detection methods for weld positional misalignment and other welding defects typically employ traditional image algorithms and visual inspection. Traditional image algorithms suffer from insufficient robustness and have low detection efficiency due to their high correlation between the product and the image. Visual inspection requires manual intervention, has low detection efficiency, and is prone to missed detections.

[0081] To address the aforementioned problems, this application provides a method, apparatus, and storage medium for detecting defects in welded workpieces. When it is necessary to detect the offset of the welding position of a certain welded workpiece, a welding image at the welding position of the workpiece can be acquired. This welding image is input into a pre-trained first detection model, which outputs the welding position information of the welding image in the welded workpiece. Based on the welding position information output by the first detection model and preset welding position information of the workpiece, it is determined whether the welding position of the workpiece has offset. If offset is determined, a welding position offset defect is identified. In this application, the first detection model can output the welding position information of different welded workpieces, which is then compared with the preset welding position to determine whether the welding position of the workpiece has offset, enhancing robustness. Furthermore, no manual intervention is required in the process of detecting whether the welding position of the workpiece has offset defects, improving detection efficiency. A detailed description follows.

[0082] See Figure 1 This is a hardware architecture diagram of a defect detection device for welded workpieces provided in an embodiment of this application. An embodiment of this application provides a defect detection device 100 for welded workpieces, including a communicator 101 and a processor 102.

[0083] A communicator 101 is used to establish a communication channel, enabling the defect detection device 100 for the welded workpiece to communicate with other devices. It receives data from other devices or sends data to other devices. In this embodiment, the communicator 101 can receive welding images of the welded workpiece acquired by an image acquisition module and receive preset welding position information of the welded workpiece from an input device. The image acquisition module is used to acquire images of the welded workpiece after welding. The image acquisition module can be a CCD camera or other imaging devices. The image acquisition module can use a high angle, such as 70 degrees, and a ring-shaped lighting method to acquire welding images, which ensures that the acquired welding images are clearer and the light and shadow are more balanced. Figure 2a , Figure 2b and Figure 2c As shown. The welded workpiece comprises at least two parts welded together. For example, it may include a stud and a flange. The welding image of the stud and flange welded together is acquired by the image acquisition module and sent to the communicator 101. The communicator 101 transmits the received welding image and preset welding position information to the processor 102. The processor 102 calls the first detection model to analyze and extract the welding position in the welding image, and finally determines whether there is a defect of positional misalignment of the welded workpiece.

[0084] The processor 102 serves as the control center of the welded workpiece defect detection device 100. It connects to various parts of the device via various interfaces and lines, and executes software programs and / or modules stored in the memory 103, as well as calling data stored in the memory, to perform various functions and / or process data of the welded workpiece defect detection device 100. The processor 102 can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 102 may only include a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0085] See Figure 3 This is a schematic flowchart illustrating a defect detection method for welded workpieces provided in an embodiment of this application. This method is applied to... Figure 1 In the aforementioned defect detection device for welded workpieces (hereinafter referred to as the defect detection device). For example... Figure 3 As shown, the defect detection method for this welded workpiece includes:

[0086] Step S301: Obtain the welding image of the workpiece and the preset welding position information of the workpiece.

[0087] In this embodiment, to ensure welding quality, defect detection is required at the welding position of the workpiece to determine whether defects such as welding position deviation, incomplete welding, or weld point climbing are present. During defect detection at the welding position of the workpiece, the detection device can acquire an image at the welding position to obtain a welding image. Alternatively, welding images at the welding position of the workpiece can be pre-acquired by other devices. In this case, the defect detection device can directly obtain the welding image from the other devices without needing to acquire it itself.

[0088] Since various welding parameters are preset during the welding process to ensure that welding is performed at the appropriate positions, and these parameters are pre-set based on actual welding requirements, the defect detection device for the welded workpiece can acquire the pre-set welding position information to detect any defects such as welding position deviation.

[0089] It should be understood that the defect detection device may obtain the pre-welding position information from its internal memory, from other external devices, or from user-input pre-welding position information; this application does not limit this.

[0090] Furthermore, the welding image of the workpiece is an image of the first workpiece and the second workpiece after welding. The first workpiece can be a flange, and the second workpiece can be a stud. That is, the welding workpiece includes a first workpiece and a second workpiece, the first workpiece being a flange and the second workpiece being a stud. The welding image is an image of the first and second workpieces after welding, that is, an image formed after the flange and the stud are welded together.

[0091] Step S302: Input the welding image of the welded workpiece into the pre-trained first detection model to obtain the outline of the welded workpiece in the welding image output by the first detection model.

[0092] The first detection model is pre-trained and is used to extract contour features from the input welding image and output a model of the contour of the welded workpiece in the welding image.

[0093] In this embodiment, to facilitate accurate and rapid determination of the position information of the welded workpiece in a welding image, a first detection model capable of outputting the contour of the welded workpiece in the welding image is pre-trained. Thus, after acquiring the welding image, the defect detection device can use the welding image as input to the first detection model. The first detection model performs welding feature extraction and stitching processing on the input welding image to obtain the contour of the welded workpiece in the welding image and outputs the contour of the welded workpiece in the welding image, such as... Figure 5aAs shown. After receiving the outline of the welded workpiece in the welding image output by the first detection model, the defect detection device can determine the position of the welded workpiece based on the outline of the welded workpiece in the welding image.

[0094] It should be understood that the welded workpiece includes a first workpiece and a second workpiece. In this case, the contour of the welded workpiece output by the first detection model can be the contour of the first workpiece, the contour of the second workpiece, or the contours of both the first and second workpieces, and can be set according to actual needs; this application does not impose any restrictions on this. Since the defect detection model can determine whether the welded workpiece in the welding image has a welding position offset defect by detecting the position of the first workpiece or the position of the second workpiece in the welding image, in order to reduce the training complexity of the first detection model, the contour of the welded workpiece output by the first detection model can be only the contour of the first workpiece or the contour of the second workpiece.

[0095] As one possible implementation, such as Figure 4 As shown, the first detection model can be trained in the following way:

[0096] Step S401: Obtain the first image.

[0097] In this embodiment, a first welding sample image can be acquired, and the user can pre-mark the contour information of the welded workpiece in the first welding sample image. After acquiring the first welding sample image, it is cropped, rotated, flipped, etc., to form more images, thereby forming a training sample set for the first preset network model. The first image is then identified from the training sample set. In this way, by cropping, rotating, flipping, etc., the first welding sample image can be used to form more images, which expands the training sample set of the first preset network model, improves the accuracy of the training of the first preset network model, reduces manual intervention in the formation of the training sample set of the first preset network model, improves the training efficiency of the first preset network model, and reduces its training cost.

[0098] Step S402: Input the first image into the first preset network model.

[0099] In this embodiment, a first preset network model needs to be trained to form a first detection model. At this time, the acquired first image can be input into the first preset network model.

[0100] Step S403: The first preset network model performs convolution and pooling processing on the input first image to extract feature information of each contour in the first image. Deconvolution and feature concatenation processing are then performed on the feature information of each contour in the first image to obtain first feature information. Based on the first feature information, a classifier is used to classify each pixel in the first image to obtain the contour of the welded workpiece in the first image.

[0101] In this embodiment, a first image is transmitted to a first preset network model. The first preset network model performs convolution and pooling processing on the first image to extract multi-channel feature information, forming feature information of each contour in the first image. The first preset network model then performs deconvolution processing on the feature information of each contour in the first image to form a deconvolution result. Since the size of the first image decreases after multiple pooling processes, while deconvolution increases the image size, the deconvolution process only increases the size and does not restore the original first image. Therefore, to reduce data loss, the convolution result after the previous convolution processing is usually cropped into an image of the same size after deconvolution and then directly stitched together to increase the image's feature information. That is, the deconvolution result and the convolution result after the previous convolution processing are combined to obtain the first feature information. The first feature information contains the contour information of the welded workpiece in the first image. A binary classification convolutional layer is used to classify the first feature information, outputting an image of the welded workpiece's contour and a background image.

[0102] Step S404: Obtain the welding workpiece outline marking information of the first image.

[0103] The welding workpiece outline marking information in the first image is the outline information of the welding workpiece in the first image that has been marked in advance.

[0104] Step S405: Using the first preset loss function, calculate the first loss value based on the outline of the welded workpiece output by the first preset network model and the outline marking information of the welded workpiece in the first image.

[0105] In this embodiment of the application, after the first preset network model outputs the welding position information of the welding workpiece in the first image, the first preset loss function can be used to calculate the difference between the welding position information of the welding workpiece in the first image and the welding workpiece contour mark information in the first image to obtain the first loss value.

[0106] Step S406: Determine whether the first loss value is greater than the first preset threshold. If the first loss value is greater than the first preset threshold, adjust the model parameters of the first preset network model and re-execute steps S401 to S406 until the first loss value is less than the first preset threshold to obtain the first detection model.

[0107] In this embodiment, when the first loss value is greater than the first preset threshold, it indicates that the welding position information of the welded workpiece in the first image output by the first preset network model has a large error. At this time, the model parameters of the first preset network model, such as convolution kernel, pooling kernel, and deconvolution kernel, can be adjusted to reduce the contour error of the welded workpiece in the first image output by the first preset network model. After adjusting the model parameters of the first preset network model, the above steps S401-S406 can be re-executed. By continuously adjusting the model parameters of the first preset network model, the output result of the first preset network model becomes more and more accurate until the first loss value is less than the first preset threshold. At this time, it can be considered that the training of the first preset network model is complete, and the first detection model is obtained.

[0108] It should be noted that the first preset network model is a pre-set neural network model, which can be a U-Net neural network model or other neural network models. This application does not limit this.

[0109] It should be noted that the first preset loss function is a pre-set function used to calculate the loss value. This function can be the cross-entropy loss function or other loss functions; this application does not impose any restrictions on this. The first preset threshold is a threshold set according to actual needs to measure whether the first preset network model has completed training. When the first loss value is greater than the first preset threshold, the output result of the first preset network model is considered to have a large error, and training of the first preset network model needs to continue. When the first loss value is less than the first preset threshold, the output result of the first preset network model is considered to be relatively accurate, and training of the first preset network model can be terminated at this time.

[0110] Through the above steps S401-S406, the first preset network model can be trained to obtain the first detection model. This first detection model can then be directly used for anomaly detection at the welding position of welded workpieces, significantly reducing detection complexity. Furthermore, this first detection model is applicable to various welded workpieces, improving the robustness of welding position anomaly detection.

[0111] As one possible implementation, the first detection model includes: a first encoder, a first decoder, and a first classifier. Among them,

[0112] The first encoder is used to extract contour feature information from the input welding image.

[0113] The first decoder is used to perform deconvolution and feature concatenation on the contour feature information extracted by the first encoder to obtain the first feature information.

[0114] The first classifier is used to perform convolution processing on the first feature information to obtain the outline of the welded workpiece.

[0115] In this embodiment, the first detection model includes a first encoder, a first decoder, and a first classifier. After a welding image is input into the first detection model, the first encoder performs multi-channel convolution and pooling processing on the input welding image using preset convolution and pooling kernels to extract feature information of each contour in the welding image. The first decoder performs deconvolution on the contour feature information extracted by the first encoder and concatenates it with the feature information extracted from the multi-channel welding image after convolution processing in the first encoder to obtain first feature information of the welding image size. This first feature information is then input into the first classifier, which classifies the welded workpiece and the background in the welding image and outputs the contour of the welded workpiece in the welding image.

[0116] In other words, the first detection model, through the cooperation of different devices, can extract welding information from multiple channels from the welding image input to it, and obtain the contour of the welded workpiece in the welding image. This enables the first detection model to accurately extract the contour of each welded workpiece in the welding image, thereby improving the accuracy of subsequent defect detection of the welded workpiece.

[0117] Step S303: Based on the outline of the welded workpiece and the preset welding position information of the welded workpiece output by the first detection model, determine whether the welding position of the welded workpiece is abnormal.

[0118] In the application embodiment, after the defect detection device outputs the outline of the welded workpiece in the welding image of the first detection model, it can locate the welding position of the welded workpiece according to the outline of the welded workpiece. Then, based on the welding position of the welded workpiece and the preset welding position information of the welded workpiece, it can determine whether the welding position of the welded workpiece is abnormal, that is, whether an offset has occurred.

[0119] As one possible implementation, determining whether the welding position of the welded workpiece is abnormal based on the outline of the welded workpiece output by the first detection model and the preset welding position information of the welded workpiece includes:

[0120] Based on the outline of the welded workpiece output by the first detection model, the welding position of the welded workpiece in the welding image is determined; based on the welding position of the welded workpiece in the welding image and the preset welding position information of the welded workpiece, the interval distance between the welding position of the welded workpiece in the welding image and the preset welding position of the welded workpiece is calculated; a preset distance threshold is obtained, and it is determined whether the interval distance is greater than the preset distance threshold; if the interval distance is greater than the preset distance threshold, it is determined that the welding position of the welded workpiece is abnormal; otherwise, the welding position of the welded workpiece is not abnormal.

[0121] In other words, after receiving the outline of the welded workpiece in the welding image output by the first detection model, the defect detection device can determine the welding position of the welded workpiece based on the position of the outline in the welding image. For example, the welding position of the welded workpiece can be determined based on the pixel position of the welded workpiece in the welding image. After determining the welding position of the welded workpiece, the welding position of the welded workpiece can be compared with the preset welding position information of the welded workpiece to calculate the offset distance, that is, to calculate the interval distance between the welding position of the welded workpiece in the welding image and the preset welding position of the welded workpiece; obtain the preset distance threshold, and compare the calculated interval distance with the preset distance threshold to determine whether the calculated interval distance is greater than the preset distance threshold. If the calculated interval distance is greater than the preset distance threshold, it is considered that the welding position of the welded workpiece has deviated from the preset welding position when the welding machine is welding the workpiece, and at this time, it can be determined that the welding position of the welded workpiece is abnormal. Figure 5b As shown.

[0122] Alternatively, if the calculated interval distance is not greater than the preset distance threshold, it is considered that the welding position of the welding machine is not offset from the preset welding position when welding the workpiece, and it can be determined that the welding position of the workpiece is not abnormal.

[0123] It should be noted that the preset distance threshold is set in advance according to actual needs and is used to measure whether the welding position of the workpiece being welded by the welding machine has shifted. That is, if the interval between the welding position of the workpiece and the preset welding position is greater than the preset distance threshold, it is considered that the welding position of the workpiece has shifted relative to the preset welding position, and the welding position of the workpiece can be determined to be abnormal. If the interval between the welding position of the workpiece and the preset welding position is not greater than the preset distance threshold, it is considered that the welding position of the workpiece has not shifted relative to the preset welding position, and the welding position of the workpiece can be determined to be normal. Figure 5c As shown.

[0124] In other words, the defect detection device can determine the welding position of the workpiece in the welding image by using the welding contour output by the first detection model. By comparing the determined welding position of the workpiece with the preset welding position information of the workpiece, it can detect whether the welding position of the workpiece has deviated from the preset welding position. The detection result is more accurate and can reduce detection time and increase detection cost.

[0125] Step S304: If an abnormality is found, it is determined that there is a defect in the positional deviation of the welded workpiece.

[0126] In this embodiment of the application, when the defect detection device determines that the welding position of the welded workpiece is abnormal, it can determine that the welded workpiece has a welding workpiece positional deviation defect.

[0127] For different welded workpieces, the first detection model can output the welding position information of the workpiece, and then compare it with the preset welding position to determine whether the welding position of the workpiece has shifted, which enhances robustness. Furthermore, no manual intervention is required in the process of detecting whether the welding position of the workpiece has shifted, thus improving detection efficiency.

[0128] See Figure 6 This is a schematic flowchart illustrating another defect detection method for welded workpieces provided in an embodiment of this application. This method is applied to... Figure 1 The aforementioned defect detection device. Compared to the above... Figure 3 The defect detection method shown in this application embodiment adds detection steps for other welding defects. For example... Figure 6 As shown, the method includes:

[0129] Step S601: Obtain the welding image of the workpiece and the preset welding position information of the workpiece.

[0130] For details, please refer to step S301, which will not be repeated here.

[0131] Step S602: Input the welding image of the welded workpiece into the pre-trained first detection model to obtain the outline of the welded workpiece in the welding image output by the first detection model.

[0132] The first detection model is pre-trained and is used to extract contour features from the input welding image and output a model of the contour of the welded workpiece in the welding image.

[0133] For details, please refer to step S302, which will not be repeated here.

[0134] Step S603: Based on the outline of the welded workpiece and the preset welding position information of the welded workpiece output by the first detection model, determine whether the welding position of the welded workpiece is abnormal.

[0135] For details, please refer to step S303, which will not be repeated here.

[0136] It should be noted that the steps performed by the defect detection device will differ depending on the determined results. When it is determined that the welding position of the workpiece is abnormal, step S604 is performed. When it is determined that the welding position of the workpiece is not abnormal, step S605 is performed.

[0137] Step S604: If an abnormality is found, it is determined that there is a defect in the positional deviation of the welded workpiece.

[0138] For details, please refer to step S304, which will not be repeated here.

[0139] Step S605: If there is no abnormality in the welding position of the workpiece, then obtain the target welding image.

[0140] The target welding image is the welding image at the welding position of the workpiece.

[0141] In this embodiment, when the defect detection device determines that the welding position of the welded workpiece is normal, it indicates that the welding position of the welded workpiece has not shifted. At this point, it can further detect whether there are other welding defects in the welded workpiece. The defect detection device can acquire a target welding image of the welded workpiece, that is, acquire an image of the welding position of the welded workpiece.

[0142] As one possible implementation, obtaining the target welding image includes: cutting the welding position area of ​​the welding workpiece in the welding image according to the contour of the welding workpiece to obtain the target welding image.

[0143] That is, in step S602 above, the outline of the welded workpiece in the acquired welding image is output. At this time, the welding position of the welded workpiece can be determined based on the output outline of the welded workpiece. Then, the welding position area of ​​the welded workpiece in the welding image can be cut, other content contained in the welding image is deleted, and only the part of the welded workpiece at the welding position is retained to obtain the target welding image. (Refer to...) Figure 7a As shown.

[0144] In this way, the detection device cuts the welding position area of ​​the welding workpiece in the welding image by using the outline of the welding workpiece output by the first detection model to obtain the target welding image, which is a part of the welding position area of ​​the welding workpiece. This can reduce the complexity of the target welding image and thus improve the accuracy of detecting other defects of the welding workpiece in the welding image.

[0145] Step S606: Input the target welding image into the pre-trained second detection model to obtain the welding information of the welded workpiece in the target welding image output by the second detection model.

[0146] The second detection model is pre-trained and is used to perform semantic segmentation on the input target welding image, extract multi-channel welding feature information, and output the welding information of the welded workpiece in the target welding image.

[0147] In this embodiment of the application, in order to more conveniently and accurately detect welding defects in the welded workpiece, a model that can output welding information of the welded workpiece in the input welding image can be pre-trained. The defect detection device can input the acquired target welding image into the second detection model, and the second detection model extracts the welding contour of the target welding image and outputs the welding information of the welded workpiece.

[0148] It should be understood that welding information is information that can characterize the welding status of each welded workpiece.

[0149] As one possible implementation, the welding information includes at least one of the following: weld point contour, first workpiece contour, second workpiece contour, weld point center, and defect point. This allows for the extraction of different welding feature information from the target welding image to obtain the welding information of the welded workpiece. Consequently, when using this welding information to perform defect detection on the welded workpiece, it is possible to detect different welding defects, thereby improving the accuracy of defect detection.

[0150] As one possible implementation, the first workpiece can be a flange, and the second workpiece can be a stud. The welding information then includes at least one of the following: weld point profile, flange profile, stud profile, weld point center, and defect point. (Refer to...) Figure 7b As shown.

[0151] As one possible implementation, such as Figure 8 As shown, the second detection model can be trained in the following way:

[0152] Step S801: Obtain the second image.

[0153] In this embodiment, a second welding sample image can be acquired. The user pre-marks the welding information of the workpiece in the second welding sample image, for example, marking at least one of the following: weld point outline, flange outline, stud outline, weld center, and defect point. After acquiring the second welding sample image, it is cropped, rotated, flipped, etc., to form more images, thus forming a training sample set for a second preset network. The second image is then determined from the training sample set.

[0154] Step S802: Input the second image into the second preset network model.

[0155] In this embodiment, a second preset network model needs to be trained to form a second detection model. At this time, the acquired second image can be input into the second preset network model.

[0156] Step S803: The second preset network model performs convolution and pooling processing on the input second image to extract feature information of each welded workpiece in the second image. Deconvolution and feature concatenation processing are then performed on the feature information of each welded workpiece in the second image to obtain second feature information. Based on the second feature information, a classifier is used to classify each pixel in the second image to obtain the welding information in the second image.

[0157] In this embodiment, the second image is transmitted to a second preset network model. The second preset network model performs convolution and pooling processing on the second image to extract multi-channel feature information, such as weld point feature information, flange contour feature information, stud contour feature information, and weld point center feature information, forming feature information of the welded workpiece in the second image. The second preset network model performs deconvolution processing on the feature information of the welded workpiece in the second image to form a deconvolution result. Since the size of the second image decreases after multiple pooling processes, while deconvolution increases the image size, the deconvolution process only increases the size and does not restore the original second image. Therefore, to reduce data loss, the convolution result after the previous convolution processing is usually cropped into an image of the same size after deconvolution and then directly stitched together to increase the image's feature information. That is, the deconvolution result and the convolution result after convolution processing are combined to obtain the second feature information. The second feature information contains the feature information of the welded workpiece in the second image. A classification convolutional layer is used to classify the second feature information and output the welding information of the welded workpiece. The classification convolutional layer can be a single layer or multiple layers. When there are multiple layers, they can be categorized as follows: flange profile classification convolutional layer, stud profile classification convolutional layer, weld point classification convolutional layer, notch classification convolutional layer, and weld point center classification convolutional layer. After the second feature information is transmitted to the classification convolutional layer, it is classified to form welding information containing different welding features.

[0158] Step S804: Obtain the welding mark information of the welding workpiece in the second image.

[0159] The welding mark information of the welding workpiece in the second image is the welding information of the welding workpiece in the pre-marked second image.

[0160] Step S805: Using the second preset loss function, calculate the second loss value based on the welding information of the welded workpiece output by the second preset network model and the welding mark information of the welded workpiece in the second image.

[0161] In this embodiment of the application, after the welding information of the welding workpiece in the second image is output by the second preset network model, the difference between the welding information of the welding workpiece in the second image and the welding mark information of the welding workpiece in the second image can be calculated using the second preset loss function to obtain the second loss value.

[0162] Step S806: Determine whether the second loss value is greater than the second preset threshold; if the second loss value is greater than the second preset threshold, adjust the model parameters of the second preset network model and re-execute steps S801 to S806 until the second loss value is less than the second preset threshold to obtain the second detection model.

[0163] In this embodiment, when the second loss value is greater than the second preset threshold, it indicates that the welding information of the welded workpiece in the second image output by the second preset network model has a large error. At this time, the model parameters of the second preset network model can be adjusted, such as convolution kernel, pooling kernel, deconvolution kernel, etc., to reduce the error of the welding information of the welded workpiece in the second image output by the second preset network model. After adjusting the model parameters of the second preset network model, the above steps S801-S806 can be re-executed. By continuously adjusting the model parameters of the second preset network model, the output result of the second preset network model becomes more and more accurate until the second loss value is less than the second preset threshold. At this time, it can be considered that the training of the second preset network model is complete, and the second detection model is obtained.

[0164] It should be noted that the second preset network model is a pre-set neural network model, which can be a U-Net neural network model or other neural network models. This application does not limit this.

[0165] It should be noted that the second preset loss function is a pre-set function used to calculate the loss value. This function can be the cross-entropy loss function or other loss functions; this application does not impose any restrictions on this. The second preset threshold is a threshold set according to actual needs to measure whether the second preset network model has completed training. When the second loss value is greater than the second preset threshold, the output result of the second preset network model is considered to have a large error, and training of the second preset network model needs to continue. When the second loss value is less than the second preset threshold, the output result of the second preset network model is considered to be relatively accurate, and training of the second preset network model can be terminated at this time.

[0166] Through the above steps S801-S806, the training of the second preset network model can be completed to obtain the second detection model. Therefore, when a target welding image is input into the second detection model, the model can extract multi-channel welding feature information from the target welding image and output the welding information of the welded workpiece in the target welding image, improving the accuracy of the welding information. Furthermore, when using the welding information of the welded workpiece to detect whether there are other welding defects, the accuracy of defect detection can also be improved, and no manual intervention is required, thus increasing detection time and reducing detection costs.

[0167] As one possible implementation, the second detection model includes: a second encoder, a second decoder, and a second classifier. Among them,

[0168] The second encoder is used to perform semantic segmentation on the input target welding image and extract multi-channel welding feature information.

[0169] The second decoder is used to upsample and fuse the welding feature information of multiple channels to obtain the second feature information.

[0170] A second classifier is used to classify the second feature information to obtain the welding information of the welded workpiece.

[0171] In this embodiment, the second detection model includes a second encoder, a second decoder, and a second classifier. After the target welding image is input into the second detection model, the second encoder performs multi-channel convolution and pooling processing on the input target welding image using preset convolution and pooling kernels to extract feature information of the welded workpiece in the target welding image. The second decoder performs deconvolution on the feature information of the welded workpiece extracted by the second encoder and concatenates it with the feature information of the multi-channel target welding image extracted after convolution processing in the second encoder to obtain second feature information of the target welding image size. This second feature information is then input to the second classifier, which classifies each welding feature of the welded workpiece in the target welding image and outputs the welding information of the welded workpiece in the target welding image.

[0172] In other words, the second detection model, through the cooperation of different devices, can extract welding information from multiple channels of the target welding image input to it, thereby obtaining the welding feature information of the welded workpiece in the welding image. This enables the second detection model to accurately extract the welding information of the welded workpiece in the welding image, improving the accuracy of subsequent defect detection of the welded workpiece.

[0173] Step S607: Based on the welding information of the workpiece, determine whether there are welding defects in the workpiece.

[0174] In this embodiment of the application, after obtaining the welding information of the welded workpiece, the defect detection device can detect whether the welded workpiece has defects such as missing welds, missing parts, missing welds, weld misalignment, weld climbing, flange angle defects, stud deformation, and workpiece gaps.

[0175] Among these steps, determining whether a welding defect exists in the welded workpiece based on its welding information includes:

[0176] When the welding information includes the center of the weld point, determine whether there are defects such as missing welds, missing parts, or missing weld points based on the center of the weld point; or,

[0177] When the welding information includes the first workpiece outline and the weld point outline, determine whether there is a defect of weld point misalignment based on the first workpiece outline and the weld point outline; or,

[0178] When the welding information includes both the weld joint outline and the second workpiece outline, determine whether a weld joint climbing defect exists based on the second workpiece outline and the weld joint outline; or,

[0179] When the welding information includes a first workpiece profile, the first workpiece angle is determined based on the first workpiece profile, and the presence of a defect in the first workpiece angle is determined based on the first workpiece angle; or,

[0180] When the welding information includes the contour of the second workpiece, check whether the contour of the second workpiece conforms to preset rules to determine whether the second workpiece has deformation defects; or,

[0181] When the welding information includes the center of the weld joint and the defect point, determine whether there is a gap defect based on the center of the weld joint and the defect point.

[0182] In this embodiment, when the welding information includes weld point centers, the defect detection device can detect whether there are missing welds or missing parts by detecting the presence of weld point centers. That is, if the welding information obtained by the defect detection device does not include weld point centers, then it can be determined that there are missing welds or missing parts. When the welding information includes weld point centers, the defect detection device can determine the number of weld points on the welded workpiece by detecting the number of weld point centers, and thus determine whether there is a defect of insufficient weld points. In this case, the defect detection device needs to first obtain preset welding parameters. The welding parameters include a preset number of weld points. The defect detection device can determine whether the number of weld points on the welded workpiece is less than the preset number of weld points; if it is less, then it can determine that there is a defect of insufficient weld points.

[0183] When the welding information includes the outline of a first workpiece and the outline of a weld point, the defect detection device can detect whether the weld point outline exceeds the outline of the first workpiece. If it exceeds the outline of the first workpiece, a defect of weld point misalignment is determined. Figure 9aAs shown. In one possible implementation, the weld point information output in the second detection model is an image. That is, the first workpiece contour is an image of the first workpiece contour, and the weld point contour is an image of the weld point contour. In this case, the defect detection device can determine the position of the pixels of the first workpiece contour based on the image of the first workpiece contour. The defect detection device can also detect the position of the pixels of the weld point contour based on the image of the weld point contour, and further detect whether the position of the pixels of the weld point contour exceeds the position of the pixels of the first workpiece contour. If it exceeds, it can be determined that there is a defect of weld point misalignment.

[0184] When the welding information includes both the weld joint outline and the outline of a second workpiece, the defect detection device can detect whether the outline of the second workpiece exceeds the weld joint outline. If it does, it determines that there is a defect of weld joint climbing over the weld joint. Figure 9b As shown. In one possible implementation, the weld point information output in the second detection model is an image. That is, the second workpiece contour is an image of the second workpiece contour, and the weld point contour is an image of the weld point contour. In this case, the defect detection device can determine the pixel positions of the second workpiece contour based on the image of the second workpiece contour. The defect detection device can also detect the pixel positions of the weld point contour based on the image of the weld point contour, and further detect whether the pixel positions of the second workpiece contour exceed the pixel positions of the weld point contour. If they exceed, it can be determined that there is a defect of weld point climbing over the wall.

[0185] When the welding information includes the outline of the first workpiece, the defect detection device can determine the position of the first workpiece outline based on the first workpiece outline, and then determine the angle of the first workpiece, and obtain the preset welding parameters. The welding parameters include the angle threshold of the first workpiece. When the calculated angle of the first workpiece exceeds the angle threshold of the first workpiece, it can be determined that there is a defect in the angle of the first workpiece.

[0186] As one possible implementation, determining the first workpiece angle based on the first workpiece profile includes: determining the position of the first workpiece profile in a first coordinate system based on the first workpiece profile; and determining the first workpiece angle based on the position of the first workpiece profile in the first coordinate system.

[0187] That is, when determining the angle of the first workpiece, the defect detection device can first determine the position of the first workpiece contour in the first coordinate system. For example, when the welding information is image information, the first workpiece contour is an image, and the coordinates of each pixel point of the edge contour of the first workpiece contour in the image can be determined in the first coordinate system. At this time, the welding information can also include the image of the edge of the first workpiece contour, for reference. Figure 7bAs shown. Furthermore, based on the position of the first workpiece contour's edge in the first coordinate system, the straight line with the most points passing through its contour edge in the first coordinate system can be determined, and the angle of the first workpiece can be determined based on this straight line. The defect detection device can use the Hough line detection algorithm to determine the straight line with the most points passing through its contour edge in the first coordinate system based on the position of the first workpiece contour in the first coordinate system. Here, the first coordinate system can be a Cartesian coordinate system. A straight line in the Cartesian coordinate system corresponds to a point in Hough space, and collinear points in the Cartesian coordinate system intersect the corresponding straight lines in Hough space. Therefore, the defect detection device maps the position of the first workpiece contour in the first coordinate system to Hough space, thereby determining the most common intersection points of the straight lines corresponding to the first workpiece contour's edge in Hough space. That is, determining the common intersection points corresponding to the first workpiece contour is equivalent to determining the straight line with the most points passing through the first workpiece contour's edge in the Cartesian coordinate system, such as... Figure 9c As shown, the angle of the first workpiece can be determined based on this straight line.

[0188] When the welding information includes the outline of the second workpiece, the defect detection device can determine the position of the second workpiece outline based on the outline, and then determine whether the shape corresponding to the second workpiece outline is the shape in the preset rule. If not, it can be determined that there is a defect of deformation of the second workpiece.

[0189] As one possible implementation, detecting whether the second workpiece contour conforms to a preset rule includes: determining the position of the second workpiece contour in a second coordinate system based on the second workpiece contour; determining the shape corresponding to the second workpiece contour based on the position of the second workpiece contour; and determining whether the shape corresponding to the second workpiece contour conforms to the preset rule.

[0190] The shape corresponding to the second workpiece contour can be circular, as preset in the rules. The shape can also be elliptical, semi-circular, or other shapes; this application does not impose any restrictions. In this example, a circular shape is used as an example. When determining the contour shape of the second workpiece, the defect detection device can determine it based on the position of each point within the contour. The defect detection device can first determine the position of the second workpiece contour in the second coordinate system. For example, if the welding information is an image, then the second workpiece contour is an image, and the coordinates of each pixel on the edge of the second workpiece contour in the image can be determined in the second coordinate system. Then, based on the position of the edge of the second workpiece contour in the second coordinate system, the circle with the most points passing through the edge of the second workpiece contour in the second coordinate system can be determined, and this circle is identified as the circle corresponding to the second workpiece contour. The defect detection device can use the Hough circular detection algorithm to determine the circle with the most points passing through the edge of the second workpiece contour in the second coordinate system based on the position of the second workpiece contour in the second coordinate system. At this point, the second coordinate system can be a Cartesian coordinate system. A straight line in the Cartesian coordinate system corresponds to a point in the Hough circular space, and collinear points in the Cartesian coordinate system intersect with corresponding circles in the Hough circular space. Therefore, the defect detection device maps the position of the second workpiece contour edge in the second coordinate system to the Hough circular space, thereby determining the most common intersection points of the circles corresponding to the second workpiece contour edge in the Hough circular space. In other words, determining the common intersection points of the circles corresponding to the second workpiece contour edge is equivalent to determining the circle in the Cartesian coordinate system that passes through the most points on the second workpiece contour edge. Figure 9d As shown, the circle is determined to be the circle corresponding to the contour of the second workpiece, thereby determining whether the circle corresponding to the contour of the second workpiece conforms to the preset rules.

[0191] It should be noted that if the deformation of the second workpiece contour is severe and the defect detection device cannot detect the circle corresponding to the second workpiece contour using the Hough circular detection algorithm, then it can be directly determined that the welded workpiece has a defect of deformation of the second workpiece.

[0192] It should be noted that the preset rules are pre-set size thresholds for the shape corresponding to the outline of the second workpiece. Taking a circle as an example, the preset rules set a size threshold range for a standard circle. The rules then determine whether the circle corresponding to the outline of the second workpiece is within this threshold range. If the circle is not within this threshold range, it indicates that the welded workpiece has a deformation defect and is considered a defective product. If the circle is within this threshold range, it indicates that the welded workpiece does not have a deformation defect and is considered a good product.

[0193] When welding information includes weld center points and defect points, the defect detection device can calculate the defect area ratio based on the number of weld center points and defect points. For example, when the welding information is image information, the weld center point is the image of the weld center point, and the defect point is the image of the defect point. In this case, the defect detection device can calculate the defect area ratio based on the number of pixels in the weld center point and the number of pixels in the defect point. The defect detection device acquires preset welding parameters, which include a defect area threshold. The defect detection device can compare the calculated defect area ratio with the defect area threshold to determine whether the defect area ratio exceeds the defect area threshold. If it exceeds the threshold, a notch defect is identified.

[0194] In this embodiment, the defect detection device detects different welding defects in the welded workpiece by utilizing the different contents contained in the welding information, thereby improving the efficiency of defect detection. Furthermore, this process requires no manual intervention, which improves the accuracy of defect detection and reduces detection costs.

[0195] Step S608: When it is determined that there is a welding defect in the welded workpiece, display information of the welding defect of the welded workpiece is generated, so as to display the concrete image or characteristic value of the welding defect according to the display information.

[0196] In this embodiment of the application, in order to facilitate the staff to know the welding defects in the welded workpiece, when the defect detection device detects the welding defects in the welded workpiece, it can generate display information of the welding defects of the welded workpiece based on the detected welding defects. This information can be text display information or image display information, and the display information is displayed on the display device to display the concrete image or feature value of the welding defect.

[0197] This application provides a defect detection device 100 for welded workpieces, referencing... Figure 1 As shown. The defect detection device 100 includes:

[0198] The communicator 101 is used to receive welding images of the workpiece from an image acquisition module and preset welding position information of the workpiece from an input device.

[0199] The processor 102, coupled to the communicator 101, is used to input a welding image of the welded workpiece into a pre-trained first detection model to obtain the outline of the welded workpiece in the welding image output by the first detection model. Based on the outline of the welded workpiece output by the first detection model and the preset welding position information of the welded workpiece, it is determined whether the welding position of the welded workpiece is abnormal. If abnormal, it is determined that there is a positional deviation defect in the welded workpiece.

[0200] The first detection model is pre-trained and is used to extract contour features from the input welding image and output a model of the contour of the welded workpiece in the welding image.

[0201] As one possible implementation, the processor 102 is specifically used to determine the welding position of the welding workpiece in the welding image based on the outline of the welding workpiece output by the first detection model; calculate the interval distance between the welding position of the welding workpiece in the welding image and the preset welding position information of the welding workpiece based on the welding position of the welding workpiece in the welding image and the preset welding position information of the welding workpiece; obtain a preset distance threshold and determine whether the interval distance is greater than the preset distance threshold; if the interval distance is greater than the preset distance threshold, then determine that the welding position of the welding workpiece is abnormal.

[0202] As one possible implementation, the first detection model includes: a first encoder, a first decoder, and a first classifier.

[0203] The first encoder is used to extract contour feature information from the input welding image.

[0204] The first decoder is used to perform deconvolution and feature concatenation on the contour feature information extracted by the first encoder to obtain the first feature information.

[0205] The first classifier is used to classify each pixel in the welding image based on the first feature information to obtain the welding position information of the welded workpiece.

[0206] As one possible implementation, the processor 102 is also used to acquire a target welding image if there is no abnormality in the welding position of the welded workpiece; input the target welding image into a pre-trained second detection model to obtain the welding information of the welded workpiece in the target welding image output by the second detection model; and determine whether there is a welding defect in the welded workpiece based on the welding information of the welded workpiece.

[0207] The target welding image is the welding image at the welding position of the workpiece; the second detection model is pre-trained and is used to perform semantic segmentation on the input target welding image, extract multi-channel welding feature information, and output the welding information of the workpiece in the target welding image.

[0208] As one possible implementation, the second detection model includes: a second encoder, a second decoder, and a second classifier.

[0209] The second encoder is used to perform semantic segmentation on the input target welding image and extract multi-channel welding feature information.

[0210] The second decoder is used to upsample and fuse the welding feature information of multiple channels to obtain the second feature information.

[0211] The second classifier is used to classify the second feature information to obtain the welding information of the welded workpiece.

[0212] As one possible implementation, the welding information includes at least one of the following: weld joint profile, first workpiece profile, second workpiece profile, weld joint center, and defect point.

[0213] At this time, the processor 102 is specifically used to determine whether there are defects such as missing solder, missing parts, or insufficient solder joints when the welding information includes the center of the solder joint. Alternatively, when the welding information includes the outline of a first workpiece and the outline of the solder joint, it determines whether there are defects such as solder joint misalignment based on the outline of the first workpiece and the outline of the solder joint. Alternatively, when the welding information includes the outline of the solder joint and the outline of a second workpiece, it determines whether there are defects such as solder joint climbing over the wall based on the outline of the second workpiece and the outline of the solder joint. Alternatively, when the welding information includes the outline of a first workpiece, it determines the angle of a first workpiece based on the outline of the first workpiece and determines whether there are defects such as the angle of the first workpiece. Alternatively, when the welding information includes the outline of a second workpiece, it detects whether the outline of the second workpiece conforms to a preset rule and determines whether there are defects such as deformation in the second workpiece. Alternatively, when the welding information includes the center of the solder joint and the defect point, it determines whether there are defects such as gaps based on the center of the solder joint and the defect point.

[0214] As one possible implementation, determining the first workpiece angle based on the first workpiece profile includes: determining the position of the first workpiece profile in a first coordinate system based on the first workpiece profile; and determining the first workpiece angle based on the position of the first workpiece profile in the first coordinate system.

[0215] As one possible implementation, detecting whether the contour of the second workpiece conforms to a preset rule includes:

[0216] Based on the second workpiece contour, determine the position of the second workpiece contour in the second coordinate system; based on the position of the second workpiece contour, determine the shape corresponding to the second workpiece contour; and determine whether the shape corresponding to the second workpiece contour conforms to the preset rules.

[0217] As one possible implementation, the processor 102 is specifically used to cut the welding position area of ​​the welding workpiece in the welding image according to the contour of the welding workpiece to obtain the target welding image.

[0218] As one possible implementation, the processor 102 is also used to generate display information of the welding defects of the welded workpiece when it is determined that there are welding defects in the welded workpiece, so as to display the concrete image or characteristic value of the welding defects according to the display information.

[0219] As one possible implementation, the defect detection device 100 described above also includes a memory 103, such as... Figure 10As shown.

[0220] The memory 103 is used to store various types of data in the defect detection device 100 for welded workpieces, such as various databases, program code, etc. In the embodiments of this application, the memory 103 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Electrically Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk, etc.

[0221] Corresponding to the above embodiments, this application also provides a computer-readable storage medium. This computer storage medium may store a program, which, when executed, may include some or all of the steps in the various embodiments of the defect detection method provided by this invention. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0222] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0223] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments and terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

Claims

1. A method of defect detection of a welded workpiece, characterized by, The method comprises the following steps: obtaining a welding image of a welding workpiece and preset welding position information of the welding workpiece; inputting the welding image of the welding workpiece into a pre-trained first detection model to obtain a contour of the welding workpiece in the welding image output by the first detection model; the first detection model is pre-trained and is used for extracting contour features of an input welding image and outputting a contour of a welding workpiece in the welding image; determining whether the welding position of the welding workpiece is abnormal according to the contour of the welding workpiece output by the first detection model and the preset welding position information of the welding workpiece; if abnormal, it is determined that there is a position offset defect of the welding workpiece; if the welding position of the welding workpiece is normal, a target welding image is obtained; the target welding image is a welding image at the welding position of the welding workpiece; inputting the target welding image into a pre-trained second detection model to obtain welding information of the welding workpiece in the target welding image output by the second detection model; the second detection model is pre-trained and is used for performing semantic segmentation on an input target welding image, extracting multi-channel welding feature information, and outputting welding information of the welding workpiece in the target welding image; determining whether the welding workpiece has a welding defect according to the welding information of the welding workpiece.

2. The method of claim 1, wherein, The step of determining whether the welding position of the welding workpiece is abnormal according to the contour of the welding workpiece output by the first detection model and the preset welding position information of the welding workpiece comprises the following steps: determining the welding position of the welding workpiece in the welding image according to the contour of the welding workpiece output by the first detection model; calculating a distance between the welding position of the welding workpiece in the welding image and the preset welding position of the welding workpiece according to the welding position of the welding workpiece in the welding image and the preset welding position information of the welding workpiece; obtaining a preset distance threshold and determining whether the distance is greater than the preset distance threshold; if the distance is greater than the preset distance threshold, it is determined that the welding position of the welding workpiece is abnormal.

3. The method of claim 1, wherein, The first detection model comprises a first encoder, a first decoder and a first classifier. The first encoder is used for extracting contour feature information in an input welding image. The first decoder is used for deconvolution and feature splicing of the contour feature information extracted by the first encoder to obtain first feature information. The first classifier is used for classifying each pixel in the welding image based on the first feature information to obtain welding position information of the welding workpiece.

4. The method of claim 1, wherein, The second detection model comprises a second encoder, a second decoder and a second classifier. The second encoder is used for performing semantic segmentation on an input target welding image to extract multi-channel welding feature information. The second decoder is used for up-sampling and feature fusion of the multi-channel welding feature information respectively to obtain second feature information. The second classifier is used for classifying the second feature information to obtain welding information of the welding workpiece.

5. The method according to claim 1 or 4, characterized in that, The welding information includes at least one of a welding spot contour, a first workpiece contour, a second workpiece contour, a welding spot center, and a defect point.

6. The method of claim 5, wherein, The determining whether the welding workpiece has a welding defect according to the welding information of the welding workpiece includes: When the welding information includes the welding spot center, determining whether there is a defect of a missing welding spot, a missing welding point, or a short welding spot according to the welding spot center; or, When the welding information includes the first workpiece contour and the welding spot contour, determining whether there is a defect of a welding spot deviation according to the first workpiece contour and the welding spot contour; or, When the welding information includes the welding spot contour and the second workpiece contour, determining whether there is a defect of a welding spot wall climbing according to the second workpiece contour and the welding spot contour; or, When the welding information includes the first workpiece contour, determining a first workpiece angle according to the first workpiece contour, and determining whether there is a defect of the first workpiece angle according to the first workpiece angle; or, When the welding information includes the second workpiece contour, detecting whether the second workpiece contour conforms to a preset rule, and determining whether there is a defect of a deformation of the second workpiece; or, When the welding information includes the welding spot center and the defect point, determining whether there is a defect of a gap according to the welding spot center and the defect point.

7. The method of claim 6, wherein, The determining the first workpiece angle according to the first workpiece contour includes: Determining a position of the first workpiece contour in a first coordinate system according to the first workpiece contour; Determining the first workpiece angle according to the position of the first workpiece contour in the first coordinate system.

8. The method of claim 6, wherein, The detecting whether the second workpiece contour conforms to the preset rule includes: Determining a position of the second workpiece contour in a second coordinate system according to the second workpiece contour; Determining a shape corresponding to the second workpiece contour according to the position of the second workpiece contour; Determining whether the shape corresponding to the second workpiece contour conforms to the preset rule.

9. The method of claim 1, wherein, The obtaining the target welding image includes: Cutting a welding position area of the welding workpiece in the welding image according to a contour of the welding workpiece to obtain the target welding image.

10. The method of claim 1 or 6, wherein, Further includes: When it is determined that the welding workpiece has a welding defect, forming display information of the welding defect of the welding workpiece to display a concrete or characteristic value of the welding defect according to the display information.

11. A defect detection apparatus for welding workpieces, characterized by, Further includes: A communicator configured to receive a welding image of a welding workpiece from an image acquisition module and preset welding position information of the welding workpiece from an input device; A processor coupled to the communicator and configured to input the welding image of the welding workpiece into a first detection model trained in advance to obtain a contour of the welding workpiece output by the first detection model; the first detection model is a model trained in advance and configured to extract a contour feature of an input welding image and output a contour of a welding workpiece in the welding image; Determine whether a welding position of the welding workpiece is abnormal according to the contour of the welding workpiece output by the first detection model and the preset welding position information of the welding workpiece; If it is abnormal, it is determined that there is a welding workpiece position degree deviation defect. If the welding position of the welding workpiece is normal, a target welding image is acquired; the target welding image is a welding image at the welding position of the welding workpiece; The target welding image is input into a second detection model to obtain welding information of the welding workpiece in the target welding image output by the second detection model; the second detection model is pre-trained and used for semantic segmentation of the input target welding image, extraction of multi-channel welding feature information, and output of the welding information of the welding workpiece in the target welding image; According to the welding information of the welding workpiece, it is determined whether the welding workpiece has a welding defect.

12. The apparatus for detecting defects of a welded workpiece according to claim 11, wherein, The first detection model comprises a first encoder, a first decoder and a first classifier. The first encoder is configured to extract contour feature information in the input welding image. The first decoder is configured to perform deconvolution and feature splicing on the contour feature information extracted by the first encoder to obtain first feature information. The first classifier is configured to perform classification processing on each pixel in the welding image based on the first feature information to obtain the welding position information of the welding workpiece.

13. The apparatus for detecting defects of a welded workpiece according to claim 11, wherein, The second detection model comprises a second encoder, a second decoder and a second classifier. The second encoder is configured to perform semantic segmentation on the input target welding image to extract multi-channel welding feature information. The second decoder is configured to perform upsampling and feature fusion on the multi-channel welding feature information respectively to obtain second feature information. The second classifier is configured to perform classification processing on the second feature information to obtain the welding information of the welding workpiece. The welding information comprises at least one of a welding point contour, a first workpiece contour, a second workpiece contour, a welding point center and a defect point.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the program controls the device where the computer readable storage medium is located to perform the method of any one of claims 1-10 when the program is running.

Images