Workpiece welding quality detection method, device, electronic equipment and storage medium

Abstract

The present application discloses a method, device, electronic device and storage medium for detecting the welding quality of a workpiece, the method comprising: obtaining an image to be welded of the current workstation before the workpiece is welded; performing regional segmentation on the image to be welded, and taking the welding area of ​​the workpiece as the region of interest of the image to be welded; extracting feature data of the region of interest of the image to be welded as feature data of the image to be welded; performing quality classification on the feature data of the image to be welded, and if the image to be welded is in an abnormal state, compressing the feature data of the image to be welded, and obtaining sensitive data of target image attributes corresponding to the image to be welded; based on the current parameter value of the target image attribute corresponding to the sensitive data of the target image attribute not being within the ideal range of the target parameter, obtaining a camera photographing parameter adjustment value according to the deviation between the current parameter value and the ideal range of the target parameter, and using it to adjust the camera photographing quality. The present application improves the detection accuracy of welding quality.

Description

Technical Field

[0001] The present application relates to the field of industrial production technology, and more specifically, to a method and device for detecting the welding quality of a workpiece, as well as electronic equipment and a storage medium. Background Art

[0002] In the existing solder joint quality inspection, the quality of the camera is directly related to the accuracy and reliability of the inspection results. Due to factors such as the complex environment, light changes, and reflections on the workpiece surface during the welding process, the images captured by the camera may be blurred, overexposed, or underexposed, which will interfere with the identification and quality assessment of the solder joints. It can be seen that in the relevant technology, the accuracy of welding quality inspection is low.

[0003] Therefore, how to improve the detection accuracy of welding quality is a technical problem that technical personnel in this field need to solve. Summary of the invention

[0004] The purpose of the present application is to provide a method and device for detecting the welding quality of a workpiece, an electronic device and a computer-readable storage medium, thereby improving the detection accuracy of the welding quality.

[0005] To achieve the above objectives, the present application provides a method for detecting the welding quality of a workpiece, comprising:

[0006] Obtain the image of the workpiece to be welded before welding at the current workstation;

[0007] Performing regional segmentation on the image to be welded, and taking the welding area of ​​the workpiece as the region of interest of the image to be welded;

[0008] Extracting feature data of a region of interest of the image to be welded as feature data of the image to be welded;

[0009] Performing quality classification on the feature data of the image to be welded to determine the current state of the image to be welded;

[0010] If the image to be welded is in an abnormal state, compressing the feature data of the image to be welded to obtain sensitive data of target image attributes corresponding to the image to be welded;

[0011] Based on the fact that the current parameter value of the target image attribute corresponding to the target image attribute sensitive data is not within the ideal range of the target parameter, a camera shooting parameter adjustment value is obtained according to the deviation between the current parameter value and the ideal range of the target parameter.

[0012] Wherein, the method further comprises:

[0013] Acquire multiple historical images to be welded of the current workstation;

[0014] Adjusting the parameter value of the target graphic attribute corresponding to each of the historical images to be welded, and obtaining multi-dimensional feature compression data of each of the historical images to be welded under multiple parameters of the target graphic attribute;

[0015] According to the feature compression data corresponding to the historical image to be welded, a sensitive relationship between the feature compression data of each dimension and the target graphic attribute is obtained, and the feature compression data dimension that is most sensitive to the target graphic attribute is used as the target dimension;

[0016] Extracting feature compression data of each of the historical images to be welded at a target dimension;

[0017] The ideal range of the target parameter is obtained according to the parameter value of the target graphic attribute corresponding to the feature compression data of each of the historical images to be welded in the target dimension.

[0018] The step of compressing the feature data of the image to be welded to obtain the target image attribute sensitive data corresponding to the image to be welded includes:

[0019] Acquire feature compression data of the feature data of the image to be welded in the target dimension, and use the feature compression data of the feature data of the image to be welded in the target dimension as the target image attribute sensitive data corresponding to the image to be welded.

[0020] The target image attributes include one or more of brightness, clarity and contrast.

[0021] Wherein, after performing quality classification on the feature data of the image to be welded and determining the current state of the image to be welded, the method further comprises:

[0022] If the image to be welded is in a normal state, obtaining a welded image of the current workstation after the workpiece is welded;

[0023] Performing region segmentation on the welded image, and taking the welding region of the workpiece as a region of interest of the welded image;

[0024] Extracting feature data of a region of interest of the welded image as feature data of the welded image;

[0025] According to the welding quality category model, prediction processing is performed on the feature data of the welded image to obtain the welding quality category of the welded image.

[0026] The step of performing region segmentation on the welded image and taking the welding region of the workpiece as the region of interest of the welded image includes:

[0027] Performing quality classification on the feature data of the welded image to determine the current state of the welded image;

[0028] If the welded image is in a normal state, the welded image is segmented into regions, and the welding region of the workpiece is used as a region of interest of the welded image.

[0029] After the quality classification of the characteristic data of the welded image is performed and the current state of the welded image is determined, the method further comprises:

[0030] If the welded image is in an abnormal state, adding the welded image into a welded abnormal image set;

[0031] According to the set of abnormal welded images, the quality classification of the welded images is optimized.

[0032] To achieve the above objectives, the present application provides a device for detecting the welding quality of a workpiece, comprising:

[0033] The first acquisition module is used to acquire the image to be welded at the current workstation before the workpiece is welded;

[0034] A first region segmentation module is used to perform region segmentation on the image to be welded, and use the welding region of the workpiece as the region of interest of the image to be welded;

[0035] A first extraction module, used for extracting feature data of a region of interest of the image to be welded as feature data of the image to be welded;

[0036] A first quality classification module is used to classify the quality of the feature data of the image to be welded and determine the current state of the image to be welded; if the image to be welded is in an abnormal state, the work flow of the compression module is started;

[0037] A compression module, used for compressing the feature data of the image to be welded, and obtaining target image attribute sensitive data corresponding to the image to be welded;

[0038] A judgment module, used to judge whether the current parameter value of the target image attribute corresponding to the target image attribute sensitive data is within the ideal range of the target parameter; if not, start the workflow of the camera parameter adjustment value determination module;

[0039] The module for determining the camera parameter adjustment value is used to obtain the camera shooting parameter adjustment value according to the deviation between the current parameter value and the ideal range of the target parameter, so as to adjust the camera shooting quality.

[0040] To achieve the above objectives, the present application provides an electronic device, including:

[0041] Memory for storing computer programs;

[0042] A processor is used to implement the steps of the above-mentioned method for detecting the welding quality of workpieces when executing the computer program.

[0043] To achieve the above objectives, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for detecting workpiece welding quality are implemented.

[0044] It can be seen from the above scheme that a method for detecting the welding quality of a workpiece provided in the present application includes: obtaining an image to be welded at a current workstation before the workpiece is welded; performing regional segmentation on the image to be welded, and taking the welding area of ​​the workpiece as the region of interest of the image to be welded; extracting feature data of the region of interest of the image to be welded as feature data of the image to be welded; performing quality classification on the feature data of the image to be welded, and determining the current state of the image to be welded; if the image to be welded is in an abnormal state, compressing the feature data of the image to be welded, and obtaining target image attribute sensitive data corresponding to the image to be welded; based on the fact that the current parameter value of the target image attribute corresponding to the target image attribute sensitive data is not within the ideal range of the target parameter, obtaining a camera shooting parameter adjustment value according to the deviation between the current parameter value and the ideal range of the target parameter.

[0045] The detection method of workpiece welding quality provided by the present application detects the image quality of the image to be welded before the workpiece is welded. If the image quality is abnormal, the target image attribute sensitive data is obtained according to the characteristic data of the image to be welded, and the current parameter value of the target image attribute is determined according to the target image attribute sensitive data. If there is a target image attribute whose current parameter value is not within the ideal range of the target parameter, the camera shooting parameter adjustment value is determined according to the deviation between the current parameter value and the ideal range of the target parameter, and the camera shooting parameter is adjusted according to the camera shooting parameter adjustment value to improve the camera shooting quality. It can be seen that the detection method of workpiece welding quality provided by the present application adjusts the camera shooting parameters according to the image to be welded with abnormal quality, improves the camera shooting quality, thereby avoiding the influence of the camera shooting quality on the welding point quality detection accuracy, and improves the welding quality detection accuracy. The present application also discloses a detection device for workpiece welding quality, an electronic device and a computer-readable storage medium, which can also achieve the above-mentioned technical effects.

[0046] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific implementation methods, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the drawings:

[0048] Figure 1 It is a flow chart of a method for detecting welding quality of a workpiece according to an exemplary embodiment;

[0049] Figure 2 is a schematic diagram of an image to be welded according to an exemplary embodiment;

[0050] Figure 3 is a schematic diagram showing a region of interest of an image to be welded according to an exemplary embodiment;

[0051] Figure 4 It is a flow chart of another method for detecting welding quality of a workpiece according to an exemplary embodiment;

[0052] Figure 5 is a schematic diagram showing the positions of various welding points in a welded image according to an exemplary embodiment;

[0053] Figure 6 is a partial image of each welding point according to an exemplary embodiment;

[0054] Figure 7 is a structural diagram of a device for detecting welding quality of a workpiece according to an exemplary embodiment;

[0055] Figure 8 The figure is a structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In addition, in the embodiments of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0057] The embodiment of the present application discloses a method for detecting the welding quality of a workpiece, which improves the detection accuracy of the welding quality.

[0058] See also Figure 1 , a flow chart of a method for detecting welding quality of a workpiece according to an exemplary embodiment is shown, as Figure 1 As shown, including:

[0059] S101: Acquire the image of the workpiece to be welded at the current workstation before welding;

[0060] The executor of this embodiment is a server, and its purpose is to adjust the camera's photo quality to improve the accuracy of detecting the welding quality of the workpiece. The camera is a built-in camera in the welding equipment, which is used to capture images of the current workstation at various stages of workpiece welding, such as images of the workpiece to be welded before welding and images of the workpiece after welding. The welding quality of the workpiece can be detected through the images captured by the camera.

[0061] In this step, the server obtains the image of the current workstation before the workpiece is welded, which is captured by the built-in camera of the welding equipment. Taking the workpiece as a camera as an example, the image to be welded is the image formed by the camera bracket and the middle plate before the camera is welded, such as Figure 2 shown.

[0062] S102: performing region segmentation on the image to be welded, and taking the welding region of the workpiece as a region of interest of the image to be welded;

[0063] In this step, the welding area of ​​the workpiece is separated from the image to be welded as the region of interest by image processing technology to eliminate the interference of the non-welding area in the image to be welded and improve the efficiency of subsequent image quality detection. Figure 3 shown.

[0064] In this embodiment, edge detection, color segmentation or machine learning algorithms can be used to determine the welding area of ​​the workpiece in the image to be welded. As a preferred embodiment, the image to be welded is input into a trained semantic segmentation model to determine the welding area in the image to be welded using the semantic segmentation model. The semantic segmentation model can understand the image content and distinguish different objects and scenes at the pixel level, so as to accurately determine the boundary of the welding area. When training the semantic segmentation model, firstly, a plurality of training images to be welded are collected, and the welding area of ​​the workpiece is marked in each training image to be welded. Then, the training image to be welded is input into the semantic segmentation model to obtain the predicted welding area of ​​the workpiece in the training image to be welded. The difference between the predicted welding area and the marked welding area is calculated by the loss function, and the model parameters of the semantic segmentation model are updated according to the difference, so as to gradually improve the prediction accuracy of the semantic segmentation model and obtain the trained semantic segmentation model. In the actual application stage, the image to be welded collected by the camera is input into the trained semantic segmentation model to predict the welding area of ​​the workpiece in the image to be welded.

[0065] S103: extracting feature data of the region of interest of the image to be welded as feature data of the image to be welded;

[0066] In this step, feature data of the region of interest of the image to be welded is extracted as feature data of the image to be welded, and these feature data may include color, texture, shape, size, etc. As a preferred embodiment, the last layer of semantic features extracted from the image to be welded in the semantic segmentation model is used as feature data of the image to be welded.

[0067] S104: performing quality classification on the feature data of the image to be welded, and determining the current state of the image to be welded; if the image to be welded is in an abnormal state, proceeding to S105;

[0068] In this step, the feature data of the image to be welded can be classified by quality through machine learning or deep learning models to determine the current state of the image to be welded, including normal state and abnormal state. If the image to be welded is in a normal state, that is, the image quality is normal, which means that the camera's photo quality is normal and no adjustment is required, and the process ends. If the image to be welded is in an abnormal state, that is, the image quality is abnormal, which means that the camera's photo quality is poor and the camera's photo parameters need to be adjusted to optimize the camera's photo quality, and then enter S105.

[0069] As a preferred embodiment, the feature data of the image to be welded is input into the first image quality monitoring model, so as to determine the current state of the image to be welded by using the first image quality monitoring model. The first image quality monitoring model may be a variational autoencoder (VAE) model. The training process of the variational autoencoder model includes: obtaining a first training data set, including a plurality of training images to be welded with normal image quality, and training the variational autoencoder model using the first training data set to obtain a trained variational autoencoder model. The trained variational autoencoder model is used to perform quality scoring on the input image to be welded, and the quality scoring method may adopt PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index Measure) and MS-SSIM (Multi-Scale Structural Similarity Index Measure) and the like. PSNR measures the image quality by comparing the original image with the distorted image, and the calculation formula is:

[0070] ;

[0071] in, is the maximum possible pixel value in image I. Generally, for uint8 data, the maximum pixel value is 255; for floating-point data, the maximum pixel value is 1. MSE is the mean squared error between the original image and the distorted image. The unit of PSNR is decibel (dB). The higher the value, the better the image quality.

[0072] SSIM is a measure of the similarity between two images, which takes into account the brightness, contrast and structural information of the image. The calculation of SSIM is based on three comparisons: brightness comparison, contrast comparison and structural comparison. The calculation formula is:

[0073] ;

[0074] in, and are the means of the two images, and is the variance, is the covariance, and It is a small constant set to avoid the denominator being zero. The value of SSIM ranges from 0 to 1, and the closer the value is to 1, the more similar the two images are.

[0075] MS-SSIM is an extension of SSIM, which provides a more comprehensive quality assessment by evaluating the similarity of images at multiple scales. MS-SSIM first calculates the SSIM value of the image at different scales (e.g., 1 / 2, 1 / 4, 1 / 8, etc.), and then averages or weighted averages these SSIM values ​​to obtain the final MS-SSIM value. The multi-scale method can better simulate the human eye's perception of image quality because it takes into account the visual characteristics of the image at different resolutions.

[0076] After obtaining the quality score of the image to be welded through the trained variational autoencoder model, it is determined whether it is greater than or equal to the first quality score threshold. If so, it is determined that the image quality of the image to be welded is normal; if not, it is determined that the image quality of the image to be welded is abnormal.

[0077] The process of determining the first quality score threshold includes: obtaining a first test data set, including a plurality of test images to be welded with normal image quality and a plurality of test images to be welded with abnormal image quality, inputting feature data of the plurality of test images to be welded in the first test data set into a trained variational autoencoder model to obtain image quality scores of the plurality of test images to be welded, sorting the image quality scores of the plurality of test images to be welded from large to small, and determining a critical quality score between the test images to be welded with normal image quality and the test images to be welded with abnormal image quality as the first quality score threshold.

[0078] S105: compressing the feature data of the image to be welded to obtain target image attribute sensitive data corresponding to the image to be welded;

[0079] S106: Determine whether the current parameter value of the target image attribute corresponding to the target image attribute sensitive data is within the ideal range of the target parameter; if not, proceed to S107;

[0080] S107: Obtaining a camera photographing parameter adjustment value according to a deviation between the current parameter value and the ideal range of the target parameter, so as to adjust the photographing quality of the camera.

[0081] In a specific implementation, the feature data of the image to be welded is compressed, and the compressed feature data is used as the target image attribute sensitive data corresponding to the image to be welded, and the current parameter value of the target image attribute is determined according to the target image attribute sensitive data, and the target image attribute may include brightness, clarity, contrast, etc. It is judged whether the current parameter value of each target image attribute is within the ideal target parameter range of each target image attribute. If there is a target image attribute whose current parameter value is not within the ideal target parameter range, the camera shooting parameter adjustment value is determined according to the deviation between the current parameter value and the ideal target parameter range, and the camera shooting parameter is adjusted according to the camera shooting parameter adjustment value to improve the camera shooting quality.

[0082] It should be noted that the dimension of the target image attribute sensitive data and the ideal range of the target parameters of each target image attribute are predetermined. The process of determining the dimension of the target image attribute sensitive data and the ideal range of the target parameters of each target image attribute includes: obtaining multiple historical images to be welded at the current workstation; adjusting the parameter value of the target graphic attribute corresponding to each of the historical images to be welded, and obtaining multi-dimensional feature compression data of each of the historical images to be welded under multiple parameters of the target graphic attribute; obtaining the sensitive relationship between the feature compression data and the target graphic attribute in each dimension according to the feature compression data corresponding to the historical images to be welded, and taking the dimension of the feature compression data that is most sensitive to the target graphic attribute as the target dimension; extracting the feature compression data of each of the historical images to be welded under the target dimension; and obtaining the ideal range of the target parameters according to the parameter value of the target graphic attribute corresponding to the feature compression data of each of the historical images to be welded under the target dimension.

[0083] In a specific implementation, multiple historical images to be welded of the current workstation are collected, and the target graphic attribute parameter value of each historical image to be welded is adjusted, such as increasing the brightness, reducing the clarity, etc. At the same time, the influence of adjusting the target graphic attribute parameter value on the feature compression data is recorded. The feature compression data is the feature data of the historical image to be welded after compression processing. The specific influence of different changes in target graphic attributes on the feature compression data can be evaluated through cross-validation. In the verification of the adjustment of a single target graphic attribute, how the feature compression data changes with the change of clarity is analyzed, so as to determine the influence of a single target graphic attribute on the feature compression data. In the cross-validation of the comprehensive adjustment of multiple target graphic attributes, how the feature compression data changes with the simultaneous change of multiple target graphic attributes is analyzed, so as to determine the influence of the joint action of multiple target graphic attributes on the feature compression data. For example, the target graphic attributes include clarity and brightness. In the cross-validation of clarity adjustment, the feature compression data is analyzed to see how it changes with the change of clarity, so as to determine the degree of influence of clarity on the feature compression data. In the cross-validation of brightness adjustment, the feature compression data is analyzed to see how it changes with the change of brightness, so as to determine the degree of influence of brightness on the feature compression data. In the cross-validation of comprehensive adjustment, the feature compression data is analyzed to see how it changes with the simultaneous change of clarity and brightness, so as to determine the influence of the two attributes on the feature compression data. Through the above analysis, the sensitive relationship between each dimension in the feature compression data and each target graphic attribute can be obtained. This relationship can help understand the change trend and sensitivity of the feature compression data under different image attribute conditions. The feature compression data dimension that is most sensitive to the target graphic attribute is taken as the target dimension. For example, the target dimension is two-dimensional, corresponding to clarity and brightness respectively. Further, the feature data of each historical image to be welded is compressed into feature compression data of the target dimension, and the parameter values ​​of each target graphic attribute are determined according to the feature compression data of the target dimension of each historical image to be welded, so as to determine the ideal range of the target parameters corresponding to each target graphic attribute.

[0084] On the basis of determining the dimension of the target image attribute sensitive data in the above manner, the feature data of the image to be welded is compressed to obtain the target image attribute sensitive data corresponding to the image to be welded, including: obtaining feature compression data of the feature data of the image to be welded in the target dimension, and using the feature compression data of the feature data of the image to be welded in the target dimension as the target image attribute sensitive data corresponding to the image to be welded. In a specific implementation, when analyzing the image to be welded acquired by the camera in real time, the feature data of the image to be welded is compressed into feature compression data of the target dimension, and the feature compression data of the target dimension is used as the target image attribute sensitive data of the image to be welded.

[0085] The detection method of workpiece welding quality provided in the embodiment of the present application detects the image quality of the image to be welded before the workpiece is welded. If the image quality is abnormal, the target image attribute sensitive data is obtained according to the characteristic data of the image to be welded, and the current parameter value of the target image attribute is determined according to the target image attribute sensitive data. If there is a target image attribute whose current parameter value is not within the ideal range of the target parameter, the camera shooting parameter adjustment value is determined according to the deviation between the current parameter value and the ideal range of the target parameter, and the camera shooting parameter is adjusted according to the camera shooting parameter adjustment value to improve the camera shooting quality. It can be seen that the detection method of workpiece welding quality provided in the present application adjusts the camera shooting parameters according to the image to be welded with abnormal quality, improves the camera shooting quality, thereby avoiding the influence of the camera shooting quality on the welding point quality detection accuracy, and improves the welding quality detection accuracy.

[0086] The embodiment of the present application discloses a method for detecting the welding quality of a workpiece. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically:

[0087] See also Figure 4 , a flow chart of another method for detecting workpiece welding quality according to an exemplary embodiment is shown, such as Figure 4 As shown, including:

[0088] S201: Acquire the image of the workpiece to be welded at the current workstation before welding;

[0089] S202: performing region segmentation on the image to be welded, and taking the welding region of the workpiece as a region of interest of the image to be welded;

[0090] S203: extracting feature data of the region of interest of the image to be welded as feature data of the image to be welded;

[0091] S204: performing quality classification on the feature data of the image to be welded, and determining the current state of the image to be welded; if the image to be welded is in an abnormal state, proceeding to S205; if the image to be welded is in a normal state, proceeding to S208;

[0092] In this embodiment, if the image to be welded is in a normal state, that is, the image quality is normal, indicating that the camera's photographic quality is normal, no adjustment is required, and the process directly proceeds to S208 to detect the welding quality. If the image to be welded is in an abnormal state, that is, the image quality is abnormal, indicating that the camera's photographic quality is poor, and the camera's photographic parameters need to be adjusted to optimize the camera's photographic quality, and the process proceeds to S205.

[0093] S205: compressing the feature data of the image to be welded to obtain target image attribute sensitive data corresponding to the image to be welded;

[0094] S206: Determine whether the current parameter value of the target image attribute corresponding to the target image attribute sensitive data is within the ideal range of the target parameter; if not, proceed to S207;

[0095] S207: Obtaining a camera photographing parameter adjustment value according to the deviation between the current parameter value and the ideal range of the target parameter, so as to adjust the photographing quality of the camera; and re-entering S201;

[0096] In this embodiment, after adjusting the camera's photographing quality according to the camera photographing parameter adjustment value, re-enter S201, re-capture the image to be welded by the camera, and detect the image quality of the image to be welded until the image quality of the image to be welded is normal and the camera's photographing quality is normal, then enter S208 to perform subsequent welding quality detection.

[0097] S208: Acquire a welded image of the current workstation after the workpiece is welded;

[0098] In this step, a camera with normal photographing quality is used to collect welded images of the current workstation after the workpiece is welded, for subsequent detection of the welding quality.

[0099] S209: performing region segmentation on the welded image, and taking the welding region of the workpiece as a region of interest of the welded image;

[0100] In this step, the welding area of ​​the workpiece is separated from the welded image as the region of interest by image processing technology. Here, the semantic segmentation model can also be used to determine the welding area of ​​the workpiece in the welded image.

[0101] S210: extracting feature data of the region of interest of the welded image as feature data of the welded image;

[0102] In this step, feature data of the region of interest of the welded image is extracted as feature data of the welded image. As a preferred embodiment, the last layer of semantic features extracted from the welded image in the semantic segmentation model is used as feature data of the welded image.

[0103] S211: Predictively process the feature data of the welded image according to the welding quality category model to obtain the welding quality category of the welded image.

[0104] In this step, the feature data of the welded image is input into the welding quality classification model. The welding quality classification model is used to infer the input feature data to predict the welding quality category, confidence and position of each weld in the welded image. The welding quality category may include normal welding category, false welding category, leaking welding category, and biased welding category. Taking the workpiece as an example, the camera recognizes the position of each weld point. Figure 5 As shown in Figure 2, the local images of each solder joint are as follows: Figure 6 As shown, the welding quality category results of each solder joint are: solder joint 1 is normal, solder joint 2 is normal, solder joint 3 is cold solder joint, solder joint 4 is normal, solder joint 5 is biased solder joint, solder joint 6 is normal, solder joint 7 is normal, and solder joint 8 is leaking solder joint.

[0105] On the basis of this embodiment, as a preferred implementation manner, the region segmentation of the welded image and taking the welding region of the workpiece as the region of interest of the welded image includes: quality classification of feature data of the welded image to determine the current state of the welded image; if the welded image is in a normal state, region segmentation of the welded image and taking the welding region of the workpiece as the region of interest of the welded image.

[0106] In a specific implementation, the feature data of the welded image can be quality classified by a machine learning or deep learning model to determine the current state of the welded image, including a normal state and an abnormal state. If the welded image is in a normal state, the subsequent welding quality detection process is performed.

[0107] As a preferred embodiment, the feature data of the welded image is input into the second image quality monitoring model to determine the current state of the welded image using the second image quality monitoring model. The second image quality monitoring model can be a variational autoencoder model. The training process of the variational autoencoder model can refer to the description of the previous embodiment, which will not be repeated here. The trained variational autoencoder model is used to perform quality scoring on the input welded image, and the quality scoring method can also refer to the description of the previous embodiment, which will not be repeated here.

[0108] After obtaining the quality score of the welded image through the trained variational autoencoder model, it is determined whether it is greater than or equal to the second quality score threshold. If so, it is determined that the image quality of the welded image is normal; if not, it is determined that the image quality of the welded image is abnormal.

[0109] The process of determining the second quality score threshold includes: obtaining a second test data set, including multiple test welded images with normal image quality and multiple welded images with abnormal image quality, inputting feature data of the multiple test welded images in the second test data set into a trained variational autoencoder model to obtain image quality scores of the multiple test welded images, sorting the image quality scores of the multiple test welded images from large to small, and determining a critical quality score between the test welded images with normal image quality and the test welded images with abnormal image quality as the second quality score threshold.

[0110] Furthermore, after performing quality classification on the characteristic data of the welded image and determining the current state of the welded image, the present embodiment further includes: if the welded image is in an abnormal state, adding the welded image into a set of abnormal welded images; and optimizing the quality classification of the welded image based on the set of abnormal welded images.

[0111] In a specific implementation, if the welded image is in an abnormal state, the welded image is added to a welded abnormal image set. When the number of images contained in the welded abnormal image set reaches a threshold, the images contained in the welded abnormal image set can be used to optimize the quality classification of the welded image.

[0112] It can be seen that this embodiment adjusts the camera shooting parameters according to the image to be welded with abnormal quality to improve the camera shooting quality. After the workpiece is welded, the welded image of the current workstation is collected by a camera with normal shooting quality, and the welding quality of each welding point is determined according to the welded image, thereby avoiding the influence of the camera shooting quality on the accuracy of welding point quality detection and improving the accuracy of welding quality detection.

[0113] A workpiece welding quality detection device provided in an embodiment of the present application is introduced below. The workpiece welding quality detection device described below and the workpiece welding quality detection method described above can refer to each other.

[0114] See also Figure 7 , a structural diagram of a device for detecting welding quality of a workpiece according to an exemplary embodiment is shown, such as Figure 7 As shown, including:

[0115] The first acquisition module 100 is used to acquire the image of the workpiece to be welded at the current workstation before welding;

[0116] A first region segmentation module 200 is used to perform region segmentation on the image to be welded, and to use the welding region of the workpiece as the region of interest of the image to be welded;

[0117] A first extraction module 300, used to extract feature data of a region of interest of the image to be welded as feature data of the image to be welded;

[0118] The first quality classification module 400 is used to classify the quality of the feature data of the image to be welded and determine the current state of the image to be welded; if the image to be welded is in an abnormal state, the work flow of the compression module is started;

[0119] A compression module 500 is used to compress the feature data of the image to be welded, and obtain the target image attribute sensitive data corresponding to the image to be welded;

[0120] A judgment module 600 is used to judge whether the current parameter value of the target image attribute corresponding to the target image attribute sensitive data is within the ideal range of the target parameter; if not, the workflow of the module for determining the camera parameter adjustment value is started;

[0121] The module 700 for determining the camera parameter adjustment value is used to obtain the camera shooting parameter adjustment value according to the deviation between the current parameter value and the ideal range of the target parameter, so as to adjust the camera shooting quality.

[0122] The device for detecting the welding quality of a workpiece provided in an embodiment of the present application detects the image quality of the image to be welded before the workpiece is welded. If the image quality is abnormal, the sensitive data of the target image attribute is obtained according to the characteristic data of the image to be welded, and the current parameter value of the target image attribute is determined according to the sensitive data of the target image attribute. If there is a target image attribute whose current parameter value is not within the ideal range of the target parameter, the camera shooting parameter adjustment value is determined according to the deviation between the current parameter value and the ideal range of the target parameter, and the camera shooting parameter is adjusted according to the camera shooting parameter adjustment value to improve the camera shooting quality. It can be seen that the device for detecting the welding quality of a workpiece provided in the present application adjusts the camera shooting parameters according to the image to be welded with abnormal quality, improves the camera shooting quality, thereby avoiding the influence of the camera shooting quality on the welding point quality detection accuracy, and improves the welding quality detection accuracy.

[0123] Based on the above embodiment, as a preferred implementation, the device further includes:

[0124] A module for determining a target dimension obtains a plurality of historical images to be welded of the current workstation; adjusts the parameter value of the target graphic attribute corresponding to each of the historical images to be welded, and obtains multi-dimensional feature compression data of each of the historical images to be welded under a plurality of parameters of the target graphic attribute; obtains a sensitive relationship between the feature compression data of each dimension and the target graphic attribute according to the feature compression data corresponding to the historical images to be welded, and uses the dimension of the feature compression data that is most sensitive to the target graphic attribute as the target dimension;

[0125] The module for obtaining the ideal range of target parameters is used to extract the feature compression data of each historical image to be welded under the target dimension; and obtain the ideal range of the target parameters according to the parameter value of the target graphic attribute corresponding to the feature compression data of each historical image to be welded under the target dimension.

[0126] On the basis of the above embodiments, as a preferred implementation mode, the compression module is specifically used to: obtain feature compression data of the feature data of the image to be welded under the target dimension, and use the feature compression data of the feature data of the image to be welded under the target dimension as the target image attribute sensitive data corresponding to the image to be welded.

[0127] Based on the above embodiment, as a preferred implementation manner, the target image attributes include one or more of brightness, clarity and contrast.

[0128] Based on the above embodiment, as a preferred implementation, the device further includes:

[0129] A second acquisition module is used to acquire a welded image of the current workstation after the workpiece is welded when the image to be welded is in a normal state;

[0130] A second region segmentation module is used to perform region segmentation on the welded image, and use the welding region of the workpiece as the region of interest of the welded image;

[0131] A second extraction module, used for extracting feature data of a region of interest of the welded image as feature data of the welded image;

[0132] The welding quality detection module is used to predict and process the feature data of the welded image according to the welding quality category model to obtain the welding quality category of the welded image.

[0133] Based on the above embodiment, as a preferred implementation, the device further includes:

[0134] A first quality classification module is used to perform quality classification on the feature data of the welded image and determine the current state of the welded image; if the welded image is in a normal state, the workflow of the second region segmentation module is started; if the welded image is in an abnormal state, the workflow of the adding module is started;

[0135] An adding module is used to add the welded image into a set of abnormal welded images; and optimize the quality classification of the welded image according to the set of abnormal welded images.

[0136] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0137] Based on the hardware implementation of the above program module, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an electronic device, Figure 8 FIG. 1 is a structural diagram of an electronic device according to an exemplary embodiment. Figure 8 As shown, the electronic equipment includes:

[0138] Communication interface 1, capable of exchanging information with other devices such as network devices;

[0139] The processor 2 is connected to the communication interface 1 to realize information exchange with other devices, and is used to execute the workpiece welding quality detection method provided by one or more technical solutions when running a computer program. The computer program is stored in the memory 3.

[0140] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 8 Various buses are labeled as bus system 4 .

[0141] The memory 3 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0142] It can be understood that the memory 3 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 3 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.

[0143] The method disclosed in the above embodiment of the present application can be applied to the processor 2, or implemented by the processor 2. The processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 2 or the instruction in the form of software. The above processor 2 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 3, and the processor 2 reads the program in the memory 3 and completes the steps of the above method in combination with its hardware.

[0144] When the processor 2 executes the program, the corresponding processes in the various methods of the embodiments of the present application are implemented, which will not be repeated here for the sake of brevity.

[0145] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 3 storing a computer program, and the computer program can be executed by a processor 2 to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, CD-ROM, etc.

[0146] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0147] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for an electronic device (which can be a personal computer, server, network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0148] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for detecting the welding quality of a workpiece, characterized in that: include: Obtain the image of the workpiece to be welded before welding at the current workstation; Performing regional segmentation on the image to be welded, and taking the welding area of ​​the workpiece as the region of interest of the image to be welded; Extracting feature data of a region of interest of the image to be welded as feature data of the image to be welded; Performing quality classification on the feature data of the image to be welded to determine the current state of the image to be welded; If the image to be welded is in an abnormal state, compressing the feature data of the image to be welded to obtain sensitive data of target image attributes corresponding to the image to be welded; Based on the fact that the current parameter value of the target image attribute corresponding to the target image attribute sensitive data is not within the ideal range of the target parameter, the camera shooting parameter adjustment value is obtained according to the deviation between the current parameter value and the ideal range of the target parameter.

2. The detection method according to claim 1, characterized in that: The method further comprises: Acquire multiple historical images to be welded of the current workstation; Adjusting the parameter value of the target graphic attribute corresponding to each of the historical images to be welded, and obtaining multi-dimensional feature compression data of each of the historical images to be welded under multiple parameters of the target graphic attribute; According to the feature compression data corresponding to the historical image to be welded, a sensitive relationship between the feature compression data of each dimension and the target graphic attribute is obtained, and the feature compression data dimension that is most sensitive to the target graphic attribute is used as the target dimension; Extracting feature compression data of each of the historical images to be welded at a target dimension; The ideal range of the target parameter is obtained according to the parameter value of the target graphic attribute corresponding to the feature compression data of each of the historical images to be welded in the target dimension.

3. The detection method according to claim 2, characterized in that: The step of compressing the feature data of the image to be welded to obtain target image attribute sensitive data corresponding to the image to be welded includes: Acquire feature compression data of the feature data of the image to be welded in the target dimension, and use the feature compression data of the feature data of the image to be welded in the target dimension as the target image attribute sensitive data corresponding to the image to be welded.

4. The detection method according to claim 1, characterized in that: The target image attributes include one or more of brightness, clarity and contrast.

5. The detection method according to claim 1, characterized in that: After the quality classification of the feature data of the image to be welded is performed and the current state of the image to be welded is determined, the method further includes: Based on the image to be welded being in a normal state, obtaining a welded image of the current workstation after the workpiece is welded; Performing region segmentation on the welded image, and taking the welding region of the workpiece as a region of interest of the welded image; Extracting feature data of a region of interest of the welded image as feature data of the welded image; According to the welding quality category model, prediction processing is performed on the feature data of the welded image to obtain the welding quality category of the welded image.

6. The detection method according to claim 5, characterized in that: The performing region segmentation on the welded image and taking the welding region of the workpiece as the region of interest of the welded image comprises: Performing quality classification on the feature data of the welded image to determine the current state of the welded image; Based on the fact that the welded image is in a normal state, the welded image is segmented into regions, and the welding region of the workpiece is used as a region of interest of the welded image.

7. The detection method according to claim 6, characterized in that: After the quality classification of the feature data of the welded image is performed and the current state of the welded image is determined, the method further comprises: If the welded image is in an abnormal state, adding the welded image into a welded abnormal image set; According to the set of abnormal welded images, the quality classification of the welded images is optimized.

8. A device for detecting the welding quality of a workpiece, characterized in that: include: The first acquisition module is used to acquire the image to be welded at the current workstation before the workpiece is welded; A first region segmentation module, used to perform region segmentation on the image to be welded, and use the welding region of the workpiece as the region of interest of the image to be welded; A first extraction module, used for extracting feature data of a region of interest of the image to be welded as feature data of the image to be welded; A first quality classification module is used to classify the quality of the feature data of the image to be welded and determine the current state of the image to be welded; if the image to be welded is in an abnormal state, the work flow of the compression module is started; A compression module, used for compressing the feature data of the image to be welded, and obtaining target image attribute sensitive data corresponding to the image to be welded; A judgment module, used to judge whether the current parameter value of the target image attribute corresponding to the target image attribute sensitive data is within the ideal range of the target parameter; if not, start the workflow of the camera parameter adjustment value determination module; The module for determining the camera parameter adjustment value is used to obtain the camera shooting parameter adjustment value according to the deviation between the current parameter value and the ideal range of the target parameter, so as to adjust the camera shooting quality.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for detecting the welding quality of a workpiece as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the steps of the method for detecting the welding quality of a workpiece according to any one of claims 1 to 7 are implemented.

Images