Machine vision-based industrial welding quality detection method

Abstract

The present application relates to the technical field of welding quality detection, in particular to an industrial welding quality detection method based on machine vision. The method comprises: acquiring a suspected abnormal characteristic value and a target defect characteristic value, obtaining a suspected welding defect clustering cluster according to the coordinates of connected domain pixel points, the suspected abnormal characteristic value and the target defect characteristic value, and obtaining the structure element of each pixel point in the welding image to be processed according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster; performing bottom hat operation on the welding image to be processed according to the structure element of the pixel point, and obtaining a target welding image; and performing welding defect detection and identification on the welding workpiece according to the target welding image. Moreover, the present application adaptively acquires the structure element according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster, can improve the effect of bottom hat operation, improve the enhancement effect of dark areas, and thus can improve the accuracy of welding defect detection and identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data acquisition, in particular to an industrial welding quality detection method based on machine vision. BACKGROUND

[0002] At present, welding technology is applied in manufacturing, construction, automobile, shipbuilding, aerospace and other industries, and the quality of welding is directly related to the performance, safety and service life of the welding product in use, so machine vision technology is usually used to detect the quality of welding.

[0003] Due to the existence of optical environment and the reflection characteristics of the welded workpiece, the detection result is generally negatively affected, and in order to reduce the negative influence of optical environment and the reflection characteristics of the welded workpiece on the detection result, the collected image is usually subjected to bottom hat operation to improve the detection efficiency and accuracy, but in the prior art, a fixed size structure element is usually selected for bottom hat operation, and the determination method of the size of the structure element will cause the problem of insufficient enhancement of dark areas, and insufficient enhancement of dark areas will result in low detection accuracy of welding defects, such as pores, cracks and other welding defects, which are usually represented as dark areas in the image, so how to adaptively obtain the size of the structure element and improve the accuracy of subsequent welding quality detection becomes a problem to be solved. SUMMARY

[0004] In order to solve the above problems, the present application provides an industrial welding quality detection method based on machine vision, and the technical solution is as follows:

[0005] An embodiment of the present application provides an industrial welding quality detection method based on machine vision, comprising the following steps:

[0006] Obtain a to-be-processed welding image of a welding workpiece and a to-be-analyzed connected domain on the to-be-processed welding image, wherein each point in the to-be-analyzed connected domain is referred to as a connected domain pixel point;

[0007] According to the gray value of the connected domain pixel point and the gray histogram of the to-be-processed welding image, a suspected abnormal feature value of the connected domain pixel point is obtained;

[0008] According to the area of the to-be-analyzed connected domain containing the connected domain pixel point and the area of the minimum circumscribed ellipse, the curvature, gradient and normal direction of the edge pixel point on the to-be-analyzed connected domain containing the connected domain pixel point, and the local window corresponding to the connected domain pixel point, a target defect feature value of the connected domain pixel point is obtained;

[0009] According to the coordinates of the connected domain pixel points, the suspected abnormal characteristic value and the target defect characteristic value, a suspected welding defect clustering cluster is obtained, and according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster, a structure element of each pixel point in the to-be-processed welding image is obtained, bottom hat operation is performed on the to-be-processed welding image according to the structure element of the pixel point, and a target welding image is obtained.

[0010] According to the target welding image, welding defect detection and identification of the welding workpiece are performed.

[0011] Beneficial effects: The present application first obtains a to-be-processed welding image of a welding workpiece and a to-be-analyzed connected domain on the to-be-processed welding image; then according to the gray value of the connected domain pixel points and the gray histogram of the to-be-processed welding image, a suspected abnormal characteristic value of the connected domain pixel points is obtained; then according to the area of the to-be-analyzed connected domain containing the connected domain pixel points and the area of the minimum circumscribed ellipse, the curvature, gradient and normal direction of the edge pixel points on the to-be-analyzed connected domain containing the connected domain pixel points and the local window corresponding to the connected domain pixel points, a target defect characteristic value of the connected domain pixel points is obtained; then according to the coordinates of the connected domain pixel points, the suspected abnormal characteristic value and the target defect characteristic value, a suspected welding defect clustering cluster is obtained, and according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster, a structure element of each pixel point in the to-be-processed welding image is obtained, bottom hat operation is performed on the to-be-processed welding image according to the structure element of the pixel point, and a target welding image is obtained; finally, according to the target welding image, welding defect detection and identification of the welding workpiece are performed. And the present application adaptively obtains the structure element according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster, can improve the effect of bottom hat operation, improve the enhancement effect on dark areas, and thus can improve the accuracy of detecting and identifying welding defects. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0013] Figure 1 The flowchart of the present application, an industrial welding quality detection method based on machine vision. DETAILED DESCRIPTION

[0014] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the protection scope of the embodiments of the present invention.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0016] This embodiment provides a machine vision-based industrial welding quality inspection method, which is described in detail below:

[0017] like Figure 1 As shown, this machine vision-based industrial welding quality inspection method includes the following steps:

[0018] Step S001: Obtain the welding image of the workpiece to be processed and the connected components to be analyzed on the welding image to be processed.

[0019] The main purpose of this embodiment is to improve the enhancement effect on the welding defect area by adaptively obtaining the size of the structuring element when performing bottom cap operation on the acquired welding image, thereby improving the accuracy of subsequent welding quality inspection. The structuring element is the core of the bottom cap operation.

[0020] This embodiment first acquires any completed welding workpiece and records it as the welding workpiece. Then, it acquires an image of the surface of the welding workpiece and records the acquired image as the initial welding image of the welding workpiece. The initial welding image is then processed into grayscale, and the grayscale processed image is recorded as the welding image to be processed of the welding workpiece. The acquired image of the welding workpiece includes the welding area. The device for acquiring the image of the surface of the welding workpiece includes a high-definition camera and a light source. The specific image acquisition process is well known. For example, an acquisition device that can rotate the welding workpiece can be used to acquire the image of the surface of the welding workpiece. Alternatively, the welding workpiece can be placed on a transmission device for acquiring welding quality inspection images. When the welding workpiece passes directly below the camera, the transmission belt stops, and the camera acquires the image of the welding workpiece.

[0021] Therefore, the embodiment can obtain the to-be-processed welding image of the welding workpiece through the above process. After obtaining the to-be-processed welding image, the connected domain on the to-be-processed welding image is obtained, and all the connected domains are recorded as to-be-analyzed connected domains. The pixel points in the to-be-analyzed connected domains are recorded as connected domain pixel points. Since the pixel points with welding defect features such as pores and cracks usually have connected domain features, that is, the pixel points with welding defect features such as pores and cracks are located in the connected domain, the embodiment mainly improves the accuracy of welding quality detection by improving the enhancement effect of the welding defect area. Therefore, the embodiment only analyzes the possibility of welding defects for the connected domain pixel points in the subsequent process, so as to reduce the calculation amount. That is, the purpose of obtaining the connected domain pixel points is to more conveniently and accurately analyze the possibility of the pixel points belonging to welding defects, so as to more accurately and effectively adjust the size of the structure element, improve the effect of the bottom hat operation, and improve the detection and recognition of the welding quality. In addition, in the embodiment, the specific process of obtaining the connected domain on the to-be-processed welding image is as follows: first, the edge of the to-be-processed welding image is extracted by using the canny edge detection algorithm to obtain all the edges in the to-be-processed welding image, and then the broken edges in the to-be-processed welding image are closed by morphological dilation or erosion. After that, the area contained by the closed edge is recorded as the connected domain.

[0022] In step S002, the suspected abnormal feature value of the connected domain pixel point is obtained according to the gray value of the connected domain pixel point and the gray histogram of the to-be-processed welding image.

[0023] Since the purpose of the bottom-hat operation on the welding image to be processed is to perform image enhancement, especially to enhance the features of the welding defect region, so that the features of the welding defect region are more obvious, and the subsequent identification and detection of the welding defect region are facilitated; and since the existing bottom-hat operation usually selects to use a fixed-size structure element, and the determination method of the size of such a structure element will have the problem of insufficient enhancement of dark areas, and the welding defect region usually exhibits dark area features, thereby affecting the subsequent welding quality detection, if the structure element used is too large, the dark area details cannot be captured, resulting in poor enhancement effect on the dark area; and in order to improve the effect of the bottom-hat operation, the size of the structure element will be adaptively obtained based on the possibility of the connected domain pixel belonging to the welding defect next, that is, the size of the structure element will be adaptively obtained based on the suspected abnormal representation value and the target defect representation value of the connected domain pixel; therefore, it can be known that the suspected abnormal representation value of the connected domain pixel is analyzed and obtained next, and since the reflectivity of the metal material is generally good, it will also cause the welding defect regions such as pores and cracks to generally exhibit as regions with low gray value, that is, dark regions, resulting in a large difference between the welding defect region and the overall welding device, that is, the gray value of the pixel points in the dark region accounts for a small proportion relative to the overall image, and based on the above analysis, the specific process of obtaining the suspected abnormal representation value of each connected domain pixel in the connected domain to be analyzed on the welding image to be processed is:

[0024] First, according to the frequency of each gray value in the to-be-processed welding image, a gray histogram of the to-be-processed welding image is constructed. The construction process of the gray histogram is known, and the implementer needs to set the gray level of the constructed gray histogram according to the actual situation, for example, an 8-level gray histogram or a 256-level gray histogram can be constructed. Each gray level on the 8-level gray histogram represents a gray value range, for example, the gray level 1 on the 8-level gray histogram represents a gray value range of 0 to 31. For any connected domain pixel point b: on the gray histogram of the to-be-processed welding image, the gray level containing the gray value of the connected domain pixel point b is obtained, and is recorded as the gray level corresponding to the connected domain pixel point b. The pixel number corresponding to the gray level corresponding to the connected domain pixel point b is obtained, and is recorded as the pixel number characteristic value of the gray level corresponding to the connected domain pixel point b. The pixel number characteristic value of the gray level corresponding to the connected domain pixel point b refers to the number of pixel points belonging to the gray level corresponding to the connected domain pixel point b in the to-be-processed welding image. The maximum gray level on the gray histogram of the to-be-processed welding image is subtracted from the gray level corresponding to the connected domain pixel point b, and is recorded as the relative gray size characteristic value corresponding to the connected domain pixel point b. The pixel number characteristic value of the gray level corresponding to the connected domain pixel point b is negatively correlated, and the negatively correlated mapping result of the relative gray size characteristic value corresponding to the connected domain pixel point b and the pixel number characteristic value of the gray level corresponding to the connected domain pixel point b is multiplied to obtain the suspected abnormal characteristic value of the connected domain pixel point b, that is, the suspected abnormal characteristic value of the connected domain pixel point b is the product of the negatively correlated mapping result of the pixel number characteristic value and the relative gray size characteristic value. The negatively correlated mapping result of the pixel number characteristic value of the gray level corresponding to the connected domain pixel point b is the reciprocal of the pixel number characteristic value of the gray level corresponding to the connected domain pixel point b. The greater the suspected abnormal characteristic value of the connected domain pixel point b, the greater the possibility that the connected domain pixel point b belongs to a welding defect.

[0025] In this embodiment, the specific calculation expression of the suspected abnormal characteristic value of the connected domain pixel point b is:

[0026] ;

[0027] Wherein, is the suspected abnormal characteristic value of the connected domain pixel point b, is the maximum gray level on the gray histogram of the to-be-processed welding image, is the gray level corresponding to the connected domain pixel point b, is the pixel number characteristic value of the gray level corresponding to the connected domain pixel point b; and The greater, the more obvious the dark area feature represented by the gray value of the connected domain pixel point b relative to the image, The smaller, the lower the proportion of the gray value of the connected domain pixel point b in the image, the more obvious the dark area feature represented, and the more likely it belongs to the welding defect area, and The larger, the more obvious the dark area feature represented by the connected domain pixel point b, the more likely it belongs to the welding defect area, and The smaller, the less obvious the dark area feature represented by the connected domain pixel point b, the less likely it belongs to the welding defect area. The larger, the more obvious the dark area feature represented by the connected domain pixel point b, the more likely it belongs to the welding defect area, and The smaller, the less obvious the dark area feature represented by the connected domain pixel point b, the less likely it belongs to the welding defect area. The smaller, the less obvious the dark area feature represented by the connected domain pixel point b, the less likely it belongs to the welding defect area.

[0028] Therefore, the embodiment can obtain the suspected abnormal representation value of each connected domain pixel point through the above process.

[0029] Step S003, obtaining the target defect representation value of the connected domain pixel point according to the area of the to-be-analyzed connected domain containing the connected domain pixel point and the minimum circumscribed ellipse, the curvature, gradient and normal direction of the edge pixel point on the to-be-analyzed connected domain containing the connected domain pixel point, and the local window corresponding to the connected domain pixel point.

[0030] Because not only the welding defect area in the image shows the characteristics of lower gray value and different from the normal part, but also the noise generated by various external factors and the texture area of the welding workpiece itself may also have the characteristics of lower gray value and different from the normal part, therefore, only based on the above-mentioned suspected abnormal representation value, the size of the structure element is adaptively obtained, which may cause some areas to be excessively smoothed, details to be lost or small noise to be left, etc., resulting in poor effect of bottom hat operation or processing, affecting the detection result of subsequent welding quality, and in order to avoid the above-mentioned situation as much as possible, the embodiment needs to further analyze the connected domain pixel point to obtain the target defect representation value of the connected domain pixel point, and then further combine the obtained target defect representation value of the connected domain pixel point to complete the adaptive acquisition of the size of the structure element, so the specific process of the target defect representation value of the connected domain pixel point is:

[0031] Since the porosity welding defect is a welding quality problem caused by a process defect, and the noise is random interference caused by the external environment, the porosity will appear as an approximately circular or elliptical area with a relatively smooth boundary, while the noise will appear as irregular and small spots without a fixed shape. Since the gray level changes from the inside of the hole to the surrounding matrix are relatively steep, the gray level gradient of the porosity edge is large, and the boundary is clear and identifiable. The edge of the noise is blurred, and the gray level transition between pixels is relatively smooth without a clear boundary. Therefore, based on the differences between the characteristics of the porosity and the noise, that is, the shape of the connected domain pixel, the smoothness of the connected domain boundary, and the clarity, the first defect representation value of the connected domain pixel is first obtained. The first defect representation value of any connected domain pixel b is a key indicator for determining the target defect representation value of the connected domain pixel. The specific process for obtaining the first defect representation value of any connected domain pixel b is as follows:

[0032] First, the connected domain to be analyzed containing the connected domain pixel b is obtained and denoted as connected domain A. The minimum circumscribed ellipse of connected domain A is obtained, and the curvature, gradient, and normal direction of each edge pixel in connected domain A are calculated. The process for obtaining the minimum circumscribed ellipse, the curvature, gradient, and normal direction of the pixel is known. Then, based on the area of connected domain A, the area of the minimum circumscribed ellipse of connected domain A, and the curvature, gradient, and normal direction of each edge pixel in connected domain A, the first defect representation value of connected domain pixel b is obtained. The specific process for obtaining the first defect representation value of connected domain pixel b based on the area of connected domain A, the area of the minimum circumscribed ellipse of connected domain A, and the curvature, gradient, and normal direction of each edge pixel in connected domain A is as follows:

[0033] First, the area of the minimum circumscribed ellipse of connected domain A and the area of connected domain A are obtained, and the ratio of the area of the minimum circumscribed ellipse of connected domain A to the area of connected domain A is calculated and denoted as the first ratio. The absolute value of the difference between the constant 1 and the first ratio is calculated and denoted as the first difference. The result of negatively normalizing the first difference is denoted as the first index value of connected domain pixel b. The smaller the first difference, the larger the output result after negative normalization. The first difference is where S1 is the area of the minimum circumscribed ellipse of connected domain A, and S2 is the area of connected domain A. Generally, S1 is greater than or equal to S2, and the smaller the ratio of S1 to S2, that is, or the larger the first index value, the more likely it is that connected domain A is an approximately circular or elliptical area, and the more it has the characteristics of the porosity welding defect. Therefore, connected domain pixel b also has the characteristics of the porosity welding defect.

[0034] In addition, the formula for negatively normalizing the first difference is where a maximum value of a value interval corresponding to the first difference value, a minimum value of a value interval corresponding to the first difference value, x is the first difference value, and the method of negative normalization of other data is the same as that of the first difference value. As another real-time method, other known negative normalization methods can also be selected for normalization.

[0035] Then, the standard deviation of the curvature of all edge pixels in the connected domain A is calculated, and the result of negative normalization of the standard deviation of the curvature of all edge pixels in the connected domain A is recorded as the second index value of the connected domain pixel b. The smaller the standard deviation of the curvature, the larger the output result after negative normalization. The larger the standard deviation of the curvature, the worse the smoothness of the edge of the connected domain A, and the less consistent with the characteristics of the pore defect. That is, the smaller the standard deviation of the curvature or the larger the second index value, the smoother the edge of the connected domain A or the more the connected domain A has the characteristics of the pore welding defect, and then the connected domain pixel b also has the characteristics of the pore welding defect.

[0036] After that, each edge pixel in the connected domain A is taken as a starting point, and a corresponding analysis window of the corresponding edge pixel is constructed along the normal direction of the corresponding edge pixel. The analysis window of the edge pixel is located on the normal line of the edge pixel, and in specific applications, the implementer needs to set the size of the analysis window according to the actual situation. For example, the size of the analysis window can be set to 1x4 in this embodiment, that is, the number of pixels that can be accommodated by the analysis window is 4. For example, for any edge pixel, the edge pixel is taken as a starting point, and the normal direction of the edge pixel is traversed. According to the order of traversal, the window composed of the first three pixels and the edge pixel is recorded as the analysis window corresponding to the edge pixel. Then, the edge feature value of each edge pixel in the connected domain A is obtained, and the specific calculation process of the edge feature value of any edge pixel is as follows: the mean value of the gray values of all pixels in the analysis window corresponding to the edge pixel except the edge pixel is calculated and recorded as the neighborhood mean value of the edge pixel. The absolute value of the difference between the gray value of the edge pixel and the neighborhood mean value of the edge pixel is calculated and recorded as the gray neighborhood difference value of the edge pixel. The addition result of the gray neighborhood difference value of the edge pixel and the gradient value of the edge pixel is calculated and recorded as the edge feature value of the edge pixel. The calculation expression of the edge feature value of the edge pixel is D1 is the absolute value of the difference between the gray value of the edge pixel point and the neighborhood average value of the edge pixel point, and D2 is the gradient value of the edge pixel point; and the greater the gray neighborhood difference value of the edge pixel point and the gradient value of the edge pixel point, the clearer the edge of the connected domain A, the more consistent with the edge characteristics of the welding defect of the gas hole; then the mean value of the edge feature values of all edge pixel points in the connected domain A is calculated, and the result of normalizing the mean value of the edge feature values of all edge pixel points in the connected domain A is taken as the third index value of the connected domain pixel point b, and the greater the third index value, the clearer the edge of the connected domain A, the more consistent with the edge characteristics of the welding defect of the gas hole, and then the connected domain pixel point b also has the characteristics of the welding defect of the gas hole.

[0037] Finally, the product of the first index value, the second index value and the third index value of the connected domain pixel point b is calculated, and is taken as the first defect representation value of the connected domain pixel point b, that is, the calculation expression of the first defect representation value of the connected domain pixel point b is Where H1 is the first index value of the connected domain pixel point b, H2 is the second index value of the connected domain pixel point b, and H3 is the third index value of the connected domain pixel point b; the greater the first defect representation value of the connected domain pixel point b, the more the connected domain pixel point b has the characteristics of the welding defect of the gas hole, or the greater the probability that the connected domain pixel point b belongs to the welding defect area.

[0038] Since the crack welding defect is a linear fracture caused by welding stress or process defects, and the normal texture area is the organizational structure of the material itself, the crack will be a continuous fine line, without periodicity or repetition, while the normal texture is a fine granular, short linear or network structure, with regularity, such as the directionality of rolling texture, and the crack also has the characteristics of significant contrast with the surrounding matrix gray, clear boundary, obvious mutation of gray jump, etc. That is, the texture of the crack welding defect area is relatively uneven, and the gray feature of the neighborhood is relatively dissimilar, and the normal texture area has the characteristics of relatively flat gray change, low contrast with the adjacent area, and no obvious mutation of gray jump; therefore, the second defect representation value of the connected domain pixel point will be obtained by analyzing the texture characteristics and the gray jump characteristics in the following embodiment, and the second defect representation value of the connected domain pixel point is a key index for determining the target defect representation value of the connected domain pixel point, and the specific acquisition process of the second defect representation value of any connected domain pixel point b is as follows:

[0039] With any pixel point as the center, a rectangular window corresponding to the pixel point is constructed and recorded as a local window corresponding to the pixel point, and in specific applications, the implementer needs to set the size of the rectangular window according to the actual situation, such as setting it to an empirical value, such as 7x7; then the gray level co-occurrence matrix and the gray level histogram of the local window corresponding to the connected domain pixel point b are constructed, and the eigenvalues of the gray level co-occurrence matrix of the local window corresponding to the connected domain pixel point are calculated; then according to the eigenvalues of the gray level co-occurrence matrix of the local window corresponding to the connected domain pixel point b, the gray level histogram of the local window corresponding to the connected domain pixel point b, and the similarity between the local window corresponding to the connected domain pixel point b and the local window corresponding to the neighborhood pixel point of the connected domain pixel point b, the second defect characteristic value of the connected domain pixel point b is obtained.

[0040] In this embodiment, the specific process of obtaining the second defect characteristic value of the connected domain pixel point b according to the eigenvalues of the gray level co-occurrence matrix of the local window corresponding to the connected domain pixel point b, the gray level histogram of the local window corresponding to the connected domain pixel point b, and the similarity between the local window corresponding to the connected domain pixel point b and the local window corresponding to the neighborhood pixel point of the connected domain pixel point b is as follows:

[0041] First, according to the energy and uniformity of the gray level co-occurrence matrix of the local window corresponding to the connected domain pixel point b and the frequency difference of the same gray value appearing in the local window corresponding to the connected domain pixel point b and the local window corresponding to the neighborhood pixel point of the connected domain pixel point b, the target texture distribution characteristic value of the connected domain pixel point b is obtained; then, according to the gray level histogram of the local window corresponding to the connected domain pixel point b, the gray level jump degree of the connected domain pixel point b is obtained; finally, the product of the target texture distribution characteristic value and the gray level jump degree of the connected domain pixel point b is calculated, and is taken as the second defect characteristic value of the connected domain pixel point b, and the greater the target texture distribution characteristic value of the connected domain pixel point b, the more uneven the local texture distribution of the connected domain pixel point b, the more similar the local gray distribution of adjacent pixels, the more the connected domain pixel point b has the characteristics of crack welding defects, the greater the gray level jump degree of the connected domain pixel point b, the greater the gray level span in the local window of the connected domain pixel point b, the more the connected domain pixel point b has the characteristics of crack welding defects, and the greater the target texture distribution characteristic value and the gray level jump degree of the connected domain pixel point b, the greater the second defect characteristic value of the connected domain pixel point b, so the greater the second defect characteristic value of the connected domain pixel point b, the more obvious the crack welding defect characteristics of the connected domain pixel point b, or the greater the probability that the connected domain pixel point b belongs to the welding defect region.

[0042] In this embodiment, the specific process of obtaining the target texture distribution characteristic value of the connected domain pixel point b is as follows:

[0043] First, the product of the energy and the uniformity of the gray level co-occurrence matrix of the local window corresponding to the connected domain pixel b is calculated, and the result of the negative normalization of the product of the energy and the uniformity is recorded as the first texture distribution representation value of the connected domain pixel b. The greater the energy and the uniformity, the smaller the first texture distribution representation value, the more uniform the local texture distribution of the connected domain pixel b, and the energy is also called the angular second moment, and the uniformity is also called the inverse difference. Then, any pixel point in the local window corresponding to the connected domain pixel b except the connected domain pixel b is selected as the neighborhood pixel point of the connected domain pixel b, the types of all gray values appearing in the local window corresponding to the connected domain pixel b and the local window corresponding to the neighborhood pixel point of the connected domain pixel b are counted, and the set constructed by the counted types of all gray values appearing is recorded as a comprehensive set. The frequency difference value corresponding to each gray value in the comprehensive set is calculated, and the frequency difference value corresponding to any gray value a in the comprehensive set is the absolute value of the difference between the frequency of the gray value a appearing in the local window corresponding to the connected domain pixel b and the frequency of the gray value a appearing in the local window corresponding to the neighborhood pixel point of the connected domain pixel b. That is, for the gray value 25 in the comprehensive set, the frequency of the gray value 25 appearing in the local window corresponding to the connected domain pixel b is 5, the frequency of the gray value 25 appearing in the local window corresponding to the neighborhood pixel point of the connected domain pixel b is 1, and then the frequency difference value corresponding to the gray value 25 is 4. Then, the result of the accumulation and normalization of the frequency difference values corresponding to all gray values in the comprehensive set is calculated, and is recorded as the second texture distribution representation value of the connected domain pixel b. The normalization function Norm() is used here. The greater the second texture distribution representation value, the greater the difference between the gray characteristics of the connected domain pixel b and the neighborhood pixel point, which indicates that the gray characteristics of the connected domain pixel b and the neighborhood pixel point are less similar.

[0044] In the embodiment, the specific acquisition process of the gray level jump degree of the connected domain pixel point b is obtained according to the gray level histogram of the local window corresponding to the connected domain pixel point b. The gray level histogram of the local window corresponding to the connected domain pixel point b is denoted as an analysis histogram. The middle gray level on the analysis histogram is obtained. The middle gray level on the analysis histogram refers to the middle gray level between the maximum gray level and the minimum gray level on the analysis histogram, or the middle gray level on the analysis histogram refers to one half of the maximum gray level on the analysis histogram, and the vertical coordinates of the maximum gray level and the minimum gray level on the analysis histogram are not 0. In the embodiment, the middle gray level is required to be an integer. If one half of the maximum gray level is not an integer, the integer part of one half of the maximum gray level is taken as the middle gray level on the analysis histogram. The number of gray levels in the construction of the analysis histogram is set by the implementer according to experience. The maximum vertical coordinate value corresponding to the gray level on the left side of the middle gray level on the analysis histogram is obtained and denoted as a left representative gray level. The maximum vertical coordinate value corresponding to the gray level on the right side of the middle gray level on the analysis histogram is obtained and denoted as a right representative gray level. The acquisition process of the histogram is known. Finally, the absolute value of the difference between the left representative gray level and the right representative gray level is normalized and taken as the gray level jump degree of the connected domain pixel point b. The normalization is also performed by using the normalization function Norm (). The greater the absolute value of the difference between the left representative gray level and the right representative gray level, the greater the degree of gray level jump in the local window of the connected domain pixel point b, and the more the connected domain pixel point b has the characteristics of the crack welding defect.

[0045] In the embodiment, the specific calculation expression of the second defect representation value of the connected domain pixel point b is:

[0046] ;

[0047] wherein, is the second defect representation value of the connected domain pixel point b, EH is the first texture distribution representation value of the connected domain pixel point b, Norm () is a normalization function, K is a comprehensive set, is the frequency difference value corresponding to the kth gray value in the comprehensive set, G1 is the left representative gray level, and G2 is the right representative gray level, is the second texture distribution representation value of the connected domain pixel point b, is the target texture distribution representation value of the connected domain pixel point b, is the gray level jump degree of the connected domain pixel point b; and is greater, and is greater, is greater, and The greater the value of the target defect feature value of the connected domain pixel point b is, the more obvious the welding defect feature of the connected domain pixel point b is, and the greater the possibility of the connected domain pixel point b belonging to the welding defect is. The smaller the value of the target defect feature value of the connected domain pixel point b is, the less obvious the welding defect feature of the connected domain pixel point b is, and the smaller the possibility of the connected domain pixel point b belonging to the welding defect is.

[0048] Since the first defect feature value and the second defect feature value can reflect the possibility of the connected domain pixel point belonging to the welding defect, the embodiment takes the mean value of the first defect feature value and the second defect feature value of each connected domain pixel point as the target defect feature value of the corresponding connected domain pixel point, that is, the target defect feature value of the connected domain pixel point b is the mean value of the first defect feature value of the connected domain pixel point b and the second defect feature value of the connected domain pixel point b. The greater the target defect feature value of the connected domain pixel point b is, the more obvious the welding defect feature of the connected domain pixel point b is, and the greater the possibility of the connected domain pixel point b belonging to the welding defect is. And based on the above, it is known that the target defect feature value of the connected domain pixel point is obtained according to the area of the to-be-analyzed connected domain containing the connected domain pixel point and the area of the minimum circumscribed ellipse, the curvature, gradient and normal direction of the edge pixel point on the to-be-analyzed connected domain containing the connected domain pixel point, and the local window corresponding to the connected domain pixel point.

[0049] In step S004, a suspected welding defect clustering cluster is obtained according to the coordinates, suspected abnormal feature value and target defect feature value of the connected domain pixel point, and a structure element of each pixel point in the to-be-processed welding image is obtained according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster. The bottom-hat operation is performed on the to-be-processed welding image according to the structure element of the pixel point, and a target welding image is obtained. The welding workpiece is detected and identified for welding defects according to the target welding image.

[0050] Since the welding defect area is relatively small and the pixel points in the welding defect area are relatively concentrated, after obtaining the suspected abnormal feature value and the target defect feature value, the embodiment performs mean shift clustering on all connected domain pixel points based on the coordinates, suspected abnormal feature value and target defect feature value of each connected domain pixel point on the to-be-processed welding image, to obtain each clustering cluster. When clustering, the distance measurement between the connected domain pixel points is the Euclidean distance between the feature vectors of the connected domain pixel points, and the feature vector of the connected domain pixel point is composed of the coordinates, suspected abnormal feature value and target defect feature value of the corresponding connected domain pixel point. The process of mean shift clustering is known.

[0051] After the clustering is completed, the suspected welding defect cluster is obtained according to the suspected abnormality feature values and the target defect feature values of the connected domain pixel points in each cluster and the density of the connected domain pixel points in the cluster, and the suspected welding defect cluster is the key to determining the parameter value of the structure element subsequently, and then the specific acquisition process of the suspected welding defect cluster is as follows:

[0052] For any cluster: obtain the total number of pixel points in the cluster and the area of the minimum circumscribed circle of the cluster, calculate the ratio of the total number of pixel points in the cluster to the area of the minimum circumscribed circle of the cluster, and record it as the characteristic ratio of the cluster. The characteristic ratio can reflect the density of connected domain pixel points in the cluster. The greater the characteristic ratio, the greater the density of connected domain pixel points in the cluster, and the more likely the cluster is a suspected welding defect cluster. Add the mean of the suspected abnormality feature values of all connected domain pixel points in the cluster to the mean of the target defect feature values of all connected domain pixel points in the cluster, and record the result as the comprehensive feature value of the cluster. The normalized result of multiplying the comprehensive feature value of the cluster by the characteristic ratio of the cluster is used as the target abnormality degree feature value of each connected domain pixel point in the cluster. The target abnormality degree feature values of each connected domain pixel point in the same cluster are the same, i.e. the target abnormality degree feature value of any connected domain pixel point in the cluster is the normalized result of multiplying the comprehensive feature value of the cluster by the characteristic ratio of the cluster. The comprehensive feature value of the cluster is the sum of the corresponding suspected abnormality mean and target defect mean. The corresponding suspected abnormality mean of the cluster is the mean of the suspected abnormality feature values of all connected domain pixel points in the corresponding cluster. The corresponding target defect mean of the cluster is the mean of the target defect feature values of all connected domain pixel points in the corresponding cluster. Here, the normalization function Norm() is used for normalization. Calculate the mean of the target abnormality degree feature values of all connected domain pixel points in the cluster, and use it as the abnormality degree judgment index value of the cluster. Since the target abnormality degree feature values of all connected domain pixel points in the cluster are consistent, the normalized result of multiplying the comprehensive feature value of the cluster by the characteristic ratio of the cluster can also be directly used as the abnormality degree judgment index value of the cluster. The greater the abnormality degree judgment index value of the cluster, the greater the density of suspected welding defect pixel points in the cluster, and the greater the probability that the cluster is a suspected welding defect cluster. Therefore, it is determined whether the abnormality degree judgment index value of the cluster is greater than a preset abnormality threshold. If yes, the cluster is determined to be a suspected welding defect cluster, i.e. the probability that the cluster belongs to a welding defect region is greater. However, it cannot be accurately stated that the cluster is a welding defect cluster, so the cluster is recorded as a suspected welding defect cluster. Otherwise, it is determined that the cluster is not a suspected welding defect cluster, but a normal region. In specific applications, the implementer needs to set the preset abnormality threshold according to the actual situation such as the rounding range of the abnormality degree judgment index value. In this embodiment, the preset abnormality threshold is set to 0.7.

[0053] After the suspected welding defect clustering cluster is obtained, the structural element parameter value is adaptively obtained based on the suspected welding defect clustering cluster, that is, after the suspected welding defect clustering cluster is obtained, the structural element of each pixel point on the welding image to be processed is obtained according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster. The specific obtaining process is as follows:

[0054] For any pixel point in the welding image to be processed: if it is judged that the pixel point does not belong to the suspected welding defect clustering cluster, the preset structural element parameter value is taken as the structural element parameter value of the pixel point; if it is judged that the pixel point belongs to the suspected welding defect clustering cluster, the structural element parameter value of the pixel point is obtained according to the diameter of the minimum circumscribed circle of the clustering cluster to which the pixel point belongs, and the structural element parameter value is obtained according to the rule that the larger the diameter of the minimum circumscribed circle of the clustering cluster to which the pixel point belongs, the larger the structural element parameter value of the corresponding pixel point, that is, the structural element parameter value of the pixel point is taken as the structural element parameter value of the pixel point, c is an adjustment coefficient, c is used to make the determined structural element size cover the welding defect area, so as to achieve the purpose of enhancing the welding defect area, and in this embodiment, c is set as an empirical value, for example, c is set as 1.5, r is the diameter of the minimum circumscribed circle of the clustering cluster to which the pixel point belongs, is a down rounding symbol; the structural element of the pixel point is VxV, and V is the structural element parameter value of the pixel point, that is, the size of the structural element of the pixel point is VxV. In addition, the way of determining the structural element of the pixel point in this embodiment can effectively enhance all welding defect areas as much as possible, that is, it can effectively enhance all welding defect areas as much as possible or can effectively avoid the problem of insufficient enhancement of the welding defect area.

[0055] It should be noted that when is an even number, in order to ensure the effectiveness of the size, the result of is added by a constant 1 to be taken as the structural element parameter value of the pixel point. In addition, in specific applications, the implementer needs to set the preset structural element parameter value according to the actual situation, for example, in this embodiment, the preset structural element parameter value is set as an empirical value, for example, it is set as 5, that is, when the pixel point does not belong to the suspected welding defect clustering cluster, the size of the structural element of the pixel point is 5x5, and it is required that the set preset structural element parameter value can take into account noise suppression and detail retention; as other implementation manners, the structural element of each pixel point on the welding image to be processed can also be obtained according to the side length of the minimum circumscribed rectangle of the suspected welding defect clustering cluster, and when the structural element of each pixel point on the welding image to be processed is obtained according to the side length of the minimum circumscribed rectangle of the suspected welding defect clustering cluster, r represents the side length of the minimum circumscribed rectangle.

[0056] After obtaining the structure elements of each pixel point in the to-be-processed welding image, bottom-hat operation is performed on the to-be-processed welding image according to the structure elements of each pixel point, and the image after the bottom-hat operation is recorded as a target welding image. The process of performing bottom-hat operation on the to-be-processed welding image under the premise of knowing the size of the structure element of the pixel point is known. Then, the welding workpiece is detected and recognized for welding defects according to the target welding image and the welding defect network. That is, the trained welding defect network is obtained, the target welding image is input into the trained welding defect network, and the welding defect region on the target welding image is output. That is, the welding defect network can mark the position of the welding defect region on the target welding image, and the image after the bottom-hat operation can improve the accuracy and efficiency of defect region recognition and detection. The welding defect network is a convolutional neural network (such as a CNN network) or a target detection model (such as a YOLO or Faster R-CNN model). The training process of the welding defect network is a known technology, and therefore, the specific description of the embodiment is not given.

[0057] Up to now, the detection and recognition of welding defects of the welding workpiece are completed in the embodiment. That is, the structure element is adaptively obtained according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster in the embodiment, which can make the bottom-hat operation better, the enhancement effect of the dark area better, and the accuracy of welding defect detection and recognition based on the image with better dark area enhancement effect higher.

[0058] In summary, the to-be-processed welding image of the welding workpiece and the to-be-analyzed connected domain on the to-be-processed welding image are obtained in the embodiment. Then, the suspected abnormality characteristic value of the connected domain pixel point is obtained according to the gray value of the connected domain pixel point and the gray histogram of the to-be-processed welding image. After that, the target defect characteristic value of the connected domain pixel point is obtained according to the area of the minimum circumscribed ellipse of the to-be-analyzed connected domain containing the connected domain pixel point, the curvature, gradient and normal direction of the edge pixel point on the to-be-analyzed connected domain containing the connected domain pixel point, and the local window corresponding to the connected domain pixel point. Then, the suspected welding defect clustering cluster is obtained according to the coordinates, suspected abnormality characteristic value and target defect characteristic value of the connected domain pixel point. The structure element of each pixel point in the to-be-processed welding image is obtained according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster. The bottom-hat operation is performed on the to-be-processed welding image according to the structure element of the pixel point, and the target welding image is obtained. Finally, the welding workpiece is detected and recognized for welding defects according to the target welding image. The structure element is adaptively obtained according to the diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster in the embodiment, which can improve the effect of bottom-hat operation, the enhancement effect of the dark area, and the accuracy of detection and recognition of welding defects.

[0059] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A machine vision-based industrial welding quality inspection method, characterized by, The method comprises the following steps: Obtaining a to-be-processed welding image of a welding workpiece and a to-be-analyzed connected domain on the to-be-processed welding image, wherein each point in the to-be-analyzed connected domain is referred to as a connected domain pixel point; Obtaining a suspected abnormality characteristic value of the connected domain pixel point according to a gray value of the connected domain pixel point and a gray histogram of the to-be-processed welding image; Obtaining a target defect characteristic value of the connected domain pixel point according to an area of a minimum circumscribed ellipse of the to-be-analyzed connected domain containing the connected domain pixel point, a curvature, a gradient and a normal direction of an edge pixel point on the to-be-analyzed connected domain containing the connected domain pixel point, and a local window corresponding to the connected domain pixel point; Clustering all connected domain pixel points according to coordinates, suspected abnormality characteristic values and target defect characteristic values of the connected domain pixel points to obtain each clustering cluster, taking a ratio of a total number of pixel points in the clustering cluster to an area of a minimum circumscribed circle of the clustering cluster as a feature ratio, and taking a normalized result of a multiplication of a mean value of suspected abnormality characteristic values of all connected domain pixel points in the clustering cluster and a mean value of target defect characteristic values of all connected domain pixel points in the clustering cluster as an abnormality degree judgment index value of the clustering cluster, wherein if the abnormality degree judgment index value of the clustering cluster is greater than a preset abnormality threshold value, the clustering cluster is recorded as a suspected welding defect clustering cluster, and a structure element of each pixel point in the to-be-processed welding image is obtained according to a diameter of the minimum circumscribed circle of the suspected welding defect clustering cluster, and a bottom-hat operation is performed on the to-be-processed welding image according to the structure element of the pixel point to obtain a target welding image; Performing welding defect detection and identification on the welding workpiece according to the target welding image; The method for obtaining the structure element of each pixel point in the to-be-processed welding image comprises the following steps: for any pixel point in the to-be-processed welding image, if the pixel point does not belong to a suspected welding defect clustering cluster, a preset structure element parameter value is taken as a structure element parameter value of the corresponding pixel point, and if the pixel point belongs to the suspected welding defect clustering cluster, a structure element parameter value of the pixel point is obtained according to a diameter of a minimum circumscribed circle of a clustering cluster to which the pixel point belongs, V×V is taken as the structure element of the pixel point, and V is the structure element parameter value of the pixel point, wherein the greater the diameter of the minimum circumscribed circle of the clustering cluster to which the pixel point belongs, the greater the structure element parameter value of the pixel point.

2. The machine vision-based industrial weld quality inspection method of claim 1, wherein, The method for obtaining the suspected abnormality characteristic value of the connected domain pixel point comprises the following steps: For any connected domain pixel point, the gray level containing the gray value of the connected domain pixel point and the pixel number corresponding to the gray level containing the gray value of the connected domain pixel point are obtained on the gray histogram of the to-be-processed welding image, and are respectively denoted as the gray level corresponding to the connected domain pixel point and the pixel number representing value of the gray level corresponding to the connected domain pixel point. The relative gray size representing value is denoted as the result of subtracting the gray level corresponding to the connected domain pixel point from the maximum gray level on the gray histogram of the to-be-processed welding image. The product of the result of the relative gray size representing value and the negative correlation mapping of the gray level corresponding to the connected domain pixel point is taken as the suspected abnormality representing value of the connected domain pixel point.

3. The machine vision-based industrial weld quality inspection method of claim 1, wherein, The method for obtaining the target defect representing value of the connected domain pixel point comprises: For any connected domain pixel point: the to-be-analyzed connected domain containing the connected domain pixel point is denoted as connected domain A. The first defect representing value of the connected domain pixel point is obtained according to the area of the connected domain A, the area of the minimum circumscribed ellipse of the connected domain A, and the curvature, gradient and normal direction of each edge pixel point in the connected domain A. The second defect representing value of the connected domain pixel point is obtained according to the eigenvalue of the gray co-occurrence matrix of the local window corresponding to the connected domain pixel point, the gray histogram of the local window corresponding to the connected domain pixel point, and the similarity between the local window corresponding to the connected domain pixel point and the local window corresponding to the neighborhood pixel point of the connected domain pixel point. The local window corresponding to any pixel point is a rectangular window constructed with the corresponding pixel point as the center. The mean value of the first defect representing value and the second defect representing value is taken as the target defect representing value of the connected domain pixel point.

4. The machine vision-based industrial weld quality inspection method of claim 3, wherein, The method for obtaining the first defect representing value of the connected domain pixel point comprises: The ratio of the area of the minimum circumscribed ellipse of the connected domain A to the area of the connected domain A is denoted as a first ratio. The absolute value of the difference between the constant 1 and the first ratio is negatively normalized to obtain a first index value. The standard deviation of the curvatures of all edge pixel points on the connected domain A is negatively normalized to obtain a second index value. The to-be-analyzed window corresponding to each edge pixel point in the connected domain A is constructed along the normal direction of the corresponding edge pixel point with the edge pixel point as the starting point. The third index value is obtained according to the to-be-analyzed window corresponding to each edge pixel point in the connected domain A. The product of the first index value, the second index value and the third index value is taken as the first defect representing value of the connected domain pixel point.

5. The machine vision-based industrial weld quality inspection method of claim 4, wherein, The method for obtaining the third index value comprises: The gray mean value of the remaining pixel points in the to-be-analyzed window corresponding to the edge pixel point except the corresponding edge pixel point is denoted as the neighborhood mean value of the corresponding edge pixel point. The edge feature value of the corresponding edge pixel point is denoted as the result of adding the gradient value of the corresponding edge pixel point to the absolute value of the difference between the gray value of the corresponding edge pixel point and the neighborhood mean value of the corresponding edge pixel point. The mean value of the edge feature values of all edge pixel points in the connected domain A is normalized to obtain the third index value.

6. The machine vision-based industrial weld quality inspection method of claim 5, wherein, The method for obtaining the second defect representation value of the connected domain pixel point comprises the following steps: According to the energy and uniformity of the gray level co-occurrence matrix of the local window corresponding to the connected domain pixel point, and the frequency difference of the same gray value appearing in the local window corresponding to the connected domain pixel point and the local window corresponding to the neighborhood pixel point of the connected domain pixel point, a target texture distribution representation value of the connected domain pixel point is obtained; according to the gray level histogram of the local window corresponding to the connected domain pixel point, a gray level jump degree of the connected domain pixel point is obtained; and the product of the target texture distribution representation value and the gray level jump degree is taken as the second defect representation value of the connected domain pixel point.

7. The machine vision-based industrial weld quality inspection method of claim 6, wherein, The method for obtaining the target texture distribution representation value of the connected domain pixel point comprises the following steps: The product of the energy and uniformity of the gray level co-occurrence matrix of the local window corresponding to the connected domain pixel point is negatively normalized, and the result is taken as a first texture distribution representation value; A set constructed by all the gray value types appearing in the local window corresponding to the connected domain pixel point and the local window corresponding to the neighborhood pixel point of the connected domain pixel point is taken as a comprehensive set, and the frequency difference value corresponding to each gray value in the comprehensive set is obtained; the frequency difference value corresponding to any gray value a in the comprehensive set is the absolute value of the difference between the frequency of the gray value a appearing in the local window corresponding to the connected domain pixel point and the frequency of the gray value a appearing in the local window corresponding to the neighborhood pixel point of the connected domain pixel point; the result of accumulating and then normalizing all the frequency difference values corresponding to the gray values in the comprehensive set is taken as a second texture distribution representation value; The mean value of the first texture distribution representation value and the second texture distribution representation value is taken as the target texture distribution representation value of the connected domain pixel point.

8. The machine vision-based industrial weld quality inspection method of claim 6, wherein, The method for obtaining the gray level jump degree of the connected domain pixel point comprises the following steps: The gray level histogram of the local window corresponding to the connected domain pixel point is taken as an analysis histogram, the gray level corresponding to the maximum ordinate value on the left side of the middle gray level on the analysis histogram is taken as a left representative gray level, the gray level corresponding to the maximum ordinate value on the right side of the middle gray level on the analysis histogram is taken as a right representative gray level, and the absolute value of the difference between the left representative gray level and the right representative gray level is normalized to obtain the gray level jump degree of the connected domain pixel point.

Images