Weldment surface defect intelligent detection method based on 3D visual sensing

Abstract

The invention discloses an intelligent weldment surface defect detection method based on 3D visual sensing, relates to the technical field of weldment detection, and solves the problem of low recognition accuracy of a surface defect corresponding to a target weldment. The method comprises the following steps: fixing the target weldment on a static platform, and collecting weldment outer surface images of the target weldment under multiple visual angles; determining a weld contour type of a weld corresponding to the target weldment according to the outer surface image; selecting different scanning modes according to the weld contour type of the weld corresponding to the target weldment to obtain a point cloud data set of the weld in the target weldment; analyzing the point cloud data set of the welding seam in the target weldment, and identifying the welding seam defect of the target weldment; according to the method, the existence condition of the welding fault defect corresponding to the target weldment is known according to the detection result of the surface of the target weldment, the welding seam corresponding to the target weldment is accurately scanned in combination with 3D vision, and accurate recognition of the surface defect corresponding to the target weldment is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of weldment recognition, and specifically relates to an intelligent detection method for surface defects of weldments based on 3D vision sensing. Background Art

[0002] A weldment refers to a component or structural part formed by connecting two or more metal (or non-metal) materials together through a welding process, having a certain shape, size, and performance; it is a common basic component in manufacturing and is widely used in fields such as machinery, construction, shipbuilding, automobiles, aerospace, etc.; when there are mistakes in the welding operation of a weldment, weldment defects may be difficult to detect in the initial stage, but will rapidly expand under long-term use or specific working conditions, resulting in structural failure and property losses; therefore, it is necessary to accurately detect the defects of weldments. However, at the present stage, when identifying the surface defects of weldments, it is impossible to accurately locate the position of the weld seam, and thus it is impossible to collect targeted data for different weld seam types. Therefore, there are deviations in data collection, resulting in large errors in identifying the surface defects of weldments and unable to ensure the accuracy of defect detection. Therefore, the present invention proposes an intelligent detection method for surface defects of weldments based on 3D vision sensing. Summary of the Invention

[0003] The purpose of the present invention is to propose an intelligent detection method for surface defects of weldments based on 3D vision sensing to solve the problems raised in the above background art.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions: An intelligent detection method for surface defects of weldments based on 3D vision sensing, the method includes: Step S1, fixing the target weldment on a stationary platform and collecting the outer surface images of the target weldment from multiple perspectives. Step S2, determining the weld contour type of the corresponding weld seam of the target weldment based on the outer surface images. Step S3, selecting different scanning methods according to the weld contour type of the corresponding weld seam of the target weldment to obtain a point cloud data set of the weld seam in the target weldment. Step S4, analyzing the point cloud data set of the weld seam in the target weldment to identify the weld defects of the target weldment. Step S5, determining the existence of welding fault defects corresponding to the target weldment based on the detection results on the surface of the target weldment.

[0005] Further, the step S2 includes the following sub-steps: Step S21: Obtain the outer surface images of the target welded part from multiple perspectives, and splice the outer surface images of the welded part to obtain the spliced outer surface image of the target welded part; Step S22: Convert the spliced outer surface image of the target welded part into a grayscale image. The conversion process of the grayscale image is as follows: Step S221: Obtain the pixel values of multiple pixel points in the spliced outer surface image, and divide the pixel value of each pixel point into an R value component Ri, a G value component Gi, and a B value component Bi; where i is the number of the pixel point, i = 1, 2,..., z, and z is the upper limit value of the number; Step S222: Calculate the pixel grayscale value XHi of each pixel point through the formula. The formula is as follows; XHi = Ri × A1 + Gi × A2 + Bi × A3; where A1, A2, and A3 in the formula are weight coefficients, and A2 > A1 > A3; Step S223: Compare the pixel grayscale value of each pixel point with the pixel grayscale threshold; if the pixel grayscale value of the pixel point is greater than or equal to the pixel grayscale threshold, set the corresponding pixel point to black; if the pixel grayscale value of the pixel point is less than the pixel grayscale threshold, set the corresponding pixel point to white; Step S224: Connect multiple groups of adjacent black pixel points and record them as the to-be-determined weld contours to obtain a binary contour image.

[0006] Further, step S2 further includes the following sub-steps: Step S23: Obtain the spliced outer surface image of the target welded part, and identify the coordinates of any weld pixel point located in the weld in the spliced outer surface image; match the coordinates of the weld pixel point with all the to-be-determined weld contours in the binary contour image; if the coordinates match successfully, record the corresponding to-be-determined weld contour as the screened weld contour of the target welded part; if the coordinates match fails, generate an abnormal signal; Step S24: Match the screened weld contour with the standard weld contour; Step S25: If the screened weld contour matches the first-class standard weld contour successfully, record the screened weld contour as the first-class weld contour; if the screened weld contour matches the second-class standard weld contour successfully, record the screened weld contour as the second-class weld contour; if the screened weld contour fails to match both the first-class standard weld and the second-class standard weld, record the screened weld contour as the third-class weld contour.

[0007] Further, the process of the contour matching is as follows: Step S241: Set multiple contour matching pixel points at equal intervals in the screened weld contour, and then collect the coordinates of each contour matching pixel point, and record the number of the contour matching pixel point as n; Step S242: Arbitrarily select a contour matching pixel point near the geometric center of the weld seam and denote it as the selected pixel point; Step S243: Set the number of radial bins r and the number of angular bins θ in the filtered weld seam contour with the selected pixel point as the origin, divide the filtered weld seam contour into multiple bins, and the number of bins corresponding to the filtered weld seam contour is obtained by multiplying the number of radial bins by the number of angular bins.

[0008] Furthermore, the process of contour matching further includes: Step S244: Calculate the filtered bin value FS(f) of the f-th bin in the filtered weld seam contour = the number of contour matching pixel points falling into the f-th bin ÷ (the number of contour matching pixel points - 1); where f is the number of different bins, f = 1, 2, ……, r×θ; Step S245: Similarly calculate the standard bin value FBn(f) corresponding to the f-th bin of the standard weld seam contour; Step S246: Compare the filtered bin value with the standard bin value. If the absolute value of the difference between all filtered bin values and the standard bin values is less than or equal to the preset threshold, it is determined that the matching is successful. If the absolute value of the difference between any one filtered bin value and the standard bin value is greater than the preset threshold, it is determined that the matching fails.

[0009] Furthermore, step S4 includes the following sub-steps: Step S41: Obtain the point cloud data set of the weld seam in the target welded part; Step S42: Perform a downsampling operation on the point cloud data set to obtain a voxel point cloud set; specifically, the downsampling operation is as follows: Step S421: Obtain the point cloud range [Xmin, Xmax]×[Ymin, Ymax]×[Zmin, Zmax] from the coordinates of different scanned pixel points in the point cloud data set. Assume the voxel side length is v, and obtain multiple voxels within the point cloud range; Step S422: Calculate the index s(x', y', z') of the corresponding voxel, and the specific calculation formula is as follows: x' = (X - Xmin) / v; y' = (Y - Ymin) / v; z' = (Z - Zmin) / v; where (X, Y, Z) are the coordinates of the center point corresponding to the voxel; Step S43: Perform a denoising operation on the voxel point cloud set to obtain a denoised voxel point cloud set; Step S44: Perform surface reconstruction based on the denoised voxel point cloud set and extract the mesh.

[0010] Furthermore, step S4 also includes the following sub-steps: Step S45: For each point in the triangular mesh, calculate the covariance matrix of its local neighborhood, and determine the curvature by the ratio of the minimum eigenvalue to the sum of eigenvalues: Curvature = TZ1 ÷ (TZ1 + TZ2 + TZ3); where TZ1 ≤ TZ2 ≤ TZ3, and TZ1, TZ2, and TZ3 are the eigenvalues of the covariance matrix; Step S46: For each point s in the triangular mesh, calculate the height difference GCs between its height and the average height of k neighboring points; ; where q is the number of the k neighboring points; Step S47: For any point in the triangular mesh, if the height difference of the corresponding point is greater than or equal to the height difference threshold or the curvature is greater than or equal to the curvature threshold, it is determined that there is a welding defect, and a defect signal is generated; if the height differences of all points are less than the height difference threshold and the curvature is less than the curvature threshold, it is determined that there is no welding defect in the target weldment, and a normal signal is generated.

[0011] Furthermore, the denoising operation is specifically as follows: Step S431: Select any voxel point ps, calculate the Euclidean distances {OS1, OS2, ……, OSk} from the voxel point ps to k nearest neighboring voxel points, sum up all the Euclidean distances corresponding to the voxel points and take the mean to obtain the calculated Euclidean distance of the corresponding voxel point; Step S432: Similarly, calculate the calculated Euclidean distances of all voxel points, sum up the calculated Euclidean distances of all voxel points and take the mean to obtain the average comprehensive distance of all voxel points; Step S433: Calculate the standard deviation of the average comprehensive distance of all voxel points based on the average comprehensive distance and the calculated Euclidean distance of each voxel point; Step S434: Set the denoising weight coefficient, add the average comprehensive distance plus the product of the standard deviation of the average comprehensive distance and the denoising weight coefficient as the discrete denoising threshold; compare the average Euclidean distance of the voxel point with the discrete denoising threshold; Step S435: If the average Euclidean distance of the voxel point is greater than or equal to the discrete denoising threshold, discard the corresponding voxel point; if the average Euclidean distance of the voxel point is less than the discrete denoising threshold, include the voxel point in the set to obtain the denoised voxel point cloud set.

[0012] Furthermore, the process of surface reconstruction is specifically as follows: Step S441: Select any voxel point pu, and select k neighboring points {pu1, pu2, ……, puk} corresponding to the voxel point pu; where u is the number of the voxel point in the denoised voxel point cloud set; Step S442: Calculate the covariance matrix JZu corresponding to the voxel point pu through the formula. The specific formula is as follows: ; wherein, , m is the serial number of the neighborhood points, and T represents the transpose of the matrix; the eigenvalues TZ1, TZ2, and TZ3 of the covariance matrix are obtained.

[0013] Furthermore, the process of the surface reconstruction further includes: Step S443, define an indicator function ZS(t), the indicator function is one inside the denoised voxel point cloud set and zero outside the denoised voxel point cloud set; then the normal vector field F(t) corresponding to the denoised voxel point cloud set satisfies: ▽ZS(t)=F(t); wherein, t is the independent variable, and the coordinate value is substituted during calculation; Step S444, solve the Poisson equation: ▽ 2 ZS(t)=▽F(t); set the isosurface ZS(t)=λ Step S445, extract the triangular mesh by the marching cubes algorithm.

[0014] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are as follows: 1. The present invention first fixes the target welded part on a stationary platform, and collects the outer surface images of the target welded part from multiple perspectives; then determines the weld contour type of the corresponding weld of the target welded part according to the outer surface images; and then selects different scanning methods according to the weld contour type of the corresponding weld of the target welded part to obtain the point cloud data set of the weld in the target welded part, and the present invention realizes the accurate scanning of the corresponding weld of the target welded part.

[0015] 2. The present invention analyzes the point cloud data set of the weld in the target welded part to identify the weld defects of the target welded part; finally, according to the detection result on the surface of the target welded part, the existence situation of the corresponding welding fault defects of the target welded part is known, and the present invention realizes the accurate detection of the defects of the target welded part. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] For the convenience of understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0017] Figure 1 is the overall method flow chart of the present invention; Figure 2 is the schematic block diagram of the architecture of the method in the present invention; Figure 3 is the binning schematic diagram for screening weld contours in the present invention; Figure 4 is the schematic structural diagram of the voxel in the present invention; Figure 5 is the schematic structural diagram of the computer device in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] Next, the technical solution of the present invention will be clearly and completely described in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present invention.

[0019] Example 1, please refer to Figures 1-4 As shown, the technical solution provided by the present invention is: an intelligent detection method for surface defects of welded parts based on 3D vision sensing. By initially analyzing the outer surface image of the target welded part, setting the weld seam scanning method of the target welded part, and then obtaining the corresponding point cloud set of the target welded part; performing data processing on the point cloud set, and then determining the triangular mesh of the point cloud set through surface reconstruction, and then extracting the feature information of the triangular mesh, and judging the defect situation of the target welded part according to the feature information; In the present invention, the welded part to be detected is calibrated as the target welded part, and the intelligent detection method for surface defects of the welded part is specifically as follows: Step S1, fix the target welded part on a stationary platform and collect the outer surface images of the target welded part from multiple perspectives. Step S2, determine the weld contour type of the weld corresponding to the target welded part according to the outer surface image. Among them, the weld contour types include type I weld contour, type II weld contour, and type III weld contour; type I weld contour is the contour of the circumferential weld of the pipe, type II weld contour is the contour of the straight weld on the plane, and type III weld contour is the contour of the weld other than type I weld contour and type II weld contour; In this embodiment, step S2 includes the following sub-steps: Step S21, obtain the outer surface images of the target welded part from multiple perspectives, and splice the outer surface images of the welded part to obtain the spliced outer surface image of the target welded part; Step S22, convert the spliced outer surface image of the target welded part into a grayscale image. In this embodiment, the conversion process of the grayscale image is specifically as follows: Step S221, obtain the pixel values of multiple pixel points in the spliced outer surface image, and divide the pixel value of each pixel point into an R value component Ri, a G value component Gi, and a B value component Bi; where i is the number of the pixel point, i = 1, 2,..., z, and z is the upper limit value of the number; Step S222, calculate the pixel grayscale value XHi of each pixel point through the formula, and the formula is specifically as follows; XHi = Ri × A1 + Gi × A2 + Bi × A3; where A1, A2, and A3 are weighting coefficients, and A2 > A1 > A3; specifically, A1 can be 0.299, A2 can be 0.587, and A3 can be 0.114; Step S223: Compare the pixel gray value of each pixel with the pixel gray threshold; if the pixel gray value of the pixel is greater than or equal to the pixel gray threshold, set the corresponding pixel to black; if the pixel gray value of the pixel is less than the pixel gray threshold, set the corresponding pixel to white; Step S224: Connect multiple groups of adjacent black pixels and record them as the to-be-determined weld contours to obtain a binary contour image; Step S23: Obtain the splicing outer surface image of the target weldment, and identify the coordinates of any weld pixel located in the weld in the splicing outer surface image; match the coordinates of the weld pixel with all the to-be-determined weld contours in the binary contour image; if the coordinates match successfully, record the corresponding to-be-determined weld contour as the screened weld contour of the target weldment; if the coordinate match fails, generate an abnormal signal; Among them, the weld pixel is the pixel that has been determined to be in the weld; Step S24: Match the screened weld contour with the standard weld contour; where the standard weld contour includes a first-class standard weld contour and a second-class standard weld contour; Step S25: If the screened weld contour matches the first-class standard weld contour successfully, record the screened weld contour as the first-class weld contour; if the screened weld contour matches the second-class standard weld contour successfully, record the screened weld contour as the second-class weld contour; if the screened weld contour fails to match both the first-class standard weld and the second-class standard weld, record the screened weld contour as the third-class weld contour; In this embodiment, the process of contour matching is specifically as follows: Step S241: Set multiple contour matching pixels at equal intervals in the screened weld contour, and then collect the coordinates of each contour matching pixel, and record the number of the contour matching pixel as n; Step S242: Arbitrarily select a contour matching pixel near the geometric center of the weld and record it as the selected pixel; Step S243: As Figure 3 shown, set the radial bin number r and the angular bin number θ in the screened weld contour with the selected pixel as the origin, divide the screened weld contour into multiple bins, and multiply the radial bin number by the angular bin number to obtain the bin number corresponding to the screened weld contour; This step can be regarded as dividing the screened weld contour into multiple sub-regions according to a certain rule, and each sub-region is a bin; Among them, the number of radial bins and the number of angular bins are the core parameters for controlling the resolution and sensitivity of the feature descriptor. The number of radial bins is the number of distance quantization intervals, which is used to control the sensitivity of distance, and the number of angular bins is the number of direction quantization intervals, which is used to control the sensitivity of direction; Step S244, calculate the screening bin value FS(f) of the f-th bin in the screened weld profile: FS(f) = the number of contour matching pixel points falling into the f-th bin ÷ (the number of contour matching pixel points - 1); where f is the number of different bins, f = 1, 2,..., r×θ; Step S245, similarly calculate the standard bin value FBn(f) corresponding to the f-th bin of the standard weld profile; Step S246, compare the screening bin value with the standard bin value. If the absolute value of the difference between all screening bin values and the standard bin values is less than or equal to the preset threshold, it is determined that the matching is successful. If the absolute value of the difference between any screening bin value and the standard bin value is greater than the preset threshold, it is determined that the matching fails.

[0020] Step S3, select different scanning methods according to the weld profile type of the corresponding weld of the target weldment to obtain the point cloud data set of the weld in the target weldment; Among them, the point cloud data set is specifically the XYZ coordinates of the scanned pixel points in the corresponding weld of the target weldment; the scanned pixel points are the pixel points collected by laser scanning; in specific scanning, a six-axis robot can be selected to cooperate with a laser line scanning sensor to ensure that the scanning path covers all weld areas; For the selection of the scanning method, the equidistant spiral method is selected for scanning for the first type of weld profile; the Zig-Zag scanning method is selected for scanning for the second type of weld profile; for the third type of weld profile, estimate the curvature of the local part of the target weldment, and adjust the scanning density of the scanning line according to the estimated curvature; Specifically, ; where SM is the scanning density and QL is the curvature of the local part of the target weldment.

[0021] Step S4, analyze the point cloud data set of the weld in the target weldment to identify the weld defects of the target weldment; In this embodiment, the step S4 includes the following sub-steps: Step S41, obtain the point cloud data set of the weld in the target weldment; Step S42, perform a downsampling operation on the point cloud data set to obtain a voxel point cloud set; In this embodiment, the downsampling operation is specifically as follows: Step S421, please refer to Figure 4As shown, the point cloud range [Xmin, Xmax] × [Ymin, Ymax] × [Zmin, Zmax] is obtained from the coordinates of different scanned pixel points in the point cloud data set. Assuming the voxel side length is v, multiple voxels within the point cloud range are obtained; The principle of downsampling is as follows: The three-dimensional space is divided into uniform voxels (cubes), and only one representative point is retained within each voxel; Step S422, calculate the index s(x', y', z') of the corresponding voxel. The specific calculation formula is as follows: x' = (X - Xmin) / v; y' = (Y - Ymin) / v; z' = (Z - Zmin) / v; where (X, Y, Z) are the coordinates of the center point corresponding to the voxel; Step S43, perform a denoising operation on the voxel point cloud set to obtain a denoised voxel point cloud set; In this embodiment, the denoising operation is specifically as follows: Step S431, select any voxel point ps, calculate the Euclidean distances {OS1, OS2, ……, OSk} from the voxel point ps to k nearest neighboring voxel points, and sum up all the Euclidean distances corresponding to the voxel points and take the average to obtain the calculated Euclidean distance of the corresponding voxel point; Step S432, similarly calculate the calculated Euclidean distances of all voxel points, sum up the calculated Euclidean distances of all voxel points and take the average to obtain the average comprehensive distance of all voxel points; Step S433, calculate the standard deviation of the average comprehensive distance of all voxel points based on the average comprehensive distance and the calculated Euclidean distance of each voxel point; Step S434, set the denoising weight coefficient, and take the average comprehensive distance plus the standard deviation of the average comprehensive distance multiplied by the denoising weight coefficient as the discrete denoising threshold; compare the average Euclidean distance of the voxel point with the discrete denoising threshold; Step S435, if the average Euclidean distance of the voxel point is greater than or equal to the discrete denoising threshold, discard the corresponding voxel point; if the average Euclidean distance of the voxel point is less than the discrete denoising threshold, then incorporate the voxel point into the set to obtain a denoised voxel point cloud set; Step S44, perform surface reconstruction based on the denoised voxel point cloud set and extract the mesh; In this embodiment, the process of surface reconstruction is specifically as follows: Step S441, select any voxel point pu, and select k neighborhood points {pu1, pu2, ……, puk} corresponding to the voxel point pu; where u is the number of the voxel point in the denoised voxel point cloud set; Step S442, calculate the covariance matrix JZu corresponding to the voxel point pu through the formula. The specific formula is as follows: ; where where \(m\) is the number of neighboring points, and \(T\) represents the transpose of the matrix; the eigenvalues \(TZ1\), \(TZ2\), and \(TZ3\) of the covariance matrix are obtained; It should be additionally noted that the normal vector is the eigenvector corresponding to the minimum eigenvalue of the covariance matrix \(JZu\); In step S443, define the indicator function \(ZS(t)\), which is one inside the denoised voxel point cloud set and zero outside the denoised voxel point cloud set; then the normal vector field \(F(t)\) corresponding to the denoised voxel point cloud set approximately satisfies: \(\nabla ZS(t)=F(t)\); where \(t\) is the independent variable and the coordinate value is substituted during calculation; the inverted triangle symbol (\(\nabla\)) is called the Nabla operator, which is the core symbol in vector calculus and is used to represent gradient, divergence, and curl; In step S444, solve the Poisson equation: \(\nabla\) 2 \(ZS(t)=\nabla F(t)\); set the isosurface \(ZS(t)=\lambda\), and the preferred value of \(\lambda\) is 0.5; It should be noted that the process of solving the Poisson equation is denoted as Poisson reconstruction, which is the process of converting the point cloud normal vector field into an implicit surface function; In step S445, extract the triangular mesh through the marching cubes algorithm; where the triangular mesh is used to simulate the surface of a complex object; the marching cubes algorithm divides the space into cube cells, determines whether the surface passes through the cube according to the \(ZS(t)\) value at the vertices, and generates the corresponding triangular mesh; Specifically, the process of determining whether the surface passes through the cube is as follows: substitute the coordinates of the eight vertices corresponding to the cube into \(ZS(t)\) in turn, and compare the \(ZS(t)\) value with the isosurface \(\lambda\); If \(ZS(t)\geq\lambda\), mark the corresponding vertex as an internal point, and if \(ZS(t)<\lambda\), mark the corresponding vertex as an external point; the state combinations of the eight vertices match fifteen predefined basic triangular patch patterns, and the triangular mesh is extracted according to the basic triangular patch patterns; Exemplarily, if only one vertex is internal, one triangle is generated; if two adjacent vertices are internal, one quadrilateral (decomposed into two triangles) is generated In step S45, for each point in the triangular mesh, calculate the covariance matrix of its local neighborhood, and determine the curvature by the ratio of the minimum eigenvalue to the sum of the eigenvalues: Curvature \(=TZ1\div(TZ1 + TZ2 + TZ3)\); where \(TZ1\leq TZ2\leq TZ3\), and \(TZ1\), \(TZ2\), and \(TZ3\) are the eigenvalues of the covariance matrix; In step S46, for each point \(s\) in the triangular mesh, calculate the height difference \(GCs\) between its height and the average height of \(k\) neighboring points; ; where q is the number of k neighboring points; Step S47, for any point in the triangular mesh, if the height difference of the corresponding point is greater than or equal to the height difference threshold or the curvature is greater than or equal to the curvature threshold, it is determined that there is a welding defect and a defect signal is generated; if the height difference of all points is less than the height difference threshold and the curvature is less than the curvature threshold; it is determined that there is no welding defect in the target weldment and a normal signal is generated.

[0022] Step S5, based on the detection result on the surface of the target weldment, the presence of welding fault defects corresponding to the target weldment is known.

[0023] In this application, if there are corresponding calculation formulas, the above calculation formulas are all dimensionless and take their numerical values for calculation. For the coefficients such as weight coefficients and proportionality coefficients existing in the formulas, the magnitudes set are for obtaining a result value by quantifying each parameter. Regarding the magnitudes of the weight coefficients and proportionality coefficients, as long as the proportional relationship between the parameters and the result value is not affected.

[0024] Embodiment 2 Figure 5 It is a schematic structural diagram of a computer device. The computer device may include: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus. The processor can call the logical instructions in the memory to execute an intelligent detection method for weldment surface defects based on 3D vision sensing. The method includes: fixing the target weldment on a stationary platform, collecting the outer surface images of the target weldment from multiple perspectives; determining the weld profile type of the weld corresponding to the target weldment based on the outer surface images; selecting different scanning methods according to the weld profile type of the weld corresponding to the target weldment to obtain the point cloud data set of the weld in the target weldment; analyzing the point cloud data set of the weld in the target weldment to identify the weld defects of the target weldment; based on the detection result on the surface of the target weldment, the presence of welding fault defects corresponding to the target weldment is known.

[0025] In addition, when the logical instructions in the above-mentioned memory are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0026] On the other hand, this application also provides a computer program product. The computer program product includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute a method for intelligent detection of surface defects of welded parts based on 3D vision sensing provided above. The method includes: fixing a target welded part on a stationary platform, and collecting outer surface images of the target welded part from multiple perspectives; determining the weld profile type of the weld corresponding to the target welded part based on the outer surface images; selecting different scanning methods according to the weld profile type of the weld corresponding to the target welded part to obtain a point cloud data set of the weld in the target welded part; analyzing the point cloud data set of the weld in the target welded part to identify weld defects of the target welded part; and learning about the existence of welding fault defects corresponding to the target welded part based on the detection results on the surface of the target welded part.

[0027] On another aspect, this application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute a method for intelligent detection of surface defects of welded parts based on 3D vision sensing provided above. The method includes: fixing a target welded part on a stationary platform, and collecting outer surface images of the target welded part from multiple perspectives; determining the weld profile type of the weld corresponding to the target welded part based on the outer surface images; selecting different scanning methods according to the weld profile type of the weld corresponding to the target welded part to obtain a point cloud data set of the weld in the target welded part; analyzing the point cloud data set of the weld in the target welded part to identify weld defects of the target welded part; and learning about the existence of welding fault defects corresponding to the target welded part based on the detection results on the surface of the target welded part.

[0028] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0029] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. An intelligent detection method for surface defects of welded parts based on 3D vision sensing, characterized in that, The method includes: Step S1, fixing the target weldment on a stationary platform and collecting the outer surface images of the target weldment from multiple perspectives; Step S2, determining the weld contour type of the corresponding weld of the target weldment based on the outer surface images; Step S3, selecting different scanning methods according to the weld contour type of the corresponding weld of the target weldment to obtain the point cloud data set of the weld in the target weldment; Step S4, analyzing the point cloud data set of the weld in the target weldment to identify the weld defects of the target weldment; Step S5, learning about the existence of the welding fault defects corresponding to the target weldment based on the detection results on the surface of the target weldment.

2. The intelligent detection method for surface defects of welded parts based on 3D vision sensing according to claim 1, wherein, The said Step S2 includes the following sub-steps: Step S21, obtaining the outer surface images of the target weldment from multiple perspectives and splicing the outer surface images of the weldment to obtain the spliced outer surface image of the target weldment; Step S22, converting the spliced outer surface image of the target weldment into a grayscale image, and the conversion process of the grayscale image is as follows: Step S221, obtaining the pixel values of multiple pixel points in the spliced outer surface image, and dividing the pixel value of each pixel point into an R value component Ri, a G value component Gi, and a B value component Bi; where i is the number of the pixel point, i = 1, 2,..., z, and z is the upper limit value of the number; Step S222, calculating the pixel grayscale value XHi of each pixel point through a formula, and the formula is as follows; XHi = Ri × A1 + Gi × A2 + Bi × A3; in the formula, A1, A2, and A3 are weight coefficients, where A2 > A1 > A3; Step S223, comparing the pixel grayscale value of each pixel point with the pixel grayscale threshold; if the pixel grayscale value of the pixel point is greater than or equal to the pixel grayscale threshold, then set the corresponding pixel point to black; if the pixel grayscale value of the pixel point is less than the pixel grayscale threshold, then set the corresponding pixel point to white; Step S224, connecting multiple groups of adjacent black pixel points and recording them as the to-be-determined weld contours to obtain a binary contour image.

3. The intelligent detection method for surface defects of welded parts based on 3D vision sensing according to claim 2, wherein The said Step S2 further includes the following sub-steps: Step S23, obtaining the spliced outer surface image of the target weldment, and identifying the coordinates of any weld pixel point located in the weld in the spliced outer surface image; matching the coordinates of the weld pixel point with all the to-be-determined weld contours in the binary contour image; if the coordinates match successfully, then record the corresponding to-be-determined weld contour as the screened weld contour of the target weldment; if the coordinates match fails, then generate an abnormal signal; Step S24, matching the screened weld contour with the standard weld contour; Step S25, if the screened weld contour matches the first-class standard weld contour successfully, then record the screened weld contour as the first-class weld contour; if the screened weld contour matches the second-class standard weld contour successfully, then record the screened weld contour as the second-class weld contour; if the screened weld contour fails to match both the first-class standard weld and the second-class standard weld, then record the screened weld contour as the third-class weld contour.

4. An intelligent detection method for surface defects of welded parts based on 3D vision sensing according to claim 3, characterized in that, The process of the said contour matching is as follows: Step S241, setting multiple contour matching pixel points at equal intervals in the screened weld contour, then collecting the coordinates of each contour matching pixel point, and recording the number of the contour matching pixel point as n; Step S242: Arbitrarily select a contour matching pixel point near the geometric center of the weld seam and denote it as the selected pixel point; Step S243: Set the number of radial bins r and the number of angular bins θ in the filtered weld seam contour with the selected pixel point as the origin, divide the filtered weld seam contour into multiple bins, and multiply the number of radial bins by the number of angular bins to obtain the number of bins corresponding to the filtered weld seam contour.

5. The intelligent detection method for surface defects of welded parts based on 3D vision sensing according to claim 4, characterized in that, The process of contour matching further includes: Step S244: Calculate the filtered bin value FS(f) of the f-th bin in the filtered weld seam contour = the number of contour matching pixel points falling into the f-th bin ÷ (the number of contour matching pixel points - 1); where f is the number of different bins, f = 1, 2,..., r×θ; Step S245: Similarly calculate the standard bin value FBn(f) corresponding to the f-th bin of the standard weld seam contour; Step S246: Compare the filtered bin value with the standard bin value. If the absolute value of the difference between all filtered bin values and the standard bin values is less than or equal to the preset threshold, it is determined that the matching is successful. If the absolute value of the difference between any one filtered bin value and the standard bin value is greater than the preset threshold, it is determined that the matching fails.

6. The intelligent detection method for surface defects of welded parts based on 3D vision sensing according to claim 1, wherein Step S4 includes the following sub-steps: Step S41: Obtain the point cloud data set of the weld seam in the target welded part; Step S42: Perform a downsampling operation on the point cloud data set to obtain a voxel point cloud set; where the downsampling operation is specifically as follows: Step S421: Obtain the point cloud range [Xmin, Xmax]×[Ymin, Ymax]×[Zmin, Zmax] from the coordinates of different scanned pixel points in the point cloud data set. Let the voxel side length be v, and obtain multiple voxels within the point cloud range; Step S422: Calculate the index s(x', y', z') of the corresponding voxel, and the specific calculation formula is as follows: x' = (X - Xmin) / v; y' = (Y - Ymin) / v; z' = (Z - Zmin) / v; where (X, Y, Z) is the coordinate of the center point corresponding to the voxel; Step S43: Perform a denoising operation on the voxel point cloud set to obtain a denoised voxel point cloud set; Step S44: Perform surface reconstruction based on the denoised voxel point cloud set and extract the mesh.

7. An intelligent detection method for surface defects of welded parts based on 3D vision sensing according to claim 6, characterized in that, Step S4 also includes the following sub-steps: Step S45: For each point in the triangular mesh, calculate the covariance matrix of its local neighborhood, and determine the curvature through the ratio of the minimum eigenvalue to the sum of eigenvalues: Curvature = TZ1÷(TZ1 + TZ2 + TZ3); where TZ1 ≤ TZ2 ≤ TZ3, and TZ1, TZ2, and TZ3 are the eigenvalues of the covariance matrix; Step S46: For each point s in the triangular mesh, calculate the height difference GCs between it and the average height of k neighboring points; ; where q is the numbering of k neighboring points; Step S47: For any point in the triangular mesh, if the height difference of the corresponding point is greater than or equal to the height difference threshold or the curvature is greater than or equal to the curvature threshold, it is determined that there is a welding defect and a defect signal is generated; if the height difference of all points is less than the height difference threshold and the curvature is less than the curvature threshold, it is determined that there is no welding defect in the target welded part and a normal signal is generated.

8. An intelligent detection method for surface defects of welded parts based on 3D vision sensing according to claim 6, characterized in that, The denoising operation is specifically as follows: Step S431: Select any voxel point ps, calculate the Euclidean distances {OS1, OS2, ……, OSk} from the voxel point ps to its k nearest neighboring voxel points, sum up all the Euclidean distances corresponding to the voxel points and take the average to obtain the calculated Euclidean distance of the corresponding voxel point. Step S432: Similarly, calculate the calculated Euclidean distances of all voxel points, sum up the calculated Euclidean distances of all voxel points and take the average to obtain the average comprehensive distance of all voxel points. Step S433: Calculate the standard deviation of the average comprehensive distance of all voxel points based on the average comprehensive distance and the calculated Euclidean distance of each voxel point. Step S434: Set the denoising weight coefficient, and take the sum of the average comprehensive distance and the product of the standard deviation of the average comprehensive distance and the denoising weight coefficient as the discrete denoising threshold; compare the average Euclidean distance of the voxel point with the discrete denoising threshold. Step S435: If the average Euclidean distance of the voxel point is greater than or equal to the discrete denoising threshold, discard the corresponding voxel point; if the average Euclidean distance of the voxel point is less than the discrete denoising threshold, include the voxel point in the set to obtain the denoised voxel point cloud set.

9. The intelligent detection method for surface defects of welded parts based on 3D vision sensing according to claim 6, characterized in that, The process of surface reconstruction is specifically as follows: Step S441: Select any voxel point pu, and select the k neighborhood points {pu1, pu2, ……, puk} corresponding to the voxel point pu; where u is the number of the voxel point in the denoised voxel point cloud set. Step S442: Calculate the covariance matrix JZu corresponding to the voxel point pu through the formula. The specific formula is as follows: ; wherein, , m is the number of neighborhood points, T represents the transpose of the matrix; the eigenvalues TZ1, TZ2 and TZ3 of the covariance matrix are obtained.

10. The intelligent detection method for surface defects of welded parts based on 3D vision sensing according to claim 9, wherein, The process of surface reconstruction further includes: Step S443: Define the indicator function ZS(t). The indicator function is 1 inside the denoised voxel point cloud set and 0 outside the denoised voxel point cloud set; then the normal vector field F(t) corresponding to the denoised voxel point cloud set satisfies: ▽ZS(t) = F(t). Among them, t is the independent variable, and the coordinate value is substituted during calculation. Step S444, solve the Poisson equation: ▽ 2 ZS(t) = ▽F(t); set the isosurface ZS(t) = λ Step S445: Extract the triangular mesh through the marching cubes algorithm.

Images