An intelligent detection method for surface defects of welded parts based on 3D vision sensing

Abstract

The present invention discloses an intelligent detection method for weld surface defects based on 3D visual sensing, which relates to the technical field of weld detection and solves the problem of low accuracy in identifying surface defects corresponding to target welds. The method comprises: fixing the target weld on a stationary platform, and collecting outer surface images of the target weld under multiple viewing angles; determining the weld contour type of the weld corresponding to the target weld based on the outer surface image; selecting different scanning modes based on the weld contour type of the weld corresponding to the target weld, and obtaining a point cloud data set of the weld in the target weld; analyzing the point cloud data set of the weld in the target weld, and identifying the weld defects of the target weld; and knowing the existence of welding fault defects corresponding to the target weld based on the detection result of the target weld surface. The present invention combines 3D vision to accurately scan the weld corresponding to the target weld, and realizes accurate identification of surface defects corresponding to the target weld.

Description

Technical Field

[0001] The present invention belongs to the technical field of weldment identification, and specifically is an intelligent detection method for weldment surface defects based on 3D visual sensing. Background Art

[0002] A weldment is a component or structure with a specific shape, size, and performance, formed by joining two or more metal (or non-metal) materials together through welding. It is a common basic component in the manufacturing industry and is widely used in machinery, construction, shipbuilding, automobiles, aerospace, and other fields. When welding operations are incorrect, weldment defects may be difficult to detect at first, but they will rapidly expand under long-term use or specific working conditions, leading to structural failure and property loss. Therefore, accurate detection of weldment defects is necessary.

[0003] However, at present, when identifying surface defects of welds, it is impossible to accurately locate the position of the weld, and thus it is impossible to collect targeted data for different weld types. As a result, there are deviations in data collection, resulting in large errors in identifying surface defects of welds, and the accuracy of defect detection cannot be guaranteed.

[0004] To this end, the present invention proposes an intelligent detection method for weld surface defects based on 3D visual sensing. Summary of the Invention

[0005] The purpose of the present invention is to propose an intelligent detection method for weld surface defects based on 3D visual sensing to solve the problems raised in the above background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for intelligently detecting surface defects of weldments based on 3D visual sensing, the method comprising:

[0008] Step S1, fixing the target weldment on a stationary platform, and collecting images of the weldment's outer surface from multiple viewing angles;

[0009] Step S2, determining the weld profile type of the weld corresponding to the target weldment based on the outer surface image;

[0010] Step S3, selecting different scanning modes according to the weld contour type of the target weldment corresponding to the weld, to obtain a point cloud data set of the weld in the target weldment;

[0011] Step S4, analyzing the point cloud data set of the weld in the target weldment to identify weld defects of the target weldment;

[0012] Step S5: The presence of welding fault defects corresponding to the target weldment is determined based on the detection result of the target weldment surface.

[0013] Furthermore, step S2 includes the following sub-steps:

[0014] Step S21, acquiring outer surface images of a target weldment at multiple viewing angles, and splicing the outer surface images of the weldment to obtain a spliced outer surface image of the target weldment;

[0015] Step S22: converting the target weldment's outer surface image into a grayscale image. The grayscale image conversion process is as follows:

[0016] Step S221, obtaining pixel values of multiple pixels in the spliced outer surface image, and dividing the pixel value of each pixel into an R value component Ri, a G value component Gi, and a B value component Bi; wherein i is the number of the pixel, i=1, 2, ..., z, and z is the upper limit of the number;

[0017] Step S222, calculate the pixel grayscale value XHi of each pixel using the formula, the specific formula is as follows;

[0018] XHi=Ri×A1+Gi×A2+Bi×A3; where A1, A2, and A3 are weight coefficients, and A2>A1>A3;

[0019] Step S223, comparing the pixel grayscale value of each pixel with the pixel grayscale threshold; if the pixel grayscale value of the pixel is greater than or equal to the pixel grayscale threshold, the corresponding pixel is set to black; if the pixel grayscale value of the pixel is less than the pixel grayscale threshold, the corresponding pixel is set to white;

[0020] Step S224 , connecting multiple groups of adjacent black pixels and recording them as the undetermined weld contour to obtain a binary contour image.

[0021] Furthermore, the step S2 further includes the following sub-steps:

[0022] Step S23, obtaining the spliced outer surface image of the target weldment, identifying the coordinates of any weld pixel point located in the weld in the spliced outer surface image; matching the coordinates of the weld pixel point with all pending weld contours in the binary contour image; if the coordinates are successfully matched, recording the corresponding pending weld contour as the screened weld contour of the target weldment; if the coordinates are not matched, generating an abnormal signal;

[0023] Step S24, screening the weld profile and matching it with the standard weld profile;

[0024] Step S25: If the screened weld profile successfully matches the first-class standard weld profile, the screened weld profile is recorded as the first-class weld profile; if the screened weld profile successfully matches the second-class standard weld profile, the screened weld profile is recorded as the second-class weld profile; if the screened weld profile fails to match both the first-class standard weld and the second-class standard weld, the screened weld profile is recorded as the second-class weld profile or the third-class weld profile.

[0025] Furthermore, the contour matching process is specifically as follows:

[0026] Step S241, setting a plurality of contour matching pixel points at equal intervals in the weld contour screening, and then collecting the coordinates of each contour matching pixel point, and recording the number of the contour matching pixel point as n;

[0027] Step S242, randomly selecting a contour matching pixel point at a position close to the geometric center of the weld and recording it as the selected pixel point;

[0028] Step S243, setting the radial bin number r and the angular bin number θ in the screened weld profile with the selected pixel point as the origin, dividing the screened weld profile into multiple bins, and multiplying the radial bin number by the angular bin number to obtain the bin number corresponding to the screened weld profile.

[0029] Furthermore, the contour matching process further includes:

[0030] Step S244, calculating the screening bin value FS(f) of the fth bin in the screened weld contour = the number of contour matching pixels falling into the fth bin ÷ (contour matching pixels - 1); where f is the number of different bins, f = 1, 2, ..., r × θ;

[0031] Step S245 , similarly calculating the standard bin value FBn(f) of the standard weld profile corresponding to the fth bin;

[0032] Step S246: The filtered bin values are compared with the standard bin values. If the absolute value of the difference between all filtered bin values and the standard bin values is less than or equal to the preset threshold, the match is considered successful. If the absolute value of the difference between any filtered bin value and the standard bin value is greater than the preset threshold, the match is considered unsuccessful.

[0033] Furthermore, step S4 includes the following sub-steps:

[0034] Step S41, obtaining a point cloud data set of the weld in the target weldment;

[0035] Step S42: downsampling the point cloud data set to obtain a voxel point cloud set. The downsampling operation is as follows:

[0036] Step S421: Obtain a point cloud range [Xmin, Xmax]×[Ymin, Ymax]×[Zmin, Zmax] from the coordinates of different scanned pixel points in the point cloud data set, set the voxel side length to v, and obtain multiple voxels within the point cloud range;

[0037] Step S422: Calculate the index s (x', y', z') of the corresponding voxel. The calculation formula is as follows:

[0038] x'=(X-Xmin) / v;y'=(Y-Ymin) / v;z'=(Z-Zmin) / v;where (X, Y, Z) are the coordinates of the center point of the voxel;

[0039] Step S43, performing a denoising operation on the voxel point cloud set to obtain a denoised voxel point cloud set;

[0040] Step S44 : performing surface reconstruction based on the denoised voxel point cloud set to extract a mesh.

[0041] Furthermore, the step S4 further includes the following sub-steps:

[0042] Step S45: For each point in the triangular mesh, calculate the covariance matrix of its local neighborhood, and determine the curvature by the ratio of the minimum eigenvalue to the sum of the eigenvalues:

[0043] Curvature = TZ1 ÷ (TZ1 + TZ2 + TZ3); where TZ1 ≤ TZ2 ≤ TZ3, and TZ1, TZ2, and TZ3 are the eigenvalues of the covariance matrix;

[0044] Step S46, for each point s in the triangular mesh, calculate the height difference GCs between the point s and the average height of the k neighboring points;

[0045] ; Where q is the number of k neighborhood points;

[0046] Step S47: For any point in the triangular mesh, if the height difference of the corresponding point is greater than or equal to the height difference threshold or the curvature is greater than or equal to the curvature threshold, it is determined that there is a welding defect and a defect signal is generated; if the height difference of all points is less than the height difference threshold and the curvature is less than the curvature threshold; it is determined that there is no welding defect in the target weldment and a normal signal is generated.

[0047] Furthermore, the denoising operation is specifically as follows:

[0048] Step S431, select any voxel point ps, calculate the Euclidean distances from the voxel point ps to the k nearest neighboring voxel points {OS1, OS2, ..., OSk}, add up all the Euclidean distances corresponding to the corresponding voxel point and take the average value to obtain the calculated Euclidean distance of the corresponding voxel point;

[0049] Step S432: Similarly, the calculated Euclidean distances of all voxel points are calculated, and the calculated Euclidean distances of all voxel points are summed up and averaged to obtain the average comprehensive distance of all voxel points;

[0050] Step S433, calculating the standard deviation of the average comprehensive distance of all voxel points based on the average comprehensive distance and the calculated Euclidean distance of each voxel point;

[0051] Step S434: Set a denoising weight coefficient, and multiply the average comprehensive distance plus the standard deviation of the average comprehensive distance by the denoising weight coefficient as a discrete denoising threshold; compare the average Euclidean distance of the voxel points with the discrete denoising threshold;

[0052] In step S435 , if the average Euclidean distance of the voxel points is greater than or equal to the discrete denoising threshold, the corresponding voxel points are discarded; if the average Euclidean distance of the voxel points is less than the discrete denoising threshold, the voxel points are grouped into a set to obtain a denoised voxel point cloud set.

[0053] Furthermore, the surface reconstruction process is specifically as follows:

[0054] Step S441: select any voxel point pu, and select k neighboring points {pu1, pu2, ..., puk} corresponding to the voxel point pu; where u is the number of the voxel point in the denoised voxel point cloud set;

[0055] In step S442, the covariance matrix JZu corresponding to the voxel point pu is calculated using the formula, which is as follows:

[0056] Where, , m is the number of the neighborhood point, T represents the transpose of the matrix; the eigenvalues TZ1, TZ2 and TZ3 of the covariance matrix are obtained.

[0057] Furthermore, the surface reconstruction process further includes:

[0058] Step S443 , defining an indicator function ZS(t), which is one inside the denoised voxel point cloud set and zero outside the denoised voxel point cloud set; then the normal vector field F(t) corresponding to the denoised voxel point cloud set satisfies: ▽ZS(t)=F(t);

[0059] Among them, t is the independent variable, and the coordinate value is input during the calculation;

[0060] Step S444, solve the Poisson equation: 2 ZS(t)=▽F(t); set the isosurface ZS(t)=λ

[0061] Step S445 , extracting a triangular mesh using a marching cubes algorithm.

[0062] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0063] 1. The present invention first fixes the target weldment on a stationary platform and collects images of the weldment's outer surface from multiple viewing angles. The weld contour type of the target weldment's corresponding weld is then determined based on the outer surface images. Different scanning methods are then selected based on the weld contour type of the target weldment's corresponding weld to obtain a point cloud data set of the weld in the target weldment. The present invention achieves precise scanning of the weld corresponding to the target weldment.

[0064] 2. The present invention analyzes the point cloud data set of the weld in the target weldment to identify the weld defects of the target weldment; finally, based on the detection results of the target weldment surface, the existence of the corresponding welding fault defects of the target weldment is known, and the present invention realizes the accurate detection of the defects of the target weldment. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0066] Figure 1 is a flow chart of the overall method of the present invention;

[0067] Figure 2 A schematic block diagram of the architecture of the method of the present invention;

[0068] Figure 3 Schematic diagram of the boxing for screening weld profiles in the present invention;

[0069] Figure 4 Schematic diagram of the structure of the voxel in the present invention;

[0070] Figure 5 It is a structural diagram of the computer device in the present invention. DETAILED DESCRIPTION

[0071] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] For example 1, please refer to Figure 1-Figure 4As shown, the technical solution provided by the present invention is: a method for intelligent detection of weld surface defects based on 3D visual sensing, which performs an initial analysis of the outer surface image of the target weld, sets the weld scanning mode of the target weld, and then obtains a point cloud set corresponding to the target weld; performs data processing on the point cloud set, and then determines the triangular mesh of the point cloud set through surface reconstruction, and then extracts feature information of the triangular mesh, and judges the defect of the target weld based on the feature information;

[0073] In the present invention, the weldment to be inspected is calibrated as the target weldment, and the intelligent detection method for weldment surface defects is specifically as follows:

[0074] Step S1, fixing the target weldment on a stationary platform, and collecting images of the weldment's outer surface from multiple viewing angles;

[0075] Step S2, determining the weld profile type of the weld corresponding to the target weldment based on the outer surface image;

[0076] Among them, the weld profile types include Class I weld profile, Class II weld profile and Class III weld profile; Class I weld profile is the profile of the pipe girth weld, Class II weld profile is the profile of the plane straight line weld, and Class III weld profile is the profile of the weld other than Class I and Class II weld profiles;

[0077] In this embodiment, step S2 includes the following sub-steps:

[0078] Step S21, acquiring outer surface images of a target weldment at multiple viewing angles, and splicing the outer surface images of the weldment to obtain a spliced outer surface image of the target weldment;

[0079] Step S22, converting the spliced outer surface image of the target weldment into a grayscale image;

[0080] In this embodiment, the grayscale image conversion process is as follows:

[0081] Step S221, obtaining pixel values of multiple pixels in the spliced outer surface image, and dividing the pixel value of each pixel into an R value component Ri, a G value component Gi, and a B value component Bi; wherein i is the number of the pixel, i=1, 2, ..., z, and z is the upper limit of the number;

[0082] Step S222, calculate the pixel grayscale value XHi of each pixel using the formula, the specific formula is as follows;

[0083] XHi=Ri×A1+Gi×A2+Bi×A3; where A1, A2, and A3 are weight coefficients, wherein A2>A1>A3; specifically, A1 can be 0.299, A2 can be 0.587, and A3 can be 0.114;

[0084] Step S223, comparing the pixel grayscale value of each pixel with the pixel grayscale threshold; if the pixel grayscale value of the pixel is greater than or equal to the pixel grayscale threshold, the corresponding pixel is set to black; if the pixel grayscale value of the pixel is less than the pixel grayscale threshold, the corresponding pixel is set to white;

[0085] Step S224, connecting multiple groups of adjacent black pixels and recording them as the undetermined weld contour to obtain a binary contour image;

[0086] Step S23, obtaining the spliced outer surface image of the target weldment, identifying the coordinates of any weld pixel point located in the weld in the spliced outer surface image; matching the coordinates of the weld pixel point with all pending weld contours in the binary contour image; if the coordinates are successfully matched, recording the corresponding pending weld contour as the screened weld contour of the target weldment; if the coordinates are not matched, generating an abnormal signal;

[0087] The weld pixel points are pixels that have been determined to be in the weld;

[0088] Step S24, screening the weld profile and matching it with the standard weld profile; wherein the standard weld profile includes a first-class standard weld profile and a second-class standard weld profile;

[0089] Step S25: If the screened weld profile successfully matches the first-class standard weld profile, the screened weld profile is recorded as the first-class weld profile; if the screened weld profile successfully matches the second-class standard weld profile, the screened weld profile is recorded as the second-class weld profile; if the screened weld profile fails to match both the first-class standard weld and the second-class standard weld, the screened weld profile is recorded as the second-class weld profile or the third-class weld profile;

[0090] In this embodiment, the contour matching process is specifically as follows:

[0091] Step S241, setting a plurality of contour matching pixel points at equal intervals in the weld contour screening, and then collecting the coordinates of each contour matching pixel point, and recording the number of the contour matching pixel point as n;

[0092] Step S242, randomly selecting a contour matching pixel point at a position close to the geometric center of the weld and recording it as the selected pixel point;

[0093] Step S243, as Figure 3 As shown, the radial bin number r and the angular bin number θ in the screened weld contour are set with the selected pixel point as the origin, and the screened weld contour is divided into multiple bins. The radial bin number is multiplied by the angular bin number to obtain the bin number corresponding to the screened weld contour;

[0094] This step can be regarded as dividing the screened weld contour into multiple sub-areas according to a certain rule, and each sub-area is a bin;

[0095] Among them, the number of radial bins and the number of angular bins are the core parameters that control the resolution and sensitivity of the feature descriptor. The number of radial bins is the number of distance quantization intervals, which is used to control the sensitivity of the distance. The number of angular bins is the number of direction quantization intervals, which is used to control the sensitivity of the direction.

[0096] Step S244, calculating the screening bin value FS(f) of the fth bin in the screened weld contour = the number of contour matching pixels falling into the fth bin ÷ (contour matching pixels - 1); where f is the number of different bins, f = 1, 2, ..., r × θ;

[0097] Step S245 , similarly calculating the standard bin value FBn(f) of the standard weld profile corresponding to the fth bin;

[0098] Step S246: The filtered bin values are compared with the standard bin values. If the absolute value of the difference between all filtered bin values and the standard bin values is less than or equal to the preset threshold, the match is considered successful. If the absolute value of the difference between any filtered bin value and the standard bin value is greater than the preset threshold, the match is considered unsuccessful.

[0099] Step S3, selecting different scanning modes according to the weld contour type of the target weldment corresponding to the weld, to obtain a point cloud data set of the weld in the target weldment;

[0100] The point cloud data set specifically consists of the XYZ coordinates of the scanned pixel points in the weld of the target weldment. The scanned pixel points are collected by laser scanning. During the specific scanning, a six-axis robot can be used in conjunction with a laser line scan sensor to ensure that the scanning path covers all weld areas.

[0101] The scanning method is selected. For the first type of weld contour, the equidistant spiral method is selected for scanning; for the second type of weld contour, the Zig-Zag scanning method is selected for scanning; for the third type of weld contour, the local curvature of the target weldment is estimated, and the scanning density of the scanning line is adjusted according to the estimated curvature;

[0102] Specifically, ; Where SM is the scanning density and QL is the local curvature of the target weldment.

[0103] Step S4, analyzing the point cloud data set of the weld in the target weldment to identify weld defects of the target weldment;

[0104] In this embodiment, step S4 includes the following sub-steps:

[0105] Step S41, obtaining a point cloud data set of the weld in the target weldment;

[0106] Step S42, performing a downsampling operation on the point cloud data set to obtain a voxel point cloud set;

[0107] In this embodiment, the downsampling operation is specifically as follows:

[0108] Step S421, please refer to Figure 4 As shown, the point cloud range [Xmin, Xmax]×[Ymin, Ymax]×[Zmin, Zmax] is obtained from the coordinates of different scanned pixel points in the point cloud data set. Assuming the voxel side length is v, multiple voxels within the point cloud range are obtained;

[0109] The principle of downsampling is to divide the three-dimensional space into uniform voxels (cubes), and only retain one representative point in each voxel;

[0110] Step S422: Calculate the index s (x', y', z') of the corresponding voxel. The calculation formula is as follows:

[0111] x'=(X-Xmin) / v;y'=(Y-Ymin) / v;z'=(Z-Zmin) / v;where (X, Y, Z) are the coordinates of the center point of the voxel;

[0112] Step S43, performing a denoising operation on the voxel point cloud set to obtain a denoised voxel point cloud set;

[0113] In this embodiment, the denoising operation is specifically as follows:

[0114] Step S431, select any voxel point ps, calculate the Euclidean distances from the voxel point ps to the k nearest neighboring voxel points {OS1, OS2, ..., OSk}, add up all the Euclidean distances corresponding to the corresponding voxel point and take the average value to obtain the calculated Euclidean distance of the corresponding voxel point;

[0115] Step S432: Similarly, the calculated Euclidean distances of all voxel points are calculated, and the calculated Euclidean distances of all voxel points are summed up and averaged to obtain the average comprehensive distance of all voxel points;

[0116] Step S433, calculating the standard deviation of the average comprehensive distance of all voxel points based on the average comprehensive distance and the calculated Euclidean distance of each voxel point;

[0117] Step S434: Set a denoising weight coefficient, and multiply the average comprehensive distance plus the standard deviation of the average comprehensive distance by the denoising weight coefficient as a discrete denoising threshold; compare the average Euclidean distance of the voxel points with the discrete denoising threshold;

[0118] Step S435: If the average Euclidean distance of the voxel points is greater than or equal to the discrete denoising threshold, the corresponding voxel points are discarded; if the average Euclidean distance of the voxel points is less than the discrete denoising threshold, the voxel points are grouped into a set to obtain a denoised voxel point cloud set;

[0119] Step S44, performing surface reconstruction based on the denoised voxel point cloud set to extract the grid;

[0120] In this embodiment, the surface reconstruction process is specifically as follows:

[0121] Step S441: select any voxel point pu, and select k neighboring points {pu1, pu2, ..., puk} corresponding to the voxel point pu; where u is the number of the voxel point in the denoised voxel point cloud set;

[0122] In step S442, the covariance matrix JZu corresponding to the voxel point pu is calculated using the formula, which is as follows:

[0123] Where, , m is the number of the neighborhood point, T represents the transpose of the matrix; get the eigenvalues TZ1, TZ2 and TZ3 of the covariance matrix;

[0124] It should be noted that the normal vector is the eigenvector corresponding to the minimum eigenvalue of the covariance matrix JZu;

[0125] Step S443 , defining an indicator function ZS(t), which is one inside the denoised voxel point cloud set and zero outside the denoised voxel point cloud set; then the normal vector field F(t) corresponding to the denoised voxel point cloud set approximately satisfies: ▽ZS(t)=F(t);

[0126] Where t is the independent variable, which is used to input the coordinate value during the calculation. The inverted triangle symbol (▽) is called the Nabla operator, which is a core symbol in vector calculus and is used to represent gradient, divergence, and curl.

[0127] Step S444, solve the Poisson equation: 2 ZS(t)=▽F(t); set the isosurface ZS(t)=λ, and the preferred value of λ is 0.5;

[0128] It should be noted that the process of solving the Poisson equation is recorded as Poisson reconstruction, which is the process of converting the point cloud normal vector field into an implicit surface function;

[0129] Step S445 , extracting a triangular mesh using a marching cubes algorithm; wherein the triangular mesh is used to simulate the surface of a complex object; the marching cubes algorithm divides the space into cubic units, determines whether the surface passes through the cube based on the ZS(t) values at the vertices, and generates a corresponding triangular mesh;

[0130] Specifically, the process of determining whether a surface passes through the cube is as follows: the coordinates of the eight corresponding vertices of the cube are substituted into ZS(t) in sequence, and the ZS(t) value is compared with the isosurface λ;

[0131] If ZS(t) ≥ λ, the corresponding vertex is recorded as an internal point; if ZS(t) < λ, the corresponding vertex is recorded as an external point; the state combination of the eight vertices matches the predefined fifteen basic triangle patch modes, and the triangle mesh is extracted according to the basic triangle patch mode;

[0132] For example, if only one vertex is internal, a triangle is generated; if two adjacent vertices are internal, a quadrilateral is generated (decomposed into two triangles).

[0133] Step S45: For each point in the triangular mesh, calculate the covariance matrix of its local neighborhood, and determine the curvature by the ratio of the minimum eigenvalue to the sum of the eigenvalues:

[0134] Curvature = TZ1 ÷ (TZ1 + TZ2 + TZ3); where TZ1 ≤ TZ2 ≤ TZ3, and TZ1, TZ2, and TZ3 are the eigenvalues of the covariance matrix;

[0135] Step S46, for each point s in the triangular mesh, calculate the height difference GCs between the point s and the average height of the k neighboring points;

[0136] ; Where q is the number of k neighborhood points;

[0137] Step S47: For any point in the triangular mesh, if the height difference of the corresponding point is greater than or equal to the height difference threshold or the curvature is greater than or equal to the curvature threshold, it is determined that there is a welding defect and a defect signal is generated; if the height difference of all points is less than the height difference threshold and the curvature is less than the curvature threshold; it is determined that there is no welding defect in the target weldment and a normal signal is generated.

[0138] Step S5: The presence of welding fault defects corresponding to the target weldment is determined based on the detection result of the target weldment surface.

[0139] In this application, if a corresponding calculation formula appears, the above calculation formula is dimensionless and its numerical calculation is performed. The weight coefficient, proportional coefficient and other coefficients in the formula are set to a result value obtained by quantifying each parameter. Regarding the size of the weight coefficient and the proportional coefficient, as long as it does not affect the proportional relationship between the parameter and the result value, it is acceptable.

[0140] Example 2: Figure 5The present invention is a structural diagram of a computer device, which may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The processor may call logic instructions in the memory to execute a method for intelligent detection of weld surface defects based on 3D visual sensing, the method comprising: fixing a target weld on a stationary platform and collecting images of the weld's outer surface from multiple viewing angles; determining the weld profile type of the target weld's corresponding weld based on the outer surface image; selecting different scanning modes based on the weld profile type of the target weld's corresponding weld to obtain a point cloud data set of the weld in the target weld; analyzing the point cloud data set of the weld in the target weld to identify weld defects in the target weld; and determining the presence of welding fault defects in the target weld based on the detection results of the target weld's surface.

[0141] In addition, the logical instructions in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0142] On the other hand, the present application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the above-mentioned intelligent detection method for weld surface defects based on 3D visual sensing, the method including: fixing the target weld on a stationary platform, and collecting the outer surface images of the weld from multiple perspectives of the target weld; determining the weld contour type of the weld corresponding to the target weld based on the outer surface image; selecting different scanning methods based on the weld contour type of the weld corresponding to the target weld to obtain a point cloud data set of the weld in the target weld; analyzing the point cloud data set of the weld in the target weld to identify the weld defects of the target weld; and knowing the existence of the welding fault defects corresponding to the target weld based on the detection results of the target weld surface.

[0143] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned intelligent detection method for weld surface defects based on 3D visual sensing, the method comprising: fixing the target weld on a stationary platform, and collecting images of the weld outer surface from multiple perspectives of the target weld; determining the weld contour type of the weld corresponding to the target weld based on the outer surface image; selecting different scanning methods based on the weld contour type of the weld corresponding to the target weld, and obtaining a point cloud data set of the weld in the target weld; analyzing the point cloud data set of the weld in the target weld to identify the weld defects of the target weld; and knowing the existence of welding fault defects corresponding to the target weld based on the detection results of the target weld surface.

[0144] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0145] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An intelligent detection method for weldment surface defects based on 3D visual sensing, characterized in that: Methods include: Step S1, fixing the target weldment on a stationary platform, and collecting images of the weldment's outer surface from multiple viewing angles; Step S2, determining the weld profile type of the weld corresponding to the target weldment based on the outer surface image; Step S3, selecting different scanning modes according to the weld contour type of the target weldment corresponding to the weld, to obtain a point cloud data set of the weld in the target weldment; Step S4, analyzing the point cloud data set of the weld in the target weldment to identify weld defects of the target weldment; The step S4 includes the following sub-steps: Step S41, obtaining a point cloud data set of the weld in the target weldment; Step S42: downsampling the point cloud data set to obtain a voxel point cloud set. The downsampling operation is as follows: Step S421: obtaining a point cloud range [Xmin, Xmax]×[Ymin, Ymax]×[Zmin, Zmax] from the coordinates of different scanned pixel points in the point cloud data set, assuming that the voxel side length is v, and obtaining multiple voxels within the point cloud range; Step S422: Calculate the index s (x', y', z') of the corresponding voxel. The calculation formula is as follows: x'=(X-Xmin) / v;y'=(Y-Ymin) / v;z'=(Z-Zmin) / v;where (X, Y, Z) are the coordinates of the center point of the voxel; Step S43, performing a denoising operation on the voxel point cloud set to obtain a denoised voxel point cloud set; Step S44, performing surface reconstruction based on the denoised voxel point cloud set to extract the grid; Step S45: For each point in the triangular mesh, calculate the covariance matrix of its local neighborhood, and determine the curvature by the ratio of the minimum eigenvalue to the sum of the eigenvalues: Curvature = TZ1 ÷ (TZ1 + TZ2 + TZ3); where TZ1 ≤ TZ2 ≤ TZ3, and TZ1, TZ2, and TZ3 are the eigenvalues of the covariance matrix; Step S46, for each point s in the triangular mesh, calculate the height difference GCs between the point s and the average height of the k neighboring points; ; Where q is the number of k neighborhood points; Step S47: For any point in the triangular mesh, if the height difference of the corresponding points is greater than or equal to the height difference threshold or the curvature is greater than or equal to the curvature threshold, it is determined that a welding defect exists and a defect signal is generated; if the height difference of all points is less than the height difference threshold and the curvature is less than the curvature threshold, it is determined that there is no welding defect in the target weldment and a normal signal is generated; Step S5: The presence of welding fault defects corresponding to the target weldment is determined based on the detection result of the target weldment surface.

2. The method for intelligent detection of weld surface defects based on 3D visual sensing according to claim 1, characterized in that: The step S2 includes the following sub-steps: Step S21, acquiring outer surface images of a target weldment at multiple viewing angles, and splicing the outer surface images of the weldment to obtain a spliced outer surface image of the target weldment; Step S22: converting the target weldment's outer surface image into a grayscale image. The grayscale image conversion process is as follows: Step S221, obtaining pixel values of multiple pixels in the spliced outer surface image, and dividing the pixel value of each pixel into an R value component Ri, a G value component Gi, and a B value component Bi; wherein i is the number of the pixel, i=1, 2, ..., z, and z is the upper limit of the number; Step S222, calculate the pixel grayscale value XHi of each pixel using the formula, the specific formula is as follows; XHi=Ri×A1+Gi×A2+Bi×A3; where A1, A2, and A3 are weight coefficients, and A2>A1>A3; Step S223, comparing the pixel grayscale value of each pixel with the pixel grayscale threshold; if the pixel grayscale value of the pixel is greater than or equal to the pixel grayscale threshold, the corresponding pixel is set to black; if the pixel grayscale value of the pixel is less than the pixel grayscale threshold, the corresponding pixel is set to white; Step S224 , connecting multiple groups of adjacent black pixels and recording them as the undetermined weld contour to obtain a binary contour image.

3. The method for intelligent detection of weld surface defects based on 3D visual sensing according to claim 2, characterized in that: The step S2 further includes the following sub-steps: Step S23, obtaining the spliced outer surface image of the target weldment, identifying the coordinates of any weld pixel point located in the weld in the spliced outer surface image; matching the coordinates of the weld pixel point with all pending weld contours in the binary contour image; if the coordinates are successfully matched, recording the corresponding pending weld contour as the screened weld contour of the target weldment; if the coordinates are not matched, generating an abnormal signal; Step S24, screening the weld profile and matching it with the standard weld profile; Step S25: If the screened weld profile successfully matches the first-class standard weld profile, the screened weld profile is recorded as the first-class weld profile; if the screened weld profile successfully matches the second-class standard weld profile, the screened weld profile is recorded as the second-class weld profile; if the screened weld profile fails to match both the first-class standard weld and the second-class standard weld, the screened weld profile is recorded as the second-class weld profile or the third-class weld profile.

4. The method for intelligent detection of weld surface defects based on 3D visual sensing according to claim 3, characterized in that: The contour matching process is as follows: Step S241, setting a plurality of contour matching pixel points at equal intervals in the weld contour screening, and then collecting the coordinates of each contour matching pixel point, and recording the number of the contour matching pixel point as n; Step S242, randomly selecting a contour matching pixel point at a position close to the geometric center of the weld and recording it as the selected pixel point; Step S243, setting the radial bin number r and the angular bin number θ in the screened weld profile with the selected pixel point as the origin, dividing the screened weld profile into multiple bins, and multiplying the radial bin number by the angular bin number to obtain the bin number corresponding to the screened weld profile.

5. The method for intelligent detection of weldment surface defects based on 3D visual sensing according to claim 4, characterized in that: The contour matching process further includes: Step S244, calculating the screening bin value FS(f) of the fth bin in the screened weld contour = the number of contour matching pixels falling into the fth bin ÷ (contour matching pixels - 1); where f is the number of different bins, f = 1, 2, ..., r × θ; Step S245 , similarly calculating the standard bin value FBn(f) of the standard weld profile corresponding to the fth bin; Step S246: The filtered bin values are compared with the standard bin values. If the absolute value of the difference between all filtered bin values and the standard bin values is less than or equal to the preset threshold, the match is considered successful. If the absolute value of the difference between any filtered bin value and the standard bin value is greater than the preset threshold, the match is considered unsuccessful.

6. The method for intelligent detection of weldment surface defects based on 3D visual sensing according to claim 1, characterized in that: The denoising operation is specifically as follows: Step S431, select any voxel point ps, calculate the Euclidean distances from the voxel point ps to the k nearest neighboring voxel points {OS1, OS2, ..., OSk}, add up all the Euclidean distances corresponding to the corresponding voxel point and take the average value to obtain the calculated Euclidean distance of the corresponding voxel point; Step S432: Similarly, the calculated Euclidean distances of all voxel points are calculated, and the calculated Euclidean distances of all voxel points are summed up and averaged to obtain the average comprehensive distance of all voxel points; Step S433, calculating the standard deviation of the average comprehensive distance of all voxel points based on the average comprehensive distance and the calculated Euclidean distance of each voxel point; Step S434: Set a denoising weight coefficient, and multiply the average comprehensive distance plus the standard deviation of the average comprehensive distance by the denoising weight coefficient as a discrete denoising threshold; compare the average Euclidean distance of the voxel points with the discrete denoising threshold; In step S435 , if the average Euclidean distance of the voxel points is greater than or equal to the discrete denoising threshold, the corresponding voxel points are discarded; if the average Euclidean distance of the voxel points is less than the discrete denoising threshold, the voxel points are grouped into a set to obtain a denoised voxel point cloud set.

7. The method for intelligent detection of weldment surface defects based on 3D visual sensing according to claim 1, characterized in that: The surface reconstruction process is as follows: Step S441: select any voxel point pu, and select k neighboring points {pu1, pu2, ..., puk} corresponding to the voxel point pu; where u is the number of the voxel point in the denoised voxel point cloud set; In step S442, the covariance matrix JZu corresponding to the voxel point pu is calculated using the formula, which is as follows: Where, , m is the number of the neighborhood point, T represents the transpose of the matrix; the eigenvalues TZ1, TZ2 and TZ3 of the covariance matrix are obtained.

8. The method for intelligent detection of weldment surface defects based on 3D visual sensing according to claim 7, characterized in that: The surface reconstruction process further includes: Step S443 , defining an indicator function ZS(t), which is one inside the denoised voxel point cloud set and zero outside the denoised voxel point cloud set; then the normal vector field F(t) corresponding to the denoised voxel point cloud set satisfies: ▽ZS(t)=F(t); Among them, t is the independent variable, and the coordinate value is input during the calculation; Step S444, solve the Poisson equation: 2 ZS(t)=▽F(t); set ZS(t)=λ, where λ is the isosurface; Step S445 , extracting a triangular mesh using a marching cubes algorithm.

Images