Welding spot positioning and detection integrated control method and system based on 3D vision
By adopting a 3D vision-based integrated control method for weld point positioning and detection, combined with dynamic point cloud acquisition and ICP registration, the problems of flexibility and real-time performance in weld point detection in flexible manufacturing environments are solved. This achieves integrated control of high-precision weld point positioning and detection, thereby improving the overall level of welding quality management.
Patent Information
- Application Number
- CN202511628516.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-07
- Publication Date
- 2026-01-20
AI Technical Summary
Existing weld point detection technologies lack flexibility in flexible manufacturing environments, making it difficult to achieve high-precision, real-time weld point positioning and detection. Furthermore, the separation between pre-weld positioning and post-weld detection leads to increased equipment costs and information silos, hindering full-process traceability and closed-loop control.
A 3D vision-based integrated control method for weld point positioning and detection is adopted. By combining dynamic point cloud acquisition with ICP registration and kd tree structure, pre- and post-weld integration is achieved. Weld point defects are determined by orientation centrality index, and a closed-loop control system is constructed.
It significantly improves adaptability and real-time performance in flexible manufacturing environments, enhances the accuracy and robustness of weld defect identification, reduces equipment redundancy and maintenance costs, and achieves integrated traceability and quality control throughout the entire process.
Smart Images
Figure CN121361086A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of intelligent welding control of vehicles, in particular to a welding spot positioning and detection integrated control method and system based on 3D vision. BACKGROUND
[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.
[0003] In the automobile manufacturing body welding production line, the positioning accuracy of the welding spot and the welding quality directly affect the structural strength and safety performance of the whole vehicle. With the continuous improvement of the light weight and automation level of the whole vehicle, the production line puts forward higher requirements for the welding spot detection technology, especially in the high-beat and flexible production environment, it is urgent to have a detection system with high precision, high efficiency and real-time feedback capability to realize accurate identification and process control of the welding spot and ensure the consistency and reliability of the welding quality.
[0004] At present, the welding spot detection method based on vision is gradually introduced in the industry to replace the inefficient and subjective manual detection. However, the inventors found in the research that the existing scheme generally relies on fixed scanning stations, requires the body parts to be scanned statically at a specific station, has poor flexibility and is difficult to adapt to the flexible manufacturing demand. At the same time, the traditional three-dimensional reconstruction speed is slow and it is difficult to realize the real-time processing of "scanning while reconstructing". In actual production, due to the reflection, color change or shielding of the body surface, it is difficult to obtain stable accuracy by relying on two-dimensional images or simple three-dimensional identification. In addition, the existing system often separates the pre-welding positioning and post-welding quality detection into independent processes, resulting in increased equipment cost, interrupted data chain, and inability to realize the whole-process tracing and closed-loop control of a single welding spot, which limits the ability of overall welding quality management. SUMMARY
[0005] In order to solve the above problems, the present application provides a welding spot positioning and detection integrated control method and system based on 3D vision, which can realize efficient, accurate and real-time welding spot positioning, and can integrate pre-welding welding guidance and post-welding detection to comprehensively improve the intelligent level and welding quality of automobile body welding.
[0006] In order to achieve the above purpose, the present application adopts the following technical scheme: One or more embodiments provide a welding spot positioning and detection integrated control method based on 3D vision, comprising the following steps: performing three-dimensional point cloud scanning on the body of the current vehicle to obtain three-dimensional point cloud data of the body; The scanning data is matched with the pre-stored standard model by ICP registration, coordinate system calibration is performed to obtain a spatial correspondence, and a welding point position coordinate in a coordinate system of an industrial robot performing welding is obtained to control the industrial robot to perform a welding operation. A three-dimensional point cloud scanning is performed on the vehicle body after welding to obtain welding point cloud data. The welding point cloud data after welding and the standard model point cloud are respectively constructed into k-d trees, and then a direction centrality index is calculated, and whether the welding point has defects is determined based on the direction centrality index. One or more embodiments provide a 3D vision-based welding point positioning and detection integrated control system, comprising: An AVG trolley, an industrial robot, and a main control system; The AVG trolley is used to carry the industrial robot to realize movement. The industrial robot is used to carry a welding gun to realize welding on the vehicle body, or carry a scanner to scan the vehicle body to obtain vehicle body point cloud data. The main control system is configured to perform the above-mentioned 3D vision-based welding point positioning and detection integrated control method to control the positioning of the welding point and the detection of the welding quality.
[0007] One or more embodiments provide a 3D vision-based welding point positioning and detection integrated control system, comprising: A first acquisition module configured to perform three-dimensional point cloud scanning on the vehicle body of a current vehicle to obtain vehicle body three-dimensional point cloud data; A welding point position identification module configured to match the scanning data with a pre-stored standard model by ICP registration, perform coordinate system calibration to obtain a spatial correspondence, and obtain welding point position coordinates in a coordinate system of an industrial robot performing welding to control the industrial robot to perform a welding operation; A second acquisition module configured to perform three-dimensional point cloud scanning on the vehicle body of a vehicle after welding to obtain welding point cloud data after welding; A detection module configured to construct the welding point cloud data after welding and the standard model point cloud into k-d trees respectively, then calculate a direction centrality index, and determine whether the welding point has defects based on the direction centrality index.
[0008] An electronic device comprising a memory and a processor, and computer instructions stored on the memory and running on the processor, when the computer instructions are run by the processor, the steps in the above-mentioned 3D vision-based welding point positioning and detection integrated control method are completed.
[0009] A computer readable storage medium for storing computer instructions, when the computer instructions are executed by a processor, the steps in the above-mentioned 3D vision-based welding point positioning and detection integrated control method are completed.
[0010] Compared with the prior art, the present application has the following beneficial effects: The present application realizes "scanning and positioning simultaneously" by dynamic point cloud acquisition combined with ICP registration, significantly improves the adaptability and real-time performance in flexible manufacturing environment. With the help of k-d tree and direction centrality index, the accuracy and robustness of weld defect identification are improved, especially suitable for processing complex reflection, shielding or deformation welding environment. In addition, this scheme integrates the positioning and detection process before and after welding, realizes the whole process tracing, eliminates information island, improves the integration level of quality control, and effectively reduces the equipment redundancy and maintenance cost The advantages of the present application and the advantages of the additional aspects will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS
[0011] The drawings accompanying the specification of this application form a part thereof, serve to provide further understanding of the application, and together with the description of the exemplary embodiments of the application and explanations thereof serve to explain the application, and do not constitute limitations on the application.
[0012] Fig. 1 is a process schematic diagram of the 3D vision-based weld positioning and detection integrated control method of embodiment 1 of the present application; Fig. 2 is a weld defect identification process schematic diagram of embodiment 1 of the present application; Fig. 3 is a weld deformation amount identification process schematic diagram of embodiment 1 of the present application; DETAILED DESCRIPTION The present application will be further described below in conjunction with the drawings and embodiments.
[0013] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.
[0014] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and furthermore, it should be understood that when the terms "comprise" and / or "include" are used in the specification, they indicate the presence of a feature, step, operation, device, component and / or combinations thereof. It should be noted that the various embodiments and features in the present application can be combined with each other without conflict. The embodiments will be described in detail below in conjunction with the drawings.
[0015] Embodiment 1 The embodiment provides a welding spot positioning and detection integrated control system based on 3D vision, comprising an AVG trolley, an industrial robot and a main control system. The AVG trolley is used for carrying the industrial robot to realize movement. The industrial robot is used for carrying a welding gun to realize welding on a vehicle body or carrying a scanner to scan the vehicle body to obtain vehicle body point cloud data. The main control system is configured to execute a welding spot positioning and detection integrated control method based on 3D vision to control positioning of the welding spot and detection of welding quality. In one or more embodiments, the technical solutions disclosed in the embodiments are as follows. Figs. 1 to 3 The embodiment also provides a welding spot positioning and detection integrated control method based on 3D vision, which is configured to be implemented in the main control system and comprises a welding spot positioning method and a quality detection method. Step 1: performing three-dimensional point cloud scanning on the vehicle body of the current vehicle to obtain three-dimensional point cloud data of the vehicle body. Step 2: performing ICP registration on the scanning data and the pre-stored standard model to obtain a spatial correspondence relationship through coordinate system calibration, obtaining the position coordinates of the welding spot in the coordinate system of the industrial robot performing welding, and controlling the industrial robot to perform welding operation. Step 3: performing three-dimensional point cloud scanning on the vehicle body of the vehicle after welding to obtain welding spot point cloud data. Step 4: constructing the welding spot point cloud data after welding and the standard model point cloud into k-d trees respectively, and then calculating a direction centrality index to determine whether the welding spot has defects. In a further technical solution, step 5: for the welded vehicle with defects in the welding spot, performing mesh division analysis to calculate the welding deformation. In the embodiment, an integrated 3D vision scanning system is deployed to collect dynamic three-dimensional point cloud of the vehicle body, and high-density laser or structured light scanning technology is used to obtain complete three-dimensional geometric information. In step 1, the obtained vehicle body point cloud is used for pre-welding positioning. In step 2, the real-time point cloud is registered with the pre-stored standard model point cloud through an iterative closest point (ICP) algorithm, the spatial coordinates are aligned, the target position of the welding spot is mapped to the coordinate system of the industrial robot, and high-precision welding control is realized. Then in step 3, the vehicle body is scanned again to extract the point cloud information of the welding spot area. In step 4, k-d tree index structure is established based on the post-welding point cloud and the standard point cloud respectively to improve the query efficiency of the point cloud data, and then the direction centrality index is used to analyze the geometric feature distribution around the welding spot to detect whether there are defects such as welding bumps, virtual welding and missed welding, and quantitative determination of post-welding quality is realized. The process forms a closed-loop path from pre-welding positioning to post-welding detection.
[0016] The embodiment realizes "scanning and positioning at the same time" through dynamic point cloud acquisition combined with ICP registration, significantly improves the adaptability and real-time performance in a flexible manufacturing environment. With the help of k-d tree and direction centrality index, the accuracy and robustness of weld defect identification are improved, especially suitable for handling complex reflection, occlusion or deformation welding environment. In addition, the scheme integrates the positioning and detection process before and after welding, realizes the whole process tracking, eliminates information island, improves the integration level of quality control, and effectively reduces the equipment redundancy and maintenance cost.
[0017] The k-d tree, i.e. K-Dimensional Tree, is a data structure for organizing K-dimensional space data, mainly used for searching key data in multi-dimensional space, such as range search and nearest neighbor search.
[0018] In step 1, the AGV car carrying the mechanical arm can be controlled to move the gripped scanner to the current welding process specified vehicle body welding part area, and the vehicle body in the welding process area is scanned to obtain high-precision current welding part three-dimensional data; Optionally, the driving path of the AGV car is preset, and the AGV car carrying the mechanical arm is controlled to drive along the preset process path to the current welding process specified vehicle body welding part area.
[0019] Further, through real-time communication between the AGV car and the main control system, the welding process planning data of the current vehicle model is obtained, including the three-dimensional boundary box coordinates of the to-be-scanned area, the sequence of scanning path planning points, and the scanner parameter configuration parameters. Based on the obtained data, the mechanical arm gripped 3D line laser scanner is controlled to scan the target area according to the preset scanning trajectory.
[0020] Further, in step 2, the scanning data and the pre-stored standard model are registered by ICP, the coordinate system calibration is performed to obtain the spatial correspondence, and the method for obtaining the welding point position coordinates in the coordinate system of the industrial robot performing welding includes the following steps: Step 21, the scanning data and the pre-stored standard model are matched by using the ICP algorithm, the standard model point cloud is matched with the current collected vehicle body point cloud, the rotation matrix and the translation vector between the two are obtained, and the conversion relationship between the visual tracking system coordinate system and the standard model coordinate system of the current collected data is obtained; Step 22, the conversion relationship between the visual tracking system coordinate system and the industrial robot coordinate system is obtained by the hand-eye calibration method; Step 23, the welding point position in the visual tracking system coordinate system is obtained based on the welding point position of the standard model, the welding coordinate position in the industrial robot coordinate system is obtained based on the conversion relationship between the visual tracking system coordinate system and the industrial robot coordinate system, and the industrial robot is controlled based on the obtained welding coordinate position to perform welding; The body point cloud data of the current vehicle obtained in step 1 is iteratively closest point matched with the pre-stored standard body model with welding point information, so as to accurately locate the coordinates of the current body welding point. First, a standard body model point cloud containing welding point information is constructed, and the standard body model point cloud is in a standard model coordinate system; the real-time scanned body point cloud obtained in step 1 is in a visual tracking system coordinate system, that is, a visual tracking system scanner itself coordinate system; Step 21, the scanning data is subjected to ICP algorithm (Iterative Closest Point) with the pre-stored standard model, the standard model point cloud (P) is matched with the current collected body point cloud (Q), the rotation matrix and the translation vector between the two are obtained, that is, the conversion relationship between the visual tracking system coordinate system of the current collected data and the standard model coordinate system is obtained, including the following steps: Step 211, for each point in the standard model point cloud set P , the nearest point in the scanned body point cloud set Q is found , the point pair is established ; Step 212, after the established point pair is decentralized, the covariance matrix H of the standard model point cloud set and the body point cloud set is constructed; ; Among them, is the point after decentralization, is the point after decentralization, indicates the number of point pairs; Step 213, whether the convergence condition is met is judged, whether the minimum distance between the standard model point cloud set and the body point cloud set is lower than the threshold value is judged, when the convergence condition is not met, step 211 is executed for iterative matching, until the minimum value of the distance square between the matched point pairs in the two point cloud sets is lower than the set threshold value; Among them, the setting is:
[0021] Among them, indicates the i-th point in the standard model point cloud set P, indicates the j-th point in the body point cloud set Q, indicates the number of all points in the standard model point cloud and the body point cloud, is the nearest point in the set Q to .
[0022] Step 214, the covariance matrix H matched by iteration is decomposed by SVD to obtain the rotation matrix and the translation vector, that is, the conversion relationship between the standard model coordinate system and the visual tracking system coordinate system of the current collected data; The covariance matrix H is decomposed by SVD: ; The rotation matrix is obtained as: ; The translation vector is: ; Finally, the rigid conversion relationship from the standard model coordinate system (M) to the visual tracking system coordinate system of the current collected data is obtained: ; In the above embodiment, through the ICP algorithm, the standard model point cloud and the current vehicle body point cloud are iteratively registered, the optimal rotation and translation between the two are calculated by using the nearest point matching and SVD decomposition, and the spatial correspondence relationship between the model welding point coordinates and the actual workpiece position is established, thereby providing accurate spatial reference for subsequent welding guidance.
[0023] In step 22, the conversion relationship of the coordinate system is identified. First, the scanner system and the industrial welding robot system are calibrated, the three-dimensional checkerboard pattern calibration piece is scanned, the coordinates of all corner points of the calibration piece are calculated by the face-face intersection method, the RANSAC robust plane fitting method is used to calculate the maximum plane, and the required corner point coordinates are selected. According to the basic formula AX=XB, the conversion relationship between the visual tracking system coordinate system under the scanner system and the industrial robot coordinate system is solved.
[0024] Further, the rotation matrix and the translation vector obtained by the ICP matching method in step 21 are combined, the welding point position information under the current visual tracking system is calculated based on the welding point position of the standard model. Through the eye-in-hand calibration method, the conversion relationship between the visual tracking system coordinate system and the industrial robot coordinate system is determined, and the welding point position information under the visual tracking system coordinate system is again translated and rotated to obtain the welding point position information under the industrial robot coordinate system
[0025] When the AVG car exits the working area, the industrial robot carries the welding gun to weld the car body; specifically, after completing the scanning task in step 1, the AGV sends a "request to exit" signal to the main control system; after receiving the signal, the main control system confirms the safe standby position. The AGV follows the preset exit path. After the AGV completely exits the working area, it sends an "exit confirmation" signal to the main control system, and the main control system activates the safety protection system of the working area. The welding robot receives the actual welding point coordinate data calculated in step 2, and performs welding operations according to the preset welding process plan, completes the welding of all welding points, and returns to the safe standby position.
[0026] After the welding operation is completed, the mechanical arm holding the scanner is controlled again to scan the welded car body part, obtain the point cloud data of the welded car body, and compare it with the designed standard model data to complete the automatic detection of welding quality.
[0027] In step 4, the post-welding point cloud data and the standard model point cloud are respectively constructed into k-d trees, and then the direction centrality index is calculated. The method for determining whether the welding point has defects based on the direction centrality index includes the following steps: Step 41, the obtained welding point cloud data and the standard model point cloud data are constructed into k-d trees; Step 42, use k-d tree for fast k-nearest neighbor point search to obtain the nearest neighbor point set for local feature calculation.
[0028] Step 43, based on the k-d tree corresponding to the point cloud data, analyze the angle and triangular facet distribution of the welding point neighborhood point cloud, and calculate the direction centrality index (DCM); Step 44, calculate the difference value of the obtained direction centrality index (DCM value), and judge whether the welding point has defects based on the set threshold value; In this embodiment, through the point cloud analysis and DCM calculation method, combined with spatial grid division and distance evaluation, the welding point defects and car body welding deformation are automatically identified and positioned, the welding quality detection and visual analysis are realized; the spatial topography is judged by means of k-d tree acceleration structure, triangular area variance, Qhull algorithm, etc.; the whole car body welding deformation is quantified by grid analysis. The three-dimensional topographic analysis method is innovatively introduced, which fills the gap of post-welding automatic detection.
[0029] In step 4, the post-welding point cloud data and the standard model point cloud are respectively constructed into k-d trees, and then the direction centrality index is calculated. The method for determining whether the welding point has defects based on the direction centrality index includes the following steps: In step 41, the obtained welding point cloud data and the standard model point cloud data are constructed into k-d trees; The method of constructing two point cloud data into k-d trees is the same. The method of establishing a k-d tree structure for the welding point cloud comprises the following steps: Step 411, according to the welding point position obtained in step 2, extracting the point cloud data of the post-welding welding point; Step 412, statistical filtering and preprocessing of the welding point cloud data: Specifically, the extracted welding point cloud is filtered and processed, such as denoising, sparsification, etc.
[0030] Step 413, using k-d tree as the spatial index structure of the welding point cloud data, constructing k-d tree: creating left and right sub-trees, in the process of constructing k-d tree, according to the variance of coordinate dimension difference as the basis for partitioning, recursively dividing the point cloud data into left and right sub-trees, until each leaf node contains only one data point; This embodiment adopts a method based on the size of dimension variance to determine the cutting order; the variance calculation method of each dimension of the point cloud data point is as follows: ; In the formula, is the coordinate mean, is the dimension variance.
[0031] Step 42, using k-d tree for fast k-nearest neighbor point search to obtain a set of neighbor points for local feature calculation.
[0032] Each welding point in the welding point cloud performs k-nearest neighbor search in its own point cloud and the standard model point cloud, respectively, to obtain two local point sets: 1) Local point cloud of post-welding scanning points, which is the actual collected point cloud after welding; 2) Local point cloud of standard model points, which is the pre-stored point cloud of the standard model; This embodiment performs neighbor point search on the scanned body point cloud data, creates a point set P to store point data, and k is the number of neighbor points to be searched.
[0033] Step 43, based on the k-d tree corresponding to the point cloud data, performing angle and triangular facet distribution analysis on the spatial structure of the welding point neighborhood point cloud, calculating the direction centrality index (DCM), comprising the following steps: Step 431, two-dimensional local direction centrality (DCM) calculation: in a two-dimensional projection plane (such as the x-y plane), taking the welding point as the center, connecting the neighborhood points to form an included angle, calculating the variance of the angle to obtain the two-dimensional local direction centrality (DCM) value; The embodiment distinguishes the boundary points and internal points in the point set according to the distribution of the points in the neighborhood. To measure the difference of the directional distribution, the variance of the angle formed by connecting the query point with the points in the neighborhood in the two-dimensional space is defined as the local directional centrality (DCM). The DCM method is used to realize the evaluation of the solder joint forming quality.
[0034] ; In the formula, k represents the number of adjacent points in the neighborhood of the query point, represents the included angle between the connecting line of the two adjacent points and the query point.
[0035] Step 432, the neighborhood points are connected into several triangular facets by using the convex hull algorithm (such as Qhull), the area S of each triangle is calculated, and the discrete degree of the areas of all the triangles is used as the local directional consistency value (DCMT), that is, the local directional centrality value in the three-dimensional space; The adjacent points are connected with the adjacent points on the spherical surface to form spherical triangles, the DCM is expanded to the variance of the solid angle of the triangle, and the Qhull algorithm is used to construct the convex complex of the adjacent points. The area of the triangle on the three-dimensional spherical surface is calculated as follows: assuming that the vertices of the triangle are A(x0, y0, z0), B(x1, y1, z1), and C(x2, y2, z2). The area S of the triangle to be solved is: ; To measure the local directional distribution of the points in the three-dimensional space, the variance of the area of the triangle connected by the convex hull of the points in the neighborhood of the query point in the three-dimensional space is defined as the local directional centrality in the three-dimensional space: ; In the formula, Si is the area of the i-th triangle, is the average value of the area.
[0036] The directional centrality indexes constructed in the embodiment include the two-dimensional local directional centrality (DCM) value and the local directional centrality value in the three-dimensional space. The two-dimensional local directional centrality (DCM) value reflects the uniformity of the directions around the point in the projection plane, and the local directional centrality value in the three-dimensional space reflects the distribution consistency of the neighborhood facets of the point in the three-dimensional space. The two indexes can be used as the characteristic parameters for evaluating the local forming quality of the solder joint.
[0037] Step 44, the difference value is calculated for the obtained directional centrality index (DCM value), and whether the solder joint has defects is judged based on the set threshold value; The difference value of the directional centrality index (DCM value) is calculated as follows: ; In the formula, DCM detectDCM represents the local directional centrality of a point in the post-weld solder point cloud within its own neighborhood, and DCM reference DCM represents the local directional centrality of a point in the post-weld solder point cloud within its own neighborhood, and DCM
[0038] In the area where no surface change occurs, the spatial distribution of the scanned solder point cloud data is similar to that of the standard model solder data, and the value of ΔDCM is small. In the changed area, the spatial distribution of the scanned solder point cloud data is quite different from that of the standard model solder data, and the value of ΔDCM is also higher. When the value of ΔDCM is higher than the set threshold, it is considered that the current solder has defects.
[0039] The above realizes the identification of the solder, and for the shape deformation of the vehicle body, the post-weld vehicle body point cloud can be collected, the post-weld vehicle body point cloud is divided into a three-dimensional grid, and by comparison with the standard model, the welding deformation area is calculated and located, and the deformation degree is quantified, thereby providing data support for subsequent process optimization; In step 5, the method for calculating the welding deformation amount of the post-weld vehicle body point cloud includes the following steps: Step 51, pre-process the post-weld vehicle body point cloud, and register the point cloud data of the standard model before welding; As shown in Fig. 3 The embodiment performs statistical filtering, downsampling and other preprocessing operations on the current post-weld complete vehicle body point cloud. The post-weld vehicle body point cloud data is registered with the pre-weld standard point cloud data.
[0040] Step 52, grid division is performed on the registered post-weld point cloud data, which is divided into regular three-dimensional grid units, each grid being a spatial voxel, and a regular three-dimensional grid is established.
[0041] The grid division formula is as follows:
[0042] Wherein, represents the grid unit of the i-th row, j-th column and k-th layer, is the point cloud data point falling into the grid.
[0043] Step 53, neighborhood structure construction in the grid: a local k-d tree data structure is constructed for the point cloud data in each grid unit to accelerate the search for near neighbors; and is used for fast correspondence with the same position point cloud of the standard model.
[0044] Step 54, calculate the average Euclidean distance between the post-weld point cloud and the standard model point cloud in each grid unit; the average distance is used as an index of the welding deformation amount, and is used to judge whether the local welding exceeds the design allowable range; Optionally, the distance threshold can be set according to the material properties of the vehicle body, welding process requirements and design tolerances, and when the average distance in the grid cell exceeds the threshold, it is determined that the vehicle body part in the region is severely deformed and does not meet the process requirements. The number and distribution of all grid cells exceeding the threshold are counted to generate a vehicle body welding deformation distribution map, and the position coordinates and deformation of the severely deformed region are output.
[0045] This step realizes the spatial quantitative evaluation of welding deformation by dividing the post-weld vehicle body point cloud into three-dimensional grid cells, constructing a local KD tree for fast matching, and calculating the average distance of the point cloud in the grid to the standard model, providing accurate basis for defect positioning and process improvement.
[0046] In this embodiment, based on the spatial geometric analysis method of three-dimensional point cloud, not only the accurate positioning of the welding point is realized before welding through ICP registration, but also the DCM (local direction centrality) algorithm is proposed after welding, which quantifies the welding point forming quality from the three-dimensional topography angle by combining the spatial features such as triangle area variance and neighborhood angle distribution difference. At the same time, by constructing a three-dimensional grid, the welding deformation of the whole vehicle body is analyzed, and a visual deformation distribution map can be output, which is more suitable for intelligent evaluation of welding quality of large-size complex curved surface structures such as automobile body-in-white.
[0047] Embodiment 2 Based on embodiment 1, an integrated control system for welding point positioning and detection based on 3D vision is provided in this embodiment, which includes: The first acquisition module is configured to perform three-dimensional point cloud scanning on the vehicle body of the current vehicle to obtain three-dimensional point cloud data of the vehicle body; The welding point position recognition module is configured to use ICP registration on the scanning data and the pre-stored standard model to perform coordinate system calibration to obtain the spatial correspondence, and obtain the welding point position coordinates in the coordinate system of the industrial robot performing welding, so as to control the industrial robot to perform welding operation; The second acquisition module is configured to perform three-dimensional point cloud scanning on the vehicle body of the vehicle after welding to obtain post-weld welding point cloud data; The detection module is configured to construct k-d trees for the post-weld welding point cloud data and the standard model point cloud respectively, and then calculate the direction centrality index, and determine whether the welding point has defects based on the direction centrality index.
[0048] It should be noted that each module in this embodiment corresponds to each step in embodiment 1, and the specific implementation process is the same, which will not be repeated here.
[0049] Embodiment 3 The embodiment provides an electronic device, comprising a memory and a processor and computer instructions stored on the memory and running on the processor, when the computer instructions are run by the processor, steps in the 3D vision-based welding point positioning and detection integrated control method of the embodiment 1 are completed.
[0050] Embodiment 4 The embodiment provides a computer readable storage medium for storing computer instructions, when the computer instructions are executed by a processor, steps in the 3D vision-based welding point positioning and detection integrated control method of the embodiment 1 are completed.
[0051] The above merely provides preferred embodiments of the present application but is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
[0052] The above describes the specific embodiments of the present application in combination with the accompanying drawings, but is not intended to limit the protection scope of the present application, and those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.
Claims
1. A 3D vision-based integrated control method for solder joint positioning and inspection, characterized in that, The method comprises the following steps: a three-dimensional point cloud scanning is performed on the body of the current vehicle to obtain three-dimensional point cloud data of the body; the scanning data is matched with the pre-stored standard model by using ICP registration to perform coordinate system calibration to obtain the spatial correspondence, and the position coordinates of the welding points in the coordinate system of the industrial robot performing welding are obtained to control the industrial robot to perform welding operation; a three-dimensional point cloud scanning is performed on the body of the vehicle after welding to obtain post-welding point cloud data of the welding points; the post-welding point cloud data of the welding points and the standard model point cloud are respectively constructed into k-d trees, and then the direction centrality index is calculated to determine whether the welding points have defects based on the direction centrality index.
2. The 3D vision-based integration control method of the solder joint positioning and inspection according to claim 1, characterized in that: For the post-welding body point cloud, grid division analysis is performed to calculate the welding deformation, comprising the following steps: the post-welding body point cloud is preprocessed, and the pre-welding standard model point cloud data is matched; the matched post-welding point cloud data is divided into regular three-dimensional grid units, each grid is a spatial voxel, and a regular three-dimensional grid is established; a local k-d tree data structure is constructed for the point cloud data in each grid unit to accelerate the search for neighboring points; the average Euclidean distance between the post-welding point cloud and the standard model point cloud in each grid unit is calculated; the average distance is used as the welding deformation index.
3. The 3D vision-based integration control method of solder joint positioning and inspection according to claim 1, characterized in that: The method for matching the scanning data with the pre-stored standard model by using ICP registration to perform coordinate system calibration to obtain the spatial correspondence and the position coordinates of the welding points in the coordinate system of the industrial robot performing welding comprises the following steps: the scanning data is matched with the pre-stored standard model by using ICP algorithm to match the standard model point cloud with the current collected body point cloud to obtain the rotation matrix and translation vector between them, and the conversion relationship between the visual tracking system coordinate system of the scanning collected data and the standard model coordinate system is obtained; the conversion relationship between the visual tracking system coordinate system and the industrial robot coordinate system is obtained by hand-eye calibration method; the welding point position in the visual tracking system coordinate system is obtained based on the welding point position of the standard model, that is, the welding point position information in the current visual tracking system; based on the conversion relationship between the visual tracking system coordinate system and the industrial robot coordinate system, the welding coordinate position in the industrial robot coordinate system is obtained.
4. The 3D vision-based integration control method of solder joint positioning and inspection according to claim 3, characterized in that: The method for matching the scanning data with the pre-stored standard model by using ICP algorithm to match the standard model point cloud with the current collected body point cloud to obtain the rotation matrix and translation vector comprises the following steps: Step 211, for each point in the standard model point cloud set P , find the nearest point in the scanned body point cloud set Q , establish a point pair ; step 212, after the established point pairs are decentralized, the covariance matrix H of the standard model point cloud set and the body point cloud set is constructed; step 213, whether the convergence condition is met is judged, that is, whether the minimum distance between the standard model point cloud set and the body point cloud set is lower than the threshold value, and when the convergence condition is not met, step 211 is executed for iterative matching until the two point cloud sets are lower than the set threshold value; step 214, the covariance matrix H after iterative matching is decomposed by SVD to obtain the rotation matrix and translation vector.
5. The 3D vision-based integration control method of solder joint positioning and inspection according to claim 1, characterized in that: The post-weld spot point cloud data and the standard model point cloud are respectively constructed into k-d trees, and then a direction centrality index is calculated. The acquired spot point cloud data and the standard model point cloud data are constructed into k-d trees. The k-d trees are used for fast k-neighbor point searching to obtain a neighbor point set for local feature calculation. The spatial structure of the spot neighborhood point cloud is analyzed based on the k-d tree corresponding to the point cloud data, and a direction centrality index is calculated. A difference value of the obtained direction centrality index is calculated, and whether the spot has a defect is determined based on a set threshold.
6. The 3D vision-based integration control method of solder joint positioning and inspection according to claim 5, characterized in that: The spatial structure of the spot neighborhood point cloud is analyzed based on the k-d tree corresponding to the point cloud data, and a direction centrality index is calculated. Two-dimensional local direction centrality calculation: in a two-dimensional projection plane, the neighborhood points are connected to form an angle with the spot as the center, and the variance of the angle is calculated to obtain a two-dimensional local direction centrality value. The neighborhood points are connected into several triangular facets using a convex hull algorithm, the area S of each triangle is calculated, and the discrete degree of the area of all the triangles is used as a local direction consistency value, i.e., a local direction centrality value in a three-dimensional space.
7. The integrated control system for the positioning and detection of welding points based on 3D vision, characterized in that, It comprises: An AVG car, an industrial robot arm, and a main control system; The AVG car is used to carry the industrial robot arm to realize movement; The industrial robot arm is used to carry a welding gun to realize welding on the vehicle body or carry a scanner to scan the vehicle body to obtain vehicle body point cloud data; The main control system is configured to execute the 3D vision-based spot positioning and detection integrated control method of any one of claims 1-6 to control the positioning of the spot and the detection of the welding quality.
8. The integrated control system for the positioning and detection of welding points based on 3D vision, characterized in that, It comprises: A first acquisition module configured to perform three-dimensional point cloud scanning on the vehicle body of the current vehicle to obtain three-dimensional point cloud data of the vehicle body; A spot position recognition module configured to use ICP registration on the scanned data and the pre-stored standard model to perform coordinate system calibration to obtain a spatial correspondence, and obtain a spot position coordinate in the coordinate system of the industrial robot arm performing welding to control the industrial robot arm to perform welding operation; A second acquisition module configured to perform three-dimensional point cloud scanning on the vehicle body of the welded vehicle to obtain post-weld spot point cloud data; A detection module configured to construct the post-weld spot point cloud data and the standard model point cloud into k-d trees, and then calculate a direction centrality index, and determine whether the spot has a defect based on the direction centrality index.
9. An electronic device, comprising: It comprises a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the steps in the 3D vision-based spot positioning and detection integrated control method of any one of claims 1-6 are completed.
10. A computer-readable storage medium, characterized in that, It is used to store computer instructions, when the computer instructions are executed by the processor, the steps in the 3D vision-based spot positioning and detection integrated control method of any one of claims 1-6 are completed.