Automatic riveting welding method based on AI visual intelligent system

Abstract

The invention discloses an automatic riveting welding method based on an AI visual intelligent system, and belongs to the technical field of intelligent welding workpieces. According to the method, an ROS is used for commanding a stereoscopic vision system Stero to measure and sense an image, the image is used for constructing a polygonal mesh three-dimensional model, and then qualified workpieces are compared and screened through Harrris corner detection. In the positioning stage, the system performs convolution operation according to an SIFT feature matching algorithm to obtain a dynamic weight sum index, and automatically adjusts a workpiece space angle through UG numerical control to realize millimeter-level accurate positioning. According to the method, a workpiece placement angle-welding quality mapping model is constructed through deep learning of welding process historical data, workpiece inclination angle parameters are optimized in real time in the UG numerical control adjustment process, and compared with traditional manual positioning, the welding deformation is greatly reduced, the fatigue life of products is prolonged, and the manufacturing qualification rate of high-value workpieces is remarkably increased.

Description

Technical Field

[0001] This invention belongs to the field of intelligent welding workpiece technology, specifically, it relates to an automated riveting and welding method based on an AI vision intelligent system. Background Technology

[0002] Precision riveting and welding is a core process for achieving highly reliable connections in metal structural components, holding strategic significance in high-end manufacturing fields such as aerospace and new energy vehicles. Compared to traditional welding, intelligent welding features dynamic perception and closed-loop control. It integrates machine vision, artificial intelligence, and real-time control technologies, achieving a leapfrog upgrade from experience-driven to data-driven approaches, which is crucial for extending equipment lifespan.

[0003] Automated riveting and welding methods based on AI vision intelligence systems play an irreplaceable role in improving social safety management and preventing and reducing safety accidents. With continuous technological advancements and expanding applications, these methods will become increasingly intelligent and precise, better serving social development and people's needs.

[0004] In existing technologies, common automated riveting and welding methods based on AI vision intelligence systems involve image recognition of the workpiece to be processed, obtaining and processing the image; performing positioning detection on the pre-processed image to obtain the riveting point position; then planning the riveting path based on the riveting point position to obtain the riveting path, and achieving automatic riveting based on the riveting path. However, this method is less effective when welding precision workpieces, potentially significantly shortening the workpiece's lifespan after welding and easily leading to large errors during the welding process. To address these issues, this invention provides the following technical solution. Summary of the Invention

[0005] The purpose of this invention is to provide an automated riveting and welding method based on an AI vision intelligent system, which solves the problem that large errors in welding precision workpieces in the prior art lead to a significant reduction in the service life of the welded workpieces.

[0006] The objective of this invention can be achieved through the following technical solutions: An automated riveting and welding method based on an AI vision intelligence system includes: Real-time image capture of the car door workpiece is achieved using an industrial camera. The actual size and structural features of the workpiece are calculated based on the image, and defect detection is performed. A three-dimensional model of the workpiece is constructed, and the overlap index is calculated based on the comparison results between the three-dimensional model and the preset theoretical model to evaluate the accuracy of the workpiece placement position. When the overlap index meets the preset threshold, the robotic arm is controlled to perform a welding operation.

[0007] Furthermore, calculating the actual dimensions of the workpiece includes: Pixel equivalents are used to convert pixel distances in an image into physical dimensions.

[0008] Furthermore, defect detection includes: Gray-level histograms were used to analyze the gray-level distribution of the image, and the area, perimeter, and circularity of connected regions were calculated using Blob analysis to identify surface contaminants.

[0009] Furthermore, constructing a 3D model of the workpiece includes: performing a 3D contour scan based on the principle of triangulation to obtain the 3D point cloud data of the workpiece.

[0010] Furthermore, defect detection also includes: using a deep learning-based image recognition algorithm to identify the location of solder joints and outputting a set of solder joint bounding boxes.

[0011] Furthermore, it also includes: monitoring the temperature of the weld joint in real time using a thermal imager, and recording the welding completion status when the temperature reaches the determination temperature in multiple consecutive frames of images.

[0012] Furthermore, calculating the coincidence index includes: scanning the workpiece from multiple angles using a stereo vision system, extracting feature points, comparing their positions and angles with the corresponding feature points in the theoretical model, and calculating the coincidence index based on the deviation rate.

[0013] Furthermore, the feature points are identified using the Harris corner detection algorithm and compared with a qualified workpiece corner model to screen for quality defects.

[0014] Furthermore, controlling the robotic arm to perform welding operations includes: using UG CNC technology to plan the robotic arm's motion trajectory, and achieving precise placement and welding based on a vision servo controller and impedance contact control.

[0015] Furthermore, it also includes: using a robot operating system to collaboratively control multiple industrial cameras to acquire workpiece images from different angles, and scheduling a vision processing module to perform 3D reconstruction and feature extraction.

[0016] The beneficial effects of this invention are: Improve connection accuracy and reliability; By integrating machine vision and real-time control technologies, dynamic perception and closed-loop control are achieved, significantly reducing welding errors and ensuring the connection strength and long-term reliability of metal structural components (especially in the fields of aerospace and new energy vehicles).

[0017] Extend the service life of equipment; By optimizing data-driven intelligent processes, the quality of riveting can be precisely controlled, avoiding workpiece damage caused by traditional methods and fundamentally improving the service life of the welded workpiece.

[0018] Enhance safety production guarantees; Intelligent systems can proactively identify risks and adjust processes in real time, playing a key role in preventing safety accidents and reducing potential human error hazards, thereby improving the overall level of safety management.

[0019] Promote the intelligent upgrading of manufacturing; Transforming traditional, experience-dependent processes into quantifiable and replicable data models enables a leap from "experience-driven" to "data-driven" approaches, providing standardized and efficient solutions for the high-end manufacturing sector.

[0020] Adaptable to complex and precision scenarios; To address the shortcomings of existing technologies in processing precision workpieces, this solution optimizes visual recognition and path planning algorithms to ensure high-precision workpiece welding quality and meet the stringent process requirements of high-end fields. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the processing flow of the method of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, this application provides an automated riveting and welding method based on an AI vision intelligence system; As an embodiment 1 of this application, it specifically includes: Real-time image capture of the car door workpiece is achieved using an industrial camera. The actual size and structural features of the workpiece are calculated based on the image, and defect detection is performed. A three-dimensional model of the workpiece is constructed, and the overlap index is calculated based on the comparison results between the three-dimensional model and the preset theoretical model to evaluate the accuracy of the workpiece placement position. When the overlap index meets the preset threshold, the robotic arm is controlled to perform a welding operation.

[0024] Example 2; This example is implemented based on Example 1, and specifically includes the following steps: Step one, taking a car door welding production line in a car manufacturing plant as an example, involves a high-precision external industrial camera installed high up next to a robotic arm on a modern industrial automated welding production line to capture real-time images of the workpiece in the welding area. The pixel equivalent is determined in advance by calculating the camera calibration using a standard calibration board. Pixel equivalent = ; Pixel equivalents are used to calculate the actual size of the workpiece. Actual size = ; Obtain the actual dimensions of the workpiece.

[0025] Structural form recognition is performed through feature point matching, and Euclidean distance is used to determine workpiece similarity. d= ; First, straight lines are fitted to the two edges of the bevel to obtain the corresponding direction vectors for each edge, denoted as a and b. Then, the angle θ between the two lines is calculated using the relationship between the dot product of the vectors and their magnitudes. The formula for calculating the bevel angle is: ; For measuring the root gap and blunt edge height, the method used is to first obtain the pixel distance, and then convert the pixel distance into physical size according to the size measurement formula, so as to obtain the actual value of the root gap or blunt edge height.

[0026] For the workpiece surface, grayscale histogram analysis is used to analyze the image's grayscale distribution and detect abnormal areas. Then, blob analysis is used to distinguish contaminants (irregular spots) from normal surfaces, calculating the area A, perimeter P, and roundness C of connected regions. The closer the value is to 1, the closer the shape is to a circular oil stain. ; The workpiece is scanned and a 3D model is constructed using 3D contour scanning based on the principle of triangulation. f is the camera focal length, d is the baseline distance between the camera and the laser emission point, and x... shift The pixel offset of the laser line in the image ; Obtain 3D point cloud.

[0027] The visual processing module employs a deep learning-based image recognition algorithm, using Faster R-CNN as the base model. Let the input image be I, (x...) i y i For convenient use of the center coordinates, w i h i s represents the width and height of the bounding box. iConfidence scores are assigned to the weld joints. The model is trained using a large amount of post-welding image data, and outputs a set of weld joint bounding boxes. Based on the bounding box BK, let the bounding box mask be M, where ⊙ represents element-wise multiplication. Extract the thermal imaging region: During the welding of the door frame weld joints, the thermal imager monitored the weld joint temperature T(t). ; As the solder joint temperature gradually decreases from the high temperature during welding, and reaches the judgment temperature for three consecutive frames... The system records this data, where Δt is the image sampling interval, typically Δt = 100ms; image segmentation technology is used to analyze the weld surface, assuming the segmentation network output is a pixel-level mask. When S(x,y)=1, the image segmentation is considered to correspond to the weld area; conversely, when S(x,y)=0, the image segmentation is considered to correspond to the background. When the machine welds the weld at the connection between the door panel and the frame, impurities are detected within the weld area S. ; Image segmentation algorithms analyze three consecutive frames of images to determine... ; If the conditions are met, the weld surface is considered clean and free of impurities. When welding the door handle mounting point, extract the point cloud of the weld cross-section P={(x j y j , z j The system calculates the weld width W based on the distance between the two boundary points. ; The system calculates the maximum height H relative to the substrate plane. ; Weld joints are only considered acceptable if the tolerance is within 1.2mm. Only when all three conditions are met in three consecutive frames of images will the system finally determine that the previous welding operation is complete and control the robotic arm to reset to the workpiece placement origin, preparing for the placement of a new workpiece.

[0028] Before placing a new workpiece, the Stereo stereo vision system, integrated directly above the worktable, initiates the scanning process. During operation, when scanning the car door workpiece, the system obtains its three-dimensional coordinates through binocular parallax-depth mapping. ; Wherein, the focal length of the left / right camera is f, the baseline distance is b, the pixel parallax is d, and (c x ,cy () represents the optical center coordinates; binocular parallax technology is existing technology, so it will not be described in detail here.

[0029] Then, quadratic surface fitting is used to improve accuracy. ; Where C(d) is the matching cost function.

[0030] Then, feature point extraction is performed using scale-space extremum detection. ; Where G is the Gaussian kernel, K= ; Using a semi-global matching cost function for dense point cloud reconstruction E(D) = ; Where P1=0.8 and P2=2.5 are penalty coefficients.

[0031] Structural features are determined through edge detection and void feature analysis. Where Gx and Gy are the results of Sobel convolution.

[0032] The above technologies enable the rapid construction of a 3D model of the car door, clearly presenting its shape and structure; the Stero stereo vision system is existing technology, so it will not be described in detail.

[0033] Step two: After scanning the car door workpiece, the system immediately uses Harris corner detection and comparison technology to perform preliminary screening of the workpiece's quality and positioning. The Harris corner detection algorithm identifies corners by calculating the corner response function of each pixel in the image based on local grayscale changes in the image. These corners usually correspond to key parts of the car door workpiece, such as the edges of the door frame and the corners of the handle mounting holes.

[0034] The system will compare the detected corner information with the pre-stored qualified workpiece corner model in the following aspects: When analyzing the door weld area, if the detected corner distribution density at a certain weld section is... ; And the corresponding grayscale value is ; This indicates that the weld seam is now exhibiting a spot-like pattern, different from a normal weld. The system combines the grayscale value and the distribution of corner points to determine that the area may have porosity defects. For example, on the weld seam where the side panel and door panel of a car door are welded, the system finds a dense area of ​​corner points with a darker grayscale value, which differs from historical data by 30%. This indicates the presence of porosity, and the car door part is immediately deemed unqualified and removed from the production line. If corner points are observed to cluster, and the grayscale distribution of the clustered area exhibits abnormal blocky or striped characteristics, the system will identify whether there is a slag inclusion problem. The system will first calculate the corner clustering degree. ; When Cluster > 0.8, it is considered an abnormal cluster; Next, calculate the gray-level co-occurrence matrix features, θ contrast =2 * Preset Normal Value ; For example, at the welding point of the door hinge installation area, the corners are clustered and the gray scale is unevenly distributed. The system judges that there is slag inclusion, and the door cannot pass the quality inspection. When a break in the continuity of a weld occurs at a corner, the system will determine whether there are defects such as incomplete penetration or lack of fusion at that point.

[0035] The system first uses corner continuity analysis to determine the physical length of the corner fracture caused by defects along the weld path. ; Defect Index = ; L critical =0.1 * total weld length; If the weld between the bottom crossbeam and the door panel is interrupted at a point 10cm from the door panel weld, the system calculates the ratio of the interruption length to the maximum interruption length. If the ratio is greater than 1, the interruption affects the welding and there is incomplete penetration. The door will be deemed unqualified. If the ratio is less than 1, the welding will not be affected. If a crack appears in an image as an abnormal extension of a corner point along a specific direction, the system detects the crack by analyzing the connection relationship of the corner points.

[0036] ; ; Where, θ chain Let θ be the principal direction of the corner chain calculated based on PCA. Design To determine the direction of stress. L max W represents the maximum crack length. avg This represents the average width.

[0037] In this example, a corner with a stress concentration point on the car door was found to have a thin, elongated trend. The system detected the crack in time and removed the car door from the production line. In addition, the system can also identify defects such as dents, undercuts, and weld beads by analyzing the geometric distribution and shape characteristics of the corner points. Once these problems are detected, the car door workpiece cannot pass quality inspection.

[0038] In the initial positioning screening stage, key feature points of the car door workpiece, such as the four corners of the door and the edges of mounting holes, are obtained through Harris corner detection. These points are compared with the feature point positions in the theoretical model to calculate the positional deviation. If the deviation is within the allowable range (±0.3mm), the car door is considered to have initially passed the positioning stage and can proceed to subsequent positioning steps; otherwise, the workpiece will be readjusted or deemed unqualified. For example, in one inspection, it was found that the corner point on one edge of the car door deviated from the theoretical position by 0.4mm, exceeding the allowable range. The system controlled the robotic arm to readjust the car door until the deviation met the requirements before allowing the car door to proceed to the next welding process.

[0039] Step 3: After the door component successfully passes the Harris corner detection and comparison, Let the workpiece coordinate system obtained from corner detection be... The base coordinate system of the robotic arm is F b ={O b ,X b ,Y b Z b} The rotation matrix R is calculated using the three-point positioning method. , , ; Specify welding position p target =(0,0,z0) in the tooling coordinate system F f Expressions in: ; Catch point The displacement to the target point is ; After confirming that the basic requirements for welding work are met, the robotic arm will be used with the help of UG CNC technology to precisely place the car door.

[0040] The objective function of UG CNC trajectory is ; Depend on ; constraint.

[0041] in Joint angle, This is an obstacle zone.

[0042] The motion control of the robotic arm is calculated using the damped least squares method. ; Method for calculating the J elements of the Jacobian matrix ; Among them, z K Let K be the axis of rotation. is the damping coefficient.

[0043] Precise placement control Visual conferencing controller Image-based Visual Gifts (IBVS): ; Image error ; Interaction Matrix ; Impedance contact control Dynamic model: ; Parameter settings: ; Select multiple tangent points evenly along the workpiece contour line. Draw a tangent line at each tangent point and calculate the angle between the tangent line and the horizontal axis. Denote this angle as the characteristic angle α. Calculate the contour characteristic angle. ; Where dx and dy are the differentials of the contour line at the tangent point.

[0044] Then, perform linear regression on all characteristic angles α and their corresponding X coordinates to obtain the slope K of the regression line. The slope K reflects the tilting trend.

[0045] ; After obtaining the standard slope K, calculate the deviation rate between the current workpiece and the standard. ; Then the deviation rate is converted into the coincidence index θ ; Detect outliers that deviate excessively from the regression line. If there are more than 5 points, it is determined that the placement angle is incorrect.

[0046] In this example, door welding requires welding to multiple parts, including the door panel, side panel, and door reinforcement beam. The welding process parameters differ for each part, and the welding sequence is strictly required. UG software comprehensively considers these factors, and combined with the 3D model of the door, plans a complete motion trajectory for the robotic arm, from grasping the door to placing it at the designated welding position, and then welding at each point. The rotation angle and movement distance of each joint of the robotic arm at different times are precisely calculated by the UG software and a CNC program is generated.

[0047] The robotic arm, based on the CNC program generated by UG, places the car door at the designated welding position (X=0, Y=0). This designated welding position is the optimal welding location determined before the welding work begins through a comprehensive analysis of various factors, including the structural characteristics of the car door, welding process requirements, and the motion characteristics of the robotic arm. Taking the welding of the car door to the body frame as an example, to ensure the sealing and smooth opening and closing of the car door after installation, the relative position of the car door and the welding fixture must be precise during welding. Through multiple simulations and actual tests, considering factors such as the car door's center of gravity distribution, the robotic arm's gripping point on the car door, and accessibility during welding, technicians finally determined the designated welding position of the car door on the welding fixture.

[0048] After the robotic arm places the car door at the designated welding position, the system obtains the coincidence index θ, which indicates whether the door's position coincides with the theoretical door position. The system scans the placed door using the Stereo stereo vision system, reconstructs a 3D model of the door, and compares it with a pre-stored theoretical door model. By calculating parameters such as positional and angular deviations of key feature points (e.g., door corners, mounting hole centers) between the two models, a specific algorithm calculates the coincidence index θ. For example, if the deviations of the door's key feature points from the theoretical position are small after placement, the calculated coincidence index θ will be high; conversely, if there are large deviations, the coincidence index θ will be low. The system compares the coincidence index θ with a preset threshold β. Only when the coincidence index θ is greater than or equal to β does it indicate that the actual position of the car door is highly coincident with the theoretical position, meeting the positioning accuracy requirements for welding. The system will then control the welding equipment to start and perform welding work according to the preset welding parameters and process path. If the coincidence index θ is less than β, it means that the placement of the car door does not meet the requirements for welding work. The system will issue an alarm and control the robotic arm to readjust or reposition the car door until the coincidence index θ meets the conditions before welding work can begin.

[0049] Step 4: Once the workpieces are placed and the overlap index condition is met, welding can begin. During the initial debugging phase, the technical team comprehensively considered the positioning accuracy requirements of the laser welding process (the error needs to be controlled within a very small range to ensure uniform welds and good sealing), the ±0.2mm error of the robotic arm during repeated positioning, and factors that may affect welding quality, such as workshop ambient temperature and vibration. After extensive simulation tests and actual verification, the team finally determined the overlap index threshold β to be 0.92.

[0050] During a production run, the system calculated the overlap index θ of the current car door to be 0.95. Since 0.95 ≥ 0.92, meaning θ is greater than or equal to β, this indicates a high degree of overlap between the actual and theoretical positions of the car door, meeting the positioning accuracy requirements for welding. At this point, the system immediately starts the welding equipment. The welding robot performs precise welding at the connection between the car door and the door frame according to preset welding parameters (e.g., welding current 200A, welding speed 30cm / min) and the process path (welding the top edge of the car door first, then the sides and bottom in sequence). During the welding process, a camera monitors the weld in real time to ensure stable welding quality.

[0051] In another operation, the system calculated an overlap index θ of only 0.88, which was less than the threshold β. This meant that the door's placement did not meet the requirements for welding. The system immediately issued a sharp alarm and controlled the robotic arm to readjust the door. Based on the deviation data fed back by the system, the robotic arm precisely adjusted the door's angle and position, and recalculated the overlap index. After two adjustments, the overlap index θ increased to 0.93, meeting the requirements, and the welding equipment restarted to begin the welding process.

[0052] Example 3 Based on Example 2, basic information about the workpiece is obtained; In the door welding process of the aforementioned automobile manufacturing plant, a Robot Operating System (ROS) is introduced for collaborative control based on Example 1 in order to obtain more comprehensive information about the door components. Taking the production of a new type of SUV door as an example, this door has a complex structure, including not only conventional door panels and side panels, but also newly added components such as sound insulation layers and anti-collision reinforcements, which places extremely high demands on the welding process, making accurate acquisition of component information crucial.

[0053] Structural form recognition is performed through feature point matching, and Euclidean distance is used to determine workpiece similarity. d= ; First, straight lines are fitted to the two edges of the bevel to obtain the corresponding direction vectors for each edge, denoted as a and b. Then, the angle θ between the two lines is calculated using the relationship between the dot product of the vectors and their magnitudes. The formula for calculating the bevel angle is: ; For measuring the root gap and blunt edge height, the method used is to first obtain the pixel distance, and then convert the pixel distance into physical size according to the size measurement formula, so as to obtain the actual value of the root gap or blunt edge height.

[0054] For the workpiece surface, grayscale histogram analysis is used to analyze the image's grayscale distribution and detect abnormal areas. Then, blob analysis is used to distinguish contaminants (irregular spots) from normal surfaces, calculating the area A, perimeter P, and roundness C of connected regions. The closer the value is to 1, the closer the shape is to a circular oil stain. ; The workpiece is scanned and a 3D model is constructed using 3D contour scanning based on the principle of triangulation. f is the camera focal length, d is the baseline distance between the camera and the laser emission point, and x... shift The pixel offset of the laser line in the image ; Obtain 3D point cloud.

[0055] The visual processing module employs a deep learning-based image recognition algorithm, using Faster R-CNN as the base model. Let the input image be I, (x...) i y i For convenient use of the center coordinates, w i h i s represents the width and height of the bounding box. i Confidence scores are assigned to the weld joints. The model is trained using a large amount of post-welding image data, and outputs a set of weld joint bounding boxes. Based on the bounding box BK, let the bounding box mask be M, where ⊙ represents element-wise multiplication. Extract the thermal imaging region: During the welding of the door frame weld joints, the thermal imager monitored the weld joint temperature T(t). ; As the solder joint temperature gradually decreases from the high temperature during welding, and reaches the judgment temperature for three consecutive frames... The system records this data, where Δt is the image sampling interval, typically Δt = 100ms; image segmentation technology is used to analyze the weld surface, assuming the segmentation network output is a pixel-level mask. When S(x,y)=1, the image segmentation is considered to correspond to the weld area; conversely, when S(x,y)=0, the image segmentation is considered to correspond to the background. When the machine welds the weld at the connection between the door panel and the frame, impurities are detected within the weld area S. ; Image segmentation algorithms analyze three consecutive frames of images to determine... ; If the conditions are met, the weld surface is considered clean and free of impurities. When welding the door handle mounting point, extract the point cloud of the weld cross-section P={(x j y j , z j The system calculates the weld width W based on the distance between the two boundary points. ; The system calculates the maximum height H relative to the substrate plane. ; Weld joints are only considered acceptable if the tolerance is within 1.2mm. Only when all three conditions are met in three consecutive frames of images will the system finally determine that the previous welding operation is complete and control the robotic arm to reset to the workpiece placement origin, preparing for the placement of a new workpiece.

[0056] Before placing a new workpiece, the Stereo stereo vision system, integrated directly above the worktable, initiates the scanning process. During operation, when scanning the car door workpiece, the system obtains its three-dimensional coordinates through binocular parallax-depth mapping. ; Wherein, the focal length of the left / right camera is f, the baseline distance is b, the pixel parallax is d, and (c x ,c y () represents the optical center coordinates; binocular parallax technology is existing technology, so it will not be described in detail here.

[0057] Then, quadratic surface fitting is used to improve accuracy. ; Where C(d) is the matching cost function.

[0058] Then, feature point extraction is performed using scale-space extremum detection. ; Where G is the Gaussian kernel, K= ; Using a semi-global matching cost function for dense point cloud reconstruction E(D) = ; Where P1=0.8 and P2=2.5 are penalty coefficients.

[0059] Structural features are determined through edge detection and void feature analysis. Where Gx and Gy are the results of Sobel convolution.

[0060] The above technologies enable the rapid construction of a 3D model of the car door, clearly presenting its shape and structure; the Stero stereo vision system is existing technology, so it will not be described in detail.

[0061] The ROS efficient command vision system captures images of the car door from multiple angles. After the car door is placed on the workbench, ROS, according to a pre-set shooting plan, directs eight industrial cameras distributed around and above the workbench to begin working. These cameras capture images of the car door from all angles, following the order of "overall view first, then details; outer view first, then inner view," ensuring that every weld point and every structural transition on the car door is clearly recorded.

[0062] The acquired image data is transmitted to the AI ​​processing module in real time, where the AI ​​uses the Stereo stereo vision system to perform depth analysis. For the complex 3D structure of the car door, the AI ​​employs the MonoDepth2 deep learning-based model to calculate the depth information of various parts of the door from the 2D image. For example, for the anti-collision reinforcement on the inside of the door, the MonoDepth2 model analyzes the texture and shadow features of the reinforcement in the image to accurately calculate its distance from the door surface, constructing the door's 3D spatial coordinates. Subsequently, a mesh generation algorithm is used to transform the discrete 3D point cloud data into a polygonal mesh 3D model. This model accurately reproduces the geometry of the car door and visually displays the subtle grooves, holes, and other structural features on the door surface, while clearly showing the 0.5mm depth change of the door seal mounting groove. ROS, Stereo, and MonoDepth2 models are existing technologies and will not be described in detail.

[0063] Example 4 Building upon Example 2, to achieve accurate assessment of the door placement position, the calculation method for the overlap index θ is further optimized. Taking the welding of the aforementioned SUV door as an example, this door uses a new type of lightweight alloy material, requiring extremely high welding precision. Traditional methods for calculating the overlap index are no longer sufficient to meet production demands.

[0064] Let the workpiece coordinate system obtained from corner detection be... The base coordinate system of the robotic arm is F b ={O b ,X b ,Y b Z b} The rotation matrix R is calculated using the three-point positioning method. , , ; Specify welding position p target =(0,0,z0) in the tooling coordinate system Ff Expressions in: ; Catch point The displacement to the target point is ; After confirming that the basic requirements for welding work are met, the robotic arm will be used with the help of UG CNC technology to precisely place the car door.

[0065] The objective function of UG CNC trajectory is ; Depend on ; constraint.

[0066] in Joint angle, This is an obstacle zone.

[0067] The motion control of the robotic arm is calculated using the damped least squares method. ; Method for calculating the J elements of the Jacobian matrix ; Among them, z K Let K be the axis of rotation. is the damping coefficient.

[0068] Precise placement control Visual conferencing controller Image-based Visual Gifts (IBVS): ; Image error ; Interaction Matrix ; Impedance contact control Dynamic model: ; Parameter settings: ; Once the robotic arm places the car door at the welding station, the Stereo stereo vision system quickly captures images of the door from multiple angles. The system's high-definition industrial camera simultaneously captures images from eight directions, including the front, sides, and top of the door, obtaining detailed information such as texture and contours on the door surface. This information is then used to construct a high-precision polygonal mesh 3D model, completely reproducing the shape and structure of the door, including minute features such as the door handle recesses and window rails.

[0069] Next, the Scale Invariant Feature Transform (SIFT) algorithm was used to process the 3D model. The SIFT algorithm scans the car door model at different scales, detecting numerous extreme points and calculating their descriptors. These extreme points correspond to distinctive features on the car door, such as door edges and weld hole edges. After processing by the algorithm, feature points with scale and rotation invariance are extracted, ensuring that these feature points remain stable and identifiable even if the car door is slightly scaled or rotated during shooting or placement.

[0070] Based on factors such as actual welding precision requirements (the door welding error must be controlled within ±0.1mm), production efficiency (60 doors need to be welded per hour), and system performance, the factory's technical team, after two weeks of repeated experiments and experience summarization, finally set a reasonable overlap index β of 0.9. This optimized calculation method comprehensively considers the current characteristics and historical data of the door, enabling the overlap index θ to more accurately reflect the degree of conformity between the door's placement position and its theoretical position. If the calculated overlap index θ value is greater than or equal to 0.92, it indicates that the door placement is accurate, and the welding process can be started immediately; if the θ value is less than 0.92, the system will control the robotic arm to fine-tune the door position until the welding requirements are met, thereby effectively improving the quality and stability of welding production and ensuring that every door meets high-standard welding process requirements.

[0071] Example 5 Another possible solution for the above content is as follows: Select multiple tangent points evenly along the workpiece contour line. Draw a tangent line at each tangent point and calculate the angle between the tangent line and the horizontal axis. Denote this angle as the characteristic angle α. Calculate the contour characteristic angle. ; Where dx and dy are the differentials of the contour line at the tangent point.

[0072] Then, perform linear regression on all characteristic angles α and their corresponding X coordinates to obtain the slope K of the regression line. The slope K reflects the tilting trend.

[0073] ; After obtaining the standard slope K, calculate the deviation rate between the current workpiece and the standard. ; Then the deviation rate is converted into the coincidence index θ ; Select multiple tangent points evenly along the workpiece contour line. Draw a tangent line at each tangent point and calculate the angle between the tangent line and the horizontal axis. Denote this angle as the characteristic angle α. Calculate the contour characteristic angle. ; Where dx and dy are the differentials of the contour line at the tangent point.

[0074] Then, perform linear regression on all characteristic angles α and their corresponding X coordinates to obtain the slope K of the regression line. The slope K reflects the tilting trend.

[0075] ; After obtaining the standard slope K, calculate the deviation rate between the current workpiece and the standard. ; Then the deviation rate is converted into the coincidence index θ ; Detect outliers that deviate excessively from the regression line. If there are more than 5 points, it is determined that the placement angle is incorrect.

[0076] After the high-end vehicle door is placed in the aforementioned automobile manufacturing plant and the welding requirements are met by the coincidence index θ, the system immediately starts to detect and analyze the welding posture when the door enters the welding process. Specifically, inertial analysis is used to ensure the welding quality of the door.

[0077] First, the system retrieves historical welding deformation data from the database. This data covers the overlap index and welding evaluation values ​​of over 2,000 similar car door workpieces produced in the past. For example, car door number WH-001 has an overlap index of 0.93 and is rated A for its uniform weld and lack of defects; car door number WH-002 has an overlap index of only 0.88 and is rated C for its porosity. This historical data provides a reference for the current analysis of car door welding posture.

[0078] Next, the contour of the car door to be welded is acquired. A high-precision laser scanner is used to scan the door, quickly obtaining its contour line, which is then placed into a two-dimensional coordinate system. Thirty tangent points are evenly selected on the door contour, such as the top edge, bottom corner, and door handle contour. A tangent line is drawn at each point, and the acute angle formed by the intersection of the tangent line and the horizontal coordinate axis is defined as the characteristic angle α. For example, at a tangent point on the top edge of the door, the characteristic angle α formed by the intersection of the tangent line and the horizontal coordinate axis is 35°.

[0079] After acquiring multiple feature angles α, the system draws a discrete graph based on these angles and the inclination of the tangent. Using a data analysis algorithm, the points in the discrete graph are divided into regression lines and outliers, and the inclination of the regression lines is calculated to determine the overall inclination of the car door profile. Simultaneously, the system counts the number of outliers. If, in a given detection, the number of outliers reaches 7, exceeding the set threshold of 5, the system immediately determines that the car door's placement angle is incorrect and controls the robotic arm to reposition and adjust the door.

[0080] Next, the system calculates the ratio of the current feature value of the welded door to the historical feature values ​​of the door workpiece, obtaining the contour irregularity. For example, comparing the contour dimension of the current door at the door handle with the historical standard dimension, the calculated contour irregularity is 1.1. The system then plots a histogram based on a series of contour irregularity data. By observing the peak angle of the histogram, the dominant feature direction of the door contour variation is determined. Finally, the system defines an angle range. If the door offset angle exceeds the acceptable range of ±2°, the door posture is readjusted until the welding requirements are met, ensuring the door is welded in a precise posture and improving the quality and consistency of the welded product.

Claims

1. An automated riveting and welding method based on an AI visual intelligence system, characterized in that, include: Real-time image capture of the car door workpiece is achieved using an industrial camera. The actual size and structural features of the workpiece are calculated based on the image, and defect detection is performed. A three-dimensional model of the workpiece is constructed, and the overlap index is calculated based on the comparison results between the three-dimensional model and the preset theoretical model to evaluate the accuracy of the workpiece placement position. When the overlap index meets the preset threshold, the robotic arm is controlled to perform a welding operation.

2. The automated riveting and welding method based on an AI visual intelligence system according to claim 1, characterized in that, Calculating the actual dimensions of the workpiece includes: Pixel equivalents are used to convert pixel distances in an image into physical dimensions.

3. The automated riveting and welding method based on an AI visual intelligence system according to claim 1, characterized in that, Defect detection includes: Gray-level histograms were used to analyze the gray-level distribution of the image, and the area, perimeter, and circularity of connected regions were calculated using Blob analysis to identify surface contaminants.

4. The automated riveting and welding method based on an AI visual intelligence system according to claim 1, characterized in that, Constructing a 3D model of the workpiece includes: performing a 3D contour scan based on the principle of triangulation to obtain the 3D point cloud data of the workpiece.

5. The automated riveting and welding method based on an AI visual intelligence system according to claim 1, characterized in that, Defect detection also includes: using deep learning-based image recognition algorithms to identify solder joint locations and outputting a set of solder joint bounding boxes.

6. The automated riveting and welding method based on an AI visual intelligence system according to claim 1, characterized in that, Also includes: The temperature of the weld joint is monitored in real time using a thermal imager, and the welding completion status is recorded when the temperature reaches the judgment temperature in multiple consecutive frames of images.

7. The automated riveting and welding method based on an AI visual intelligence system according to claim 1, characterized in that, The calculation of the coincidence index includes: scanning the workpiece from multiple angles using a stereo vision system, extracting feature points, comparing their positions and angles with the corresponding feature points in the theoretical model, and calculating the coincidence index based on the deviation rate.

8. The automated riveting and welding method based on an AI visual intelligence system according to claim 7, characterized in that, The feature points are identified using the Harris corner detection algorithm and compared with the corner model of a qualified workpiece to screen for quality defects.

9. The automated riveting and welding method based on an AI visual intelligence system according to claim 1, characterized in that, Controlling the robotic arm to perform welding operations includes: using UG CNC technology to plan the robotic arm's motion trajectory, and using a vision servo controller and impedance contact control to achieve precise placement and welding.

10. The automated riveting and welding method based on an AI visual intelligence system according to claim 1, characterized in that, Also includes: The robot operating system coordinates the control of multiple industrial cameras to acquire workpiece images from different angles, and schedules the vision processing module to perform 3D reconstruction and feature extraction.

Images