Adjustment method of melting thickness of hot plate welding of plastic embryo based on visual SLAM

Abstract

The present invention discloses a method for adjusting the melting thickness of hot plate welding of a plastic embryo body based on visual SLAM, comprising the following steps: 1) acquiring image key frames with a camera; 2) constructing a visual SLAM system; 3) performing an adaptive welding posture correction method based on plane geometry; 4) outputting a robot working posture. The present invention solves the influence of different welding depths caused by size differences of the plastic embryo body by correcting the rotation posture of the robot based on the welding plane and the translation posture of the robot based on the distance from the welding point to the welding head, and adaptively determines the melting posture of the hot plate welding of the plastic embryo body by correcting the rotation posture of the robot based on the welding plane and the translation posture of the robot based on the distance from the welding point to the welding head, thereby determining the welding depth, and effectively improving the welding quality of the plastic embryo body.

Description

Technical Field

[0001] The invention relates to the field of machine vision SLAM, and in particular to a method for adjusting the melting thickness of hot plate welding of a plastic embryo body based on vision SLAM. Background Art

[0002] For hot plate welding of metal bodies, Long Chaoxiang et al. designed a fixture and a fuel tank punching and welding device using this fixture (invention patent number: CN209453043U). This device clamps the plastic body with the fixture, offering advantages such as stable support and convenient material removal. However, this device does not achieve automated production and is only suitable for hot plate welding of a single plastic body structure.

[0003] Currently, the preform body is typically composed of six layers: virgin material layer, adhesive layer, barrier layer, adhesive layer, recycled material layer, and virgin material layer. These layers offer advantages such as lightweight, impact resistance, corrosion resistance, and wide design freedom. They are typically blow molded in a single step, enabling the creation of complex, custom-shaped products. Therefore, hot plate welding of the preform body is typically performed using a robotic intelligent flexible production line.

[0004] In 2018, Pei Ruiying and others proposed a loading and unloading and positioning device suitable for a flexible production line with robot loading and unloading (invention patent authorization number: CN109366242B). The device realizes automatic loading and unloading and grasping by robots and can adapt to the diversity of workpieces; however, the device uses position sensors and code readers to achieve positioning, and is not suitable for plastic bodies of various shapes, and only a fixed value can be set for the welding depth.

[0005] SLAM (Simultaneous Localization and Mapping) technology refers to the robot using its own sensors to perceive and construct a 3D environmental map, and at the same time determine its own position in the map. According to the different sensors, SLAM is divided into visual SLAM and laser SLAM, but this method will result in inaccurate results for complex indoor environments. In 2017, Long Chaoxiang and others proposed a car fuel tank welding positioning method and system based on robot 3D vision (invention patent publication number: CN 109421043A). This method sends the posture deviation value to the robot through a 3D camera to correct the robot's welding posture. This method effectively improves the welding quality of the hot plate of the plastic embryo body by fine-tuning the welding device at the end of the robot, but the welding depth still requires a sensor to determine.

[0006] In 2020, Guo Hengkai et al. proposed a monocular SLAM initialization method, device and electronic device (invention patent publication number: CN 113129366 A). This method replaces the original matrix decomposition method for obtaining camera pose and plane normal vector with variable optimization, thereby improving the efficiency of monocular SLAM initialization. However, this method does not eliminate the case of mismatched images, which may cause the results to not converge. In 2020, LI et al. proposed a mobile robot visual SLAM system with enhanced semantic segmentation (F Li, et al. A Mobile Robot Visual SLAM System With Enhanced Semantics Segmentation [J]. IEEE Access, 2020, PP (99): 1-1.). By using image processing algorithms to enhance the semantic segmentation effect, fusing the information of RGB-D cameras and encoders, performing positioning and creating a dense color octree map without dynamic objects, this method improves the positioning accuracy of the system. However, this method sacrifices the accuracy of target extraction through image downsampling methods and is not suitable for hot plates of plastic embryo bodies that have precise requirements for welding points.

[0007] To sum up, the existing plastic embryo body hot plate welding device is not suitable for plastic embryo body hot plates with complex shapes and high welding precision requirements. The welding depth of the plastic embryo body hot plate is generally based on multi-sensor control, and there are generally problems of inaccurate welding depth and limited applicable scenarios. Summary of the Invention

[0008] The purpose of the present invention is to overcome the defects of the prior art and provide a method for adjusting the melting thickness of the hot plate welding of the plastic embryo body based on visual SLAM. This method is based on the robot rotation posture correction of the welding plane and the robot translation posture correction method based on the distance from the welding point to the welding head, which better solves the influence of the depth of the hot plate of the welding plastic embryo body.

[0009] The object of the present invention is achieved as follows: a method for adjusting the melting thickness of a plastic embryo body hot plate welding based on visual SLAM, comprising the following steps:

[0010] Step 1) The camera acquires an image key frame;

[0011] Step 2) Visual SLAM system construction;

[0012] Step 3) performing an adaptive welding posture correction method based on plane geometry;

[0013] Step 4) Output the robot's working posture.

[0014] As a further limitation of the present invention, the step 2) comprises:

[0015] Step 2.1) Initial welding posture of the robot;

[0016] Step 2.2) Match ORB feature points between frames;

[0017] Step 2.3) P3P-based welding robot pose estimation;

[0018] Step 2.4) Global SLAM backend optimization;

[0019] Step 2.5) Robot scene map fusion.

[0020] As a further limitation of the present invention, the step 3) comprises:

[0021] Step 3.1) Path planning based on genetic method;

[0022] Step 3.2) Correcting the robot rotation posture based on the welding plane;

[0023] Step 3.3) Correcting the robot translation pose based on the welding point to the welding head;

[0024] Step 3.4) Check whether the welding tool plane interferes with the hot plate welding plane of the plastic body; if it is, return to step 3.1); otherwise, continue to step 4).

[0025] As a further limitation of the present invention, the step 2.1) comprises

[0026] Step 2.1.1) Extract corner points based on the FAST method;

[0027] Step 2.1.2) Use BRIEF feature descriptor to describe the feature points;

[0028] As a further limitation of the present invention, the specific method flow of step 2.1.1) is as follows:

[0029] (1) Select a point in the image and set it as pixel p0. Set the grayscale value of the pixel to be

[0030] (2) Set the grayscale threshold as

[0031] (3) Take 16 pixels on the Bresenham discrete circle with a radius of three pixels and centered at pixel p0, and denote them as p1-p 16 ;

[0032] (4) In the selected p1-p 16 Among these 16 pixels, if there are N0 consecutive pixels with grayscale values ​​equal to The absolute value of the difference is greater than Then pixel p0 is the corner point. Let N0 = 12.

[0033]

[0034] (5) Repeat the above steps (1)-(4) for all pixels in the image.

[0035] As a further limitation of the present invention, the specific method flow of step 2.1.2) is as follows:

[0036] (1) Taking pixel p0 as the center, take an S×S image block as the neighborhood;

[0037] (2) Select n0 point pairs in the neighborhood according to the Gaussian distribution mode, and take n0 = 128;

[0038] (3) For each pair of pixels (x, y), L(x) represents the pixel at x = (u, v) in the image. T The brightness of the pixel at , (u, v) is the pixel coordinate, and τ is the result of comparing the pixel brightness:

[0039]

[0040] (4) The BRIEF descriptor can be represented as an n0-dimensional binary code string:

[0041]

[0042] in, is the binary code string function of n0 point pairs, L is the image at the jth point pair (x j ,y j ), τ is the pixel brightness at the jth point pair (x j ,y j )The result of pixel brightness comparison.

[0043] As a further limitation of the present invention, step 3.2) includes initializing the hardware such as the welding robot and the camera; after the initialization is completed, the camera detects the plane Ω to be welded, and the normal vector is Adjust the camera pose so that the vector OW represented by the line connecting the camera optical center O and the welding point W is The camera transformation matrix R1 is obtained at this time. Let the transformation matrix of the camera relative to the welding robot be R2, and the rotation matrix of the robot relative to the welding plane be R3; where R3 = R1R2.

[0044] As a further limitation of the present invention, the step 3.3) includes, based on the depth information, knowing that the distance from the camera lens O to the welding point W is D1, the distance from the camera lens O to the welding head U is D2, and the distance from the camera lens O to the welding plane WH is D3, then the distance from the welding point W to the welding head U is D Δ It can be expressed by geometric relations:

[0045]

[0046] Therefore, the correction robot translation based on the distance from the welding point to the welding head is: D Δ , record the posture at that time (x 11 ,x 21 ,x 31 ).

[0047] As a further limitation of the present invention, the step 3.4) includes the motion trajectory planning, assuming that the welding tool plane is denoted as α, which is represented by a0x+b0y+c0z+d0=0, with an area of ​​S1, and the hot plate welding plane of the plastic embryo body is denoted as β, which is represented by e0x+f0y+g0z+h0=0, with an area of ​​S2, where the normal vector of plane α is d0≠0 means that plane α is a general position plane in space, and the normal vector of plane β is h0≠0 means that plane β is a general position plane in space;

[0048] The equation of the straight line obtained by simultaneous equations is a1x+b1y+c1=0; if the straight line satisfies both the welding tooling plane α and the hot plate welding plane β of the plastic embryo body, it is judged as interference. If the solutions of a1x+b1y+c1=0 and a0x+b0y+c0z+d0=0 are within S1 and the solutions of a1x+b1y+c1=0 and e0x+f0y+g0z+h0=0 are within S2, then replan the path and return to step 3.1).

[0049] The present invention adopts the above technical solution, and compared with the existing technology, the beneficial effects are: 1) The method of the present invention corrects the posture of the hot plate welding robot of the plastic embryo body based on visual SLAM, thereby effectively correcting the welding depth of the hot plate of the plastic embryo body; and using the fitness function designed in this way, the efficiency and convergence of the method operation are improved by adopting a genetic method; 2) Through the rotation posture of the welding robot based on the welding plane correction and the robot translation posture correction method based on the distance from the welding point to the welding head, the influence of different welding depths of the plastic embryo body caused by the size difference is better solved. Through the robot rotation posture based on the welding plane correction and the robot translation posture correction method based on the distance from the welding point to the welding head, the posture of the hot plate welding melting of the plastic embryo body is adaptively determined, thereby determining the welding depth, and effectively improving the welding quality of the plastic embryo body. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Flowchart of the present invention.

[0051] Figure 2 Schematic diagram of FAST corner points and BRIEF descriptors in the present invention.

[0052] Figure 3 Schematic diagram of camera pose estimation based on P3P (Perspective-3-Point) in the present invention.

[0053] Figure 4 Schematic diagram of the global pose graph optimization model in the present invention.

[0054] Figure 5 This is a flow chart of path planning based on the genetic method in the present invention.

[0055] Figure 6 Schematic diagram of the translation posture of the correction robot based on the distance from the welding point to the welding head in the present invention. DETAILED DESCRIPTION

[0056] like Figure 1 The method for adjusting the melting thickness of hot plate welding of a plastic embryo body based on visual SLAM includes the following steps:

[0057] Step 1) The camera acquires an image key frame;

[0058] Use RGB-D camera to collect n frames of color images, denoted as I1, I2, ..., I n .

[0059] Step 2) Visual SLAM (Simultaneous Localization and Mapping) system construction;

[0060] Step 2.1) Initial welding posture of the robot;

[0061] like Figure 2 (a) shows a schematic diagram of FAST corner points; ORB features are composed of FAST corner points and BRIEF descriptors, and have the characteristics of rotation and scale invariance.

[0062] Step 2.1.1) Extract corner points based on the FAST method (Features From Accelerated Segment Test). This method determines whether a pixel is a corner point by judging the brightness difference between a pixel and its neighboring pixels.

[0063] The specific method process is as follows:

[0064] (1) Select a point in the image and set it as pixel p0. Set the grayscale value of the pixel to be

[0065] (2) Set the grayscale threshold as

[0066] (3) Take 16 pixels on the Bresenham discrete circle with a radius of three pixels and centered at pixel p0, and denote them as p1-p 16 ;

[0067] (4) In the selected p1-p 16 Among these 16 pixels, if there are N0 consecutive pixels with grayscale values ​​equal to The absolute value of the difference is greater than Then pixel p0 is the corner point. Let N0 = 12.

[0068]

[0069] (5) Repeat the above steps (1)-(4) for all pixels in the image.

[0070] Step 2.1.2) Use BRIEF (Binary Robust Independent Element Feature) to describe the feature points; Figure 2 (b) is a schematic diagram of the BRIEF descriptor. The BRIEF descriptor uses some point pairs around the key point to describe a feature. The algorithm flow is:

[0071] (1) Taking pixel p0 as the center, take an S×S image block as the neighborhood;

[0072] (2) Select n0 point pairs in the neighborhood according to the Gaussian distribution mode, and take n0 = 128;

[0073] (3) For each pair of pixels (x, y), L(x) represents the pixel at x = (u, v) in the image. T The brightness of the pixel at , (u, v) is the pixel coordinate, and τ is the result of comparing the pixel brightness:

[0074]

[0075] (4) The BRIEF descriptor can be represented as an n0-dimensional binary code string:

[0076]

[0077] in, is the binary code string function of n0 point pairs, L is the image at the jth point pair (x j ,y j ), τ is the pixel brightness at the jth point pair (x j ,y j )The result of pixel brightness comparison.

[0078] Step 2.2) Match ORB (Oriented FAST and Rotated BRIEF) feature points between frames;

[0079] like Figure 3 As shown in the figure, how does P3P (Perspective-3-Point) estimate the camera pose based on three 3D space points and their projections (2D). Assume that three pairs of matching points Aa, Bb, and Cc are known, where A, B, and C are points in the world coordinate system, a, b, and c are the projections of A, B, and C in the image coordinate system, and O is the camera optical center. According to the cosine theorem of ΔOab and ΔOAB, we have:

[0080]

[0081] Let e=OA / OC, g=OB / OC, m0=AB in the above equation. 2 / OC 2 ,m1m0=BC 2 / OC 2 ,m2m0=AC 2 / OC 2 , which can be simplified to:

[0082]

[0083] Camera-calibrated cos<a,b> ,cos<b,c> ,cos<a,c> All are known quantities. Once the world coordinates of the three points are known, the camera's coordinates relative to the world coordinate system can be calculated using elimination. Record the robot's initial welding point (x1, x2, x3), denote the rotation matrix M0, and the translation vector T0.

[0084] Step 2.3) P3P-based welding robot pose estimation;

[0085] like Figure 4 As shown, the global pose graph optimization model is based on X i is the robot pose, T ij is the transfer matrix between the robot position i and position j. In an ideal situation, T ij and X i , X j The relationship is: X j =T ij X i ;

[0086] However, due to external noise, there will be an error term E ij :E ij =X j -T ij X i , the sum of the error terms' two norms is: The Gauss-Newton method is used to find the solution that minimizes the error term.

[0087] Preferably, this method can effectively reduce the error between the pose of each node and the previously estimated pose, thereby achieving the purpose of improving accuracy.

[0088] Step 2.4) Global SLAM backend optimization;

[0089] Step 2.5) Robot scene map fusion.

[0090] Step 3) performing an adaptive welding posture correction method based on plane geometry;

[0091] Step 3.1) Path planning based on genetic method;

[0092] like Figure 5 As shown in FIG, after the map is established in step 2.4), the walking space model of the mobile robot is established using the grid method, and the fitness function is calculated to judge the optimization result.

[0093] Step 3.1.1 Initialize the population;

[0094] Initializing the population can randomly generate multiple feasible paths. A feasible path requires first generating a discontinuous path, and then connecting the discontinuous paths into a continuous path. During initialization, a barrier-free grid is randomly selected from each row in order to form a discontinuous path. When connecting the discontinuous paths into a continuous path, the first step is to determine whether the two adjacent grids are continuous grids, starting from the first grid. The method for determining whether the grid is continuous is as follows:

[0095] D max =max{abs(x ω+1 -x ω ),abs(y ω+1 -y ω )} (6)

[0096] If D max =1, it means that the two adjacent grids are continuous, D max ≠1, the two adjacent grids are discontinuous. For discontinuous grids, take the midpoint grid of the two grids, the midpoint grid (x mid ,y mid ) is calculated as: If the new grid is an obstacle-free grid and is not in the path, it is inserted into the path; if the new grid is an obstacle-free grid, it is inserted between two discontinuous grids. Continue to check whether the newly inserted grid is continuous with the previous grid. If not, repeat the above steps until the two grids are continuous.

[0097] Step 3.1.2 fitness function;

[0098] The fitness function is used as the evaluation criterion for individuals or paths, where the adaptive function ψ is:

[0099]

[0100] d represents the sum of the distances between every two adjacent genes in an individual, 1 / n1-1 represents the penalty element, n1 represents the number of grids of individuals, and d / n1-1 represents the average coefficient.

[0101] Preferably, the smaller the grid area, the more accurate the representation of the environment information in the space, but the search time of this method will increase. The larger the grid area, the less accurate the representation of the environment information in the space, and the more likely collision problems will occur.

[0102] Step 3.2) Correcting the robot rotation posture based on the welding plane;

[0103] In the present invention, the welding robot and camera hardware are initialized. After the initialization is completed, the camera detects the plane to be welded Ω, and the normal vector is Adjust the camera pose so that the vector OW represented by the line connecting the camera optical center O and the welding point W is The camera transformation matrix R1 is obtained at this time. Let the transformation matrix of the camera relative to the welding robot be R2, and the rotation matrix of the robot relative to the welding plane be R3; where R3 = R1R2.

[0104] Step 3.3) Correcting the robot translation pose based on the welding point to the welding head;

[0105] like Figure 6 As shown, based on the depth information, it is known that the distance from camera lens O to welding point W is D1, the distance from camera lens O to welding head U is D2, and the distance from camera lens O to welding plane WH is D3. Then the distance from welding point W to welding head U is D Δ It can be expressed by geometric relations:

[0106]

[0107] Therefore, the correction robot translation based on the distance from the welding point to the welding head is: D Δ , record the posture at that time (x 11 ,x 21 ,x 31 ).

[0108] Step 3.4) Check whether the welding tool plane interferes with the hot plate welding plane of the plastic body; if it is, return to step 3.1); otherwise, continue to step 4).

[0109] In motion trajectory planning, let the welding tool plane be denoted as α, which is expressed as a0x+b0y+c0z+d0=0, with an area of ​​S1, and the hot plate welding plane of the plastic embryo body be denoted as β, which is expressed as e0x+f0y+g0z+h0=0, with an area of ​​S2, where the normal vector of plane α is d0≠0 means that plane α is a general position plane in space, and the normal vector of plane β is h0≠0 means that plane β is a general position plane in space;

[0110] The equation of the straight line obtained by simultaneous equations is a1x+b1y+c1=0; if the straight line satisfies both the welding tooling plane α and the hot plate welding plane β of the plastic embryo body, it is judged as interference. If the solutions of a1x+b1y+1c=0 and a0x+b0y+c0z+d0=0 are within S1 and the solutions of a1x+b1y+c1=0 and e0x+f0y+g0z+h0=0 are within S2, then replan the path and return to step 3.1).

[0111] Step 4) Outputting the robot's welding working posture;

[0112] After multiple iterations, the path planning with the lowest complexity and shortest distance for the welding robot is obtained. Record the robot welding points The rotation matrix is ​​denoted as M * , the translation vector is denoted as T * .

[0113] The method of the present invention solves the influence of different welding depths of the plastic embryo body due to size difference through the rotation posture of the welding robot based on the welding plane correction and the translation posture of the robot based on the distance from the welding point to the welding head, and adaptively determines the posture of the hot plate welding melting of the plastic embryo body through the method of the rotation posture of the robot based on the welding plane correction and the translation posture of the robot based on the distance from the welding point to the welding head, thereby determining the welding depth and effectively improving the welding quality of the plastic embryo body.

[0114] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solutions disclosed in the present invention, those skilled in the art can make some substitutions and modifications to some of the technical features therein according to the disclosed technical content without creative labor, and these substitutions and modifications are all within the protection scope of the present invention.

Claims

1. A method for adjusting the melting thickness of hot plate welding of a plastic embryo body based on visual SLAM, characterized in that: The following steps are involved: Step 1) The camera obtains an image key frame; Step 2) Visual SLAM system construction; Step 3) performing an adaptive welding posture correction method based on plane geometry; Step 4) Outputting the robot's working posture; The step 2) comprises: Step 2.1) Initial welding posture of robot; Step 2.2) ORB feature point matching between frames; Step 2.3) P3P-based welding robot pose estimation; Step 2.4) Global SLAM backend optimization; Step 2.5) Robot scene map fusion; The step 3) comprises: Step 3.1) Path planning based on genetic method; Step 3.2) The robot rotation posture is corrected based on the welding plane; Step 3.3) Correcting the robot translation pose based on the welding point to the welding head; Step 3.4) Whether the welding tool plane interferes with the hot plate welding plane of the plastic embryo body; if it is judged to be interference, return to step 3.1); otherwise, continue to step 4); The step 2.1) comprises: Step 2.1.1) Extract corner points based on FAST method; Step 2.1.2) Use BRIEF feature descriptor to describe the feature points; The specific method flow of step 2.1.1) is as follows: (1) Select a point in the image and set it as pixel p0. Set the gray value of the pixel to (2) Set the grayscale threshold as (3) Take 16 pixels on the Bresenham discrete circle with a radius of three pixels and centered at pixel p0, denoted as p1-p 16 ; (4) In the selected p1-p 16 Among these 16 pixels, if there are N0 consecutive pixels with grayscale values ​​equal to The absolute value of the difference is greater than Then the pixel point p0 is the corner point, and N0=12; (5) Repeat the above steps (1)-(4) for all pixels in the image; The specific method flow of step 2.1.2) is as follows: (1) Taking pixel p0 as the center, take an S×S image block as the neighborhood; S represents the side length of a square image block; (2) Select n0 point pairs in the neighborhood according to the Gaussian distribution mode, and take n0 = 128; (3) For each pair of pixels (x, y), L(x) represents the pixel located in the horizontal direction of the image x = (u, v) T The pixel brightness at (u, v) is the pixel coordinate, and τ is the result of comparing the pixel brightness: Among them, L(y) represents the vertical gradient of the image y=(u,v) T The brightness of the pixel at (4) The BRIEF descriptor is represented as an n0-dimensional binary code string: in, is the binary code string function of n0 point pairs, L is the image at the jth point pair (x j ,y j ), τ is the pixel brightness at the jth point pair (x j ,y j ) The result of pixel brightness comparison; The step 3.2) includes initializing the welding robot and the camera hardware. After the initialization is completed, the camera detects the plane Ω to be welded, and the normal vector is Adjust the camera position so that the line OW between the camera optical center O and the welding point W represents the vector The camera transformation matrix R1 is obtained at this time. Let the transformation matrix of the camera relative to the welding robot be R2, then the rotation matrix of the robot relative to the welding plane is R3; where R3 = R1R2; The step 3.3) includes, based on the depth information, knowing that the distance from the camera lens O to the welding point W is D1, the distance from the camera lens O to the welding head U is D2, and the distance from the camera lens O to the welding plane WH is D3, Therefore, the correction robot translation based on the distance from the welding point to the welding head is D Δ , record the posture at that time (x 11 ,x 21 ,x 31 ); x 11 ,x 21 ,x 31 They are the coordinates in the X, Y and Z directions for correcting the robot's translation posture respectively; The step 3.4) includes the motion trajectory planning, assuming that the welding tool plane is denoted as α, represented by a0x+b0y+c0z+d0=0, with an area of ​​S1, and the hot plate welding plane of the plastic embryo body is denoted as β, represented by e0x+f0y+g0z+h0=0, with an area of ​​S2, where the normal vector of plane α is d0≠0 means that plane α is a general position plane in space, and the normal vector of plane β is h0≠0 means that plane β is a general position plane in space; a0, b0, c0, d0 are known constants, and a0, b0, c0 are not zero at the same time, (x, y, z) are the coordinates of a point on the plane, d0 is the intercept of the plane, e0, f0, g0, h0 are known constants, and e0, f0, g0 are not zero at the same time, (x, y, z) are the coordinates of a point on the plane, h0 is the intercept of the plane; The equation of the straight line obtained by simultaneous calculation is a1x+b1y+c1=0; a1, b1, c1 are known constants that can be obtained, and a1, b1 are not zero at the same time; if the straight line satisfies that it is on the welding tooling plane α and the plastic body hot plate welding plane β at the same time, it is judged as interference, and the solutions that satisfy a1x+b1y+c1=0 and a0x+b0y+c0z+d0=0 are in S1 and the solutions that satisfy a1x+b1y+c1=0 and e0x+f0y+g0z+h0=0 are in S2, then replan the path and return to step 3.1).

Images