Automatic tin sheet identifying, positioning and brazing method based on depth vision collaborative robot

Abstract

The invention particularly relates to an automatic tin sheet identifying, positioning and brazing method based on a depth vision collaborative robot, and relates to the technical field of image processing. Fusing a multi-source data input layer; tin sheet identification positioning and light reflection inhibition; and robot motion planning and brazing control are conducted. According to the method, multi-source hardware collaboration and intelligent algorithm are fused, so that the problem of micro tin sheet identification is solved; the near-infrared camera and the RGB camera are coaxially arranged, binocular structured light is combined, all-directional sensing of tin sheet reflection characteristics, color differences and three-dimensional forms is achieved, and a space-time binding mechanism triggered by an FPGA ensures synchronization of data and poses of a robot. And the RSM module decouples metal reflection through three-dimensional characteristics of abnormal compactness, spanning degree and change degree, and inhibits reflection and instantaneous noise interference in cooperation with k and u dynamic optimization and Sigmoid function adjustment.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for automatic identification and positioning of solder sheets for brazing based on a depth vision collaborative robot. Background Technology

[0002] Tin sheets are widely used in electronic packaging, precision instruments, and other fields due to their excellent electrical and thermal conductivity. However, tin sheets are tiny (usually a few millimeters to tens of millimeters) and their surfaces are easily reflective. Furthermore, the brazing process requires extremely high positioning accuracy and welding quality, making it difficult for traditional manual brazing methods to meet production demands.

[0003] Existing robot-based soldering technology has the following shortcomings: First, solder sheet identification and positioning are easily affected by metal reflection, resulting in large positioning errors; second, the accuracy of multi-source sensor data fusion is low, making it difficult to accurately extract solder sheet features; third, robot motion planning and soldering parameter control lack adaptability and cannot be dynamically adjusted according to the actual state of the solder sheet; and fourth, the soldering quality detection and feedback mechanism is imperfect, making it difficult to achieve continuous process optimization.

[0004] Therefore, there is an urgent need for an automated method that can achieve high-precision identification and positioning of tin sheets, effectively suppress reflective interference, accurately control the brazing process, and ensure welding quality. Summary of the Invention

[0005] The purpose of this invention is to propose an automatic identification and positioning brazing method for tin sheets based on a deep vision collaborative robot in order to solve the above-mentioned problems.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A method for automatic identification and positioning of solder sheets for soldering based on deep vision collaborative robots includes: Hardware sensing and multi-source data acquisition: Through hardware synchronous control and environmental adaptation components, the acquisition and spatiotemporal binding of tin sheet-related data are realized; Multi-source data input layer fusion: Clearly define calibration and fusion quantification indicators, and align the heterogeneous features of near-infrared images, depth maps, and RGB images through spatiotemporal calibration, heterogeneous data fusion processing, and noise preprocessing; Tin sheet identification and localization and reflection suppression: Coarse detection is performed using a lightweight network, metal reflection features are decoupled through an RSM module, and then precise localization is performed to complete the identification and localization of tin sheets and suppress the influence of reflection; Robot motion planning and brazing control: Combining robot dynamics characteristics, it enables precise mapping between positioning results and robot actions, covering pose transformation, motion planning, adaptive control of brazing parameters, and safety control.

[0007] Preferably, the hardware sensing and multi-source data acquisition specifically include: Multispectral imaging is achieved using near-infrared and RGB cameras; A binocular structured light sensor is used to output point cloud density, and the initial attitude is obtained by fitting the point cloud plane on the tin sheet surface. Based on the FPGA trigger board, a synchronous pulse signal is generated, and the pose feedback from the robot controller is received at the same time. The spatiotemporal binding of the image acquisition time and the robot pose time is realized through timestamp alignment.

[0008] Preferably, the multi-source data input layer fusion specifically includes: Spatiotemporal calibration and data alignment include camera intrinsic calibration, hand-eye calibration, and coordinate transformation; Near-infrared feature extraction uses 3×3 convolution; depth feature extraction uses 5×5 convolution. A 64×64 learnable mapping matrix is ​​designed, and reflection and geometric association weights are learned through backpropagation. The fused feature dimension is 128. A 3×3 Gaussian filter is used to smooth high-frequency noise, and a 3×3 erosion operation is combined to process near-infrared reflective noise. By using depth continuity detection, a bilateral filter is employed to preserve edges while suppressing noise.

[0009] Preferably, the tin sheet identification, positioning, and reflection suppression specifically include: Input the fused 128×128 feature map and output candidate boxes with a confidence score ≥ 0.85; Interference from fixtures and weld slag is eliminated by measuring the average reflection intensity and depth flatness; then, the metal reflection characteristics are decoupled. Subpixel edge detection is performed using the Zernike moment algorithm; Edge points are fitted using the minimum bounding rectangle, and the center coordinates and rotation angle are calculated. The thickness is calculated using the depth difference.

[0010] Preferably, the decoupling of the metallic reflective features specifically includes: The tin sheet image is divided into 3×3 regions. The gray values ​​of each pixel in each region are obtained and the mean is calculated to obtain the mean gray value of the region. Regions with a mean gray value > 200 are extracted and their compactness is calculated. After marking the regions with a compactness less than 0.3, the number of regions is counted. The total number of marked regions is divided by the total number of regions in the tin sheet image to obtain the abnormal compactness. Obtain the grayscale value of the center pixel of a 3×3 region, and the grayscale values ​​of the 8 pixels outside the center of the region. Sort the grayscale values ​​of the 8 pixels in ascending order of numerical value, extract the k nearest minimum grayscale values, take the average value, calculate the difference between the average value and the grayscale value of the center pixel of the region, and take the absolute value to obtain the span difference value. Preset a span difference threshold, divide the span difference value by the span difference threshold to obtain the span. The difference between the average grayscale value of the 3×3 region in the current frame and the average grayscale value of the region in the previous u frames is calculated and then divided by the average grayscale value of the 3×3 region in the current frame. The absolute value is then taken to obtain the degree of change. The abnormal compactness, span, and variability are weighted and fused to obtain the fused feature value, which is then input into the Sigmoid function for optimization and output as the reflectivity probability.

[0011] Preferably, the Sigmoid function is optimized as follows: in, This represents the probability of reflection. This is the gain coefficient; To fuse feature values; This is the offset threshold.

[0012] Preferably, the process further includes optimizing k and u, and designing a k-value optimization function based on the edge grayscale variance:

[0013] in, The grayscale variance of the 8 edge pixels: in, It represents the absolute value of the grayscale difference between the current frame and the previous frame.

[0014] Preferably, the robot motion planning and brazing control specifically includes: The coordinates of the solder sheet output by visual recognition are converted into the coordinates of the end effector of the soldering gun using a hand-eye conversion matrix. The path is generated based on a heuristic function and then smoothed using a 5th-order B-spline; a trapezoidal velocity curve is used to control the welding torch speed to ensure a smooth acceleration / deceleration process. Adaptive control of brazing parameters, including current adjustment and welding torch oscillation, and real-time correction; By monitoring both force and distance, mechanical collisions between the soldering torch and the solder sheet / workpiece are prevented, thus protecting the solder sheet and equipment. A force sensor is integrated at the end of the welding torch to collect the contact force in real time and set a safety force threshold. When the contact force is greater than or equal to the safety force threshold, an emergency stop command is triggered. The vertical distance between the tip of the welding torch and the workpiece surface is calculated in real time based on the depth map, and the welding torch speed is adjusted accordingly based on the vertical distance.

[0015] Preferably, the brazing quality inspection and feedback optimization involves: evaluating quality through post-weld quality inspection and an SVM classifier, establishing a quality-parameter correlation model, achieving closed-loop optimization, and recording relevant data.

[0016] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention solves the problem of identifying tiny tin sheets by integrating multi-source hardware collaboration and intelligent algorithms; the coaxial layout of near-infrared and RGB cameras combined with binocular structured light enables all-round perception of tin sheet reflection characteristics, color differences and three-dimensional morphology; the spatiotemporal binding mechanism triggered by FPGA ensures that data is synchronized with robot pose; the RSM module decouples metal reflection through three-dimensional features of abnormal compactness, span and variability, and suppresses reflection and instantaneous noise interference by dynamic optimization of k and u and adjustment of Sigmoid function.

[0017] 2. This invention avoids solder sheet collision and deformation through motion planning using a 5th-order B-spline smooth path and trapezoidal velocity curve, combined with dual-dimensional safety monitoring of force and distance. The adaptive control of brazing parameters dynamically adjusts the current and swing amplitude according to the solder sheet thickness and positioning deviation, correcting deviations in real time. After soldering, multiple methods such as vision, laser, and ultrasound are used for inspection. The quality and parameters are correlated by an SVM classifier and a random forest model, and high-frequency defect parameters are optimized weekly, improving the brazing pass rate. Moreover, the process parameters do not require manual adjustment, adapting to different batches of solder sheets and environmental changes, significantly reducing debugging costs in industrial applications. Attached Figure Description

[0018] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0019] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0020] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0021] Example 1

[0022] Its specific implementation method is combined with the appendix Figure 1 Please provide a detailed explanation.

[0023] Appendix Figure 1 The flowchart of the automatic identification and positioning brazing method for tin sheets based on a deep vision collaborative robot provided in the embodiments of the present invention illustrates the complete steps from hardware perception and multi-source data acquisition to robot motion planning and brazing control.

[0024] In this embodiment, it includes: Hardware sensing and multi-source data acquisition: Adapting to the tiny size of tin sheets, through hardware synchronous control and environmental adaptation components such as near-infrared cameras, RGB cameras, and binocular structured light sensors, high resolution combined with low latency is used to acquire and spatiotemporally bind tin sheet-related data. Specifically, it includes: Multispectral imaging is achieved using near-infrared and RGB cameras; Near-infrared camera: A 12-megapixel global shutter camera (30fps) is used, equipped with a 650nm narrowband filter (to suppress visible light interference), with an imaging resolution of 1920×1080, ensuring pixel-level preservation of the reflective characteristics of the tin sheet (each tin sheet covers at least 50×50 pixels). RGB camera: Coaxially mounted with near-infrared camera (baseline distance ≤ 5mm), 8 megapixels (1600×1200), used to help distinguish tin sheets from fixtures of the same material (by color difference, such as the tin sheet surface oxide layer being light yellow).

[0025] A binocular structured light sensor (measurement range 0.3-1.5m) with a depth resolution of 0.1mm (at a working distance of 0.5m) is used to output a point cloud density of 2 million points / frame. The initial attitude is obtained by fitting the point cloud plane on the tin sheet surface (flatness error ≤0.03mm). Based on the FPGA trigger board (response delay ≤50μs), a synchronous pulse signal (10Hz trigger frequency, matching the robot's working rhythm) is generated. At the same time, the pose feedback (6-axis joint angle, update frequency 1kHz) of the robot controller (such as the RTDE interface of UR5e) is received. The spatiotemporal binding of the image acquisition time and the robot pose time is realized through timestamp alignment (error ≤0.5ms). A near-infrared LED array (wavelength 850nm) is used with a 45° tilt, and the illumination uniformity is ≥90% (to avoid additional reflections on the tin sheet surface due to uneven illumination). Equipped with an automatic cleaning device (air jetting once every 30 minutes), the lens has a focal length of 16mm (at a working distance of 0.5m, the field of view is 20°×15°, which just covers a 200×150mm working area).

[0026] Multi-source data input layer fusion: Clearly define calibration and fusion quantification indicators, and align the heterogeneous features of near-infrared images, depth maps, and RGB images through spatiotemporal calibration, heterogeneous data fusion processing, and noise preprocessing; Specifically, it includes: Spatiotemporal calibration and data alignment include camera intrinsic calibration, hand-eye calibration, and coordinate transformation; Camera intrinsic parameter calibration: Using a 12×9 checkerboard grid (grid size 10mm×10mm), 20 sets of images from different angles were captured, and the intrinsic parameters were solved using Zhang Zhengyou's algorithm (focal length error ≤ 0.5 pixels, distortion coefficient ≤ 0.01). Hand-eye calibration: The "eye on hand" mode is adopted (the camera is fixed to the end of the robot). The robot carries the camera to take pictures of a fixed chessboard (collecting 30 pose-image pairs), and iterative optimization is performed using the Tsai-Lenz algorithm (calibration error ≤ 0.02mm). Coordinate transformation: Using the calibrated extrinsic matrix (4×4 homogeneous matrix), the image pixel coordinates are combined with the depth values ​​and transformed into robot base coordinate system coordinates with a transformation error ≤0.03mm.

[0027] Near-infrared feature extraction uses 3×3 convolution (outputting 64-dimensional features, focusing on the reflection intensity gradient); depth feature extraction uses 5×5 convolution (outputting 64-dimensional features, focusing on the height change rate). Design a 64×64 learnable mapping matrix (initialized using the Identity matrix), and learn reflection and geometric association weights through backpropagation (e.g., the weights of reflection features and depth gradient features in the tin edge region are increased by 20%), resulting in a 128-dimensional feature after fusion; A 3×3 Gaussian filter (σ=1.2) is used to smooth high-frequency noise, and a 3×3 erosion operation is combined to process near-infrared reflective noise (to remove isolated reflective points with an area ≤3 pixels). By detecting depth continuity (where the depth difference between adjacent pixels is greater than 0.1 mm, it is determined to be noise), a bilateral filter (spatial standard deviation of 5, grayscale standard deviation of 0.05) is used to preserve edges while suppressing noise.

[0028] Tin sheet identification and localization and reflection suppression: Coarse detection is performed using a lightweight network, metal reflection features are decoupled through an RSM module, and then precise localization is performed to complete the identification and localization of tin sheets and suppress the influence of reflection; Specifically, it includes: Input the fused 128×128 feature map (corresponding to the original 200×200mm region), output candidate boxes with confidence ≥0.85 (IOU threshold 0.5), and inference speed ≤10ms / frame; Interference from fixtures (low reflection intensity) and weld slag (poor flatness) was eliminated by using the dual criteria of mean reflection intensity > 200 (near-infrared gray value) and depth flatness < 0.05 mm; then, the metal reflection characteristics were decoupled. The Zernike moment algorithm (template order 8) is used to perform subpixel edge detection with an edge localization accuracy of 0.01 pixels (corresponding to an actual accuracy of 0.005 mm). The center coordinates (X, Y error ≤ 0.02 mm) and rotation angle (based on the long side direction, error ≤ 0.1°) are calculated by fitting the edge points with the minimum bounding rectangle (≥ 20 valid points). The thickness is calculated by the depth difference (depth difference between the upper and lower surfaces, error ≤ 0.01 mm).

[0029] Metallic reflective feature decoupling (RSM module), specifically including: The tin sheet image is divided into 3-pixel × 3-pixel regions. The gray values ​​of each pixel in each region are obtained and the mean is calculated to obtain the mean gray value of the region. Regions with a mean gray value > 200 (reflection candidate areas) are extracted, and the compactness is calculated. Compactness = 4π × area / perimeter squared. After marking the regions with a compactness less than 0.3, the number of regions is counted. The total number of marked regions is divided by the total number of regions in the tin sheet image to obtain the abnormal compactness. Metallic reflections appear as small, irregularly distributed areas (such as dots or stripes) in images. Features with a compactness of <0.3 can accurately match this physical form. The compactness of normal tin sheet areas (regular edges, uniform reflection) is usually >0.6. A compactness of <0.3 can eliminate more than 90% of non-reflective interference (such as tin sheet edges and oxide spots). Obtain the grayscale value of the center pixel of a 3×3 region, and the grayscale values ​​of the 8 pixels outside the center of the region. Sort the grayscale values ​​of the 8 pixels in ascending order of numerical value, extract the k nearest minimum grayscale values, take the average value, calculate the difference between the average value and the grayscale value of the center pixel of the region, and take the absolute value to obtain the span difference value. Preset a span difference threshold, divide the span difference value by the span difference threshold to obtain the span. The core characteristic of metallic reflection is that the center is extremely bright and the edges are suddenly dark. This gradient can be quantified by the gray-level difference between the center pixel and the k smallest edge pixels. The gray-level difference between the center and the edge of the normal area is ≤30, while the difference in the reflective area is usually >50. Combined with dynamic optimization of the k value (adjusting k according to the gray-level dispersion of the edge), the gradient calculation distortion caused by a single k value can be avoided. The difference between the average grayscale value of the 3×3 region in the current frame and the average grayscale value of the region in the previous u frames is calculated and then divided by the average grayscale value of the 3×3 region in the current frame. The absolute value is then taken to obtain the degree of change. Metallic reflection is a stable physical phenomenon (such as continuous reflection on the surface of tin foil), while transient noise (such as dust obstruction or sensor fluctuations) presents as sudden changes in grayscale. The degree of change can be distinguished between the two by the proportion of grayscale difference between the current frame and the previous u-frame. The degree of change in the reflective area is usually <0.1 (stable grayscale), while the degree of change in the noise area is >0.2 (abrupt grayscale). Combined with dynamic optimization of the u-value (the stronger the noise, the larger the u-value), it can filter out more than 80% of transient interference.

[0030] The abnormal compactness, span, and variability are weighted and fused to obtain the fused feature value, which is then input into the Sigmoid function for optimization and output as the reflectivity probability. The weighted fusion calculation process includes: pre-setting weight factors for anomaly compactness, span, and variability; multiplying anomaly compactness, span, and variability with their corresponding weight factors respectively; and summing the results to obtain the fusion feature value.

[0031] Sigmoid function optimization: in, The probability of reflection is the final probability of determining whether a region is a reflective area (in the range of 0-1). This is the gain coefficient, which controls the steepness of the sigmoid curve. The larger the value, the more pronounced the feature differences are; in scenarios involving tin sheet identification and reflection suppression, Typically, a value of 5.0-8.0 is used, but the specific value should be selected based on the differences in characteristics between the normal area and the reflective area. A smarter approach is to dynamically adjust based on the feature dispersion of the current frame. :

[0032] Feature dispersion: Calculates the dispersion of samples in the normal region. The larger the standard deviation and dispersion, the more difficult it is to distinguish between reflective and normal area features. The larger; Effect: When there is severe feature overlap (e.g., in low light). Automatically boosted to 7.0-8.0 to enhance differences, especially in situations where features are clear (e.g., under strong light). Maintain a speed of 5.0-6.0 to ensure stability.

[0033] To fuse feature values; As an offset threshold, align the mean of the fusion features in the normal region so that the midpoint of the Sigmoid curve matches the mean of the normal region; This also includes optimizing k and u to make the probability output by the Sigmoid function more accurate: The k value (the number of adjacent minimum gray values ​​among 8 edge pixels) directly affects the stability of the span calculation and needs to be dynamically adapted to the dispersion of the edge gray distribution through a function. A k value that is too small (e.g., k=2) is easily affected by interference from a single noisy pixel, while a k value that is too large (e.g., k=6) will include high grayscale pixels, leading to distorted gradient calculations. An optimization function for the k value is designed based on the edge grayscale variance. in, The grayscale variance of the 8 edge pixels (reflecting the degree of grayscale dispersion at the edges); When the edge grayscale is uniform ( ≤30, normal solder sheet edge); The mean value is calculated by taking the three smallest gray values ​​to balance stability and gradient sensitivity. When the edge grayscale is discrete ( ≤60, slightly reflective edges); Increase the sample size to combat noise; When the edge grayscale fluctuates drastically ( >60, highly reflective edges); Suppressing highlight interference by using more low grayscale pixels; Limit k to the range of 2-6 to avoid extreme values ​​that could cause calculation failures.

[0034] The u-value (the reference frame number for the average grayscale value of the first u frames) needs to be adapted to the grayscale change frequency to dynamically balance noise immunity and real-time performance.

[0035] The u-value optimization function is designed based on the grayscale change rate between the current frame and the previous frame: in, (Grayscale difference between the current frame and the previous frame); When grayscale is stable ( ≤5, no transient noise); Reduce reliance on historical frames and ensure real-time performance; When the grayscale fluctuates slightly (5 < ≤10 (dust interference may exist). Suppress mutation noise by using more historical frames; When the grayscale changes drastically ( >10, strong transient noise); or Suppress mutation noise by using more historical frames; Limit u to the range of 2-5 to avoid storing too much historical data, which could cause delays.

[0036] Without the need for manual adjustment of k and u, the function can automatically adapt to the reflective intensity and noise level based on the real-time grayscale distribution.

[0037] Robot motion planning and brazing control: Combining robot dynamics characteristics, it enables precise mapping between positioning results and robot actions, covering pose transformation, motion planning, adaptive control of brazing parameters, and safety control; Specifically, it includes: The coordinates of the solder sheet output by visual recognition are converted into the coordinates of the end effector of the soldering gun using a hand-eye conversion matrix. Hand-eye transformation matrix: A 4×4 homogeneous transformation matrix is ​​used. Includes rotation matrix Translation vector The expression is: Wherein, rotation matrix (3×3) Used to convert the solder sheet rotation angle (Euler angle) To the welding torch posture, translation vector (Unit: mm) Used to compensate for the installation offset between the camera and the welding torch.

[0038] Coordinate transformation formula: If the coordinates of the center of the solder sheet in the robot's base coordinate system are... Then the target coordinates at the end of the welding torch Calculated using the following formula:

[0039] Rotation angle conversion is achieved through trigonometric function operations from Euler angles to rotation matrices (such as ZYX follow-through), with a conversion error ≤0.05°; translation conversion error ≤0.02mm (guaranteed by hand-eye calibration accuracy).

[0040] After generating the path based on the heuristic function, it is smoothed by a 5th-order B-spline (maximum curvature ≤ 0.5 rad / mm to avoid robot vibration); a trapezoidal velocity curve is used to control the welding torch movement speed to ensure smooth acceleration / deceleration. Speed ​​curve segmentation: acceleration segment (0.5s, speed increases linearly from 0 to 10mm / s) to constant speed segment (1s, speed remains at 10mm / s) to deceleration segment (0.5s, speed decreases linearly from 10mm / s to 0). Near-target deceleration mechanism: When the straight-line distance d between the soldering gun and the solder sheet is less than or equal to 5mm, the low-speed mode is triggered, and the speed is reduced to 2mm / s (to avoid airflow disturbance or collision risk caused by high-speed approach).

[0041] Heuristic function Based on the Euclidean distance from the current node to the target node, the expression is: The coefficient 1.2 is used to balance search speed and path optimality, and to prevent the algorithm from getting trapped in local optima. Smoothing is achieved using a 5th-order B-spline curve to ensure the path curvature meets the following requirements: ; (curvature) Calculations using the first and second derivatives of the path show that maximum curvature control can prevent end-effector vibrations caused by sudden stops and turns of robot joints. Adaptive control of brazing parameters, including current adjustment and welding torch oscillation, and real-time correction; Current adjustment: Base current 8A, increase current by 1A for every 0.1mm increase in solder sheet thickness (maximum 15A); when positioning deviation > 0.03mm, decrease current by 1A (to avoid solder overflow caused by misalignment). Welding torch oscillation: Default amplitude ±0.2mm, frequency 2Hz; when positioning accuracy <0.02mm, the oscillation amplitude is reduced to ±0.1mm; Real-time correction: Based on visual feedback of 10ms / frame, calculate the center deviation (ΔX, ΔY) between the soldering gun and the solder sheet, and adjust the end position in real time through PID control (correction amount per frame ≤ 0.01mm). By monitoring force and distance in two dimensions, mechanical collisions between the soldering gun and the solder sheet and workpiece are prevented, protecting the solder sheet (thickness 0.1-0.3mm, easily deformable) and the equipment; A 6-axis force sensor is integrated at the end of the welding torch to collect the contact force in real time and set a safety force threshold. When the contact force is greater than or equal to the safety force threshold, an emergency stop command is triggered. The vertical distance between the tip of the welding torch and the workpiece surface is calculated in real time based on the depth map, and the welding torch speed is adjusted accordingly based on the vertical distance. When the vertical distance is less than 2mm, the welding torch speed automatically decreases. When the vertical distance is less than 0.5mm (approaching collision), a pre-stop is triggered (the speed drops to 0.1mm / s) to provide buffer time for emergency stop.

[0042] Brazing quality inspection and feedback optimization: Quality is evaluated through post-weld quality inspection and SVM classifier, a quality-parameter correlation model is established, closed-loop optimization is achieved, and relevant data is recorded.

[0043] A 20-megapixel industrial camera is used to inspect the appearance of the solder to ensure uniformity of spread (edge ​​unevenness ≤0.1mm), coverage area (≥95%), and adhesion (≥90%). A laser contour sensor scans and generates a 3D point cloud to verify geometric parameters such as solder thickness (design value ±0.02mm) and height difference (≤0.03mm). A 50MHz ultrasonic probe focuses on the interface between the solder and the tin sheet to check for pores and inclusions with a diameter >0.01mm (foreign object reflection signal <0.1V).

[0044] Six features, including spreading uniformity and thickness deviation, are used to construct vectors. An SVM classifier (RBF kernel, accuracy ≥98%) is used to output a pass, need optimization, or fail grade. Based on a random forest regression model (prediction error ≤0.005mm), when needing optimization, parameters such as brazing current (increase ≤1A) and welding time are dynamically adjusted; when fail, the hand-eye matrix translation is corrected (compensation = 0.8 × center deviation).

[0045] Full-process data recording supports traceability and long-term optimization: The database stores information such as pre-welding positioning coordinates, process brazing parameters, and post-welding quality reports. Weekly analysis reports are generated to lock high-frequency defect-related parameters, and the model is updated monthly with new data.

[0046] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0047] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0048] It should be noted that, in this document, the use of relational terms such as "first" and "second" is merely for distinguishing one entity or operation from another, and does not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0049] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0050] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0051] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0052] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0053] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0054] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0055] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for automatic identification, positioning, and brazing of solder sheets based on a depth vision collaborative robot, characterized in that, include: Hardware sensing and multi-source data acquisition: Through hardware synchronous control and environmental adaptation components, the acquisition and spatiotemporal binding of tin sheet-related data are realized; Multi-source data input layer fusion: Clearly define calibration and fusion quantification indicators, and align the heterogeneous features of near-infrared images, depth maps, and RGB images through spatiotemporal calibration, heterogeneous data fusion processing, and noise preprocessing; Tin sheet identification and localization and reflection suppression: Coarse detection is performed using a lightweight network, metal reflection features are decoupled through an RSM module, and then precise localization is performed to complete the identification and localization of tin sheets and suppress the influence of reflection; Robot motion planning and brazing control: Combining robot dynamics characteristics, it enables precise mapping between positioning results and robot actions, covering pose transformation, motion planning, adaptive control of brazing parameters, and safety control.

2. The method for automatic identification, positioning, and brazing of solder sheets based on a depth vision collaborative robot according to claim 1, characterized in that, Hardware sensing and multi-source data acquisition specifically include: Multispectral imaging is achieved using near-infrared and RGB cameras; A binocular structured light sensor is used to output point cloud density, and the initial attitude is obtained by fitting the point cloud plane on the tin sheet surface. Based on the FPGA trigger board, a synchronous pulse signal is generated, and the pose feedback from the robot controller is received at the same time. The spatiotemporal binding of the image acquisition time and the robot pose time is realized through timestamp alignment.

3. The method for automatic identification, positioning, and brazing of solder sheets based on a depth vision collaborative robot according to claim 1, characterized in that, Multi-source data input layer fusion, specifically including: Spatiotemporal calibration and data alignment include camera intrinsic calibration, hand-eye calibration, and coordinate transformation; Near-infrared feature extraction uses 3×3 convolution; depth feature extraction uses 5×5 convolution. A 64×64 learnable mapping matrix is ​​designed, and reflection and geometric association weights are learned through backpropagation. The fused feature dimension is 128. A 3×3 Gaussian filter is used to smooth high-frequency noise, and a 3×3 erosion operation is combined to process near-infrared reflective noise. By using depth continuity detection, a bilateral filter is employed to preserve edges while suppressing noise.

4. The method for automatic identification, positioning, and brazing of solder sheets based on a depth vision collaborative robot according to claim 1, characterized in that, Tin sheet identification and positioning, and reflection suppression, specifically include: Input the fused 128×128 feature map and output candidate boxes with a confidence score ≥ 0.85; Interference from fixtures and weld slag is eliminated by measuring the average reflection intensity and depth flatness; then, the metal reflection characteristics are decoupled. Subpixel edge detection is performed using the Zernike moment algorithm; Edge points are fitted using the minimum bounding rectangle, and the center coordinates and rotation angle are calculated. The thickness is calculated using the depth difference.

5. The method for automatic identification, positioning, and brazing of solder sheets based on a depth vision collaborative robot according to claim 4, characterized in that, Decoupling of metallic reflective features specifically includes: The tin sheet image is divided into 3×3 regions. The gray values ​​of each pixel in each region are obtained and the mean is calculated to obtain the mean gray value of the region. Regions with a mean gray value > 200 are extracted and their compactness is calculated. After marking the regions with a compactness less than 0.3, the number of regions is counted. The total number of marked regions is divided by the total number of regions in the tin sheet image to obtain the abnormal compactness. Obtain the grayscale value of the center pixel of a 3×3 region, and the grayscale values ​​of the 8 pixels outside the center of the region. Sort the grayscale values ​​of the 8 pixels in ascending order of numerical value, extract the k nearest minimum grayscale values, take the average value, calculate the difference between the average value and the grayscale value of the center pixel of the region, and take the absolute value to obtain the span difference value. Preset a span difference threshold, divide the span difference value by the span difference threshold to obtain the span. The difference between the average grayscale value of the 3×3 region in the current frame and the average grayscale value of the region in the previous u frames is calculated and then divided by the average grayscale value of the 3×3 region in the current frame. The absolute value is then taken to obtain the degree of change. The abnormal compactness, span, and variability are weighted and fused to obtain the fused feature value, which is then input into the Sigmoid function for optimization and output as the reflectivity probability.

6. The method for automatic identification, positioning, and brazing of tin sheets based on a depth vision collaborative robot according to claim 5, characterized in that, Sigmoid function optimization: in, This represents the probability of reflection. This is the gain coefficient; To fuse feature values; This is the offset threshold.

7. The method for automatic identification, positioning, and brazing of solder sheets based on a depth vision collaborative robot according to claim 6, characterized in that, This also includes optimizing k and u, and designing an optimization function for the k value based on the edge grayscale variance: in, The grayscale variance of the 8 edge pixels: in, It represents the absolute value of the grayscale difference between the current frame and the previous frame.

8. The method for automatic identification, positioning, and brazing of solder sheets based on a depth vision collaborative robot according to claim 1, characterized in that, Robot motion planning and brazing control, specifically including: The coordinates of the solder sheet output by visual recognition are converted into the coordinates of the end effector of the soldering gun using a hand-eye conversion matrix. The path is generated based on a heuristic function and then smoothed using a 5th-order B-spline; a trapezoidal velocity curve is used to control the welding torch speed to ensure a smooth acceleration / deceleration process. Adaptive control of brazing parameters, including current adjustment and welding torch oscillation, and real-time correction; By monitoring both force and distance, mechanical collisions between the soldering torch and the solder sheet / workpiece are prevented, thus protecting the solder sheet and equipment. A force sensor is integrated at the end of the welding torch to collect the contact force in real time and set a safety force threshold. When the contact force is greater than or equal to the safety force threshold, an emergency stop command is triggered. The vertical distance between the tip of the welding torch and the workpiece surface is calculated in real time based on the depth map, and the welding torch speed is adjusted accordingly based on the vertical distance.

9. The method for automatic identification, positioning, and brazing of solder sheets based on a depth vision collaborative robot according to claim 1, characterized in that, Brazing quality inspection and feedback optimization: Quality is evaluated through post-weld quality inspection and SVM classifier, a quality-parameter correlation model is established, closed-loop optimization is achieved, and relevant data is recorded.

Images