A robot projection welding vision positioning method and device for vehicle body welding, electronic equipment and computer readable storage medium

Abstract

The application provides a robot projection welding visual positioning method and device for vehicle body welding, electronic equipment and computer readable storage medium, and relates to the field of robot visual positioning. The method comprises the following steps: constructing prior positions and geometric constraints of welding points based on vehicle body design information data and tooling information data; constructing a three-dimensional reference model based on the prior positions and geometric constraints, and outputting a geometric mapping relationship; extracting a welding point positioning feature set in a welding point image; associating the welding point positioning feature set with the geometric mapping relationship through feature constraint, and outputting a first spatial pose of the welding point in a three-dimensional space through the three-dimensional reference model; performing a precision verification operation on the first spatial pose and correcting it to a second spatial position; converting the second spatial pose to a robot coordinate system, and controlling a target robot to perform projection welding visual positioning. The application solves the problem of low positioning accuracy of the traditional robot projection welding visual positioning method for vehicle body welding under complex working conditions.

Description

Technical Field

[0001] This application relates to the field of robot vision positioning, and more particularly to a robot projection welding vision positioning method, apparatus, electronic device, and computer-readable storage medium for vehicle body welding. Background Technology

[0002] Against the backdrop of continuous upgrading in intelligent manufacturing and the automotive industry, automated body welding is developing towards higher precision, flexibility, and intelligence. With the increasing complexity of body structures and the widespread application of lightweight materials, traditional welding methods relying on fixed tooling and rigid paths are no longer sufficient to meet production demands.

[0003] Traditional methods typically rely on tooling reference points and fixed compensation parameters to achieve robotic projection welding vision positioning for car body welding. However, in the real-world scenario of a car body welding production line, various interference factors can cause different types of positioning errors. For example, strong reflections and localized highlights on the car body exterior can blur the edges of weld point images, leading to unstable feature extraction; thermal drift and joint clearance accumulation during long-term robot operation can reduce hand-eye calibration accuracy, resulting in pose calculation deviations; and dense weld point distribution and surface scratch textures can cause overlapping candidate regions, leading to false positives or false negatives. In such cases, the methods described above alone cannot correct for the errors caused by these factors, resulting in a decrease in the overall accuracy of robotic projection welding vision positioning.

[0004] Therefore, there is an urgent need for a robotic projection welding visual positioning method, device, electronic equipment, and computer-readable storage medium for automotive body welding. Summary of the Invention

[0005] This application provides a robotic projection welding visual positioning method, device, electronic device, and computer-readable storage medium for automotive body welding, which solves the problem of low positioning accuracy in traditional robotic projection welding visual positioning methods for automotive body welding under complex working conditions.

[0006] The first aspect of this application provides a visual positioning method for projection welding of a robot for vehicle body welding. The method includes: acquiring vehicle body design information data and tooling information data of the vehicle to be welded, and constructing a priori position and geometric constraints of the weld point based on the vehicle body design information data and tooling information data; constructing a three-dimensional reference model based on the priori position and geometric constraints, and outputting a geometric mapping relationship, the geometric mapping relationship including the spatial relationship between the target robot, the target camera and the weld point; acquiring a weld point image based on the target camera, and extracting a set of weld point positioning features from the weld point image; associating the weld point positioning feature set with the geometric mapping relationship through feature constraints, and outputting a first spatial pose of the weld point in three-dimensional space through the three-dimensional reference model; performing an accuracy verification operation on the first spatial pose, and correcting the first spatial pose to a second spatial position based on welding disturbance factors; converting the second spatial pose to the robot coordinate system, and controlling the target robot to complete the projection welding visual positioning through the converted second spatial pose.

[0007] Optionally, the vehicle body design information data and tooling information data of the vehicle to be welded are obtained, and the prior position and geometric constraints of the weld points are constructed based on the vehicle body design information data and tooling information data. Specifically, this includes: extracting the vehicle body structure model based on the vehicle body design information data, and marking the nominal space coordinates and local normal directions corresponding to the weld points in the vehicle body structure model; using the nominal space coordinates and local normal directions as prior positions; determining the constraints of the fixtures, jigs, and process reference surfaces based on the tooling information data, and correcting the prior positions under the constraints to obtain corrected positions that meet the constraints; and combining the corrected positions with the clamping direction, reference plane normal, and adjacency relationship in the tooling information data to generate geometric constraints.

[0008] Optionally, a three-dimensional reference model is constructed based on prior positions and geometric constraints, and geometric mapping relationships are output. Specifically, this includes: combining prior positions and geometric constraints through spatial fusion operations, and establishing the three-dimensional spatial distribution of weld points in the vehicle body coordinate system based on the combination results; and constructing a three-dimensional reference model based on the three-dimensional spatial distribution to output geometric mapping relationships.

[0009] Optionally, the solder joint image is acquired based on the target camera, and a set of solder joint positioning features is extracted from the solder joint image. Specifically, this includes: performing reflection suppression and multi-exposure fusion processing on the solder joint image to enhance the imaging quality of the area where the solder joint is located; extracting the edge contour features and local geometric features of the solder joint in the enhanced solder joint image, and performing sub-pixel refinement processing on the local geometric features; combining the edge contour features and the refined local geometric features through feature fusion operation, and outputting a set of solder joint positioning features, which includes shape features, position features, and normal features.

[0010] Optionally, the solder joint location feature set is associated with the geometric mapping relationship through feature constraints. Specifically, this includes: matching the shape features in the solder joint location feature set with the nominal geometric constraints in the geometric mapping relationship to establish a geometric correspondence between the solder joints; registering the position features in the solder joint location feature set with the prior position in the geometric mapping relationship to establish a spatial correspondence between the solder joints; and applying consistency constraints between the normal features in the solder joint location feature set and the local normal direction in the geometric mapping relationship to establish an attitude correspondence between the solder joints; and forming a feature constraint association through the geometric correspondence, spatial correspondence, and attitude correspondence.

[0011] Optionally, an accuracy verification operation is performed on the first spatial pose, and the first spatial pose is corrected to a second spatial position based on welding disturbance factors. Specifically, this includes: calculating the first welding accuracy corresponding to the weld point based on the first spatial pose; if the first welding accuracy is determined to be less than a preset accuracy, outputting welding disturbance factors through a welding disturbance recognition model; outputting a weld point correction strategy corresponding to the welding disturbance factors through a welding disturbance recognition model based on the welding disturbance factors; correcting the first spatial pose based on the weld point correction strategy to obtain a second spatial pose; calculating the second welding accuracy corresponding to the weld point based on the second spatial pose, and completing the accuracy verification operation when it is confirmed that the second welding accuracy is greater than or equal to the preset accuracy.

[0012] Optionally, a welding disturbance recognition model is constructed, specifically including: acquiring production line environmental monitoring information, including ambient lighting parameters, body assembly deviation parameters, robot thermal drift parameters, and camera imaging stability parameters; using historical weld point positioning feature sets, geometric mapping relationships, and production line environmental monitoring information as training data, and constructing welding disturbance features, including feature residual distribution features, pose drift quantification index features, assembly tolerance offset features, and imaging deviation features; constructing a first correspondence between welding disturbance features and welding disturbance factors; based on welding disturbance factors, constructing a weld point correction strategy through an adaptive correction generation mechanism, including an imaging parameter compensation strategy, a pose optimization compensation strategy, a trajectory constraint correction strategy, and a calibration drift compensation strategy; constructing a second correspondence between welding disturbance factors and weld point correction strategies; and constructing a welding disturbance recognition model based on the first and second correspondences.

[0013] A second aspect of this application provides a robotic projection welding vision positioning device for vehicle body welding, the device including an acquisition module and an output module, wherein,

[0014] The acquisition module is used to acquire the body design information data and tooling information data of the vehicle to be welded, and construct the prior position and geometric constraints of the weld point based on the body design information data and tooling information data; construct a three-dimensional benchmark model based on the prior position and geometric constraints, and output the geometric mapping relationship, which includes the spatial relationship between the target robot, the target camera and the weld point; acquire the weld point image based on the target camera, and extract the weld point positioning feature set in the weld point image.

[0015] The output module is used to associate the weld point positioning feature set with the geometric mapping relationship through feature constraints, and output the first spatial pose of the weld point in three-dimensional space through the three-dimensional reference model; to perform accuracy verification operation on the first spatial pose, and correct the first spatial pose to the second spatial position based on welding disturbance factors; to transform the second spatial pose to the robot coordinate system, and to control the target robot to complete the projection welding visual positioning through the transformed second spatial pose.

[0016] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described above.

[0017] A fourth aspect of this application provides a computer-readable storage medium storing a computer program, which is executed by a processor using the method described in any of the foregoing descriptions.

[0018] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0019] 1. Based on vehicle body design information and tooling information, construct the prior position and geometric constraints of the weld points; construct a three-dimensional reference model based on the prior position and geometric constraints, and output the geometric mapping relationship; extract the weld point positioning feature set from the weld point image; associate the weld point positioning feature set with the geometric mapping relationship through feature constraints, and output the first spatial pose of the weld point in three-dimensional space through the three-dimensional reference model; perform accuracy verification operation on the first spatial pose and correct it to the second spatial position; convert the second spatial pose to the robot coordinate system, and control the target robot to perform projection welding visual positioning, thereby achieving dynamic alignment and high-precision positioning of the weld points from the design state to the actual assembly state under the complex working conditions of the vehicle body welding production line. This effectively reduces the positioning error caused by factors such as assembly tolerance, calibration drift, imaging reflection and environmental interference, improves the stability and consistency of robot projection welding operations, and ensures that the welding quality meets production requirements.

[0020] 2. Perform reflection suppression and multi-exposure fusion processing on the solder joint image to enhance the imaging quality of the area where the solder joint is located; extract the edge contour features and local geometric features of the solder joint in the enhanced solder joint image, and perform sub-pixel refinement processing on the local geometric features; combine the edge contour features and the refined local geometric features through feature fusion operation, and output the solder joint positioning feature set, thereby obtaining a stable solder joint positioning feature set that combines shape features, position features and normal features, providing high-quality input for subsequent feature constraint association and spatial pose calculation based on the 3D benchmark model, and improving the accuracy and robustness of solder joint recognition and positioning.

[0021] 3. Based on the first spatial pose, calculate the first welding accuracy corresponding to the weld point; if the first welding accuracy is determined to be less than the preset accuracy, output the welding disturbance factor through the welding disturbance identification model; based on the welding disturbance factor, output the weld point correction strategy corresponding to the welding disturbance factor through the welding disturbance identification model; correct the first spatial pose based on the weld point correction strategy to obtain the second spatial pose; calculate the second welding accuracy corresponding to the weld point based on the second spatial pose, and complete the accuracy verification operation when it is confirmed that the second welding accuracy is greater than or equal to the preset accuracy. In this way, targeted compensation for different disturbance factors is achieved in the process of weld point spatial pose calculation, ensuring that the weld point positioning result still has verifiable accuracy and stability under complex working conditions, and improving the robustness and consistency of robot projection welding operation. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a visual positioning method for robot projection welding in vehicle body welding, provided in an embodiment of this application.

[0023] Figure 2 This is a schematic diagram of a robot projection welding vision positioning device for car body welding provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0025] Explanation of reference numerals in the attached figures: 21. Acquisition module; 22. Output module; 301. Processor; 302. Communication bus; 303. User interface; 304. Network interface; 305. Memory. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0027] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0028] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0029] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0030] Please refer to Figure 1 The diagram illustrates a flowchart of a robotic projection welding visual positioning method for vehicle body welding provided in an embodiment of this application. The flowchart mainly includes the following steps: S101 to S106.

[0031] Step S101: Obtain the body design information data and tooling information data of the vehicle to be welded, and construct the prior position and geometric constraints of the weld point based on the body design information data and tooling information data.

[0032] Specifically, in the vehicle body welding production line scenario, the vehicle body-in-white is composed of multiple covering parts and structural components, and tooling fixtures are used to fix the body in the workstation. However, in this scenario, the following technical problems arise: In the highly reflective environment of the vehicle body exterior panels, specular reflection and strong light spots often appear in the weld point area, causing the weld point edges in the image captured by the camera to be blurred or locally overexposed, making it difficult to perform stable sub-pixel fitting. This is because the surface gloss of the electroplated or painted layer of the exterior panel is high, and even a slight deviation in the light source and imaging angle will produce strong reflection; Under complex working conditions, the robot hand-eye calibration is difficult to maintain accuracy in long-term operation, causing the positioning result to drift from the actual weld point position. This is due to the accumulation of robot joint gaps, thermal expansion and mechanical deformation, as well as the camera's pose shift caused by vibration or collision; In scenarios with densely distributed weld points, the distance between adjacent weld points is too close and accompanied by surface textures or scratches. Interference causes overlap of weld point feature areas, leading to false positives or false negatives. This is because weld points are concentrated in one area, and image processing algorithms struggle to distinguish similar local features. In body assembly deviation scenarios, millimeter-level assembly tolerances cause overall weld point offsets, making it impossible to align the prior CAD weld point model with the actual position. This is due to fixture errors, heat treatment deformation, and sheet metal springback. In high-speed production line scenarios, cameras need to quickly acquire and process data, but complex image enhancement and 3D computation introduce delays due to high algorithm complexity and limited computing power. In the dusty, spattery, and temperature-prone environment of the welding site, the stability of the imaging link decreases. This is because spatter adheres to the lens, reducing contrast, and temperature fluctuations increase sensor noise or even cause calibration thermal drift. These combined problems present multiple challenges to the visual positioning of body welding robots, simultaneously impacting algorithm accuracy, hardware stability, and environmental adaptability.

[0033] To address the technical problems that may arise in the aforementioned scenarios, this application embodiment constructs a three-dimensional reference model and analyzes welding disturbance factors. When deviations are detected, multiple correction strategies are invoked to ensure the positioning accuracy of the weld points. First, in step S101, the vehicle body design information data and tooling information data of the vehicle to be welded are acquired. Based on the vehicle body design information data, the prior position of the weld point is determined. Then, based on the tooling information data, geometric constraints are determined. Finally, the prior position and geometric constraints are combined to form the prior spatial distribution of the weld points, providing input conditions for the subsequent construction of the three-dimensional reference model.

[0034] In one possible implementation, step S101 further includes: acquiring the vehicle body design information data and tooling information data of the vehicle to be welded, and constructing the prior position and geometric constraints of the weld point based on the vehicle body design information data and tooling information data. Specifically, this includes: extracting the vehicle body structure model based on the vehicle body design information data, and marking the nominal space coordinates and local normal directions corresponding to the weld point in the vehicle body structure model; using the nominal space coordinates and local normal directions as the prior position; determining the constraints of the fixture, jig, and process reference surface based on the tooling information data, and correcting the prior position under the constraints to obtain a corrected position that meets the constraints; and combining the corrected position with the clamping direction, reference plane normal, and adjacency relationship in the tooling information data to generate geometric constraints.

[0035] Specifically, in order to address the potential error factors in the above scenarios, this application embodiment combines the body design information data in the design state with the tooling information data in the actual assembly state, thereby eliminating the systematic offset caused by assembly tolerances, clamping directions, and reference planes. This allows the a priori position and geometric constraints of the weld points to truly reflect the constrained geometric relationships at the workstation and serve as input for the subsequent construction of the three-dimensional reference model.

[0036] When extracting the body structure model based on body design information data, a body coordinate system is established and the properties of weld point-related entities and layers are analyzed to locate the geometric attachment objects of the weld point in the design state, such as surface attachment points, reinforcement plate hole edges, and flange segments. The normal direction is calculated on the attachment surface to obtain the nominal spatial coordinates and local normal direction of each weld point. The nominal spatial coordinates and local normal directions are solidified as a priori positions, and the topological references between the weld point and its attachment surface, sheet metal, and assembly level are recorded for subsequent constraints and traceability.

[0037] When determining constraints based on tooling information data, the process reference surfaces and clamping directions defined by the locating pins, V-blocks, clamping blocks, and vacuum adsorption surfaces are analyzed. The "datum A / B / C" constraints of the station on key assemblies, the thickness configuration of the fixture compensation shims, and the clamping stroke and preload range are read to form a set of constraints. The set of constraints is then checked for consistency, and conditions that conflict with or contradict the CAD assembly relationship are eliminated. Priority rules are given, such as datum surfaces taking precedence over local clamping surfaces and assembly datums taking precedence over part datums.

[0038] When correcting the prior position under constraints, the prior position is first projected along the normal of the corresponding process reference surface with a limited amplitude to align with the clamping datum at the workstation. When there is a deviation between the clamping direction and the local normal direction, the local normal direction is constrained and converged according to the incident angle limit set by the clamping direction to make the normal consistent with the reachable incident angle range. When the jig shim thickness or assembly tolerance library gives a systematic offset, the offset is superimposed on the nominal space coordinates according to priority to obtain a corrected position that conforms to the clamping boundary. During the correction process, the correction trajectory and cause label are retained to ensure that every offset can be traced and rolled back.

[0039] When combining the corrected position with the clamping direction, reference plane normal, and adjacency relationship in the tooling information data to generate geometric constraints, each weld point is constructed with the following constraint terms: position tolerance interval to limit the allowable spatial drift range of the weld point, normal consistency interval to limit the angle range between the actual incident normal and the local normal direction, adjacency topology to limit the relative order and minimum spacing between weld points within the same partition, and occlusion mask and visual reachability constraints to mark the no-entry zone and priority observation field of the camera and welding clamp; the above constraint terms, together with the corrected position, form the geometric constraints of the weld point and maintain a one-to-one mapping relationship with the prior position, which can be directly consumed by the subsequent 3D reference model.

[0040] To ensure the availability of data within the production line closed loop, the prior set of solder joints is organized into structured records. Core fields include solder joint ID, nominal spatial coordinates and local normal direction of the prior position, corrected position, attached object reference, workstation reference identifier, clamping direction, reference plane normal, list of adjacent solder joint IDs, position tolerance and normal tolerance, occlusion mask, weight, and effective version number. The records are versioned and bound to the tooling configuration sheet, so that any tooling change or compensation adjustment can automatically trigger the regeneration and comparison of prior positions and geometric constraints.

[0041] Taking the overlap section between the outer and inner panels of the left front door as an example, the CAD layer provides the geometric definitions of the outer panel surface and the inner panel ribs, as well as the weld point distribution mesh. After extracting the nominal spatial coordinates and local normal direction of each weld point, the "datum A" of the section in the tooling information data is read as the outer surface datum of the outer panel, the "datum B" is the upper and lower positioning pin axis, and the "datum C" is the door frame reference surface. At the same time, the clamping direction of the clamping block on the outer panel is read. Under the constraint conditions, the prior position is projected along the normal of datum A with millimeter-level constraints, and the incident angle of the local normal direction is limited according to the direction of the clamping block. If the fixture compensation shim has a uniform thickness in the section, the thickness is mapped as a systematic translation along the normal of datum A to generate the corrected position. Finally, the corrected position, clamping direction, reference plane normal, and adjacency relationship within the section are packaged into geometric constraints to form a data object that can be directly called by the downstream three-dimensional datum model.

[0042] Step S102: Construct a three-dimensional reference model based on prior position and geometric constraints, and output geometric mapping relationships, including the spatial relationships between the target robot, the target camera and the welding points.

[0043] Specifically, the corrected prior positions are first spatially fused with the corresponding geometric constraints to generate a three-dimensional spatial distribution of the weld points. Then, this spatial distribution is aligned with the vehicle body coordinate system, and the target robot base coordinate system and the target camera coordinate system are also included to establish a coordinate mapping relationship between them and the weld points. The final output geometric mapping relationship is used to describe the correspondence and constraint relationship between the robot end effector, the camera field of view, and the weld points in three-dimensional space, thereby providing a unified benchmark for subsequent image feature matching and pose calculation.

[0044] In one possible implementation, step S102 further includes: combining the prior position and geometric constraints through spatial fusion operation, and establishing a three-dimensional spatial distribution of the weld point in the vehicle body coordinate system based on the combination result; constructing a three-dimensional reference model based on the three-dimensional spatial distribution to output the geometric mapping relationship.

[0045] Specifically, using prior positions and geometric constraints as input, a spatial fusion operation is first performed to align the prior positions of each weld point with their corresponding geometric constraints within the vehicle body coordinate system. During alignment, the prior positions are subjected to restricted projection and amplitude correction using the reference datum plane where the workstation is active and the clamping direction as constraint boundaries, ensuring that the position and normal direction simultaneously meet the process incident angle range, position tolerance range, and minimum spacing requirements. For weld points with clamping zone boundaries, conflicts are handled according to constraint priority, and the three-dimensional spatial distribution of the weld points in the vehicle body coordinate system is solidified.

[0046] After obtaining the 3D spatial distribution, a data skeleton for the 3D reference model is constructed. A node record is created for each weld point, containing the corrected 3D coordinates, local normal direction, clamping area identifier, adjacency topology, and visible reachability marker. Position tolerance intervals and normal consistency intervals are attached to the nodes as constraints. A sub-model is generated for each partition, recording the reference plane normal, occlusion mask, and preferred viewing area for that partition. The nodes and sub-models are organized into a hierarchical structure, with the vehicle body coordinate system as the sole upper-level coordinate reference, forming a 3D reference model that can be directly consumed by downstream solvers.

[0047] Geometric mapping relationships are generated on the 3D baseline model. First, the identifiers and placeholder mappings of the target robot base coordinate system and the target camera coordinate system in the vehicle body coordinate system are registered. Coordinate mapping placeholders for the robot base coordinate system, robot flange coordinate system, and target camera coordinate system are defined. A process local coordinate system is derived at each weld point node, providing a set of normal alignment directions and candidate approach directions. Then, the "viewpoint-reachability" intersection is calculated for each weld point. Camera visibility, welder reachability, and tooling occlusion constraints are integrated to generate a correspondence between the candidate set of camera observation poses and the candidate set of robot positioning poses. These correspondences, together with the weld point nodes, constitute the core content of the geometric mapping relationships.

[0048] Perform consistency checks and version fixation on the geometric mapping relationships. Check whether the candidate approach directions and normal intervals of all weld points are compatible, whether the minimum spacing within the partition meets the constraints, and whether the visible reachable set is non-empty. Output a flag for nodes that fail the checks, awaiting subsequent disturbance identification and adaptive correction strategies. Write the nodes that pass the checks and their view-reachable correspondences into the baseline version of the current workstation configuration to ensure that subsequent steps run under the same baseline.

[0049] After the above processing, two types of objects are output. First, a 3D baseline model containing weld point nodes, partition sub-models, and hierarchical indexes. Second, a geometric mapping describing the spatial relationship between the target robot, target camera, and weld points, including paired mappings of the weld point local coordinate system, candidate approach direction set, candidate camera observation pose set, and candidate robot positioning pose set. These two types of outputs serve as the sole input for the next step, guiding the selection of the viewport for image acquisition, the setting of the search range for the weld point localization feature set, and the constraint assembly for subsequent pose calculation.

[0050] Step S103: Acquire solder joint images based on the target camera and extract the solder joint location feature set from the solder joint images.

[0051] Specifically, to address the high reflectivity and complex local textures in the solder joint area, reflection suppression and multi-exposure fusion are first applied to the image to improve local contrast and detail fidelity. Subsequently, edge detection and geometric contour extraction are performed on the solder joint area in the enhanced image, combined with sub-pixel-level refinement processing, to obtain a set of solder joint localization features including positional features, shape features, and normal features, providing input for subsequent feature constraint association and pose calculation.

[0052] In one possible implementation, step S103 further includes: performing reflection suppression and multi-exposure fusion processing on the solder joint image to enhance the imaging quality of the area where the solder joint is located; extracting the edge contour features and local geometric features of the solder joint in the enhanced solder joint image, and performing sub-pixel refinement processing on the local geometric features; combining the edge contour features and the refined local geometric features through feature fusion operation, and outputting a solder joint positioning feature set, which includes shape features, position features, and normal features.

[0053] Specifically, guided by the three-dimensional benchmark model and geometric mapping relationship, the acquisition of solder joint images and the generation of "solder joint positioning feature set" are completed. First, reflection suppression and multi-exposure fusion are performed to obtain solder joint images with high dynamic range and high detail fidelity. Then, edge contour features and local geometric features are extracted and sub-pixel refinement is performed. Finally, the solder joint positioning feature set containing shape features, position features and normal features is output through feature fusion operation.

[0054] When performing reflection suppression on solder joint images, a specular mask is first constructed in the HSV color space, and the intensity of the V channel is denoted as... saturation is With threshold Identify the highlight area:

[0055]

[0056] in, For pixel coordinates, The intensity threshold, The saturation threshold, It is a binary specular mask. This represents logical AND. This is an indicator function. Morphological closing and small connected component removal are performed to suppress noise; the structuring element size is denoted as... In the highlight region, neighborhood-guided filtering or bilateral filtering is used to recover details, with the filter window radius denoted as... The intensity similarity parameter is denoted as .

[0057] When performing multi-exposure fusion on solder joint images, the set of exposure times is acquired. Corresponding multi-frame image grayscale Using the irradiance response function with weight function To merge:

[0058]

[0059] in, This is the exponential reduction result of the fused logarithmic irradiance. For the first Frame exposure time, For camera response function, This is a median-priority weighting function used to suppress underexposed and overexposed pixels. The camera response function can be estimated using a standard calibration sequence. The weighting function can be triangular or Gaussian weighted, and its shape parameter is denoted as . .

[0060] When extracting edge contour features from the enhanced solder joint image, the noise is first smoothed using a Gaussian kernel, and the standard deviation is denoted as... Then, the Canny operator with hysteresis thresholding is used to obtain the binary edges, with the low threshold and high threshold being respectively... Obtain the edge set. The contour curve is then refined to obtain a single-pixel width for stable subsequent fitting and normal estimation.

[0061] When extracting local geometric features and performing sub-pixel refinement on the enhanced solder joint image, the features located at... Each edge point Calculate gradient One-dimensional quadratic curve fitting is performed along the gradient normal direction to obtain the sub-pixel edge position, with the parameter being the sampling step size. With radius An attempt was made to fit the sampling intensity sequence to... The offset of the sub-pixel edge position along the normal direction is:

[0062]

[0063] in, These are the coefficients for fitting the local quadratic curve. This represents the sub-pixel offset in the normal direction. Control the sampling density and range.

[0064] In the stage of globalizing local geometric features, the contours of the weld point caps or convex points are fitted with ellipses using a quadratic curve model, and the edge subsets are selected. Solution:

[0065]

[0066] in, The parameters of the conic section are defined, and the constraints ensure that the conic section is an ellipse. The center of the ellipse can be obtained analytically. Spindle length With rotation angle These quantities serve as the basic representations of shape and position features.

[0067] In normal feature estimation, a structural tensor is used to robustly solve for the locally dominant directions. The gradient field of the enhanced image is then calculated using a Gaussian kernel. Smoothing yields the structure tensor

[0068]

[0069] in, The principal eigenvectors correspond to the edge tangential direction, and the normal direction is taken as its orthogonal direction. The normal angle is denoted as . , The eigenvector corresponding to the largest eigenvalue. This is the normal smoothing scale. To improve consistency with the local normal directions in the 3D baseline model, it can be... A small regularization is applied to the vicinity, biasing it towards the range of incident angles allowed by the geometric mapping. The regularization strength is denoted as... .

[0070] When performing feature fusion operations on edge contour features and sub-pixel geometric features, a solder joint localization feature vector is constructed:

[0071]

[0072] in, For feature fusion weights, satisfying , Shaft ratio, The rotation angle of the ellipse. For sub-pixel center coordinates, This is the component representation of the unit normal vector. To improve robustness, we can... For whitening normalization, the mean and variance are estimated from historical samples, and the normalization scale is denoted as... .

[0073] In feature quality measurement and candidate selection, a consistency score is introduced, which is a weighted sum of reprojection consistency, marginal support, and normal bias:

[0074]

[0075] in, This represents the mean of the local reprojection residuals obtained based on the current geometric mapping relationship. The residual scaling factor. The percentage of valid edge points within the fitted ellipse. The deviation angle between the image normal and the center of the allowable incident angle. The angle scale factor, As the scoring weight, satisfying .in accordance with Set threshold Filter out stable and consistent sets of solder joint location features.

[0076] After completing the above processing, a set of solder joint positioning features is output, which includes shape features. Location features With normal features And accompanied by quality scores Uncertainty estimates (such as sub-pixel center and normal variance obtained from local Jacobian and noise covariance propagation, used for weighting subsequent feature constraint association and pose solution) are used to ensure that subsequent steps can use the set in a consistent and weighted manner.

[0077] Step S104: Associate the solder joint positioning feature set with the geometric mapping relationship through feature constraints, and output the first spatial pose of the solder joint in three-dimensional space through the three-dimensional reference model.

[0078] Specifically, the shape features in the localization feature set are matched with the nominal geometry in the geometric constraints, the position features are spatially registered with the prior position, and the normal features are constrained with the local normal direction to establish a stable correspondence between the two-dimensional image features and the three-dimensional reference data. After completing the constraint matching of multi-dimensional features, the first spatial pose of the weld point in three-dimensional space is calculated through the three-dimensional reference model, providing the basic input for subsequent accuracy verification and correction.

[0079] In one possible implementation, step S104 further includes: matching the shape features in the solder joint positioning feature set with the nominal geometric constraints in the geometric mapping relationship to establish a geometric correspondence of the solder joints; registering the position features in the solder joint positioning feature set with the prior position in the geometric mapping relationship to establish a spatial correspondence of the solder joints; applying consistency constraints based on the normal features in the solder joint positioning feature set with the local normal direction in the geometric mapping relationship to establish an attitude correspondence of the solder joints; and forming a feature constraint association through the geometric correspondence, spatial correspondence, and attitude correspondence.

[0080] Specifically, the 3D reference model provides nominal geometric constraints, prior position, and local normal direction. The weld point positioning feature set provides shape features, position features, and normal features. A unified pose calculation objective function is constructed through feature constraint association, and the first spatial pose is solved under the coordinate scale of the 3D reference model. The specific processing chain is as follows: a geometric correspondence is established between shape features and nominal geometric constraints; a spatial correspondence is established between position features and prior position; and an attitude correspondence is established between normal features and local normal direction. Then, the three correspondences are unified into an optimization problem constrained by a robust kernel and weight scheduling. Finally, the rotation and translation of the target camera relative to the local reference of the weld point are solved using the Gauss-Newton method or the Levenberg-Marquardt method, and the first spatial pose is output.

[0081] Construct a reprojection consistency term from location features and prior location:

[0082]

[0083] in, For rotation matrix, It is a translation vector. For the third in the three-dimensional benchmark model a priori position To and The sub-pixel coordinates of the corresponding image location features, For the camera intrinsic parameter matrix, For the distortion parameter vector, This is an imaging projection model with distortion. The weighting coefficients are derived from the quality score in step S103. A robust kernel function for suppressing outliers.

[0084] Constructing a normal consistency term from normal features and local normal directions:

[0085]

[0086] in, For the three-dimensional benchmark model in The local normal direction at that location The unit direction vector is obtained by applying back-projection constraints to the image normal features. The weight of the normal consistency term, is the robust kernel function for the normal term.

[0087] A contour shape consistency term is constructed from shape features and nominal geometric constraints. Let the implicit quadratic curve of the nominal contour on the camera imaging plane be... Image shape features provide parameters for the center and principal axis directions of the ellipse. , with the set of sampled contour points Measurement cost:

[0088]

[0089] in, The set of contour points obtained by fitting ellipses with shape features. For the weight of the shape item, For the shape term robust kernel function, By using a three-dimensional nominal round cap or convex surface in The implicit expression obtained after downprojection and parameter elimination.

[0090] By combining the three types of constraints, a unified pose calculation objective is obtained:

[0091]

[0092] Among them, the three items correspond to spatial correspondence, attitude correspondence, and geometric correspondence, respectively. Adjusting relative contributions, It reflects the confidence level at a single point.

[0093] Linearize the Lie algebra at the optimum, let To minimize the parameterization increment, update the pose as follows: The linearized forms of the three types of residuals are summarized using the Jacobian matrix:

[0094]

[0095] in, To connect the incremental stacked vectors of the three types of residuals, This is the corresponding block Jacobian matrix. For the pose increment, weighted least squares are used to solve the normal equations:

[0096]

[0097] in, For the reason And the diagonal weight matrix formed by the weights derived from the robust kernel. The damping coefficient is... This is the current residual stacking vector. It is an identity matrix.

[0098] To generate uncertainty and support subsequent accuracy verification, the pixel-level covariance of the propagation near the optimal solution and the normal measurement noise is calculated:

[0099]

[0100] in, The covariance approximation of the pose increment can be further mapped to the uncertainty of the position and attitude components, which is used for the calculation of the first welding accuracy and the determination of the threshold.

[0101] The above optimization convergence result Recorded as the first spatial pose, and associated with The outputs are combined and used as input for subsequent accuracy verification and welding disturbance identification. The above process takes the set of weld point positioning features and geometric mapping relationships as input, and completes joint calculation under the coordinate scale provided by the three-dimensional reference model to ensure that the source of the first spatial pose is clear and measurable.

[0102] Step S105: Perform accuracy verification on the first spatial pose and correct the first spatial pose to the second spatial position based on welding disturbance factors.

[0103] Specifically, the first spatial pose obtained through calculation is used in conjunction with the tolerance range provided by the 3D reference model to verify whether the position and orientation of the weld point meet the preset accuracy threshold. When the verification result is lower than the preset accuracy, a welding disturbance identification model is introduced to determine possible disturbance factors such as imaging reflection, hand-eye calibration drift, assembly tolerance offset, and dense interference from weld points. Based on the identified disturbance factors, corresponding adaptive correction strategies, such as imaging parameter compensation, pose optimization compensation, or trajectory constraint correction, are called from the disturbance factor correction database to correct the first spatial pose, resulting in a second spatial pose. After completing the second accuracy verification and confirming that it meets the preset accuracy, this second spatial pose serves as the final positioning input for subsequent welding trajectory generation and robot control.

[0104] In one possible implementation, step S105 further includes: calculating a first welding accuracy corresponding to the weld point based on a first spatial pose; if it is determined that the first welding accuracy is less than a preset accuracy, outputting welding disturbance factors through a welding disturbance identification model; acquiring production line environment monitoring information, including ambient lighting parameters, vehicle body assembly deviation parameters, robot operation thermal drift parameters, and camera imaging stability parameters; outputting a weld point correction strategy corresponding to the welding disturbance factors through a welding disturbance identification model based on the welding disturbance factors; correcting the first spatial pose based on the weld point correction strategy to obtain a second spatial pose; calculating a second welding accuracy corresponding to the weld point based on the second spatial pose, and completing the accuracy verification operation when it is confirmed that the second welding accuracy is greater than or equal to the preset accuracy.

[0105] Specifically, using the first spatial pose and its covariance approximation as input, a first welding accuracy is first formed and threshold verification is performed. Then, the welding disturbance identification model provides welding disturbance factors and corresponding weld point correction strategies. Subsequently, the pose correction target is constructed using strategy constraints, and the second spatial pose is solved. Finally, the second welding accuracy is generated, and the verification loop is completed. The first welding accuracy uses the Mahalanobis distance of the pose error as the core metric. A three-dimensional reference model is used to provide the nominal pose. The first spatial pose is Construct a six-dimensional error vector ,in , Let the pose increment covariance obtained from propagation in step S104 be... The error distance and the first welding accuracy are defined as follows:

[0106]

[0107]

[0108] in, Represents the translation deviation vector. This represents the attitude deviation vector obtained from the rotational logarithmic mapping. The covariance approximation for pose uncertainty This represents the overall deviation normalized to uncertainty. The precision scale parameter is used to map distances to precision fractions between 0 and 1. This indicates the first welding precision. If... Then it enters the disturbance identification and correction process, in which This is the preset accuracy threshold.

[0109] The welding disturbance identification model takes a feature vector composed of multi-source residuals and scene measurements as input, and outputs the welding disturbance factors and their confidence distribution. Let the feature vector be:

[0110]

[0111] in, Represents the mean of the reprojection residuals, Indicates the normal deviation angle, Indicates the percentage of saturated pixels, Indicates the variance of exposure time sequence, Indicators of fuzzy indices Indicates hand-eye extrinsic parameter offset, Indicates assembly offset indication, This represents the shape consistency residual. A set of perturbation factors is obtained using multinomial logistic regression. Posterior probability:

[0112]

[0113] in, Indicates the first The posterior probability of perturbation factors, This represents the corresponding weight vector. This represents a nonlinear mapping or normalization transformation of the input features. Based on a threshold. Alternatively, the Top-K criterion can be used to determine the set of welding disturbance factors, and a weld point correction strategy can be generated based on the strategy mapping embedded in the model.

[0114] Solder joint correction strategies are selected from the candidate pool in a score-maximizing manner, achieving an optimal balance between correction benefits and costs. Let the candidate strategy set be... The strategy score is defined as follows:

[0115]

[0116] in, Representation strategy Overall score Indicates the first The importance weight of perturbation factors Indicates an indicator function, Representation strategy For the first The utility coefficient of perturbation factors. Indicates cost weight, This represents a measure of policy latency or cost. The policy or policy combination with the highest score is used for pose correction modeling.

[0117] Pose correction uses the first spatial pose as the initial value and iteratively updates it in the direction that minimizes the target residual. Let the homogeneous transformation of the first spatial pose be... The second spatial pose is Incremental updates using Lie algebra:

[0118]

[0119] in, This indicates the six-dimensional pose increment in this correction. Let the cap operator map represent the mapping from a Lie algebra vector to a Lie group. The policy constraints are incorporated into the pose objective function in the form of quadratic priors or reweighted residuals, and the solution is obtained. :

[0120]

[0121] in, This represents the reprojection residual corresponding to the space. Indicates the normal consistency residual. Indicates shape consistency residuals. This represents the weighted matrix of the three types of residuals. This represents the corresponding weight coefficient. Indicates the first The prior weights of the strategy, This represents the linear prior terms generated by the solder joint correction strategy, such as the small extrinsic parameter offset prior given by the extrinsic parameter fine-tuning strategy, the displacement prior along the tooling reference normal given by the assembly offset field strategy, the feature reweighting prior induced by the imaging compensation strategy, and the sparse prior of the correspondence between adjacent solder joints by the adjacency disambiguation strategy. Damped least squares are used to solve the increment, and the updated prior is obtained. That is, the second spatial pose.

[0122] The second welding accuracy uses a consistent measurement method to ensure comparability between the preceding and following closed loops. Let the second spatial pose be... Construction error And with a new Gaussian approximation covariance calculate:

[0123]

[0124]

[0125] in, This indicates the corrected overall deviation. Indicates in The pose increment covariance obtained by linearization. This indicates the second welding precision. If... Then the accuracy verification operation is completed and the second spatial pose is used for subsequent control; if it does not meet the standard, it can be improved in the policy library. Alternatively, a different strategy combination can be used to solve the problem again, in order to ensure that the verification loop has a convergence path and traceability.

[0126] In one possible implementation, step S105 further includes: using historical weld point location feature sets, geometric mapping relationships, and production line environment monitoring information as training data, and constructing welding disturbance features, including feature residual distribution features, pose drift quantification index features, assembly tolerance offset features, and imaging deviation features; constructing a first correspondence between welding disturbance features and welding disturbance factors; constructing a weld point correction strategy based on welding disturbance factors through an adaptive correction generation mechanism, including an imaging parameter compensation strategy, a pose optimization compensation strategy, a trajectory constraint correction strategy, and a calibration drift compensation strategy; constructing a second correspondence between welding disturbance factors and weld point correction strategies; and constructing a welding disturbance recognition model based on the first and second correspondences.

[0127] Specifically, the training data consists of a set of historical weld point location features, geometric mapping relationships, and production line environmental monitoring information. The set of historical weld point location features provides statistical trajectories of shape features, position features, and normal features. The geometric mapping relationships provide nominal geometric constraints, prior positions, and local normal directions. The production line environmental monitoring information provides contextual measurements such as illumination, temperature rise, vibration, clamping status, and fixture configuration. Based on the above training data, welding disturbance features are constructed, and two types of correspondences are learned: the first correspondence between welding disturbance features and welding disturbance factors, and the second correspondence between welding disturbance factors and the set of weld point correction strategies S = {imaging parameter compensation strategy, pose optimization compensation strategy, trajectory constraint correction strategy, calibration drift compensation strategy}. The adaptive correction generation mechanism outputs specific strategies or combinations of strategies accordingly.

[0128] The characteristic residual distribution is defined as a statistical characterization of the positioning residual in time and space, used to characterize the stability of image reprojection consistency and contour shape consistency. It includes the mean of reprojection residuals, residual variance, long-tail proportion, quantile of elliptical contour fitting error, and spatial clustering of residual hot zones. If the long-tail residual near the bright area coincides with the local overexposed mask, it indicates a tendency for reflection residue caused by imaging deviation; if the residual rises uniformly globally and has the same period as the temperature rise of the workstation or the change in robot load, it indicates a tendency for calibration or mechanism drift.

[0129] The pose drift quantification index is defined as the offset strength and direction characteristics of the calculated pose relative to the nominal pose of the three-dimensional reference model. It includes the projection offset of the translation component along the reference A / B / C axes, the Euler angle or axis angle offset of the pose component, and the principal axis length and principal direction of the pose increment covariance. If a systematic translation along the same reference normal occurs in a single station or multiple batches, it indicates a structural offset caused by assembly or fixture. If the pose component drifts linearly with the working time and is related to the ambient temperature rise, it indicates hand-eye extrinsic parameters or thermal drift of the mechanism.

[0130] The assembly tolerance offset feature is defined as the assembly offset amount derived from the jig status, shim configuration, locating pin wear, clamping stroke statistics, and CAD alignment residuals. It quantifies the systematic difference between the nominal prior position and the current assembly configuration. This feature may include displacement along the reference plane normal, slippage along the locating pin axis, and changes in the relative spacing between adjacent solder joints within a partition. If this feature is consistently positive in a certain partition, it indicates that consistency correction is needed at the trajectory layer or prior layer.

[0131] Imaging deviation features are defined as quality metrics related to the image formation process, including saturated pixel ratio, local contrast, motion blur index, polarization consistency index, exposure timing stability, and lens occlusion confidence. High saturation ratio, poor polarization consistency, and high overlap between the highlight mask and solder joint area indicate residual reflection; increased blur index and simultaneous occurrence of kinematic acceleration peaks indicate motion blur.

[0132] An imaging parameter compensation strategy is used to mitigate perturbations dominated by imaging bias features. An example of the strategy generation is as follows: when the imaging bias feature triggers a threshold, and the feature residual distribution has a significant long tail in the highlight region, the adaptive correction generation mechanism selects to increase the polarizer annihilation angle, switch to short reference exposure, shorten the exposure time, and increase the gain cap. Simultaneously, it limits the field of view incident angle range and enables local HDR synthesis, supplemented by guided filtering restoration in the highlight region. The output includes a fine-tuned parameter set for camera exposure, gain, polarization angle, light source power, and camera pose, and the feature weights are reconfigured to reduce the matching contribution in overexposed areas.

[0133] The pose optimization compensation strategy is used to mitigate solution instability caused by feature residual distribution or pose drift quantification metrics. An example of strategy generation is as follows: When the overall reprojection residual increases without significant imaging bias, the mechanism reweights the residuals, increasing the weights of the normal consistency and shape consistency terms. Robust kernel suppression is applied to suspected outlier edge points. Damping and prior regularization are introduced at the solver layer, tightening proximity constraints and the incident angle interval. Iterative updates yield a stable solution. The output includes optimization weights, robust kernel type and scale, damping coefficients, and prior regularization strength, and returns the corrected pose increment.

[0134] The trajectory constraint correction strategy addresses issues dominated by assembly tolerance offsets. An example of the strategy generation is as follows: when the assembly tolerance offset feature shows a consistent offset along the reference plane normal and the spacing between adjacent weld points within the partition remains stable, the mechanism introduces a translation offset field at the trajectory layer to translate the entire arrival path to the corrected position, while maintaining normal alignment and minimum spacing constraints. If adjacent weld points become confused, adjacency disambiguation rules are simultaneously enabled, limiting candidate correspondences to the allowed range of the topological adjacency list. Outputs include the trajectory translation vector, proximity attitude adjustment, adjacency disambiguation whitelist, and safety buffer distance.

[0135] The calibration drift compensation strategy is used to address hand-eye or mechanism thermal drift. An example of strategy generation is as follows: When the pose drift quantification index is highly correlated with temperature rise and runtime, and multiple zones experience simultaneous rise, the mechanism introduces a small rigid body correction at the extrinsic parameter layer. A linear or piecewise calibration model based on temperature and runtime is used to compensate the extrinsic parameters from the camera to the flange or from the flange to the base. In the next round of calculation, the degrees of freedom of this extrinsic parameter are fixed or weakened. To avoid overfitting, the observation set is time-weighted, making the latest batch more sensitive to the compensation estimate. The output includes the hand-eye extrinsic parameter fine-tuning amount, the temperature rise compensation coefficient, and the time weight configuration.

[0136] The first correspondence is constructed using a combination of supervised learning and rule-based complementary methods. Labeled historical samples are used to map four types of welding disturbance features to the probability distribution of welding disturbance factors. Interpretable rules are used to calibrate boundary conditions; for example, "high saturation ratio and overlapping specular masks" are prioritized for imaging deviation, and "monotonically offset along the reference normal and jig shim change" is prioritized for assembly tolerance deviation. The second correspondence is constructed using historical validation results of strategy gains and costs as supervisory signals. It learns the optimal strategy or strategy sequence under different combinations of disturbance factors and different workstation contexts. When necessary, multi-armed gambling or small-scale reinforcement learning is introduced to select the strategy with the highest expected return without disrupting the cycle time.

[0137] The adaptive correction generation mechanism first obtains the posterior distribution of perturbation factors from the first correspondence during inference, then retrieves candidate strategies and calculates expected returns from the second correspondence, and selects a single strategy or a finite combination according to the return ranking. When the returns are close or highly complementary, they are executed in series in the order of "imaging priority - solution stability - structural consistency", and constraints are injected at each step to maintain consistent terminology and data flow: the input is the first spatial pose and its uncertainty, as well as the set of weld point positioning features of the current batch and production line environment monitoring information, and the output is the second spatial pose and the updated feature weights, extrinsic parameters or trajectory parameters, followed by the second welding accuracy verification and coordinate system transformation.

[0138] Step S106: Transform the second spatial pose to the robot coordinate system, and control the target robot to complete the projection welding visual positioning through the transformed second spatial pose.

[0139] Specifically, taking the second spatial pose as input, and combining the relationship between hand-eye calibration and robot kinematics, the pose is transformed from the camera coordinate system or vehicle coordinate system to the robot base coordinate system. This includes: first, calling the hand-eye calibration matrix completed during the production line initialization phase to map the second spatial pose to the robot flange coordinate system; then, using the robot's forward kinematics model, further transforming the pose in the flange coordinate system to the robot base coordinate system, thereby obtaining the final spatial pose of the target tool's center point in the base coordinate system.

[0140] After coordinate system transformation, a welding path conforming to process constraints is generated based on the target tool's center point pose, including normal alignment, incident angle limits, and minimum spacing requirements. This path is then sent to the robot controller. The robot drives the welding gun according to the path to complete precise alignment and crimping, ultimately achieving vision-guided projection welding. The entire process ensures that the accuracy of the second spatial pose passes closed-loop verification, and the motion trajectory in the robot coordinate system can be directly executed, thus completing the vision-guided weld point positioning and welding action.

[0141] Please refer to Figure 2 This illustration shows a schematic diagram of a robot projection welding vision positioning device for car body welding provided in an embodiment of this application. The device includes an acquisition module 21 and an output module 22, wherein...

[0142] The acquisition module 21 is used to acquire the body design information data and tooling information data of the vehicle to be welded, and construct the prior position and geometric constraints of the weld point based on the body design information data and tooling information data; construct a three-dimensional reference model based on the prior position and geometric constraints, and output the geometric mapping relationship, which includes the spatial relationship between the target robot, the target camera and the weld point; acquire the weld point image based on the target camera, and extract the weld point positioning feature set in the weld point image.

[0143] Output module 22 is used to associate the weld point positioning feature set with the geometric mapping relationship through feature constraints, and output the first spatial pose of the weld point in three-dimensional space through the three-dimensional reference model; to perform accuracy verification operation on the first spatial pose, and to correct the first spatial pose to the second spatial position based on welding disturbance factors; to convert the second spatial pose to the robot coordinate system, and to control the target robot to complete the projection welding visual positioning through the converted second spatial pose.

[0144] In one possible implementation, the acquisition module 21 is used to acquire the vehicle body design information data and tooling information data of the vehicle to be welded, and to construct the prior position and geometric constraints of the weld point based on the vehicle body design information data and tooling information data. Specifically, this includes: extracting the vehicle body structure model based on the vehicle body design information data, and marking the nominal space coordinates and local normal directions corresponding to the weld point in the vehicle body structure model; using the nominal space coordinates and local normal directions as the prior position; determining the constraints of the fixture, jig, and process reference surface based on the tooling information data, and correcting the prior position under the constraints to obtain a corrected position that meets the constraints; and combining the corrected position with the clamping direction, reference plane normal, and adjacency relationship in the tooling information data to generate geometric constraints.

[0145] In one possible implementation, the acquisition module 21 is used to construct a three-dimensional reference model based on prior positions and geometric constraints, and output geometric mapping relationships. Specifically, it includes: combining prior positions and geometric constraints through spatial fusion operations, and establishing the three-dimensional spatial distribution of weld points in the vehicle body coordinate system based on the combination results; constructing a three-dimensional reference model based on the three-dimensional spatial distribution to output geometric mapping relationships.

[0146] In one possible implementation, the acquisition module 21 is used to acquire a solder joint image based on the target camera and extract a set of solder joint positioning features from the solder joint image. Specifically, this includes: performing reflection suppression and multi-exposure fusion processing on the solder joint image to enhance the imaging quality of the area where the solder joint is located; extracting the edge contour features and local geometric features of the solder joint in the enhanced solder joint image, and performing sub-pixel refinement processing on the local geometric features; combining the edge contour features and the refined local geometric features through feature fusion operation, and outputting a set of solder joint positioning features, which includes shape features, position features, and normal features.

[0147] In one possible implementation, the output module 22 is used to perform feature constraint association between the solder joint positioning feature set and the geometric mapping relationship, specifically including: matching the shape features in the solder joint positioning feature set with the nominal geometric constraints in the geometric mapping relationship to establish a geometric correspondence of the solder joints; registering the position features in the solder joint positioning feature set with the prior position in the geometric mapping relationship to establish a spatial correspondence of the solder joints; performing consistency constraints on the normal features in the solder joint positioning feature set with the local normal direction in the geometric mapping relationship to establish an attitude correspondence of the solder joints; and forming a feature constraint association through the geometric correspondence, spatial correspondence, and attitude correspondence.

[0148] In one possible implementation, the output module 22 is used to perform an accuracy verification operation on the first spatial pose and correct the first spatial pose to a second spatial position based on welding disturbance factors. Specifically, it includes: calculating the first welding accuracy corresponding to the weld point based on the first spatial pose; if it is determined that the first welding accuracy is less than a preset accuracy, outputting welding disturbance factors through a welding disturbance recognition model; outputting a weld point correction strategy corresponding to the welding disturbance factors through a welding disturbance recognition model based on the welding disturbance factors; correcting the first spatial pose based on the weld point correction strategy to obtain a second spatial pose; calculating the second welding accuracy corresponding to the weld point based on the second spatial pose, and completing the accuracy verification operation when it is confirmed that the second welding accuracy is greater than or equal to the preset accuracy.

[0149] In one possible implementation, the output module 22 is used to construct a welding disturbance recognition model, specifically including: acquiring production line environment monitoring information, including ambient lighting parameters, body assembly deviation parameters, robot operation thermal drift parameters, and camera imaging stability parameters; using historical weld point positioning feature sets, geometric mapping relationships, and production line environment monitoring information as training data, and constructing welding disturbance features, including feature residual distribution features, pose drift quantification index features, assembly tolerance offset features, and imaging deviation features; constructing a first correspondence between welding disturbance features and welding disturbance factors; based on welding disturbance factors, constructing a weld point correction strategy through an adaptive correction generation mechanism, including an imaging parameter compensation strategy, a pose optimization compensation strategy, a trajectory constraint correction strategy, and a calibration drift compensation strategy; constructing a second correspondence between welding disturbance factors and weld point correction strategies; and constructing a welding disturbance recognition model based on the first and second correspondences.

[0150] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0151] This application also provides an electronic device. (See reference...) Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: at least one processor 301, at least one communication bus 302, a user interface 303, at least one network interface 304, and a memory 305.

[0152] The communication bus 302 is used to enable communication between these components.

[0153] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0154] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0155] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0156] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. (Refer to...) Figure 3 The memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a robot projection welding vision positioning application for vehicle body welding.

[0157] exist Figure 3In the illustrated electronic device, the user interface 303 is primarily used to provide an input interface for the user and acquire user input data; while the processor 301 can be used to call the robot projection welding vision positioning application for car body welding stored in the memory 305. When executed by one or more processors 301, the electronic device performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0158] This application also provides a computer-readable storage medium storing instructions. When executed by one or more processors, these instructions cause an electronic device to perform one or more of the methods described in the above embodiments.

[0159] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0160] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0161] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0162] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0163] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0164] The above description is merely an exemplary embodiment disclosed in this application and should not be construed as limiting the scope of this application. Any equivalent changes and modifications made in accordance with the teachings of this application shall still fall within the scope of this application.

[0165] This application is intended to cover any variations, uses, or adaptations disclosed herein that follow the general principles disclosed herein and include common knowledge or customary technical means in the art that are not described in this application.

Claims

1. A robotic projection welding vision positioning method for vehicle body welding, characterized in that, The method includes: Obtain the body design information data and tooling information data of the vehicle to be welded, and construct the prior position and geometric constraints of the weld point based on the body design information data and the tooling information data; A three-dimensional benchmark model is constructed based on the prior position and the geometric constraints, and the geometric mapping relationship is output, which includes the spatial relationship between the target robot, the target camera and the welding point; The solder joint image is acquired based on the target camera, and the set of solder joint positioning features in the solder joint image is extracted. The set of solder joint positioning features is associated with the geometric mapping relationship through feature constraints, and the first spatial pose of the solder joint in three-dimensional space is output through the three-dimensional reference model. The accuracy verification operation for the first spatial pose and the correction of the first spatial pose to a second spatial position based on welding disturbance factors specifically includes: calculating the first welding accuracy corresponding to the weld point based on the first spatial pose; if it is determined that the first welding accuracy is less than a preset accuracy, outputting the welding disturbance factor through a welding disturbance recognition model; outputting the weld point correction strategy corresponding to the welding disturbance factor through the welding disturbance recognition model based on the welding disturbance factor; correcting the first spatial pose based on the weld point correction strategy to obtain a second spatial pose; calculating the second welding accuracy corresponding to the weld point based on the second spatial pose, and completing the accuracy verification operation when it is confirmed that the second welding accuracy is greater than or equal to the preset accuracy. The second spatial pose is converted to the robot coordinate system, and the target robot is controlled to complete the projection welding visual positioning through the converted second spatial pose.

2. The method according to claim 1, characterized in that, The process of acquiring the vehicle body design information data and tooling information data of the vehicle to be welded, and constructing the prior position and geometric constraints of the weld point based on the vehicle body design information data and the tooling information data, specifically includes: Based on the vehicle body design information data, extract the vehicle body structure model, and mark the nominal space coordinates and local normal directions corresponding to the weld points in the vehicle body structure model; The nominal spatial coordinates and the local normal direction are used as the prior position; Based on the tooling information data, the constraints of the fixture, jig, and process reference surface are determined, and the prior position is corrected under the constraints to obtain a corrected position that meets the constraints. The corrected position is combined with the clamping direction, reference plane normal, and adjacency relationship in the tooling information data to generate the geometric constraint.

3. The method according to claim 1, characterized in that, The process of constructing a 3D baseline model based on the prior location and the geometric constraints, and outputting the geometric mapping relationship, specifically includes: The prior location and the geometric constraint are combined through spatial fusion operation, and the three-dimensional spatial distribution of the weld point in the vehicle body coordinate system is established based on the combination result. The three-dimensional benchmark model is constructed based on the three-dimensional spatial distribution to output the geometric mapping relationship.

4. The method according to claim 1, characterized in that, The step of acquiring solder joint images based on the target camera and extracting a set of solder joint location features from the solder joint images specifically includes: The solder joint image is subjected to reflection suppression and multi-exposure fusion processing to enhance the imaging quality of the area where the solder joint is located; The edge contour features and local geometric features of the solder joint are extracted from the enhanced solder joint image, and the local geometric features are refined to subpixel level. The edge contour features and the refined local geometric features are combined through feature fusion operations to output the solder joint positioning feature set, which includes shape features, position features and normal features.

5. The method according to claim 4, characterized in that, The step of associating the set of solder joint positioning features with the geometric mapping relationship through feature constraints specifically includes: The geometric correspondence between the solder joints is established by matching the shape features in the set of solder joint positioning features with the nominal geometric constraints in the geometric mapping relationship. Registration is performed based on the positional features in the set of solder joint positioning features and the prior positions in the geometric mapping relationship to establish the spatial correspondence of the solder joints; Consistency constraints are applied between the normal features in the solder joint positioning feature set and the local normal directions in the geometric mapping relationship to establish the orientation correspondence of the solder joints. Feature constraint associations are formed through the geometric correspondence, the spatial correspondence, and the attitude correspondence.

6. The method according to claim 1, characterized in that, The construction of the welding disturbance identification model specifically includes: Acquire production line environmental monitoring information, which includes ambient lighting parameters, vehicle assembly deviation parameters, robot operation thermal drift parameters, and camera imaging stability parameters. The historical weld point positioning feature set, the geometric mapping relationship, and the production line environment monitoring information are used as training data to construct welding disturbance features, which include feature residual distribution features, pose drift quantification index features, assembly tolerance offset features, and imaging deviation features. Construct a first correspondence between the welding disturbance characteristics and the welding disturbance factors; Based on the welding disturbance factors, the weld point correction strategy is constructed through an adaptive correction generation mechanism. The weld point correction strategy includes an imaging parameter compensation strategy, a pose optimization compensation strategy, a trajectory constraint correction strategy, and a calibration drift compensation strategy. Construct a second correspondence between the welding disturbance factors and the weld point correction strategy; The welding disturbance identification model is constructed based on the first correspondence and the second correspondence.

7. A robotic projection welding vision positioning device for vehicle body welding, characterized in that, The device includes an acquisition module and an output module, wherein, The acquisition module is used to acquire the body design information data and tooling information data of the vehicle to be welded, and construct the prior position and geometric constraints of the weld point based on the body design information data and the tooling information data; construct a three-dimensional reference model based on the prior position and the geometric constraints, and output the geometric mapping relationship, which includes the spatial relationship between the target robot, the target camera and the weld point; acquire the weld point image based on the target camera, and extract the weld point positioning feature set in the weld point image; The output module is used to associate the weld point positioning feature set with the geometric mapping relationship through feature constraints, and output the first spatial pose of the weld point in three-dimensional space through the three-dimensional reference model; to perform an accuracy verification operation on the first spatial pose, and to correct the first spatial pose to a second spatial position based on welding disturbance factors, specifically including: calculating the first welding accuracy corresponding to the weld point based on the first spatial pose; if it is determined that the first welding accuracy is less than a preset accuracy, outputting the welding disturbance factor through the welding disturbance recognition model; outputting the weld point correction strategy corresponding to the welding disturbance factor through the welding disturbance recognition model based on the welding disturbance factor; correcting the first spatial pose based on the weld point correction strategy to obtain a second spatial pose; calculating the second welding accuracy corresponding to the weld point based on the second spatial pose, and completing the accuracy verification operation when it is confirmed that the second welding accuracy is greater than or equal to the preset accuracy; converting the second spatial pose to the robot coordinate system, and controlling the target robot to complete the projection welding visual positioning through the converted second spatial pose.

8. An electronic device, characterized in that, The device includes a processor, a communication bus, a user interface, a network interface, and a memory. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1 to 6.

Images