A joint pose-deformation estimation method and system for micro-contact assembly processes

Abstract

This application discloses a pose-deformation joint estimation method and system for micro-contact assembly processes. The method involves multi-view scanning of a workpiece using a mobile device and reconstructing its global point cloud model. Next, the overall pose parameters and local surface morphology of the workpiece are simultaneously calculated to construct displacement and strain fields to quantitatively characterize the deformation state. Subsequently, the image, point cloud, pose, and deformation parameters are fused, and a pre-trained joint estimation model is used to rapidly predict the current coupling state of the workpiece. A built-in consistency verification and online update mechanism ensures the reliability of the prediction results, and the final reliable state estimate is converted into robot assembly correction instructions in real time, thus forming a closed loop of perception, estimation, and control. This invention enables sub-millimeter-level pose measurement and high-precision deformation prediction, significantly improving the dynamic adaptability, robustness, and automation level of the assembly process. It is suitable for the automated assembly of high-precision components in fields such as aerospace and aviation.

Description

Technical Field

[0001] This invention relates to the field of precision assembly and intelligent measurement technology, specifically to a method and system for joint estimation of pose and deformation for micro-contact assembly processes. Background Technology

[0002] In the precision assembly of large, thin-walled, and weakly rigid components such as aerospace panels, stringers, frame beams, and composite material shells, the workpiece undergoes a complex evolution during the clamping, positioning, fitting, and micro-contact stages, characterized by the coupling of overall pose changes (translation and rotation) and local deformations (bending, warping, and contact area undulations). Existing technologies typically treat these two aspects separately: for overall pose, methods such as laser trackers and visual markers are often used; for local deformation, strain gauges, digital image correlation (DIC), or point cloud fitting analysis are relied upon. With the development of 3D vision technology, multi-view scanning and point cloud stitching provide new approaches to acquiring the global geometric state, but a unified modeling and real-time prediction capability for the coupled state of pose and deformation is still lacking.

[0003] Currently, some studies have begun to attempt data fusion and joint modeling, but their methods often separate the perception, estimation and control links, making it difficult for the model to be updated adaptively online. This results in insufficient robustness in dynamic and uncertain assembly environments, and the prediction results cannot be directly and reliably used to guide real-time closed-loop control.

[0004] Therefore, designing an integrated solution that can achieve collaborative perception and rapid joint prediction of pose and deformation states, and can perform online correction based on real-time measurement feedback, ultimately forming a closed loop of perception-estimation-control, is an urgent technical problem to be solved. Summary of the Invention

[0005] To address the aforementioned shortcomings of existing technologies, this invention provides a pose-deformation joint estimation method and system for micro-contact assembly processes. It aims to achieve coordinated, real-time, and high-precision perception and prediction of the overall workpiece pose and local deformation states, and ensures the reliability of the estimation results through online verification and updating mechanisms. Finally, the predicted states are transformed into assembly control commands, realizing intelligent closed-loop control from perception to execution.

[0006] In a first aspect, embodiments of this application provide a pose-deformation joint estimation method for micro-contact assembly processes, comprising the following steps: S1: Perform multi-view scanning of the assembled workpiece using a mobile device to acquire multiple sets of image data covering key areas of the workpiece and corresponding local point cloud data. S2: Based on the image data, the local point cloud data, the device pose parameters and the hand-eye calibration matrix, perform three-dimensional reconstruction and point cloud stitching to construct a global point cloud model covering the workpiece surface; S3: Register the global point cloud model with the reference model of the workpiece, solve the overall pose parameters of the workpiece, and calculate the normal deviation of each point on the workpiece surface relative to the reference model to obtain the local topography information of the workpiece. S4: Based on the overall pose parameters and the local topography information, construct the displacement field and strain field of the workpiece, and calculate the current deformation parameters characterizing the local deformation state. S5: Input the image data, the global point cloud model, the overall pose parameters and the current deformation parameters into the pre-trained pose-deformation joint estimation model, and output the joint estimation result of the workpiece's current pose state and deformation state. S6: Construct a consistency verification index to verify the joint estimation result and the geometric measurement result; if the verification fails, perform an online update on the joint estimation model until the verification passes; the geometric measurement result includes the local morphology information and the current deformation parameter. S7: The verified joint estimation results are converted into robot assembly correction instructions to achieve closed-loop control of micro-contact assembly.

[0007] Preferably, the step of performing 3D reconstruction and point cloud stitching to construct a global point cloud model covering the workpiece surface includes: The pixels from each viewpoint are back-projected onto the camera coordinate system to obtain the local 3D point coordinates. Based on the device pose parameters, the hand-eye calibration matrix, and the world coordinate system transformation matrix, the local point cloud data from each viewpoint is transformed to the world coordinate system to obtain the point cloud coordinates in the world coordinate system under the corresponding viewpoint. The point clouds from various perspectives are registered in the world coordinate system. The objective function is constructed and optimized by minimizing the distance between corresponding points in the overlapping area to form the global point cloud model.

[0008] Preferably, in step S3, solving the overall pose parameters of the workpiece includes: The optimal rotation matrix and translation vector for rigidly aligning the reference model point set with the global point cloud model point set are obtained by using singular value decomposition. The rotation matrix is ​​decomposed into Euler angles about the X, Y, and Z axes; The translation vector and Euler angles constitute the overall pose parameters.

[0009] Preferably, in step S4, constructing the displacement field and strain field of the workpiece includes: Based on the reference model points and the corresponding points of the current global point cloud model, the displacement vector is calculated and a displacement field is formed. Based on the small deformation assumption, the Cauchy strain tensor is calculated by the displacement field gradient; The maximum principal strain is extracted from the strain tensor as the current deformation parameter.

[0010] Preferably, in step S5, the pose-deformation joint estimation model is a machine learning model built on a multi-branch neural network, which minimizes the joint loss function. Conduct training: (1); in, For pose prediction error loss, For deformation prediction error loss, The loss is based on the geometric consistency between the global point cloud model and the reference model. For regularization loss, , , , These are the weighting coefficients for each loss term.

[0011] Preferably, in step S6, performing online updates to the joint estimation model includes: S61: Concatenate the local topography information and the current deformation parameters at the current moment into a target state vector; S62: The original input features at the current time step are weighted and fused with the hidden layer features of the model at the previous stable time step to form the updated input features; S63: Calculate the error between the joint estimation result of the current output of the model and the target state vector, and map the error to the hidden layer of the model to obtain the hidden layer feature correction amount; S64: The original hidden layer features of the model are superimposed with the hidden layer feature correction amount to obtain the updated fused features, and the model parameters are updated based on this. S65: Re-execute step S5 using the updated model parameters and output the updated joint estimation results.

[0012] Preferably, in step S7, converting the verified joint estimation result into a robot assembly correction instruction specifically involves: The joint estimation results of the pose state are converted into pose corrections for the robot end effector. The joint estimation results of the pose state and the joint estimation results of the deformation state are combined through a preset mapping function to generate control commands for adjusting the assembly trajectory, contact force or assembly sequence.

[0013] Secondly, embodiments of this application provide a pose-deformation joint estimation system for micro-contact assembly processes designed based on the pose-deformation joint estimation method for micro-contact assembly processes described in the first aspect, comprising: The data acquisition module is used to control the mobile device to perform multi-view scanning and acquire multiple sets of image data covering key areas of the workpiece and corresponding local point cloud data. The 3D reconstruction and point cloud processing module is used to perform step S2 and construct the global point cloud model. The geometric state measurement module is used to execute steps S3 and S4 to obtain the overall pose parameters, the local topography information, and the current deformation parameters. The joint estimation and prediction module, which incorporates the pose-deformation joint estimation model, is used to execute step S5 and output the joint estimation result. The verification and model update module is used to perform step S6, verify the joint estimation results and update the joint estimation model online; The assembly control instruction generation module is used to execute step S7 and generate the robot assembly correction instruction.

[0014] The pose-deformation joint estimation method for micro-contact assembly processes provided by this invention, through the construction of an integrated technical system of multi-view collaborative perception, joint modeling and prediction, online verification and update, and closed-loop control execution, produces significant beneficial effects compared to existing separate measurement and open-loop control methods, as detailed below: (1) Measurement and prediction accuracy have been improved by leaps and bounds, fundamentally ensuring assembly quality. This invention establishes a unified geometric benchmark through precise global point cloud stitching, eliminating the inconsistency problem of multi-sensor data at its source. Based on this, the innovative pose-deformation joint estimation model does not simply treat two tasks in parallel, but deeply integrates images, point clouds, and geometric parameters at the feature level, forcing the model to learn the physical coupling relationship between them through a joint loss function. Examples show that this method stably controls pose measurement errors within ±0.05 mm, and the prediction accuracy of key deformation parameters reaches ±0.01%. This collaborative high-precision perception enables the system to keenly capture subtle deformations caused by microscopic contact forces and their reaction to the overall pose, thereby reducing assembly docking deviations by an average of over 30%, providing a reliable quantitative basis for high-precision assembly of thin-walled aerospace components, etc. (2) The system's real-time performance and dynamic robustness are greatly enhanced, enabling it to adapt to complex working conditions. The online verification and update mechanism designed in this invention forms the core of the system. Consistency verification indicators monitor the reliability of the model output in real time; once the deviation exceeds the limit, the system can trigger an update process in less than 1 second, quickly adjusting the model through lightweight operations such as feature fusion and hidden layer correction, rather than retraining. The forward prediction time of the joint estimation model itself is less than 0.3 seconds, and the entire process cycle is less than 3 seconds, achieving sub-second response, meeting the stringent real-time requirements of dynamic assembly processes, and significantly improving stability and reliability under uncertain environments. (3) Forming intelligent closed-loop control, significantly improving the level of assembly automation and overall efficiency. By employing a deterministic mapping function from pose-deformation state to control commands, reliable joint estimation results are directly transformed into pose corrections and force / trajectory control parameters for the robot's end effector. This enables online dynamic optimization of the assembly process, allowing the robot to compensate for pose deviations caused by workpiece deformation in real time. Simultaneously, precise control avoids over-assembly and under-assembly, reducing damage to workpieces and tools and improving resource utilization by 25%.

[0015] (4) It has outstanding engineering practicality and economic benefits, and has broad application prospects. This approach integrates advanced algorithms with low-cost sensors, using mobile robots equipped with general-purpose 3D vision cameras to replace some high-cost dedicated measurement equipment, thus reducing the overall system's hardware dependence and deployment costs. The proposed framework is highly modular and versatile, adaptable to different robot platforms and workpiece types, and easily upgraded in existing production lines. It provides an effective solution to the long-standing challenge of automated assembly of large, weakly rigid structural components in the aerospace industry, not only improving product consistency and quality but also enabling continuous optimization of assembly processes and the construction of digital twins, demonstrating significant long-term engineering value and widespread application potential. Attached Figure Description

[0016] Exemplary embodiments of the present invention can be more fully understood by referring to the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain the present invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 A flowchart of a pose-deformation joint estimation method for micro-contact assembly process provided as an exemplary embodiment of this application; Figure 2 A schematic diagram of the structure of a pose-deformation joint estimation system for micro-contact assembly process provided as another exemplary embodiment of this application. Detailed Implementation

[0018] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0019] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0020] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0021] Example 1 Reference Figure 1 This embodiment discloses a pose-deformation joint estimation method for micro-contact assembly processes, including the following steps: S1: Perform multi-view scanning of the assembled workpiece using a mobile device to acquire multiple sets of image data covering key areas of the workpiece and corresponding local point cloud data. Specifically, the assembly process of a thin-walled aerospace panel and its support frame was selected as the experimental object. An experimental system consisting of one mobile robot, two sets of 3D vision cameras, and one industrial control computing platform was built. The robot was positioned with multiple observation points along the length of the panel to acquire multi-view images and local point clouds of the workpiece. The 3D vision cameras had a resolution of 2448×2048 pixels, a depth measurement accuracy of ±0.03 mm, and a robot repeatability accuracy of ±0.05 mm. The transformation matrix between the camera and the robot's end effector was obtained through hand-eye calibration, with the calibration error controlled within ±0.04 mm. The experiment was conducted in a simulated assembly environment, with applied contact forces of 50-100 N, covering typical deformation scenarios such as panel warping and local undulations.

[0022] Specifically, in the actual assembly site, the mobile robot scans the surface of the panel from 5-8 viewpoints, collecting multiple sets of image data covering the key areas of the workpiece and local point cloud data of about 100,000 points / viewpoint.

[0023] S2: Based on image data, local point cloud data, device pose parameters and hand-eye calibration matrix, perform 3D reconstruction and point cloud stitching to construct a global point cloud model covering the workpiece surface; Specifically, for each pixel in each viewpoint, combining depth information and the camera intrinsic parameter matrix, it is back-projected into the camera coordinate system to obtain the local 3D point coordinates, the expression of which is: (2); in, Let i represent a 3D point in the camera coordinate system from the i-th viewpoint; This represents the depth value corresponding to that point; K represents the camera intrinsic parameter matrix, and the superscript -1 indicates the inversion operation; Represents the homogeneous coordinates of a pixel, with superscript. This indicates the transpose operation.

[0024] Subsequently, the robot's pose parameters and camera intrinsic parameter matrix (focal length) were combined. =2450 pixels, =2450 pixels) and a hand-eye calibration matrix are used to uniformly transform the local point clouds of each viewpoint to the world coordinate system. The coordinate transformation relationship is as follows: (3); in, This represents the point cloud coordinates in the world coordinate system from the i-th viewpoint; This represents the homogeneous transformation matrix from the robot's base coordinate system to the world coordinate system; This represents the homogeneous transformation matrix from the robot base coordinate system to the end effector coordinate system at time i; This represents the hand-eye calibration transformation matrix from the end effector coordinate system to the camera coordinate system; This represents a local point cloud in the camera coordinate system.

[0025] After coordinate unification, coarse and fine registration are performed on the point clouds from various viewpoints to form a global point cloud model of the workpiece, providing a geometric basis for subsequent pose, topography, and deformation measurements. Fine registration is achieved by minimizing the distance between corresponding points in the overlapping area, with the objective function being: (4); in, This represents the operation of finding the minimum value; Let represent the rotation matrix of the point cloud at the i-th viewpoint; Represents the translation vector of the point cloud at the i-th viewpoint; Indicates the pair from the 1st to the 2nd. Accumulate the errors at corresponding points; This represents the k-th point in the source point cloud; This represents the k-th corresponding point in the target point cloud; This represents the squared L2 norm. This step outputs a global point cloud model, which is directly used as input for the next pose measurement step to ensure geometric consistency.

[0026] Specifically, coarse registration uses the SAC-IA algorithm (Sample Consistency Initial Registration), and fine registration uses the ICP algorithm (Iterative Closest Point), with the objective function minimizing the error threshold set to 0.05 mm. The output global point cloud model contains approximately 1.1 million points, covering 90% of the panel surface. Experiments have shown that it constructs a complete geometric model within 1-2 seconds. Compared to traditional single-view scanning, the point cloud stitching accuracy is improved by 20%, providing a stable data foundation for subsequent steps, avoiding assembly errors caused by local blind spots, and reducing the overall geometric deviation from 0.8 mm to 0.2 mm, significantly improving measurement efficiency and reliability.

[0027] S3: Register the global point cloud model with the reference model of the workpiece, solve the overall pose parameters of the workpiece, and calculate the normal deviation of each point on the workpiece surface relative to the reference model to obtain the local topography information of the workpiece. Specifically, after obtaining the global point cloud model of the workpiece, it is rigidly aligned with the reference model to measure the overall pose change of the workpiece in its current state. The rigid body transformation relationship between the reference model and the current measured model is expressed as: (5); in, Represents the coordinates of a measured point in the world coordinate system; Represents the coordinates of a point in the reference model; Represents the rotation matrix; This represents the translation vector.

[0028] For ease of engineering representation, the rotation matrix is ​​further decomposed into Euler angle form: (6); in, Represents the total rotation matrix; , , These represent matrices that rotate about the x, y, and z axes, respectively. , , These represent roll angle, pitch angle, and yaw angle, respectively.

[0029] After solving the overall pose, the normal deviation of the measured point relative to the reference surface is calculated to obtain the local surface morphology changes of the workpiece. Finally, both the overall pose information and the local morphology deviation information of the workpiece are obtained simultaneously. The specific expression is: (7); in, Indicates the normal deviation of the measured point; Indicates the current measured point; Indicates the corresponding point on the reference surface; · represents the unit normal vector of the reference surface at the corresponding point; · represents the vector dot product operation.

[0030] Based on the above process, in actual operation, the global point cloud is rigidly aligned with the CAD reference model (mesh resolution 0.5 mm), and the rotation matrix R and translation vector t are solved using SVD (Singular Value Decomposition). After decomposing the Euler angles, the pose parameters are obtained: x, y, and z translations are 0.62 mm, -0.41 mm, and 0.35 mm, respectively; roll, pitch, and yaw angles are 0.09°, -0.16°, and 0.21°, respectively. The normal deviation is further calculated using a kd-tree nearest neighbor search with a deviation threshold of 0.1 mm. The maximum normal deviation is 0.74 mm, and the average deviation is 0.18 mm.

[0031] S4: Based on the overall pose parameters and the local topography information, construct the displacement field and strain field of the workpiece, and calculate the current deformation parameters that characterize the local deformation state. Based on pose decoupling and topography measurement, the displacement and strain fields of the workpiece are further established to quantitatively characterize the local deformation state. First, according to the correspondence between the reference state and the current state, the displacement vector at each point is calculated, and its expression is: (8); in, Indicates the location The displacement vector at that point; Indicates the position of the point in the current state; This indicates the position of the corresponding point under the reference state.

[0032] Based on the displacement field, the strain field is further calculated. Under small deformation conditions, the strain tensor is expressed as: (9); in, Represents the strain tensor; Represents the gradient operator; Represents the displacement field ; This indicates transpose.

[0033] If unfolded in two-dimensional surface coordinates, the strain components can be expressed as: (10); in, express Positive strain direction; express Positive strain direction; Indicates shear strain; and These represent the displacement vectors at... and Components in direction.

[0034] Furthermore, to characterize the most significant deformation in a local region, the maximum principal strain is calculated: (11); in, This represents the maximum principal strain. This step outputs deformation parameters (such as displacement field and maximum principal strain) as inputs for the next step of the joint model.

[0035] Specifically, a displacement field was established using the point cloud after pose decoupling: the sampling grid step size was 0.5 mm, and the displacement vector u was calculated through local interpolation, with a maximum displacement of 0.93 mm. Then, the strain field was calculated, and the gradient was obtained using the finite difference method to obtain the principal strain distribution, with a maximum principal strain of 0.047%. In the experiment, the small deformation assumption (strain <5%) was considered, and the consistency with strain gauge measurements was verified (error <0.01%). The output deformation parameters include displacement field diagrams and thermograms.

[0036] S5: Input the image data, global point cloud model, overall pose parameters and the current deformation parameters into the pre-trained pose-deformation joint estimation model, and output the joint estimation result of the current pose state and deformation state of the workpiece. After obtaining image data, global point cloud, pose parameters and deformation parameters, a pose-deformation joint estimation model is constructed to predict the pose-deformation coupling state of the workpiece at the current moment in real time.

[0037] Let the joint estimation model be Then the input-output relationship of the model can be expressed as: (12); in, The parameter is The joint estimation model; Represents a collection of images from multiple perspectives; Represents the global point cloud or its feature representation; This represents geometric prior information, including the reference model, normal vectors, and pose and deformation parameters obtained in the preceding steps; This represents the pose state prediction result output by the model; This represents the deformation state prediction result output by the model; This represents the predicted value.

[0038] To enable the model to simultaneously achieve accuracy in pose estimation, deformation estimation, and geometric consistency, a joint loss function is constructed: (1); in, Represents the total loss function; , , , These represent the weighting coefficients of each loss term; Indicates pose error loss; Indicates deformation error loss; This represents the geometric consistency loss; This represents the regularization term.

[0039] The pose error loss can be expressed as: (13); in, and These represent the predicted translation vector and the measured translation vector, respectively. and These represent the predicted attitude angle vector and the measured attitude angle vector, respectively. This represents the weight coefficient of the attitude term.

[0040] The deformation error loss can be expressed as: (14); in, Indicates the total number of sampling points; and Let these represent the predicted displacement and the measured displacement of the m-th point, respectively. and These represent the predicted maximum principal strain and the measured maximum principal strain at the m-th point, respectively; This represents the weighting coefficient of the principal strain error term. This step ultimately establishes a unified pose-deformation joint estimation model and outputs the pose-deformation joint prediction result at the current moment. The prediction result output in this step is directly input into the next verification step, ensuring a logical closure from modeling to verification.

[0041] Specifically, the aforementioned data (images, point clouds, pose parameters, and deformation parameters) were integrated and input into a neural network model (a 3-branch structure: CNN image branch, PointNet point cloud branch, and MLP parameter branch). The training dataset consisted of 6200 sets of data, a batch size of 16, a learning rate of 0.0005, and 180 iterations. The joint loss function weights were λ1=0.4, λ2=0.4, λ3=0.1, and λ4=0.1. The output prediction results were: pose error <0.05 mm and principal strain error <0.01%. Experimental results show that the model predicts the pose-deformation state within 0.3 s, achieving a fast response to dynamic changes in the assembly (prediction accuracy is 20% higher than pure measurement), reducing the real-time computational burden. The experiment successfully predicted the deformation evolution under varying contact loads, improving the system's adaptability to uncertainties.

[0042] S6: Construct a consistency verification index to verify the joint estimation result and the geometric measurement result; if the verification fails, perform an online update on the joint estimation model until the verification passes; the geometric measurement result includes the local morphology information and the current deformation parameter. After obtaining the joint estimation results, the consistency between the model output and the geometric measurement results is verified. The verification index is defined as follows: (15); in, Indicates the indicators for result verification; and These represent the weighting coefficients for the pose and deformation terms, respectively. and These represent the pose results from joint estimation and geometric measurement, respectively. and These represent the deformation results from joint estimation and geometric measurement, respectively. If... Then output the current pose-deformation state result; if If so, return to the joint estimation model step and update the model online.

[0043] Specifically, the geometric measurement results are first constructed as the correction target for the current moment: (16); in, This represents the target state vector at the current moment; and These represent the geometric deformation result and the geometric pose result at the current moment, respectively; [.] indicates that the pose state and deformation state are concatenated into a unified vector.

[0044] Then, the current input features are fused with the steady-state features from the previous time step to form the updated input features: (17); in, This represents the updated input features at time t; Represents the original input features at time t; This represents the characteristics of the steady state at time t-1; This represents the feature fusion coefficient.

[0045] Then, the state correction between the current output and the target is calculated: (18); in, This represents the state correction at time t.

[0046] This correction is then mapped onto the hidden layer feature space to obtain the hidden feature correction term: (19); in, This represents the hidden layer feature correction term; Represents the modified mapping matrix; This indicates the bias term.

[0047] The original hidden features are superimposed with the correction terms to obtain the updated fused features: (20); in, This represents the updated hidden layer fusion features; Represents the original hidden layer features; This indicates the correction step size coefficient.

[0048] Based on this, the model parameters are updated using gradients: (twenty one); in, This represents the model parameters before the update; This represents the updated model parameters; Indicates the learning rate; This represents the gradient of the loss function with respect to the model parameters.

[0049] To prevent excessive parameter fluctuations, a smoothing constraint term is added to the loss update: (twenty two); in, This represents the total loss during the update phase; Indicates the smoothing coefficient of the parameter; This indicates a constraint term for the range of parameter changes.

[0050] After the update is complete, the joint estimation model is re-executed to obtain the updated pose-deformation state results: (twenty three); in, This represents the updated pose estimation result; This represents the updated deformation estimation result. The verification index is then recalculated: (twenty four); when If the condition is met, the updated pose-deformation state result is output; otherwise, the model update process continues until the threshold requirement is met. This step outputs the verified reliable result, which is directly input into the next assembly correction step to ensure a logical closed loop from verification to control.

[0051] Specifically, the verification index δ (weight) is calculated. =0.6、 =0.4, threshold =0.15). Under normal operating conditions, δ=0.12, direct output is performed; under abnormal operating conditions, δ=0.23, an update is performed: fusion coefficient η=0.6, correction step size μ=0.4, learning rate α=0.0002, and after 3 gradient updates, δ decreases to 0.11. Real measurement data is used as the target, and the update time is <0.5 s. Experimental results show that adaptive model adjustment improves robustness by 25% when operating conditions change abruptly (load increases by 20%). The experiment avoided 5 potential prediction errors and achieved continuous and stable output, providing highly reliable data for assembly control.

[0052] S7: The verified joint estimation results are converted into robot assembly correction instructions to achieve closed-loop control of micro-contact assembly.

[0053] After obtaining the final pose-deformation joint estimation result, it is further converted into assembly correction instructions. Let the robot end-effector pose correction amount be expressed as: (25); in, This indicates a robot end-effector pose correction command; Indicates the end-point translation correction amount; Indicates the end-effector attitude correction amount; This indicates the transpose operation.

[0054] Furthermore, the pose-deformation joint estimation results are mapped to assembly control commands: (26); in, Indicates assembly control commands; This represents the mapping function from the pose-deformation joint estimation result to the control quantity; Indicates the estimated pose state; This indicates the estimated deformation state. The control commands can be fed back to the robot control system to correct the end effector pose, adjust the assembly trajectory, optimize the contact posture, and control the loading process, thereby guiding the dynamic adjustment of subsequent assembly processes. This step completes closed-loop control and can cyclically return to data acquisition for continuous optimization.

[0055] Specifically, based on the final prediction results, correction commands are generated: end-effector translation correction Δt = [0.58 mm, -0.37 mm, 0.31 mm], attitude correction Δα = [0.08°, -0.14°, 0.19°]. The mapping function g is linearly transformed and fed back to the robot controller to adjust the trajectory (speed 0.1 m / s, force control threshold 50 N). The closed-loop feedback cycle interval is 1 s. In the experiment, the assembly deviation decreased from 1.2 mm to 0.3 mm. The overall experiment shows that this method achieves high precision (error <0.1 mm), high real-time performance (total process <3 s), and strong robustness in micro-contact assembly, with an assembly success rate of 98%, demonstrating significant beneficial effects.

[0056] The pose-deformation joint estimation method proposed in this application for micro-contact assembly processes achieves the organic unity of collaborative perception, joint prediction, and online closed-loop correction of the overall pose and local deformation state of the workpiece. This improves the accuracy, dynamic robustness, and intelligent control level of state monitoring for large, thin-walled, and weakly rigid structural components during precision assembly, providing reliable technical support for high-precision automated assembly. In the future, its universality and deployment convenience in complex and variable industrial environments can be further enhanced by introducing a lighter model architecture, multi-physics coupled prior knowledge, and a more efficient online adaptive mechanism.

[0057] Example 2 Based on the same inventive concept, this application also provides a pose-deformation joint estimation system for micro-contact assembly processes, used to execute the method described in the above embodiments. The solution provided by this system is similar to the implementation scheme described in the above method; therefore, the specific limitations in the embodiments of the pose-deformation joint estimation system for micro-contact assembly processes provided below can be found in the limitations of the pose-deformation joint estimation method for micro-contact assembly processes described above, and will not be repeated here.

[0058] Reference Figure 2 This application discloses a pose-deformation joint estimation system 20 for micro-contact assembly processes, comprising: The data acquisition module 201 is used to control the mobile device to perform multi-view scanning and acquire multiple sets of image data covering key areas of the workpiece and corresponding local point cloud data. The 3D reconstruction and point cloud processing module 202 is used to perform step S2 and construct the global point cloud model. The geometric state measurement module 203 is used to execute steps S3 and S4 to obtain the overall pose parameters, the local topography information and the current deformation parameters; The joint estimation and prediction module 204, which incorporates the pose-deformation joint estimation model, is used to execute step S5 and output the joint estimation result. The verification and model update module 205 is used to perform step S6, verify the joint estimation results and update the joint estimation model online; The assembly control instruction generation module 206 is used to execute step S7 and generate the robot assembly correction instruction.

[0059] The pose-deformation joint estimation system proposed in this application for micro-contact assembly processes achieves the organic unity of collaborative perception, joint prediction, and online closed-loop correction of the overall workpiece pose and local deformation state. This improves the accuracy, dynamic robustness, and intelligent control level of state monitoring for large, thin-walled, and weakly rigid structural components during precision assembly, providing reliable technical support for high-precision automated assembly. In the future, by introducing a lighter model architecture, multi-physics coupled prior knowledge, and a more efficient online adaptive mechanism, its universality and deployment convenience in complex and variable industrial environments can be further enhanced.

[0060] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0061] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0062] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0063] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0064] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application.

Claims

1. A pose-deformation joint estimation method for micro-contact assembly process, characterized in that, Includes the following steps: S1: Perform multi-view scanning of the assembled workpiece using a mobile device to acquire multiple sets of image data covering key areas of the workpiece and corresponding local point cloud data. S2: Based on the image data, the local point cloud data, the device pose parameters and the hand-eye calibration matrix, perform three-dimensional reconstruction and point cloud stitching to construct a global point cloud model covering the workpiece surface; S3: Register the global point cloud model with the reference model of the workpiece, solve the overall pose parameters of the workpiece, and calculate the normal deviation of each point on the workpiece surface relative to the reference model to obtain the local topography information of the workpiece. S4: Based on the overall pose parameters and the local topography information, construct the displacement field and strain field of the workpiece, and calculate the current deformation parameters characterizing the local deformation state. S5: Input the image data, the global point cloud model, the overall pose parameters and the current deformation parameters into the pre-trained pose-deformation joint estimation model, and output the joint estimation result of the workpiece's current pose state and deformation state. S6: Construct a consistency verification index to verify the joint estimation result and the geometric measurement result; if the verification fails, perform an online update on the joint estimation model until the verification passes; the geometric measurement result includes the local morphology information and the current deformation parameter. S7: The verified joint estimation results are converted into robot assembly correction instructions to achieve closed-loop control of micro-contact assembly.

2. The method according to claim 1, characterized in that, The process of performing 3D reconstruction and point cloud stitching to construct a global point cloud model covering the workpiece surface includes: The pixels from each viewpoint are back-projected onto the camera coordinate system to obtain the local 3D point coordinates. Based on the device pose parameters, the hand-eye calibration matrix, and the world coordinate system transformation matrix, the local point cloud data from each viewpoint is transformed to the world coordinate system to obtain the point cloud coordinates in the world coordinate system under the corresponding viewpoint. The point clouds from various perspectives are registered in the world coordinate system. The objective function is constructed and optimized by minimizing the distance between corresponding points in the overlapping area to form the global point cloud model.

3. The method according to claim 1, characterized in that, In step S3, solving the overall pose parameters of the workpiece includes: The optimal rotation matrix and translation vector for rigidly aligning the reference model point set with the global point cloud model point set are obtained by using singular value decomposition. The rotation matrix is ​​decomposed into Euler angles about the X, Y, and Z axes; The translation vector and Euler angles constitute the overall pose parameters.

4. The method according to claim 1, characterized in that, In step S4, constructing the displacement field and strain field of the workpiece includes: Based on the reference model points and the corresponding points of the current global point cloud model, the displacement vector is calculated and a displacement field is formed. Based on the small deformation assumption, the Cauchy strain tensor is calculated by the displacement field gradient; The maximum principal strain is extracted from the strain tensor as the current deformation parameter.

5. The method according to claim 1, characterized in that, In step S5, the pose-deformation joint estimation model is a machine learning model built on a multi-branch neural network, which minimizes the joint loss function. Conduct training: （1）； in, For pose prediction error loss, For deformation prediction error loss, The loss is based on the geometric consistency between the global point cloud model and the reference model. For regularization loss, , , , These are the weighting coefficients for each loss term.

6. The method according to claim 1 or 5, characterized in that, In step S6, performing online updates to the joint estimation model includes: S61: Concatenate the local topography information and the current deformation parameters at the current moment into a target state vector; S62: The original input features at the current time step are weighted and fused with the hidden layer features of the model at the previous stable time step to form the updated input features; S63: Calculate the error between the joint estimation result of the current output of the model and the target state vector, and map the error to the hidden layer of the model to obtain the hidden layer feature correction amount; S64: The original hidden layer features of the model are superimposed with the hidden layer feature correction amount to obtain the updated fused features, and the model parameters are updated based on this. S65: Re-execute step S5 using the updated model parameters and output the updated joint estimation results.

7. The method according to claim 1, characterized in that, In step S7, the conversion of the verified joint estimation result into robot assembly correction instructions specifically involves: The joint estimation results of the pose state are converted into pose corrections for the robot end effector. The joint estimation results of the pose state and the joint estimation results of the deformation state are combined through a preset mapping function to generate control commands for adjusting the assembly trajectory, contact force or assembly sequence.

8. A pose-deformation joint estimation system for implementing the method of any one of claims 1-7 in a micro-contact assembly process, characterized in that, include: The data acquisition module is used to control the mobile device to perform multi-view scanning and acquire multiple sets of image data covering key areas of the workpiece and corresponding local point cloud data. The 3D reconstruction and point cloud processing module is used to perform step S2 and construct the global point cloud model. The geometric state measurement module is used to execute steps S3 and S4 to obtain the overall pose parameters, the local topography information, and the current deformation parameters. The joint estimation and prediction module, which incorporates the pose-deformation joint estimation model, is used to execute step S5 and output the joint estimation result. The verification and model update module is used to perform step S6, verify the joint estimation results and update the joint estimation model online; The assembly control instruction generation module is used to execute step S7 and generate the robot assembly correction instruction.

Images