Automatic gluing method and system based on upper limb action teleoperation mechanical arm

Abstract

The invention relates to the technical field of teleoperation robot design, and discloses an automatic gluing method and system based on an upper limb action teleoperation mechanical arm, and the method comprises the steps: collecting and processing upper limb action data of a human body, and obtaining the position and posture information of all bones of the upper limb; on the basis of the position and posture information of all the bones of the upper limb, real-time angles of all joints of a slave end mechanical arm are obtained through motion mapping calculation, and the motion mapping relation between the tail end of the human hand and the mechanical arm is established; a mechanical arm is controlled to conduct multi-angle scanning on a workpiece, and point cloud data are collected and preprocessed; 3D point cloud reconstruction is carried out, and a point cloud model of the target workpiece is extracted from the environment; performing edge extraction on the point cloud model of the target workpiece, and generating a gluing path according to an edge extraction result; and controlling a mechanical arm to execute automatic gluing based on the gluing path, and detecting the thickness uniformity of the adhesive tape to complete the automatic gluing operation. The gluing operation quality and the production flexibility are synchronously improved.

Description

Technical Field

[0001] This invention relates to the field of teleoperated robot design technology, specifically to an automatic glue application method and system for a teleoperated robotic arm based on upper limb movements. Background Technology

[0002] To meet the needs of social production, workers in some industries often need to work in harsh and high-intensity environments. To ensure worker safety and improve work efficiency, there is an urgent need for robotic equipment that can replace human labor in dangerous environments. As a mainstream automation device in the robotics field, the robotic arm integrates core technologies from multiple fields, including computer technology, mechanical structure design, sensor technology, and automatic control. It possesses advantages such as high efficiency, flexibility, and reliability, and can provide isolated remote interaction, ensuring safety when interacting with the surrounding dangerous environment. It can even complete tasks that are difficult for humans to perform, thus becoming a core tool for liberating humans from high-risk work environments.

[0003] In terms of control methods, teleoperation offers greater flexibility and can adapt to special, complex, and rapidly changing work scenarios. By transmitting commands and actions from the operator at the master end to the remote robotic arm and driving it to perform actions, it achieves precise operation and control of the remote environment. Under the remote control of professional technicians, it can perfectly replace manual labor in completing high-risk work tasks and has significant application value in high-risk work scenarios.

[0004] Glue application is a crucial step in industrial production. However, most glues used in gluing are mixtures of volatile organic solvents and isocyanates, which are released into the air during the application process, posing serious health risks to workers. Therefore, industrial robotic arms have been widely adopted in gluing, assembly, and other manufacturing processes. However, in actual production scenarios, the technological limitations of existing industrial robotic arms are becoming increasingly apparent, specifically in the following three aspects: First, most existing industrial robotic arms use a "manual teaching" mode or offline programming for control. These control methods suffer from cumbersome operation procedures and insufficient gluing positioning accuracy, making it difficult to meet the demands of high-precision gluing operations. Second, the gluing process relies excessively on mechanical positioning. Specialized mechanical positioning devices are required to ensure the material's position is fixed during the operation, which not only increases the structural complexity of the entire production line but also significantly raises the design and manufacturing costs. Third, when products are updated, existing control programs and tooling adapted to older products cannot be directly reused. Often, redesigning the program and modifying the tooling for the new product is necessary, leading to high production line upgrade costs and severely impacting production efficiency and product iteration speed.

[0005] To address some of the aforementioned issues, existing technologies propose a gluing-related technical solution based on upper limb teleoperation. The core idea is as follows: A robotic arm is controlled to move to multiple desired postures via upper limb teleoperation. A depth camera mounted at the end of the robotic arm captures point cloud data of the target workpiece from different perspectives. Then, combining the robotic arm pose parameters corresponding to each individual point cloud data, all point cloud data are fused into a unified coordinate system through coordinate transformation. A clustering segmentation algorithm is used to extract the target workpiece from the environment, obtaining a complete surface model of the target workpiece. Based on this surface model, an edge-strength-based glue seam detection algorithm is used to locate the path points of the gluing robot. A smooth and continuous gluing trajectory is then generated through fitting and interpolation. Finally, the robot executes different gluing paths in segments according to human commands to complete the gluing task.

[0006] Despite the improvements made in existing technologies, the requirements for programming ease of use, teaching difficulty, operational efficiency, and adaptability to complex scenarios in industrial production continue to increase. Existing solutions still have room for further optimization. Therefore, developing a coating method that can effectively reduce programming and teaching difficulty, significantly improve coating efficiency, and adapt to complex workpieces and diverse unstructured industrial scenarios has become an urgent technical problem to be solved in this field. Summary of the Invention

[0007] In order to overcome the shortcomings of the existing technology, the present invention aims to provide an automatic glue application method and system based on upper limb motion teleoperation robotic arm, so as to solve the technical problems of existing industrial robotic arms in glue application operations, such as cumbersome operation, reliance on workpiece CAD models, strong positioning dependence, and insufficient environmental adaptability.

[0008] This invention is achieved through the following technical solution: In a first aspect, the present invention provides an automated adhesive application method for a teleoperated robotic arm based on upper limb movements, comprising: Collect and process human upper limb movement data to obtain the position and posture information of all bones in the upper limb, providing data input for the teleoperation of the robotic arm; Based on the position and posture information of all bones in the upper limb, the real-time angles of each joint of the end-effector robotic arm are calculated through motion mapping, and the motion mapping relationship between the end of the human hand and the robotic arm is established. Based on the motion mapping relationship, the robotic arm is controlled to perform multi-angle scanning of the workpiece, and point cloud data is collected and preprocessed. 3D point cloud reconstruction is performed based on preprocessed point cloud data to extract the point cloud model of the target workpiece from the environment. Edge extraction is performed on the point cloud model of the target workpiece, and an adhesive application path is generated based on the edge extraction results; The robotic arm is controlled based on the glue application path to perform automatic glue application and detect the uniformity of the glue strip thickness, thus completing the automatic glue application operation.

[0009] Preferably, human upper limb motion data is acquired through two IMU sensors placed on the upper arm and forearm respectively. The acquired data includes three-axis acceleration, gyroscope, and magnetometer data, and a set of data is acquired for calibration when the system starts. The processing procedure includes constructing a multi-skeletal model containing i bones and i-1 rotational joints, and calculating the position and posture information of all bones of the upper limb based on the forward kinematics solution process, through recursive operation of transformation matrix multiplication, and defining joint lengths.

[0010] Preferably, the motion mapping calculation adopts a master-slave heterogeneous joint algorithm. The specific process includes establishing a kinematic model of the robotic arm based on the DH method, obtaining the coordinate values ​​of the robotic arm end effector under the base coordinates through the transformation matrix; based on the position mapping of Cartesian space, the position and posture of the upper limb end effector obtained by upper limb motion capture are used as the expected pose of the robotic arm end effector, and the inverse kinematics is solved to obtain the real-time angles of each joint of the robotic arm; wherein, the robotic arm used is a ball-wrist robotic arm, the first three joints control the position, and the last three joints control the posture. The angles of the first three joints are calculated first, and then the angles of the last three joints are calculated.

[0011] Preferably, the preprocessing includes density reduction sampling and point cloud filtering; the density reduction sampling reduces the amount of data while preserving geometric structural features, suppresses local noise and redundant points, and balances the spatial distribution of the point cloud; the point cloud filtering adopts a statistical analysis-based method, which uses the average K-nearest neighbor distance to detect and remove outliers in the point cloud, and uses a dynamic threshold method to identify data points that deviate from the overall distribution.

[0012] Preferably, the specific process of 3D point cloud reconstruction includes the following: Point cloud registration based on TEASER++: Spatial transformation parameters are determined through multi-stage optimization, and local point clouds from different perspectives are aligned to a unified coordinate system and fused. Specifically, this includes constructing a geometric invariance constraint graph, solving the rotation matrix in stages using semidefinite programming relaxation techniques, and calculating the translation vector in parallel using an adaptive voting strategy. Point cloud segmentation based on Euclidean clustering: Based on the spatial proximity criterion, the topological relationship of Euclidean distance between point clouds is constructed. Through iterative expansion, connected clusters are formed, the point cloud of the target workpiece is separated, and a triangular mesh model is generated. Point cloud coordinate system establishment based on principal component analysis algorithm: The centroid of the original point cloud of the workpiece is calculated as the origin of the coordinate system. The principal plane of the workpiece is segmented by RANSAC algorithm and the Z-axis direction is determined by its normal vector. The principal direction is extracted based on PCA as the Y-axis. The orthogonal X-axis is obtained by cross product, thus completing the construction of the right-handed coordinate system.

[0013] Preferably, edge extraction is achieved by calculating the edge intensity of each point in the point cloud data. The judgment rules include: if a point has two high feature values, then the point is an edge intersection point; if a point has a single high feature value, then the point is an edge point; the edge intersection point is used as the start and end point of the adhesive application path, and the edge point is used to fit the adhesive application path.

[0014] Preferably, the glue application path generation employs a user-guided, geometrically defined, spatially filtered, and robustly fitted algorithm, specifically including: For circular arc trajectories: Users manually select three key points—start point, midpoint, and end point—to determine the circular arc path and construct geometric prior constraints. A local cylindrical space centered on the circle is constructed as a region filter to screen the point cloud. Points within the cylinder are used for local fitting to generate the adhesive application path. For straight-line trajectories: The user selects two key points, the start point and the end point, and constructs a finite-length cylinder with the two points as the axis to capture the points around the path. The least-squares straight-line fitting is then performed, or a first-order spline or polynomial smoothing process is used to generate the adhesive application path.

[0015] Preferably, the uniformity of the adhesive strip thickness is detected using a line structured light measurement method. The specific process includes extracting the red channel from the original image, setting a brightness threshold to filter out background noise, scanning line by line along the vertical direction to find the center position of the laser line, detecting the gap of the adhesive strip by the interruption position of the laser line, recording the positions of the two laser lines above and below the interruption area and calculating the vertical pixel distance to obtain the gap width.

[0016] Preferably, the quantitative evaluation indicators for the uniformity of adhesive coating width include average width, standard deviation, uniformity index, maximum and minimum width difference, and the proportion of samples exceeding the specification range.

[0017] Secondly, the present invention also provides an automatic glue application system based on an upper limb motion-operated robotic arm, comprising: The motion data processing module is used to collect and process human upper limb motion data to obtain the position and posture information of all bones in the upper limb, providing data input for the remote operation of the robotic arm; The motion mapping module is used to calculate the real-time angles of each joint of the end-effector robotic arm based on the position and posture information of all bones in the upper limb, and to establish the motion mapping relationship between the end of the human hand and the robotic arm. The point cloud acquisition module is used to control the robotic arm to perform multi-angle scanning of the workpiece based on the motion mapping relationship, and to acquire and preprocess point cloud data. The model reconstruction module is used to reconstruct 3D point clouds based on preprocessed point cloud data and extract the point cloud model of the target workpiece from the environment. The path generation module is used to extract the edges of the point cloud model of the target workpiece and generate the adhesive application path based on the edge extraction results. The glue application detection module is used to control the robotic arm to perform automatic glue application based on the glue application path, and to detect the uniformity of the glue strip thickness to complete the automatic glue application operation.

[0018] Compared with the prior art, the present invention has the following beneficial technical effects: This invention provides an automated adhesive application method based on an upper limb motion-controlled robotic arm. Through a complete technical chain consisting of upper limb motion acquisition and processing, motion mapping establishment, point cloud acquisition and preprocessing, 3D point cloud reconstruction, adhesive application path generation, and automated adhesive application and uniformity detection, it precisely overcomes the core technical problems of existing industrial robotic arm adhesive application operations, such as cumbersome operation, reliance on workpiece CAD models, strong positioning dependence, and insufficient environmental adaptability. At the same time, it simultaneously improves the quality of adhesive application and production flexibility.

[0019] Furthermore, by collecting human upper limb movement data and converting it into robotic arm control commands, the robotic arm can be naturally teleoperated. Operators do not need to master complex programming or teaching skills; they can guide the robotic arm to complete core tasks such as workpiece scanning and gluing simply through conventional upper limb movements. This greatly simplifies the operation process, lowers the operation threshold, effectively shortens the pre-operation preparation time, and significantly improves the overall gluing operation efficiency.

[0020] Furthermore, by remotely guiding the robotic arm to scan the workpiece from multiple angles and collect the original point cloud data, the target workpiece point cloud model is obtained through preprocessing and 3D reconstruction. Without providing any workpiece CAD design drawings in advance, the subsequent glue application path planning can be completed. This not only greatly expands the applicability of the system, but also accurately adapts to the glue application needs of various customized workpieces without drawings and irregularly shaped workpieces, and can better respond to the needs of personalized production scenarios.

[0021] Furthermore, after completing the workpiece point cloud reconstruction, a dedicated coordinate system corresponding to the physical posture of the workpiece is automatically constructed through principal component analysis algorithm. This can accurately identify the real-time pose of the workpiece in space without relying on any mechanical positioning device. This simplifies the overall structure of the production line, effectively reduces the design, manufacturing and maintenance costs of the production line, and improves the flexibility of workpiece positioning. It can adapt to the gluing requirements of different workpiece placement postures and reduce the frequency and cost of tooling fixture replacement.

[0022] Furthermore, by employing techniques such as point cloud filtering and Euclidean clustering segmentation, the point cloud of the target workpiece can be accurately separated from complex environments, demonstrating excellent adaptability to unstructured environments. When products iterate, there is no need to redesign or modify the original program or tooling; simply rescanning the new workpiece, reconstructing the point cloud model, and generating a suitable gluing path via remote operation allows for rapid switching of operations, significantly improving the system's reusability and versatility, substantially reducing production line upgrade costs, and enhancing the ability to respond quickly to product iterations. Simultaneously, this invention performs glue strip thickness uniformity detection during the automatic gluing process, promptly identifying defects such as glue strip gaps and uneven thickness, preventing batch defects, improving gluing accuracy and product pass rate, and providing feedback data to support glue parameter optimization, forming a closed-loop control system for gluing, detection, and optimization, ensuring the stability of gluing quality. Attached Figure Description

[0023] Figure 1 This is a flowchart of an automatic glue application method for a robotic arm based on upper limb movements in an embodiment of the present invention. Figure 2 This is a schematic diagram of the upper limb motion capture system based on an inertial measurement unit in an embodiment of the present invention; Figure 3 This is a schematic diagram of the composition of the robotic arm adhesive application system in an embodiment of the present invention; Figure 4 This is a schematic diagram of ROS and a software structure diagram in an embodiment of the present invention; Figure 5 This is a schematic diagram of the speed sensing module in an embodiment of the present invention; Figure 6 This is a flowchart of a remotely operated robot gluing system in an embodiment of the present invention; Figure 7 This is a schematic diagram of the automatic glue application system based on the upper limb movement teleoperation of the robotic arm in an embodiment of the present invention; In the diagram: 1. Motion data processing module; 2. Motion mapping module; 3. Point cloud acquisition module; 4. Model reconstruction module; 5. Path generation module; 6. Adhesive application detection module. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] The purpose of this invention is to provide an automatic glue application method and system based on an upper limb motion-operated robotic arm, so as to solve the technical problems of existing industrial robotic arms in glue application operations, such as cumbersome operation, reliance on workpiece CAD models, strong positioning dependence, and insufficient environmental adaptability.

[0027] The present invention will now be described in further detail with reference to the accompanying drawings: See Figure 1 In one embodiment of the present invention, an automatic adhesive application method based on an upper limb motion-operated robotic arm is provided, comprising: Step 1: Collect and process human upper limb movement data to obtain the position and posture information of all bones in the upper limb, providing data input for the teleoperation of the robotic arm; Specifically, human upper limb motion data is collected to provide reliable data input for robotic arm teleoperation. This upper limb motion data includes triaxial accelerometer, gyroscope, and magnetometer data from two IMU sensors placed on the upper arm and forearm (a set of data is reported for calibration upon system startup), collected in a specific format at a certain frequency. After obtaining this data, referring to the solution process of forward kinematics, when the pose of the parent skeleton and the relative rotation angle and offset between the child and parent skeletons are known, the position and orientation information of the child skeleton can be calculated. First, a system containing... A skeleton and A multi-bone model with rotational joints, wherein the bones For the root bone (hip joint), bone and For a parent-child skeleton pair, the pose in the reference coordinate system at the initial moment is as follows:

[0028] In the formula, Root skeleton This refers to the change in quantity during translational motion (walking, squatting, etc.). For the skeleton in Changes in rotational motion , , And so on. Using transformation matrix multiplication, the pose of the i-th skeleton in the reference coordinate system can be expressed as:

[0029] The algorithm principle is shown in Table 1: Table 1. Dynamic Visualization Algorithm for Human Upper Limb Movements

[0030] By performing recursive calculations and defining joint lengths, the position and posture information of all bones in the upper limb (including the position and posture of the upper limb end) can be obtained. In practical applications, when the data input is a quaternion output by the IMU, the above steps can be followed for calculation.

[0031] Step 2: Based on the position and posture information of all bones in the upper limb, the real-time angles of each joint of the end-effector robotic arm are calculated through motion mapping to establish the motion mapping relationship between the end of the human hand and the robotic arm. Specifically, based on the collected upper limb motion data, a master-slave heterogeneous joint algorithm is used to map the motion of the human hand's end effector onto the slave-end robotic arm. When the robotic arm performs real-time tasks, each state update requires real-time calculation of how the joint angles should change to meet the robotic arm's kinematic objectives in the current environment. This is especially crucial in teleoperation, where real-time performance is paramount. Dynamically calculating the joint angles at each moment is essential to ensure the robotic arm can respond to operational commands promptly and accurately. Due to the heterogeneity of the master-slave teleoperation system, the joint angles at the master end cannot be directly applied as control variables to the slave end. Instead, a position mapping method is used to indirectly obtain the joint angle values ​​at the slave end. This direct point-to-point method can be implemented by solving the inverse kinematics (IK) during each update.

[0032] Using the DH method, a kinematic model of the robotic arm was established. The unique coordinates (position and orientation) of the robotic arm's end effector in the base coordinate system can be obtained through transformation matrices. The inverse kinematics solution process involves calculating the joint angles based on the desired pose of the robotic arm's end effector in the coordinate system. 1-6, the robot's desired pose expression is as follows:

[0033]

[0034] Since the JAKA ZU 5 assisted robotic arm has a spherical wrist (the axes of the last three joints intersect at one point), the first three joints generally control the position, and the last three joints control the posture. First, calculate the angles of the first three joints:

[0035] Among them, the motion mapping relationship of the human hand end is based on the position mapping of Cartesian space. The position and posture of the upper limb end obtained by the upper limb motion capture system are used as the expected pose of the robotic arm end. The real-time angles of each joint of the robotic arm are obtained through the inverse kinematics algorithm.

[0036] Step 3: Based on the motion mapping relationship, control the robotic arm to perform multi-angle scanning of the workpiece, collect and preprocess point cloud data; Specifically, the workpiece is scanned from multiple angles based on the motion mapping relationship of the human end effector, and point cloud data is saved. After obtaining the original point cloud data of the 3D target workpiece through teleoperation-guided scanning, the data acquired by the camera needs to be processed. Downsampling and point cloud filtering are common preprocessing methods. The role of downsampling is to effectively reduce the amount of data while preserving geometric structural features as much as possible, thereby improving the computational efficiency and stability of subsequent algorithms. First, this method can reduce the density of the point cloud, significantly reducing the pressure on storage and computing resources. Second, appropriate downsampling helps to suppress local noise and redundant points, improving the overall quality of the point cloud. In addition, downsampling can also balance the distribution of the point cloud in space to a certain extent, providing a better data foundation for subsequent tasks such as registration.

[0037] When acquiring point cloud data, due to factors such as equipment precision, electromagnetic diffraction characteristics, and environmental conditions (e.g., occlusion), some noise points and discrete points far from the main point cloud will inevitably appear. Statistical analysis-based methods can be used. The core idea is to assume that most points in the point cloud are uniformly distributed or have similar local density, while the neighborhood statistical characteristics of outliers are significantly different from those of the majority of points. Point cloud filtering is based on the detection and removal of outliers in point clouds using the average K-nearest neighbor distance. Through a dynamic thresholding method, it can effectively identify data points that deviate from the overall distribution, improve point cloud quality, and provide more reliable input for subsequent point cloud processing steps such as reconstruction and registration.

[0038] Step 4: Perform 3D point cloud reconstruction based on the preprocessed point cloud data to extract the point cloud model of the target workpiece from the environment. Specifically, the process of 3D point cloud reconstruction is as follows: Point cloud registration based on TEASER++: During teleoperation-guided scanning, due to limitations imposed by the workpiece's structure and environmental occlusion, point clouds acquired from multiple perspectives often suffer from data gaps, accompanied by rotation and translation deviations between viewpoints. To construct a complete 3D model, point cloud registration technology is needed to determine the optimal spatial transformation parameters, aligning local point clouds from different perspectives to a unified coordinate system, and ultimately fusing them into a complete global point cloud dataset. TEASER++ is a point cloud registration algorithm designed for noisy scenarios, achieving high accuracy and real-time performance through multi-stage optimization. Its core employs truncated least squares (TLS) to reconstruct the problem model, significantly suppressing the interference of outlier matching on the results. The algorithm separates the optimization processes of scale, rotation, and translation using a graph theory framework: first, a geometrically invariant constraint graph is constructed, and maximum clique search is used to eliminate outliers exceeding 90%; then, a semidefinite programming relaxation technique is used to solve the rotation matrix in stages to avoid local optima traps; finally, an adaptive voting strategy is used to compute the translation vector in parallel, combined with dynamic adjustment of the threshold based on the residuals. A stable rotation matrix and translation vector synthesize the point cloud into the target point cloud.

[0039] Point cloud segmentation based on Euclidean clustering: Subsequently, point cloud segmentation based on Euclidean clustering was performed. Euclidean clustering extraction is a classic point cloud segmentation algorithm whose principle fully conforms to the distribution characteristics of discrete point cloud data in three-dimensional space. Based on the spatial proximity criterion, this algorithm constructs Euclidean distance topological relationships between point clouds, forming connected clusters through iterative expansion. The clustering radius (threshold) is a key parameter of this algorithm. A smaller radius produces more clusters, each containing fewer points, potentially segmenting the same object into multiple parts and being sensitive to noise. Conversely, a larger radius produces fewer clusters, potentially merging different objects into the same cluster, leading to loss of detailed information. Using clustering segmentation, the point cloud of the target workpiece was successfully separated, and a clear triangular mesh model was generated based on this. The model essentially reconstructs the shape of the workpiece, including the central hole and the surrounding structure.

[0040] Point cloud coordinate system establishment based on principal component analysis (PCA) algorithm: After extracting the original point cloud data of the workpiece, in order to accurately identify the workpiece's pose in complex environments and design the subsequent adhesive application path, it is necessary to annotate the workpiece with a clear and consistent coordinate system. Coordinate system annotation is also an important bridge connecting the modeling and execution stages.

[0041] The method employed in this application first calculates the centroid of the workpiece's original point cloud, using this as the origin of the coordinate system. Next, the RANSAC algorithm is used to segment the workpiece's principal plane, and the Z-axis direction of the coordinate system is determined by the normal vector of this plane, ensuring a physical correspondence between the coordinate system and the workpiece's spatial orientation. Subsequently, PCA (Principal Component Analysis) is used to analyze the direction of the point cloud, extracting the principal direction as the Y-axis, reflecting the workpiece's long side or main structural features. Finally, the orthogonal X-axis is obtained through a cross product, completing the construction of the right-handed coordinate system.

[0042] Step 5: Extract the edges of the point cloud model of the target workpiece and generate the adhesive application path based on the edge extraction results; Specifically, edge extraction is performed based on the point cloud model to obtain edge extraction results, and a glue application path is generated based on the edge extraction results. In mechanical equipment or pipelines, the edges are often the boundary areas where other parts contact and connect with other parts, and glue application at these locations is a common process. Statistics show that edge glue application typically accounts for a large proportion in many industrial applications because edges are often critical areas for sealing, protection, and fixation. Therefore, an edge recognition-based glue application path extraction algorithm was designed. The edge strength of each point in the point cloud data can be calculated using the following formula:

[0043] When extracting the edges of a point cloud, if two high feature values ​​are observed coexisting at a point, it means that the point is located at two intersecting edges (edge ​​intersection point). If a single high feature value is observed, it means that it is a point on the edge. In subsequent operations, this characteristic can be used to take the edge intersection point as the start and end point, while other edge points can be used to fit the adhesive application path.

[0044] Edge recognition can be considered as a preliminary extraction of the glue application area. However, to extract the glue application trajectory, it is necessary to identify the glue application trajectory points, then fit the data, and finally generate the glue application path.

[0045] This embodiment presents a user-guided, geometrically defined, spatially filtered, and robustly fitted algorithm for extracting glue application trajectory points. Taking a circular contour as an example, the specific steps are as follows: 1) By having the user manually select three key points (start point, midpoint, and end point), a circular arc path is determined in space, and geometric prior constraints are constructed accordingly.

[0046] 2) Construct a local cylindrical space centered on the circle and use it as a region filter to effectively filter out noise points and non-target region points.

[0047] 3) By using points inside the cylinder for local fitting and combining them with real point cloud data, a coating path that is closer to the actual trajectory, more robust to noise and outliers, and has strong smoothness and continuity is generated.

[0048] The generation of straight-line trajectories is similar to that of circular trajectories. Users select two points: the starting point and the ending point, and construct a finite-length cylinder with these two points as its axis. This captures points around the path that are "approximately distributed along a straight line". Least-squares straight-line fitting can be performed directly. If it is desired to retain some natural offset or the actual curve feel of the machine-executed trajectory, linear splines or polynomials can be used for smoothing.

[0049] Step 6: Based on the glue application path, control the robotic arm to perform automatic glue application and detect the uniformity of the glue strip thickness to complete the automatic glue application operation.

[0050] Specifically, the system automatically applies adhesive based on the application path and detects the uniformity of the adhesive strip thickness. The specific process is as follows: When measuring the width of adhesive strips using structured light, the acquired images undergo a series of processing steps to extract useful information. First, the red channel is extracted from the original image, as laser lines typically have a distinct red characteristic. Then, by setting a brightness threshold, only pixels with intensity exceeding the threshold are retained, effectively filtering out background noise and other interference. Next, the image is scanned line by line along its vertical direction, and the center position of the laser line in each line is located, typically using the median method to improve robustness. By observing the continuity of the laser line in the vertical direction, the location of any interruption can be detected; this interruption usually corresponds to a gap between adhesive strips. Further, the positions of the laser lines above and below the interrupted area are recorded, and the vertical pixel distance between them is calculated to obtain the gap width (in pixels).

[0051] When quantitatively evaluating the uniformity of adhesive coating width, several key indicators were selected, including average width, standard deviation, uniformity index, maximum and minimum width difference, and the proportion of samples exceeding the specification range. Average width reflects the overall thickness of the adhesive layer and is a core parameter for measuring the basic coating amount. However, the average value alone cannot fully reveal the uniformity of the width; therefore, the standard deviation was introduced to quantify the dispersion of the width data relative to the average value. The smaller the standard deviation, the smaller the fluctuation in the adhesive coating width, and the better the uniformity. To more intuitively evaluate uniformity, a uniformity index was also defined. This index is usually calculated based on the standard deviation or coefficient of variation, and can directly reflect the overall consistency level of the adhesive coating width with a single value. Furthermore, the maximum and minimum width difference directly examines the extreme deviations of the adhesive coating width throughout the entire coating range; a large difference indicates significant local width variations and poor uniformity.

[0052] In this embodiment, the teleoperated robotic arm system based on upper limb motion control includes an upper limb motion capture part and a master-slave heterogeneous teleoperated robotic arm system part.

[0053] The upper limb motion capture component includes: motion capture sensors, respectively placed on the upper arm, forearm, and handle; a sensor data processing component; and an upper limb posture calculation component. The motion capture sensors are inertial nine-axis IMU sensors, and the sensor data processing component includes data calibration and filtering algorithms for accelerometers and gyroscopes. The filtering algorithm is a second-order Kalman filter algorithm.

[0054] The master-slave heterogeneous teleoperated robotic arm system includes: a robotic arm hardware and software system, and a real-time robotic arm inverse kinematics solution method. The software system is built using a Robot Operating System (ROS), and the hardware system consists of a six-degree-of-freedom collaborative robotic arm, a depth camera, and an adhesive applicator. The real-time robotic arm inverse kinematics solution method calculates the corresponding robotic arm joint angles based on the posture of the upper limb end effector to achieve master-slave pose mapping.

[0055] Based on the aforementioned teleoperated robotic arm system, an automatic glue application system was developed, comprising a point cloud data processing section, a glue application path extraction section, and a glue application quality detection section. The point cloud data processing section first performs source point cloud de-density sampling, which reduces the density of the point cloud and significantly reduces the pressure on storage and computing resources. Secondly, it combines outlier point processing to identify and remove data points that deviate from the overall distribution, improving point cloud quality. Finally, it registers the point clouds collected from multiple angles and merges them into a complete global point cloud dataset. The point cloud registration algorithm is based on the TEASER++ algorithm.

[0056] The adhesive application path extraction section includes point cloud data extraction based on edge recognition, calculation of adhesive application trajectory points based on user guidance, geometric definition, and spatial filtering, and adhesive application path fitting based on least squares. The adhesive application quality detection utilizes line structured light to effectively detect the thickness of the adhesive strip.

[0057] like Figure 2As shown, the upper limb motion capture system comprises four main parts: data acquisition, data processing, posture calculation, and upper-layer application. The data acquisition part collects data from multiple sensors (three-axis accelerometer, gyroscope, and magnetometer – a set of data is reported for calibration upon system startup) in a specific format at a certain frequency. The data processing part is responsible for eliminating sensor bias, magnetometer calibration, and data fusion filtering. The processed sensor data is mapped to a global coordinate system, and the upper limb posture and position are calculated using a defined human motion model. The application part involves data publishing and transmission from relevant nodes. Ultimately, upper-layer applications (such as visualization software and robotic arm motion control nodes) subscribe to this data to reconstruct real-time upper limb motion and apply trajectory and pose information.

[0058] The automated glue-applying system based on the human upper limb movement-controlled robotic arm in this implementation can be divided into a hardware system and a software system. The hardware system includes a teleoperation hardware system comprising: a multi-axis inertial measurement unit mounted on the human upper limb as the master control device; a JAKA ZU 5 six-degree-of-freedom robotic arm and its control box; a Linux system computer; and a glue-applying system. The JAKA ZU5 robotic arm is a flexible, intelligent, collaborative industrial robot characterized by its small size, high flexibility, ease of rapid deployment, high precision, and sensitive response ensuring accurate operation and real-time feedback. Furthermore, it provides rich full-stack secondary development interfaces, facilitating algorithm integration and master-slave control development. The glue-applying system consists of three parts: a vision system, a glue-applying system, and a glue supply system. Figure 3 As shown, the vision system consists of a point cloud camera, a laser generator, and an industrial camera, responsible for reconstructing the 3D part model and inspecting the adhesive application quality, respectively. The adhesive application system consists of a computer and a six-degree-of-freedom robotic arm, primarily serving as the motion mechanism to apply the adhesive. The adhesive supply system consists of a suction valve, an adhesive supply controller, and an adhesive storage tank, mainly performing the functions of storing, conveying, and discharging adhesive. Each system is mounted at the end effector of the robotic arm via a specially designed connecting flange.

[0059] The software component, including the teleoperation software system, is developed using a Robot Operating System (ROS). Its main advantages include a modular architecture that facilitates task allocation and collaborative development, and support for distributed operation and network communication. ROS nodes are its basic building blocks, capable of executing specific tasks, similar to processes in an operating system. Nodes communicate through messages and can publish or subscribe to specific topics. Topics serve as the channels for data communication between nodes. The specific operation of inter-node communication is as follows... Figure 4As shown, the communication mechanism of the Robot Operating System (ROS) can be used to develop a data transmission network for a teleoperation system. The master controller, kinematics solver, and slave robotic arm can operate as three nodes. The master controller node receives data from the inertial sensors via Bluetooth and calculates the position and orientation of the human upper limb. It publishes the target orientation through the ROS topic / Pose Topic. The kinematics solver subscribes to this topic and obtains the calculated angles of the six joints of the robotic arm at that moment using a specific master-slave mapping algorithm. After receiving the data, the slave JAKA ZU5 robotic arm, according to the underlying control logic, drives the arm to move to the target position, completing the desired motion.

[0060] In the embodiment, when the robotic arm performs a real-time task, it is necessary to calculate in real time how the joint angles should change each time the state is updated in order to meet the kinematic objectives of the robotic arm in the current environment. Especially in teleoperation, real-time performance is of utmost importance. Due to the heterogeneity of the teleoperation master-slave system, the joint angles of the master end cannot be directly used as control variables for the slave end. The joint angle values ​​of the slave end can only be indirectly obtained by using position mapping. This direct point-to-point method can be achieved by solving the inverse kinematics (IK) each time it is updated.

[0061] This embodiment differs from traditional numerical solutions by designing a real-time motion synthesis method based on velocity perception. During each system update, it receives the desired posture of a robotic arm, including the target position. and target direction It outputs the robot arm joint angle corresponding to the posture. To achieve this goal, the following conditions must be met: ，

[0062] That is, the current position of the robotic arm's end effector. (This can be obtained through forward kinematics (FK) at each joint angle) and the target position is equal. However, directly solving a nonlinear transcendental vector equation is too complex. Instead, we try to minimize the L2 error between the robot's end-effector position and the target position. In this way, solving the inverse kinematics problem can be transformed into solving an optimization control problem. Similarly, the end-effector posture of the robotic arm can be achieved. With target attitude To achieve a smooth joint motion while ensuring the continuity of the robotic arm's joint angle solution sequence, optimization terms can be added to improve the smoothness of the robotic arm's operation and reduce wear. ， ，

[0063] The objective function is described as a weighted sum of various errors. The weights balance the priorities among different objectives. A module that dynamically adjusts the weights based on motion velocity is included, and its principle is as follows: Figure 5 As shown, this module aims to automatically adjust the weight parameters of each target item based on the real-time speed information of the robotic arm's end effector or joints. During high-speed motion, the system will reduce the weight of end effector pose accuracy and increase the priority of posture continuity and naturalness to avoid problems such as drastic posture jumps and inverse kinematics instability. During low-speed or fine operation, the accuracy weight will be increased again to ensure the accuracy and reliability of task completion.

[0064] This embodiment uses a velocity-aware real-time motion synthesis principle to transform the inverse kinematics problem into an optimization control problem. It minimizes the weighted sum of the end-effector pose error and the joint motion smoothness index, dynamically adjusting the weights to balance accuracy and continuity. Based on this, a Sequenced Quadratic Programming (SLSQP) algorithm is introduced, which can efficiently handle nonlinear constraint optimization and obtain a stable, continuous joint angle sequence that meets the task requirements.

[0065] In this embodiment, the automatic glue application method operates on the basis of the aforementioned teleoperated robotic arm system. Based on this, the automatic glue application system includes a point cloud data processing part, a glue application path extraction part, and a glue application quality detection part.

[0066] The system operation process is as follows: Figure 6 As shown, firstly, under the remote control of gestures, the robotic arm is moved to several desired poses. A depth camera mounted at the end of the robotic arm captures point cloud data of the target workpiece from different perspectives. Then, using the robotic arm pose parameters corresponding to all individual point cloud data, coordinate transformation is performed to fuse all point cloud data into a unified coordinate system. A clustering segmentation algorithm is then used to extract the target workpiece from the environment, obtaining a complete surface model of the target workpiece. Based on this, an edge-strength-based glue seam detection algorithm is used to locate the glue-applying robot's path points, and fitting and interpolation processing is performed to generate a smooth and continuous glue-applying trajectory. Finally, the robot executes different glue-applying paths in segments according to human instructions to achieve the glue-applying task on the target workpiece.

[0067] The specific process is as follows: Point Cloud Processing: After obtaining the original point cloud data of the 3D target workpiece through teleoperation-guided scanning, the data acquired by the camera needs to be processed. De-density sampling and outlier removal are common preprocessing methods. De-density sampling involves defining a voxel size (voxel_size) to divide the cube of the spatial mesh. Then, the voxel index of the current point is calculated. For an input point (x, y, z) and a given voxel_size, the index of the voxel in the x, y, and z directions is calculated. These voxel indices are then combined into a unique identifier, and the current point is added to the corresponding voxel. If the voxel_id entry does not already exist in the voxel mesh, a new entry is created, and the coordinates of the current point are added to the voxel's list. If the voxel_id entry already exists in the voxel mesh, the coordinates of the current point are added to the existing list for that voxel. Iterate through each non-empty voxel in the previously created voxel grid. For each non-empty voxel, select a point to represent that voxel based on the information stored in step 3, in order to reduce the point cloud density.

[0068] The discrete point processing employs the following steps: First, it is necessary to determine the key parameters, namely the number of neighborhood points used to evaluate the local statistical information of each point. And the standard deviation multiple threshold for determining outliers. Then, for each point in the point cloud Algorithm search includes A point cloud set consisting of 10 adjacent points Calculate the point based on these adjacent points. The average distance to each point in its neighborhood is used to determine outliers, based on the following formula: ,

[0069] In the formula, This is the global mean estimate of the average distance between all points in the point cloud and their neighborhoods. This is the global standard deviation of the average distance. If the point... To all of them If the distance between all adjacent points is greater than this threshold, the point is determined to be an outlier and needs to be removed.

[0070] During remotely operated guided scanning, due to limitations imposed by the workpiece's structure and environmental occlusion, point clouds acquired from multiple perspectives often suffer from data gaps, accompanied by rotational and translational deviations between viewpoints. To construct a complete 3D model, point cloud registration technology is needed to determine the optimal spatial transformation parameters, aligning local point clouds from different perspectives to a unified coordinate system, and finally fusing them into a complete global point cloud dataset—that is, achieving point cloud registration. The point cloud registration algorithm is based on TEASER++.

[0071] After extracting the original point cloud data of the workpiece, a clear and consistent coordinate system is needed to accurately identify the workpiece's pose in complex environments and design the subsequent adhesive application path. Coordinate system annotation is also a crucial bridge connecting the modeling and execution stages. The method employed first calculates the centroid of the original workpiece point cloud, using this as the origin of the coordinate system. Next, the RANSAC algorithm is used to segment the principal plane of the workpiece, and the Z-axis direction of the coordinate system is determined by the normal vector of this plane, ensuring a physical correspondence between the coordinate system and the workpiece's spatial pose. Subsequently, PCA (Principal Component Analysis) is used to analyze the direction of the point cloud, extracting the principal direction as the Y-axis, reflecting the workpiece's long side or main structural features. Finally, the orthogonal X-axis is obtained through cross product, completing the construction of the right-handed coordinate system.

[0072] The method for extracting the adhesive application path is as follows: The application scenario of the adhesive application system in this embodiment is also edge adhesive application. After obtaining a complete 3D point cloud model of the part (or environment), an adhesive seam detection algorithm based on edge intensity gradient is designed to extract points belonging to the edge of the part and prepare for subsequent adhesive application path generation. The edge extraction algorithm is based on edge intensity gradient, which is used to measure the degree of abrupt change in geometry / attributes within the neighborhood of a point, thereby identifying "boundary points" or "contour points" similar to image edges. If two high feature values ​​are observed coexisting at a point, it means that the point is located at two intersecting edges (edge ​​intersection point). If a single high feature value is observed, it means that it is a point on the edge. In subsequent operations, this characteristic can be used to take the edge intersection point as the start and end point, while other edge points can be used to fit the adhesive application path.

[0073] Specifically, a user-guided, geometrically defined, spatially filtered, and robustly fitted algorithm for extracting glue application trajectory points was designed. Taking a circular contour as an example, the specific steps are as follows: 1) By having the user manually select three key points (start point, midpoint, and end point), a circular arc path is determined in space, and geometric prior constraints are constructed accordingly.

[0074] 2) Construct a local cylindrical space centered on the circle and use it as a region filter to effectively filter out noise points and non-target region points.

[0075] 3) By using points inside the cylinder for local fitting and combining them with real point cloud data, a coating path that is closer to the actual trajectory, more robust to noise and outliers, and has strong smoothness and continuity is generated.

[0076] The generation of straight-line trajectories is similar to that of circular trajectories. Users select two points: the starting point and the ending point, and construct a finite-length cylinder with these two points as its axis. This captures points around the path that are "approximately distributed along a straight line". Least-squares straight-line fitting can be performed directly. If it is desired to retain some natural offset or the actual curve feel of the machine-executed trajectory, linear splines or polynomials can be used for smoothing.

[0077] The adhesive coating quality inspection process in this embodiment is as follows: Adhesive coating quality inspection is based on the principle of line structured light. By projecting light of a specific shape (usually a thin line or stripe) onto the surface of an object, and using the deformation of the light observed by a camera, changes in the object's surface shape or size can be inferred, as shown in Figure x. Because the adhesive strip itself has raised surface features, the light undergoes a significant change when the laser line sweeps across the area containing the adhesive strip. In the image, the light that does not pass through the adhesive strip remains unchanged, while the light above the adhesive strip disappears or shifts. This breakage characteristic is very noticeable in the image captured by the camera.

[0078] Specifically, image processing algorithms can be used to analyze the positional continuity of the laser lines along the vertical direction of the image. The acquired image undergoes a series of processing steps to extract useful information. First, the red channel is extracted from the original image, as laser lines typically have a distinct red characteristic. Then, by setting a brightness threshold, only pixels with intensity exceeding the threshold are retained, effectively filtering out background noise and other interference. Next, the image is scanned line by line along the vertical direction, searching for the center position of the laser line in each line, typically using the median method to improve robustness. By observing the vertical continuity of the laser lines, the location of any interruption can be detected; this interruption usually corresponds to a gap between adhesive strips. Further, the positions of the laser lines above and below the interrupted area are recorded, and the vertical pixel distance between them is calculated to obtain the width of the gap (in pixels), which can be considered the width of the adhesive strip.

[0079] Subsequently, the uniformity of the adhesive coating width was quantitatively evaluated, selecting several key indicators including average width, standard deviation, uniformity index, maximum and minimum width difference, and the proportion of samples exceeding the specification range. Average width reflects the overall thickness of the adhesive layer and is a core parameter for measuring the basic coating amount. Standard deviation quantifies the dispersion of width data relative to the average value; a smaller standard deviation indicates less fluctuation in the adhesive coating width and better uniformity. A uniformity index was also defined, typically calculated based on standard deviation or coefficient of variation, directly reflecting the overall consistency level of the adhesive coating width with a single value. When the uniformity index is greater than 0.1, it indicates poor uniformity and low quality in that section of the coating, requiring recording in the system.

[0080] In summary, the automatic adhesive application method based on an upper limb teleoperated robotic arm provided in this embodiment has several advantages. First, the teleoperated robotic arm can flexibly change the position and orientation of the sensors according to the characteristics of the environment and the workpiece, while bypassing obstructions to effectively eliminate scanning blind spots. Second, in complex environments, the teleoperated robotic arm can manually control the distance between the scanning device and the workpiece to ensure point cloud density and accuracy. Third, thanks to the flexibility of the teleoperated robotic arm, it can perform omnidirectional scanning around the target object, capturing every angle and detail to generate a high-quality point cloud model.

[0081] This embodiment combines point cloud scanning and modeling, enabling workpiece identification and tracking without relying on complex mechanical positioning and tooling fixtures, thus reducing production line design and manufacturing costs. It can quickly adapt to different workpiece shapes and orientations, achieving flexible cross-product applications and avoiding high-cost program and tooling modifications due to product iterations.

[0082] This embodiment combines remote operation with automation. In the early stage, human-machine collaboration is used to quickly complete workpiece identification and modeling, and in the later stage, the robot autonomously performs the glue application operation, achieving a balance between efficiency and accuracy.

[0083] This embodiment avoids direct exposure of operators to harmful environments containing volatile organic solvents, isocyanates, etc., reducing occupational health risks and enhancing the system's applicability in complex or hazardous environments.

[0084] Example 2 according to Figure 7 As shown, this embodiment also provides an automatic glue application system based on an upper limb motion-operated robotic arm, including: The motion data processing module 1 is used to collect and process human upper limb motion data to obtain the position and posture information of all bones in the upper limb, providing data input for the remote operation of the robotic arm. The motion mapping module 2 is used to calculate the real-time angles of each joint of the end-effector robotic arm based on the position and posture information of all bones of the upper limb, and to establish the motion mapping relationship between the end of the human hand and the robotic arm. Point cloud acquisition module 3 is used to control the robotic arm to perform multi-angle scanning of the workpiece based on the motion mapping relationship, and to acquire and preprocess point cloud data; Model reconstruction module 4 is used to perform 3D point cloud reconstruction based on preprocessed point cloud data and extract the point cloud model of the target workpiece from the environment. The path generation module 5 is used to extract the edges of the point cloud model of the target workpiece and generate the glue application path based on the edge extraction results. The glue application detection module 6 is used to control the robotic arm to perform automatic glue application based on the glue application path, and to detect the uniformity of the glue strip thickness to complete the automatic glue application operation.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. An automated adhesive application method based on an upper limb teleoperated robotic arm, characterized in that, include: Collect and process human upper limb movement data to obtain the position and posture information of all bones in the upper limb, providing data input for the teleoperation of the robotic arm; Based on the position and posture information of all bones in the upper limb, the real-time angles of each joint of the end-effector robotic arm are calculated through motion mapping, and the motion mapping relationship between the end of the human hand and the robotic arm is established. Based on the motion mapping relationship, the robotic arm is controlled to perform multi-angle scanning of the workpiece, and point cloud data is collected and preprocessed. 3D point cloud reconstruction is performed based on preprocessed point cloud data to extract the point cloud model of the target workpiece from the environment. Edge extraction is performed on the point cloud model of the target workpiece, and an adhesive application path is generated based on the edge extraction results; The robotic arm is controlled based on the glue application path to perform automatic glue application and detect the uniformity of the glue strip thickness, thus completing the automatic glue application operation.

2. The automatic glue application method for a teleoperated robotic arm based on upper limb movements according to claim 1, characterized in that, The human upper limb motion data is acquired through two IMU sensors placed on the upper arm and forearm respectively. The acquired data includes three-axis acceleration, gyroscope, and magnetometer data, and a set of data is acquired for calibration when the system starts. The processing includes constructing a multi-skeletal model containing i bones and i-1 rotational joints, and calculating the position and posture information of all bones in the upper limb based on the forward kinematics solution process, through recursive operation of transformation matrix multiplication, and defining joint lengths.

3. The automatic glue application method for a teleoperated robotic arm based on upper limb movements according to claim 1, characterized in that, The motion mapping calculation adopts a master-slave heterogeneous joint algorithm. The specific process includes establishing a kinematic model of the robotic arm based on the DH method, obtaining the coordinate values ​​of the robotic arm end in the base coordinates through the transformation matrix; based on the position mapping of Cartesian space, the position and posture of the upper limb end obtained by upper limb motion capture are used as the expected pose of the robotic arm end, and the inverse kinematics is solved to obtain the real-time angles of each joint of the robotic arm; wherein, the robotic arm used is a ball-wrist robotic arm, the first three joints control the position, and the last three joints control the posture. The angles of the first three joints are calculated first, and then the angles of the last three joints are calculated.

4. The automatic glue application method for a teleoperated robotic arm based on upper limb movements according to claim 1, characterized in that, The preprocessing includes density reduction sampling and point cloud filtering. The density reduction sampling reduces the amount of data while preserving geometric structural features, suppresses local noise and redundant points, and balances the spatial distribution of the point cloud. The point cloud filtering adopts a statistical analysis-based method, which uses the average K-nearest neighbor distance to detect and remove outliers in the point cloud, and uses a dynamic threshold method to identify data points that deviate from the overall distribution.

5. The automatic glue application method for a teleoperated robotic arm based on upper limb movements according to claim 1, characterized in that, The specific process of 3D point cloud reconstruction includes the following: Point cloud registration based on TEASER++: Spatial transformation parameters are determined through multi-stage optimization, and local point clouds from different perspectives are aligned to a unified coordinate system and fused. Specifically, this includes constructing a geometric invariance constraint graph, solving the rotation matrix in stages using semidefinite programming relaxation techniques, and calculating the translation vector in parallel using an adaptive voting strategy. Point cloud segmentation based on Euclidean clustering: Based on the spatial proximity criterion, the topological relationship of Euclidean distance between point clouds is constructed. Through iterative expansion, connected clusters are formed, the point cloud of the target workpiece is separated, and a triangular mesh model is generated. Point cloud coordinate system establishment based on principal component analysis algorithm: The centroid of the original point cloud of the workpiece is calculated as the origin of the coordinate system. The principal plane of the workpiece is segmented by RANSAC algorithm and the Z-axis direction is determined by its normal vector. The principal direction is extracted based on PCA as the Y-axis. The orthogonal X-axis is obtained by cross product, thus completing the construction of the right-handed coordinate system.

6. The automatic glue application method for a teleoperated robotic arm based on upper limb movements according to claim 1, characterized in that, The edge extraction is achieved by calculating the edge intensity of each point in the point cloud data. The judgment rules include: if a point has two high feature values, then the point is an edge intersection point; if a point has a single high feature value, then the point is an edge point; the edge intersection point is used as the start and end point of the adhesive application path, and the edge point is used to fit the adhesive application path.

7. The automatic glue application method for a teleoperated robotic arm based on upper limb movements according to claim 1, characterized in that, The glue application path generation employs a user-guided, geometrically defined, spatially filtered, and robustly fitted algorithm, specifically including: For circular arc trajectories: Users manually select three key points—start point, midpoint, and end point—to determine the circular arc path and construct geometric prior constraints. A local cylindrical space centered on the circle is constructed as a region filter to screen the point cloud. Points within the cylinder are used for local fitting to generate the adhesive application path. For straight-line trajectories: The user selects two key points, the start point and the end point, and constructs a finite-length cylinder with the two points as the axis to capture the points around the path. The least-squares straight-line fitting is then performed, or a first-order spline or polynomial smoothing process is used to generate the adhesive application path.

8. The automatic glue application method for a teleoperated robotic arm based on upper limb movements according to claim 1, characterized in that, The uniformity of the adhesive strip thickness is detected using a line structured light measurement method. The specific process includes extracting the red channel from the original image, setting a brightness threshold to filter out background noise, scanning line by line along the vertical direction to find the center position of the laser line, detecting the gap of the adhesive strip by the interruption position of the laser line, recording the positions of the two laser lines above and below the interruption area, and calculating the vertical pixel distance to obtain the gap width.

9. The automatic glue application method for a teleoperated robotic arm based on upper limb movements according to claim 1, characterized in that, The quantitative evaluation indicators for the uniformity of adhesive coating width include average width, standard deviation, uniformity index, maximum and minimum width difference, and the proportion of samples exceeding the specification range.

10. An automated adhesive application system based on an upper limb teleoperated robotic arm, characterized in that, include: The motion data processing module is used to collect and process human upper limb motion data to obtain the position and posture information of all bones in the upper limb, providing data input for the remote operation of the robotic arm; The motion mapping module is used to calculate the real-time angles of each joint of the end-effector robotic arm based on the position and posture information of all bones in the upper limb, and to establish the motion mapping relationship between the end of the human hand and the robotic arm. The point cloud acquisition module is used to control the robotic arm to perform multi-angle scanning of the workpiece based on the motion mapping relationship, and to acquire and preprocess point cloud data. The model reconstruction module is used to reconstruct 3D point clouds based on preprocessed point cloud data and extract the point cloud model of the target workpiece from the environment. The path generation module is used to extract the edges of the point cloud model of the target workpiece and generate the adhesive application path based on the edge extraction results. The glue application detection module is used to control the robotic arm to perform automatic glue application based on the glue application path, and to detect the uniformity of the glue strip thickness to complete the automatic glue application operation.

Images