System and method for intelligent plug-in mounting self-checking of line sequence of humanoid robot

Abstract

The invention discloses a system and method for intelligent plug-in mounting and line sequence self-checking of a humanoid robot. The system comprises a mechanical arm, an end effector, a laser projection system, a mobile vision system and a six-dimensional force sensor. According to the system, laser guiding and SIFT feature recognition are adopted, and submillimeter-level automatic pre-alignment of a mechanical arm and an actuator is achieved; in the plugging process, process monitoring is carried out through six-dimensional force and torque data in combination with a standard curve and a DTW algorithm, and plugging abnormity is found in real time. After plugging is completed, a sweep frequency signal is injected, a poor contact point is accurately detected through impedance phase shift and amplitude analysis, and a defect position is delayed and positioned through signal reflection. For poor contact points, the system adopts a compensation path based on the Hausdorff distance, and drives the shape memory alloy to automatically repair, so that high-precision line sequence self-inspection and fault repair are realized. According to the system, the assembly accuracy and efficiency are greatly improved, the rework rate and the manufacturing cost are reduced, and the intelligence and reliability of the humanoid robot production process are remarkably enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial robot automation, in particular to a system and method for intelligent plug-in self-checking line sequence of humanoid robots. BACKGROUND

[0002] As high-end intelligent equipment, the limb structure of a humanoid robot is highly complex, usually integrating more than 200 precision cables and hydraulic pipelines. These components need to be plugged in with sub-millimeter precision to ensure the stability and functional integrity of the system. However, the traditional manual plugging method relies on the experience and manual precision of the operator, resulting in a misdiagnosis rate of more than 15%. Common problems include cable misplacement, loose connection, and other defects. These misdiagnoses not only affect assembly efficiency, but also cause the need for post-production rework, resulting in an increase of more than 40% in overall costs. To address such problems, existing visual detection systems can assist in surface inspection, but cannot effectively identify hidden defects such as line sequence misplacement and virtual connection, leading to potential failures being exposed in the later production or actual application, further exacerbating manufacturing risks and economic losses. Existing technology only uses force sensors to determine the completion of plugging, cannot detect line sequence logic, and relies on pre-set paths for compensation, without addressing the pose deviation caused by pin deformation. SUMMARY

[0003] Based on the above analysis, the present application provides a system for intelligent plug-in self-checking line sequence of humanoid robots, which includes a robot, an end effector, and a computer. The computer is connected to the robot, the robot is connected to the mechanical arm, the mechanical arm is connected to the end effector, the computer is connected to a modulator, the end effector is connected to a demodulator, the computer sends a sweep frequency signal to the demodulator through the modulator to detect whether there is a poor contact point between the mechanical arm and the end effector. The impedance of the poor contact point will cause the phase of the sweep frequency signal to move and the amplitude to attenuate. The computer determines whether there is a poor contact point by detecting the phase and amplitude changes of the sweep frequency signal.

[0004] Preferably, the computer locates the poor contact point by calculating the transmission distance of the sweep frequency signal:

[0005] Δt=

[0006] Signal transmission distance=

[0007] Where Δt is the reflection delay, Δφ is the difference between the phase value of the sweep frequency signal at the poor contact point and the phase value of the sweep frequency signal at the normal position, v is the propagation speed of the sweep frequency signal, and f is the frequency of the sweep frequency signal.

[0008] Preferably, the frequency of the sweep signal is 1 kHz to 10 MHz.

[0009] Preferably, a mobile robot vision system is mounted on the robot, the mobile robot vision system is connected to the computer, the mobile robot vision system observes the relative position of the robot arm and the end effector, and the computer adjusts the position of the robot arm according to the relative position to pre-align the robot arm and the end effector.

[0010] Preferably, the mobile robot vision system uses a feature matching algorithm to observe the relative position.

[0011] Preferably, the computer is connected to a laser projection system, the laser projection system generates an optical guide grid on the docking surface between the robot arm and the end effector to assist the mobile robot vision system in observing the relative position of the robot arm and the end effector.

[0012] Preferably, the control accuracy of the laser projection system is 5 μm.

[0013] Preferably, the computer is connected to a six-dimensional force sensor, the six-dimensional force sensor is connected to the robot arm, the six-dimensional force sensor is used to monitor the actual values of the force and torque received by the robot arm during the docking process of the robot arm and the end effector, the theoretical values of the force and torque received by the robot arm are plotted into standard plug-in force curves and standard plug-in torque curves, and the actual values of the force and torque are compared with the standard plug-in force curves and the standard plug-in torque curves. If the actual value of the force or torque exceeds ±10% of the standard plug-in force curve or the standard plug-in torque curve, it is determined that there is an abnormality in the docking process of the robot arm and the end effector and the docking process is terminated.

[0014] Preferably, the force and torque collected by the six-dimensional force sensor include Fx, Fy, Fz, Mx, My and Mz, wherein F is force, M is torque, and x, y and z are x-axis, y-axis and z-axis in Cartesian coordinate system, respectively.

[0015] Preferably, during the plug-in process, if the value of the force exceeds ±10% of the standard plug-in force curve or the value of the torque exceeds ±10% of the standard plug-in torque curve, it is determined that there is an abnormality in the plug-in process and the process is terminated.

[0016] Preferably, the problem that the data of the six-dimensional force and the six-dimensional torque are not synchronized with the standard plug-in force curve and the standard plug-in torque curve in time axis due to the fluctuation of the plug-in speed is solved by a DTW (Dynamic Time Warping) algorithm.

[0017] A method for repairing the poor contact point in the system, if the computer judges that the poor contact point exists, the computer repairs the poor contact point through a dynamic compensation algorithm, and the dynamic compensation algorithm comprises:

[0018] S1: a compensation path is generated, the compensation path calls two parameters, including: the Hausdorff distance between the poor contact point and the correct position and the stiffness of the material of the mechanical arm;

[0019] S2: the compensation path drives the shape memory alloy mechanism integrated on the end effector, the shape memory alloy mechanism has the property of expansion and contraction, and the shape memory alloy mechanism drives the mechanical arm to adjust the position of the poor contact until the mechanical arm is in the correct position

[0020] The beneficial effects of the present application are:

[0021] (1) The present application realizes pre-alignment through a laser projection system and a feature matching algorithm, reduces the misjudgment rate of traditional manual insertion, and improves the assembly accuracy and functional integrity of the limb assembly of the humanoid robot.

[0022] (2) The six-dimensional force sensor is combined with the DTW algorithm to monitor the insertion process in real time, the contact poor point is detected through the sweep frequency signal, the instant recognition of the implicit defect and the automatic repair of the shape memory alloy driving are realized, the demand for rework in the later period is reduced, and the overall manufacturing cost is reduced.

[0023] (3) The system integrates an intelligent fault repair mechanism, quantifies the deviation by using the Hausdorff distance and generates a compensation path, ensures that the line sequence logic and the pose offset problem are effectively solved, improves the production efficiency and long-term operation stability of the humanoid robot, and reduces the potential fault exposure risk. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 The system flowchart in the present application. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Example 1

[0028] like Figure 1 As shown, a system for intelligent insertion and self-testing of a humanoid robot is used to realize the insertion and self-testing between the robot arm and its end effector. The system components are a robot arm, an end effector, and a computer. The computer is connected to the robot via cables, the robot is connected to the robot arm via cables, and the robot arm and end effector are connected by insertion.

[0029] Pre-alignment phase

[0030] The robot is equipped with a mobile robot vision system, which is connected to a computer. The mobile robot vision system observes the relative position of the robotic arm and the end effector. It uses a feature matching algorithm to observe the relative position, and the computer adjusts the position of the robotic arm based on this relative position to perform pre-alignment between the robotic arm and the end effector. Specifically, this includes:

[0031] Image acquisition and preprocessing:

[0032] The mobile robot's vision system captures image data from the end of the wiring harness as the current image, and uses an image of the correctly connected robotic arm and end effector as a reference image.

[0033] Preprocess the current image:

[0034] Grayscale conversion: Converts an image to grayscale.

[0035] Noise Removal: Use a Gaussian filter or median filter to remove noise from the grayscale image.

[0036] Feature detection and matching:

[0037] The SIFT (Scale Invariant Feature Transform) algorithm is used to detect keypoints in different scale spaces. The SIFT algorithm is decomposed into the following four steps:

[0038] Scale-space extremum detection: Searches for image locations across all scales. Keypoints are identified using the difference of Gaussian function, which is invariant to scale and rotation. Extrema are detected using different Gaussian filters in different scale spaces.

[0039] Key point localization: Local extrema detected in the Gaussian filter need to be further verified to be accurately located as key points. The exact location of the extrema is obtained by Taylor series expansion in scale space. If the gray value of the extrema is less than 0.03 or 0.04, the extrema will be ignored.

[0040] The DoG algorithm is used to locate key points, but because it is highly sensitive to boundaries, these boundaries must be removed. Therefore, the Harris corner detection algorithm is used to detect boundaries. In the Harris corner detection algorithm, a key point is considered a boundary when one feature value is significantly larger than another. In the DoG algorithm, if the ratio of the curvature parallel to the boundary to the curvature perpendicular to the boundary at an extreme point is greater than 10, that extreme point is considered a boundary and is ignored. Removing extreme points with grayscale values ​​less than 0.03 or 0.04, as well as boundary extreme points, yields accurate key points.

[0041] Keypoint orientation determination: After the above two steps, these keypoints are scale-invariant. To achieve rotation invariance, each keypoint needs to be assigned an orientation angle. Histograms are used to statistically analyze the gradient magnitude and direction of pixels in the neighborhood of each keypoint. Specifically, the 360° area is divided into 36 pillars, each pillar representing 10°. Within a region with radius r, pixels whose gradient direction lies within a particular pillar are identified. The magnitudes of these pixels in that gradient direction are then summed to determine the pillar height, ultimately achieving rotational invariance for the keypoints.

[0042] Keypoint Description: Through the above steps, each keypoint has been assigned location, scale, and orientation information. Next, a descriptor is created for each keypoint. This descriptor is both distinguishable and invariant to certain variables, such as lighting and viewpoint. The descriptor is a vector used to capture local image features of the keypoint and its surrounding area. By dividing the image region around the keypoint into 16 4×4 blocks, and using an 8-dimensional gradient histogram for each block, the 8-dimensional gradient histograms of the 16 blocks are concatenated end-to-end, resulting in 16×8=128, forming a 128-dimensional feature vector, thus abstracting the image information.

[0043] Then, a brute-force matching method is used to match the key points of each current image with the reference image to achieve pre-alignment of the robotic arm and the end effector.

[0044] In addition, the computer is connected to a laser projection system. The laser projection system generates an optical guidance grid on the docking surface between the robotic arm and the end effector to assist the mobile robot's vision system in observing the relative position of the robotic arm and the end effector. The guidance grid consists of intersecting lines, dot matrix, or custom geometry. The guidance grid can dynamically adjust the density, spacing, and orientation between grids according to real-time needs to provide accurate three-dimensional spatial reference coordinates. The control accuracy of the laser projection system is 5μm.

[0045] Connecting process monitoring

[0046] A computer is connected to a six-dimensional force sensor, which is linked to the robotic arm. The six-dimensional force sensor monitors the actual values ​​of the force and torque experienced by the robotic arm during the docking process with the end effector. The data collected by the six-dimensional force sensor includes Fx, Fy, Fz, Mx, My, and Mz, where F represents force, M represents torque, and x, y, and z are the x-axis, y-axis, and z-axis in a Cartesian coordinate system, respectively. The theoretical values ​​of the force and torque experienced by the robotic arm during the contact, alignment, and insertion phases are plotted into standard docking force curves and standard docking torque curves. The theoretical values ​​refer to the force and torque values ​​under error-free docking conditions. The actual values ​​of force and torque are compared with the standard docking force and torque curves. If the actual value of force or torque exceeds ±10% of the standard docking force or torque curve, an anomaly is identified during the docking process between the robotic arm and the end effector, and the docking process is terminated to ensure docking safety and reliability.

[0047] During the connection process between a robotic arm and an end effector, fluctuations in connection speed are common. For example, due to factors such as robotic arm control precision, load variations, or external interference, the actual operating speed may vary. This can cause the data collected by the six-dimensional force sensor to be out of sync with the standard connection force curve or standard connection torque curve on the time axis. For instance, a process that theoretically takes 5 seconds may actually take 4 or 6 seconds in practice. Small differences in connection speed, such as slight variations in local movements, can lead to significant timing misalignments. Therefore, during the connection process, a DTW (Dynamic Time Warping) algorithm is used to compare the six-dimensional force sensor data with the standard connection force curve or standard connection torque curve in real time to resolve the timing asynchrony problem caused by connection speed fluctuations. Specifically, this includes:

[0048] The DTW algorithm calculates the minimum distance between a six-dimensional force sensor sequence A and a standard insertion force curve or standard insertion torque curve B, allowing for non-linear alignment paths to quantify similarity. Even if velocity fluctuations cause temporal stretching / compression of A, the DTW algorithm can still match corresponding points in A and B. For example, if the insertion phase is prolonged due to slow speed, the DTW algorithm compensates by repeatedly matching certain points without affecting the overall similarity assessment.

[0049] Constructing a multidimensional vector of the standard insertion force / torque curve:

[0050] =[ ]

[0051] The six-dimensional force sensor samples at a frequency of 100Hz to form a real-time sequence. ;

[0052] In sequence A sliding window W is set up. Each time the six-dimensional force sensor scans a new point, it is placed into the sliding window W, while the oldest point is removed to ensure the sequence. Timeliness of data.

[0053] In a six-dimensional scenario, the local distance function d at each time point is expressed using the multidimensional Euclidean formula:

[0054]

[0055] Where k corresponds to Fx, Fy, Fz, Mx, My, and Mz. The DTW distance D is calculated by accumulating the minimum local distance function d, which is the minimum cumulative distance under the optimal alignment path.

[0056] The sequence can be determined based on the DTW distance D. A certain point corresponds to Which point in it?

[0057] Check each pair ( The six-dimensional relative error of the corresponding points:

[0058]

[0059] If any of the six dimensions If the value is greater than 0.1, the connection process is considered abnormal, and the connection is terminated.

[0060] Self-test thread sequence stage

[0061] A computer is connected to a modulator, and the end effector is connected to a demodulator. The computer sends a swept-frequency signal (1kHz to 10MHz) to the demodulator via the modulator to detect any poor contact points between the robotic arm and the end effector. Under ideal conditions, the swept-frequency signal transmits through the conductive lines with good impedance matching and almost no signal loss. However, poor contact points introduce additional impedance, causing phase shifts and amplitude attenuation in the swept-frequency signal. The computer determines the presence of poor contact points by detecting these phase and amplitude changes. The computer locates the poor contact points by calculating the transmission distance of the swept-frequency signal.

[0062] Δt=

[0063] Signal transmission distance =

[0064] Where Δt is the reflection delay, Δϕ is the difference between the phase value of the sweep signal at the poor contact point and the phase value of the sweep signal at the normal position, v is the propagation speed of the sweep signal, and f is the frequency of the sweep signal.

[0065] Δt=

[0066] Signal transmission distance =

[0067] Where Δt is the reflection delay, Δϕ is the phase difference compared to the standard baseline, v is the signal propagation speed, and f is the signal frequency. The standard baseline of the sweep signal refers to the phase value of the input reference signal when the sweep signal is not affected by the impedance of the poor contact point. If a poor contact point exists, the system uses a dynamic compensation system to repair it.

[0068] Example 2

[0069] A method for repairing poor contact points in the system of Embodiment 1: If the computer determines that poor contact points exist, the computer repairs the poor contact points using a dynamic compensation algorithm. The dynamic compensation algorithm includes:

[0070] S1: Generate compensation path. The compensation path calls two parameters, including: Hausdorf distance between the poor contact point and the correct position and the stiffness of the robotic arm material.

[0071] S2: The compensation path drives the shape memory alloy mechanism integrated on the end effector. The shape memory alloy mechanism is flexible and drives the robotic arm to adjust the position of poor contact according to the compensation path until the robotic arm is in the correct position.

[0072] The code for the dynamic compensation algorithm is as follows:

[0073] Python

[0074] if pin_matrix != target_matrix:

[0075] generate_compensation_path(

[0076] deviation= calc_hausdorff_distance(pin_matrix, target_matrix),

[0077] stiffness = current_material_stiffness )

[0079] activate_shape_memory_alloy(SMA) # Drive the fine-tuning mechanism

[0080] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0081] The embodiments of the present invention have been described in detail above. The description of the embodiments above is only for the purpose of helping to understand the system of the present invention and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A system for intelligent insertion and self-testing of assembly lines using a humanoid robot, comprising a robotic arm, an end effector, and a computer, characterized in that: The computer is connected to the robot, the robot is connected to the robotic arm, the robotic arm is connected to the end effector, the computer is connected to a modulator, and the end effector is connected to a demodulator. The computer sends a sweep frequency signal to the demodulator through the modulator to detect whether there is a poor contact point between the robotic arm and the end effector. The impedance of the poor contact point will cause the phase of the sweep frequency signal to shift and the amplitude to attenuate. The computer determines whether there is a poor contact point by detecting the phase and amplitude changes of the sweep frequency signal.

2. The system according to claim 1, characterized in that, The computer locates the poor contact point by calculating the transmission distance of the frequency sweep signal: Δt= Signal transmission distance = Where Δt is the reflection delay, Δϕ is the difference between the phase value of the sweep frequency signal at the point of poor contact and the phase value of the sweep frequency signal at the normal position, v is the propagation speed of the sweep frequency signal, and f is the frequency of the sweep frequency signal.

3. The system according to claim 1, characterized in that, The frequency of the sweep signal is from 1 kHz to 10 MHz.

4. The system according to claim 1, characterized in that, The robot is equipped with a mobile robot vision system, which is connected to the computer. The mobile robot vision system observes the relative position of the robotic arm and the end effector. The computer adjusts the position of the robotic arm based on the relative position to pre-align the robotic arm and the end effector.

5. The system according to claim 4, characterized in that, The mobile robot's vision system uses a feature matching algorithm to observe the relative position.

6. The system according to claim 4, characterized in that, The computer is connected to a laser projection system, which generates an optical guiding grid on the mating surface between the robotic arm and the end effector to assist the mobile robot's vision system in observing the relative positions of the robotic arm and the end effector.

7. The system according to claim 1, characterized in that, The computer is connected to a six-dimensional force sensor, which is connected to the robotic arm. The six-dimensional force sensor is used to monitor the actual values ​​of the force and torque experienced by the robotic arm during the docking process with the end effector. The theoretical values ​​of the force and torque experienced by the robotic arm are plotted into a standard insertion force curve and a standard insertion torque curve. The actual values ​​of the force and torque are compared with the standard insertion force curve and the standard insertion torque curve. If the actual value of the force or torque exceeds ±10% of the standard insertion force curve or the standard insertion torque curve, it is determined that there is an abnormality in the docking process between the robotic arm and the end effector, and the docking process is terminated.

8. The system according to claim 7, characterized in that, The forces and torques collected by the six-dimensional force sensor include Fx, Fy, Fz, Mx, My, and Mz, where F is force, M is torque, and x, y, and z are the x-axis, y-axis, and z-axis in the Cartesian coordinate system, respectively.

9. The system according to claim 7, characterized in that, The computer uses the DTW (Dynamic Time Warping) algorithm to align the actual value with the standard insertion force curve and the standard insertion torque curve to avoid the actual value being out of sync with the standard insertion force curve and the standard insertion torque curve on the time axis due to the speed fluctuation of the robotic arm during the docking process.

10. A method for repairing poor contact points in the system of claim 1, characterized in that, If the computer determines that a contact defect exists, the computer repairs the contact defect using a dynamic compensation algorithm, the dynamic compensation algorithm including: S1: Generate a compensation path, which calls two parameters, including: the Hausdorf distance between the poor contact point and the correct position and the stiffness of the robotic arm material; S2: The compensation path drives a shape memory alloy mechanism integrated on the end effector. The shape memory alloy mechanism is telescopic. The shape memory alloy mechanism drives the robotic arm to adjust the position of poor contact according to the compensation path until the robotic arm is in the correct position.

Images