A screw locking control method for a humanoid robot

Abstract

The application relates to a screw locking control method for a humanoid robot, which comprises the following steps: acquiring a hole position image of a workpiece to be locked and generating hole position positioning data by a visual positioning component, and acquiring a fastened hole position image; acquiring task parameters of screw locking by a task interaction component; generating a screwdriver end control instruction based on the hole position positioning data, the task parameters and the geometric parameters of the screwdriver by a control unit; acquiring working parameters of an automatic screwdriver execution component during real-time operation, and dynamically correcting the screwdriver end control instruction based on the working parameters; performing screw locking operation according to the dynamically corrected screwdriver end control instruction by the automatic screwdriver execution component; and performing reinspection based on the fastened hole position image by the control unit to generate a quality result, so that the locking stability and quality tracing capability of the humanoid robot in a complex screw locking scene can be improved.

Description

Technical Field

[0001] This application relates to the field of screw fastening technology, and in particular to a screw fastening control method for humanoid robots. Background Technology

[0002] Screw fastening is a frequent connection process in electronic assembly, automotive parts maintenance, robot repair, laboratory prototype assembly, and large equipment on-site maintenance. Existing solutions utilize humanoid robots for fastening. These robots, equipped with arms, head or chest sensing capabilities, and waist posture adjustment and movement, can theoretically simulate the "observation-alignment-tooth recognition-screwing-tightening-review" process of manual fastening. However, the visual positioning schemes used typically focus on hole position coordinate recognition, and the torque closed-loop schemes usually focus on torque threshold control of a single motor or electric screwdriver. If the visual coordinates, robot joint states, end effector posture, screwdriver speed, real-time torque, cumulative rotation angle, and axial feed are not time-aligned around the same task number and hole position number, the control unit struggles to distinguish whether the abnormal torque originates from misaligned teeth, hole position deviation, material hardness changes, incorrect screw specifications, bit slippage, or communication synchronization failure.

[0003] Therefore, a control method for locking humanoid robots at complex workstations is needed, which integrates stage state estimation, torque feedback correction, visual re-inspection and judgment, and synchronous communication into a closed-loop control process, enabling humanoid robots to achieve stable, safe, and verifiable locking operations in scenarios with multiple postures, specifications, and holes, as well as traceable quality management. Summary of the Invention

[0004] Based on this, in order to address the problems of poor adaptability of existing screw fastening schemes to non-standard workstations, insufficient coupling between visual positioning and torque control, untimely correction of hole position deviations, single dimension of quality judgment, and difficulty in tracing abnormal screw fastening, it is necessary to provide a screw fastening control method for humanoid robots. The humanoid robot includes a visual positioning component, a task interaction component, a control unit, and an automatic screwdriver execution component mounted on the robot body. The method includes: S1: Obtain the hole position image of the workpiece to be fastened and generate hole position positioning data through the vision positioning component, and obtain the hole position image after fastening; obtain the task parameters for screw fastening through the task interaction component; S2: The control unit generates screwdriver end control commands based on hole positioning data, task parameters, and screwdriver geometric parameters; obtains the working parameters of the automatic screwdriver execution component during real-time operation, and dynamically corrects the screwdriver end control commands based on the working parameters; S3: The automatic screwdriver execution component performs screw fastening operations according to the dynamically corrected screwdriver end control command; S4: The control unit performs a re-inspection based on the image of the tightened hole position to generate a quality result.

[0005] The task parameters include screw specifications, target torque, allowable rotation angle, allowable float height, and number of screws.

[0006] Preferably, the step of acquiring the hole position image of the workpiece to be fastened and generating hole position positioning data includes: The hole position images of the workpiece to be locked are obtained from multiple perspectives. Based on the three-dimensional vision recognition algorithm, the center coordinates and axis direction of the target hole position are established from multiple perspectives. The center coordinates and axis direction of the target hole position from multiple perspectives are then fused across perspectives. The fused center coordinates and axis direction of the target hole position are the hole position positioning data of the target hole position.

[0007] Preferably, the step of generating screwdriver end control commands based on hole positioning data, task parameters, and screwdriver geometric parameters includes: Based on the hole positioning data, determine the position vector of the target hole center in the robot base coordinate system and the direction vector of the hole axis; Based on the screwdriver bit length, installation offset, and axial compliance mechanism compression, determine the coordinate transformation relationship of the screwdriver bit's point of action relative to the end flange of the robot's right arm. With the goal of aligning the bit's point of action with the center of the hole and the bit's axis with the hole's axis, the target joint variables of the right arm and the target pose of the screwdriver's end are solved by combining the right arm dynamics model, joint range of motion, collision constraints, and robot body posture constraints. Based on the current working stage, obtain the reference rotational speed, reference axial feed amount, and target torque from the process template corresponding to the screw specifications; The system generates control commands corresponding to the target pose, reference rotation speed, reference axial feed, and target torque, which serve as control commands for the screwdriver end effector of the automatic screwdriver execution assembly located on the right side of the robot body.

[0008] Preferably, the step of acquiring the working parameters of the automatic screwdriver execution component during real-time operation and dynamically correcting the screwdriver end control command based on the working parameters includes: The working parameters of the automatic screwdriver execution component during real-time operation are obtained, including: cumulative rotation angle, real-time torque, torque change rate, axial feed, feed change rate, axial compression, end posture error, and hole position deviation. The working parameters are synchronized according to the task number, hole position number, serial number and timestamp, and the real-time torque, cumulative rotation angle and axial feed are filtered. Calculate the torque-angle slope based on the relationship between the real-time torque and the cumulative angle within a preset sliding window; Calculate the feed-angle ratio based on the relationship between the axial feed rate and the cumulative rotation angle within the preset sliding window; The current working stage is determined based on the axial compression, real-time torque, torque-angle slope, cumulative angle, feed-angle ratio, end attitude error, and hole position error. The upper and lower envelopes of the expected torque corresponding to the current working stage are invoked, and the signed distance between the real-time torque and the upper or lower envelope of the expected torque is taken as the torque residual. Calculate the theoretical axial feed based on the screw pitch, the starting angle and cumulative angle of the current working stage, and use the difference between the axial feed and the theoretical axial feed as the feed residual. The anomaly attribution results are determined based on working parameters, torque residual, feed residual, end attitude error, and hole position deviation. Based on the anomaly attribution results, correction instructions are generated to dynamically correct the screwdriver end control instructions.

[0009] Preferably, the working stages include the contact stage, tooth recognition stage, screwing-in stage, fitting stage, fastening stage, and abnormal stage.

[0010] Preferably, the step of generating correction instructions based on the anomaly attribution results to dynamically correct the screwdriver end control instructions includes: When the torque residual increases, the end attitude error increases, and the feed residual does not meet the preset requirements, the abnormal attribution result is determined to be helical tooth or orifice interference, and correction commands such as reducing the speed, reducing the axial pressure, reversing the tool, or repositioning are generated. When the cumulative rotation angle continues to increase, the real-time torque continues to be lower than the lower limit and the axial feed remains unchanged, the abnormal attribution result is determined to be stripped thread, free spin, or screwdriver tip not engaging, and a correction instruction is generated to stop rotation, recheck working parameters, or pick up the screw again. When the real-time torque reaches the expected torque but the cumulative rotation angle and axial feed do not meet the preset requirements, the abnormal attribution result is floating lock, foreign objects in the hole or mismatch of screw length, generating a correction command for tool retraction and relocking or triggering a re-inspection process. When the hole position deviation is less than the first threshold, the abnormal attribution result is determined to be a slight hole position deviation, and a correction instruction is generated to perform end fine-tuning based on the hole position deviation along the tangential direction of the workpiece surface; when the hole position deviation is between the first threshold and the second threshold, the abnormal attribution result is determined to be a moderate hole position deviation, and a correction instruction is generated to stop rotation and replan the end pose; when the hole position deviation is greater than the second threshold, the abnormal attribution result is determined to be a severe hole position deviation, and a correction instruction is generated to stop the target hole position locking and output a manual verification mark.

[0011] Preferably, S4 includes: The re-inspection image features are extracted from the hole position image after tightening. The re-inspection image features include the height of the screw head relative to the workpiece surface, the eccentricity of the screw head center relative to the hole position center coordinates, the tilt angle of the screw head plane relative to the workpiece plane, the integrity of the screw head edge contour, the degree of damage to the screw head groove, the area of ​​scratches or indentations around the hole, the probability of missing lock, and the re-inspection confidence level. The quality result is determined based on the image features, peak torque, final stable torque, final rotation angle, final feed amount, torque residual, feed residual, end attitude error, and hole position deviation. The quality result includes qualified, automatically reworkable, requiring manual review, or not suitable for further fastening. When the quality result is qualified, the next hole position is automatically locked. When the quality result indicates that automatic rework is possible, based on the abnormal attribution result, choose to perform secondary tightening, tool re-locking, re-picking up the screw, or replacing the screwdriver. When the quality result requires manual review, the task interaction component displays the current hole number, the hole image after tightening, and the anomaly attribution result. When the quality result indicates that the fastening process cannot continue, the current hole position is locked and automatic fastening of the next hole position is prohibited until manual confirmation or the fastening process is released.

[0012] Preferably, it also includes recording the working information for each hole position. The working information includes the center coordinates of the hole position, the direction of the hole axis, the positioning confidence, the screw specification, the target torque, the peak torque, the final stable torque, the cumulative rotation angle, the axial feed, the locking time, the stage state sequence, the dynamic correction action, the re-inspection image features, the quality results, and the anomaly attribution results. The working information is then stored in association with the task number, the hole position number, and the timestamp.

[0013] Preferably, it also includes: encapsulating hole positioning data, screwdriver end control commands, and real-time working parameters into a synchronous communication frame.

[0014] Beneficial effects: This method acquires hole position images of the workpiece to be fastened and generates hole position positioning data through the visual positioning component, and acquires hole position images after fastening; acquires screw fastening task parameters through the task interaction component; generates screwdriver end control commands based on hole position positioning data, task parameters, and screwdriver geometric parameters through the control unit; acquires the real-time operating parameters of the automatic screwdriver execution component, and dynamically corrects the screwdriver end control commands based on the operating parameters; performs screw fastening operations through the automatic screwdriver execution component according to the dynamically corrected screwdriver end control commands; and performs re-inspection based on the fastened hole position images through the control unit to generate quality results, which can improve the fastening stability and quality traceability capability of humanoid robots in complex screw fastening scenarios. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the humanoid robot in an embodiment of this application.

[0017] Figure 2 This is a flowchart of a screw fastening control method for a humanoid robot, as described in this application.

[0018] Figure 3 This is a schematic diagram illustrating the handling of various abnormal situations in the embodiments of this application.

[0019] Figure 4 This is a schematic diagram of dynamic correction of hole position deviation in an embodiment of this application.

[0020] Figure 5 This is a schematic diagram of the torque-rotation angle curve in an embodiment of this application.

[0021] Figure 6 This is a schematic diagram of an optional format of the synchronization communication frame in the embodiments of this application. Detailed Implementation

[0022] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0023] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0024] like Figure 1 As shown, this embodiment provides a humanoid robot for screw fastening control, including: The robot body and the vision positioning component, task interaction component, control unit, and automatic screwdriver execution component set on the robot body.

[0025] Specifically, the robot body includes a torso, waist, two arms, two hands or end effector interfaces, legs or a mobile chassis, and a robot main controller. Preferably, the humanoid robot has the ability to adjust its waist pitch or yaw, each arm has no less than five degrees of freedom, the right arm end effector can carry an automatic screwdriver actuator and maintain axial clamping, and the left arm or head / chest can perform close-range or global observation of hole positions.

[0026] A vision positioning component is installed on the left hand, left wrist, left forearm, head, or chest; an automatic screwdriver execution component is installed on the right hand, right wrist, or right forearm; a task interaction component is installed on the chest, head, or remote interface; and a control unit is installed on the back or waist. The control unit encapsulates the hole position coordinates, hole axis direction, positioning confidence, task parameters, robot joint state, end effector posture, screwdriver speed, real-time torque, cumulative rotation angle, axial feed, stage status, and exception code into a synchronous communication frame, forming multi-source data alignment for the fastening process.

[0027] In this embodiment, the visual positioning component may include one or more of the following: an RGB industrial camera, a global shutter camera, a depth camera, a binocular camera, a structured light camera, a ToF depth sensor, a laser ranging unit, a ring-shaped supplementary lighting component, a polarizing filter, and a dustproof transparent protective window. A left-hand or left-wrist visual component is used for local fine positioning, while a head or chest-mounted visual component is used for global coarse positioning and work area monitoring.

[0028] Furthermore, the humanoid robot also includes a safety and human-robot collaboration module electrically connected to the control unit. This module can perform graded safety control based on factors such as operator distance, robot foot stability, waist tilt angle, joint load, end effector speed, axial clamping force, and screwdriver rotation status. When a person is detected entering the protected area, the end effector speed exceeds a threshold, joint torque is abnormal, or communication synchronization fails, the control unit can reduce the overall speed, stop the screwdriver rotation, release axial pressure, hold the end effector, or perform a safe screwdriver retraction.

[0029] In some alternative embodiments, the humanoid robot preferably has a standing height of 1200 mm to 1800 mm to limit the humanoid platform's ability to approach workpieces and adjust its posture; a body width of 350 mm to 600 mm and a body thickness of 150 mm to limit its spatial adaptability to non-standard workstations, sidewall holes, and deep cavity holes; a total weight of 30 kg to 80 kg; and an effective operating radius of 300 mm to 750 mm per arm to match the common operating range of manual close-range attachment; and a sustainable load of 1 kg to 5 kg per arm to match servo motor electric screwdrivers, cameras, and sensor components. For example, a small humanoid robot configuration can have a standing dimensions of approximately 1320 mm × 450 mm × 200 mm, a weight of approximately 35 kg with battery, a total degree of freedom of 23 to 43, a single arm degree of freedom of 5 or more, and an arm span of approximately 0.45 m.

[0030] The body length of the automatic screwdriver actuator is preferably between 120 mm and 260 mm, and the mounting outer diameter is preferably between 20 mm and 50 mm. The overall mass is matched to the single-arm load to accommodate the load at the right arm end and the bit axis compensation. The coordinates of the screwdriver bit center point should be compensated based on the bit length, bit diameter, quick-change interface offset, axial compliance mechanism compression, and the right arm end coordinate system, so that the planned end pose of the right arm corresponds to the actual point of action of the bit, rather than just the center of the robot flange.

[0031] Screw sizes are preferably ranging from M1.4 to M8, and screw lengths are preferably from 3 mm to 80 mm to cover electronic assembly, robot repair, and structural component assembly. Target torques of 0.02 N·m to 0.2 N·m can be used for micro screws, 0.2 N·m to 1.5 N·m for small screws, 1.5 N·m to 6 N·m for medium screws, and 6 N·m to 12 N·m for larger bolts. These torque ranges can be templated and configured based on thread diameter, pitch, material, hole type, and assembly grade.

[0032] The preferred rotation speed for the tooth recognition stage is 30 r / min to 150 r / min, the preferred rotation speed for the screwing-in stage is 150 r / min to 800 r / min, and the preferred rotation speed for the tightening stage is 20 r / min to 120 r / min, to support stage state estimation and segmented control. The preferred axial clamping force is 2 N to 80 N (preferably 5 to 40 N), to support tooth recognition stability, screwing-in consistency, and floating lock identification. Small screws require smaller axial pressure, while structural connecting bolts require larger axial pressure. The visual positioning accuracy target can be ±0.05 mm to ±0.50 mm, and the hole axis angle error threshold can be 0.5° to 5°, to support hole position residuals, posture residuals, and visual re-inspection judgment.

[0033] The preferred synchronous communication cycle is 1 ms to 20 ms. Key torque feedback, stage status, and abnormal alarm data are preferably transmitted with a cycle of 5 ms to 10 ms to support real-time torque feedback, dynamic correction, and safe tool retraction. The task data recording cycle can be 10 ms to 100 ms. For high-priority torque mutations, over-tightening risks, and emergency stop events, the control unit prioritizes processing based on the sequence number, timestamp, and priority field to avoid low-frequency recording tasks affecting real-time shutdown or tool retraction actions.

[0034] Before automatic fastening, the system performs initialization and calibration checks: the control unit performs self-tests on the vision positioning component, automatic screwdriver execution component, axial compliance mechanism, torque sensor, angle encoder, robot main controller, and synchronous communication link, and reads camera intrinsic parameters, hand-eye calibration matrix, bit length, screwdriver axis offset, right arm kinematic model, and safety threshold. If any key calibration version does not match the task parameters, the automatic fastening process is stopped.

[0035] like Figure 2 As shown, this embodiment provides a screw fastening control method for a humanoid robot. The method employs the aforementioned humanoid robot and includes: S1: Obtain the hole position image of the workpiece to be fastened and generate hole position positioning data through the vision positioning component, and obtain the hole position image after fastening; obtain the task parameters for screw fastening through the task interaction component.

[0036] It is worth noting that the step of acquiring the hole position image of the workpiece to be locked and generating hole position positioning data includes: The hole position images of the workpiece to be locked are obtained from multiple perspectives. Based on the three-dimensional vision recognition algorithm, the center coordinates and axis direction of the target hole position are established from multiple perspectives. The center coordinates and axis direction of the target hole position from multiple perspectives are then fused across perspectives. The fused center coordinates and axis direction of the target hole position are the hole position positioning data of the target hole position.

[0037] In this embodiment, the vision positioning component outputs the center coordinates of the hole, the outline of the hole edge, the estimated hole diameter, the direction of the hole axis, the local normal vector of the workpiece, the positioning confidence level, the occlusion ratio, the reflection status, the acquisition timestamp, the coordinate system identifier, and the hole version number. For inclined holes or sidewall holes, the control unit can estimate the direction of the hole axis based on the major and minor axes of the hole edge ellipse, the local point cloud normal vector, and the workpiece plane fitting result; for deep cavities or occluded holes, the three-dimensional position can be obtained through multi-view sampling and confidence-weighted fusion.

[0038] Furthermore, the visual positioning component on the head or chest acquires the overall posture of the workpiece and the boundary of the working area, while the visual positioning component on the left hand or wrist performs local precision positioning near the target hole. The control unit fuses the hole center, hole axis direction, and positioning reliability from multiple perspectives to generate hole positioning data with timestamps and coordinate system identifiers.

[0039] In this embodiment, the task parameters include screw specifications, target torque, allowable rotation angle, allowable float height, and number of screws.

[0040] Furthermore, the task interaction component obtains the product model, hole position list, screw specifications, target torque, allowable rotation angle range, allowable float, allowable eccentricity, number of locks, material type, and quality grade. The task parsing node maps the above parameters to the rotational speed, torque limit, axial clamping force, feed threshold, residual threshold, and re-inspection threshold for the tooth recognition, screwing, fitting, tightening, and re-inspection stages.

[0041] Task parameters are stored according to screw specifications, pitch, length, head type, material, hole type, assembly level, and workpiece material. Each template includes the thread engagement speed, thread insertion speed, tightening speed, target torque, upper limit of torque, allowable rotation angle range, axial clamping force range, allowable float, allowable eccentricity, allowable tilt angle, residual threshold, and re-inspection threshold. The parameter descriptions are shown in Table 1.

[0042] Table 1 Example of Task Parameters

[0043] S2: The control unit generates screwdriver end control commands based on hole positioning data, task parameters, and screwdriver geometric parameters; obtains the working parameters of the automatic screwdriver execution component during real-time operation, and dynamically corrects the screwdriver end control commands based on the working parameters.

[0044] In this embodiment, the control unit can integrate an edge computing board, a real-time controller, a robot communication gateway, a screwdriver driver, a power management module, an I / O expansion module, a vision computing module, a data storage module, a safety shutdown module, a wireless communication module, a heat dissipation component, and a wiring harness management component. The software is divided into vision nodes, task parsing nodes, motion planning nodes, synchronous communication nodes, torque feedback nodes, stage state estimation nodes, dynamic correction nodes, re-inspection and judgment nodes, safety monitoring nodes, and data recording nodes.

[0045] The stage state estimation node is used to determine the current clamping stage and its confidence level based on real-time torque, cumulative rotation angle, axial feed, end attitude error, and hole position deviation; the dynamic correction node is used to output end fine-tuning, speed reduction, torque limiting, pressure reduction, pressure increase, pulse screwing in, reverse tool retraction, or re-visual positioning commands based on stage residuals and anomaly types; the re-inspection judgment node is used to generate a quality judgment vector and output the quality results.

[0046] Specifically, the step of generating screwdriver end control commands based on hole positioning data, task parameters, and screwdriver geometric parameters includes: Based on the hole positioning data, determine the position vector of the target hole center in the robot base coordinate system and the direction vector of the hole axis; Based on the screwdriver bit length, installation offset, and axial compliance mechanism compression, determine the coordinate transformation relationship of the screwdriver bit's point of action relative to the end flange of the robot's right arm. With the goal of aligning the bit's point of action with the center of the hole and the bit's axis with the hole's axis, the target joint variables of the right arm and the target pose of the screwdriver's end are solved by combining the right arm dynamics model, joint range of motion, collision constraints, and robot body posture constraints. Based on the current working stage, obtain the reference rotational speed, reference axial feed amount, and target torque from the process template corresponding to the screw specifications; The system generates control commands corresponding to the target pose, reference rotation speed, reference axial feed, and target torque, which serve as control commands for the screwdriver end effector of the automatic screwdriver execution assembly located on the right side of the robot body.

[0047] It is worth noting that the specific solution process for the target joint and target pose of the right arm includes: First, based on the coordinate system identifier and acquisition timestamp of the visual positioning component, the coordinates of the hole center and the direction of the hole axis are transformed to the robot base coordinate system; Then, based on the bit length, bit installation offset, quick-change interface offset, and axial compliance mechanism compression, calculate the coordinate transformation relationship between the actual point of action of the bit and the right arm end flange. With the actual point of action of the bit aligned with the center of the hole and the bit axis collinear with the hole axis as the geometric target, the target pose of the bit is constructed, and the target pose of the right arm end flange is obtained inversely through the coordinate transformation relationship. Under the conditions of satisfying the constraints of joint range of motion, joint velocity, joint torque, collision distance, singular configuration, trunk posture and end-effector load, the target joint variables are solved by using right arm inverse kinematics or constraint optimization algorithm, and the continuous end-effector trajectory between the pre-approach, alignment, contact, tooth recognition, screwing in, fitting, fastening and retraction stages is generated.

[0048] Furthermore, the control unit calculates the target pose of the bit center point based on the hole center coordinates, hole axis direction, bit geometry parameters, right arm kinematic model, and torso posture constraints, ensuring that the angle between the bit axis and the hole axis is below a preset threshold. If the right arm alone cannot meet the posture constraints, the shoulder, elbow, wrist, waist, or moving chassis are coordinated for posture adjustment.

[0049] It is worth noting that the step of acquiring the working parameters of the automatic screwdriver execution component during real-time operation and dynamically correcting the screwdriver end control command based on the working parameters includes: The working parameters of the automatic screwdriver execution component during real-time operation are obtained. The working parameters include: cumulative rotation angle, real-time torque, torque change rate, axial feed amount, feed change rate, axial compression amount, end posture error, and hole position deviation. The end posture error is used to characterize the deviation of the angle between the bit axis and the hole axis, and the hole position deviation is used to characterize the offset between the center of the repositioning hole and the center of the bit. The working parameters are synchronized according to the task number, hole position number, serial number and timestamp, and the real-time torque, cumulative rotation angle and axial feed are filtered. Calculate the torque-angle slope based on the relationship between the real-time torque and the cumulative angle within a preset sliding window; Calculate the feed-angle ratio based on the relationship between the axial feed rate and the cumulative rotation angle within the preset sliding window; The current working stage is determined based on the axial compression, real-time torque, torque-angle slope, cumulative angle, feed-angle ratio, end attitude error, and hole position error. The upper and lower envelopes of the expected torque corresponding to the current working stage are invoked. The signed distance between the real-time torque and the upper or lower envelope of the expected torque is used as the torque residual. The torque residual is used to characterize the degree of deviation of the real-time torque curve from the expected torque envelope. Based on the screw pitch, the starting angle and cumulative angle of the current working stage, the theoretical axial feed is calculated, and the difference between the axial feed and the theoretical axial feed is used as the feed residual. The feed residual is used to characterize the consistency between the cumulative angle and the axial feed. The anomaly attribution results are determined based on working parameters, torque residual, feed residual, end attitude error, and hole position deviation. Based on the anomaly attribution results, correction instructions are generated to dynamically correct the screwdriver end control instructions.

[0050] The control quantities in the revised screwdriver end control command are limited by safety and encapsulated together with the task number, hole number, stage number, serial number and timestamp before being sent to the robot's right arm controller and automatic screwdriver execution component.

[0051] In this embodiment, the working stages include the contact stage, tooth recognition stage, screw-in stage, fitting stage, fastening stage, and abnormal stage.

[0052] It is worth noting that: During the contact phase: the end of the right arm approaches the target hole along the hole axis, and the axial compliance mechanism enters the pre-compression zone. The control unit determines the contact state between the bit and the hole based on the axial compression, end force, hole position deviation, and visual confidence level, and performs millimeter-level or sub-millimeter-level position fine adjustments before and after contact.

[0053] For the tooth-entry stage: the actuator rotates at low speed, low torque, and low axial pressure. The stage state estimation node continuously calculates early torque spikes, angle increases, axial feed changes, and end-effector attitude errors. If there is an angle increase with insufficient feed, an early torque spike, or an increased hole position deviation, it is determined to be a non-entry, oblique, or hole-mouth interference situation, and the process enters a reverse retraction or repositioning procedure.

[0054] For the screw-in phase: After successful tooth recognition, the actuator enters the screw-in phase. The control unit establishes the desired rotation angle-feed relationship and the desired torque envelope, and calculates the residuals of the actual rotation angle, actual feed, and torque curves relative to the desired envelope in real time. When the residual is small, the screw-in phase is maintained; when the residual is moderate, fine adjustments are made to the rotation speed, axial pressure, or end position; when the residual exceeds a threshold, rotation is paused and anomaly attribution is performed.

[0055] For the engagement and tightening phases: When the axial feed approaches the set depth or the torque curve reaches the engagement inflection point, the control unit enters the engagement and tightening phases. The actuator reduces its speed and uses a combined stopping condition based on the target torque, final rotation angle, torque rise slope, and retraction angle. If overtightening is detected, rotation is immediately stopped and some axial pressure is released; if necessary, a small-angle reverse rotation is performed to reduce the risk of thread damage.

[0056] Specifically, such as Figure 3As shown, the step of generating correction instructions based on anomaly attribution results to dynamically correct screwdriver end control instructions includes: When the torque residual increases, the end attitude error increases, and the feed residual does not meet the preset requirements, the abnormal attribution result is determined to be helical tooth or hole interference, and correction commands are generated to reduce the speed, reduce the axial pressure, reverse the tool and reposition it. For the helical tooth, the specific treatment is to first reduce the axial pressure and reverse it 30° to 360°, then retract it 0.5 mm to 5 mm and then re-identify the tooth. When the cumulative rotation angle continues to increase, the real-time torque continues to be lower than the lower limit and the axial feed remains unchanged, the abnormal attribution result is determined to be stripped, free rotation or screwdriver tip not engaging, and a correction instruction is generated to stop rotation, recheck working parameters or pick up the screw again. The specific handling for stripped or bit not engaging is to stop rotation and perform bit status recheck. When the real-time torque exceeds the maximum allowable torque for a continuous preset period, or when the automatic screwdriver's actuator triggers overcurrent, stall, or overload protection, it is directly judged as overtightening. When the real-time torque exceeds the warning torque, and at least a preset number of the following trend characteristics appear simultaneously, such as excessive additional rotation angle after contact, excessive torque-angle slope, torque continuing to rise even after axial feed has stopped, or excessive cumulative tightening work after contact, it is judged as overtightening risk. The specific handling for overtightening is to stop rotation, release axial pressure, and record the overtightening risk mark. When the real-time torque reaches the expected torque but the cumulative rotation angle and axial feed do not meet the preset requirements, the abnormal attribution result is floating lock, foreign objects in the hole or mismatch of screw length, generating a correction command for tool retraction and relocking or triggering a re-inspection process. When the real-time torque suddenly increases and communication data is continuously lost, the abnormality is attributed to feedback out-of-step or actuator protection triggering, thus triggering the re-inspection process.

[0057] like Figure 4 As shown, the method for dynamic correction of hole position deviation is as follows: When the hole position deviation is less than the first threshold, the abnormal attribution result is determined to be a slight hole position deviation. A correction instruction is generated based on the hole position deviation to perform end fine adjustment along the tangential direction of the workpiece surface. Specifically, end fine adjustment of 0.05 mm to 1.0 mm is performed along the tangential direction of the workpiece surface while maintaining low-speed tooth recognition. When the hole position deviation is between the first threshold and the second threshold, the abnormal attribution result is determined to be a medium hole position deviation, and a correction instruction is generated to stop the rotation and replan the end pose. Specifically, the rotation is paused and the wrist, elbow, shoulder or waist are linked to realign the hole axis. When the hole position deviation exceeds the second threshold, the abnormal attribution result is determined to be a serious hole position deviation, and a correction instruction to stop the target hole position locking and output the manual review mark is generated.

[0058] After the correction command is executed, the control unit rereads the hole position positioning data, end attitude data, and torque feedback data, and calculates the residual changes before and after the correction. If the residual decreases and the stage confidence level recovers to above the threshold, the current hole position locking continues; if the residual continues to increase or the number of repeated corrections exceeds the threshold, the automatic locking of the current hole position ends and a rework or manual review mark is output.

[0059] Furthermore, each correction records the triggering stage, triggering residual, anomaly attribution, correction action, correction magnitude, execution time, post-correction residual change, and final quality result. This record can be used for subsequent parameter updates of the same product model and can be used to distinguish between process parameter deviations, workpiece hole position errors, bit wear, and robot positioning errors.

[0060] S3: The automatic screwdriver actuator performs screw fastening operations according to the dynamically modified screwdriver end control command.

[0061] In this embodiment, the automatic screwdriver actuation component is installed on the right hand, right wrist, or right forearm end, and may include one or more of the following: servo electric screwdriver, electric screwdriver, pneumatic screwdriver, brushless motor drive unit, reduction mechanism, quick-change bit interface, screw adsorption or clamping mechanism, axial floating compensation mechanism, torque sensor, angle encoder, motor current detection unit, axial displacement detection unit, and end force / torque detection unit.

[0062] The automatic screwdriver actuator has axial compliance and coaxial compensation capabilities. The axial floating compensation mechanism can output axial compression, contact status, or equivalent feed amount, which the control unit can use to determine whether the workpiece surface has been reached, whether the screws are properly engaged, and whether there is any floating lock, misaligned screws, or jamming.

[0063] Furthermore, the automatic screwdriver actuation assembly holds the screw through vacuum adsorption, magnetic retention, mechanical grippers, elastic sleeves, or guide nozzle structures. The control unit determines whether the screw is secured based on the screw retention sensor, bit end image, mass sensor, or vacuum pressure, and writes the secured status into the data frame of the current hole position.

[0064] S4: The control unit performs a re-inspection based on the image of the tightened hole position to generate a quality result.

[0065] Specifically, the steps include: The re-inspection image features are extracted from the hole position image after tightening. The re-inspection image features include the height of the screw head relative to the workpiece surface, the eccentricity of the screw head center relative to the hole position center coordinates, the tilt angle of the screw head plane relative to the workpiece plane, the integrity of the screw head edge contour, the degree of damage to the screw head groove, the area of ​​scratches or indentations around the hole, the probability of missing lock, and the re-inspection confidence level. Based on the aforementioned image features, peak torque, final stable torque, final rotation angle, final feed rate, torque residual, feed residual, end-point attitude error, hole position deviation, and torque-rotation angle curve (curve shown in Figure 1), Figure 5 (As shown) Determine the quality result, which includes qualified, automatically reworkable, requiring manual review, or unable to continue locking; When the quality result is qualified, the next hole position is automatically locked. When the quality result indicates that automatic rework is possible, based on the abnormal attribution result, choose to perform secondary tightening, tool re-locking, re-picking up the screw, or replacing the screwdriver. When the quality result requires manual review, the task interaction component displays the current hole number, the hole image after tightening, and the anomaly attribution result. When the quality result indicates that the fastening process cannot continue, the current hole position is locked and automatic fastening of the next hole position is prohibited until manual confirmation or the fastening process is released.

[0066] In some optional embodiments, the method further includes recording working information for each hole, including hole center coordinates, hole axis direction, positioning confidence, screw specifications, target torque, peak torque, final stable torque, cumulative rotation angle, axial feed, locking time, stage state sequence, dynamic correction actions, re-inspection image features, quality results, and anomaly attribution results. This working information is then associated with and stored using task number, hole number, and timestamp. After statistically analyzing historical curves for the same product or the same hole, the threshold envelope and stage residual threshold can be updated.

[0067] In some optional embodiments, the method further includes encapsulating hole positioning data, screwdriver end control commands, and real-time operating parameters into a synchronous communication frame.

[0068] Furthermore, synchronization communication frames can operate in UDP, TCP, DDS, ROS 2 QoS, CAN-FD, EtherCAT, RS485, UART, USB, Wi-Fi, or Bluetooth communication modes. A synchronization communication frame includes a frame header, protocol version, device address, task number, aperture number, stage number, data type, coordinate system identifier, data length, sequence number, timestamp, priority, data payload, CRC checksum, and frame trailer. One optional format for synchronization communication frames is as follows: Figure 6 As shown.

[0069] The data payload includes one or more of the following: hole positioning data, end effector pose command, screwdriver control command, torque feedback data, rotation angle data, axial feed data, visual inspection data, stage status data, residual data, quality judgment data, abnormal alarm data, and heartbeat data.

[0070] The control unit processes data according to priority. Torque surges, overtightening risks, personnel entering the protected area, communication timeouts, and emergency stop events are high priority; hole position corrections, stage status updates, and visual re-inspections are medium priority; data statistics, remote uploads, and historical parameter learning are low priority. High-priority events can bypass the normal task queue and directly control the screwdriver to stop rotating, release axial pressure, or perform a safe tool retraction.

[0071] It is worth noting that the triggering conditions for the re-inspection process can be at least one of the following: the current hole position reaches the target torque or a combined shutdown condition, the torque-angle-feed residual exceeds the preset threshold, a reverse retraction occurs during the tooth recognition or screwing-in stage, the positioning reliability is lower than the threshold, there is packet loss or out-of-order communication frames, or the operator issues a re-inspection command through the task interaction component.

[0072] The re-inspection can employ one or more of the following methods: single-view close-range sampling, multi-view surround sampling, and fusion of global head and chest sampling with local left-hand sampling. The control unit selects the observation posture and adjusts the supplementary lighting brightness, exposure time, and camera distance based on the orifice plane normal vector and screw head shape. For reflective or obstructed areas, polarization filtering, low-angle supplementary lighting, or multi-view confidence weighting can be enabled.

[0073] To better illustrate the advantages of this method, this embodiment provides three examples, as follows: 1. A binocular camera and a ring-shaped lighting assembly are installed on the left arm of the humanoid robot, while a servo electric screwdriver with an axial compliant mechanism is installed on the right arm. The tool bag on its back controls the operation of vision nodes, stage state estimation nodes, and re-inspection judgment nodes. The robot first determines the workpiece position through the head and chest vision assembly, then performs local fine positioning of the target hole using the left arm camera, calculating the hole coordinates and hole axis direction. The right arm generates the end-target pose based on the screwdriver bit length and hole axis direction, and performs a secondary visual verification before contact.

[0074] 2. When tightening M3 screws, use low speed and low axial pressure during the thread recognition phase. If the cumulative rotation angle reaches the preset range and the axial feed is below the lower limit, and an early torque spike occurs, the control unit determines that there is a risk of misalignment and executes reverse rotation, backtracking, and thread recognition re-entry. If the torque residual and feed residual decrease after thread recognition, continue tightening; if there are two consecutive abnormalities, lock the current hole position and output a manual verification mark.

[0075] 3. After tightening, the left wrist camera performs a close-up re-inspection of the screw head, extracting the screw head height, center offset, and tilt angle. If the final torque reaches the target range but the screw head height exceeds the allowable value, and the final rotation angle is insufficient, the quality judgment module outputs a floating lock result, and the control unit performs low-speed secondary tightening or tool retraction and relocking according to the process template; if the secondary re-inspection still fails, the abnormal image, curve, and hole position number are retained for manual processing.

[0076] The screw fastening control method for humanoid robots provided in this embodiment has the following beneficial effects: 1. By combining the advantages of humanoid robot dual-arm collaboration with staged locking control, the hole alignment accuracy in non-standard workstations can be improved. 2. By using the torque-angle-feed residual model to attribute anomalies, the risks of stripping, skewed teeth, floating lock, missed lock, and over-tightening are reduced. 3. Avoid relying solely on torque or number of revolutions for quality judgment by using visual re-inspection vectors; 4. Traceable quality data is generated by recording task number, hole position number, timestamp, serial number, and locking curve. 5. Applicable to mobile robot platforms with similar dual-arm collaborative operation capabilities.

[0077] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A screw fastening control method for a humanoid robot, the humanoid robot comprising a vision positioning component, a task interaction component, a control unit, and an automatic screwdriver execution component disposed on the robot body, characterized in that, include: S1: Obtain the hole position image of the workpiece to be fastened and generate hole position positioning data through the vision positioning component, and obtain the hole position image after fastening; obtain the task parameters for screw fastening through the task interaction component; S2: The control unit generates screwdriver end control commands based on hole positioning data, task parameters, and screwdriver geometric parameters; obtains the working parameters of the automatic screwdriver execution component during real-time operation, and dynamically corrects the screwdriver end control commands based on the working parameters; S3: The automatic screwdriver execution component performs screw fastening operations according to the dynamically corrected screwdriver end control command; S4: The control unit performs a re-inspection based on the image of the tightened hole position to generate a quality result.

2. The method according to claim 1, characterized in that, The task parameters include screw specifications, target torque, allowable rotation angle, allowable float height, and number of screws.

3. The method according to claim 1, characterized in that, The step of acquiring the hole position image of the workpiece to be locked and generating hole position positioning data includes: The hole position images of the workpiece to be locked are obtained from multiple perspectives. Based on the three-dimensional vision recognition algorithm, the center coordinates and axis direction of the target hole position are established from multiple perspectives. The center coordinates and axis direction of the target hole position from multiple perspectives are then fused across perspectives. The fused center coordinates and axis direction of the target hole position are the hole position positioning data of the target hole position.

4. The method according to claim 2, characterized in that, The process of generating screwdriver end control commands based on hole positioning data, task parameters, and screwdriver geometry parameters includes: Based on the hole positioning data, determine the position vector of the target hole center in the robot base coordinate system and the direction vector of the hole axis; Based on the screwdriver bit length, installation offset, and axial compliance mechanism compression, determine the coordinate transformation relationship of the screwdriver bit's point of action relative to the end flange of the robot's right arm. With the goal of aligning the bit's point of action with the center of the hole and the bit's axis with the hole's axis, the target joint variables of the right arm and the target pose of the screwdriver's end are solved by combining the right arm dynamics model, joint range of motion, collision constraints, and robot body posture constraints. Based on the current working stage, obtain the reference rotational speed, reference axial feed amount, and target torque from the process template corresponding to the screw specifications; The system generates control commands corresponding to the target pose, reference rotation speed, reference axial feed, and target torque, which serve as control commands for the screwdriver end effector of the automatic screwdriver execution assembly located on the right side of the robot body.

5. The method according to claim 2, characterized in that, The step of acquiring the real-time operating parameters of the automatic screwdriver execution component and dynamically correcting the screwdriver end control commands based on the operating parameters includes: The working parameters of the automatic screwdriver execution component during real-time operation are obtained, including: cumulative rotation angle, real-time torque, torque change rate, axial feed amount, feed change rate, axial compression amount, end posture error, and hole position deviation. The working parameters are synchronized according to the task number, hole position number, serial number and timestamp, and the real-time torque, cumulative rotation angle and axial feed are filtered. Calculate the torque-angle slope based on the relationship between the real-time torque and the cumulative angle within a preset sliding window; Calculate the feed-angle ratio based on the relationship between the axial feed rate and the cumulative rotation angle within the preset sliding window; The current working stage is determined based on the axial compression, real-time torque, torque-angle slope, cumulative angle, feed-angle ratio, end attitude error, and hole position error. The upper and lower envelopes of the expected torque corresponding to the current working stage are invoked, and the signed distance between the real-time torque and the upper or lower envelope of the expected torque is taken as the torque residual. Calculate the theoretical axial feed based on the screw pitch, the starting angle and cumulative angle of the current working stage, and use the difference between the axial feed and the theoretical axial feed as the feed residual. The anomaly attribution results are determined based on working parameters, torque residual, feed residual, end attitude error, and hole position deviation. Based on the anomaly attribution results, correction instructions are generated to dynamically correct the screwdriver end control instructions.

6. The method according to claim 5, characterized in that, The working stages include the contact stage, tooth recognition stage, screwing-in stage, fitting stage, fastening stage, and abnormality stage.

7. The method according to claim 5, characterized in that, The step of generating correction instructions based on anomaly attribution results to dynamically correct screwdriver end control instructions includes: When the torque residual increases, the end attitude error increases, and the feed residual does not meet the preset requirements, the abnormal attribution result is determined to be helical tooth or orifice interference, and correction commands such as reducing the speed, reducing the axial pressure, reversing the tool, or repositioning are generated. When the cumulative rotation angle continues to increase, the real-time torque continues to be lower than the lower limit and the axial feed remains unchanged, the abnormal attribution result is determined to be stripped thread, free spin, or screwdriver tip not engaging, and a correction instruction is generated to stop rotation, recheck working parameters, or pick up the screw again. When the real-time torque reaches the expected torque but the cumulative rotation angle and axial feed do not meet the preset requirements, the abnormal attribution result is floating lock, foreign objects in the hole or mismatch of screw length, generating a correction command for tool retraction and relocking or triggering a re-inspection process. When the hole position deviation is less than the first threshold, the abnormal attribution result is determined to be a slight hole position deviation, and a correction instruction is generated to perform end fine-tuning based on the hole position deviation along the tangential direction of the workpiece surface; when the hole position deviation is between the first threshold and the second threshold, the abnormal attribution result is determined to be a moderate hole position deviation, and a correction instruction is generated to stop rotation and replan the end pose; when the hole position deviation is greater than the second threshold, the abnormal attribution result is determined to be a severe hole position deviation, and a correction instruction is generated to stop the target hole position locking and output a manual verification mark.

8. The method according to claim 5, characterized in that, S4 includes: The re-inspection image features are extracted from the hole position image after tightening. The re-inspection image features include the height of the screw head relative to the workpiece surface, the eccentricity of the screw head center relative to the hole position center coordinates, the tilt angle of the screw head plane relative to the workpiece plane, the integrity of the screw head edge contour, the degree of damage to the screw head groove, the area of ​​scratches or indentations around the hole, the probability of missing lock, and the re-inspection confidence level. The quality result is determined based on the image features, peak torque, final stable torque, final rotation angle, final feed amount, torque residual, feed residual, end attitude error, and hole position deviation. The quality result includes qualified, automatically reworkable, requiring manual review, or not suitable for further fastening. When the quality result is qualified, the next hole position is automatically locked. When the quality result indicates that automatic rework is possible, based on the abnormal attribution result, choose to perform secondary tightening, tool re-locking, re-picking up the screw, or replacing the screwdriver. When the quality result requires manual review, the task interaction component displays the current hole number, the hole image after tightening, and the anomaly attribution result. When the quality result indicates that the fastening process cannot continue, the current hole position is locked and automatic fastening of the next hole position is prohibited until manual confirmation or the fastening process is released.

9. The method according to claim 8, characterized in that, It also includes recording the working information for each hole, including the center coordinates of the hole, the direction of the hole axis, the positional reliability, the screw specification, the target torque, the peak torque, the final stable torque, the cumulative rotation angle, the axial feed, the locking time, the stage state sequence, the dynamic correction action, the re-inspection image features, the quality results, and the anomaly attribution results. The working information is then stored in association with the task number, the hole number, and the timestamp.

10. The method according to claim 1, characterized in that, Also includes: The hole positioning data, screwdriver end control commands, and real-time working parameters are encapsulated into a synchronous communication frame.

Images