Robot control method, device and equipment, robot and medium

Abstract

The invention provides a robot control method, device and equipment, a robot and a medium. The robot comprises a mechanical arm; the robot control method comprises the steps that according to a workpiece image of a target workpiece, target pose information of a target hole of the target workpiece is determined; according to the target pose information, multiple actual contact forces between the tail end of the mechanical arm and the target workpiece based on the first direction are collected; according to the multiple actual contact forces, it is determined that the tail end of the mechanical arm is aligned with the target hole; and the mechanical arm is controlled to conduct shaft hole assembling operation on the target hole. The technical problem that in the prior art, shaft hole alignment cannot be accurately achieved, and therefore the shaft hole assembling effect is affected is solved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a robot control method, device, equipment, robot, and medium. Background Art

[0002] With the rapid development of science and technology, robots are increasingly being used to replace manual labor in more and more scenarios. As a result, robots with various functions have emerged, providing users with a variety of business services. For example, robots can be used to perform shaft hole assembly operations.

[0003] In the related art, visual positioning technology is usually used to locate the holes on the workpiece, and the robot's mechanical arm is controlled to align with the holes according to the positioning results to perform the shaft-hole assembly operation.

[0004] In this way, the shaft-hole alignment cannot be achieved accurately, which will affect the shaft-hole assembly effect. Summary of the Invention

[0005] The present disclosure aims to solve one of the technical problems in the related art at least to a certain extent.

[0006] To this end, the present disclosure proposes a robot control method, a robot control device, an electronic device, a robot, a computer-readable storage medium and a computer program product, which can accurately achieve shaft-hole alignment, thereby improving the shaft-hole assembly effect.

[0007] The first aspect of the present disclosure provides a robot control method, wherein the robot includes: a robotic arm; including: determining target posture information of a target hole of a target workpiece based on a workpiece image of the target workpiece; collecting multiple actual contact forces based on a first direction between the end of the robotic arm and the target workpiece based on the target posture information; determining that the end of the robotic arm is aligned with the target hole based on the multiple actual contact forces; and controlling the robotic arm to perform an axial hole assembly operation on the target hole.

[0008] In some embodiments of the present disclosure, determining target pose information of a target hole of a target workpiece according to a workpiece image of the target workpiece includes:

[0009] Process the workpiece image to obtain a workpiece grayscale image;

[0010] Identify and obtain feature points corresponding to the target hole from the grayscale image of the workpiece;

[0011] According to the feature points, the reference pose information of the target hole in the camera coordinate system is determined, wherein there is a mapping relationship between the camera coordinate system and the base coordinate system of the robot;

[0012] The reference pose information is processed according to the mapping relationship to obtain the target pose information.

[0013] In some embodiments of the present disclosure, collecting a plurality of actual contact forces based on a first direction between an end of a robotic arm and a target workpiece according to target pose information includes:

[0014] Controlling the robotic arm to move to a position corresponding to the target posture information, and collecting an actual contact force based on a first direction between the end of the robotic arm and the target workpiece at a first reference time;

[0015] controlling the movement of the robotic arm in a force control mode according to the actual contact force corresponding to the first reference time;

[0016] Controlling the robot arm to move in a second direction in a position control mode, wherein the first direction and the second direction are perpendicular to each other;

[0017] At the second reference time, the actual contact force between the end of the robotic arm and the target workpiece based on the first direction is collected to obtain the actual contact force corresponding to the second reference time, and the first reference time is updated according to the second reference time until multiple actual contact forces are continuously collected.

[0018] In some embodiments of the present disclosure, controlling the movement of the robotic arm in a force control mode according to the actual contact force corresponding to the first reference time includes:

[0019] determining a position deviation of the end of the robotic arm based on an actual contact force corresponding to the first reference time;

[0020] determining joint differential motions associated with joints of the robotic arm based on the position deviation;

[0021] Determine the current joint angles of the joints of the robotic arm;

[0022] According to the joint differential motion, the current joint angle of the robot arm is controlled to control the movement of the robot arm.

[0023] In some embodiments of the present disclosure, determining the position deviation of the end of the robotic arm according to the actual contact force corresponding to the first reference time includes:

[0024] determining an expected contact force corresponding to a third reference time, an actual contact force corresponding to the third reference time, and a position compensation amount corresponding to the third reference time, wherein the third reference time is a previous reference time of the first reference time;

[0025] determining an update rate based on the actual contact force corresponding to the first reference time;

[0026] determining a position compensation amount corresponding to the first reference time based on the update rate, the expected contact force corresponding to the third reference time, the actual contact force corresponding to the third reference time, and the position compensation amount corresponding to the third reference time;

[0027] A position deviation of the end of the robot arm is determined based on the position compensation amount corresponding to the first reference time.

[0028] In some embodiments of the present disclosure, determining the update rate according to the actual contact force corresponding to the first reference time includes:

[0029] Determine the upper limit of the update rate;

[0030] The update rate is determined according to the update rate upper limit, the actual contact force corresponding to the first reference time, and the expected contact force corresponding to the first reference time.

[0031] In some embodiments of the present disclosure, controlling the robotic arm to move based on the second direction in the position control mode includes:

[0032] Determine the radial velocity and rotational speed of the robot arm;

[0033] The robotic arm is controlled to move in a second direction according to the first reference time and the radial speed and the rotation speed of the robotic arm.

[0034] In some embodiments of the present disclosure, determining that the end of the robotic arm is aligned with the target hole based on multiple actual contact forces includes:

[0035] If each of the multiple actual contact forces is less than or equal to the contact force threshold, and the accumulated sampling duration of the multiple reference times corresponding to the multiple actual contact forces reaches the duration threshold, it is determined that the end of the robotic arm is aligned with the target hole;

[0036] If at least one of the multiple actual contact forces is greater than the contact force threshold, or the cumulative sampling duration of multiple reference times corresponding to the multiple actual contact forces does not reach the duration threshold, the actual contact force between the end of the robot arm and the target workpiece based on the first direction is re-collected to obtain a new actual contact force until it is determined that the end of the robot arm is aligned with the target hole based on the multiple actual contact forces collected continuously.

[0037] In some embodiments of the present disclosure, controlling a robotic arm to perform a shaft-hole assembly operation on a target hole includes:

[0038] Control the movement of the robotic arm in force control mode and control the robotic arm to perform shaft-hole assembly operations on the target hole.

[0039] In some embodiments of the present disclosure, the first direction is the direction indicated by the Z axis of the tool coordinate system of the robotic arm, and the tool coordinate system also includes the X axis and the Y axis. The direction indicated by the X axis and the direction indicated by the Y axis are collectively regarded as the second direction.

[0040] An embodiment of the second aspect of the present disclosure proposes a robot control device, wherein the robot includes: a robotic arm; including: a first determination module, used to determine the target posture information of the target hole of the target workpiece based on the workpiece image of the target workpiece; an acquisition module, used to acquire multiple actual contact forces based on a first direction between the end of the robotic arm and the target workpiece based on the target posture information; a second determination module, used to determine that the end of the robotic arm is aligned with the target hole based on the multiple actual contact forces; and a control module, used to control the robotic arm to perform an axial hole assembly operation on the target hole.

[0041] In some embodiments of the present disclosure, the first determining module is specifically configured to:

[0042] Process the workpiece image to obtain a workpiece grayscale image;

[0043] Identify and obtain feature points corresponding to the target hole from the grayscale image of the workpiece;

[0044] According to the feature points, the reference pose information of the target hole in the camera coordinate system is determined, wherein there is a mapping relationship between the camera coordinate system and the base coordinate system of the robot;

[0045] The reference pose information is processed according to the mapping relationship to obtain the target pose information.

[0046] In some embodiments of the present disclosure, the acquisition module is specifically configured to:

[0047] Controlling the robotic arm to move to a position corresponding to the target posture information, and collecting an actual contact force based on a first direction between the end of the robotic arm and the target workpiece at a first reference time;

[0048] controlling the movement of the robotic arm in a force control mode according to the actual contact force corresponding to the first reference time;

[0049] Controlling the robot arm to move in a second direction in a position control mode, wherein the first direction and the second direction are perpendicular to each other;

[0050] At the second reference time, the actual contact force between the end of the robotic arm and the target workpiece based on the first direction is collected to obtain the actual contact force corresponding to the second reference time, and the first reference time is updated according to the second reference time until multiple actual contact forces are continuously collected.

[0051] In some embodiments of the present disclosure, the acquisition module is specifically configured to:

[0052] determining a position deviation of the end of the robotic arm based on an actual contact force corresponding to the first reference time;

[0053] determining joint differential motions associated with joints of the robotic arm based on the position deviation;

[0054] Determine the current joint angles of the joints of the robotic arm;

[0055] According to the joint differential motion, the current joint angle of the robot arm is controlled to control the movement of the robot arm.

[0056] In some embodiments of the present disclosure, the acquisition module is specifically configured to:

[0057] determining an expected contact force corresponding to a third reference time, an actual contact force corresponding to the third reference time, and a position compensation amount corresponding to the third reference time, wherein the third reference time is a previous reference time of the first reference time;

[0058] determining an update rate based on the actual contact force corresponding to the first reference time;

[0059] determining a position compensation amount corresponding to the first reference time based on the update rate, the expected contact force corresponding to the third reference time, the actual contact force corresponding to the third reference time, and the position compensation amount corresponding to the third reference time;

[0060] A position deviation of the end of the robot arm is determined based on the position compensation amount corresponding to the first reference time.

[0061] In some embodiments of the present disclosure, the acquisition module is specifically configured to:

[0062] Determine the upper limit of the update rate;

[0063] The update rate is determined according to the update rate upper limit, the actual contact force corresponding to the first reference time, and the expected contact force corresponding to the first reference time.

[0064] In some embodiments of the present disclosure, the acquisition module is specifically configured to:

[0065] Determine the radial velocity and rotational speed of the robot arm;

[0066] The robotic arm is controlled to move in a second direction according to the first reference time and the radial speed and the rotation speed of the robotic arm.

[0067] In some embodiments of the present disclosure, the second determining module is configured to:

[0068] If each of the multiple actual contact forces is less than or equal to the contact force threshold, and the accumulated sampling duration of the multiple reference times corresponding to the multiple actual contact forces reaches the duration threshold, it is determined that the end of the robotic arm is aligned with the target hole;

[0069] If at least one of the multiple actual contact forces is greater than the contact force threshold, or the cumulative sampling duration of multiple reference times corresponding to the multiple actual contact forces does not reach the duration threshold, the actual contact force between the end of the robot arm and the target workpiece based on the first direction is re-collected to obtain a new actual contact force until it is determined that the end of the robot arm is aligned with the target hole based on the multiple actual contact forces collected continuously.

[0070] In some embodiments of the present disclosure, the control module is specifically configured to:

[0071] Control the movement of the robotic arm in force control mode and control the robotic arm to perform shaft-hole assembly operations on the target hole.

[0072] In some embodiments of the present disclosure, the first direction is the direction indicated by the Z axis of the tool coordinate system of the robotic arm, and the tool coordinate system also includes the X axis and the Y axis. The direction indicated by the X axis and the direction indicated by the Y axis are collectively regarded as the second direction.

[0073] The electronic device proposed in the third embodiment of the present disclosure includes: a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the robot control method proposed in the first embodiment of the present disclosure.

[0074] The fourth embodiment of the present disclosure proposes a robot, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the robot control method proposed in the first embodiment of the present disclosure is implemented.

[0075] The fifth embodiment of the present disclosure proposes a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robot control method proposed in the first embodiment of the present disclosure.

[0076] The sixth embodiment of the present disclosure proposes a computer program product. When the instructions in the computer program product are executed by a processor, the robot control method proposed in the first embodiment of the present disclosure is executed.

[0077] The robot control method, robot control device, electronic device, robot, computer-readable storage medium, and computer program product provided herein determine target pose information of a target hole of a target workpiece based on a workpiece image of the target workpiece, collect multiple actual contact forces between the end of a robotic arm and the target workpiece in a first direction based on the target pose information, determine that the end of the robotic arm is aligned with the target hole based on the multiple actual contact forces, and control the robotic arm to perform a shaft-hole assembly operation on the target hole. This method can accurately achieve shaft-hole alignment, thereby improving the shaft-hole assembly effect.

[0078] Additional aspects and advantages of the present disclosure will be given in part in the description below and in part will be obvious from the description below, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0080] Figure 1 A schematic flow chart of a robot control method provided by an embodiment of the present disclosure;

[0081] Figure 2 is a flow chart of the visual positioning algorithm in the embodiment of the present disclosure;

[0082] Figure 3 A flowchart of another robot control method provided by an embodiment of the present disclosure;

[0083] Figure 4 is a schematic diagram of the robot control process in an embodiment of the present disclosure;

[0084] Figure 5 A schematic structural diagram of a robot control device provided by an embodiment of the present disclosure;

[0085] Figure 6 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown;

[0086] Figure 7 A schematic structural diagram of a robot provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0087] Some embodiments of the present disclosure will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but may be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, for the sake of clarity and brevity, descriptions of features known in the art may be omitted.

[0088] The embodiments described in the following examples of the present disclosure do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0089] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0090] Figure 1 A flowchart of a robot control method provided by an embodiment of the present disclosure.

[0091] This embodiment is described by taking the robot control method as an example in which the robot control device is configured.

[0092] In this embodiment, the robot control method can be configured in a robot control device, which can be set in a server, or can also be set in a mobile device. The mobile device can be, for example, a terminal, a robot, etc., and this embodiment of the present disclosure does not limit this.

[0093] The robot in this embodiment may include: a robotic arm, which can be used to replace manual operation to perform corresponding operations.

[0094] It should be noted that the execution entity of this embodiment can be, for example, a central processing unit (CPU) in a server or mobile device in terms of hardware, and can be, for example, a related background service in a server or mobile device in terms of software, without limitation.

[0095] like Figure 1 As shown, the robot control method includes:

[0096] S101: Determine target position and posture information of a target hole of the target workpiece according to a workpiece image of the target workpiece.

[0097] The workpiece to be assembled with the shaft and hole can be referred to as the target workpiece, and the hole on the target workpiece into which the shaft is to be assembled can be referred to as the target hole. The number of target holes can be one or more, and there is no limitation on this.

[0098] The image of the target workpiece captured by the camera device can be called a workpiece image, and the relative position and posture information of the target hole on the target workpiece can be called target posture information.

[0099] In some embodiments, a workpiece image of the target workpiece may be captured, and image recognition may be performed on the workpiece image to identify the target hole therefrom, and target position information of the target hole relative to the target workpiece may be analyzed, without limitation.

[0100] In some embodiments, the workpiece image may be input into a posture recognition model, and target posture information output by the posture recognition model may be obtained, without limitation.

[0101] Of course, any other possible method may be used to determine the target position information of the target hole of the target workpiece based on the workpiece image of the target workpiece, such as automatic calibration, mathematical analysis, etc.

[0102] Optionally, in some embodiments, in the process of determining the target pose information of the target hole of the target workpiece based on the workpiece image of the target workpiece, the workpiece image can be processed to obtain a workpiece grayscale image, and feature points corresponding to the target hole can be identified from the workpiece grayscale image. Based on the feature points, reference pose information of the target hole in the camera coordinate system is determined, wherein a mapping relationship exists between the camera coordinate system and the base coordinate system of the robot, and the reference pose information is processed based on the mapping relationship to obtain the target pose information. Since the target pose information of the target hole is identified by reference to the workpiece image through a combined image processing and camera calibration method, the accuracy of target pose information recognition can be effectively improved, supporting the improvement of shaft hole alignment accuracy.

[0103] In some embodiments, a workpiece image can be captured based on a camera device, where the workpiece image is an RGB image (an RGB image is an image composed of three color channels: red (Red, R), green (Green, G), and blue (Blue, B)). Then, a Gaussian filter is performed on the RGB image to convert the RGB image into a workpiece grayscale image.

[0104] In some embodiments, such as Figure 2 As shown, Figure 2 This is a flow chart of the visual positioning algorithm in the embodiment of the present disclosure. It can perform edge detection or corner detection on the grayscale image of the workpiece to extract the feature points corresponding to the target hole, and use the pose estimation algorithm to process the feature information related to the feature points to solve the six-dimensional pose information of the target hole in the camera coordinate system (reference pose information is an optional example, and the external parameter matrix of the camera device will be solved during the solution process). Then, combined with the calibration method, the six-dimensional pose information is converted to the base coordinate system of the robot to obtain the target pose information of the target hole in the base coordinate system of the robot. Furthermore, the target pose information is used to control the robot arm to move near the target hole.

[0105] The camera coordinate system is the coordinate system used to describe the internal parameters of the camera device and the relationship between the camera device and the world coordinate system. The robot's base coordinate system is the reference coordinate system defined within the robot system, used to describe the robot's position and posture. It is often used as a benchmark for robot motion and control.

[0106] In some embodiments, a camera calibration method can be used to determine the mapping relationship between the camera coordinate system and the base coordinate system of the robot, and then the mapping relationship is used to convert the reference pose information of the target hole in the camera coordinate system into the target pose information in the base coordinate system of the robot. There is no limitation on this.

[0107] S102: According to the target posture information, a plurality of actual contact forces based on a first direction between the end of the robot arm and the target workpiece are collected.

[0108] After the target posture information of the target hole is collected, the robot's mechanical arm can be controlled to move to the position indicated by the target posture information, and the robot can be triggered to perform the shaft-hole alignment operation.

[0109] In some embodiments, the first direction may be, for example, a perpendicular direction between the end of the robotic arm and the contact surface of the target workpiece. The first direction may be customized or may be a direction determined based on a tool coordinate system.

[0110] In the embodiment of the present disclosure, the tool coordinate system of the robotic arm can be used to define the first direction, wherein the tool coordinate system refers to the coordinate system defined relative to the end effector (the end of the robotic arm) in the robotic system. The tool coordinate system is generally used to describe the position and posture of the end effector of the robot for precise positioning and control. The tool coordinate system is usually not coincident with the base coordinate system of the robot because the end effector may undergo changes such as rotation and translation. By defining the tool coordinate system, the motion and control of the end effector can be converted into motion and control relative to the tool coordinate system, thereby simplifying the design and programming of the robot control system.

[0111] Optionally, in some embodiments, the first direction is the direction indicated by the Z axis of the tool coordinate system of the robot arm, and the tool coordinate system also includes the X axis and the Y axis. The direction indicated by the X axis and the direction indicated by the Y axis are collectively referred to as the second direction. Thus, by defining the first direction based on the tool coordinate system of the robot arm, the control flexibility and effect of the shaft-hole alignment can be effectively improved during the shaft-hole alignment process, thereby supporting improved shaft-hole alignment accuracy.

[0112] In some embodiments, in the process of acquiring multiple actual contact forces between the end of the robotic arm and the target workpiece based on the first direction, the movement of the end of the robotic arm can be controlled, and multiple actual contact forces can be continuously acquired during the movement. Alternatively, multiple reference times can be set, and corresponding actual contact forces can be acquired at each reference time, thereby obtaining multiple actual contact forces. There is no limitation on this.

[0113] It can be understood that when the end of the robotic arm contacts the surface of the target workpiece, the actual contact force at the target hole position and the contact force at the non-target hole position on the target workpiece are different. Usually, after the end of the robotic arm is inserted into the target hole, the actual contact force in the first direction at this position will be relatively small. Therefore, in the embodiment of the present disclosure, multiple actual contact forces based on the first direction between the end of the robotic arm and the target workpiece can be collected and analyzed to determine whether the end of the robotic arm is aligned with the target hole.

[0114] The multiple actual contact forces mentioned above may refer to multiple actual contact forces collected continuously. For example, the multiple actual contact forces are multiple actual contact forces collected at multiple consecutive reference times.

[0115] S103: Determine, based on the multiple actual contact forces, whether the end of the robotic arm is aligned with the target hole.

[0116] In some embodiments, the reference distribution characteristics of multiple real contact forces based on the first direction between the end of the robotic arm and the target workpiece when the end of the robotic arm is aligned with the target hole can be analyzed in advance. Then, during the search for the target hole, the same characteristic analysis is performed on the collected multiple actual contact forces based on the first direction between the end of the robotic arm and the target workpiece to obtain the actual distribution characteristics, and it is determined whether the actual distribution characteristics and the reference distribution characteristics match. If they match, it is determined that the end of the robotic arm has been aligned with the target hole, and there is no restriction on this.

[0117] In some embodiments, a mechanical analysis can be performed based on multiple actual contact forces collected between the end of the robotic arm and the target workpiece in the first direction, and based on the analysis results, it can be determined whether the end of the robotic arm is aligned with the target hole. Of course, the analysis of whether the end of the robotic arm is aligned with the target hole based on multiple actual contact forces can also be achieved based on any other possible method, such as modeling, mathematical calculation, etc., and there is no limitation to this.

[0118] In the disclosed embodiments, a contact force threshold can be pre-calculated. This contact force threshold refers to the contact force in a first direction between the end of the robotic arm and the target workpiece when the end of the robotic arm is aligned with the target hole. This contact force threshold can be pre-calculated and calibrated. After obtaining multiple actual contact forces, a comparison analysis can be performed between the multiple actual contact forces and the contact force threshold to determine whether the end of the robotic arm is aligned with the target hole.

[0119] Optionally, in some embodiments, in the process of determining that the end of the robotic arm is aligned with the target hole based on multiple actual contact forces, it can be that each actual contact force among the multiple actual contact forces is less than or equal to the contact force threshold, and the cumulative sampling time of multiple reference times corresponding to the multiple actual contact forces reaches the time threshold, then it is determined that the end of the robotic arm is aligned with the target hole; if at least one of the multiple actual contact forces is greater than the contact force threshold, or the cumulative sampling time of multiple reference times corresponding to the multiple actual contact forces does not reach the time threshold, the actual contact force based on the first direction between the end of the robotic arm and the target workpiece is re-collected to obtain a new actual contact force, until it is determined that the end of the robotic arm is aligned with the target hole based on the continuously collected multiple actual contact forces, thereby enabling convenient and rapid determination of whether the end of the robotic arm is aligned with the target hole, improving the control efficiency of the shaft hole alignment, and improving the user experience of the robot.

[0120] For example, assume that multiple actual contact forces collected continuously include: actual contact force 1, actual contact force 2, actual contact force 3, and actual contact force 4, and it is determined that the time taken for sampling actual contact force 1, actual contact force 2, actual contact force 3, and actual contact force 4 (that is, the cumulative sampling time of multiple reference times) has reached a time threshold (for example, 10 seconds), and actual contact force 1, actual contact force 2, actual contact force 3, and actual contact force 4 are all less than or equal to the contact force threshold, then it indicates that the end of the robotic arm has been aligned with the target hole. If the actual contact force 3 is greater than the contact force threshold, then it indicates that the end of the robotic arm has not yet been aligned with the target hole. At this time, the actual contact force 5, actual contact force 6, and actual contact force 7 can continue to be collected at subsequent reference times. If it is determined that the end of the robotic arm is aligned with the target hole based on actual contact force 4, actual contact force 5, actual contact force 6, and actual contact force 7, then the sampling can be stopped and the shaft hole assembly operation can be triggered. If it is still not determined that the end of the robotic arm is aligned with the target hole, the sampling process can be continued.

[0121] S104: Control the robotic arm to perform a shaft-hole assembly operation on the target hole.

[0122] After determining that the end of the robotic arm has been aligned with the target hole, the robotic arm can be controlled to perform the shaft-hole assembly operation on the target hole.

[0123] In this embodiment, target pose information of a target hole in the target workpiece is determined based on a workpiece image of the target workpiece. Based on the target pose information, multiple actual contact forces in a first direction are collected between the end of the robotic arm and the target workpiece. Based on the multiple actual contact forces, the end of the robotic arm is determined to be aligned with the target hole, and the robotic arm is controlled to perform a shaft-hole assembly operation on the target hole. This allows for accurate shaft-hole alignment, thereby improving shaft-hole assembly performance.

[0124] Figure 3 A flowchart of another robot control method provided in an embodiment of the present disclosure.

[0125] like Figure 3 As shown, the robot control method includes:

[0126] S301: Determine target position and posture information of a target hole of the target workpiece according to the workpiece image of the target workpiece.

[0127] For the detailed description of S301, please refer to the above embodiment and will not be repeated here.

[0128] S302: Control the robot arm to move to a position corresponding to the target posture information, and collect the actual contact force between the end of the robot arm and the target workpiece based on the first direction at a first reference time.

[0129] The target pose information represents the position and posture of the target hole. Therefore, the target pose information can be used to determine the location of the target hole on the target workpiece and control the robot arm to move to that location, thereby roughly aligning the target hole. To improve the accuracy of the shaft-hole alignment, multiple actual contact forces between the end of the robot arm and the target workpiece in the first direction can be continuously collected to trigger the execution of subsequent steps.

[0130] In some embodiments, when the robotic arm is controlled to move to a position corresponding to the target posture information, it is detected whether the current time is the first reference time. If the current time is the first reference time, it triggers the collection of the actual contact force based on the first direction between the end of the robotic arm and the target workpiece.

[0131] S303: Controlling the movement of the robotic arm in a force control mode according to the actual contact force corresponding to the first reference time.

[0132] In some embodiments, the motion of the robotic arm can be controlled in a force-controlled mode. The force-controlled mode refers to a mode in which the motion of the robotic arm is controlled based on a reference force. In the disclosed embodiments, the reference force may, for example, refer to the actual contact force collected at a previous reference time.

[0133] In some embodiments, after the actual contact force is sampled at the first reference time, the motion displacement required for motion control of the robotic arm can be calculated with reference to the actual contact force sampled at the first reference time, and then the robotic arm can be controlled to move based on the motion displacement; or the robotic arm can be motion controlled based on the actual contact force sampled at the first reference time and combined with a motion control model, and there is no restriction on this.

[0134] Optionally, in some embodiments, in the process of controlling the motion of the robotic arm in the force control mode based on the actual contact force corresponding to the first reference time, the position deviation of the end of the robotic arm may be determined based on the actual contact force corresponding to the first reference time, and based on the position deviation, the joint differential motion associated with the joint of the robotic arm is determined, and the current joint angle of the joint of the robotic arm is determined. Based on the joint differential motion, the current joint angle of the robotic arm is controlled to control the motion of the robotic arm. This allows precise control of the motion of the robotic arm in the force control mode.

[0135] The position deviation represents the deviation between the current position of the end of the robotic arm and the desired position. The joint differential motion represents the magnitude of the adjustment required for the current joint angle of the joint of the robotic arm.

[0136] In some embodiments, the position deviation can be determined with reference to the actual contact force corresponding to the first reference time, and then the joint differential motion associated with the joint of the robotic arm can be calculated based on the position deviation, and the joint differential motion is accumulated based on the current joint angle, thereby achieving control of the robotic arm movement in force control mode.

[0137] Optionally, in some embodiments, in the process of determining the position deviation of the end of the manipulator based on the actual contact force corresponding to the first reference time, the expected contact force corresponding to the third reference time, the actual contact force corresponding to the third reference time, and the position compensation amount corresponding to the third reference time may be determined, wherein the third reference time is the previous reference time of the first reference time, and an update rate is determined based on the actual contact force corresponding to the first reference time, and the position compensation amount corresponding to the first reference time is determined based on the update rate, the expected contact force corresponding to the third reference time, the actual contact force corresponding to the third reference time, and the position compensation amount corresponding to the third reference time. Finally, the position deviation of the end of the manipulator is determined based on the position compensation amount corresponding to the first reference time. In this way, the position deviation of the end of the manipulator is accurately determined in the force control mode, thereby improving the control accuracy of the manipulator movement.

[0138] For example, the robot arm can be controlled based on the position control mode in the directions indicated by the X-axis and Y-axis of the tool coordinate system, and the robot arm can be controlled based on the force control mode in the direction indicated by the Z-axis, that is, force-position hybrid control. The robot arm control in the force control mode is implemented in the direction indicated by the Z-axis of the tool coordinate system (an optional example of the first direction). For example, an adaptive admittance control formula can be used to guide the robot arm control in the force control mode, and the desired control force can be set to calculate the position compensation (also called adaptive compensation). The adaptive admittance control formula is as follows:

[0139]

[0140]

[0141] Wherein, λ is the sampling period (the sampling period represents the duration between two adjacent sampling time points), α is the update rate, ψ(t) represents the adaptive compensation corresponding to the first reference time t (an optional example of the position compensation amount), ψ(t-λ) represents the adaptive compensation corresponding to the third reference time t-λ (an optional example of the position compensation amount), and f d (t) represents the expected contact force corresponding to the first reference time t, f e (t) represents the actual contact force corresponding to the first reference time t, f d (t-λ) represents the expected contact force corresponding to the third reference time t-λ, f e (t-λ) represents the actual contact force corresponding to the third reference time t-λ, m represents the inertia coefficient, b represents the damping coefficient, k represents the stiffness coefficient, Δx represents the position deviation, Indicates the speed deviation, represents the acceleration deviation. Therefore, by the contact force deviation f e (t)-f d (t) is integrated to obtain ψ(t), which can eliminate the force tracking steady-state error and thus achieve precise force control.

[0142] Optionally, in determining the update rate based on the actual contact force corresponding to the first reference time, an upper limit for the update rate may be determined, and the update rate may be determined based on the upper limit, the actual contact force corresponding to the first reference time, and the expected contact force corresponding to the first reference time. This allows for dynamic adjustment of the update rate, making it more adaptable to changes in the actual contact force, and effectively improving control accuracy and effectiveness.

[0143] The update rate α in the above formula can be dynamically adjusted by the force tracking error. The adjustment method is:

[0144] α=Ae -βΔf ;

[0145] Where A is the upper limit of the update rate, e is the natural exponent, β is the adjustment coefficient of the adaptive update rate, Δf = f e (t)-f d (t) represents the force deviation between the contact forces. At the beginning of contact, the force deviation is large, the update rate α is small, and the adjustment is slow, effectively preventing overshoot. As the force deviation decreases, the update rate α gradually increases, increasing the adjustment speed and effectively shortening the adjustment time. This effectively balances overshoot and adjustment time, ensuring good dynamic response of the system.

[0146] Furthermore, the adaptive admittance control formula can be discretized into the following iterative form:

[0147]

[0148] Among them, in the initial stage, the expected control force f can be set d =F_target, adaptive compensation ψ, position deviation Δx and speed deviation The initial values ​​are all set to zero, and the current actual contact force f is obtained in each iteration cycle (one iteration cycle can be associated with a reference time) e , the adaptive compensation ψ can be calculated, and then the acceleration deviation can be obtained The position deviation Δx of the end of the manipulator is obtained by iterating the equation, and then the pseudo-inverse of the Jacobian matrix J of the manipulator is obtained. -1 Calculate the joint differential motion associated with the joints of the robotic arm, then solve the current joint angle and send instructions to the robotic arm controller to make it move.

[0149]

[0150] Where Δq represents the differential motion of the joint, q(t) represents the current joint angle, and q(t-1) represents the previous joint angle.

[0151] S304: Control the robotic arm to move in a second direction in the position control mode, wherein the first direction and the second direction are perpendicular to each other.

[0152] While controlling the movement of the robotic arm in the force control mode, the robotic arm can also be controlled to move in a second direction in the position control mode. The second direction is perpendicular to the first direction. The second direction can be, for example, the direction indicated by the X-axis and the direction indicated by the Y-axis of the tool coordinate system.

[0153] In some embodiments, the position control mode refers to a mode of calculating a reference position and controlling the movement of the robotic arm based on the reference position.

[0154] Optionally, in some embodiments, the radial velocity and rotational velocity of the robotic arm can be determined, and the robotic arm can be controlled to move in the second direction based on the first reference time and the radial velocity and rotational velocity of the robotic arm, thereby accurately controlling the movement of the robotic arm in the position control mode.

[0155] For example, the position control mode can be used in the direction indicated by the X-axis and the Y-axis of the tool coordinate system. The spiral search can be used to continuously rotate to find the position of the hole. The spiral equation is as follows:

[0156]

[0157] Wherein, v is the radial velocity, ω is the rotational velocity, and t is the current time (an optional example of the first reference time).

[0158] S305: At the second reference time, the actual contact force between the end of the robot arm and the target workpiece based on the first direction is collected to obtain the actual contact force corresponding to the second reference time, and the first reference time is updated according to the second reference time until multiple actual contact forces are continuously collected.

[0159] After controlling the movement of the robotic arm, the actual contact force between the end of the robotic arm and the target workpiece based on the first direction can be collected at the second reference time, and the movement of the robotic arm can be controlled again based on the above-mentioned force control mode and position control mode, and the actual contact force can be collected again at the next reference time after the movement.

[0160] S306: Determine, based on the multiple actual contact forces, whether the end of the robotic arm is aligned with the target hole.

[0161] For the detailed description of S306 , please refer to the above embodiment and will not be repeated here.

[0162] S307: Control the movement of the robotic arm in the force control mode, and control the robotic arm to perform the shaft-hole assembly operation on the target hole.

[0163] That is to say, if the actual contact force in the direction indicated by the z-axis of the tool coordinate system is less than or equal to the contact force threshold for a period of time, it means that the target hole has been aligned, and then the pure force control mode is switched to, that is, adaptive admittance control is used in the directions indicated by the X-axis, Y-axis and Z-axis of the tool coordinate system, the expected contact force in the X-axis and Y-axis directions is zero, and the expected contact force in the Z-axis direction is F_target, to ensure that the end of the robot arm automatically adjusts and smoothly inserts into the hole. For example, the position deviation Δx can be iteratively calculated in each dimension, and then the pseudo-inverse of the Jacobian matrix J of the robot arm is used. -1 Calculate the corresponding joint differential motion, and then solve the current joint angle to control the movement of the robotic arm.

[0164] In this embodiment, the target posture information of the target hole of the target workpiece is determined based on the workpiece image of the target workpiece, and based on the target posture information, multiple actual contact forces based on the first direction between the end of the robot arm and the target workpiece are collected, and based on the multiple actual contact forces, it is determined that the end of the robot arm is aligned with the target hole, and the robot arm is controlled to perform the shaft-hole assembly operation on the target hole. The shaft-hole alignment can be accurately achieved, thereby improving the shaft-hole assembly effect. Therefore, the method provided in the embodiment of the present disclosure searches for the target hole through force-position hybrid control in the high-precision shaft-hole assembly or multi-shaft-hole assembly operation scenario, and uses the robot differential motion to achieve posture fine-tuning to ensure that the shaft-hole assembly is completed smoothly. Through adaptive admittance control, the actual contact force is kept within the set range to prevent damage to the workpiece during operation. The position deviation of the end of the robot arm is solved by the adaptive admittance control formula, and the joint differential motion is calculated using the pseudo-inverse of the Jacobian matrix to ensure the continuity and smoothness of the robot motion.

[0165] like Figure 4 As shown, Figure 4 It is a flow chart of robot control in the embodiment of the present disclosure. The target posture information of the target hole can be obtained based on visual positioning, and the robot's mechanical arm can be controlled to move to the position indicated by the target posture information. With the tool coordinate system as the reference, Z-axis force control, X-axis and Y-axis position control, and search according to the spiral trajectory. When the expected contact force is detected in the direction indicated by the Z-axis, the hole can be explored to determine whether the actual contact force collected multiple times in the direction indicated by the Z-axis is less than or equal to the contact force threshold. If so, force control is adopted in the X-axis, Y-axis, and Z-axis directions to achieve smooth jacking action. If not, Z-axis force control, X-axis and Y-axis position control are continued to be implemented, and search is carried out according to the spiral trajectory.

[0166] Figure 5 This is a schematic diagram of the structure of a robot control device provided by an embodiment of the present disclosure. The robot in the embodiment of the present disclosure includes: a robotic arm.

[0167] like Figure 5 As shown, the robot control device 50 includes:

[0168] The first determining module 501 is configured to determine target position and posture information of a target hole of the target workpiece according to the workpiece image of the target workpiece.

[0169] The acquisition module 502 is configured to acquire a plurality of actual contact forces between the end of the robot arm and the target workpiece based on a first direction according to the target posture information.

[0170] The second determining module 503 is configured to determine, based on a plurality of actual contact forces, whether the end of the robot arm is aligned with the target hole.

[0171] The control module 504 is used to control the robot arm to perform the shaft-hole assembly operation on the target hole.

[0172] In some embodiments of the present disclosure, the first determining module 501 is specifically configured to:

[0173] Process the workpiece image to obtain a workpiece grayscale image;

[0174] Identify and obtain feature points corresponding to the target hole from the grayscale image of the workpiece;

[0175] According to the feature points, the reference pose information of the target hole in the camera coordinate system is determined, wherein there is a mapping relationship between the camera coordinate system and the base coordinate system of the robot;

[0176] The reference pose information is processed according to the mapping relationship to obtain the target pose information.

[0177] In some embodiments of the present disclosure, the acquisition module 502 is specifically configured to:

[0178] Controlling the robotic arm to move to a position corresponding to the target posture information, and collecting an actual contact force based on a first direction between the end of the robotic arm and the target workpiece at a first reference time;

[0179] controlling the movement of the robotic arm in a force control mode according to the actual contact force corresponding to the first reference time;

[0180] Controlling the robot arm to move in a second direction in a position control mode, wherein the first direction and the second direction are perpendicular to each other;

[0181] At the second reference time, the actual contact force between the end of the robotic arm and the target workpiece based on the first direction is collected to obtain the actual contact force corresponding to the second reference time, and the first reference time is updated according to the second reference time until multiple actual contact forces are continuously collected.

[0182] In some embodiments of the present disclosure, the acquisition module 502 is specifically configured to:

[0183] determining a position deviation of the end of the robotic arm based on an actual contact force corresponding to the first reference time;

[0184] determining joint differential motions associated with joints of the robotic arm based on the position deviation;

[0185] Determine the current joint angles of the joints of the robotic arm;

[0186] According to the joint differential motion, the current joint angle of the robot arm is controlled to control the movement of the robot arm.

[0187] In some embodiments of the present disclosure, the acquisition module 502 is specifically configured to:

[0188] determining an expected contact force corresponding to a third reference time, an actual contact force corresponding to the third reference time, and a position compensation amount corresponding to the third reference time, wherein the third reference time is a previous reference time of the first reference time;

[0189] determining an update rate based on the actual contact force corresponding to the first reference time;

[0190] determining a position compensation amount corresponding to the first reference time based on the update rate, the expected contact force corresponding to the third reference time, the actual contact force corresponding to the third reference time, and the position compensation amount corresponding to the third reference time;

[0191] A position deviation of the end of the robot arm is determined based on the position compensation amount corresponding to the first reference time.

[0192] In some embodiments of the present disclosure, the acquisition module 502 is specifically configured to:

[0193] Determine the upper limit of the update rate;

[0194] The update rate is determined according to the update rate upper limit, the actual contact force corresponding to the first reference time, and the expected contact force corresponding to the first reference time.

[0195] In some embodiments of the present disclosure, the acquisition module 502 is specifically configured to:

[0196] Determine the radial velocity and rotational speed of the robot arm;

[0197] The robotic arm is controlled to move in a second direction according to the first reference time and the radial speed and the rotation speed of the robotic arm.

[0198] In some embodiments of the present disclosure, the second determining module 503 is configured to:

[0199] If each of the multiple actual contact forces is less than or equal to the contact force threshold, and the accumulated sampling duration of the multiple reference times corresponding to the multiple actual contact forces reaches the duration threshold, it is determined that the end of the robotic arm is aligned with the target hole;

[0200] If at least one of the multiple actual contact forces is greater than the contact force threshold, or the cumulative sampling duration of multiple reference times corresponding to the multiple actual contact forces does not reach the duration threshold, the actual contact force between the end of the robot arm and the target workpiece based on the first direction is re-collected to obtain a new actual contact force until it is determined that the end of the robot arm is aligned with the target hole based on the multiple actual contact forces collected continuously.

[0201] In some embodiments of the present disclosure, the control module 504 is specifically configured to:

[0202] Control the movement of the robotic arm in force control mode and control the robotic arm to perform shaft-hole assembly operations on the target hole.

[0203] In some embodiments of the present disclosure, the first direction is the direction indicated by the Z axis of the tool coordinate system of the robotic arm, and the tool coordinate system also includes the X axis and the Y axis. The direction indicated by the X axis and the direction indicated by the Y axis are collectively regarded as the second direction.

[0204] It should be noted that the aforementioned explanation of the robot control method is also applicable to the robot control device of this embodiment and will not be repeated here.

[0205] In this embodiment, target pose information of a target hole in the target workpiece is determined based on a workpiece image of the target workpiece. Based on the target pose information, multiple actual contact forces in a first direction are collected between the end of the robotic arm and the target workpiece. Based on the multiple actual contact forces, the end of the robotic arm is determined to be aligned with the target hole, and the robotic arm is controlled to perform a shaft-hole assembly operation on the target hole. This allows for accurate shaft-hole alignment, thereby improving shaft-hole assembly performance.

[0206] Figure 6 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 6 The electronic device 12 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0207] like Figure 6 As shown, electronic device 12 is implemented as a general purpose computing device. Components of electronic device 12 may include, but are not limited to, one or more processors or processing units 16, memory 28, and a bus 18 that connects various system components (including memory 28 and processing unit 16).

[0208] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of such architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnection (PCI) bus.

[0209] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0210] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 6 Not shown, often called a "hard drive").

[0211] although Figure 6 Although not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a Compact Disc Read Only Memory (hereinafter referred to as: CD-ROM), a Digital Video Disc Read Only Memory (hereinafter referred to as: DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.

[0212] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methods of the embodiments described herein.

[0213] The electronic device 12 can also communicate with one or more external devices 14 (e.g., a keyboard, pointing device, display 24, etc.), one or more devices that enable human interaction with the electronic device 12, and / or any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). This communication can occur via an input / output (I / O) interface 22. Furthermore, the electronic device 12 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the electronic device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0214] The processing unit 16 executes various functional applications and data processing by running the programs stored in the memory 28 , such as implementing the robot control method mentioned in the above embodiment.

[0215] Figure 7 A schematic structural diagram of a robot provided in an embodiment of the present disclosure.

[0216] The robot includes:

[0217] Memory 701 , processor 702 , and computer programs stored in the memory 701 and executable on the processor 702 .

[0218] When the processor 702 executes the program, the robot control method provided in the above embodiment is implemented.

[0219] In one possible implementation, the robot further includes:

[0220] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0221] The memory 701 is used to store computer programs that can be run on the processor 702 .

[0222] The memory 701 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0223] The processor 702 is configured to implement the robot control method of the above embodiment when executing a program.

[0224] If the memory 701, processor 702, and communication interface 703 are implemented independently, the communication interface 703, memory 701, and processor 702 can be interconnected via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0225] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0226] The processor 702 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure.

[0227] In order to implement the above embodiments, the present disclosure also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the above embodiments.

[0228] In order to implement the above embodiments, the present disclosure further proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0229] In order to implement the above embodiments, the present disclosure further provides a computer program product, including a computer program, which implements the methods provided in the above embodiments when executed by a processor.

[0230] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0231] It is important to note that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold beyond these legitimate uses. Furthermore, such collection / sharing should be conducted only after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes the relevant user information before using the feature. Furthermore, any necessary steps must be taken to safeguard and secure access to such personal information and ensure that others with access to personal information comply with its privacy policy and procedures.

[0232] This disclosure contemplates providing implementations that allow users to selectively block the use or access of personal information data. Specifically, this disclosure contemplates providing hardware and / or software to prevent or block access to such personal information data. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. Furthermore, where applicable, such personal information can be de-identified to protect user privacy.

[0233] In the descriptions of the aforementioned embodiments, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are mutually inconsistent.

[0234] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0235] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0236] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0237] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0238] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0239] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0240] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. A person of ordinary skill in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A robot control method, characterized in that: The robot comprises: a robotic arm; the method comprises: Determining target position information of a target hole of the target workpiece according to the workpiece image of the target workpiece; collecting, based on the target posture information, a plurality of actual contact forces between the end of the robotic arm and the target workpiece in a first direction; determining, based on the plurality of actual contact forces, that the end of the robotic arm is aligned with the target hole; and The robotic arm is controlled to perform a shaft-hole assembly operation on the target hole.

2. The method according to claim 1, characterized in that Determining target position information of a target hole of the target workpiece according to the workpiece image of the target workpiece includes: Processing the workpiece image to obtain a workpiece grayscale image; Identify and obtain feature points corresponding to the target hole from the workpiece grayscale image; Determine reference pose information of the target hole in a camera coordinate system according to the feature points, wherein a mapping relationship exists between the camera coordinate system and a base coordinate system of the robot; The reference pose information is processed according to the mapping relationship to obtain the target pose information.

3. The method according to claim 1, characterized in that The step of collecting a plurality of actual contact forces based on a first direction between the end of the robotic arm and the target workpiece according to the target posture information includes: Controlling the robotic arm to move to a position corresponding to the target posture information, and collecting an actual contact force based on a first direction between the end of the robotic arm and the target workpiece at a first reference time; controlling the movement of the robotic arm in a force control mode according to the actual contact force corresponding to the first reference time; Controlling the robotic arm to move in a second direction in a position control mode, wherein the first direction and the second direction are perpendicular to each other; At the second reference time, the actual contact force between the end of the robotic arm and the target workpiece based on the first direction is collected to obtain the actual contact force corresponding to the second reference time, and the first reference time is updated according to the second reference time until multiple actual contact forces are continuously collected.

4. The method according to claim 3, characterized in that The controlling the movement of the robotic arm in a force control mode according to the actual contact force corresponding to the first reference time includes: determining a position deviation of the end of the robotic arm based on an actual contact force corresponding to the first reference time; determining a joint differential motion associated with a joint of the robotic arm based on the position deviation; determining current joint angles of the joints of the robotic arm; The current joint angle of the robotic arm is controlled according to the joint differential motion amount to control the motion of the robotic arm.

5. The method according to claim 4, characterized in that The determining the position deviation of the end of the robotic arm according to the actual contact force corresponding to the first reference time includes: determining an expected contact force corresponding to a third reference time, an actual contact force corresponding to the third reference time, and a position compensation amount corresponding to the third reference time, wherein the third reference time is a previous reference time of the first reference time; determining an update rate based on the actual contact force corresponding to the first reference time; determining a position compensation amount corresponding to the first reference time based on the update rate, the expected contact force corresponding to the third reference time, the actual contact force corresponding to the third reference time, and the position compensation amount corresponding to the third reference time; A position deviation of the end of the robot arm is determined according to the position compensation amount corresponding to the first reference time.

6. The method according to claim 4, characterized in that The determining the update rate according to the actual contact force corresponding to the first reference time includes: Determine the upper limit of the update rate; The update rate is determined according to the update rate upper limit, the actual contact force corresponding to the first reference time, and the expected contact force corresponding to the first reference time.

7. The method according to claim 3, characterized in that The controlling the robotic arm to move based on the second direction in the position control mode includes: determining a radial velocity and a rotational velocity of the robotic arm; The robotic arm is controlled to move in a second direction according to the first reference time and the radial speed and the rotational speed of the robotic arm.

8. The method according to claim 1, characterized in that Determining, based on the multiple actual contact forces, that the end of the robotic arm is aligned with the target hole comprises: If each of the plurality of actual contact forces is less than or equal to a contact force threshold, and the accumulated sampling duration of a plurality of reference times respectively corresponding to the plurality of actual contact forces reaches a duration threshold, it is determined that the end of the robotic arm is aligned with the target hole; If at least one of the multiple actual contact forces is greater than the contact force threshold, or the cumulative sampling duration of multiple reference times corresponding to the multiple actual contact forces does not reach the duration threshold, the actual contact force based on the first direction between the end of the robot arm and the target workpiece is re-collected to obtain a new actual contact force until it is determined based on multiple continuously collected actual contact forces that the end of the robot arm is aligned with the target hole.

9. The method according to claim 3, characterized in that The controlling the robotic arm to perform the shaft-hole assembly operation on the target hole includes: The movement of the robotic arm is controlled in the force control mode, and the robotic arm is controlled to perform a shaft-hole assembly operation on the target hole.

10. The method according to any one of claims 1 to 9, characterized in that The first direction is the direction indicated by the Z axis of the tool coordinate system of the robot arm. The tool coordinate system also includes an X axis and a Y axis. The direction indicated by the X axis and the direction indicated by the Y axis are collectively used as the second direction.

11. A robot control device, characterized in that: The robot includes: a mechanical arm; the device includes: A first determining module is configured to determine target pose information of a target hole of the target workpiece according to a workpiece image of the target workpiece; an acquisition module, configured to acquire a plurality of actual contact forces between the end of the robotic arm and the target workpiece in a first direction according to the target posture information; a second determining module, configured to determine, based on the plurality of actual contact forces, that the end of the robotic arm is aligned with the target hole; and A control module is used to control the robotic arm to perform an axis-hole assembly operation on the target hole.

12. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 10.

13. A robot, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 10 is implemented.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.

15. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 10 when being executed by a processor.

Images