Shaft hole assembly method, apparatus, and medium

Abstract

This application discloses a shaft-hole assembly method, equipment, and medium. The method includes: when a multi-axis workpiece is in contact with a multi-hole workpiece, performing a calculation operation on the deviation between the shaft in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece to obtain the deviation; if the deviation is greater than a preset deviation, controlling a robotic arm to move the multi-axis workpiece based on the deviation until the deviation is less than or equal to the preset deviation; controlling the robotic arm to perform a spiral hole-searching operation on the multi-axis workpiece until the actual contact force value of the contact surface between the robotic arm and the multi-axis workpiece is less than or equal to a fourth preset force value; if the actual contact force value of the contact surface between the robotic arm and the multi-axis workpiece is less than or equal to the fourth preset force value, performing an operation to determine the center position of the bottom of the shaft already inserted in the hole in the multi-axis workpiece; controlling the robotic arm to move the multi-axis workpiece on a set spherical surface until the contact component of the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to a third preset force value.

Description

Technical Field

[0001] This application relates to the field of automatic control technology, and more specifically, to a shaft hole assembly method, equipment, and medium. Background Technology

[0002] The assembly problem of multi-axis workpieces with multiple holes is more complex than that of single-axis workpieces with a single hole. In the assembly process of multi-axis and multi-hole workpieces, vision alone cannot accurately determine the current assembly position and orientation. Therefore, six-dimensional force information and compliant control methods for robotic arms are often required to complete the alignment of the shaft and hole axes.

[0003] However, when performing pose adjustment in multi-axis hole assembly tasks, on the one hand, due to the coupling of forces on multiple axes, it is difficult to obtain the position deviation information of the shaft holes through the planar force characteristics; on the other hand, there is no mature method for determining the deviation during attitude adjustment. This makes multi-axis hole assembly tasks complex and difficult to implement. Summary of the Invention

[0004] One objective of this application is to provide a new technical solution for shaft-hole assembly.

[0005] According to a first aspect of this application, a shaft hole assembly method is provided, comprising:

[0006] In a multi-axis workpiece, when a single axis is aligned with a corresponding hole in a multi-hole workpiece, the operation of determining the bottom center position of the inserted shaft in the multi-axis workpiece is performed; wherein, the inserted shaft is the shaft in the multi-axis workpiece that is aligned with a hole in the multi-hole workpiece;

[0007] The robotic arm is controlled to move the multi-axis workpiece on a set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to a third preset force value; wherein, the set spherical surface is a spherical surface with the bottom center of the inserted shaft as the center and a set distance as the radius, and the set distance is the distance between the bottom center of the inserted shaft and the end of the robotic arm;

[0008] Before aligning the single axis in the multi-axis workpiece with the corresponding hole in the multi-hole workpiece, the method further includes:

[0009] When a multi-axis workpiece is controlled to contact a multi-hole workpiece by a robotic arm, the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is calculated to obtain the deviation.

[0010] If the deviation is greater than the preset deviation, the robotic arm is controlled to move the multi-axis workpiece based on the deviation, and the calculation operation is repeated until the deviation is less than or equal to the preset deviation.

[0011] The robotic arm is controlled to drive the multi-axis workpiece to perform a spiral hole-searching operation until the actual contact force value between the robotic arm and the multi-axis workpiece is less than or equal to the fourth preset force value.

[0012] If the actual contact force value of the contact surface between the robotic arm and the multi-axis workpiece is less than or equal to the fourth preset force value, it is determined that the single axis of the multi-axis workpiece is aligned with the corresponding hole of the multi-hole workpiece.

[0013] The calculation operation includes: acquiring a first image containing the contact surface of the multi-axis workpiece and the multi-hole workpiece, and detecting the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece based on the first image.

[0014] Optionally, controlling the robotic arm to drive the multi-axis workpiece to perform a helical hole-searching operation includes:

[0015] Obtain the tolerance between the central shaft of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece;

[0016] Determine the helix pitch based on the aforementioned tolerance;

[0017] Based on the pitch of the helix, the robotic arm is controlled according to the Archimedes spiral to drive the multi-axis workpiece to perform a helical hole-searching operation.

[0018] Optionally, before calculating the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece when the multi-axis workpiece is controlled to contact the multi-hole workpiece by a robotic arm, and before obtaining the deviation, the method further includes:

[0019] The robotic arm is controlled to be in a compliant impedance mode.

[0020] Optionally, detecting the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece based on the first image includes:

[0021] Detect pairs of arc lines in the first image;

[0022] For any of the said arc line pairs, the first centroid position corresponding to the first centroid of the first arc line is determined according to the first arc line in the arc line pair, and the second centroid position corresponding to the second arc line is determined according to the second arc line in the arc line pair. The deviation between the first centroid position and the second centroid position is calculated as the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0023] Optionally, detecting the pairs of arc lines in the first image includes:

[0024] Perform a first image preprocessing operation on the first image to obtain a processed first image. The first image preprocessing operation includes: grayscale processing, noise reduction processing, binarization processing, and interference filtering processing.

[0025] Extract the arc lines from the processed first image to obtain at least two arc lines;

[0026] Two intersecting arcs among the at least two arcs are defined as an arc pair.

[0027] Optionally, before calculating the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece by controlling the multi-axis workpiece to contact the multi-axis workpiece with the robotic arm, and obtaining the deviation, the method further includes:

[0028] Obtain a second image containing the porous workpiece;

[0029] Detect the edges that are closed contours in the second image to obtain at least one closed contour edge;

[0030] From the at least one closed contour edge, select closed contour edges with a roundness greater than a preset roundness to obtain at least one circular edge;

[0031] For any of the circular edges, determine the position of the third centroid of the circular edge;

[0032] Based on the third centroid position, the robotic arm controls the multi-axis workpiece to contact the porous workpiece.

[0033] Optionally, detecting edges that are closed contours in the second image to obtain at least one closed contour edge includes:

[0034] Perform a second image preprocessing operation on the second image to obtain a processed second image. The first image preprocessing operation includes: grayscale processing, noise reduction processing, binarization processing, and circular hole display processing.

[0035] Detect the edges of closed contours in the processed second image to obtain at least one closed contour edge.

[0036] Optionally, controlling the robotic arm to move the multi-axis workpiece on a set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to a third preset force value includes:

[0037] Based on the first rotation angle, the robotic arm is controlled to drive the multi-axis workpiece to move around the fourth direction on the set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is greater than or equal to the first preset force value.

[0038] When the actual contact force in the third direction is greater than or equal to the first preset force value, the robotic arm is controlled based on the second rotation angle to drive the multi-axis workpiece to move around the fifth direction on the set spherical surface until the actual contact force in the third direction between the robotic arm and the multi-axis workpiece is greater than or equal to the second preset force value.

[0039] When the actual contact force in the third direction is greater than or equal to the second preset force value, the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is calculated to obtain the deviation vector.

[0040] Based on the deviation vector control, the robotic arm drives the multi-axis workpiece to move around the sixth direction on the set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to the third preset force value.

[0041] The sixth direction is the extension direction of the shaft that has been inserted into the hole, and the fourth and fifth directions are perpendicular in a plane that is perpendicular to the sixth direction.

[0042] Optionally, the first rotation angle is determined by the following steps:

[0043] The second contact force of the contact surface between the robotic arm and the multi-axis workpiece is obtained in the first direction and the second contact resultant force in the second direction, and the third contact component force of the second contact force in the third direction.

[0044] The third rotation angle is determined based on the second contact resultant force and the third contact component force;

[0045] The first rotation angle is determined based on the third rotation angle and the preset angle deviation value;

[0046] The second rotation angle is determined by the following steps:

[0047] When the actual contact force in the third direction is greater than or equal to the first preset force value, the third contact force of the contact surface between the robotic arm and the multi-axis workpiece is obtained as the third contact resultant force in the first and second directions and the fourth contact component of the third contact force in the third direction.

[0048] The fourth rotation angle is determined based on the third contact resultant force and the fourth contact component force.

[0049] The second rotation angle is determined based on the fourth rotation angle and the preset angle deviation value.

[0050] Optionally, the calculation operation includes:

[0051] Obtain a first image containing the contact surface between the multi-axis workpiece and the multi-hole workpiece;

[0052] Based on the first image, the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is detected.

[0053] Optionally, detecting the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece based on the first image includes:

[0054] Detect pairs of arc lines in the first image;

[0055] For any pair of arc lines, the first centroid position corresponding to the first centroid of the first arc line is determined according to the first arc line in the pair of arc lines, and the second centroid position corresponding to the second arc line is determined according to the second arc line in the pair of arc lines. The deviation between the first centroid position and the second centroid position is calculated as the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0056] Optionally, controlling the robotic arm to move the multi-axis workpiece around a first direction on the set spherical surface based on the first rotation angle includes:

[0057] Based on the first rotation angle and the first pose of the robotic arm end, determine the first arc trajectory of the robotic arm end on the set spherical surface;

[0058] The robotic arm end effector is controlled to drive the multi-axis workpiece to move along the first circular arc trajectory.

[0059] Optionally, the operation of determining the center position of the bottom of the inserted shaft in the multi-axis workpiece includes:

[0060] Obtain the first contact force at the contact surface between the robotic arm and the multi-axis workpiece;

[0061] Based on the first contact force, determine the shaft that has been inserted into the hole in the multi-axis workpiece;

[0062] Obtain the first position of the end effector of the robotic arm;

[0063] Based on the first position, determine the center position of the bottom of the inserted shaft.

[0064] Optionally, the first contact force includes a first contact force component along a first direction and a second contact force component along a second direction, wherein the first direction and the second direction are perpendicular. Determining the inserted shaft in the multi-axis workpiece based on the first contact force includes:

[0065] The contact azimuth angle between the multi-axis workpiece and the multi-hole workpiece is determined based on the first contact force component value and the second contact force component value.

[0066] Based on the contact azimuth angle and the first correspondence, the shaft that has entered the hole in the multi-axis workpiece is determined; wherein, the first correspondence is a range of contact azimuth angles when different shafts of the multi-axis workpiece contact the multi-hole workpiece.

[0067] Optionally, determining the center position of the bottom of the inserted shaft based on the first position includes:

[0068] The bottom center position of the multi-axis workpiece is determined based on the first position and the first relative position relationship; wherein, the first relative position relationship is the relative position relationship between the end of the robotic arm and the bottom center of the multi-axis workpiece;

[0069] Based on the inserted shaft and the second correspondence, the second relative position relationship corresponding to the inserted shaft is determined; wherein, the second correspondence relationship is the relative position relationship between the bottom centers of different shafts of the multi-axis workpiece and the bottom center of the multi-axis workpiece.

[0070] The bottom center position of the inserted shaft is determined based on the bottom center position of the multi-axis workpiece and the second relative position relationship.

[0071] According to a second aspect of this application, an electronic device is provided, the electronic device comprising a force sensor, a robotic arm, a memory, and a processor, the robotic arm being configured to move a shaft workpiece under the control of the processor, the force sensor being configured to acquire the actual contact force value of the contact surface between the robotic arm and the shaft workpiece, the memory being configured to store computer instructions, and the processor being configured to retrieve the computer instructions from the memory to execute the method as described in any one of the first aspects.

[0072] According to a third aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the first aspects.

[0073] This application provides a shaft-hole assembly method. Based on the above, this method uses computer vision to calculate the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece after they come into contact. This assists the robotic arm in performing rapid spiral hole searching, improving the hole searching efficiency. Furthermore, this method eliminates the need for high-precision determination of the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece, reducing the accuracy requirements for this deviation. In other words, only the fuzzy deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is needed to complete rapid hole searching, achieving single-axis rapid insertion of the multi-axis workpiece into a hole. Moreover, in the case of single-axis insertion of the multi-axis workpiece, the robotic arm is controlled to move the multi-axis workpiece on a sphere with the bottom center of the inserted shaft as the center and a set distance as the radius, until the non-inserted shaft in the multi-axis workpiece aligns with the corresponding hole. This avoids damage to the inserted shaft during the multi-axis insertion process. Furthermore, a corresponding motion trajectory is planned for multi-axis insertion, improving the efficiency and accuracy of multi-axis insertion and achieving rapid insertion of multiple shafts into multiple holes.

[0074] Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0075] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.

[0076] Figure 1 This is a schematic flowchart of a shaft hole assembly method provided in this application;

[0077] Figure 2 This is a schematic diagram of a first image provided in this application, showing that when a multi-axis workpiece is in contact with a multi-hole workpiece, the corresponding holes on the multi-axis workpiece are not aligned.

[0078] Figure 3 This is a schematic diagram of a processed first image corresponding to a first image provided in this application;

[0079] Figure 4 This is an assembly diagram illustrating an example provided in this application of a triaxial workpiece in contact with a three-hole workpiece.

[0080] Figure 5 This is a schematic diagram of the posture adjustment during the alignment process of a multi-axis workpiece and a multi-hole workpiece provided in this application;

[0081] Figure 6 This is a schematic diagram of the azimuth angle range corresponding to the three axes of a three-axis workpiece, as provided in this application.

[0082] Figure 7This is a schematic diagram of a method for controlling a robotic arm to drive a multi-axis workpiece to rotate around a fourth direction, as provided in this application.

[0083] Figure 8 This is a schematic diagram of a method for controlling a robotic arm to drive a multi-axis workpiece to rotate around a fifth direction, as provided in this application.

[0084] Figure 9 This is a schematic diagram of a method for controlling a robotic arm to drive a multi-axis workpiece to rotate around a sixth direction, as provided in this application.

[0085] Figure 10 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0086] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0087] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0088] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0089] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0090] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0091] This application provides a shaft-hole assembly method. This method uses computer vision to calculate the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece after they come into contact. This assists a robotic arm in performing a helical hole-finding process, achieving single-axis insertion of the multi-axis workpiece into a hole. Furthermore, in the case of single-axis insertion of the multi-axis workpiece, the robotic arm is controlled to move the multi-axis workpiece on a sphere with the bottom center of the inserted shaft as the center and a set distance as the radius. This aligns the shaft to be inserted in the multi-axis workpiece with the corresponding hole in the multi-hole workpiece, thus assisting the robotic arm in assembling the shaft and hole of the multi-axis and multi-hole workpieces.

[0092] It should be noted that this application stipulates that the number of shafts in a multi-axis workpiece is the same as the number of holes in a multi-hole workpiece, and does not limit the number of shafts in a multi-axis workpiece or the number of holes in a multi-hole workpiece.

[0093] like Figure 1 As shown, the shaft hole assembly method provided in this application includes the following steps S1100 to S1600.

[0094] Step S1100: Under the condition that the multi-axis workpiece and the multi-hole workpiece are in contact controlled by the robotic arm, the deviation calculation operation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is performed to obtain the deviation.

[0095] The calculation operation includes the following steps S1110 and S1120.

[0096] Step S1110: Obtain a first image containing the contact surface of the multi-axis workpiece and the multi-hole workpiece.

[0097] Step S1120: Based on the first image, detect the deviation between the central shaft of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0098] In this embodiment, traditional assembly methods can be used to control the assembly of multi-axis and multi-hole workpieces. However, due to various errors, issues such as… Figure 2 The diagram illustrates a situation where a multi-axis workpiece and a multi-hole workpiece are in contact; the corresponding holes on the multi-axis workpiece and the multi-hole workpiece are not aligned. In other words, the shafts in the multi-axis workpiece and the corresponding holes in the multi-hole workpiece do not completely overlap. Therefore, the multi-axis workpiece cannot be assembled with the multi-hole workpiece. Based on this, a first image containing the contact surface between the multi-axis workpiece and the multi-hole workpiece is acquired using a camera. Image analysis is then performed on the first image to determine the deviation between the shaft in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece after contact.

[0099] In one embodiment of this application, deep learning can be used to perform image analysis on the first image, thereby obtaining the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece after the multi-axis workpiece comes into contact with the multi-hole workpiece.

[0100] In step S1200, if the deviation is greater than the preset deviation, the robotic arm is controlled to move the multi-axis workpiece based on the deviation, and the calculation operation is repeated until the deviation is less than or equal to the preset deviation.

[0101] The preset deviation is the maximum deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece when the robotic arm performs the spiral hole-searching operation and finds the hole in the multi-hole workpiece within the required time. It can be set according to requirements.

[0102] If the deviation calculated above exceeds the preset deviation, it indicates that the deviation between the shaft in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is too large. If the robotic arm is controlled to perform a helical hole-searching operation based on this deviation, the robotic arm needs to perform a large amount of trajectory search motion to find the hole in the multi-hole workpiece. In this case, the robotic arm is controlled to move the multi-axis workpiece based on the deviation. Specifically, the larger the deviation, the greater the movement of the robotic arm. If the currently calculated deviation is larger than the previously calculated deviation, the robotic arm moves the multi-axis workpiece in the opposite direction to the previous movement. If the currently calculated deviation is smaller than the previously calculated deviation, the robotic arm moves the multi-axis workpiece in the same direction as the previous movement. In this way, the shaft in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece gradually approach each other.

[0103] When the robotic arm moves the multi-axis workpiece, the position of the central axis of the multi-axis workpiece changes. At this time, the calculation operation is repeated to calculate the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece after the robotic arm moves the multi-axis workpiece. Based on this deviation, the above step S1200 is repeated until the deviation is less than or equal to the preset deviation.

[0104] Through steps S1100 and S1200, based on the first image containing the contact surface of the multi-axis workpiece and the multi-hole workpiece, the deviation between the corresponding shaft in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is controlled to be less than or equal to a preset deviation. In other words, the deviation between the shaft in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is reduced using computer vision.

[0105] Step S1300: Control the robotic arm to drive the multi-axis workpiece to perform a spiral hole-searching operation until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to the fourth preset force value.

[0106] In this embodiment, a force sensor is installed between the contact surface of the robotic arm and the multi-axis workpiece. The force collected by this force sensor is recorded as the actual contact force. The actual contact force is the contact force in the tool coordinate system T. The tool coordinate system T can be a coordinate system with the center of the flange at the end of the robotic arm as the origin, the Z-axis along the tool extension direction, the X-axis aligned with the long side of the flange locating pin / gripper, and the Y-axis determined according to the right-hand rule. The tool coordinate system T can be as follows: Figure 4 The coordinate system shown is as follows. The X-axis extends in the first direction, the Y-axis extends in the second direction, and the Z-axis extends in the third direction. The contact component of the actual contact force in the third direction can be expressed as... It can be read directly from the force sensor.

[0107] The fourth preset force value is the contact component of the maximum actual contact force collected by the force sensor in the third direction when a single axis in a multi-axis workpiece is aligned with the corresponding hole in a multi-hole workpiece. Theoretically, the fourth preset force value is 0. Considering factors such as error, the fourth preset force value can be a force value close to 0.

[0108] Step S1400: If the actual contact force value of the contact surface between the robotic arm and the multi-axis workpiece is less than or equal to the fourth preset force value, determine that the single axis of the multi-axis workpiece is aligned with the corresponding hole of the multi-hole workpiece.

[0109] When the deviation between the central axis of a multi-axis workpiece and the corresponding hole in a multi-hole workpiece is less than or equal to a preset deviation, the robotic arm can find the hole in the multi-hole workpiece within the required time during the spiral hole-finding operation. At this time, the robotic arm is controlled to drive the multi-axis workpiece to perform the spiral hole-finding operation. If the actual contact force value of the contact surface between the robotic arm and the multi-axis workpiece is less than or equal to a fourth preset force value, it indicates that the single axis in the multi-axis workpiece is aligned with the corresponding hole in the multi-hole workpiece, and the hole-finding task is completed.

[0110] Based on the premise that the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is less than or equal to a preset deviation, the hole-searching task can be quickly completed by controlling the robotic arm to drive the multi-axis workpiece to perform a helical hole-searching operation. Furthermore, the shaft-hole assembly method provided in this application does not directly control the robotic arm to move the multi-axis workpiece by an amount corresponding to the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece to achieve hole searching. Thus, it is not necessary to determine the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece with high precision, reducing the accuracy requirements for the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece. In other words, in this embodiment, it is sufficient to obtain only the fuzzy deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0111] Based on the above, it can be seen that by using computer vision to calculate the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece after they come into contact, the robotic arm can perform rapid spiral hole searching, thus improving the hole searching efficiency.

[0112] Step S1500: When a single axis in a multi-axis workpiece is aligned with a corresponding hole in a multi-hole workpiece, the operation of determining the bottom center position of the shaft that has entered the hole in the multi-axis workpiece is performed.

[0113] Wherein, the inserted shaft is the shaft in the multi-axis workpiece that is aligned with the hole in the multi-hole workpiece.

[0114] In a multi-axis workpiece, when a single axis is aligned with a corresponding hole in a multi-hole workpiece, determine which axis in the multi-axis workpiece is the inserted shaft aligned with a hole in the multi-hole workpiece, and determine the bottom center position of the inserted shaft in the multi-axis workpiece. Specifically, this can be the position coordinates of the bottom center of the inserted shaft in the multi-axis workpiece in the base coordinate system.

[0115] Step S1600: Control the robotic arm to drive the multi-axis workpiece to move on a set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to the third preset force value.

[0116] The set spherical surface is a spherical surface with the bottom center of the inserted shaft as the center and a set distance as the radius, wherein the set distance is the distance between the bottom center of the inserted shaft and the end of the robotic arm.

[0117] In this embodiment, when the actual contact force between the robotic arm and the multi-axis workpiece is less than or equal to the third preset force value in the third direction, the multi-axis in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece are completely aligned.

[0118] The third preset force value is the contact component of the maximum actual contact force collected by the force sensor in the third direction when the shaft in a multi-axis workpiece and the hole in a multi-hole workpiece are perfectly aligned. Theoretically, the third preset force value is 0. However, considering factors such as error, the third preset force value can be a force value close to 0, for example, 1 N.

[0119] like Figure 5 As shown, after obtaining the position of the bottom center of the inserted shaft in the base coordinate system, a coordinate system for the inserted shaft is constructed with this position as the origin and the extension direction of the inserted shaft as the Z-axis, and mutually perpendicular X-axis and Y-axis are constructed on the bottom plane of the multi-axis workpiece. For ease of description, the process of performing step S1200 will be referred to as attitude adjustment. Furthermore, the attitude of the inserted shaft during the attitude adjustment process will be represented by the inserted shaft coordinate system. Coordinate system This represents the orientation of the shaft already inserted into the hole before attitude adjustment. Coordinate system. This shows the orientation of the shaft after attitude adjustment, once it has been inserted into the hole. (Coordinate system) The coordinate system represents the pose of the end effector flange of the robotic arm before attitude adjustment. The coordinate system represents the pose of the end effector flange of the robotic arm after attitude adjustment. The orientation of the bottom center of the multi-axis workpiece. Coordinate system. This is the tool coordinate system, with its origin at the center of the end flange of the robotic arm, and the Z-axis along the flange's outgoing axis. The coordinate system is a multi-axis workpiece coordinate system, with its origin at the bottom center of the multi-axis workpiece and its z-axis along the extension direction of the axis. N is the distance between the bottom center of the inserted shaft and the bottom center of the multi-axis workpiece, M is the vertical distance between the bottom center of the multi-axis workpiece and the center of the end flange of the robotic arm, and L is the spatial straight line length between the bottom center of the inserted shaft and the center of the end flange of the robotic arm, i.e., the set distance. The three form a right triangle with N and M as the legs and L as the hypotenuse. During the attitude adjustment process, these three satisfy the following condition of formula (1):

[0120]

[0121] This ensures that the multi-axis workpiece always treats the bottom center of the inserted shaft as an immovable hinge during posture adjustment, moving on a set spherical surface with a radius of a set distance L, thus avoiding pulling out or bending the inserted shaft.

[0122] Based on the above, this method uses computer vision to calculate the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece after they come into contact. This assists the robotic arm in performing rapid spiral hole searching, improving the efficiency of hole searching. Furthermore, this method eliminates the need for high-precision determination of the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece, reducing the accuracy requirements for this deviation. In other words, only the fuzzy deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is needed to complete rapid hole searching, achieving single-axis rapid insertion of the multi-axis workpiece into a hole. Moreover, in the case of single-axis insertion of the multi-axis workpiece, the robotic arm is controlled to move the multi-axis workpiece on a sphere with the bottom center of the inserted shaft as the center and a set distance as the radius, until the non-inserted shaft in the multi-axis workpiece aligns with the corresponding hole. This avoids damage to the inserted shaft during the multi-axis insertion process. Furthermore, a corresponding motion trajectory is planned for multi-axis insertion, improving the efficiency and accuracy of multi-axis insertion and achieving rapid insertion of multiple holes from multiple axes.

[0123] In one embodiment of this application, the above step S1300 is specifically implemented by the following steps S1310 to S1330.

[0124] Step S1310: Obtain the tolerance between the central shaft of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0125] The tolerances between the central shaft of a multi-axis workpiece and the corresponding holes in a multi-hole workpiece can be specified by the operator based on requirements or experience.

[0126] Step S1320: Determine the helix pitch according to the tolerance.

[0127] In this embodiment, the helical pitch is... Determined to be less than The value of this is determined to avoid the problem of the axis on the multi-axis workpiece missing the corresponding hole in the multi-hole workpiece due to excessively large screw pitch when the robotic arm drives the multi-axis workpiece to move spirally. Among these, This refers to the tolerance between the central shaft of a multi-axis workpiece and the corresponding holes in a multi-hole workpiece.

[0128] Step S1330: According to the helix pitch, the robotic arm is controlled based on the Archimedes spiral to drive the multi-axis workpiece to perform a helical hole-searching operation.

[0129] Based on the pitch of the spirals, an Archimedean spiral trajectory is planned. Furthermore, the robotic arm is controlled to move the multi-axis workpiece along the planned Archimedean spiral trajectory.

[0130] In one embodiment of this application, the shaft hole assembly method provided in this application further includes the following step S1130 before the above step S1100.

[0131] Step S1130: Control the robotic arm to be in compliant impedance mode.

[0132] By using the above step S1300, the problem of excessive contact force between the axis of the multi-axis workpiece and the multi-hole workpiece when the robotic arm moves the multi-axis workpiece can be avoided, which could lead to damage to at least one of the robotic arm, the multi-axis workpiece, and the multi-hole workpiece.

[0133] In one embodiment of this application, step S1120 is specifically implemented through the following steps S1121 and S1122.

[0134] Step S1121: Detect the pairs of arc lines in the first image.

[0135] In cases where multi-axis and multi-hole workpieces are misaligned, the shaft in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece do not completely overlap. Since the edges of both the shaft and the hole in the multi-axis workpiece are arc-shaped, the aforementioned misaligned portions form a pair of crescent-shaped arc lines. This pair includes the arc line corresponding to the shaft edge and the arc line corresponding to the hole edge. It is understood that this pair of arc lines reflects the deviation between the shaft in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece. Therefore, after obtaining the first image, the pair of arc lines in the first image is detected.

[0136] In one embodiment of this application, arc pairs in the first image can be detected using deep learning.

[0137] In one embodiment of this application, the above step S1121 is specifically implemented by the following steps S1121-1 to S1121-3.

[0138] Step S1121-1: Perform a first image preprocessing operation on the first image to obtain the processed first image.

[0139] The first image preprocessing operation includes: grayscale processing, noise reduction, binarization, and interference filtering.

[0140] Specifically, after performing grayscale processing on the first image, a grayscale image is obtained that highlights the multi-axis workpiece and the multi-hole workpiece in the first image; the grayscale image is then denoised to remove noise and obtain a denoised image; the denoised image is then binarized according to a first preset grayscale threshold to obtain a binarized image that is convenient for subsequent arc extraction, wherein the first preset grayscale threshold is set based on experience; the binarized image is then subjected to interference elimination processing to obtain a first image after processing without interference information.

[0141] Noise removal in grayscale images can be achieved using a Gaussian filter. In one example, the Gaussian kernel in the Gaussian filter is set to a width and height of 5 pixels each, with a standard deviation of 0 in the X direction. Furthermore, the first preset grayscale threshold can be set empirically.

[0142] In one example, with Figure 2 For example, the first image after processing is as follows: Figure 3 As shown.

[0143] The processed first image obtained through step S1121-1 above facilitates the extraction of the arc in step S1121-2 below.

[0144] Step S1121-2: Extract arc lines from the processed first image to obtain at least two arc lines.

[0145] In one embodiment of this application, the processed first image is input to the Hough transform algorithm, and the Hough transform algorithm outputs at least two arcs in the first image.

[0146] The Hough transform is a method that "votes" pixels in the image space into a parameter space, and then searches for peaks of geometric shapes (lines, circles, ellipses, polygons, etc.) in the parameter space. It transforms discrete edge points, which are difficult to aggregate directly in the pixel domain, into a clustering problem in the parameter domain, making it particularly suitable for detecting regular geometric contours in noisy or incomplete environments. In Hough transform-based Hough arc detection, Canny edge detection is first used to obtain the edges, and then the plane equation of the circle is used... "Vote" for each edge to all possible A counter is established in the discretized parameter space. All edge points increment the count of the corresponding cell according to their possible parameter combinations. When the number of "votes" for a certain parameter cell exceeds the threshold, it is considered that an arc has appeared.

[0147] It should be noted that, in the case of single-axis multi-axis workpieces and single-hole multi-hole workpieces, if the multi-axis workpiece and the multi-hole workpiece are not aligned after contact, the processed first image will contain two arc lines. One arc line corresponds to the edge of the central axis of the multi-axis workpiece, and the other arc line corresponds to the edge of the central hole of the multi-hole workpiece. In the case of two or more multi-axis workpieces and two or more multi-hole workpieces, the processed first image will contain more than two arc lines.

[0148] Step S1121-3: Determine two intersecting arcs from at least two arcs as an arc pair.

[0149] In this embodiment, when the multi-axis workpiece and the multi-hole workpiece are not aligned after contact, the non-overlapping portion of any axis on the multi-axis workpiece and the corresponding hole in the multi-hole workpiece forms a crescent-shaped arc pair. Therefore, two intersecting arcs among at least two arcs are defined as an arc pair.

[0150] Step S1122: For any pair of arc lines, determine the first centroid position corresponding to the first centroid of the first arc line based on the first arc line in the pair of arc lines, determine the second centroid position corresponding to the second arc line based on the second arc line in the pair of arc lines, and calculate the deviation between the first centroid position and the second centroid position as the deviation between the central shaft of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0151] It is understandable that after obtaining the arc pair, the arc with the smaller curvature in the arc pair corresponds to the arc edge of the hole. The arc with the larger curvature in the arc pair corresponds to the arc edge of the shaft. Based on this, the arc corresponding to the hole edge in the arc pair is designated as the first arc, and the arc corresponding to the shaft edge in the arc pair is designated as the second arc. Alternatively, the arc corresponding to the hole edge in the arc pair is designated as the second arc, and the arc corresponding to the shaft edge in the arc pair is designated as the first arc.

[0152] For any pair of arcs, calculate the position of the first centroid of the first arc and record it as the first centroid position. Simultaneously calculate the position of the second centroid of the second arc and record it as the second centroid position. The deviation between the first centroid position and the second centroid position is recorded as the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0153] Through the above steps S1121 and S1122, the deviation between the central shaft of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece can be calculated.

[0154] In one embodiment of this application, the shaft-hole assembly method provided in this application further includes, before the above-described step S1100, the steps S1140 to S1180 shown below, steps for controlling the contact between the multi-axis workpiece and the multi-hole workpiece.

[0155] Step S1140: Obtain a second image containing the porous workpiece.

[0156] In this embodiment, an image of the porous workpiece to be assembled can be acquired by a camera to obtain a second image containing the porous workpiece.

[0157] Step S1150: Detect the edges of closed contours in the second image to obtain at least one closed contour edge.

[0158] In one embodiment of this application, the edges in the second image are detected using, for example, the Canny edge detection algorithm, to obtain at least one edge. Furthermore, since the edges of holes in a porous workpiece are closed contours, the closed contours of the aforementioned detected at least one edge are determined to be at least one circular edge, and the holes in the porous workpiece can be determined based on this circular edge.

[0159] Step S1160: Select closed contour edges with a roundness greater than a preset roundness from at least one closed contour edge to obtain at least one circular edge.

[0160] The preset roundness is the minimum roundness of a circle. The roundness of the closed contour edge can be calculated using the roundness calculation formula.

[0161] If the roundness of the closed contour edge is greater than or equal to the preset roundness, then the closed contour edge can be determined as a circular edge, denoted as a circular edge.

[0162] Step S1170: For any circular edge, determine the position of the third centroid of the circular edge.

[0163] For any circular edge, calculate the position of its centroid, and denote it as the third centroid position. This third centroid position reflects the position of the holes in the porous workpiece.

[0164] Step S1180: Based on the third centroid position, the robotic arm controls the multi-axis workpiece to contact the multi-hole workpiece.

[0165] Based on the position of the third centroid, the position of the hole in the multi-hole workpiece can be determined, thus achieving the positioning of the hole in the multi-hole workpiece. At this time, the multi-axis workpiece is controlled to contact the multi-hole workpiece.

[0166] When a multi-axis workpiece is held by a robotic arm, and the robotic arm controls the assembly of the multi-axis workpiece with the multi-hole workpiece, it is necessary to convert the third center of mass position into the center of mass position in the robotic arm coordinate system. Further, the robotic arm is controlled to move the multi-axis workpiece according to the converted center of mass position in the robotic arm coordinate system until the multi-axis workpiece contacts the multi-hole workpiece.

[0167] In one embodiment of this application, in order to accurately obtain at least one circular edge, the above step S1140 is specifically implemented through the following steps S1140-1 and S1140-2.

[0168] Step S1140-1: Perform a second image preprocessing operation on the second image to obtain the processed second image.

[0169] The second image preprocessing operation includes: grayscale processing, noise reduction processing, binarization processing, and circular hole display processing.

[0170] It should be noted that the grayscale processing, denoising processing, and binarization processing in the second image preprocessing operation are described in the same way as the relevant processing in the first image preprocessing operation, and will not be repeated here.

[0171] The above-mentioned circular hole display processing can be achieved by contour level filling. The circular hole display processing is used to highlight the circular holes in the image, which facilitates the detection of the edges of the closed contour in the following step S1140-2.

[0172] Step S1140-2: Detect the edges of closed contours in the processed second image to obtain at least one circular edge.

[0173] In this embodiment, edge detection of closed contours is performed on the processed second image, which can accurately detect at least one circular edge.

[0174] In one embodiment of this application, the operation of determining the bottom center position of the shaft that has entered the hole in the multi-axis workpiece in step S1500 includes: steps S1500.1 to S1500.4.

[0175] Step S1500.1: Obtain the first contact force of the contact surface between the robotic arm and the multi-axis workpiece.

[0176] In this embodiment, the first contact force is the actual contact force currently collected by the force sensor when a single axis in a multi-axis workpiece is aligned with a corresponding hole in a multi-hole workpiece. The first contact force is a vector.

[0177] The first contact force can characterize the current force exerted by the porous workpiece on the multi-axis workpiece.

[0178] Step S1500.2: Based on the first contact force, determine the shaft that has been inserted into the hole in the multi-axis workpiece.

[0179] In one embodiment of this application, the first contact force includes a first contact force component along a first direction and a second contact force component along a second direction, wherein the first direction and the second direction are perpendicular. Step S1500.2 determines the inserted shaft in the multi-axis workpiece based on the first contact force, including steps S1500.21 and S1500.22.

[0180] Step S1500.21: Determine the contact azimuth angle between the multi-axis workpiece and the multi-hole workpiece based on the first contact force component value and the second contact force component value.

[0181] In this embodiment, the first contact force is the current force exerted by the porous workpiece on the multi-axis workpiece. By decomposing the first contact force along the first direction and the second direction, we can obtain a first contact component along the first direction and a second contact component along the second direction. The first contact component can be expressed as... The second contact force can be expressed as Both can be read directly from the force sensor. The first contact force component represents the component of the force exerted by the porous workpiece on the multi-axis workpiece in the first direction. The second contact force component represents the component of the force exerted by the porous workpiece on the multi-axis workpiece in the second direction. The value of the first contact force component can be expressed as... The second contact force component can be expressed as .

[0182] To determine the inserted shaft in a multi-axis workpiece, it is necessary to know the force exerted by the multi-axis workpiece on the multi-hole workpiece. Therefore, based on action and reaction forces, by adding a negative sign to the first contact component, we obtain the contact component of the current force exerted by the multi-axis workpiece on the multi-hole workpiece in the first direction, which is called the reaction first contact component. Adding a negative sign to the second contact force yields the contact force in the second direction of the current force exerted by the multi-axis workpiece on the multi-hole workpiece; this is called the reverse second contact force. .

[0183] Since the relative positions of each axis in a multi-axis workpiece are fixed and known, let's assume that the number of axes in the multi-axis workpiece is n. Then, based on the number of axes n, we assign an angle range of 360° / n to each axis to obtain the azimuth angle range corresponding to each axis in the multi-axis workpiece.

[0184] For example, such as Figure 6As shown, the multi-axis workpiece is a three-axis workpiece. First, the azimuth angles of the three axes are measured in the xy plane of the tool coordinate system. Then, the three axes are translated into the drawing coordinate system. Next, the three axes are equally divided in the drawing coordinate system, i.e., each axis corresponds to 360° / 3 = 120°, resulting in three reference lines: 0°, 120°, and 240°. This divides the 0–360° circle into three equal parts, forming the azimuth angle ranges corresponding to different axes. The azimuth angle ranges corresponding to the three axes are as follows: , , Accordingly, the first axis corresponds to the first and second quadrants, the second axis to the third quadrant, and the third axis to the fourth quadrant. Similarly, the azimuth range of the four axes in a four-axis workpiece is... , , , The four axes correspond to the first quadrant, the second quadrant, the third quadrant, and the fourth quadrant, respectively.

[0185] In one example, when a multi-axis workpiece is a four-axis workpiece, there is one axis in each of the four quadrants. In this case, the axis already inserted in the hole in the four-axis workpiece can be directly determined based on the anti-first contact force and the anti-second contact force. That is, if For positive and If positive, then the shaft already inserted into the hole in the four-axis workpiece is... The axis. If negative and If positive, then the shaft already inserted into the hole in the four-axis workpiece is... The axis, if negative and If the value is negative, then the shaft that has entered the hole in the four-axis workpiece is... The axis, if For positive and If the value is negative, then the shaft that has entered the hole in the four-axis workpiece is... The axis.

[0186] In another example, if at least two axes exist in at least one of the four quadrants, or if one axis corresponds to at least two quadrants, the shaft already in the hole cannot be directly determined based on the anti-first contact force and the anti-second contact force. In this case, the contact azimuth angle between the multi-axis workpiece and the multi-hole workpiece can be calculated according to the following formula (2). :

[0187]

[0188] in, According to the anti-first contact component force Confirmed, if For positive and If it is positive, then The value is 0. If... negative and If it is positive, then The value is 1, if negative and If it is negative, then The value is 2, if For positive and If it is negative, then The value is 3.

[0189] The contact azimuth angle between the multi-axis workpiece and the multi-hole workpiece can characterize the resultant force azimuth angle of the force exerted by the multi-axis workpiece on the multi-hole workpiece in the xy plane of the tool coordinate system.

[0190] Step S1500.22: Determine the inserted shaft in the multi-axis workpiece according to the contact azimuth angle and the first correspondence relationship;

[0191] Wherein, the first correspondence is the range of contact azimuth angles when different axes of the multi-axis workpiece contact the multi-hole workpiece.

[0192] In this embodiment, since the distribution positions (i.e., the distribution azimuth range) of each axis in the multi-axis workpiece are different, the contact azimuth angles generated when different axes in the multi-axis workpiece come into contact with the multi-hole workpiece are also different.

[0193] For ease of calculation, the range of azimuth angles corresponding to different axes in a multi-axis workpiece is taken as the range of contact azimuth angles when different axes of the multi-axis workpiece come into contact with the multi-hole workpiece.

[0194] In other words, the azimuth ranges corresponding to the above three axes are as follows: , , This refers to the contact azimuth angle range corresponding to each of the three axes. The azimuth angle ranges corresponding to the four axes in a four-axis workpiece are as follows: , , , This refers to the range of contact azimuth angles corresponding to the four axes, which will not be elaborated here.

[0195] Continuing with the example above where the multi-axis workpiece is a three-axis workpiece, if when At that time, the shaft that has entered the hole is the shaft located in the first and second quadrants; when At that time, the shaft that has entered the hole is the shaft located in the third quadrant; when At that time, the shaft that has entered the hole is the shaft located in the fourth quadrant.

[0196] Step S15100.3: Obtain the first position of the end effector of the robotic arm.

[0197] In this embodiment, since the robotic arm end effector does not move before step S1500.3 after the single axis in the multi-axis workpiece is aligned with the corresponding hole in the multi-hole workpiece, the first position is the current position of the robotic arm end effector when the single axis in the multi-axis workpiece is aligned with the corresponding hole in the multi-hole workpiece. Specifically, it is the position coordinate of the robotic arm end effector in the base coordinate system. The base coordinate system is the coordinate of the robotic arm's installation position in the world coordinate system.

[0198] Step S1500.4: Determine the center position of the bottom of the inserted shaft based on the first position.

[0199] In this embodiment, the center position of the bottom of the inserted shaft can be specifically the position coordinates of the center of the bottom of the inserted shaft in the base coordinate system.

[0200] In one embodiment of this application, step S1500.4 determines the center position of the bottom of the inserted shaft based on the first position, including steps S1500.41 to S1500.43.

[0201] Step S1500.41: Determine the bottom center position of the multi-axis workpiece based on the first position and the first relative position relationship;

[0202] Wherein, the first relative positional relationship is the relative positional relationship between the end of the robotic arm and the bottom center of the multi-axis workpiece.

[0203] In this embodiment, the first relative positional relationship can be the relative positional relationship between the end effector of the robotic arm and the bottom center of the multi-axis workpiece in the base coordinate system. The position of the bottom center of the multi-axis workpiece can be the position coordinates of the bottom center of the multi-axis workpiece in the base coordinate system. This first relative positional relationship can be obtained through the following steps: Given that the method by which the robotic arm grips the multi-axis workpiece is known, the distance between the end effector of the robotic arm and the bottom center of the multi-axis workpiece is a fixed value, denoted as M. Then, the relative positional relationship between the end effector of the robotic arm and the bottom center of the multi-axis workpiece in the tool coordinate system can be expressed as follows: Next, the rotation matrix (also known as the first rotation matrix) between the robotic arm's end effector and the base coordinate system is calculated. and Multiply them to obtain the first relative positional relationship.

[0204] Then, in the base coordinate system, the first position is transformed through the first relative position relationship to obtain the position coordinates of the bottom center of the multi-axis workpiece in the base coordinate system.

[0205] The bottom center position of a multi-axis workpiece can be represented by formula (3) below. Calculation method:

[0206]

[0207] in, This is the current first position of the robotic arm's end effector.

[0208] Step S1500.42: Determine the second relative position relationship corresponding to the shaft that has been inserted into the hole based on the shaft that has been inserted into the hole and the second correspondence relationship;

[0209] The second correspondence is a storage of the relative positional relationship between the bottom centers of different axes of the multi-axis workpiece and the bottom center of the multi-axis workpiece.

[0210] In this embodiment, the second correspondence can specifically refer to the relative positional relationship between the bottom centers of different axes of the multi-axis workpiece and the bottom center of the multi-axis workpiece in the base coordinate system. Thus, the second relative positional relationship corresponding to the inserted shaft is the relative positional relationship between the inserted shaft and the bottom center of the multi-axis workpiece in the base coordinate system.

[0211] Taking a three-axis workpiece as an example, the second correspondence is explained as follows: the three axes and the center position of the bottom of the three-axis workpiece (i.e. Figure 6 The distances between the centers of the three axes are equal, each being N. Then, what is the phase relationship between the three axes and the workpiece bottom center in the tool coordinate system? It can be represented as: , , Then, the rotation matrix (also known as the second rotation matrix) from the bottom center of the multi-axis workpiece to the base coordinate system is used. The three relative positions Transform to the base coordinate system to obtain the positional transformation relationships between the three axes of the three-axis workpiece and the bottom center of the three-axis workpiece in the base coordinate system. Finally, based on the inserted shaft and this second correspondence, find the second relative positional relationship corresponding to the inserted shaft.

[0212] Step S1500.43: Determine the bottom center position of the inserted shaft based on the bottom center position of the multi-axis workpiece and the second relative position relationship.

[0213] In this embodiment, the center position of the bottom of the inserted shaft can be the position coordinate of the center of the bottom of the inserted shaft in the base coordinate system, denoted as: Then the center position of the bottom of the shaft that has been inserted into the hole can be calculated using the following formula (4):

[0214]

[0215] in, The second rotation matrix, i.e., the rotation matrix from the bottom center of the multi-axis workpiece to the base coordinate system, is determined by the fixed connection of the multi-axis workpiece. With the first rotation matrix The same can be determined by the posture of the robotic arm's end effector.

[0216] In one embodiment of this application, step S1600 involves controlling the robotic arm to move the multi-axis workpiece on a set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to a third preset force value, including steps S2100 to S2400.

[0217] Step S2100: Based on the first rotation angle, control the robotic arm to drive the multi-axis workpiece to move around the fourth direction on the set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is greater than or equal to the first preset force value.

[0218] In this embodiment, the first rotation angle can be determined based on the actual contact force collected by the current force sensor, or it can be a set small rotation angle; no limitation is made here.

[0219] The fourth direction is the X-axis direction of the already inserted hole axis coordinate system.

[0220] like Figure 7 As shown, it is a schematic diagram of a robotic arm driving a multi-axis workpiece to move around the X-axis of the already inserted hole axis coordinate system on a set spherical surface with radius L.

[0221] The first preset force value can be the critical force value at which the actual contact force measured by the force sensor begins to rise sharply in the third direction and the deviation between the multi-axis workpiece and the multi-hole workpiece begins to increase when the multi-axis workpiece rotates around the X-axis of its own coordinate system. If the actual contact force in the third direction is greater than or equal to the first preset force value, it indicates that the multi-hole workpiece and the multi-axis workpiece have begun to deviate. Continuing to rotate in the same direction will only increase the contact deformation between the two and will not improve the alignment accuracy. Therefore, the rotation around the X-axis of the already inserted hole axis should be stopped immediately.

[0222] The first preset force value can be, for example, 10N.

[0223] In one embodiment of this application, the first rotation angle in step S2100 is determined by the following steps: steps S2100.1 to S2100.3.

[0224] Step S2100.1: Obtain the second contact force of the contact surface between the robotic arm and the multi-axis workpiece in the first direction and the second contact resultant force in the second direction, and the third contact component force of the second contact force in the third direction.

[0225] In this embodiment, the second contact force can be the contact force collected by the current force sensor, and the second contact force can be equal to the first contact force.

[0226] The second contact resultant force can be expressed as: The third contact force is expressed as .

[0227] Step S2100.2: Determine the third rotation angle based on the second contact resultant force and the third contact component force.

[0228] In this embodiment, the third rotation angle It can be calculated using the following formula (5):

[0229]

[0230] Step S2100.3: Determine the first rotation angle based on the third rotation angle and the preset angle deviation value.

[0231] In this embodiment, the preset angle deviation can be set according to the measurement error of the system, and its specific value is not limited here.

[0232] By setting the angle deviation, After appropriate magnification, the first rotation angle is obtained. This prevents issues arising from a given... If it is too small, the X-axis attitude adjustment around the already inserted hole axis coordinate system will be incomplete.

[0233] In one embodiment of this application, step S2100 controls the robotic arm to move the multi-axis workpiece around a fourth direction on the set spherical surface based on a first rotation angle, including steps S2100.4 and S2100.5.

[0234] Step S2100.4: Based on the first rotation angle and the first pose of the end of the robotic arm, determine the first arc trajectory of the end of the robotic arm on the set spherical surface.

[0235] In this embodiment, the first pose includes a first position and a first posture. The first pose can be the current posture of the robotic arm's end effector. It is understood that, since the position of the robotic arm's end effector does not move after the single axis is aligned with the corresponding hole in the multi-hole workpiece, the first position in this step is the same as the first position in step S1100.3.

[0236] When planning the first circular arc trajectory, the first pose is first obtained. Then, the third pose (i.e., the pose after the robotic arm's end effector rotates by the first rotation angle around the fourth direction) is determined based on the first rotation angle. Next, the second pose (i.e., the pose after the robotic arm's end effector rotates by half of the first rotation angle around the fourth direction) is determined based on half of the first rotation angle. The first circular arc trajectory is obtained based on the first, second, and third poses. The second pose includes the second position and the second orientation. The third pose includes the third position and the third orientation.

[0237] The third position can be calculated using the following formula (6):

[0238]

[0239] in, It is the third position. The center position of the bottom of the shaft that has been inserted into the hole. As the first position, To be around the fourth direction, i.e., the coordinate system of the hole axis. The rotation matrix of the axis.

[0240] Similarly, the formula for calculating the second position is to take the first rotation angle... become Substituting into the above formula (6), we can calculate the result.

[0241] The first posture can be represented as Then the third pose can be represented as add The third posture can be represented as add .

[0242] Step S2100.5: Control the end of the robotic arm to drive the multi-axis workpiece to move along the first circular arc trajectory.

[0243] In this embodiment, the robotic arm end is controlled to drive the multi-axis workpiece to move along the first arc trajectory. The actual contact force in the third direction is judged in real time to see if it is greater than or equal to the first preset force value. If it is, the attitude adjustment around the X-axis of the inserted hole shaft is ended.

[0244] Step S2200: When the actual contact force in the third direction is greater than or equal to the first preset force value, the robotic arm is controlled based on the second rotation angle to drive the multi-axis workpiece to move around the fifth direction on the set spherical surface until the actual contact force in the third direction between the robotic arm and the multi-axis workpiece is greater than or equal to the second preset force value.

[0245] In this embodiment, the second rotation angle can be determined based on the actual contact force collected by the current force sensor, or it can be a set small rotation angle; no limitation is made here.

[0246] The fifth direction is the Y-axis direction of the already inserted hole axis coordinate system.

[0247] like Figure 8 As shown, it is a schematic diagram of a robotic arm driving a multi-axis workpiece to move around the Y-axis of the already inserted hole axis coordinate system on a set spherical surface with radius L.

[0248] The second preset force value can be the critical force value at which the actual contact force measured by the force sensor begins to rise sharply in the third direction and the deviation between the multi-axis workpiece and the multi-hole workpiece begins to increase when the multi-axis workpiece rotates around the Y-axis of its own coordinate system. If the actual contact force in the third direction is greater than or equal to the second preset force value, it indicates that the multi-hole workpiece and the multi-axis workpiece have begun to deviate. Continuing to rotate in the same direction will only increase the contact deformation between the two and will not improve the alignment accuracy. Therefore, the rotation around the Y-axis of the already inserted hole axis should be stopped immediately.

[0249] The second preset force value can be, for example, 10N.

[0250] In one embodiment of this application, the second rotation angle in step S2200 is determined by the following steps: steps S2200.1 to S2200.3.

[0251] Step S2200.1: When the actual contact force in the third direction is greater than or equal to the first preset force value, obtain the third contact force of the contact surface between the robotic arm and the multi-axis workpiece in the first and second directions, and the fourth contact force of the third contact force in the third direction.

[0252] In this embodiment, the third contact force can be the current contact force collected by the force sensor when the contact component of the actual contact force in the third direction is greater than or equal to the first preset force value.

[0253] Step S2200.2: Determine the fourth rotation angle based on the third contact resultant force and the fourth contact component force.

[0254] In this embodiment, the fourth rotation angle The calculation method is the same as that for the third rotation angle in step S2100.2, and will not be repeated here.

[0255] Step S2200.3: Determine the second rotation angle based on the fourth rotation angle and the preset angle deviation value.

[0256] In this embodiment, the preset angle deviation amount can be used to... After appropriate magnification, the second rotation angle is obtained. This prevents issues arising from a given... If it is too small, the Y-axis attitude adjustment around the already inserted hole axis coordinate system will be incomplete.

[0257] In one embodiment of this application, step S2200 controls the robotic arm to move the multi-axis workpiece around the fifth direction on the set spherical surface based on the second rotation angle, including steps S2200.4 and S2200.5.

[0258] Step S2200.4: Determine the second circular arc trajectory of the robotic arm on the set spherical surface based on the second rotation angle and the third pose of the end of the robotic arm.

[0259] In this embodiment, when planning the second circular arc trajectory, the third pose is first obtained, then the fifth pose is determined based on the second rotation angle (i.e., the pose of the robotic arm end effector after rotating by the second rotation angle around the fifth direction based on the third pose), and then the fourth pose is determined based on half of the second rotation angle (i.e., the pose of the robotic arm end effector after rotating by half of the second rotation angle around the fifth direction based on the third pose). The second circular arc trajectory is obtained based on the third, fourth, and fifth poses. The third pose includes the third position and the third orientation. The fourth pose includes the fourth position and the fourth orientation. The fifth pose includes the fifth position and the fifth orientation.

[0260] The fifth position can be calculated using the following formula (7):

[0261]

[0262] in, It is the third position. The center position of the bottom of the shaft that has been inserted into the hole. It is the fifth position. For the fifth direction, i.e., the coordinate system of the hole axis. The rotation matrix of the axis.

[0263] Similarly, the formula for calculating the fourth position is to take the second rotation angle. become Substitute into the above formula (7) to calculate.

[0264] The third posture can be represented as Then the fifth posture can be represented as add The fourth posture can be represented as add .

[0265] Step S2200.5: Control the end of the robotic arm to drive the multi-axis workpiece to move along the second circular arc trajectory.

[0266] In this embodiment, the robotic arm end is controlled to drive the multi-axis workpiece to move along the second arc trajectory. The actual contact force in the third direction is judged in real time to determine whether the contact force component is greater than or equal to the second preset force value. If so, the attitude adjustment around the Y-axis of the inserted hole shaft is ended.

[0267] Step S2300: When the actual contact force in the third direction is greater than or equal to the second preset force value, the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is calculated to obtain the deviation vector.

[0268] In one embodiment of this application, the calculation operation in step S2300 includes:

[0269] Step S2300.1: Obtain a third image containing the contact surface between the multi-axis workpiece and the multi-hole workpiece.

[0270] Step S2300.2: Based on the third image, detect the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0271] In this embodiment, a third image containing the contact surface between a multi-axis workpiece and a multi-hole workpiece is acquired using a camera. Image analysis is then performed on the third image to determine the deviation vector between the central axis of the multi-axis element and the corresponding hole in the multi-hole element after the multi-axis workpiece and the multi-hole workpiece come into contact.

[0272] In one embodiment of this application, deep learning can be used to perform image analysis on a third image, thereby obtaining the deviation vector between the central axis of the multi-axis element and the corresponding hole in the multi-hole element after the multi-axis workpiece and the multi-hole workpiece come into contact.

[0273] In one embodiment of this application, step S2300.2, which detects the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece based on the third image, includes steps S2300.21 to S2300.22.

[0274] Step S2300.21: Detect the pairs of arc lines in the third image.

[0275] When a multi-axis workpiece and a multi-hole workpiece are not perfectly aligned, the shaft in the multi-axis element and the corresponding hole in the multi-hole element do not completely coincide. Since the edges of the shaft in the multi-axis workpiece and the edges of the holes in the multi-hole workpiece are both arc-shaped, the aforementioned misaligned portions form a pair of crescent-shaped arc lines. This pair of arc lines includes the arc line corresponding to the shaft edge and the arc line corresponding to the hole edge. It can be understood that this pair of arc lines reflects the deviation vector between the shaft in the multi-axis workpiece and the corresponding hole in the multi-hole workpiece. Therefore, after obtaining the third image, the pair of arc lines in the third image is detected.

[0276] In one embodiment of this application, arc pairs in a third image can be detected using deep learning.

[0277] Step S2300.22: For any of the said arc line pairs, determine the third centroid position of the third centroid corresponding to the third arc line in the arc line pair, determine the fourth centroid position of the fourth arc line corresponding to the fourth arc line in the arc line pair, and calculate the deviation vector between the third centroid position and the fourth centroid position as the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0278] It is understandable that after obtaining the arc pair, the arc with the smaller radius in the arc pair corresponds to the arc edge of the hole. The arc with the larger radius in the arc pair corresponds to the arc edge of the shaft. Based on this, the arc corresponding to the hole edge in the arc pair is designated as the third arc, and the arc corresponding to the shaft edge in the arc pair is designated as the fourth arc. Alternatively, the arc corresponding to the hole edge in the arc pair is designated as the fourth arc, and the arc corresponding to the shaft edge in the arc pair is designated as the third arc.

[0279] For any pair of arcs, calculate the position of the third centroid of the third arc and denote it as the third centroid position. Simultaneously calculate the position of the fourth centroid of the fourth arc and denote it as the fourth centroid position. The deviation vector between the third and fourth centroid positions is denoted as the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

[0280] Based on the above, the deviation vector between the central shaft of a multi-axis workpiece and the corresponding hole in a multi-hole workpiece can be calculated.

[0281] Step S2400: Based on the deviation vector, control the robotic arm to drive the multi-axis workpiece to move around the sixth direction on the set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to the third preset force value.

[0282] In this embodiment, the sixth direction is the Z-axis direction of the already inserted hole axis coordinate system.

[0283] like Figure 9 As shown, it is a schematic diagram of a robotic arm driving a multi-axis workpiece to move around the Z-axis of the already inserted hole axis coordinate system on a set spherical surface with radius L.

[0284] By controlling the direction of the deviation vector, the robotic arm drives the multi-axis workpiece to rotate clockwise or counterclockwise within a set rotation angle range until the actual contact force of the force sensor in the third direction is less than or equal to the third preset force value.

[0285] The rotation angle range is set to be greater than 0 and less than or equal to 360° / n. No specific limit is imposed here. Where n is the number of axes in a multi-axis workpiece.

[0286] For example, a multi-axis workpiece is a three-axis workpiece. If the three-axis workpiece is currently located to the left of the three-hole workpiece, the robotic arm needs to rotate the multi-axis workpiece counterclockwise within a 0°-60° angle range around the Z-axis of the already inserted hole axis coordinate system until the component of the actual contact force from the force sensor in the third direction is less than or equal to a third preset force value. Conversely, if the three-axis workpiece is currently located to the right of the three-hole workpiece, the robotic arm needs to rotate the multi-axis workpiece clockwise within a 0°-60° angle range around the Z-axis of the already inserted hole axis coordinate system until the component of the actual contact force from the force sensor in the third direction is less than or equal to a third preset force value.

[0287] It should be noted that the deviation vector calculated here is only used as a basis for determining the direction of rotation, and is not used as a basis for calculating the specific rotation angle around the sixth direction.

[0288] This application also provides another electronic device 1000, such as Figure 10 As shown, the electronic device 1000 includes a force sensor 1020, a robotic arm 1010, a memory 1030, a processor 1040, and a camera 1050. The robotic arm 1010 is used to drive a multi-axis workpiece to move under the control of the processor 1040. The force sensor 1020 is used to collect the actual contact force value of the contact surface between the robotic arm 1010 and the multi-axis workpiece. The camera 1050 is used to collect a first image including the contact surface between the multi-axis workpiece and the multi-hole workpiece. The memory 1030 is used to store computer instructions. The processor 1040 is used to call the computer instructions from the memory 1030 to execute the method as described in any of the above method embodiments.

[0289] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above-described method embodiments.

[0290] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the above-described method embodiments.

[0291] This application may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0292] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0293] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0294] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing the status information of the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0295] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0296] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0297] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0298] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0299] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.

Claims

1. A method for assembling a shaft hole, characterized in that, include: In a multi-axis workpiece, when a single axis is aligned with a corresponding hole in a multi-hole workpiece, the operation of determining the bottom center position of the inserted shaft in the multi-axis workpiece is performed; wherein, the inserted shaft is the shaft in the multi-axis workpiece that is aligned with a hole in the multi-hole workpiece; The robotic arm is controlled to move the multi-axis workpiece on a set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to a third preset force value; wherein, the set spherical surface is a spherical surface with the bottom center of the inserted shaft as the center and a set distance as the radius, and the set distance is the distance between the bottom center of the inserted shaft and the end of the robotic arm; Before aligning the single axis in the multi-axis workpiece with the corresponding hole in the multi-hole workpiece, the method further includes: When a multi-axis workpiece is controlled to contact a multi-hole workpiece by a robotic arm, the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is calculated to obtain the deviation. If the deviation is greater than the preset deviation, the robotic arm is controlled to move the multi-axis workpiece based on the deviation, and the calculation operation is repeated until the deviation is less than or equal to the preset deviation. The robotic arm is controlled to drive the multi-axis workpiece to perform a spiral hole-searching operation until the actual contact force value between the robotic arm and the multi-axis workpiece is less than or equal to the fourth preset force value. If the actual contact force value of the contact surface between the robotic arm and the multi-axis workpiece is less than or equal to the fourth preset force value, it is determined that the single axis of the multi-axis workpiece is aligned with the corresponding hole of the multi-hole workpiece. The calculation operation includes: acquiring a first image containing the contact surface of the multi-axis workpiece and the multi-hole workpiece, and detecting the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece based on the first image.

2. The method according to claim 1, characterized in that, The control of the robotic arm to drive the multi-axis workpiece to perform a helical hole-searching operation includes: Obtain the tolerance between the central shaft of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece; Determine the helix pitch based on the aforementioned tolerance; Based on the pitch of the helix, the robotic arm is controlled according to the Archimedes spiral to drive the multi-axis workpiece to perform a helical hole-searching operation.

3. The method according to claim 1, characterized in that, Before calculating the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece when the multi-axis workpiece is controlled to contact the multi-hole workpiece by a robotic arm, and before obtaining the deviation, the method further includes: The robotic arm is controlled to be in a compliant impedance mode.

4. The method according to claim 1, characterized in that, The step of detecting the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece based on the first image includes: Detect pairs of arc lines in the first image; For any of the said arc line pairs, the first centroid position corresponding to the first centroid of the first arc line is determined according to the first arc line in the arc line pair, and the second centroid position corresponding to the second arc line is determined according to the second arc line in the arc line pair. The deviation between the first centroid position and the second centroid position is calculated as the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

5. The method according to claim 4, characterized in that, The detection of arc line pairs in the first image includes: Perform a first image preprocessing operation on the first image to obtain a processed first image. The first image preprocessing operation includes: grayscale processing, noise reduction processing, binarization processing, and interference filtering processing. Extract the arc lines from the processed first image to obtain at least two arc lines; Two intersecting arcs among the at least two arcs are defined as an arc pair.

6. The method according to claim 1, characterized in that, Before calculating the deviation between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece under the control of a robotic arm to make contact with the multi-hole workpiece, and obtaining the deviation, the method further includes: Obtain a second image containing the porous workpiece; Detect the edges that are closed contours in the second image to obtain at least one closed contour edge; From the at least one closed contour edge, select closed contour edges with a roundness greater than a preset roundness to obtain at least one circular edge; For any of the circular edges, determine the position of the third centroid of the circular edge; Based on the third centroid position, the robotic arm controls the multi-axis workpiece to contact the porous workpiece.

7. The method according to claim 6, characterized in that, The step of detecting edges that are closed contours in the second image to obtain at least one closed contour edge includes: Perform a second image preprocessing operation on the second image to obtain a processed second image. The first image preprocessing operation includes: grayscale processing, noise reduction processing, binarization processing, and circular hole display processing. Detect the edges of closed contours in the processed second image to obtain at least one closed contour edge.

8. The method according to claim 1, characterized in that, The control of the robotic arm to move the multi-axis workpiece on a set spherical surface until the actual contact force between the robotic arm and the multi-axis workpiece is less than or equal to a third preset force value in the third direction includes: Based on the first rotation angle, the robotic arm is controlled to drive the multi-axis workpiece to move around the fourth direction on the set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is greater than or equal to the first preset force value. When the actual contact force in the third direction is greater than or equal to the first preset force value, the robotic arm is controlled based on the second rotation angle to drive the multi-axis workpiece to move around the fifth direction on the set spherical surface until the actual contact force in the third direction between the robotic arm and the multi-axis workpiece is greater than or equal to the second preset force value. When the actual contact force in the third direction is greater than or equal to the second preset force value, the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is calculated to obtain the deviation vector. Based on the deviation vector control, the robotic arm drives the multi-axis workpiece to move around the sixth direction on the set spherical surface until the actual contact force of the contact surface between the robotic arm and the multi-axis workpiece in the third direction is less than or equal to the third preset force value. The sixth direction is the extension direction of the shaft that has been inserted into the hole, and the fourth and fifth directions are perpendicular in a plane that is perpendicular to the sixth direction.

9. The method according to claim 8, characterized in that, The first rotation angle is determined by the following steps: The second contact force of the contact surface between the robotic arm and the multi-axis workpiece is obtained in the first direction and the second contact resultant force in the second direction, and the third contact component force of the second contact force in the third direction. The third rotation angle is determined based on the second contact resultant force and the third contact component force; The first rotation angle is determined based on the third rotation angle and the preset angle deviation value; The second rotation angle is determined by the following steps: When the actual contact force in the third direction is greater than or equal to the first preset force value, the third contact force of the contact surface between the robotic arm and the multi-axis workpiece is obtained as the third contact resultant force in the first and second directions and the fourth contact component of the third contact force in the third direction. The fourth rotation angle is determined based on the third contact resultant force and the fourth contact component force. The second rotation angle is determined based on the fourth rotation angle and the preset angle deviation value.

10. The method according to claim 8, characterized in that, The calculation operations include: Obtain a third image containing the contact surface between the multi-axis workpiece and the multi-hole workpiece; Based on the third image, the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece is detected.

11. The method according to claim 10, characterized in that, The step of detecting the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece based on the third image includes: Detect the pairs of arc lines in the third image; For any pair of arc lines, the third centroid position corresponding to the third centroid of the third arc line is determined based on the third arc line in the pair of arc lines, and the fourth centroid position corresponding to the fourth arc line is determined based on the fourth arc line in the pair of arc lines. The deviation between the third centroid position and the fourth centroid position is calculated as the deviation vector between the central axis of the multi-axis workpiece and the corresponding hole in the multi-hole workpiece.

12. The method according to claim 8, characterized in that, The method of controlling the robotic arm to move the multi-axis workpiece around a first direction on the set spherical surface based on a first rotation angle includes: Based on the first rotation angle and the first pose of the robotic arm end, determine the first arc trajectory of the robotic arm end on the set spherical surface; The robotic arm end effector is controlled to drive the multi-axis workpiece to move along the first circular arc trajectory.

13. The method according to claim 1, characterized in that, The operation of determining the center position of the bottom of the shaft already inserted in the hole in the multi-axis workpiece includes: Obtain the first contact force at the contact surface between the robotic arm and the multi-axis workpiece; Based on the first contact force, determine the shaft that has been inserted into the hole in the multi-axis workpiece; Obtain the first position of the end effector of the robotic arm; Based on the first position, determine the center position of the bottom of the inserted shaft.

14. The method according to claim 13, characterized in that, The first contact force includes a first contact force component along a first direction and a second contact force component along a second direction, wherein the first direction and the second direction are perpendicular. Determining the inserted shaft in the multi-axis workpiece based on the first contact force includes: The contact azimuth angle between the multi-axis workpiece and the multi-hole workpiece is determined based on the first contact force component value and the second contact force component value. Based on the contact azimuth angle and the first correspondence, the shaft that has entered the hole in the multi-axis workpiece is determined; wherein, the first correspondence is a range of contact azimuth angles when different shafts of the multi-axis workpiece contact the multi-hole workpiece.

15. The method according to claim 13, characterized in that, Determining the center position of the bottom of the inserted shaft based on the first position includes: The bottom center position of the multi-axis workpiece is determined based on the first position and the first relative position relationship; wherein, the first relative position relationship is the relative position relationship between the end of the robotic arm and the bottom center of the multi-axis workpiece; Based on the inserted shaft and the second correspondence, the second relative position relationship corresponding to the inserted shaft is determined; wherein, the second correspondence relationship is the relative position relationship between the bottom centers of different shafts of the multi-axis workpiece and the bottom center of the multi-axis workpiece. The bottom center position of the inserted shaft is determined based on the bottom center position of the multi-axis workpiece and the second relative position relationship.

16. An electronic device, characterized in that, The electronic device includes a force sensor, a robotic arm, a memory, and a processor. The robotic arm is used to move the shaft workpiece under the control of the processor. The force sensor is used to collect the actual contact force value of the contact surface between the robotic arm and the shaft workpiece. The memory is used to store computer instructions. The processor is used to call the computer instructions from the memory to execute the method as described in any one of claims 1-15.

17. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method according to any one of claims 1-15.

Images