Workpiece assembly method, device, system, electronic device, and storage medium

Abstract

Embodiments of the present application provide a workpiece assembly method, device, system, electronic equipment and storage medium. The method comprises: controlling a robot arm holding a target workpiece to move to an image acquisition area, and acquiring a target image collected by an image acquisition device; determining a current pose of a jig flange, a current pose of the target workpiece and a pose of an object to be assembled based on the target image and parameters of the image acquisition device calibrated in advance; calculating a final pose of the jig flange according to the current pose of the jig flange, the current pose of the target workpiece, the pose of the object to be assembled and a preset assembly pose relationship; wherein the preset assembly pose relationship is determined based on a pose relationship when a teaching workpiece corresponding to the target workpiece and a teaching object corresponding to the object to be assembled are correctly assembled; and controlling the robot arm to assemble the target workpiece to the object to be assembled based on the current pose of the jig flange and the final pose of the jig flange. The workpiece is assembled based on the preset assembly pose relationship, and the workpiece assembly precision is improved.

Description

Technical Field

[0001] This application relates to the field of industrial automation technology, and in particular to a workpiece assembly method, apparatus, system, electronic device, and storage medium. Background Technology

[0002] With the continuous development of industrial automation technology, robotic arms are widely used in workpiece assembly processes in industries such as automobile manufacturing to improve assembly efficiency. Typically, when using robotic arms for workpiece assembly, AGVs (Automated Guided Vehicles) transport the workpiece to a preset position and fix it in that position according to a taught pose using physical clamping or other methods. Then, the robotic arm, holding the target workpiece, moves to the fixed position according to the taught method and assembles the target workpiece with the workpiece. However, in actual assembly processes, there are situations where the workpiece cannot be accurately fixed in the preset position according to the taught pose. In such cases, the robotic arm cannot complete the workpiece assembly according to the taught method.

[0003] To overcome the above problems, in one assembly method, the pose relationship between the robotic arm's clamping flange and the object to be assembled during correct assembly can be obtained in advance through teaching. Thus, during the actual assembly process, the pose of the robotic arm's clamping flange during correct assembly can be calculated based on the pose of the object to be assembled and the aforementioned pose relationship, and the workpiece can be assembled according to this clamping flange pose.

[0004] However, in the above assembly method, there are cases where the relative pose of the target workpiece and the clamping flange shifts after the target workpiece is clamped by the clamping flange due to unreasonable fixture design or other reasons. In such cases, even with the calculated correct clamping flange pose of the robotic arm, the target workpiece and the object to be assembled cannot be accurately assembled, and the workpiece assembly accuracy remains low. Summary of the Invention

[0005] The purpose of this application is to provide a workpiece assembly method, apparatus, system, electronic device, and storage medium to improve workpiece assembly accuracy. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a workpiece assembly method, the method comprising:

[0007] The robotic arm holding the target workpiece moves to the image acquisition area of ​​the image acquisition device and acquires the target image acquired by the image acquisition device, wherein the pre-placed object to be assembled is located within the image acquisition area;

[0008] Based on the target image and the pre-calibrated parameters of the image acquisition device, the current pose of the gripper flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled are determined.

[0009] The final pose of the fixture flange is calculated based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship; wherein, the preset assembly pose relationship is determined during the teaching process based on the pose relationship between the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled when they are correctly assembled.

[0010] Based on the current and final positions of the clamp flange, the robotic arm is controlled to assemble the target workpiece onto the object to be assembled.

[0011] Optionally, the step of calculating the final pose of the fixture flange based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and a preset assembly pose relationship includes:

[0012] Based on the current pose relationship, the pose of the object to be assembled, and the preset assembly pose relationship, the final pose of the clamping flange when the target workpiece and the object to be assembled are correctly assembled is calculated. The current pose relationship is the pose relationship between the clamping flange of the robot arm and the target workpiece, characterized by the current pose of the clamping flange and the current pose of the target workpiece.

[0013] Optionally, the step of calculating the final position of the fixture flange when the target workpiece and the object to be assembled are correctly assembled based on the current position relationship, the position of the object to be assembled, and the preset assembly position relationship includes:

[0014] Calculate the final position of the clamp flange using the following formula:

[0015] T′2R=obj22R×(obj22obj1) -1 ×(obj12R) -1 ×T2R

[0016] Wherein, T′2R is the final pose of the clamping flange in the robot arm coordinate system; obj22R is the pose of the object to be assembled in the robot arm coordinate system; obj22obj1 is the preset assembly pose relationship matrix; obj12R is the current pose of the target workpiece in the robot arm coordinate system; and T2R is the current pose of the clamping flange in the robot arm coordinate system.

[0017] Optionally, the method for constructing the preset assembly pose relationship includes:

[0018] The control arm holding the teaching workpiece moves to the teaching image acquisition area of ​​the teaching image acquisition device and acquires the first teaching image acquired by the teaching image acquisition device, wherein the pre-placed teaching object is located within the teaching image acquisition area;

[0019] Based on the first teaching image and the pre-calibrated parameters of the teaching image acquisition device, the current pose of the teaching jig flange, the current pose of the teaching workpiece, and the current pose of the teaching object of the teaching robot arm are determined.

[0020] The robot arm is controlled to assemble the teaching workpiece onto the teaching object, and a second teaching image is acquired by the teaching image acquisition device.

[0021] Based on the second teaching image and the pre-calibrated parameters of the teaching image acquisition device, determine the final position of the teaching jig flange of the teaching robot arm when it is correctly assembled;

[0022] Based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled.

[0023] Optionally, the step of calculating the preset assembly pose relationship between the taught workpiece and the taught object when correctly assembled, based on the current pose of the taught fixture flange, the current pose of the taught workpiece, the pose of the taught object, and the final pose of the taught fixture flange, includes:

[0024] Based on the current teaching pose relationship, the pose of the teaching object, and the final pose of the teaching fixture flange, the preset assembly pose relationship between the teaching workpiece and the teaching object is calculated when the assembly is correct. The current teaching pose relationship is the teaching pose relationship between the teaching fixture flange and the teaching workpiece characterized by the current pose of the teaching fixture flange and the current pose of the teaching workpiece.

[0025] Optionally, the step of calculating the preset assembly pose relationship between the taught workpiece and the taught object when correctly assembled, based on the current pose relationship, the pose of the taught object, and the final pose of the taught fixture flange, includes:

[0026] The preset assembly pose relationship is calculated using the following formula:

[0027] obj22obj1=(obj teach1 2R teach ) -1 ×T t ′ each 2Rteach ×(T teach 2R teach ) -1

[0028] ×obj teach2 2R teach

[0029] Among them, obj teach1 2R teach The current pose of the teaching workpiece in the coordinate system of the teaching robot arm; T′ teach 2R teach The current pose of the teaching fixture flange in the coordinate system of the teaching robot arm; T teach 2R teach The final pose of the teaching fixture flange in the coordinate system of the teaching manipulator arm; obj teach2 2R teach The teaching object is the pose of the teaching object in the coordinate system of the teaching robot arm.

[0030] Optionally, the step of calculating the preset assembly pose relationship between the taught workpiece and the taught object when correctly assembled, based on the current pose of the taught fixture flange, the current pose of the taught workpiece, the pose of the taught object, and the final pose of the taught fixture flange, includes:

[0031] Based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, the pose relationship between the teaching workpiece and the teaching object when correctly assembled is calculated as a candidate assembly pose relationship.

[0032] Calculate the difference between the candidate assembly pose relationship and the preset standard assembly pose relationship, and when the difference is not greater than a preset threshold, use the candidate assembly pose relationship as the preset assembly pose relationship. The preset standard assembly pose relationship is pre-constructed and determined based on the pose relationship between the standard workpiece corresponding to the teaching workpiece and the teaching object when they are correctly assembled. The teaching workpiece is one of the workpieces in the same batch, and the standard workpiece is the production template workpiece corresponding to the workpieces in that batch.

[0033] Optionally, the step of controlling the robotic arm to assemble the target workpiece onto the object to be assembled based on the current pose and final pose of the clamping flange includes:

[0034] Based on the current pose and the final pose of the clamp flange, the movement path of the robotic arm is planned.

[0035] The robotic arm is controlled to move along the movement path, and the clamping flange of the robotic arm is controlled to assemble the target workpiece onto the object to be assembled in the final position of the clamping flange.

[0036] Secondly, embodiments of this application provide a workpiece assembly apparatus, the apparatus comprising:

[0037] The image acquisition module is used to control the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device, and acquire the target image acquired by the image acquisition device, wherein the pre-placed object to be assembled is located within the image acquisition area;

[0038] The first pose determination module is used to determine the current pose of the clamping flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled based on the target image and the pre-calibrated parameters of the image acquisition device.

[0039] The second pose determination module is used to calculate the final pose of the fixture flange based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and a preset assembly pose relationship; wherein, the preset assembly pose relationship is determined during the teaching process based on the pose relationship between the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled when they are correctly assembled.

[0040] An assembly module is used to control the robotic arm to assemble the target workpiece onto the object to be assembled, based on the current and final positions of the clamp flange.

[0041] Optionally, the second pose determination module includes:

[0042] The first pose determination unit is used to calculate the final pose of the clamp flange when the target workpiece and the object to be assembled are correctly assembled, based on the current pose relationship, the pose of the object to be assembled, and the preset assembly pose relationship. The current pose relationship is the pose relationship between the clamp flange of the robot arm and the target workpiece, characterized by the current pose of the clamp flange and the current pose of the target workpiece.

[0043] Optionally, the first pose determination unit includes:

[0044] The first calculation subunit is used to calculate the final pose of the fixture flange according to the following formula:

[0045] T′2R=obj22R×(obj22obj1) -1 ×(obj12R) -1 ×T2R

[0046] Wherein, T′2R is the final pose of the clamping flange in the robot arm coordinate system; obj22R is the pose of the object to be assembled in the robot arm coordinate system; obj22obj1 is the preset assembly pose relationship matrix; obj12R is the current pose of the target workpiece in the robot arm coordinate system; and T2R is the current pose of the clamping flange in the robot arm coordinate system.

[0047] Optionally, the device further includes a preset assembly pose relationship construction module, the preset assembly pose relationship construction module comprising:

[0048] The first image acquisition unit is used to control the teaching robot arm holding the teaching workpiece to move to the teaching image acquisition area of ​​the teaching image acquisition device, and acquire the first teaching image acquired by the teaching image acquisition device, wherein the pre-placed teaching object is located in the teaching image acquisition area;

[0049] The second pose determination unit is used to determine the current pose of the teaching jig flange, the current pose of the teaching workpiece, and the current pose of the teaching object of the teaching robot arm based on the first teaching image and the parameters of the pre-calibrated teaching image acquisition device.

[0050] The second image acquisition unit is used to control the teaching robot arm to assemble the teaching workpiece onto the teaching object, and to acquire the second teaching image acquired by the teaching image acquisition device;

[0051] The third pose determination unit is used to determine the final pose of the teaching jig flange of the teaching robot arm when it is correctly assembled, based on the second teaching image and the parameters of the pre-calibrated teaching image acquisition device.

[0052] The fourth pose determination unit is used to calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when the assembly is correct, based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange.

[0053] Optionally, the fourth pose determination unit includes:

[0054] The second calculation subunit is used to calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled, based on the current teaching pose relationship, the pose of the teaching object, and the final pose of the teaching fixture flange. The current teaching pose relationship is the teaching pose relationship between the teaching fixture flange and the teaching workpiece characterized by the current pose of the teaching fixture flange and the current pose of the teaching workpiece.

[0055] Optionally, the second calculation subunit is specifically used for:

[0056] The preset assembly pose relationship is calculated using the following formula:

[0057] obj22obj1=(obj teach1 2R teach ) -1 ×T′ teach 2R teach ×(T teach 2R teach ) -1

[0058] ×obj teach2 2R teach

[0059] Among them, obj teach1 2R teach The current pose of the teaching workpiece in the coordinate system of the teaching robot arm; T′ teach 2R teach The current pose of the teaching fixture flange in the coordinate system of the teaching robot arm; T teach 2R teach The final pose of the teaching fixture flange in the coordinate system of the teaching manipulator arm; obj teach2 2R teach The teaching object is the pose of the teaching object in the coordinate system of the teaching robot arm.

[0060] Optionally, the fourth pose determination unit includes:

[0061] The third calculation subunit is used to calculate the pose relationship between the teaching workpiece and the teaching object when the assembly is correct, based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, as a candidate assembly pose relationship.

[0062] The difference calculation subunit is used to calculate the difference between the candidate assembly pose relationship and the preset standard assembly pose relationship, and when the difference is not greater than a preset threshold, the candidate assembly pose relationship is used as the preset assembly pose relationship. The preset standard assembly pose relationship is pre-constructed and determined based on the pose relationship between the standard workpiece corresponding to the teaching workpiece and the teaching object when they are correctly assembled. The teaching workpiece is one of the workpieces in the same batch, and the standard workpiece is the production template workpiece corresponding to the workpieces in this batch.

[0063] Optionally, the assembly module includes:

[0064] The path planning unit is used to plan the movement path of the robotic arm based on the current pose and the final pose of the clamping flange.

[0065] The control unit is used to control the robotic arm to move according to the movement path, and to control the clamping flange of the robotic arm to assemble the target workpiece to the object to be assembled in the final position of the clamping flange.

[0066] Thirdly, embodiments of this application provide a workpiece assembly system, the system including an image acquisition device and a processor:

[0067] The image acquisition device is used to acquire images of various objects located in the image acquisition area;

[0068] The processor is used to execute the steps of the method described in the first aspect above.

[0069] Fourthly, embodiments of this application provide an electronic device, including:

[0070] Memory, used to store computer programs;

[0071] When a processor executes a program stored in memory, it implements the steps of the method described in the first aspect above.

[0072] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0073] In a sixth aspect, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of the method described in the first aspect above.

[0074] Beneficial effects of the embodiments in this application:

[0075] In the solution provided in this application embodiment, the electronic device can control the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device and acquire the target image acquired by the image acquisition device. The object to be assembled is pre-placed within the image acquisition area. Based on the target image and the parameters of the pre-calibrated image acquisition device, the current pose of the gripper flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled are determined. The final pose of the gripper flange is calculated based on the current pose of the gripper flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship. The preset assembly pose relationship is determined during the teaching process based on the pose relationship between the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled when they are correctly assembled. Then, based on the current pose and the final pose of the gripper flange, the robotic arm is controlled to assemble the target workpiece onto the object to be assembled.

[0076] For a target workpiece and its corresponding object to be assembled, the pose relationship between the target workpiece and the object to be assembled should satisfy a preset assembly pose relationship during correct assembly. Since the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled, it does not change with the pose relationship between the target workpiece and the clamping flange of the robotic arm. Therefore, using the preset assembly pose relationship to assemble the target workpiece and the object to be assembled can avoid assembly failures caused by pose misalignment between the target workpiece and the clamping flange due to unreasonable clamping flange design, thus improving assembly accuracy. Furthermore, since the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled during correct assembly, it is independent of the current pose of the object to be assembled. Therefore, it can adapt to arbitrary placement of the object to be assembled, without needing to fix the object to be assembled in the same position. By real-time detection of the pose of the object to be assembled and the pose relationship between the target workpiece and the clamping flange, corrective assembly of the pose of the object to be assembled and the pose relationship between the target workpiece and the clamping flange can be achieved, reducing assembly costs. Of course, implementing any product or method of this application does not necessarily require achieving all the advantages described above simultaneously. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0078] Figure 1 A flowchart illustrating a workpiece assembly method provided in an embodiment of this application;

[0079] Figure 2This is a schematic diagram of the pose of a robotic arm, a target workpiece, and an object to be assembled, provided in an embodiment of this application.

[0080] Figure 3 This is a schematic flowchart illustrating the process of constructing a preset assembly pose relationship provided in an embodiment of this application.

[0081] Figure 4(a) is a schematic diagram of the pose of the robotic arm, the target workpiece, and the object to be assembled during the teaching process;

[0082] Figure 4(b) is a schematic diagram of another pose of the robotic arm, the target workpiece, and the object to be assembled during the teaching process;

[0083] Figure 5 for Figure 3 A specific flowchart of step S305 in the illustrated embodiment;

[0084] Figure 6 for Figure 1 A specific flowchart of step S104 in the illustrated embodiment;

[0085] Figure 7 This is a schematic diagram of the structure of a workpiece assembly device provided in an embodiment of this application;

[0086] Figure 8 This is a schematic diagram of the structure of a workpiece assembly system provided in an embodiment of this application;

[0087] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0088] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0089] The following explains the technical terms used in the embodiments of this application:

[0090] Workpieces are product components in the industrial manufacturing process;

[0091] Assembly is the process of assembling workpieces according to specified technical requirements and then debugging and inspecting them to make them into qualified products.

[0092] A flange, also known as a flange plate, is a symmetrical disc-shaped structure used to connect pipes, containers, or fixed shaft mechanical parts. It is usually connected to the workpiece by bolts or threads.

[0093] AGV is a transportation vehicle characterized by wheeled movement, computer control, equipped with a power (or power conversion) device and an electromagnetic (or optical, etc.) automatic guidance device, and capable of automatically traveling along a prescribed guidance path.

[0094] Registration is achieved by teaching the pose relationship between object A and object B.

[0095] Pose is the position and orientation of an object.

[0096] To improve the assembly accuracy of workpieces, embodiments of this application provide a workpiece assembly method, apparatus, system, electronic device, computer-readable storage medium, and computer program product. The workpiece assembly method provided by the embodiments of this application will be described first.

[0097] The workpiece assembly method provided in this application can be applied to any electronic device that can control a robotic arm and has image processing capabilities. Examples include an image acquisition device that can control a robotic arm and is equipped with an image processing module, a robotic arm that can communicate with an image acquisition device and is equipped with an image processing module, and a server that communicates with an image acquisition device and provides services to the robotic arm; these are not specifically limited here. For clarity, they will be referred to as electronic devices below.

[0098] like Figure 1 As shown, a workpiece assembly method includes:

[0099] S101: Control the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device, and acquire the target image acquired by the image acquisition device;

[0100] The pre-placed object to be assembled is located within the image acquisition area;

[0101] S102: Based on the target image and the pre-calibrated parameters of the image acquisition device, determine the current pose of the clamping flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled.

[0102] S103: Calculate the final pose of the fixture flange based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship.

[0103] The preset assembly posture relationship is determined during the teaching process based on the posture relationship between the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled when they are correctly assembled.

[0104] S104: Based on the current and final positions of the clamp flange, control the robotic arm to assemble the target workpiece onto the object to be assembled.

[0105] As can be seen, in the solution provided in this application embodiment, the electronic device can control the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device and acquire the target image acquired by the image acquisition device, wherein the pre-placed object to be assembled is located within the image acquisition area; based on the target image and the pre-calibrated parameters of the image acquisition device, the current pose of the gripper flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled are determined; according to the current pose of the gripper flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship, the final pose of the gripper flange is calculated; wherein, the preset assembly pose relationship is determined during the teaching process based on the pose relationship between the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled when they are correctly assembled; and then, based on the current pose of the gripper flange and the final pose of the gripper flange, the robotic arm is controlled to assemble the target workpiece to the object to be assembled.

[0106] For a target workpiece and its corresponding object to be assembled, the pose relationship between the target workpiece and the object to be assembled should satisfy a preset assembly pose relationship during correct assembly. Since the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled, it does not change with the pose relationship between the target workpiece and the clamping flange of the robotic arm. Therefore, using the preset assembly pose relationship to assemble the target workpiece and the object to be assembled can avoid assembly failures caused by pose misalignment between the target workpiece and the clamping flange due to unreasonable clamping flange design, thus improving assembly accuracy. Furthermore, since the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled during correct assembly and is independent of the current pose of the object to be assembled, it can accommodate arbitrary placement of the object to be assembled without fixing it to a single position. By real-time detection of the pose of the object to be assembled and the pose relationship between the target workpiece and the clamping flange, corrective assembly can be achieved, reducing assembly costs.

[0107] In industries such as automobile manufacturing, robotic arms are commonly used to assemble workpieces to meet requirements for assembly precision and efficiency. Examples include assembling car door panels and bearings. For each product, to facilitate the use of robotic arms for workpiece assembly during production, the workpiece assembly process can be pre-taught. In actual production, the robotic arm can then be controlled to assemble workpieces according to the taught process.

[0108] For each product, the assembly process may include the assembly process of multiple workpieces. For each workpiece, the assembly process of that workpiece can be taught separately.

[0109] In complex workpiece assembly scenarios, to avoid workpiece assembly failures caused by deviations in the actual position of the object to be assembled compared to the taught position, or by unreasonable fixture design leading to deviations in the pose relationship between the target workpiece and the fixture flange compared to the taught pose relationship, and to meet the accuracy requirements for workpiece assembly, a 3D (three-dimensional) vision-guided robotic arm can be used for workpiece assembly. That is, 3D vision positioning guides the robotic arm to correctly assemble the workpiece with the object to be assembled.

[0110] To utilize 3D vision to guide the robotic arm in workpiece assembly, an image acquisition device can be pre-installed within the workpiece assembly area. This image acquisition device can be a monocular camera, a binocular camera, etc., and is not specifically limited here. When installing the image acquisition device, its image acquisition area should cover the workpiece assembly area, meaning that the image acquisition device can clearly and completely capture images of the robotic arm and the object to be assembled within the workpiece assembly area.

[0111] After the image acquisition device is installed, its parameters can be adjusted and calibrated. This involves calibrating the transformation relationship between the camera coordinates and the image coordinate system, as well as the transformation relationship between the camera coordinates and the robotic arm coordinate system, to facilitate subsequent calculations of pose relationships. The calibration methods can include Zhang Zhengyou's calibration method, the nine-point calibration method, etc., and are not specifically limited here.

[0112] During assembly operations, when it is desired to assemble objects, the objects to be assembled can be placed in the image acquisition area beforehand. For example, an AGV (Automated Guided Vehicle) can be controlled to transport the objects to be assembled into the image acquisition area.

[0113] Furthermore, in step S101 above, the electronic device controls the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device, and acquires the target image acquired by the image acquisition device.

[0114] When using a robotic arm to assemble a target workpiece located within an image acquisition area, the electronic equipment can control the robotic arm to move the target workpiece to its position and control the gripper flange of the robotic arm to hold the target workpiece. The electronic equipment can then control the robotic arm holding the target workpiece to move to the image acquisition area and acquire the target image captured by the image acquisition device.

[0115] In one implementation, when the electronic device controls the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device, it controls the image acquisition device to acquire the target image. The image acquisition device acquires the target image and sends it to the electronic device so that the electronic device can obtain the target image.

[0116] In one implementation, the image acquisition device acquires images of the image acquisition area at a preset frequency. The preset frequency can be set according to actual needs, such as 10 times / second, 50 times / second, etc., and is not specifically limited here. When the electronic device controls the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device, it sends an image acquisition request carrying a timestamp representing the moment the robotic arm moves to the image acquisition area, and receives the target image sent by the image acquisition device based on the aforementioned timestamp.

[0117] Since the robotic arm holding the target workpiece and the object to be assembled are both located within the image acquisition area, the target image acquired by the electronic device is an image that includes the robotic arm, the target workpiece, and the object to be assembled.

[0118] For example, after the robotic arm holding the target workpiece moves to the image acquisition area of ​​the image acquisition device under the control of the electronic device, the respective positions of the image acquisition device 210, the robotic arm 220, the target workpiece 230, and the object to be assembled 240 are as follows: Figure 2 As shown, the image acquisition device 210 can acquire clear images of the robotic arm 220, the target workpiece 230, and the object to be assembled 240. In this way, the electronic device can acquire the target image of the robotic arm 220, the target workpiece 230, and the object to be assembled 240 acquired by the image acquisition device 210.

[0119] Then, in step S102 above, the electronic device determines the current pose of the gripper flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled based on the target image and the parameters of the pre-calibrated image acquisition device.

[0120] Since the target image includes the robotic arm, the target workpiece, and the object to be assembled, the electronic device can determine the current pose of the robotic arm's clamping flange, the current pose of the target workpiece, and the pose of the object to be assembled based on the target image and the parameters of the pre-calibrated image acquisition device after acquiring the target image.

[0121] The current pose of the gripper flange of the robotic arm can be either the current pose of the gripper flange in the coordinate system of the robotic arm base or the pose of the gripper flange in the coordinate system of other reference objects in the image acquisition area; no specific limitation is made here. Similarly, the current pose of the target workpiece can be either the current pose of the target workpiece in the coordinate system of the robotic arm base or the pose of the target workpiece in the coordinate system of other reference objects in the image acquisition area; the pose of the object to be assembled can be either the current pose of the object to be assembled in the coordinate system of the robotic arm base or the pose of the object to be assembled in the coordinate system of other reference objects in the image acquisition area. For ease of calculation, the current poses of the gripper flange, the target workpiece, and the object to be assembled are usually defined as poses in the same coordinate system.

[0122] During each run, by using the target image to determine the pose of the object to be assembled, a secondary correction and positioning of the object to be assembled can be achieved, thereby solving the problems of inability to assemble or assembly deviation caused by the positional deviation of the object to be assembled in traditional assembly schemes.

[0123] For example, such as Figure 2 As shown, the electronic device can determine the current pose T2R of the clamping flange of the robotic arm 220, the current pose ibj12R of the target workpiece 230, and the pose obj22R of the object to be assembled 240 in the coordinate system of the robotic arm base based on the target image and the parameters of the pre-calibrated image acquisition device.

[0124] For various types of workpieces, the assembly process can be taught in advance using the corresponding teaching workpiece and teaching object. During the teaching process, the electronic device can determine a preset assembly pose relationship based on the pose relationship between the teaching workpiece and the teaching object when they are correctly assembled. Correspondingly, for a target workpiece, the assembly process can be taught in advance using the corresponding teaching workpiece and the teaching object corresponding to the object to be assembled. Furthermore, during the teaching process, the electronic device can determine a preset assembly pose relationship based on the pose relationship between the teaching workpiece and the teaching object when they are correctly assembled. That is, this preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled when they are correctly assembled.

[0125] When using a preset assembly pose matrix to represent the preset assembly pose relationship, the process of obtaining the preset assembly pose matrix of the target workpiece and the object to be assembled when correctly assembled through teaching is called assembly matrix registration.

[0126] For clarity, the method for establishing the preset assembly pose relationship will be explained in detail below.

[0127] Furthermore, in step S103 above, the electronic device can calculate the final pose of the fixture flange based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship.

[0128] For any clamping flange, after the clamping flange clamps the target workpiece, the positional relationship between the clamping flange and the target workpiece will not change during the assembly process. Furthermore, regardless of whether the object to be assembled is placed in the teaching position, the positional relationship between the target workpiece and the object to be assembled must satisfy the pre-calibrated preset assembly positional relationship when the assembly is successful.

[0129] Furthermore, during the assembly process, the electronic equipment assembles the target workpiece and the object to be assembled by controlling the movement of the robotic arm. When the position of the object to be assembled remains fixed, the electronic equipment can move the clamping flange of the robotic arm to achieve the correct assembly of the target workpiece and the object to be assembled. That is, when the position of the object to be assembled and the positional relationship between the clamping flange and the target workpiece remain unchanged, the electronic equipment can adjust the position of the target workpiece by controlling the movement of the clamping flange to adjust the position of the clamping flange.

[0130] After determining the current pose of the fixture flange, the target workpiece, and the object to be assembled, the electronic device can calculate the final pose of the fixture flange based on these poses and a preset assembly pose relationship. This final pose is the correct pose of the fixture flange during assembly; that is, when the fixture flange has this final pose, the target workpiece can be correctly assembled with the object to be assembled.

[0131] As one implementation method, the electronic device can calculate the pose relationship between the target workpiece and the clamping flange based on the current pose of the clamping flange and the current pose of the target workpiece, and calculate the final pose of the target workpiece when it is correctly assembled based on the pose of the object to be assembled and the preset assembly pose relationship. Then, the electronic device can calculate the final pose of the clamping flange based on the pose relationship between the target workpiece and the clamping flange and the final pose of the target workpiece.

[0132] As one implementation method, the electronic device can calculate the pose relationship between the target workpiece and the clamping flange based on the current pose of the clamping flange and the current pose of the target workpiece. Based on the pose relationship between the target workpiece and the clamping flange, as well as the preset assembly pose relationship, the electronic device can calculate the pose relationship between the clamping flange and the object to be assembled when correctly assembled. Then, the electronic device can calculate the final pose of the clamping flange based on the pose relationship between the clamping flange and the object to be assembled when correctly assembled, as well as the pose of the object to be assembled.

[0133] Furthermore, in step S104 above, the electronic device can control the robotic arm to assemble the target workpiece onto the object to be assembled based on the current and final poses of the clamping flange. After determining the final pose of the clamping flange, the electronic device can determine the robotic arm control strategy based on the current and final poses of the clamping flange, and then control the robotic arm to move according to the determined control strategy so that the robotic arm assembles the target workpiece onto the object to be assembled. Moreover, when the target workpiece is assembled onto the object to be assembled, the pose of the clamping flange of the robotic arm is the aforementioned final pose of the clamping flange.

[0134] As one embodiment of this application, the step of calculating the final pose of the fixture flange based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship may include:

[0135] Based on the current pose relationship, the pose of the object to be assembled, and the preset assembly pose relationship, the final pose of the clamping flange when the target workpiece and the object to be assembled are correctly assembled is calculated. The current pose relationship is the pose relationship between the clamping flange of the robot arm and the target workpiece, characterized by the current pose of the clamping flange and the current pose of the target workpiece.

[0136] Because the positional relationship between the clamping flange and the target workpiece remains unchanged after the clamping flange holds the target workpiece during assembly, the electronic equipment can determine the positional relationship between the clamping flange and the target workpiece based on the current positions of the clamping flange and the target workpiece, thus obtaining the current positional relationship represented by the current positions of the clamping flange and the target workpiece. By performing secondary correction and positioning on the positional relationship between the target component and the clamping flange, the problem of assembly failure caused by positional misalignment between the target component and the clamping flange due to unreasonable clamp design in traditional assembly schemes is solved.

[0137] Subsequently, the electronic device can calculate the final position of the fixture flange when the target workpiece and the object to be assembled are correctly assembled, based on the current position relationship, the position of the object to be assembled, and the preset assembly position relationship.

[0138] As one implementation method, the electronic device can calculate the final pose of the target workpiece when it is correctly assembled based on the pose of the object to be assembled and the preset assembly pose relationship, and calculate the final pose of the fixture flange based on the current pose relationship and the final pose of the target workpiece.

[0139] As one implementation method, the electronic device can calculate the pose relationship between the fixture flange and the object to be assembled when correctly assembled based on the current pose relationship and the preset assembly pose relationship, and calculate the final pose of the fixture flange based on the pose relationship between the fixture flange and the object to be assembled when correctly assembled, and the pose of the object to be assembled.

[0140] In this embodiment, after determining the current pose relationship between the fixture flange and the target workpiece, the electronic device can use this current pose relationship, the pose of the object to be assembled, and the preset assembly pose relationship to calculate the final pose of the fixture flange when the target workpiece and the object to be assembled are correctly assembled. Since the current pose relationship between the fixture flange and the target workpiece is calculated in real time, that is, for all types of fixture flanges, the current pose relationship between the fixture flange and the target workpiece needs to be recalculated after the fixture flange clamps the target workpiece, this can avoid workpiece assembly failure caused by the pose relationship between the target workpiece and the fixture flange shifting compared to the taught pose relationship due to unreasonable fixture design, thereby improving the workpiece configuration accuracy.

[0141] As one embodiment of this application, the step of calculating the final position of the fixture flange when the target workpiece and the object to be assembled are correctly assembled based on the current position relationship, the position of the object to be assembled, and the preset assembly position relationship may include:

[0142] Calculate the final position of the clamp flange using the following formula:

[0143] T′2R=obj22R×(obj22obj1) -1 ×(obj12R) -1 ×T2R

[0144] Wherein, T′2R is the final pose of the clamping flange in the robot arm coordinate system; obj22R is the pose of the object to be assembled in the robot arm coordinate system; obj22obj1 is the preset assembly pose relationship matrix; obj12R is the current pose of the target workpiece in the robot arm coordinate system; and T2R is the current pose of the clamping flange in the robot arm coordinate system.

[0145] Based on the pre-calibrated preset assembly pose relationship matrix, when the electronic device calculates the final pose of the fixture flange according to the above formula, the pose of the object to be assembled in the robot arm coordinate system can remain unchanged; the relationship between the current pose of the target workpiece in the robot arm coordinate system and the current pose of the fixture flange in the robot arm coordinate system, representing the current pose of the target workpiece and the fixture flange, can also remain unchanged. Therefore, based on the above current pose relationship, the pose of the object to be assembled, and the preset assembly pose relationship, the electronic device can calculate the final pose of the fixture flange when correctly assembled.

[0146] In this embodiment, the electronic device calculates the final position of the fixture flange when the target workpiece and the object to be assembled are correctly assembled using the above formula. This allows for more accurate and faster workpiece assembly, improving workpiece assembly precision.

[0147] As one implementation method of this application, such as Figure 3 As shown, the method for constructing the above-mentioned preset assembly pose relationship can include:

[0148] S301: Control the teaching robot arm holding the teaching workpiece to move to the teaching image acquisition area of ​​the teaching image acquisition device, and acquire the first teaching image acquired by the teaching image acquisition device;

[0149] The pre-placed teaching object is located within the teaching image acquisition area;

[0150] S302: Based on the first teaching image and the pre-calibrated parameters of the teaching image acquisition device, determine the current pose of the teaching jig flange, the current pose of the teaching workpiece, and the current pose of the teaching object of the teaching robot arm;

[0151] S303: Control the teaching robot arm to assemble the teaching workpiece onto the teaching object, and acquire the second teaching image acquired by the teaching image acquisition device;

[0152] S304: Based on the second teaching image and the pre-calibrated parameters of the teaching image acquisition device, determine the final position of the teaching jig flange of the teaching robot arm when it is correctly assembled;

[0153] S305: Based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled.

[0154] For each workpiece assembly process, the assembly process can be taught in advance using the teaching workpiece and the teaching object. During the teaching process, a teaching image acquisition device can be installed in the teaching assembly area beforehand. When installing the teaching image acquisition device, its teaching image acquisition area should cover the teaching workpiece assembly area, meaning the device can clearly and completely capture images of the teaching robot arm and the teaching object located in the teaching workpiece assembly area.

[0155] After the teaching image acquisition device is installed, its parameters can be adjusted and calibrated. This includes calibrating the transformation relationship between the camera coordinates and the image coordinate system, as well as the transformation relationship between the camera coordinates and the teaching robot arm coordinate system, to facilitate subsequent calculations of pose relationships.

[0156] During the assembly process, when it is desired to assemble the workpiece on the teaching object, the teaching object can be placed in the teaching image acquisition area in advance.

[0157] Next, the electronic device can control the teaching robot arm to move to the teaching workpiece and clamp the teaching workpiece. Then, it controls the teaching robot arm holding the teaching workpiece to move to the teaching image acquisition area of ​​the teaching image acquisition device and acquire the first teaching image acquired by the teaching image acquisition device.

[0158] Next, the electronic device can determine the current pose of the teaching jig flange, the current pose of the teaching workpiece, and the current pose of the teaching object based on the first teaching image and the parameters of the pre-calibrated teaching image acquisition device.

[0159] The teaching robot arm is then controlled to assemble the workpiece to the teaching object, and a second teaching image is acquired by the teaching image acquisition device. The electronic device can determine the final pose of the teaching fixture flange of the teaching robot arm when correctly assembled, based on the second teaching image and the pre-calibrated parameters of the teaching image acquisition device. Furthermore, based on the current pose of the teaching fixture flange, the current pose of the workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, the preset assembly pose relationship between the workpiece and the teaching object when correctly assembled is calculated.

[0160] In this way, through the teaching process, a preset assembly posture relationship can be obtained based on the posture relationship between the taught workpiece and the taught object when they are correctly assembled.

[0161] In subsequent assembly operations, for the workpieces corresponding to the taught workpieces, the preset assembly posture relationship determined by the taught workpieces and the taught objects can be used to assemble the workpieces.

[0162] For example, after the electronic device controls the teaching robot arm holding the teaching workpiece to move to the teaching image acquisition area of ​​the teaching image acquisition device, the respective positions of the teaching image acquisition device 410, the teaching robot arm 420, the teaching workpiece 430, and the teaching object 440 are shown in Figure 4(a). The teaching image acquisition device 410 can acquire clear images of the teaching robot arm 420, the teaching workpiece 430, and the teaching object 440. In this way, the electronic device can obtain the first teaching image acquired by the teaching image acquisition device 410, including the teaching robot arm 420, the teaching workpiece 430, and the teaching object 440, and use the first teaching image and the pre-calibrated parameters of the teaching image acquisition device to determine the current pose T′ of the teaching fixture flange of the teaching robot arm 420. teach 2R teach The current pose of the teaching workpiece 430 (obj) teach1 2R teach And the current pose of the teaching object 440 (obj) teach2 2R teach .

[0163] Afterwards, the electronic device can continue to control the teaching robot arm 420 to assemble the teaching workpiece 430 onto the teaching object 440. During correct assembly, the respective positions of the teaching image acquisition device 410, the teaching robot arm 420, the teaching workpiece 430, and the teaching object 440 are shown in Figure 4(b). The electronic device acquires the second teaching image captured by the teaching image acquisition device 410. Then, based on the second teaching image and the pre-calibrated parameters of the teaching image acquisition device, it can determine the final pose T of the teaching fixture flange of the teaching robot arm 420 during correct assembly. teach 2R teach .

[0164] Furthermore, the electronic device can calculate the preset assembly pose relationship between the taught workpiece and the taught object when correctly assembled, based on the current pose of the taught fixture flange, the current pose of the taught workpiece, the pose of the taught object, and the final pose of the taught fixture flange.

[0165] In one implementation, two image acquisition devices can be installed in the teaching assembly area. When the electronic device controls the teaching robotic arm to move into the image acquisition area, it acquires point cloud images from the two devices. These point cloud images are then processed to extract point cloud images containing only the target workpiece and only the object to be assembled. Next, the electronic device uses the GASD algorithm to calculate the assembly angle and position on the two point cloud images, obtaining the preset assembly pose relationship between the target workpiece and the object to be assembled when correctly assembled.

[0166] In this embodiment, since the pose relationship matrix between the teaching workpiece and the teaching object is independent of various factors such as the design, quality, and batch of the fixture flange, the preset assembly pose relationship does not need to be recalculated even if the fixture is changed during subsequent assembly operation. Assembly can be carried out smoothly by directly using the preset assembly pose relationship.

[0167] As one embodiment of this application, step S305 above, which calculates the preset assembly posture relationship between the teaching workpiece and the teaching object when correctly assembled based on the current posture of the teaching fixture flange, the current posture of the teaching workpiece, the posture of the teaching object, and the final posture of the teaching fixture flange, may include:

[0168] Based on the current teaching pose relationship, the pose of the teaching object, and the final pose of the teaching fixture flange, the preset assembly pose relationship between the teaching workpiece and the teaching object is calculated when the assembly is correct. The current teaching pose relationship is the teaching pose relationship between the teaching fixture flange and the teaching workpiece characterized by the current pose of the teaching fixture flange and the current pose of the teaching workpiece.

[0169] Since the positional relationship between the clamping flange and the target workpiece remains unchanged after the clamping flange clamps the target workpiece during the assembly process, the electronic device can determine the positional relationship between the clamping flange and the workpiece of the teaching robot arm based on the current positional position of the teaching clamping flange and the current positional position of the teaching workpiece after determining the current positional position of the teaching clamping flange and the current positional position of the teaching workpiece, and obtain the current positional relationship represented by the current positional position of the teaching clamping flange and the current positional position of the teaching workpiece.

[0170] Subsequently, the electronic device can calculate the preset assembly pose relationship between the taught workpiece and the taught object when correctly assembled, based on the current pose relationship, the pose of the taught object, and the final pose of the taught fixture flange.

[0171] In one implementation, the electronic device can calculate the final pose of the target workpiece when correctly assembled based on the current pose relationship and the final pose of the teach fixture flange, and calculate the preset assembly pose relationship between the teach workpiece and the teach object when correctly assembled based on the final pose of the target workpiece and the pose of the teach object.

[0172] In one implementation, the electronic device can calculate the pose relationship between the taught object and the taught fixture flange when correctly assembled, based on the pose of the taught object and the final pose of the taught fixture flange. Based on the pose relationship between the taught object and the taught fixture flange, and the current pose relationship, the electronic device can calculate the preset assembly pose relationship between the taught workpiece and the taught object when correctly assembled.

[0173] In this embodiment, after determining the current teaching pose relationship between the teaching fixture flange and the teaching workpiece, the electronic device can use this current teaching pose relationship, the pose of the teaching object, and the final pose of the teaching fixture flange to calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled. In this way, the obtained preset assembly pose relationship is independent of the fixture, thus avoiding workpiece assembly failures caused by deviations in the pose relationship between the target workpiece and the fixture flange compared to the teaching pose relationship due to unreasonable fixture design, thereby improving workpiece configuration accuracy.

[0174] As one embodiment of this application, the step of calculating the preset assembly posture relationship between the taught workpiece and the taught object when correctly assembled, based on the current teaching posture relationship, the teaching object posture, and the final teaching fixture flange posture, may include:

[0175] The preset assembly pose relationship is calculated using the following formula:

[0176] obj22obj1=(obj teach1 2R teach ) -1 ×T′ teach 2R teach ×(T teach 2R teach ) -1

[0177] ×obj teach2 2R teach

[0178] Among them, obj teach1 2R teach The current pose of the teaching workpiece in the coordinate system of the teaching robot arm; T′ teach 2R teach The current pose of the teaching fixture flange in the coordinate system of the teaching robot arm; T teach 2R teach The final pose of the teaching fixture flange in the coordinate system of the teaching manipulator arm; obj teach2 2R teach The teaching object is the pose of the teaching object in the coordinate system of the teaching robot arm.

[0179] During the teaching process, the pose of the taught object in the teaching robot arm coordinate system can remain unchanged; the relationship between the current poses of the taught workpiece and the taught fixture flange in the teaching robot arm coordinate system, as represented by their current poses, can also remain unchanged. Therefore, based on the aforementioned current pose relationship, the pose of the object to be assembled, and the final pose of the taught fixture flange, the electronic device can calculate the preset assembly pose relationship between the taught workpiece and the taught object for correct assembly.

[0180] In this embodiment, after the electronic device calculates the preset assembly pose relationship between the taught workpiece and the taught object using the above formula, it can use the preset assembly pose relationship to assemble the workpiece more accurately and quickly, thereby improving the workpiece assembly accuracy.

[0181] As one implementation method of this application, such as Figure 5 As shown, step S305 above, which calculates the preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, may include:

[0182] S501: Based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, calculate the pose relationship between the teaching workpiece and the teaching object when correctly assembled, and use it as a candidate assembly pose relationship.

[0183] S502: Calculate the difference between the candidate assembly pose relationship and the preset standard assembly pose relationship, and when the difference is not greater than a preset threshold, use the candidate assembly pose relationship as the preset assembly pose relationship.

[0184] The preset standard assembly posture relationship is pre-constructed and determined based on the posture relationship between the standard workpiece corresponding to the teaching workpiece and the teaching object when they are correctly assembled. The teaching workpiece is one of the workpieces in the same batch, and the standard workpiece is the production template workpiece corresponding to the workpieces in that batch.

[0185] When manufacturing each workpiece, a standard workpiece is typically first created based on the design drawings, serving as a production template. After creating the standard workpiece, it can be used to teach the object, i.e., by executing steps S301-S305 above, to obtain the preset standard assembly pose relationship between the standard workpiece and the object when correctly assembled. The electronic device stores this preset standard assembly pose relationship.

[0186] For each batch of workpieces, one workpiece can be randomly selected from that batch as the teaching workpiece and the teaching object for teaching. During the teaching process, when the electronic device determines the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, it can calculate the pose relationship between the teaching workpiece and the teaching object when correctly assembled, and use it as a candidate assembly pose relationship.

[0187] The electronic device can then calculate the difference between the candidate assembly pose and its stored preset standard assembly pose, and determine the quantitative relationship between this difference and a preset threshold. The preset threshold can be set according to actual needs, such as an angle threshold of 0.4 degrees or a coordinate threshold of 3, etc., without specific limitations here.

[0188] When the difference is not greater than a preset threshold, the candidate assembly pose relationship is taken as the preset assembly pose relationship between the taught workpiece and the taught object when the assembly is correct. When the difference is greater than the preset threshold, a notification message carrying the candidate assembly pose relationship is output so that the staff can perform corresponding verification measures based on the notification message. For example, the teaching process can be checked to determine whether there are any errors in the teaching process that cause the candidate assembly pose relationship to be too large; the batch of workpieces can be checked to determine whether the batch of workpieces is unqualified.

[0189] In this embodiment, for each batch of workpieces, the electronic device verifies the candidate assembly pose relationship between the taught workpiece and the taught object when the correct assembly is calculated during the teaching process using a preset standard assembly pose relationship. This verifies whether the teaching process is correct and / or whether the batch of workpieces is qualified. Therefore, it avoids large errors in the preset assembly pose relationship due to errors in the teaching process or the qualification of the batch of workpieces, thus improving the workpiece assembly accuracy.

[0190] As one implementation method of this application, such as Figure 6 As shown, step S104 above, the step of controlling the robotic arm to assemble the target workpiece onto the object to be assembled based on the current pose and the final pose of the clamping flange, may include:

[0191] S601: Based on the current pose and final pose of the clamping flange, plan the movement path of the robotic arm;

[0192] S602: Control the robotic arm to move along the movement path, and control the clamping flange of the robotic arm to assemble the target workpiece to the object to be assembled in the final position of the clamping flange.

[0193] After determining the current and final positions of the clamping flange, the electronic device can plan the movement path of the robotic arm based on these positions.

[0194] Then, the electronic equipment can control the robotic arm to move according to the planned movement path, and control the clamping flange of the robotic arm to assemble the target workpiece to the object to be assembled in the final position of the clamping flange, thereby completing the assembly of the target workpiece and the object to be assembled.

[0195] After assembling the target workpiece to the object to be assembled, the assembly result can be used as the new object to be assembled, and the workpiece corresponding to the new object to be assembled can be used as the new target workpiece. Then, using the preset assembly pose relationship between the new target workpiece and the new object to be assembled during correct assembly, determined through the teaching process, the above steps S101-S104 are continued to assemble the new target workpiece to the new object to be assembled. By updating the target object and the object to be assembled, the workpiece assembly of the product is completed.

[0196] In this embodiment, the electronic device can determine the movement path of the robotic arm based on the current and final positions of the clamping flange, and then control the robotic arm according to the movement path to accurately assemble the target workpiece to the object to be assembled, thereby improving the assembly accuracy.

[0197] It should be noted that in the technical solution provided in this application, the robotic arm includes a clamping flange, and the target workpiece and the object to be assembled are clamped by the clamping flange to assemble the workpiece. However, in actual production, depending on the type of workpiece and the requirements of the production process, the clamp of the robotic arm can also be other types of clamps, such as gripping clamps, vacuum adsorption clamps, etc. The type of clamp is not limited here. Any workpiece assembly scheme based on the inventive concept of this application and using the workpiece assembly method provided in the embodiments of this application is within the protection scope of this application.

[0198] Corresponding to the above-described workpiece assembly method, this application also provides a workpiece assembly device, which will be described below.

[0199] like Figure 7 As shown, a workpiece assembly device includes:

[0200] The image acquisition module 701 is used to control the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device, and acquire the target image acquired by the image acquisition device, wherein the pre-placed object to be assembled is located in the image acquisition area;

[0201] The first pose determination module 702 is used to determine the current pose of the clamping flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled based on the target image and the pre-calibrated parameters of the image acquisition device.

[0202] The second pose determination module 703 is used to calculate the final pose of the fixture flange based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and a preset assembly pose relationship; wherein, the preset assembly pose relationship is determined during the teaching process based on the pose relationship between the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled when they are correctly assembled.

[0203] Assembly module 704 is used to control the robotic arm to assemble the target workpiece onto the object to be assembled based on the current and final positions of the clamp flange.

[0204] In the solution provided in this application embodiment, the electronic device can control the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device and acquire the target image acquired by the image acquisition device. The object to be assembled is pre-placed within the image acquisition area. Based on the target image and the parameters of the pre-calibrated image acquisition device, the current pose of the gripper flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled are determined. The final pose of the gripper flange is calculated based on the current pose of the gripper flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship. The preset assembly pose relationship is determined during the teaching process based on the pose relationship between the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled when they are correctly assembled. Then, based on the current pose and the final pose of the gripper flange, the robotic arm is controlled to assemble the target workpiece onto the object to be assembled.

[0205] For a target workpiece and its corresponding object to be assembled, the pose relationship between the target workpiece and the object to be assembled should satisfy a preset assembly pose relationship during correct assembly. Since the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled, it does not change with the pose relationship between the target workpiece and the clamping flange of the robotic arm. Therefore, using the preset assembly pose relationship to assemble the target workpiece and the object to be assembled can avoid assembly failures caused by pose misalignment between the target workpiece and the clamping flange due to unreasonable clamping flange design, thus improving assembly accuracy. Furthermore, since the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled during correct assembly and is independent of the current pose of the object to be assembled, it can accommodate arbitrary placement of the object to be assembled without fixing it to a single position. By real-time detection of the pose of the object to be assembled and the pose relationship between the target workpiece and the clamping flange, corrective assembly can be achieved, reducing assembly costs.

[0206] As one embodiment of this application, the second pose determination module 703 includes:

[0207] The first pose determination unit is used to calculate the final pose of the clamp flange when the target workpiece and the object to be assembled are correctly assembled, based on the current pose relationship, the pose of the object to be assembled, and the preset assembly pose relationship. The current pose relationship is the pose relationship between the clamp flange of the robot arm and the target workpiece, characterized by the current pose of the clamp flange and the current pose of the target workpiece.

[0208] As one embodiment of this application, the first pose determination unit includes:

[0209] The first calculation subunit is used to calculate the final pose of the fixture flange according to the following formula:

[0210] T′2R=obj22R×(obj22obj1) -1 ×(obj12R) -1 ×T2R

[0211] Wherein, T′2R is the final pose of the clamping flange in the robot arm coordinate system; obj22R is the pose of the object to be assembled in the robot arm coordinate system; obj22obj1 is the preset assembly pose relationship matrix; obj12R is the current pose of the target workpiece in the robot arm coordinate system; and T2R is the current pose of the clamping flange in the robot arm coordinate system.

[0212] As one embodiment of this application, the device further includes a preset assembly pose relationship construction module, the preset assembly pose relationship construction module comprising:

[0213] The first image acquisition unit is used to control the teaching robot arm holding the teaching workpiece to move to the teaching image acquisition area of ​​the teaching image acquisition device, and acquire the first teaching image acquired by the teaching image acquisition device, wherein the pre-placed teaching object is located in the teaching image acquisition area;

[0214] The second pose determination unit is used to determine the current pose of the teaching jig flange, the current pose of the teaching workpiece, and the current pose of the teaching object of the teaching robot arm based on the first teaching image and the parameters of the pre-calibrated teaching image acquisition device.

[0215] The second image acquisition unit is used to control the teaching robot arm to assemble the teaching workpiece onto the teaching object, and to acquire the second teaching image acquired by the teaching image acquisition device;

[0216] The third pose determination unit is used to determine the final pose of the teaching jig flange of the teaching robot arm when it is correctly assembled, based on the second teaching image and the parameters of the pre-calibrated teaching image acquisition device.

[0217] The fourth pose determination unit is used to calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when the assembly is correct, based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange.

[0218] As one embodiment of this application, the fourth pose determination unit includes:

[0219] The second calculation subunit is used to calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled, based on the current teaching pose relationship, the pose of the teaching object, and the final pose of the teaching fixture flange. The current teaching pose relationship is the teaching pose relationship between the teaching fixture flange and the teaching workpiece characterized by the current pose of the teaching fixture flange and the current pose of the teaching workpiece.

[0220] As one embodiment of this application, the second computing subunit is specifically used for:

[0221] The preset assembly pose relationship is calculated using the following formula:

[0222] obj22obj1=(obj teach1 2R teach ) -1 ×T′teach 2R teach ×(T teach 2R teach ) -1

[0223] ×obj teach2 2R teach

[0224] Among them, obj teach1 2R teach The current pose of the teaching workpiece in the coordinate system of the teaching robot arm; T′ teach 2R teach The current pose of the teaching fixture flange in the coordinate system of the teaching robot arm; T teach 2R teach The final pose of the teaching fixture flange in the coordinate system of the teaching manipulator arm; obj teach2 2R teach The teaching object is the pose of the teaching object in the coordinate system of the teaching robot arm.

[0225] As one embodiment of this application, the fourth pose determination unit includes:

[0226] The third calculation subunit is used to calculate the pose relationship between the teaching workpiece and the teaching object when the assembly is correct, based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, as a candidate assembly pose relationship.

[0227] The difference calculation subunit is used to calculate the difference between the candidate assembly pose relationship and the preset standard assembly pose relationship, and when the difference is not greater than a preset threshold, the candidate assembly pose relationship is used as the preset assembly pose relationship. The preset standard assembly pose relationship is pre-constructed and determined based on the pose relationship between the standard workpiece corresponding to the teaching workpiece and the teaching object when they are correctly assembled. The teaching workpiece is one of the workpieces in the same batch, and the standard workpiece is the production template workpiece corresponding to the workpieces in this batch.

[0228] As one embodiment of this application, the assembly module 704 includes:

[0229] The path planning unit is used to plan the movement path of the robotic arm based on the current pose and the final pose of the clamping flange.

[0230] The control unit is used to control the robotic arm to move according to the movement path, and to control the clamping flange of the robotic arm to assemble the target workpiece to the object to be assembled in the final position of the clamping flange.

[0231] Corresponding to the above-described workpiece assembly method, this application also provides a workpiece assembly system, which will be described below.

[0232] like Figure 8 As shown, a workpiece assembly system includes an image acquisition device 801 and a processor 802.

[0233] The image acquisition device 801 is used to acquire images of various objects located in the image acquisition area;

[0234] The processor 802 is used to execute the steps of any workpiece assembly method provided in the embodiments of this application.

[0235] In the solution provided in this application embodiment, the electronic device can control the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device and acquire the target image acquired by the image acquisition device. The object to be assembled is pre-placed within the image acquisition area. Based on the target image and the parameters of the pre-calibrated image acquisition device, the current pose of the gripper flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled are determined. The final pose of the gripper flange is calculated based on the current pose of the gripper flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship. The preset assembly pose relationship is determined during the teaching process based on the pose relationship between the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled when they are correctly assembled. Then, based on the current pose and the final pose of the gripper flange, the robotic arm is controlled to assemble the target workpiece onto the object to be assembled.

[0236] For a target workpiece and its corresponding object to be assembled, the pose relationship between the target workpiece and the object to be assembled should satisfy a preset assembly pose relationship during correct assembly. Since the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled, it does not change with the pose relationship between the target workpiece and the clamping flange of the robotic arm. Therefore, using the preset assembly pose relationship to assemble the target workpiece and the object to be assembled can avoid assembly failures caused by pose misalignment between the target workpiece and the clamping flange due to unreasonable clamping flange design, thus improving assembly accuracy. Furthermore, since the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled during correct assembly and is independent of the current pose of the object to be assembled, it can accommodate arbitrary placement of the object to be assembled without fixing it to a single position. By real-time detection of the pose of the object to be assembled and the pose relationship between the target workpiece and the clamping flange, corrective assembly can be achieved, reducing assembly costs.

[0237] This application also provides an electronic device, such as... Figure 9 As shown, it includes:

[0238] Memory 901 is used to store computer programs;

[0239] When the processor 902 executes the program stored in the memory 901, it implements the steps of the method described in any of the above embodiments.

[0240] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 902, the communication interface, and the memory 901 communicating with each other via the communication bus.

[0241] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0242] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0243] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0244] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0245] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described workpiece assembly methods.

[0246] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the workpiece assembly methods described above.

[0247] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0248] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0249] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, systems, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0250] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A workpiece assembly method, characterized in that, The method includes: The robotic arm holding the target workpiece moves to the image acquisition area of ​​the image acquisition device and acquires the target image acquired by the image acquisition device, wherein the pre-placed object to be assembled is located within the image acquisition area; Based on the target image and the pre-calibrated parameters of the image acquisition device, the current pose of the clamping flange, the current pose of the target workpiece, and the pose of the object to be assembled are determined, wherein the current pose of the clamping flange, the current pose of the target workpiece, and the pose of the object to be assembled are three-dimensional poses. The final pose of the fixture flange is calculated based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship. The preset assembly pose relationship is the pose relationship that should be satisfied when the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled are correctly assembled during the teaching process. The preset assembly pose relationship will not change with the pose of the target workpiece and the fixture flange. Based on the current and final positions of the clamp flange, the robotic arm is controlled to assemble the target workpiece onto the object to be assembled.

2. The method according to claim 1, characterized in that, The step of calculating the final pose of the fixture flange based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship includes: Based on the current pose relationship, the pose of the object to be assembled, and the preset assembly pose relationship, the final pose of the clamping flange when the target workpiece and the object to be assembled are correctly assembled is calculated. The current pose relationship is the pose relationship between the clamping flange of the robot arm and the target workpiece, characterized by the current pose of the clamping flange and the current pose of the target workpiece.

3. The method according to claim 2, characterized in that, The step of calculating the final position of the fixture flange when the target workpiece and the object to be assembled are correctly assembled based on the current position relationship, the position of the object to be assembled, and the preset assembly position relationship includes: Calculate the final position of the clamp flange using the following formula: ； in, The final position of the clamp flange in the robot arm coordinate system; The pose of the object to be assembled in the coordinate system of the robotic arm; Preset assembly pose relationship matrix; The current pose of the target workpiece in the robot arm coordinate system. The current position of the clamp flange in the coordinate system of the robotic arm.

4. The method according to claim 1, characterized in that, The method for constructing the preset assembly pose relationship includes: The control arm holding the teaching workpiece moves to the teaching image acquisition area of ​​the teaching image acquisition device and acquires the first teaching image acquired by the teaching image acquisition device, wherein the pre-placed teaching object is located within the teaching image acquisition area; Based on the first teaching image and the pre-calibrated parameters of the teaching image acquisition device, the current pose of the teaching jig flange, the current pose of the teaching workpiece, and the current pose of the teaching object of the teaching robot arm are determined. The robot arm is controlled to assemble the teaching workpiece onto the teaching object, and a second teaching image is acquired by the teaching image acquisition device. Based on the second teaching image and the pre-calibrated parameters of the teaching image acquisition device, determine the final position of the teaching jig flange of the teaching robot arm when it is correctly assembled; Based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled.

5. The method according to claim 4, characterized in that, The step of calculating the preset assembly pose relationship between the taught workpiece and the taught object when correctly assembled, based on the current pose of the taught fixture flange, the current pose of the taught workpiece, the pose of the taught object, and the final pose of the taught fixture flange, includes: Based on the current teaching pose relationship, the pose of the teaching object, and the final pose of the teaching fixture flange, the preset assembly pose relationship between the teaching workpiece and the teaching object is calculated when the assembly is correct. The current teaching pose relationship is the teaching pose relationship between the teaching fixture flange and the teaching workpiece characterized by the current pose of the teaching fixture flange and the current pose of the teaching workpiece.

6. The method according to claim 5, characterized in that, The step of calculating the preset assembly pose relationship between the taught workpiece and the taught object when correctly assembled, based on the current pose relationship, the pose of the taught object, and the final pose of the taught fixture flange, includes: The preset assembly pose relationship is calculated using the following formula: ； in, The current pose of the teaching workpiece in the coordinate system of the teaching robot arm; The current position of the teaching fixture flange in the coordinate system of the teaching robot arm; The final pose of the teaching fixture flange in the coordinate system of the teaching manipulator arm; The teaching object is the pose of the teaching object in the coordinate system of the teaching robot arm.

7. The method according to any one of claims 4-6, characterized in that, The step of calculating the preset assembly pose relationship between the taught workpiece and the taught object when correctly assembled, based on the current pose of the taught fixture flange, the current pose of the taught workpiece, the pose of the taught object, and the final pose of the taught fixture flange, includes: Based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, the pose relationship between the teaching workpiece and the teaching object when correctly assembled is calculated as a candidate assembly pose relationship. Calculate the difference between the candidate assembly pose relationship and the preset standard assembly pose relationship, and when the difference is not greater than a preset threshold, use the candidate assembly pose relationship as the preset assembly pose relationship. The preset standard assembly pose relationship is pre-constructed and determined based on the pose relationship between the standard workpiece corresponding to the teaching workpiece and the teaching object when they are correctly assembled. The teaching workpiece is one of the workpieces in the same batch, and the standard workpiece is the production template workpiece corresponding to the workpieces in that batch.

8. The method according to any one of claims 1-6, characterized in that, The step of controlling the robotic arm to assemble the target workpiece onto the object to be assembled based on the current and final positions of the clamping flange includes: Based on the current pose and the final pose of the clamp flange, the movement path of the robotic arm is planned. The robotic arm is controlled to move along the movement path, and the clamping flange of the robotic arm is controlled to assemble the target workpiece onto the object to be assembled in the final position of the clamping flange.

9. A workpiece assembly device, characterized in that, The device includes: The image acquisition module is used to control the robotic arm holding the target workpiece to move to the image acquisition area of ​​the image acquisition device, and acquire the target image acquired by the image acquisition device, wherein the pre-placed object to be assembled is located within the image acquisition area; The first pose determination module is used to determine the current pose of the clamping flange of the robotic arm, the current pose of the target workpiece, and the pose of the object to be assembled based on the target image and the pre-calibrated parameters of the image acquisition device, wherein the current pose of the clamping flange, the current pose of the target workpiece, and the pose of the object to be assembled are three-dimensional poses. The second pose determination module is used to calculate the final pose of the fixture flange based on the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and a preset assembly pose relationship. The preset assembly pose relationship is the pose relationship that should be satisfied when the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled are correctly assembled during the teaching process. The preset assembly pose relationship will not change with the pose changes of the target workpiece and the fixture flange. An assembly module is used to control the robotic arm to assemble the target workpiece onto the object to be assembled, based on the current and final positions of the clamp flange.

10. The apparatus according to claim 9, characterized in that, The second pose determination module includes: The first pose determination unit is used to calculate the final pose of the clamping flange when the target workpiece and the object to be assembled are correctly assembled, based on the current pose relationship, the pose of the object to be assembled, and the preset assembly pose relationship. The current pose relationship is the pose relationship between the clamping flange of the robot arm and the target workpiece represented by the current pose of the clamping flange and the current pose of the target workpiece. The first pose determination unit includes: The first calculation subunit is used to calculate the final pose of the fixture flange according to the following formula: ； in, The final position of the clamp flange in the robot arm coordinate system; The pose of the object to be assembled in the coordinate system of the robotic arm; Preset assembly pose relationship matrix; The current pose of the target workpiece in the robot arm coordinate system. The current position of the clamping flange in the robot arm coordinate system; The device further includes a preset assembly pose relationship construction module, which includes: The first image acquisition unit is used to control the teaching robot arm holding the teaching workpiece to move to the teaching image acquisition area of ​​the teaching image acquisition device, and acquire the first teaching image acquired by the teaching image acquisition device, wherein the pre-placed teaching object is located in the teaching image acquisition area; The second pose determination unit is used to determine the current pose of the teaching jig flange, the current pose of the teaching workpiece, and the current pose of the teaching object of the teaching robot arm based on the first teaching image and the parameters of the pre-calibrated teaching image acquisition device. The second image acquisition unit is used to control the teaching robot arm to assemble the teaching workpiece onto the teaching object, and to acquire the second teaching image acquired by the teaching image acquisition device; The third pose determination unit is used to determine the final pose of the teaching jig flange of the teaching robot arm when it is correctly assembled, based on the second teaching image and the parameters of the pre-calibrated teaching image acquisition device. The fourth pose determination unit is used to calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when the assembly is correct, based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange. The fourth pose determination unit includes: The second calculation subunit is used to calculate the preset assembly posture relationship between the teaching workpiece and the teaching object when the assembly is correct, based on the current teaching posture relationship, the teaching object posture, and the final posture of the teaching fixture flange. The current teaching posture relationship is the teaching posture relationship between the teaching fixture flange and the teaching workpiece represented by the current posture of the teaching fixture flange and the current posture of the teaching workpiece. The second calculation subunit is specifically used for: The preset assembly pose relationship is calculated using the following formula: ； in, The current pose of the teaching workpiece in the coordinate system of the teaching robot arm; The current position of the teaching fixture flange in the coordinate system of the teaching robot arm; The final pose of the teaching fixture flange in the coordinate system of the teaching manipulator arm; The teaching object is the teaching object pose in the coordinate system of the teaching robot arm; The fourth pose determination unit includes: The third calculation subunit is used to calculate the pose relationship between the teaching workpiece and the teaching object when the assembly is correct, based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, as a candidate assembly pose relationship. The difference calculation subunit is used to calculate the difference between the candidate assembly pose relationship and the preset standard assembly pose relationship, and when the difference is not greater than a preset threshold, the candidate assembly pose relationship is used as the preset assembly pose relationship. The preset standard assembly pose relationship is pre-constructed and determined based on the pose relationship between the standard workpiece corresponding to the teaching workpiece and the teaching object when they are correctly assembled. The teaching workpiece is one of the workpieces in the same batch, and the standard workpiece is the production template workpiece corresponding to the workpieces in this batch. The assembly module includes: The path planning unit is used to plan the movement path of the robotic arm based on the current pose and the final pose of the clamping flange. The control unit is used to control the robotic arm to move according to the movement path, and to control the clamping flange of the robotic arm to assemble the target workpiece to the object to be assembled in the final position of the clamping flange.

11. A workpiece assembly system, characterized in that, The system includes an image acquisition device and a processor: The image acquisition device is used to acquire images of various objects located in the image acquisition area; The processor is configured to execute the method according to any one of claims 1-8.

12. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-8.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images