Assembly system calibration method, assembly method, assembly system, and electronic equipment

Abstract

The present application provides a calibration method, an assembly method, an assembly system, and an electronic device for an assembly system, which relate to the field of computer vision technology and are used to solve the problem of poor assembly accuracy of the assembly system when assembling 3D small components. The calibration method is applied to the processing unit of the assembly system, and the calibration method includes: taking the surface of the first orientation of the first calibration body as the reference surface, obtaining the spatial posture information when the reference surface is in the first reference plane, as the first reference information; the first reference information is used to indicate the first reference plane; obtaining the spatial posture information of the first surface of the second calibration body as the first initial information, and obtaining the spatial posture information of the second surface of the second calibration body as the second initial information; the first surface is a surface of the first orientation; according to the deviation between the first initial information and the first reference information, calibrating the second initial information to obtain the second reference information, and the second reference information is used to indicate the second reference plane in the assembly space.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a calibration method for an assembly system, an assembly method, an assembly system, and electronic equipment. Background Art

[0002] In the application scenario of 3D assembly, the calibration of the assembly system is a very critical link, and the accuracy of the calibration results directly affects the accuracy of the assembly.

[0003] However, 3D assembly of small components typically involves no calibration. Instead, the assembly process occurs directly within the assembly system. For example, each axis of the module is driven by a separate motor to control the movement of the component being assembled. However, this direct assembly approach lacks precise calibration, making it difficult to accurately determine and correct the position and orientation of small components in global coordinates. This results in poor assembly accuracy during assembly. Summary of the Invention

[0004] Based on the above technical problems, the present application provides a calibration method, assembly method, assembly system and electronic equipment for an assembly system, which are used to solve the technical problem of poor assembly accuracy of the assembly system when assembling 3D stereo small components.

[0005] In a first aspect, the present application provides a calibration method for an assembly system, which is applied to a processing unit of the assembly system. The assembly system also includes a camera group, which includes at least one camera for collecting spatial pose information of a component to be assembled placed in an assembly space. The spatial pose information is used by the processing unit to locate the spatial pose of the component to be assembled. The camera is also used to scan the spatial pose information of each surface of a calibration body placed in the assembly space. The method includes: taking a surface of a first orientation of a first calibration body as a reference surface, obtaining spatial pose information when the reference surface is in a first reference plane, as first reference information; the first reference information is used to indicate the first reference plane; obtaining spatial pose information of the first surface of a second calibration body, as first initial information, and obtaining spatial pose information of the second surface of the second calibration body, as second initial information; the first surface is a surface of a first orientation; calibrating the second initial information according to the deviation between the first initial information and the first reference information to obtain second reference information, and the second reference information is used to indicate the second reference plane in the assembly space.

[0006] In the embodiment of the present application, it is mainly used in the calibration scene, and further, it is used in the calibration scene of the 3D assembly system of small components. With one surface of the calibration body as the reference plane, the multiple surfaces of the calibration body are calibrated, and the calibrated surface is used as the reference plane to obtain the spatial posture information corresponding to the multiple reference planes, and the spatial posture information corresponding to the multiple reference planes is used to locate the spatial posture of the components to be assembled in the assembly scene. The second reference plane and the first reference plane are in a certain spatial position relationship (such as a parallel relationship), and the first reference plane and the second reference plane can locate the spatial posture of the multiple surfaces of the components to be assembled in the assembly scene, that is, the spatial posture information of the multiple reference planes is used to indicate the spatial posture information of the components to be assembled in the assembly scene. In this way, in the 3D assembly scene of small components, the defect that direct assembly is difficult to perform accurate global coordinate positioning and correction is overcome, the assembly error caused by the lack of calibration is effectively reduced, and the assembly accuracy of the assembly system is improved.

[0007] In one possible implementation, the camera assembly includes a first camera and a second camera, both of which are fixed in position. The reference surface and the first surface are within the acquisition range of the first camera, while the second surface is within the acquisition range of the second camera. The first baseline information and the first initial information are acquired by the processing unit via the first camera, while the second initial information is acquired by the processing unit via the second camera. This allows for accurate determination of the first baseline information, the first initial information, and the second initial information, thereby increasing the accuracy of the second baseline information and, consequently, the precision of the calibration results.

[0008] In one possible implementation, the assembly system further includes a drive mechanism for driving the movement of the assembly to be assembled or the calibration object. Using the surface of the first calibration object in a first orientation as a reference plane, spatial position information of the reference surface when in a first reference position is obtained as the first reference information. This includes: driving the first calibration object to move via the drive mechanism to adjust the spatial position of the first calibration object until the error between the spatial position information of the reference surface and the theoretical spatial position information corresponding to the first reference plane falls within an allowable error range, and then using the spatial position information of the reference surface as the first reference information. In this way, by adjusting the reference surface to the first reference plane (or to fall within the allowable error range), a unified spatial reference is provided for the entire assembly system, thereby making the reference information more accurate.

[0009] In a possible implementation, the first reference plane is parallel to the second reference plane.

[0010] In one possible implementation, the method further includes: when the reference surface is in the first reference plane, obtaining spatial position information of the third surface of the first calibration body as third reference information; the third surface is a surface located in a third position, and the third reference information is used to indicate the third reference plane in the assembly space. In this way, with the first reference plane as a reference, the third surface of the first calibration body is in the third reference plane, and the spatial position information of the third surface of the first calibration body is used as the third reference information. Based on this, the first reference plane or the second reference plane has a certain spatial position relationship with the third reference plane. In this way, the spatial position relationship between the first reference plane, the second reference plane, and the third reference plane can be directly applied in the assembly scene to further improve the assembly accuracy and efficiency.

[0011] In one possible implementation, the method further includes: when the reference surface is in the first reference plane, obtaining spatial position information of the fourth surface of the first calibration body as fourth reference information; the fourth surface is a surface located in a fourth orientation, and the fourth reference information is used to indicate the fourth reference plane in the assembly space. In this way, with the first reference plane as a reference, the fourth surface of the first calibration body is in the fourth reference plane, and the spatial position information of the fourth surface of the first calibration body is used as the fourth reference information. Based on this, the first reference plane or the second reference plane has a certain spatial position relationship with the fourth reference plane. In this way, the spatial position relationship between the first reference plane, the second reference plane, and the fourth reference plane can be directly applied in the assembly scene to further improve the assembly accuracy and efficiency.

[0012] In one possible implementation, the third reference plane intersects with the first or second reference plane and is parallel to the fourth reference plane. This allows the first, third, and fourth reference planes to directly locate the spatial positions of multiple surfaces in an assembly scenario, providing a more comprehensive and accurate assembly experience.

[0013] In one possible implementation, the third surface is arranged opposite and parallel to the fourth surface, and adjacent to and perpendicular to the reference surface; the third reference surface is perpendicular to the first reference surface or the second reference surface. In this way, the first, second, third, and fourth reference surfaces can be combined to locate multiple surfaces that need to be parallel and / or perpendicular in an assembly scenario.

[0014] In one possible implementation, the camera assembly includes a third camera and a fourth camera, both of which are fixed in position. The third surface is within the acquisition range of the third camera, and the fourth surface is within the acquisition range of the fourth camera. The third and fourth reference information are acquired by the processing unit using the third and fourth cameras, respectively. This allows for accurate determination of the third and fourth reference information, resulting in more precise calibration results.

[0015] In one possible implementation, the camera includes a 3D laser profile scanner, which is used to scan 3D image data of the surface of a calibration object or a component to be assembled. The 3D image data includes spatial position information of multiple points on the surface. Thus, by scanning the surface of the calibration object with a camera equipped with a 3D laser profile scanner, the laser emitted by the camera illuminates the surface of the calibration object, thereby obtaining 3D image data of the calibration object's surface. Based on this, the spatial pose information of the calibration object's surface in the assembly space can be determined based on the 3D image data.

[0016] In a second aspect, the present application provides an assembly method for use in a processing unit of an assembly system. The assembly system further includes a camera assembly and a drive mechanism. The camera assembly includes at least one camera for collecting spatial pose information of a component to be assembled placed in an assembly space. The spatial pose information is used by the processing unit to locate the spatial pose of the component to be assembled. The component to be assembled includes an assembly target and an assembly object. The camera is further configured to scan the spatial pose information of each surface of a calibration object placed in the assembly space. The drive mechanism is configured to drive the component to be assembled. The method includes: controlling the drive mechanism to drive a first preset surface of the assembly target to a first reference plane based on first reference information, wherein the first reference information is spatial pose information when a reference plane is at the first reference plane, wherein the reference plane is a surface of a first calibration object at a first orientation, wherein the first reference information is used to indicate the first reference plane, and wherein the first surface is a surface at the first orientation. Controlling the drive mechanism to drive a second preset surface of the assembly target to a second reference plane based on second reference information, wherein the second reference information is spatial pose information when a second surface of a second calibration object is at the second reference plane, wherein the second reference information is used to indicate the second reference plane. The first reference plane and the second reference plane are in a preset spatial positional relationship in the assembly space.

[0017] In the embodiments of this application, the system is primarily used in assembly scenarios, and more specifically, in assembly scenarios for 3D assembly systems of small components. By positioning a first predetermined surface of an assembly target and a second predetermined surface of an assembly object on a first reference plane and a second reference plane, respectively, the first predetermined surface of the assembly target and the second predetermined surface of the assembly object are positioned in a predetermined spatial relationship that satisfies assembly requirements. This improves the assembly accuracy of the assembly system, while also increasing assembly efficiency and meeting user assembly requirements.

[0018] In a third aspect, the present application provides an assembly system comprising: a processing unit, a camera assembly, and a drive mechanism. The camera assembly includes at least one camera for collecting spatial pose information of a component to be assembled placed in an assembly space. This spatial pose information is used by the processing unit to locate the spatial pose of the component to be assembled, which includes an assembly target and an assembly object. The camera is also used to scan the spatial pose information of each surface of a calibration object placed in the assembly space. The drive mechanism is used to drive the component to be assembled. The processing unit is used to control the drive mechanism to move a first preset surface of the assembly target to a first reference plane based on first reference information. The first reference information is the spatial pose information when a reference surface is at the first reference plane. The reference surface is a surface at a first orientation of a first calibration object. The first reference information indicates the first reference plane. The first surface is a surface at the first orientation. The processing unit is also used to control the drive mechanism to move a second preset surface of the assembly target to a second reference plane based on second reference information. The second reference information is the spatial pose information when a second surface of a second calibration object is at the second reference plane. The second reference information indicates the second reference plane. The first reference plane and the second reference plane are in a preset spatial positional relationship in the assembly space.

[0019] In a fourth aspect, the present application provides an electronic device comprising: a processor and a memory. The memory stores instructions executable by the processor. When the processor is configured to execute the instructions, the electronic device implements the above-described method. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic structural diagram of an assembly system provided in an embodiment of the present application;

[0021] Figure 2 A schematic structural diagram of another assembly system provided in an embodiment of the present application;

[0022] Figure 3 A schematic structural diagram of another assembly system provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of a flow chart of a calibration method for an assembly system provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of a surface provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of a camera group scanning a calibration object provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the surface of a calibration object scanned by a first camera according to an embodiment of the present application;

[0027] Figure 8A schematic diagram of the surface of a calibration object scanned by a second camera provided in an embodiment of the present application;

[0028] Figure 9 A schematic diagram of the surface of a calibration object scanned by a third camera provided in an embodiment of the present application;

[0029] Figure 10 A schematic diagram of the surface of a calibration object scanned by a fourth camera provided in an embodiment of the present application;

[0030] Figure 11 A schematic diagram of a flow chart of an assembly method provided in an embodiment of the present application;

[0031] Figure 12 A schematic structural diagram of a calibration device for an assembly system provided in an embodiment of the present application;

[0032] Figure 13 A schematic structural diagram of an assembly device provided in an embodiment of the present application;

[0033] Figure 14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0035] In the description of this application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in this application is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.

[0036] Furthermore, in the description of this application, unless otherwise specified, "plurality" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0037] In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit differences. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0038] The following is an illustrative introduction to the application scenarios of the embodiments of the present application.

[0039] In 3D assembly application scenarios, it is usually necessary to calibrate the assembly system first, and the accuracy of the calibration results directly affects the accuracy of the assembly.

[0040] However, 3D assembly of small components typically involves no calibration process. Instead, the assembly process occurs directly within the assembly system using the modules. For example, each axis of the module is driven independently by a separate motor to control the movement of the component being assembled. However, this direct assembly approach lacks precise calibration, making it difficult to accurately determine and correct the position and orientation of small components in global coordinates. This results in poor assembly accuracy during assembly.

[0041] In light of this, an embodiment of the present application provides a calibration method for an assembly system. Using one surface of a calibration object as a reference plane, multiple surfaces of the calibration object are calibrated. The calibrated surface is then used as the reference plane to obtain spatial pose information corresponding to the multiple reference planes. This spatial pose information is used to locate the spatial pose of components to be assembled in an assembly scenario. In this way, by calibrating the assembly system, the assembly accuracy and efficiency of the assembly system are improved.

[0042] In some embodiments, the method provided by the embodiments of the present application is applied to a processing unit of an assembly system, wherein the assembly system further comprises a camera group, the camera group comprising at least one camera for collecting spatial pose information of components to be assembled placed in the assembly space, the spatial pose information being used by the processing unit to locate the spatial pose of the components to be assembled; the camera is also used to scan the spatial pose information of each surface of a calibration body placed in the assembly space. In this method, first, the surface of a first calibration body in a first orientation is used as a reference surface, and the spatial pose information of the reference surface when it is in a first reference plane is obtained as the first reference information; the first reference information is used to indicate the first reference plane. Secondly, the spatial pose information of the first surface of the second calibration body is obtained as the first initial information, and the spatial pose information of the second surface of the second calibration body is obtained as the second initial information; the first surface is the surface of the second calibration body in the first orientation. Finally, the second initial information is calibrated based on the deviation between the first initial information and the first reference information to obtain the second reference information, and the second calibration information is used to indicate the second reference plane in the assembly space.

[0043] In this way, multiple surfaces of the calibration object are calibrated using the first reference plane as a reference. Specifically, the first surface of the second calibration object is associated with the first reference plane using the deviation between the first initial information and the first reference plane. This deviation indicates the positional difference between the second initial information and the second reference plane, thereby determining the second reference plane. In this way, the second reference plane and the first reference plane are in a specific spatial relationship (e.g., parallel), and the first and second reference planes can be used to locate the spatial position of multiple surfaces of the component to be assembled in the assembly scenario. In other words, the spatial position information of the multiple reference planes is used to indicate the spatial position information of the component to be assembled in the assembly scenario. For example, in an assembly scenario, if two surfaces have a specific spatial relationship (e.g., parallel), one surface can be positioned to the first reference plane, and the other surface can be positioned to the second reference plane. This overcomes the limitation of direct assembly in the 3D assembly of small components, effectively reducing assembly errors caused by lack of calibration and improving the assembly accuracy of the assembly system.

[0044] The following is an exemplary introduction to the system architecture of the embodiment of the present application.

[0045] Combine Figure 1 and Figure 2 As shown, an embodiment of the present application provides an assembly system. The assembly system includes a processing unit 10 and a camera group, wherein:

[0046] The camera assembly includes at least one camera for collecting spatial pose information of components to be assembled placed in the assembly space and for scanning spatial pose information of each surface of a calibration object placed in the assembly space. The spatial pose information is used by the processing unit 10 to determine the spatial pose of the components to be assembled.

[0047] The processing unit 10 is configured to use the surface of the first calibration body 201 in the first orientation as a reference surface and obtain spatial pose information when the reference surface is in the first reference plane as first reference information. The processing unit 10 is further configured to obtain spatial pose information of the first surface of the second calibration body 202 as first initial information, and to obtain spatial pose information of the second surface of the second calibration body 202 as second initial information; and further configured to calibrate the second initial information based on the deviation between the first initial information and the first reference information to obtain second reference information. The first reference information is used to indicate the first reference plane. The first surface is the surface of the second calibration body 202 in the first orientation. The second reference information is used to indicate the second reference plane in the assembly space.

[0048] The assembly system is mainly used in calibration scenarios, and further, in calibration scenarios of 3D assembly systems for small components. Calibration can be understood as determining multiple reference planes in the assembly space before the assembly system assembles the components to be assembled, and the multiple reference planes are in a certain spatial position relationship. For example, the multiple reference planes are in a parallel relationship and / or a perpendicular relationship. The reference information corresponding to the multiple reference planes can be used to locate the spatial postures of the multiple surfaces of the components to be assembled in the assembly scenario. In the assembly scenario, multiple reference planes can be used as references, and the multiple surfaces of the components to be assembled can be positioned with the corresponding reference information, so that the multiple surfaces of the components to be assembled are in the required spatial position relationship. For example, in the assembly scenario, there are two surfaces with a certain spatial position relationship (such as a parallel relationship) in the assembly requirement. Then, one of the surfaces can be positioned to the first reference plane, and the other surface can be positioned to the second reference plane to achieve the assembly requirement. In this way, the assembly accuracy of the assembly system can be improved, and the assembly efficiency can be improved at the same time.

[0049] The calibration body is an object with known structural information and high processing precision. The calibration body can be a rectangular parallelepiped, etc., and there is no restriction on the shape of the calibration body. The embodiment of the present application takes the calibration body as a rectangular parallelepiped as an example. In the calibration scenario of the assembly system, the calibration body is placed in the assembly space and can be scanned by a camera with a 3D laser contour scanner. The laser emitted by the camera is irradiated onto the surface of the calibration body, thereby obtaining 3D image data of the surface of the calibration body. Based on this, the processing unit 10 can determine the spatial posture information of the surface of the calibration body in the assembly space according to the 3D image data.

[0050] To make it easier to distinguish, Figure 2As shown, the calibration body is divided into a first calibration body 201 and a second calibration body 202 .

[0051] Exemplarily, the first calibration body 201 and the second calibration body 202 can be the same calibration body, that is, the same calibration body is placed at different positions in the assembly space and serves as the first calibration body 201 and the second calibration body 202, respectively. In addition, the first calibration body 201 and the second calibration body 202 can also be different calibration bodies, that is, two calibration bodies, and the two calibration bodies are placed at different positions in the assembly space. However, during the calibration process, no matter what position the first calibration body 201 is initially in, the upper surface of the first calibration body 201 is first placed on the first reference plane, and then the second surface of the second calibration body 202 is calibrated based on the first reference plane.

[0052] like Figure 1 and Figure 2 As shown, the orientation can be one or more of up, down, left, and right. For example, the first orientation is up, the second orientation is down, the third orientation is left, and the fourth orientation is right.

[0053] For example, in the embodiments of the present application, the first surface is the top surface, the second surface is the bottom surface, the third surface is the left surface, and the fourth surface is the right surface. For example, the first surface (i.e., the reference surface) of the first calibration object 201 can refer to the top surface of the first calibration object 201, the third surface (the surface in the third orientation) of the first calibration object 201 can refer to the left surface of the first calibration object 201, and the fourth surface (the surface in the fourth orientation) of the first calibration object 201 can refer to the right surface of the first calibration object 201. The first surface of the second calibration object 202 can refer to the top surface of the second calibration object 202, and the second surface of the second calibration object 202 can refer to the bottom surface of the second calibration object 202.

[0054] It is understandable that when the first calibration body 201 is placed in the assembly space, its upper surface can be within the acquisition range of the camera group. However, at this time, the lower surface of the first calibration body 201 cannot be scanned by the camera group. Therefore, the second calibration body 202 is introduced and placed in an arbitrary position in the assembly space where its upper and lower surfaces can be captured by the camera group. In this way, the spatial position information of the lower surface of the second calibration body 202 is determined by correlating the upper surface of the second calibration body 202 with the upper surface of the first calibration body 201.

[0055] Exemplarily, the first surface and the second surface are arranged opposite to and in parallel.

[0056] Exemplarily, the third surface is opposite to and parallel to the fourth surface, and is adjacent to and perpendicular to the reference surface.

[0057] In some embodiments, the camera includes a 3D laser profile scanner for scanning 3D image data of the surface of a calibration object or a component to be assembled, wherein the 3D image data includes spatial position information of multiple points on the surface. Specifically, the spatial position coordinates of multiple points can be determined in the 3D image data, and the spatial pose information of the surface can be calculated based on the spatial position coordinates of the multiple points. For example, when the reference surface is in the first reference plane, the spatial pose information of the reference surface calculated based on the spatial position coordinates of the multiple points in the 3D image data of the reference surface is the first reference information.

[0058] In some embodiments, the camera group includes a first camera 301 and a second camera 302 that are fixed in position.

[0059] For example, Figure 1 and Figure 2 As shown, the first camera 301 may be a camera located at an upper position, and the second camera 302 may be a camera located at a lower position.

[0060] Exemplarily, the first calibration body 201 and the second calibration body 202 are located in the assembly space, with the upper surface of the first calibration body 201 within the acquisition range of the first camera 301, and the upper surface of the second calibration body 202 within the acquisition range of the first camera 301. The lower surface of the second calibration body 202 is within the acquisition range of the second camera 302. In other words, the first calibration body 201 only needs to ensure that its upper surface is within the acquisition range of the first camera 301. The second calibration body 202 needs to ensure that its upper surface is within the acquisition range of the first camera 301 and its lower surface is within the acquisition range of the second camera 302.

[0061] In one implementation, the first camera 301 is configured to capture 3D image data of the upper surface of the first calibration object 201 when the upper surface is located on a first reference plane. The processing unit 10 is further configured to determine, based on the 3D image data of the upper surface, the spatial pose information of the upper surface of the first calibration object 201 when the upper surface is located on the first reference plane, i.e., the first reference information. In other words, the processing unit 10 is further configured to obtain the first reference information via the first camera 301.

[0062] In another implementation, the first camera 301 is also used to collect 3D image data of the upper surface of the second calibration body 202. If the second calibration body 202 is in the initial position, the 3D image data collected by the first camera 301 is used to determine the first initial information. The initial position refers to the second calibration body 202 being placed at any position where it can be scanned by the first camera 301 and the second camera 302 on its upper and lower surfaces. The processing unit 10 is also used to obtain the 3D image data of the upper surface of the second calibration body 202 when it is in the initial position, and determine the spatial posture information of the upper surface based on the 3D image data of the upper surface, that is, the first initial information. That is, the processing unit 10 is also used to obtain the first initial information through the first camera 301.

[0063] In one implementation, the second camera 302 is used to capture 3D image data of the lower surface of the second calibration body 202. The processing unit 10 is further configured to obtain 3D image data of the lower surface of the second calibration body 202 when the second calibration body 202 is in an initial position, and determine spatial pose information of the lower surface, i.e., second initial information, based on the 3D image data of the lower surface. In other words, the processing unit 10 is further configured to process the second initial information captured by the second camera 302.

[0064] In some embodiments, the processing unit 10 is further configured to obtain spatial pose information of the left surface of the first calibration object 201 as third reference information when the reference surface is located at the first reference plane. The third reference information is used to indicate a third reference plane in the assembly space.

[0065] In one implementation, the camera group includes a third camera 303 with a fixed position, and the left surface of the first calibration object 201 is located within the acquisition range of the third camera 303 .

[0066] In one implementation, the third camera 303 is configured to capture 3D image data of the left surface of the first calibration object 201 when the reference surface is located on the first reference plane. The processing unit 10 is further configured to obtain the 3D image data of the left surface of the first calibration object 201 and, based on the 3D image data of the left surface of the first calibration object 201, determine the spatial pose information of the left surface of the first calibration object 201, i.e., the third reference information. In other words, the processing unit 10 is further configured to obtain the third reference information through the third camera 303.

[0067] In some embodiments, the processing unit 10 is further configured to, when the reference surface is located at the first reference plane, obtain spatial pose information of the fourth surface of the first calibration object 201 as fourth reference information. The fourth reference information is used to indicate the fourth reference plane in the assembly space.

[0068] In one implementation, the camera group includes a fourth camera 304 that is fixed in position. For example, the fourth camera 304 may be a camera located in the right direction. The right surface of the first calibration object 201 is located within the acquisition range of the fourth camera 304.

[0069] In one implementation, the fourth camera 304 is configured to capture 3D image data of the right surface of the first calibration object 201 when the reference surface is located on the first reference plane. The processing unit is further configured to obtain the 3D image data of the right surface of the first calibration object 201 and, based on the 3D image data of the right surface of the first calibration object 201, determine spatial pose information of the right surface of the first calibration object 201, i.e., the fourth reference information. In other words, the processing unit 10 is further configured to obtain the fourth reference information through the fourth camera 304.

[0070] In the embodiment of the present application, the camera group includes a first camera 301, a second camera 302, a third camera 303 and a fourth camera 304 with fixed positions. These four different cameras respectively capture the upper, lower, left and right surfaces of the calibration object.

[0071] Exemplarily, the first reference plane is parallel to the second reference plane. In this way, the first reference plane and the second reference plane can be used to locate multiple surfaces that need to be parallel in an assembly scene.

[0072] Exemplarily, the third reference plane intersects with the first reference plane or the second reference plane and is parallel to the fourth reference plane. In this way, the first reference plane, the second reference plane, the third reference plane, and the fourth reference plane can be combined to locate multiple surfaces that need to be parallel and / or intersecting in the assembly scene.

[0073] For example, the third surface is disposed opposite and parallel to the fourth surface, and adjacent to and perpendicular to the reference surface. The third reference surface is perpendicular to the first reference surface or the second reference surface. In this way, the first, second, third, and fourth reference surfaces can be combined to locate multiple surfaces that need to be parallel and / or perpendicular in an assembly scenario.

[0074] In some embodiments, the assembly system further includes a drive mechanism 40 for driving the calibration object to move. Exemplarily, the drive mechanism 40 is used to drive the calibration object to rotate around the X-axis, the Y-axis, and the Z-axis.

[0075] In one implementation, the drive mechanism 40 is used to drive the movement of the first calibration object 201. Specifically, the first calibration object 201 is placed on the drive mechanism 40, and the drive mechanism 40 drives the first calibration object 201 to rotate about the X-axis, Y-axis, and Z-axis. For a surface, the angle of the surface relative to the X-axis, Y-axis, and Z-axis can be represented by RX, RY, and RZ, respectively. In this way, the upper surface of the first calibration object 201 is located on the first reference plane.

[0076] For example, the reference information includes two values, RX and RY. That is, RX and RY can indicate the spatial posture of the reference plane. In this way, indicating the reference plane using the reference information facilitates achieving one or more of the following: the first reference plane and the second reference plane are parallel, the third reference plane and the fourth reference plane are parallel, or the first reference plane and the third reference plane are perpendicular.

[0077] In some embodiments, the assembly system further includes a platform 50 for mounting the camera assembly. For example, the first camera 301, the second camera 302, the third camera 303, and the fourth camera 304 are mounted on the top, bottom, left, and right sides of the platform 50, respectively. The platform 50 is also used to mount the drive mechanism 40. For example, the drive mechanism 40 is disposed on the bottom of the platform 50.

[0078] In the embodiments of the present application, the second reference plane is calibrated with the reference plane being on the first reference plane as a reference, with the goal of achieving a certain spatial positional relationship between the first reference plane and the second reference plane, such as making the first reference plane and the second reference plane parallel. In this way, in an assembly scenario, the first reference information and the second reference information corresponding to the first reference plane and the second reference plane can directly determine the spatial position of the two surfaces that need to be parallel, so that the two surfaces can be accurately and efficiently parallelized.

[0079] In addition, the third and / or fourth reference planes are obtained based on the reference plane being on the first reference plane. The purpose is to achieve a certain spatial position relationship between the first, third, and fourth reference planes. For example, taking the calibration object as a rectangular parallelepiped, the third and fourth reference planes are parallel to each other and perpendicular to the first reference plane. In this way, in the assembly scenario, the third and / or fourth reference planes corresponding to the third and / or fourth reference planes can directly locate the spatial position of multiple surfaces, which is more comprehensive, making assembly more accurate and efficient.

[0080] Combine Figure 3 As shown, the embodiment of the present application provides another assembly system. The assembly system includes: a processing unit 10, a camera group and a driving mechanism; wherein,

[0081] The camera assembly includes at least one camera for collecting spatial pose information of components to be assembled placed in the assembly space and for scanning spatial pose information of each surface of a calibration object placed in the assembly space. The spatial pose information is used by the processing unit 10 to locate the spatial pose of the components to be assembled, which include an assembly target 601 and an assembly object 602.

[0082] The driving mechanism is used to drive the components to be assembled to move.

[0083] The processing unit 10 is configured to control the drive mechanism to drive the first preset surface of the assembly target 601 to be located at the first reference plane based on the first reference information, and further to control the drive mechanism to drive the second preset surface of the assembly target 602 to be located at the second reference plane based on the second reference information. The first reference information is spatial position information when the reference plane is located at the first reference plane, where the reference plane is the surface of the first calibration body in the first orientation, and the first reference information is used to indicate the first reference plane, where the first surface is the surface in the first orientation; the second reference information is spatial position information when the second surface of the second calibration body is located at the second reference plane, and the second reference information is used to indicate the second reference plane. The first reference plane and the second reference plane are in a preset spatial position relationship in the assembly space.

[0084] This assembly system is mainly used in assembly scenarios, and further, in assembly scenarios of 3D assembly systems of small components. By positioning the first preset surface of the assembly target 601 and the second preset surface of the assembly object 602 to the first reference plane and the second reference plane respectively, so that the first preset surface of the assembly target 601 and the second preset surface of the assembly object 602 are in a certain preset spatial position relationship, the preset spatial position relationship can meet the assembly requirements. For example, in the assembly scenario, there are two surfaces with an assembly requirement of a certain spatial position relationship (such as a parallel relationship), then one of the surfaces (such as the first preset surface of the assembly target 601) can be positioned to the first reference plane, and the other surface (such as the second preset surface of the assembly object 602) can be positioned to the second reference plane. In this way, the assembly accuracy of the assembly system can be improved, while the assembly efficiency can be improved, and the assembly needs of the user can be met.

[0085] Preset surfaces refer to two surfaces that need to be in a preset spatial position relationship in the assembly space in the assembly scene. The preset surface can be one or more surfaces of the upper surface, lower surface, left surface, right surface, or any surface specified by the user, etc., which are not limited here. For the sake of convenience, it is divided into a first preset surface and a second preset surface. For example, the first preset surface of the assembly target 601 can be Figure 3 The upper surface 1 in the assembly object 602 may be a second predetermined surface. Figure 3 For another example, the first preset surface of the assembly target 601 may be Figure 3 The upper surface 2 in the assembly object 602 may be a second predetermined surface. Figure 3 The lower surface 2 is determined according to the user's assembly requirements.

[0086] In some embodiments, the processing unit is further configured to obtain a user's assembly requirements. The assembly requirements include a first predetermined surface of the assembly target 601 and a second predetermined surface of the assembly object 602. In other words, the processing unit positions the first predetermined surface and the second predetermined surface to the first reference plane and the second reference plane, respectively, based on the assembly requirements.

[0087] In some embodiments, the camera group includes a first camera 301 and a second camera 302 that are fixed in position. The first camera 301 is used to collect 3D image data of a first preset surface of the assembly target 601. The processing unit 10 is further used to determine whether the first preset surface is located on a first reference plane based on the 3D image data of the first preset surface, that is, whether the spatial position information of the first preset surface is the first reference information. In other words, the processing unit 10 is further used to determine whether the first preset surface is positioned on the first reference plane through the first camera 301. The second camera 302 is used to collect 3D image data of a second preset surface of the assembly target 602. The processing unit 10 is further used to determine whether the second preset surface is located on a second reference plane based on the 3D image data of the second preset surface, that is, whether the spatial position information of the second preset surface is the second reference information. In other words, the processing unit 10 is further used to determine whether the second preset surface is positioned on the second reference plane through the second camera 302.

[0088] In some embodiments, the camera group includes a third camera 303 with a fixed position, and the third camera 303 is used to collect 3D image data of the left surface of the assembly target 601 and / or the assembly object 602. The processing unit 10 is also used to determine whether the left surface of the assembly target 601 and / or the assembly object 602 is on the third reference plane based on the 3D image data of the left surface of the assembly target 601 and / or the assembly object 602, that is, whether the spatial posture information of the left surface of the assembly target 601 and / or the assembly object 602 is the third reference information.

[0089] In some embodiments, the camera group includes a fourth camera 304 with a fixed position, and the fourth camera 304 is used to collect 3D image data of the right surface of the assembly target 601 and / or the assembly object 602. The processing unit 10 is also used to determine whether the right surface of the assembly target 601 and / or the assembly object 602 is on the fourth reference plane based on the 3D image data of the right surface of the assembly target 601 and / or the assembly object 602, that is, whether the spatial posture information of the left surface of the assembly target 601 and / or the assembly object 602 is the fourth reference information.

[0090] In this embodiment of the present application, the user's actual needs determine whether to use one or more of the first camera 301, the second camera 302, the third camera 303, and the fourth camera 304. For example, when the first preset surface and the second preset surface are both left surfaces, only the third camera 303 can be used.

[0091] In some embodiments, combined Figure 3 As shown, the driving mechanism includes a first driving mechanism 401 and a second driving mechanism 402 .

[0092] In an assembly scenario, typically, the assembly target 601 is placed on the first drive mechanism 401, and its spatial position is controlled by the first drive mechanism 401. The assembly object 602 is above the assembly target 601, and its position is controlled by the second drive mechanism 402 so that the surface to be bonded in the assembly object 602 is bonded to the surface to be bonded in the assembly target 601. The surface to be bonded refers to the two surfaces that need to be bonded. Among them, the first drive mechanism 401 drives the assembly target 601 to rotate around the X-axis, Y-axis, and Z-axis. The second drive mechanism 402 drives the assembly object 602 to rotate around the X-axis, Y-axis, and Z-axis, and also drives the assembly object 602 to translate along the X-axis, Y-axis, and Z-axis.

[0093] Exemplarily, the second driving mechanism 402 includes a driving member 4022 and a suction nozzle 4021. The lower surface of the suction nozzle 4021 is in contact with the upper surface of the assembly object 602. Driven by the driving member, the suction nozzle 4021 drives the assembly object 602 to rotate around the X-axis, Y-axis, and Z-axis, and / or translate along the X-axis, Y-axis, and Z-axis. It can be understood that the lower surface of the suction nozzle 4021 is also the upper surface of the assembly object 602. Figure 3 The lower surface 1 and the upper surface 3 in.

[0094] In this embodiment of the present application, the purpose is to assemble an assembly object 602 onto an assembly target 601 in an assembly space, achieving alignment of the surface to be bonded of the assembly object 602 with the surface to be bonded of the assembly target 601. This can be achieved by the processing unit controlling a drive mechanism to drive the assembly object 602 and the assembly target 601 based on reference information, and determining whether alignment of the surface to be bonded of the assembly object 602 with the surface to be bonded of the assembly target 601 is achieved through the camera group.

[0095] In some embodiments, the assembly system further includes the aforementioned platform 50 , the first driving mechanism 401 is disposed at the lower portion of the platform 50 , and the first driving mechanism 401 is disposed at the upper portion of the platform 50 .

[0096] In some embodiments, if it is necessary to position different surfaces of the assembly object 602 and the assembly target 601, for example, to position the upper surface of the assembly target 601 parallel to the lower surface of the assembly object 602, or to position the upper surface of the assembly target 601 perpendicular to the left surface of the assembly object 602, etc., then based on the first reference plane, the second reference plane, the third reference plane, and the fourth reference plane, two or more reference planes can be selected in combination according to the user's actual assembly requirements to achieve the spatial position relationship between two or more surfaces of the assembly object 602 and the assembly target 601. For example, when the upper surface of the assembly target 601 needs to be parallel to the lower surface of the assembly object 602, the upper surface of the assembly target 601 is positioned to the first reference plane, and the lower surface of the assembly object 602 is positioned to the second reference plane. For another example, when the upper surface of the assembly target 601 needs to be perpendicular to the left surface of the assembly object 602, the upper surface of the assembly target 601 is positioned to the first reference plane, and the left surface of the assembly object 602 is positioned to the third reference plane.

[0097] It is worth noting that if it is necessary to locate the upper surface of the assembly object 602, since the upper surface of the assembly object 602 is a blind spot and cannot be scanned by the camera, and the lower surface of the suction nozzle 4021 is in contact with the upper surface of the assembly object 602, the lower surface of the suction nozzle 4021 is regarded as the upper surface of the assembly object 602. In this case, it is equivalent to the lower surface of the suction nozzle 4021 that needs to be positioned for the assembly target 601, that is, a different surface. Therefore, two reference surfaces are required for positioning. Specifically, before the suction nozzle 4021 sucks the assembly object 602, the lower surface of the suction nozzle 4021 is positioned to the second reference surface. Afterwards, the assembly object 602 is sucked.

[0098] In other embodiments, if it is necessary to locate the same surface of the assembly object 602 and the assembly target 601, for example, to locate the left surface of the assembly object 602 and the left surface of the assembly object 602, then based on the first reference plane, the second reference plane, the third reference plane and the fourth reference plane, the spatial position relationship of the same surface of the assembly object 602 and the assembly target 601 can be achieved by selecting one of the reference planes (such as the second reference plane).

[0099] Combine Figure 3 As shown, the assembly scenario of the assembly system is illustrated. In the assembly scenario, the assembly target 601 and the assembly object 602 can be correctly matched. Before assembly, the assembly target 601 and the assembly object 602 need to be planned. For example, if you want to achieve Figure 3 The three-dimensional fit of the assembly target 601 and the assembly object 602 is as follows: the upper surface of the assembly target 601 is parallel to the upper surface of the assembly object 602, and the right surface of the assembly target 601 is parallel to the left surface of the assembly object 602.

[0100] Based on this, first, calibrate the upper surface of the assembly target 601 (i.e. Figure 3 The upper surface 1), the right surface, the lower surface of the nozzle 4021 (that is, the upper surface of the assembly object 602, that is, Figure 3 The lower surface 1 or upper surface 3) and the posture information of the left surface of the assembly object 602 are used to achieve that the upper surface of the assembly target 601 is parallel to the upper surface of the assembly object 602, and the right surface of the assembly target 601 is parallel to the left surface of the assembly object 602.

[0101] Specifically, the upper surface of the assembly target 601 needs to be positioned to the first reference plane, and the right surface of the assembly target 601 needs to be positioned to the fourth reference plane: the assembly target 601 is placed on the first driving mechanism 401, and the processing unit 10 controls the first driving mechanism 401 to drive the assembly target 601 to move, so as to calibrate the spatial position information of the upper surface of the assembly target 601 to the first reference information. During the positioning process, the first camera 301 scans the upper surface of the current assembly target 601 in real time and obtains 3D image data. The processing unit 10 determines whether the spatial position information of the upper surface of the current assembly target 601 is the first reference information based on the 3D image data, that is, whether the upper surface of the current assembly target 601 is on the first reference plane. Similarly, the right surface of assembly target 601 must be positioned relative to the fourth reference plane. During this positioning process, fourth camera 304 scans the right surface of current assembly target 601 in real time and obtains 3D image data. Processing unit 10 determines, based on this 3D image data, whether the spatial pose information of the right surface of current assembly target 601 corresponds to the fourth reference information, that is, whether the right surface of current assembly target 601 is located on the fourth reference plane. When the spatial pose information of the top surface of current assembly target 601 corresponds to the first reference information, and the spatial pose information of the right surface corresponds to the fourth reference information, calibration of assembly target 601 is complete.

[0102] In addition, the lower surface of the suction nozzle 4021 needs to be positioned to the second reference plane, and the left surface of the assembly object 602 needs to be positioned to the third reference plane: before the suction nozzle 4021 sucks the assembly object 602, the lower surface of the suction nozzle 4021 is calibrated, and the processing unit 10 controls the movement of the second drive mechanism 402 to achieve the movement of the suction nozzle 4021 and position the lower surface of the suction nozzle 4021 to the second reference plane. During the positioning process, the second camera 302 scans the current lower surface of the suction nozzle 4021 in real time and obtains a 3D image. The processing unit 10 determines whether the spatial position information of the current lower surface of the suction nozzle 4021 is the second reference information based on the 3D image, that is, whether the current suctioned lower surface is on the second reference plane. Similarly, the left surface of the assembly object 602 needs to be positioned to the third reference plane: the upper surface of the assembly object 602 is attached to the lower surface of the suction nozzle 4021. At this time, the processing unit 10 controls the second drive mechanism 402 to drive the assembly object 602 to move. During the positioning process, the third camera 303 scans the left surface of the current assembly object 602 in real time and obtains a 3D image. The processing unit 10 determines whether the spatial position information of the left surface of the current assembly object 602 is the third reference information based on the 3D image, that is, whether the left surface of the current assembly object 602 is on the third reference plane. When the spatial position information of the lower surface of the suction nozzle 4021 is the second reference information and the spatial position information of the left surface of the assembly object 602 is the third reference information, the calibration of the assembly object 602 is completed.

[0103] In this way, when the spatial posture information of the upper surface of the current assembly target 601 is the first reference information, the spatial posture information of the right surface is the fourth reference information, the spatial posture information of the lower surface of the suction nozzle 4021 is the second reference information, and the spatial posture information of the left surface of the assembly object 602 is the third reference information, the upper surface of the assembly target 601 is parallel to the upper surface of the assembly object 602, and the right surface of the assembly target 601 is parallel to the left surface of the assembly object 602.

[0104] Then, with the upper surface of assembly target 601 parallel to the upper surface of assembly object 602, and the right surface of assembly target 601 parallel to the left surface of assembly object 602, processing unit 10 controls second drive mechanism 402 to drive assembly object 602 to translate in one or more of the X-axis, Y-axis, and Z-axis directions, thereby achieving alignment of upper surface 2 with lower surface 2. For example, during assembly planning, a target point (e.g., target point 1 and target point 2) can be determined on upper surface 2 and lower surface 2, respectively. During translation, if target point 1 and target point 2 coincide, assembly is complete.

[0105] In the embodiments of this application, the first and second reference information corresponding to the first and second reference planes are used to directly locate the spatial position of two surfaces that need to conform to a predetermined spatial relationship (e.g., parallelism), thereby accurately and efficiently positioning the two surfaces. Furthermore, third and fourth reference information corresponding to the third and second reference planes are combined to locate the spatial relationship of more surfaces in the assembly scenario, further improving assembly accuracy and efficiency.

[0106] The device deploying the processing unit 10 may be an electronic device. The electronic device may be a mobile phone, tablet computer, handheld computer, personal computer (PC), cellular phone, personal digital assistant (PDA), wearable device (such as a smart watch, smart bracelet, etc.), smart home device (such as a television, etc.), vehicle computer (such as an onboard computer, etc.), smart screen, laptop computer, desktop computer, or all-in-one computer, etc.

[0107] The electronic device may also be a server. Specifically, the server may be a single server or a server cluster. The server may be a physical server or a virtualized platform. The server may be a single physical server or may be composed of two or more physical servers sharing different responsibilities, with the physical servers collaborating to implement the server's various functions.

[0108] Illustratively, the server may be a blade server, a high-density server, a rack server, a tower server, or the like.

[0109] It should be noted that the system architecture and application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0110] For ease of understanding, the following is an exemplary introduction to the calibration method of the assembly system provided by the present application in conjunction with the accompanying drawings. The calibration method of the assembly system is applicable to Figure 1 Assembled system shown.

[0111] Combine Figure 4 As shown, the embodiment of the present application provides a calibration method for an assembly system, which is applied to Figure 1 and Figure 2 The processing unit of the assembly system is composed of Figure 1 The processing unit shown is executed. The method includes:

[0112] S401 : Taking a surface of a first calibration body in a first orientation as a reference surface, obtaining spatial pose information when the reference surface is on a first reference surface as first reference information.

[0113] The calibration body is an object with known structural information and high processing precision. The calibration body can be a rectangular parallelepiped, etc., and there is no restriction on the shape of the calibration body. The embodiment of the present application takes the calibration body as a rectangular parallelepiped as an example. In the calibration scenario of the assembly system, the calibration body is placed in the assembly space and can be scanned by a camera with a 3D laser contour scanner. The laser emitted by the camera is irradiated onto the surface of the calibration body, thereby obtaining 3D image data of the surface of the calibration body. Based on this, the processing unit 10 can determine the spatial posture information of the surface of the calibration body in the assembly space according to the 3D image data.

[0114] To make it easier to distinguish, Figure 2 As shown, the calibration body is divided into a first calibration body and a second calibration body.

[0115] Exemplarily, the first calibration body and the second calibration body can be the same calibration body, that is, the same calibration body is placed at different positions in the assembly space and serves as the first calibration body and the second calibration body respectively. In addition, the first calibration body and the second calibration body can also be different calibration bodies, that is, two calibration bodies, and the two calibration bodies are placed at different positions in the assembly space. However, during the calibration process, no matter what position the first calibration body is initially in, the upper surface of the first calibration body is first placed on the first reference plane, and then the second surface of the second calibration body is calibrated based on the first reference plane.

[0116] like Figure 1 and Figure 2 As shown, the orientation can be one or more of up, down, left, and right. For example, the first orientation is up, the second orientation is down, the third orientation is left, and the fourth orientation is right.

[0117] Based on this, in the embodiment of the present application, the first surface is taken as the upper surface, the second surface is taken as the lower surface, the third surface is taken as the left surface, and the fourth surface is taken as the right surface.

[0118] It is understandable that when the first calibration body is placed in the assembly space, its upper surface can be within the acquisition range of the camera group, but the lower surface of the first calibration body cannot be scanned by the camera group. Therefore, the second calibration body is introduced and placed in an arbitrary position in the assembly space where its upper and lower surfaces can be captured by the camera group. In this way, the spatial position information of the lower surface of the second calibration body is determined by correlating the upper surface of the second calibration body with the upper surface of the first calibration body.

[0119] Exemplarily, the first surface and the second surface are arranged opposite to and in parallel.

[0120] Exemplarily, the third surface is opposite to and parallel to the fourth surface, and is adjacent to and perpendicular to the reference surface.

[0121] For example, the first calibration body and the second calibration body are located in the assembly space, with the upper surface of the first calibration body within the acquisition range of the first camera, and the upper surface of the second calibration body within the acquisition range of the first camera. The lower surface of the second calibration body is within the acquisition range of the second camera. In other words, the first calibration body only needs to ensure that its upper surface is within the acquisition range of the first camera. The second calibration body must ensure that its upper surface is within the acquisition range of the first camera and its lower surface is within the acquisition range of the second camera.

[0122] The reference plane is used to locate the surface of the components to be assembled in the assembly scene. In order to distinguish the different reference planes in the assembly space, they are divided into the first reference plane, the second reference plane, the third reference plane and the fourth reference plane. Among them, the first reference plane corresponds to the upper surface of the first calibration body, that is, when the upper surface of the first calibration body is in a specific posture (for example, horizontal), the upper surface is in the first reference plane. When the upper surface of the first calibration body is in the first reference plane, the left surface of the first calibration body is determined as the third reference plane. When the upper surface of the first calibration body is in the first reference plane, the right surface of the first calibration body is determined as the fourth reference plane. When the upper surface of the second calibration body is parallel to the first reference plane, the lower surface of the second calibration body is determined as the second reference plane.

[0123] In some embodiments, a surface of a first calibration object in a first orientation is used as a reference surface, and spatial pose information of the reference surface in a first reference pose is obtained as the first reference information. This includes: driving the first calibration object to move by a driving mechanism to adjust the spatial pose of the first calibration object until the error between the spatial pose information of the reference surface and the theoretical spatial pose information corresponding to the first reference surface falls within an allowable error range, and then using the spatial pose information of the reference surface as the first reference information. The second reference information is used to indicate a second reference surface in the assembly space.

[0124] In the embodiment of the present application, by adjusting the reference plane to the first reference plane (or falling within the allowable error range), a unified spatial reference is provided for the entire assembly system to make the reference information more accurate.

[0125] Illustratively, the theoretical spatial pose information corresponding to the first reference plane may refer to spatial pose information when the first reference plane is in a horizontal state.

[0126] For example, while driving the first calibration body, a dial indicator is used to confirm whether the reference surface is in the theoretical spatial pose information (e.g., horizontal). In this way, by gradually correcting the position of the first calibration body during the process of driving the first calibration body, the reference surface is positioned at the first reference plane, that is, the reference surface is in the theoretical spatial pose information (e.g., horizontal). Using this as a reference, the calibration result is more accurate.

[0127] The error range can be set according to actual needs.

[0128] In an embodiment of the present application, the purpose of calibrating the assembly system is to determine multiple reference planes, and the multiple reference planes are in a certain spatial relationship and spatial position relationship. For example, the multiple reference planes are in a parallel relationship and / or a perpendicular relationship. The reference information corresponding to the multiple reference planes can be used to locate the spatial postures of the multiple surfaces of the components to be assembled in the assembly scene. In the assembly scene, multiple reference planes can be used as references, and the multiple surfaces of the components to be assembled can be positioned with the corresponding reference information, so that the multiple surfaces of the components to be assembled are in the required spatial relationship and spatial position relationship. For example, in the assembly scene, there are two surfaces with an assembly requirement of a certain spatial relationship and spatial position relationship (such as a parallel relationship). Then, one of the surfaces can be positioned to the first reference plane, and the other surface can be positioned to the second reference plane to achieve the assembly requirement. In this way, the assembly accuracy of the assembly system can be improved, and the assembly efficiency can be improved at the same time.

[0129] Exemplarily, the first reference plane is parallel to the second reference plane.

[0130] Exemplarily, the third reference plane intersects with the first reference plane or the second reference plane. Further, the third reference plane is perpendicular to the first reference plane or the second reference plane.

[0131] Exemplarily, the fourth reference plane intersects with the first reference plane or the second reference plane. Further, the fourth reference plane is perpendicular to the first reference plane or the second reference plane.

[0132] Exemplarily, the third reference plane is parallel to the fourth reference plane.

[0133] It is understood that the above-mentioned multiple reference planes are used in combination according to assembly requirements. For example, in an assembly scenario, the assembly requirements are: the left surface of the assembly object and the right surface of the assembly target are parallel, and the left surface of the assembly object and the top surface of the assembly target are perpendicular. In this case, the required reference planes include: the third reference plane (for locating the left surface of the assembly object), the first reference plane (for locating the top surface of the assembly target), and the fourth reference plane (for locating the right surface of the assembly target).

[0134] Reference information is used to indicate a reference plane. It can be understood that the spatial pose information corresponding to the reference plane is reference information. For ease of distinction, reference information includes first, second, third, and fourth reference information. The first reference information indicates the first reference plane; the second reference information indicates the second reference plane in the assembly space. The third reference information indicates the third reference plane in the assembly space. The fourth reference information indicates the fourth reference plane in the assembly space.

[0135] Exemplarily, the first reference information is acquired by the processing unit through the first camera. The third reference information and the fourth reference information are acquired by the processing unit through the third camera and the fourth camera, respectively. The second reference information is determined based on the first reference information.

[0136] Spatial pose information is used by the processing unit to determine the spatial pose of the component to be assembled. It is understood that the spatial pose information of the reference plane is used to determine the spatial pose of multiple surfaces of the component to be assembled. That is, after obtaining the spatial pose information of the multiple reference planes of the calibration object in the assembly space, the spatial pose information of the multiple reference planes of the calibration object in the assembly space (i.e., the reference information) is used to adjust the spatial pose of the component to be assembled.

[0137] For example, the spatial posture information is represented by the posture information in the preset spatial coordinate system. The preset spatial coordinate system is set according to actual needs. For example, a vertex of the first reference plane can be used as the origin of the preset spatial coordinate system, combined with Figure 1 and Figure 2 As shown, the horizontal direction of the assembly space is the Y-axis direction, the vertical direction of the assembly space is the Z-axis direction, and the direction perpendicular to the Y-axis and the Z-axis is the X-axis direction.

[0138] The pose information includes the rotation angles of the reference plane relative to the X-axis and Y-axis in the assembly space, represented by RX and RY, respectively. For example, the rotation angles of the first reference plane relative to the X-axis are 30° and 40° relative to the Y-axis. Based on this, the first reference information corresponding to the first reference plane can be expressed as (30°, 40°).

[0139] In some embodiments, the first reference information is acquired by the processing unit via a first camera. Exemplarily, the first camera scans the reference surface when it is at the first reference surface, obtaining 3D image data corresponding to the reference surface at that time. The 3D image data includes spatial position information of multiple points. Based on the spatial position information of the multiple points corresponding to the reference surface, the processing unit calculates spatial pose information of the reference surface as the first reference information.

[0140] In some embodiments, the third reference information and the fourth reference information are acquired by the processing unit through the third camera and the fourth camera, respectively. Exemplarily, the third camera scans the left surface of the first calibration body when the reference surface is in the first reference surface, and obtains the 3D image data corresponding to the left surface at this time. The processing unit calculates the spatial posture information of the left surface as the third reference information based on the spatial position information of the multiple points corresponding to the left surface. Exemplarily, the fourth camera scans the right surface of the first calibration body when the reference surface is in the first reference surface, and obtains the 3D image data corresponding to the right surface at this time. The processing unit calculates the spatial posture information of the right surface as the fourth reference information based on the spatial position information of the multiple points corresponding to the right surface.

[0141] Exemplarily, the spatial position information is based on a preset spatial coordinate system, indicating the distance of a point relative to the coordinate origin in the X-axis, Y-axis, and Z-axis directions, and the point is represented by three coordinate values ​​(X, Y, Z).

[0142] Exemplarily, the number of the plurality of points is greater than or equal to 4. Specifically, the number of the plurality of points is equal to 4.

[0143] For example, combined Figure 5 As shown, for each surface, the multiple points may include preset points in each of the four areas (area 1, area 2, area 3, and area 4) of the surface. In other words, the preset points in area 1, area 2, area 3, and area 4 may be considered as the multiple points.

[0144] It is understandable that the size of each area is not limited and can be set according to the actual needs of the user.

[0145] Specifically, the preset point can be any point in the corresponding area (Area 1, Area 2, Area 3, Area 4). The preset point can also be the average of multiple points in the corresponding area. For example, if three points are randomly selected from Area 1, namely point a (Xa, Ya, Za), b (Xb, Yb, Zb), and c (Xc, Yc, Zc), the preset point can be the point corresponding to the average of these three points a, b, and c. For example, the preset points in Area 1 can be: ((Xa+Xb+Xc) / 3, (Ya+Yb+Yc) / 3, (Za+Zb+Zc) / 3).

[0146] Combine Figure 6 As shown, the embodiment of the present application provides a schematic diagram of a camera group scanning a calibration body. Figure 6 and Figure 7As shown, the surface of the calibration body scanned by the first camera is a surface composed of vertices AGHB. For example, for the surface of the calibration body scanned by the first camera, the preset point in area 1 is used as point 1, the preset point in area 2 is used as point 2, the preset point in area 3 is used as point 3, and the preset point in area 4 is used as point 4. Figure 6 and Figure 8 As shown, the surface of the calibration body scanned by the second camera is a surface composed of vertices DEFC. For example, for the surface of the calibration body scanned by the second camera, the preset point in area 1 is used as point 1, the preset point in area 2 is used as point 2, the preset point in area 3 is used as point 3, and the preset point in area 4 is used as vertex 4. Figure 6 and Figure 9 As shown, the surface of the calibration body scanned by the third camera is the surface formed by AGED. For example, for the surface of the calibration body scanned by the third camera, the preset point in area 1 is used as point 1, the preset point in area 2 is used as point 2, the preset point in area 3 is used as point 3, and the preset point in area 4 is used as point 4. Figure 6 and Figure 10 As shown, the surface of the calibration object scanned by the fourth camera is a surface formed by vertices CBHF. For example, for the surface of the calibration object scanned by the fourth camera, the preset points in area 1 are used as point 1, the preset points in area 2 are used as point 2, the preset points in area 3 are used as point 3, and the preset points in area 4 are used as point 4.

[0147] The following is an exemplary explanation of determining the spatial posture information of a surface based on the spatial position information of multiple points on the surface: for example, taking point A as the origin of the preset spatial coordinate system, the multiple points on the surface of the calibration body are point 1, point 2, point 3, and point 4, and their coordinate values ​​are (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), and (X4, Y4, Z4), respectively.

[0148] The rotation angle RX of the surface relative to the X-axis can be calculated as follows:

[0149] RX=(RX12+RX34) / 2;where,

[0150] ;

[0151] ;

[0152] In the above, RX12 is the angle between the line connecting point 1 and point 2 and the XOY plane. The XOY plane represents the plane formed by the X-axis and Y-axis in the preset spatial coordinate system. RX34 is the angle between the line connecting point 3 and point 4 and the XOY plane.

[0153] The rotation angle RY of the surface relative to the Y axis can be calculated as follows:

[0154] RY=(RY13+RY24) / 2; among them,

[0155] ;

[0156] ;

[0157] In the above, RY13 is the angle between the line connecting points 1 to 3 and the XOY plane, and RY24 is the angle between the line connecting points 2 to 4 and the XOY plane.

[0158] In summary, the spatial pose information of the surface can be expressed as: RX,RY.

[0159] Illustratively, the first reference plane, the third reference plane, and the fourth reference plane may be determined in the above manner, and the corresponding first reference information, third reference information, and fourth reference information are not described in detail here.

[0160] S402 : Acquire spatial pose information of a first surface of a second calibration body as first initial information, and acquire spatial pose information of a second surface of the second calibration body as second initial information.

[0161] The initial information refers to the spatial pose information of the second calibration body when it is in its initial position. The initial position refers to the placement of the second calibration body at any position where the first camera and the second camera can scan its upper and lower surfaces. In order to distinguish the initial information of different surfaces of the second calibration body, the initial information is divided into first initial information and second initial information. The first initial information refers to the spatial pose information obtained by scanning the surface of the second calibration body by the first camera when the second calibration body is in its initial position; the second initial information refers to the spatial pose information obtained by scanning the lower surface of the second calibration body by the second camera when the second calibration body is in its initial position.

[0162] In some exemplary embodiments, the first initial information is acquired by the processing unit through a first camera, and the second initial information is acquired by the processing unit through a second camera.

[0163] In one implementation, when the second calibration body is in the initial position, the first camera scans the upper surface of the second calibration body and obtains 3D image data of the upper surface of the second calibration body. The processing unit obtains the 3D image data of the upper surface of the second calibration body and determines the spatial pose information of the upper surface, i.e., the first initial information, based on the spatial position information of multiple points corresponding to the upper surface.

[0164] In one implementation, when the second calibration body is in the initial position, the second camera scans the lower surface of the second calibration body and obtains 3D image data of the lower surface of the second calibration body. The processing unit obtains the 3D image data of the lower surface of the second calibration body and determines the spatial pose information of the lower surface, i.e., the second initial information, based on the spatial position information of multiple points corresponding to the lower surface.

[0165] Exemplarily, the spatial posture information of the upper surface is determined based on the spatial position information of multiple points corresponding to the upper surface, and / or the spatial posture information of the lower surface is determined based on the spatial position information of multiple points corresponding to the lower surface. This can be determined according to the step of determining the spatial posture information of the surface based on the spatial position information of multiple points on the surface, which will not be repeated here.

[0166] S403: Calibrate the second initial information according to the deviation between the first initial information and the first reference information to obtain second reference information.

[0167] In this way, the first surface of the second calibration body is associated with the first reference plane through the deviation between the first initial information and the first reference plane. The deviation can indicate the posture difference between the second initial information and the second reference plane, thereby determining the second reference plane.

[0168] In some embodiments, calibrating the second initial information according to the deviation between the first initial information and the first reference information to obtain the second reference information includes: calculating the difference between the second initial information and the deviation to obtain the second reference information.

[0169] In an embodiment of the present application, the deviation between the first initial information and the first reference information can clearly determine the posture difference required to adjust the upper surface of the second calibration body to the first reference surface, providing an accurate and reasonable calibration basis for the subsequent calibration of the lower surface of the second calibration body. Since the lower surface of the second calibration body will be adjusted accordingly with the upper surface, by calculating the difference between the second initial information and the deviation, the second reference information of the lower surface of the second calibration body after adjusting the target coordinate value based on the second initial information can be obtained. It can be understood that in order to reflect the calculation principle, this process does not necessarily require actual adjustment of the second calibration body.

[0170] Exemplarily, the first initial information is expressed as (RX01, RY01), the second initial information is expressed as (RX02, RY02), and the first reference information is expressed as (RX1, RY1).

[0171] Based on this, the X-axis deviation ΔRX between the first initial information RX01 and the first reference information RX1 is calculated: ΔRX=RX01-RX1; and the Y-axis deviation ΔRY between the first initial information RY01 and the first reference information RY1 is calculated: ΔRY=RY01-RY1.

[0172] Then calculate the difference between RX02 of the second initial information and the X-axis deviation △RX to obtain RX2: RX2=RX02-△RX; and calculate the difference between RY02 of the second initial information and the Y-axis deviation △RY to obtain RY2: RY2=RY02-△RY; in this way, the second reference information (RX2, RY2) is obtained.

[0173] In an embodiment of the present application, the second reference information can be accurately obtained, and the angle of the second reference plane relative to the X-axis is RX2, and the angle relative to the Y-axis is RY2. Through these two key spatial dimensions, the spatial position of the second reference plane in the assembly space is accurately located, thereby providing accurate second reference information for the components to be assembled in the assembly scene, thereby improving the assembly accuracy.

[0174] In the embodiments of this application, the goal of calibrating the assembly system is to establish a specific spatial relationship between the first and second reference surfaces. This spatial relationship can be directly applied in assembly scenarios. For example, based on assembly requirements and reference information corresponding to the reference surface, surfaces that need to be on the reference surface can be positioned on that surface to improve assembly efficiency and accuracy.

[0175] In some embodiments, the method further includes: when the reference plane is located at the first reference plane, acquiring spatial pose information of a third surface of the first calibration body as third reference information.

[0176] In an embodiment of the present application, with the first reference plane as a reference, the left surface of the first calibration body is on the third reference plane. At this time, the spatial position information of the left surface of the first calibration body serves as the third reference information. Based on this, the first reference plane or the second reference plane has a certain spatial position relationship with the third reference plane, such as an intersection relationship, and further, a vertical relationship. In this way, the spatial position relationship between the first reference plane, the second reference plane, and the third reference plane can be directly applied in the assembly scene to further improve the assembly accuracy and efficiency.

[0177] In some embodiments, the method further includes: when the reference plane is located at the first reference plane, acquiring spatial pose information of a fourth surface of the first calibration body as fourth reference information.

[0178] In this way, with the first reference plane as a reference, the right surface of the first calibration body is now on the fourth reference plane, and the spatial position information of the right surface of the first calibration body is used as the fourth reference information. Based on this, the first reference plane or the second reference plane has a certain spatial position relationship with the fourth reference plane, such as an intersection relationship, and further, it can be a perpendicular relationship. In this way, the spatial position relationship between the first reference plane, the second reference plane, and the fourth reference plane can be directly applied in the assembly scene, further improving the assembly accuracy and efficiency.

[0179] The second reference plane is calibrated with the reference plane being on the first reference plane as the reference. The goal is to achieve a certain spatial positional relationship between the first and second reference planes, such as parallelism. In this way, in an assembly scenario, the first and second reference planes' corresponding first and second reference planes can directly determine the spatial position of the two surfaces that need to be parallel, so that the two surfaces can be parallelized accurately and efficiently.

[0180] In addition, the third and / or fourth reference planes are obtained based on the reference plane being on the first reference plane. The purpose is to achieve a certain spatial position relationship between the first, third, and fourth reference planes. For example, taking the calibration object as a rectangular parallelepiped, the third and fourth reference planes are parallel to each other and perpendicular to the first reference plane. In this way, in the assembly scenario, the third and / or fourth reference planes corresponding to the third and / or fourth reference planes can directly locate the spatial position of multiple surfaces, which is more comprehensive, making assembly more accurate and efficient.

[0181] Combine Figure 11 As shown, the embodiment of the present application provides an assembly method, which is applied to Figure 3 The processing unit of the assembly system is composed of Figure 3 The processing unit shown is executed. The method includes:

[0182] S1101 , according to first reference information, controlling a driving mechanism to drive a first preset surface of an assembly target to be located at a first reference plane.

[0183] The first reference information is the spatial posture information of the reference surface when it is in the first reference plane. The reference plane is the surface of the first orientation of the first calibration body. The first reference information is used to indicate the first reference plane. The first surface is the surface of the first orientation.

[0184] In some embodiments, according to the first reference information, the first driving mechanism is controlled to drive the first preset surface of the assembly target to be at the first reference plane.

[0185] In one implementation, while driving the assembly target, a first camera captures current 3D image data of a first preset surface in real time, and a processing unit obtains the current 3D image data of the first preset surface. Current spatial pose information corresponding to the first preset surface is determined based on the current spatial position coordinates of multiple points in the current 3D image data. This is then compared with first reference information until the current spatial pose information equals the first reference information (or is within an error range), thereby positioning the first preset surface on the first reference plane.

[0186] S1102 : According to the second reference information, control the driving mechanism to drive the second preset surface of the assembly object to be located at the second reference plane.

[0187] The second reference information is spatial position information of the second surface of the second calibration body when it is on the second reference plane. The second reference information is used to indicate the second reference plane. The first reference plane and the second reference plane are in a predetermined spatial position relationship in the assembly space. Exemplarily, the predetermined spatial position relationship can be a parallel relationship.

[0188] In some embodiments, according to the second reference information, the second driving mechanism is controlled to drive the second preset surface of the assembly object to be at the second reference plane.

[0189] In one implementation, while the assembly object is being driven to move, the second camera captures current 3D image data of the second preset surface in real time, and the processing unit obtains the current 3D image data of the second preset surface. Current spatial pose information corresponding to the second preset surface is determined based on the current spatial position coordinates of multiple points in the current 3D image data. The current spatial pose information is then compared with the second reference information until the current spatial pose information equals the second reference information (or is within an error range), thereby positioning the second preset surface on the second reference plane.

[0190] In some embodiments, the method further includes: controlling the driving mechanism to drive the third surface of the assembly object and / or the assembly target to be at a third reference plane according to the third reference information.

[0191] The third reference information is used to indicate the spatial position information of the third surface of the first calibration body when the reference surface is located on the first reference surface. The third reference information is used to indicate the third reference surface in the assembly space.

[0192] In the embodiments of the present application, the third reference plane is determined using the first reference plane as a reference, thereby determining the spatial positional relationship between the first and third reference planes. For example, the first and third reference planes intersect, and furthermore, are perpendicular to, each other. In this way, based on the spatial positional relationship between the first and third reference planes, the spatial position of multiple surfaces of a component to be assembled (for example, two surfaces that need to intersect or be perpendicular to each other) can be directly defined in an assembly scenario, thereby improving assembly accuracy and efficiency.

[0193] In one implementation, based on the third reference information, the driving mechanism is controlled to drive the third surface of the assembly object and / or the assembly target to be located at the third reference plane.

[0194] Exemplarily, while driving the assembly object and / or assembly target to move, the third camera captures current 3D image data of the left surface in real time, and the processing unit obtains the current 3D image data of the left surface. Current spatial pose information corresponding to the left surface is determined based on the current spatial position coordinates of multiple points in the current 3D image data. The current spatial pose information is compared with the third reference information until the current spatial pose information equals the third reference information (or is within an error range), thereby positioning the left surface on the third reference plane.

[0195] In some embodiments, the method further includes: controlling the driving mechanism to drive a fourth surface of the assembly object and / or the assembly target to be located at a fourth reference plane according to fourth reference information.

[0196] The fourth reference information is used to indicate the spatial position information of the fourth surface of the first calibration object when the reference surface is located on the first reference surface. The fourth reference information is used to indicate the fourth reference surface in the assembly space.

[0197] In an embodiment of the present application, the fourth reference plane is determined with reference to the first reference plane, thereby deriving the spatial positional relationship between the first and fourth reference planes. For example, the first and fourth reference planes intersect, and furthermore, are perpendicular to, each other. Thus, based on the spatial positional relationship between the first and fourth reference planes, the spatial position of multiple surfaces of the assembly to be assembled (for example, two surfaces that need to be intersecting or perpendicular to each other) can be directly defined in the assembly scenario, thereby improving assembly accuracy and efficiency. Furthermore, illustratively, the third reference plane is parallel to the fourth reference plane.

[0198] In one implementation, based on the fourth reference information, the driving mechanism is controlled to drive the fourth surface of the assembly object and / or the assembly target to be at the fourth reference plane.

[0199] Exemplarily, while driving the assembly object and / or assembly target to move, the fourth camera captures current 3D image data of the right surface in real time, and the processing unit obtains the current 3D image data of the right surface. Current spatial pose information corresponding to the right surface is determined based on the current spatial position coordinates of multiple points in the current 3D image data. The current spatial pose information is compared with the fourth reference information until the current spatial pose information equals the fourth reference information (or is within an error range), thereby positioning the right surface on the fourth reference plane.

[0200] In some embodiments, the method further includes obtaining a user's assembly requirements. The assembly requirements include a first predetermined surface of the assembly target and a second predetermined surface of the assembly object. In other words, the processing unit positions the first predetermined surface and the second predetermined surface to the first reference plane and the second reference plane, respectively, based on the assembly requirements.

[0201] In some embodiments, if it is necessary to position different surfaces of the assembly object and the assembly target, for example, to position the upper surface of the assembly target parallel to the lower surface of the assembly object, or to position the upper surface of the assembly target perpendicular to the left surface of the assembly object, etc., then based on the first reference plane, the second reference plane, the third reference plane and the fourth reference plane, more than two reference planes can be selected in combination according to the user's actual assembly needs to achieve the spatial position relationship between more than two surfaces of the assembly object and the assembly target. For example, when the upper surface of the assembly target needs to be parallel to the lower surface of the assembly object, the upper surface of the assembly target is positioned to the first reference plane, and the lower surface of the assembly object is positioned to the second reference plane. For another example, when the upper surface of the assembly target needs to be perpendicular to the left surface of the assembly object, the upper surface of the assembly target is positioned to the first reference plane, and the left surface of the assembly object is positioned to the third reference plane.

[0202] It is worth noting that if the upper surface of the assembly object needs to be located, since the upper surface of the assembly object is a blind spot and cannot be scanned by the camera, and the lower surface of the suction nozzle is in contact with the upper surface of the assembly object, the lower surface of the suction nozzle is regarded as the upper surface of the assembly object. In this case, it is equivalent to the lower surface of the suction nozzle that needs to be positioned on the assembly target, that is, a different surface. Therefore, two reference surfaces are required for positioning. Specifically, before the suction nozzle sucks the assembly object, the lower surface of the suction nozzle is positioned to the second reference surface. After that, the assembly object is sucked.

[0203] In other embodiments, if it is necessary to locate the same surface of the assembly object and the assembly target, for example, to locate the left surface of the assembly object and the left surface of the assembly object, then based on the first reference plane, the second reference plane, the third reference plane and the fourth reference plane, the spatial position relationship of the same surface of the assembly object and the assembly target can be achieved by selecting one of the reference planes (such as the second reference plane).

[0204] Combine Figure 3 As shown in the figure, the assembly scenario of the assembly system is illustrated. In the assembly scenario, the assembly target and the assembly object can be correctly matched. Before assembly, the assembly target and the assembly object need to be planned. For example, if you want to achieve Figure 3 The three-dimensional fit of the assembly target and the assembly object is as follows: the upper surface of the assembly target is parallel to the upper surface of the assembly object, and the right surface of the assembly target is parallel to the left surface of the assembly object.

[0205] Based on this, first, calibrate the upper surface of the assembly target (i.e. Figure 3 The upper surface 1), the right surface, the lower surface of the nozzle (that is, the upper surface of the assembly object, i.e. Figure 3 The lower surface 1 or upper surface 3) in the assembly target and the posture information of the left surface of the assembly object are used to achieve that the upper surface of the assembly target is parallel to the upper surface of the assembly object, and the right surface of the assembly target is parallel to the left surface of the assembly object.

[0206] Specifically, the upper surface of the assembly target needs to be positioned to the first reference plane, and the right surface of the assembly target needs to be positioned to the fourth reference plane: the assembly target is placed on the first drive mechanism, and the processing unit controls the first drive mechanism to drive the assembly target to move, so as to calibrate the spatial position information of the upper surface of the assembly target to the first reference information. During the positioning process, the first camera scans the upper surface of the current assembly target in real time and obtains 3D image data. The processing unit determines whether the spatial position information of the upper surface of the current assembly target is the first reference information based on the 3D image data, that is, whether the upper surface of the current assembly target is in the first reference plane. Similarly, the right surface of the assembly target needs to be positioned to the fourth reference plane. During the positioning process, the fourth camera scans the right surface of the current assembly target in real time and obtains 3D image data. The processing unit 10 determines whether the spatial position information of the right surface of the current assembly target is the fourth reference information based on the 3D image data, that is, whether the right surface of the current assembly target is in the fourth reference plane. When the spatial position information of the upper surface of the current assembly target is the first reference information and the spatial position information of the right surface is the fourth reference information, the calibration of the assembly target is completed.

[0207] In addition, the lower surface of the suction nozzle needs to be positioned to the second reference plane, and the left surface of the assembly object needs to be positioned to the third reference plane: the lower surface of the suction nozzle is calibrated before the suction nozzle sucks the assembly object, and the processing unit controls the movement of the second drive mechanism to realize the movement of the suction nozzle and position the lower surface of the suction nozzle to the second reference plane. During the positioning process, the second camera scans the lower surface of the current suction nozzle in real time and obtains a 3D image. The processing unit determines whether the spatial position information of the lower surface of the current suction nozzle is the second reference information based on the 3D image, that is, whether the current suctioned lower surface is on the second reference plane. Similarly, the left surface of the assembly object needs to be positioned to the third reference plane: the upper surface of the assembly object is attached to the lower surface of the suction nozzle. At this time, the processing unit controls the second drive mechanism to drive the assembly object to move. During the positioning process, the third camera scans the left surface of the current assembly object in real time and obtains a 3D image. The processing unit determines whether the spatial position information of the left surface of the current assembly object is the third reference information based on the 3D image, that is, whether the left surface of the current assembly object is on the third reference plane. When the spatial posture information of the lower surface of the suction nozzle is the second reference information and the spatial posture information of the left surface of the assembly object is the third reference information, calibration of the assembly object is completed.

[0208] In this way, when the spatial posture information of the upper surface of the current assembly target is the first reference information, the spatial posture information of the right surface is the fourth reference information, the spatial posture information of the lower surface of the suction nozzle is the second reference information, and the spatial posture information of the left surface of the assembly object is the third reference information, the upper surface of the assembly target is parallel to the upper surface of the assembly object, and the right surface of the assembly target is parallel to the left surface of the assembly object.

[0209] Then, on the basis that the upper surface of the assembly target is parallel to the upper surface of the assembly object, and the right surface of the assembly target is parallel to the left surface of the assembly object, the processing unit controls the second drive mechanism to drive the assembly object to translate in one or more directions among the X-axis, Y-axis, and Z-axis to achieve the alignment of upper surface 2 and lower surface 2. For example, during assembly planning, a target point (such as target point 1 and target point 2) can be determined on upper surface 2 and lower surface 2 respectively. During translation, if target point 1 and target point 2 coincide, assembly is complete.

[0210] In the embodiments of this application, the first and second reference information corresponding to the first and second reference planes are used to directly locate the spatial position of two surfaces that need to conform to a predetermined spatial relationship (e.g., parallelism), thereby accurately and efficiently positioning the two surfaces. Furthermore, third and fourth reference information corresponding to the third and second reference planes are combined to locate the spatial relationship of more surfaces in the assembly scenario, further improving assembly accuracy and efficiency.

[0211] like Figure 12 As shown, an embodiment of the present application provides a calibration device 200 for an assembly system, which is applied to a processing unit of the assembly system. The assembly system also includes a camera group, which includes at least one camera for collecting spatial pose information of a component to be assembled placed in an assembly space. The spatial pose information is used by the processing unit to locate the spatial pose of the component to be assembled; the camera is also used to scan the spatial pose information of each surface of a calibration body placed in the assembly space. The calibration device 200 for the assembly system includes: a first acquisition module 21, a second acquisition module 22, and a calibration module 23. The first acquisition module 21 is used to use the surface of the first orientation of the first calibration body as a reference surface, and obtain the spatial pose information when the reference surface is in the first reference plane as the first reference information. The first reference information is used to indicate the first reference plane. The second acquisition module 22 is used to obtain the spatial pose information of the first surface of the second calibration body as the first initial information, and obtain the spatial pose information of the second surface of the second calibration body as the second initial information. The first surface is the surface of the first orientation. The calibration module 23 is used to calibrate the second initial information according to the deviation between the first initial information and the first reference information to obtain second reference information, where the second reference information is used to indicate a second reference plane in the assembly space.

[0212] like Figure 13 As shown, an embodiment of the present application provides an assembly device 300 for use in a processing unit of an assembly system. The assembly system also includes a camera group and a drive mechanism. The camera group includes at least one camera for collecting spatial pose information of a component to be assembled placed in an assembly space. This spatial pose information is used by the processing unit to locate the spatial pose of the component to be assembled. The component to be assembled includes an assembly target and an assembly object. The camera is also used to scan the spatial pose information of each surface of a calibration object placed in the assembly space. The drive mechanism is used to drive the component to be assembled. The assembly device 300 includes: a first control module 31 and a second control module 32. The first control module 31 is used to control the drive mechanism to drive a first preset surface of the assembly target to a first reference plane based on first reference information. The first reference information is the spatial pose information when the reference surface is at the first reference plane. The reference surface is a surface of a first calibration object at a first orientation. The first reference information is used to indicate the first reference plane. The first surface is a surface at the first orientation. The second control module 32 is used to control the drive mechanism to drive a second preset surface of the assembly object to a second reference plane based on second reference information. The second reference information is the spatial pose information when the second surface of the second calibration object is at the second reference plane. The second reference information is used to indicate the second reference plane. The first reference plane and the second reference plane are in a preset spatial position relationship in the assembly space.

[0213] like Figure 14As shown, an embodiment of the present application provides an electronic device 500. The electronic device 500 includes a processor 510 and a memory 520 for storing processor-executable instructions. When the processor 510 is configured to execute the instructions, the electronic device 500 implements the method described above.

[0214] Figure 14 The electronic device 500 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0215] In the embodiment of the present application, the processor 510 may correspond to the above Figure 1 、 Figure 2 、 Figure 3 The processing unit 10 in.

[0216] The electronic device 500 is implemented as a general-purpose computing device and its components may include, but are not limited to, one or more processors 510 , a memory 520 , a communication bus 540 connecting various system components (including the memory 520 and the processor 510 ), and a communication interface 530 .

[0217] Communication bus 540 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnection (PCI) bus.

[0218] The electronic device 500 typically includes a variety of computer system readable media, which can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0219] The memory 520 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Figure 14 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a Compact Disc Read Only Memory (hereinafter referred to as: CD-ROM), a Digital Video Disc Read Only Memory (hereinafter referred to as: DVD-ROM), or other optical media) may be provided. In these cases, each drive can be connected to the communication bus 540 via one or more data medium interfaces. The memory 520 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0220] A program / utility having a set (at least one) of program modules may be stored in memory 520. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. The program modules generally perform the functions and / or methods of the embodiments described herein.

[0221] The electronic device 500 may also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through the communication interface 530. In addition, the electronic device 500 may also communicate with the network adapter ( Figure 14 The network adapter can communicate with other modules of the electronic device through the communication bus 540. It should be understood that although Figure 14 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Drives; hereinafter referred to as: RAID) systems, tape drives, and data backup storage systems.

[0222] The processor 510 executes various functional applications and data processing by running programs stored in the memory 520 .

[0223] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 500. In other embodiments of the present application, the electronic device 500 may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0224] The present application provides a computer program product. When the computer program product is run in an electronic device, the electronic device is enabled to execute the above method to implement the above method.

[0225] The present application provides a readable storage medium, which includes software instructions. When the software instructions are executed in an electronic device, the electronic device implements the above method.

[0226] It is understandable that the above-mentioned electronic devices, computer program products, readable storage media, etc., in order to realize the above-mentioned functions, include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the various exemplary units and algorithm steps described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0227] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device etc. according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0228] The electronic device provided in the embodiment of the present application is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0229] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0230] The functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0231] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

[0232] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A calibration method for an assembly system, characterized in that: A processing unit applied to the assembly system, the assembly system further comprising a camera group, the camera group comprising at least one camera for collecting spatial pose information of a component to be assembled placed in an assembly space, the spatial pose information being used by the processing unit to locate the spatial pose of the component to be assembled; the camera is further configured to scan the spatial pose information of each surface of a calibration body placed in the assembly space; The camera is used to scan 3D image data of the surface of the calibration object or the component to be assembled, and the 3D image data is used to determine the spatial pose information; The method comprises: Taking a surface of a first orientation of a first calibration body as a reference surface, obtaining spatial pose information when the reference surface is on the first reference plane as first reference information; the first reference information is used to indicate the first reference plane; Acquire spatial pose information of a first surface of a second calibration body as first initial information, and acquire spatial pose information of a second surface of the second calibration body as second initial information; the first surface is a surface in the first orientation; According to the deviation between the first initial information and the first reference information, the first surface of the second calibration body is associated with the first reference plane, and the second initial information is calibrated using the deviation as the difference between the second initial information and the second reference information to obtain the second reference information, which is used to indicate the second reference plane in the assembly space.

2. The calibration method according to claim 1, characterized in that: The camera group includes a first camera and a second camera at fixed positions, the reference surface and the first surface are located within a collection range of the first camera, and the second surface is located within a collection range of the second camera; the first reference information and the first initial information are acquired by the processing unit through the first camera; The second initial information is collected by the processing unit through the second camera.

3. The calibration method according to claim 1 or 2, characterized in that: The assembly system further includes a driving mechanism for driving the assembly to be assembled or the calibration object to move; the method of taking the surface of the first calibration object in the first orientation as a reference surface and obtaining spatial posture information of the reference surface in a first reference posture as the first reference information includes: The first calibration body is driven to move by the driving mechanism to adjust the spatial posture of the first calibration body until the error between the spatial posture information of the reference surface and the theoretical spatial posture information corresponding to the first reference surface falls within an allowable error range, and the spatial posture information of the reference surface is used as the first reference information.

4. The calibration method according to claim 1 or 2, characterized in that: The first reference plane is parallel to the second reference plane; the method further comprises: When the reference surface is located at the first reference plane, obtaining spatial pose information of a third surface of the first calibration body as third reference information; the third surface is a surface located at a third position, and the third reference information is used to indicate a third reference plane in the assembly space; and / or, When the reference surface is located at the first reference plane, obtaining spatial pose information of a fourth surface of the first calibration object as fourth reference information; the fourth surface is a surface located at a fourth orientation, and the fourth reference information is used to indicate a fourth reference plane in the assembly space; The third reference plane intersects with the first reference plane or the second reference plane and is parallel to the fourth reference plane.

5. The calibration method according to claim 4, characterized in that: The third surface is opposite to and parallel to the fourth surface, and is adjacent to and perpendicular to the reference surface; the third reference surface is perpendicular to the first reference surface or the second reference surface.

6. The calibration method according to claim 4, characterized in that: The camera group includes a third camera and a fourth camera with fixed positions, the third surface is located within the acquisition range of the third camera, and the fourth surface is located within the acquisition range of the fourth camera; the third reference information and the fourth reference information are respectively acquired by the processing unit through the third camera and the fourth camera.

7. The calibration method according to claim 1, characterized in that: The camera includes a 3D laser profile scanner for scanning 3D image data of the surface of the calibration object or the component to be assembled, and the 3D image data includes spatial position information of multiple points on the surface.

8. An assembly method, characterized in that: A processing unit for an assembly system, the assembly system further comprising a camera group and a drive mechanism, the camera group comprising at least one camera for collecting spatial pose information of a component to be assembled placed in an assembly space, the spatial pose information being used by the processing unit to locate the spatial pose of the component to be assembled, the component to be assembled comprising an assembly target and an assembly object, the camera further being used to scan spatial pose information of each surface of a calibration body placed in the assembly space; the camera being used to scan 3D image data of a surface of the calibration body or the component to be assembled, the 3D image data being used to determine the spatial pose information; The driving mechanism is used to drive the assembly to be assembled to move; The method comprises: controlling the driving mechanism to drive a first preset surface of the assembly target to be located at a first reference plane according to first reference information, wherein the first reference information is spatial position information of a reference plane when the reference plane is located at the first reference plane, the reference plane is a surface of a first calibration object in a first orientation, the first reference information is used to indicate the first reference plane, and the first surface is a surface in the first orientation; According to the second reference information, the driving mechanism is controlled to drive the second preset surface of the assembly object to be located at the second reference plane, where the second reference information is spatial position information of the second surface of the second calibration body when the second surface is located at the second reference plane, and the second reference information is used to indicate the second reference plane; the second reference information is determined based on the deviation between the first reference information and the first initial information, and the second initial information; the deviation is used to associate the first surface of the second calibration body with the first reference plane and indicate the difference between the second initial information and the second reference information; the first initial information and the second initial information respectively represent the spatial position information of the first surface and the second surface of the second calibration body when the second calibration body is in the assembly space; The first reference plane and the second reference plane are in a preset spatial position relationship in the assembly space.

9. An assembly system, characterized in that: include: Processing unit, camera group and driving mechanism; wherein, The camera group includes at least one camera for collecting spatial pose information of a component to be assembled placed in an assembly space, the spatial pose information being used by the processing unit to locate the spatial pose of the component to be assembled, the component to be assembled including an assembly target and an assembly object; the camera is further used to scan the spatial pose information of each surface of a calibration body placed in the assembly space; the camera is used to scan 3D image data of the surface of the calibration body or the component to be assembled, and the 3D image data is used to determine the spatial pose information; The driving mechanism is used to drive the assembly to be assembled to move; The processing unit is configured to control the driving mechanism to drive the first preset surface of the assembly target to be located at a first reference plane based on first reference information, wherein the first reference information is spatial position information of the reference plane when the reference plane is located at the first reference plane, the reference plane is a surface of a first orientation of a first calibration body, the first reference information is used to indicate the first reference plane, and the first surface is a surface of the first orientation; and is further configured to control the driving mechanism to drive the second preset surface of the assembly object to be located at a second reference plane based on second reference information, wherein the second reference information is spatial position information of the second surface of the second calibration body when the second reference plane is located at the second reference plane, and the second reference information is used to indicate the second reference plane; the second reference information is determined based on a deviation between the first reference information and the first initial information, and the second initial information; the deviation is used to associate the first surface of the second calibration body with the first reference plane, and to indicate a difference between the second initial information and the second reference information; the first initial information and the second initial information respectively represent spatial position information of the first surface and the second surface of the second calibration body when the second calibration body is in the assembly space; The first reference plane and the second reference plane are in a preset spatial position relationship in the assembly space.

10. An electronic device, characterized in that: include: processor and memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the calibration method according to any one of claims 1 to 7 or the assembly method according to claim 8.

Images