A product assembly method, apparatus and readable storage medium

Abstract

The application discloses a product assembling method, device and readable storage medium, and the method comprises the following steps: controlling a first mobile platform to drive a first product to rotate, so that a first included angle between the first product and a reference plane is less than or equal to a first threshold value; controlling a second mobile platform to drive a second product to rotate, so that a second included angle between the second product and the first product is less than or equal to a second threshold value; controlling a camera to shoot the first product and the second product to obtain a first image; according to the first image, controlling the second mobile platform to drive the second product to translate and / or rotate in the reference plane, so that the second product is aligned with the first product; and controlling the second mobile platform to drive the second product to move towards the first product, so as to assemble the first product and the second product.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic equipment, and more particularly, to a product assembling method, a product assembling device, and a computer readable storage medium. BACKGROUND

[0002] Augmented Reality (AR) technology is a new technology that superimposes computer-generated virtual information on the real world where the user is, and is an important branch of virtual reality technology. It improves the user's perception of the real world and provides a new way for humans to communicate with the world. An AR module is a core component of an AR device, responsible for generating and delivering images to the user's field of view. In order to make the AR module present the best visual effect, the parallelism of the light engine and the waveguide sheet assembly, the alignment in the xy direction, and the rotation angle in the AR module all need to meet the preset high precision requirements.

[0003] In the assembly process of the light engine and the waveguide sheet, mechanical assembly is usually performed based on a human-simulated assembly algorithm. However, due to the low accuracy of the human-simulated assembly algorithm in the mechanical control process, the coordinate error calculated based on the algorithm is large, and thus the deviation between the light engine and the waveguide sheet is large after the AR module is assembled, resulting in low assembly precision of the light engine and the waveguide sheet. SUMMARY

[0004] An object of embodiments of the present application is to provide a product assembling method, device and readable storage medium.

[0005] According to a first aspect of the present disclosure, a product assembling method is provided, comprising:

[0006] controlling a first mobile platform to rotate a first product, so that a first included angle between the first product and a reference plane is less than or equal to a first threshold value;

[0007] controlling a second mobile platform to rotate a second product, so that a second included angle between the second product and the first product is less than or equal to a second threshold value;

[0008] controlling a camera to take a photograph of the first product and the second product, to obtain a first image;

[0009] controlling the second mobile platform to translate and / or rotate the second product in the reference plane, based on the first image, so that the second product is aligned with the first product;

[0010] controlling the second mobile platform to move the second product towards the first product, to assemble the first product and the second product.

[0011] Optionally, the control of the first mobile platform to rotate the first product such that a first included angle between the first product and the reference plane is less than or equal to a first threshold value comprises:

[0012] measuring first coordinates of at least three first identification points in the first product in a set measurement coordinate system, wherein a first coordinate axis and a second coordinate axis of the measurement coordinate system are both parallel to the reference plane;

[0013] controlling the first mobile platform to rotate the first product according to the first coordinates such that the first included angle is less than or equal to the first threshold value.

[0014] Optionally, the control of the second mobile platform to rotate the second product such that a second included angle between the second product and the first product is less than or equal to a second threshold value comprises:

[0015] measuring second coordinates of at least three first identification points in the first product in a set measurement coordinate system, wherein a first coordinate axis and a second coordinate axis of the measurement coordinate system are both parallel to the reference plane;

[0016] measuring third coordinates of at least three second identification points in the second product in the measurement coordinate system;

[0017] controlling the second mobile platform to rotate the second product according to the second coordinates and the third coordinates such that the second included angle is less than or equal to the second threshold value.

[0018] Optionally, before the control of the second mobile platform to rotate the second product such that the second included angle between the second product and the first product is less than or equal to the second threshold value, the method further comprises:

[0019] controlling the second mobile platform to move the second product to below the first product.

[0020] Optionally, before the control of the second mobile platform to rotate the second product such that the second included angle between the second product and the first product is less than or equal to the second threshold value, the method further comprises:

[0021] measuring fourth coordinates of at least three second identification points in the second product in a set measurement coordinate system, wherein a first coordinate axis and a second coordinate axis of the measurement coordinate system are both parallel to the reference plane;

[0022] controlling the second mobile platform to rotate the second product according to the fourth coordinates such that a third included angle between the second product and the reference plane is less than or equal to a third threshold value.

[0023] Optionally, the controlling, according to the first image, the second mobile platform to drive the second product to translate and / or rotate in the reference plane so that the second product is aligned with the first product comprises:

[0024] determining, according to the first image, a rotation angle deviation and / or a distance deviation of the first product and the second product in the reference plane;

[0025] controlling, according to the rotation angle deviation, the second mobile platform to drive the second product to rotate in the reference plane, and / or controlling, according to the distance deviation, the second mobile platform to drive the second product to translate in the reference plane so that the second product is aligned with the first product.

[0026] Optionally, the controlling, according to the rotation angle deviation, the second mobile platform to drive the second product to rotate in the reference plane comprises:

[0027] controlling, according to the rotation angle deviation, the second mobile platform to drive the second product to rotate in the reference plane around a direction perpendicular to the reference plane so that the rotation angle deviation of the second product and the first product in the reference plane is less than or equal to a fourth threshold value.

[0028] Optionally, the controlling, according to the distance deviation, the second mobile platform to drive the second product to translate in the reference plane comprises:

[0029] controlling, according to the distance deviation, the second mobile platform to drive the second product to translate in the reference plane so that the distance deviation of the second product and the first product in the reference plane is less than or equal to a fifth threshold value.

[0030] According to a second aspect of the present disclosure, there is provided a product assembling device, comprising:

[0031] a first control module configured to control a first mobile platform to drive a first product to rotate so that a first included angle between the first product and a reference plane is less than or equal to a first threshold value;

[0032] a second control module configured to control a second mobile platform to drive a second product to rotate so that a second included angle between the second product and the first product is less than or equal to a second threshold value;

[0033] a third control module configured to control a camera to capture the first product and the second product to obtain a first image;

[0034] a fourth control module configured to control the second mobile platform to drive the second product to translate and / or rotate in the reference plane according to the first image, so that the second product is aligned with the first product;

[0035] a fifth control module configured to control the second mobile platform to drive the second product to move towards the first product, so as to assemble the first product and the second product.

[0036] According to a third aspect of the present disclosure, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0037] According to the embodiments of the present disclosure, the first mobile platform is controlled to drive the first product to rotate, so that the first included angle between the first product and the reference plane is less than or equal to the first threshold value; the second mobile platform is controlled to drive the second product to rotate, so that the second included angle between the second product and the first product is less than or equal to the second threshold value; the second mobile platform is controlled to drive the second product to translate and / or rotate in the reference plane according to the first image obtained by the camera shooting the first product and the second product, so that the second product is aligned with the first product; the second mobile platform is controlled to drive the second product to move towards the first product, so as to assemble the first product and the second product. Thus, the automatic assembly of the first product and the second product with any shape can be realized, and the assembly precision of the first product and the second product can be improved.

[0038] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments thereof, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings incorporated in and forming a part of the specification illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0040] Figure 1 is a schematic diagram of a product assembly system according to an embodiment of the present disclosure;

[0041] Figure 2 is a schematic diagram of a measurement coordinate system according to an embodiment of the present disclosure;

[0042] Figure 3 is a schematic diagram of a first mobile coordinate system according to an embodiment of the present disclosure;

[0043] Figure 4 is a schematic diagram of a second mobile coordinate system according to an embodiment of the present disclosure;

[0044] Figure 5This is a schematic diagram of a field-of-view coordinate system according to an embodiment of the present disclosure;

[0045] Figure 6 This is a flowchart of a product assembly method provided according to an embodiment of the present disclosure;

[0046] Figure 7 This is a flowchart illustrating an example of a product assembly method provided according to an embodiment of this disclosure;

[0047] Figure 8 This is a block diagram of a product assembly apparatus provided according to an embodiment of the present disclosure. Detailed Implementation

[0048] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0049] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0050] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0051] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0052] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0053] This embodiment provides a product assembly system, such as Figure 1 As shown, the product assembly system 100 may include a measuring mechanism 110, a camera 120, a first moving platform 130, and a second moving platform 140.

[0054] In this embodiment, the first product can be placed on the first moving platform 130, and the first moving platform 130 drives the first product to rotate. The first moving platform 130 can be a 2-axis moving platform, which can drive the first product to rotate around a first direction and a second direction. The first direction can be the same as the X-axis direction of the reference Cartesian coordinate system, and the second direction can be the same as the Y-axis direction of the reference Cartesian coordinate system.

[0055] In this embodiment, the second product can be placed on the second moving platform 140, which drives the second product to rotate and translate. The second moving platform 140 can be a 6-axis moving platform, which can drive the second product to translate along the first direction, the second direction, and the third direction, respectively, and can also drive the second product to rotate around the first direction, the second direction, and the third direction, respectively. The third direction can be the same as the Z-axis of the reference Cartesian coordinate system.

[0056] In one embodiment of this disclosure, before executing the product assembly method of this embodiment, the measuring mechanism 110, camera 120, first moving platform 130, and second moving platform 140 can be adjusted according to... Figure 2 The reference Cartesian coordinate system shown in (b) is used to construct the measurement coordinate system X corresponding to the measurement mechanism 110. a Y a Z a ,like Figure 2 As shown in (a); according to as Figure 3 The reference Cartesian coordinate system shown in (b) is used to construct the first moving coordinate system X1Y1Z1 corresponding to the first moving platform 130, as follows: Figure 3 As shown in (a); according to Figure 4 The reference Cartesian coordinate system shown in (b) is used to construct the second translation coordinate system X corresponding to the second moving platform 140. W Y W Z W ,like Figure 4 As shown in (a); construct the visual coordinate system X corresponding to camera 120. b OY b ,like Figure 5 As shown in (a). Wherein, Figure 2 (b) Figure 3 (b) Figure 4 The reference Cartesian coordinate system shown in (b) is the same.

[0057] In this embodiment, the measuring mechanism 110 is constructed based on the measuring coordinate system X. a Y a Z a Measurements were performed to obtain the coordinates corresponding to the measurement coordinate system X. a Y a Z a The measurement results. Camera 120 is based on the visual coordinate system X. b OY b Measurements were performed to obtain the values ​​corresponding to the visual coordinate system X. b OY b The measurement results. The first moving platform 130, based on the first moving coordinate system X1Y1Z1, drives the first product to rotate. The second moving platform 140, based on the second moving coordinate system X... WY W Z W It causes the second product to rotate and translate.

[0058] Furthermore, such as Figure 2 As shown, the X coordinate system is measured. a The axis direction is the same as the X-axis direction of the reference Cartesian coordinate system, and the Y-axis of the measurement coordinate system is... a The axis direction is the same as the Y-axis direction of the reference Cartesian coordinate system, and the Z-axis of the measurement coordinate system is the same. a The axis direction is the same as the Z-axis direction of the reference Cartesian coordinate system; such as Figure 3 As shown, the X1 axis of the first translation coordinate system is in the same direction as the X-axis of the reference Cartesian coordinate system, and the Y1 axis of the first translation coordinate system is in the same direction as the Y-axis of the reference Cartesian coordinate system; as Figure 4 As shown, the X coordinate of the second translational coordinate system W The X-axis direction is the same as the X-axis direction of the reference Cartesian coordinate system, and the Y-axis direction of the second moving coordinate system is... W The direction of the axis is the same as the Y-axis direction of the reference Cartesian coordinate system, and the Z-axis of the second moving coordinate system is... W The axis direction is the same as the Z-axis direction of the reference Cartesian coordinate system.

[0059] Furthermore, such as Figure 5 As shown, Figure 5 The visual coordinate system X shown in (a) b OY b It can be with Figure 5 X in the second moving coordinate system shown in (b) W OY W Affine transformation of the plane yields the visual coordinate system X. b OY b X in the second translational coordinate system W OY W The transformation matrix between planes, used for subsequent transformation of the visual coordinate system X. b OY b Convert the pixel data below to X W OY W Distance data in the plane.

[0060] In one embodiment, if the constructed X a OY a There is a non-negligible angle between the plane and the XOY plane of the reference Cartesian coordinate system. A rotation matrix can be used to adjust the X... a OY a The coordinate values ​​of the plane are transformed to the XOY plane. If the X... a OY a If the straightness of a single-axis robot in a planar plane does not meet the requirements, then it is necessary to address the issue in the corresponding Z-axis direction. athe measurement results of the first product.

[0061] In one embodiment, the first mobile platform 130 can be used to coarsely adjust the pitch angle of the first product, and the first mobile platform 130 can have a relatively large accuracy.

[0062] In one embodiment, before the second mobile platform 140 is used, the work coordinate system X W Y W Z W and the tool coordinate system X T Y T Z T , the work coordinate system X W Y W Z W and the tool coordinate system X T Y T Z T are associated, so that when the 6-axis second mobile platform rotates and translates, the rotation and translation center is close to the center of the second product, so that the position parameters fed back by the 6-axis second mobile platform are consistent with the parameters adjusted by the algorithm. In addition, the directions of the coordinate systems X W Y W Z W are completely parallel to the reference Cartesian coordinate system XYZ.

[0063] Figure 6 is a flowchart of a product assembly method according to an embodiment of the present disclosure.

[0064] As shown in Figure 6 , the product assembly method can include the following steps S6100-S6500:

[0065] Step S6100, control the first mobile platform to rotate the first product, so that the first included angle between the first product and the reference plane is less than or equal to the first threshold value.

[0066] The first product of the present embodiment can be a waveguide sheet.

[0067] In the present embodiment, the first threshold value can be set in advance according to the application scenario or specific requirements. For example, the first threshold value can be 1 degree.

[0068] In one embodiment of the present disclosure, controlling the first mobile platform to rotate the first product so that the first included angle between the first product and the reference plane is less than or equal to the first threshold value can include: controlling the camera to take a picture of the first product, determining the first included angle between the first product and the reference plane according to the second image obtained by taking the picture; and then controlling the first mobile platform to rotate the first product according to the first included angle, so that the first included angle is less than or equal to the first threshold value.

[0069] In the embodiment, the camera can be a 3D camera.

[0070] In another embodiment of the present disclosure, the step of controlling the first mobile platform to rotate the first product such that the first included angle between the first product and the reference plane is less than or equal to the first threshold value can include: measuring first coordinates of at least three first mark points in the first product in a set measurement coordinate system; wherein the first coordinate axis and the second coordinate axis of the measurement coordinate system are both parallel to the reference plane; and controlling the first mobile platform to rotate the first product according to the first coordinates such that the first included angle is less than or equal to the first threshold value.

[0071] In the embodiment, the step of measuring the first coordinates of the first mark points in the first product in the measurement coordinate system can include: controlling the measurement mechanism to move above the first mark points of the first product; and measuring the first coordinates of the first mark points in the first product in the measurement coordinate system.

[0072] In the embodiment, the first coordinates of the at least three first mark points in the measurement coordinate system can be used to obtain a rotation angle of the first product in the first direction relative to the reference plane and a rotation angle of the first product in the second direction relative to the reference plane.

[0073] According to the rotation angle of the first product in the first direction relative to the reference plane and the rotation angle of the first product in the second direction relative to the reference plane, the first mobile platform can be controlled to rotate the first product such that the first included angle is less than or equal to the first threshold value.

[0074] For example, the rotation angle of the first product in the first direction relative to the reference plane is A1, and the rotation angle of the first product in the second direction relative to the reference plane is A2. According to the rotation angle of the first product in the first direction relative to the reference plane and the rotation angle of the first product in the second direction relative to the reference plane, the first mobile platform can be controlled to rotate the first product around the first direction by -A1 and around the second direction by -A2, such that the first included angle is less than or equal to the first threshold value.

[0075] Further, in order to ensure that the first included angle is less than or equal to the first threshold value, the steps of measuring the first coordinates of the at least three first mark points in the first product in the set measurement coordinate system and controlling the first mobile platform to rotate the first product according to the first coordinates can be performed multiple times.

[0076] Still further, during the measurement of the first coordinates of the at least three first mark points in the first product in the set measurement coordinate system, the camera can be controlled to take a picture of the first product, and the measurement mechanism can be guided to move above the first mark points of the first product according to the image obtained by the picture to measure the first coordinates of the first mark points in the first product in the measurement coordinate system.

[0077] In step S6200, the second mobile platform is controlled to rotate the second product, so that a second included angle between the second product and the first product is less than or equal to a second threshold.

[0078] The second product in this embodiment can be an optical machine to be assembled with the waveguide sheet.

[0079] In this embodiment, the second threshold can be set in advance according to application scenarios or specific requirements. For example, the second threshold can be 1 degree.

[0080] In one embodiment of the present disclosure, the second mobile platform is controlled to rotate the second product, so that a second included angle between the second product and the first product is less than or equal to a second threshold, which can include: controlling the camera to take a picture of the first product and the second product, determining the second included angle between the second product and the first product according to a third image obtained by the taking, and then controlling the second mobile platform to rotate the second product according to the second included angle, so that the second included angle is less than or equal to the second threshold.

[0081] In another embodiment of the present disclosure, the second mobile platform is controlled to rotate the second product, so that a second included angle between the second product and the first product is less than or equal to a second threshold, which can include: measuring second coordinates of at least three first mark points in the first product in a measurement coordinate system; measuring third coordinates of at least three second mark points in the second product in the measurement coordinate system; and controlling the second mobile platform to rotate the second product according to the second coordinates and the third coordinates, so that the second included angle is less than or equal to the second threshold.

[0082] In this embodiment, the measurement mechanism can be controlled to move above the first mark point of the first product to measure the second coordinates of the first mark point of the first product in the measurement coordinate system, and the measurement mechanism can be controlled to move above the second mark point of the second product to measure the third coordinates of the second mark point of the second product in the measurement coordinate system.

[0083] In this embodiment, according to the second coordinates of the at least three first mark points in the measurement coordinate system and the third coordinates of the at least three second mark points in the measurement coordinate system, a rotation angle of the second product relative to the first product in a first direction and a rotation angle of the second product relative to the first product in a second direction can be obtained.

[0084] According to the rotation angle of the second product relative to the first product in the first direction and the rotation angle of the second product relative to the first product in the second direction, the second mobile platform is controlled to rotate the second product, so that the second included angle is less than or equal to the second threshold.

[0085] For example, the rotation angle of the second product relative to the first product in the first direction is A3, and the rotation angle of the second product relative to the first product in the second direction is A4. According to the rotation angle of the second product relative to the first product in the first direction and the rotation angle of the second product relative to the first product in the second direction, the second mobile platform is controlled to drive the second product to rotate, which can be to control the second mobile platform to drive the second product to rotate around the first direction by -A3, and control the mobile platform to drive the second product to rotate around the second direction by -A4, so as to make the second included angle less than or equal to the second threshold.

[0086] Further, in order to ensure that the second included angle is less than or equal to the second threshold, the steps of measuring the second coordinates of the at least three first mark points in the first product in the measurement coordinate system, and measuring the third coordinates of the at least three second mark points in the second product in the measurement coordinate system, can be performed multiple times.

[0087] Further, in the process of measuring the second coordinates of the at least three first mark points in the first product in the measurement coordinate system, the camera can be controlled to take a picture of the first product, and the image obtained by the picture is used to guide the measurement mechanism to move above the first mark point of the first product, so as to measure the second coordinates of the first mark point of the first product in the measurement coordinate system. In the process of measuring the third coordinates of the at least three second mark points in the second product in the measurement coordinate system, the camera can be controlled to take a picture of the second product, and the image obtained by the picture is used to guide the measurement mechanism to move above the second mark point of the second product, so as to measure the third coordinates of the second mark point of the second product in the measurement coordinate system.

[0088] In an embodiment of the method, before step S6200 is performed, the method can include: controlling the second mobile platform to drive the second product to move below the first product.

[0089] In this way, the posture of the second product can be adjusted subsequently to assemble the first product and the second product.

[0090] In an embodiment of the method, before step S6200 is performed, the method can include: controlling the second mobile platform to drive the second product to rotate, so that a third included angle between the second product and the reference plane is less than or equal to a third threshold.

[0091] In this embodiment, the third threshold can be set in advance according to application scenarios or specific requirements. For example, the third threshold can be 1 degree.

[0092] In one embodiment of the present disclosure, the control of the second mobile platform to rotate the second product such that the third included angle between the second product and the reference plane is less than or equal to the third threshold value can include: controlling the camera to take a photograph of the second product, determining the third included angle between the second product and the reference plane according to a fourth image obtained by the photographing, and then controlling the second mobile platform to rotate the second product according to the third included angle such that the third included angle is less than or equal to the third threshold value.

[0093] In another embodiment of the present disclosure, the control of the second mobile platform to rotate the second product such that the third included angle between the second product and the reference plane is less than or equal to the third threshold value can include: measuring fourth coordinates of at least three second mark points in the second product in a set measurement coordinate system; wherein the fourth coordinate axis and the second coordinate axis of the measurement coordinate system are both parallel to the reference plane; and controlling the second mobile platform to rotate the second product according to the fourth coordinates such that the third included angle is less than or equal to the third threshold value.

[0094] In the present embodiment, the control of the measurement mechanism to move above the second mark point of the second product can be to measure the fourth coordinates of the second mark point of the second product in the measurement coordinate system.

[0095] In the present embodiment, the fourth coordinates of the at least three second mark points in the measurement coordinate system can obtain the rotation angle of the second product in the first direction relative to the reference plane and the rotation angle of the second product in the second direction relative to the reference plane.

[0096] According to the rotation angle of the second product in the first direction relative to the reference plane and the rotation angle of the second product in the second direction relative to the reference plane, the control of the second mobile platform to rotate the second product can be to make the third included angle less than or equal to the third threshold value.

[0097] For example, the rotation angle of the second product in the first direction relative to the reference plane is A5, and the rotation angle of the second product in the second direction relative to the reference plane is A6. According to the rotation angle of the second product in the first direction relative to the reference plane and the rotation angle of the second product in the second direction relative to the reference plane, the control of the second mobile platform to rotate the second product can be to control the second mobile platform to rotate the second product around the first direction by -A5 and control the mobile platform to rotate the second product around the second direction by -A6, so as to make the third included angle less than or equal to the third threshold value.

[0098] In step S6300, the camera is controlled to take a photograph of the first product and the second product to obtain a first image.

[0099] In the present embodiment, the first product and the second product are both in the field of view of the camera, and the passive AA mode can be used to adjust the second product.

[0100] In step S6400, the second mobile platform is controlled to move the second product in the reference plane according to the first image, so that the second product is aligned with the first product.

[0101] In this embodiment, by controlling the second mobile platform to move the second product in the reference plane according to the first image, the first product and the second product can be aligned in the first image, that is, the position area of the first product to be assembled in the second product coincides with the first product in the first image, and the posture of the second product matches the posture of the first product, so that the assembly can be performed.

[0102] In one embodiment of the present disclosure, the second mobile platform is controlled to move the second product in the reference plane according to the first image, so that the second product is aligned with the first product, including: determining the rotation angle deviation and / or distance deviation of the first product and the second product in the reference plane according to the first image; controlling the second mobile platform to rotate the second product in the reference plane according to the rotation angle deviation, and / or controlling the second mobile platform to move the second product in the reference plane according to the distance deviation, so that the second product is aligned with the first product.

[0103] In this embodiment, the rotation angle deviation and / or distance deviation of the first product and the second product in the reference plane can be obtained by respectively performing transformation processing on the pixel angle and pixel distance of the position area of the first product to be assembled in the second product in the first image according to the transformation matrix obtained by performing the affine transformation in advance.

[0104] In this embodiment, the second mobile platform is controlled to rotate the second product in the reference plane according to the rotation angle deviation, which can reduce the rotation angle deviation of the first product and the second product in the reference plane; the second mobile platform is controlled to move the second product in the reference plane according to the distance deviation, which can reduce the distance deviation of the first product and the second product in the reference plane.

[0105] In one embodiment of the present disclosure, the second mobile platform is controlled to rotate the second product in the reference plane according to the rotation angle deviation, including: controlling the second mobile platform to rotate the second product in the reference plane around the direction perpendicular to the reference plane according to the rotation angle deviation, so that the rotation angle deviation of the second product and the first product in the reference plane is less than or equal to a fourth threshold value.

[0106] In one embodiment of the present disclosure, the second mobile platform is controlled to move the second product in the reference plane according to the distance deviation, including: controlling the second mobile platform to move the second product in the reference plane according to the distance deviation, so that the distance deviation of the second product and the first product in the reference plane is less than or equal to a fifth threshold value.

[0107] By the embodiment, the second product can be aligned with the first product by controlling the second mobile platform to drive the second product to translate and / or rotate in the reference plane, so as to facilitate the subsequent assembly of the first product and the second product.

[0108] In step S6500, the second mobile platform is controlled to drive the second product to move towards the first product, so as to assemble the first product and the second product.

[0109] In one embodiment of the present disclosure, the control of the second mobile platform to drive the second product to move towards the first product can be control of the second mobile platform to drive the second product to translate in a direction perpendicular to the reference plane and close to the first product.

[0110] By the embodiment of the present disclosure, the first mobile platform is controlled to drive the first product to rotate so that the first included angle between the first product and the reference plane is less than or equal to the first threshold value; the second mobile platform is controlled to drive the second product to rotate so that the second included angle between the second product and the first product is less than or equal to the second threshold value; the second mobile platform is controlled to drive the second product to translate and / or rotate in the reference plane according to the first image obtained by the camera shooting the first product and the second product, so that the second product is aligned with the first product; and the second mobile platform is controlled to drive the second product to move towards the first product, so as to assemble the first product and the second product, which can realize the automatic assembly of the first product and the second product of any shape, and can improve the assembly precision of the first product and the second product.

[0111] Figure 7 is a flowchart of an example of a product assembly method according to an embodiment of the present disclosure.

[0112] As shown in Figure 7 , the product assembly method can include the following steps:

[0113] In step S7001, first coordinates of at least three first mark points in a first product in a set measurement coordinate system are measured.

[0114] In step S7002, the first mobile platform is controlled to drive the first product to rotate according to the first coordinates, so that the first included angle is less than or equal to the first threshold value.

[0115] In step S7003, fourth coordinates of at least three second mark points in a second product in a set measurement coordinate system are measured.

[0116] In step S7004, the second mobile platform is controlled to drive the second product to rotate according to the fourth coordinates, so that the third included angle is less than or equal to the third threshold value.

[0117] In step S7005, the second mobile platform is controlled to drive the second product to move below the first product.

[0118] Step S7006, control the camera to capture the first product and the second product to obtain a first image.

[0119] Step S7007, determine a rotation angle deviation of the first product and the second product in the reference plane according to the first image.

[0120] Step S7008, control the second mobile platform to drive the second product to rotate in the reference plane around a direction perpendicular to the reference plane according to the rotation angle deviation, so that the rotation angle deviation of the second product and the first product in the reference plane is less than or equal to a fourth threshold value.

[0121] Step S7009, measure second coordinates of the at least three first mark points in the first product in a measurement coordinate system.

[0122] Step S7010, measure third coordinates of the at least three second mark points in the second product in the measurement coordinate system.

[0123] Step S7011, control the second mobile platform to drive the second product to rotate according to the second coordinates and the third coordinates, so that a second included angle is less than or equal to a second threshold value.

[0124] Step S7012, control the camera to capture the first product and the second product to obtain a first image.

[0125] Step S7013, determine a distance deviation of the first product and the second product in the reference plane according to the first image.

[0126] Step S7014, control the second mobile platform to drive the second product to translate in the reference plane according to the distance deviation, so that the distance deviation of the second product and the first product in the reference plane is less than or equal to a fifth threshold value.

[0127] Step S7015, control the second mobile platform to drive the second product to move to the first product, which can be to control the second mobile platform to drive the second product to translate in a direction perpendicular to the reference plane and close to the first product, so as to assemble the first product and the second product.

[0128] The embodiment also provides a product assembling device 8000, as shown in the figure, the product assembling device 8000 comprises a first control module 8100, a second control module 8200, a third control module 8300, a fourth control module 8400 and a fifth control module 8500. Figure 8

[0129] The first control module 8100 is used for controlling the first mobile platform to drive the first product to rotate, so that a first included angle between the first product and the reference plane is less than or equal to a first threshold value. ​

[0130] The second control module 8200 is configured to control the second mobile platform to rotate the second product, so that a second included angle between the first product and the second product is less than or equal to a second threshold.

[0131] The third control module 8300 is configured to control the camera to capture the first product and the second product to obtain a first image.

[0132] The fourth control module 8400 is configured to control the second mobile platform to translate and / or rotate the second product in the reference plane according to the first image, so that the first product and the second product are aligned.

[0133] The fifth control module 8500 is configured to control the second mobile platform to move the second product to the first product to assemble the first product and the second product.

[0134] In an embodiment of the present disclosure, the first control module 8100 is specifically configured to:

[0135] measure first coordinates of at least three first mark points in the first product in a set measurement coordinate system; wherein a first coordinate axis and a second coordinate axis of the measurement coordinate system are both parallel to the reference plane;

[0136] control the first mobile platform to rotate the first product according to the first coordinates, so that the first included angle is less than or equal to a first threshold.

[0137] In an embodiment of the present disclosure, the second control module 8200 is specifically configured to:

[0138] measure second coordinates of at least three first mark points in the first product in a set measurement coordinate system; wherein a first coordinate axis and a second coordinate axis of the measurement coordinate system are both parallel to the reference plane;

[0139] measure third coordinates of at least three second mark points in the second product in the measurement coordinate system;

[0140] control the second mobile platform to rotate the second product according to the second coordinates and the third coordinates, so that the second included angle is less than or equal to a second threshold.

[0141] In an embodiment of the present disclosure, the product assembling device 8000 further comprises:

[0142] a sixth control module configured to control the second mobile platform to move the second product to below the first product.

[0143] In an embodiment of the present disclosure, the product assembling device 8000 further comprises:

[0144] a measuring module configured to measure fourth coordinates of at least three second identification points in the second product in a set measuring coordinate system, wherein the first coordinate axis and the second coordinate axis of the measuring coordinate system are both parallel to the reference plane;

[0145] a seventh control module configured to control the second moving platform to rotate the second product according to the fourth coordinates, so that a third included angle between the second product and the reference plane is less than or equal to a third threshold value.

[0146] In an embodiment of the present disclosure, the fourth control module 8400 is specifically configured to:

[0147] determine a rotation angle deviation and / or a distance deviation of the first product and the second product in the reference plane according to the first image;

[0148] control the second moving platform to rotate the second product in the reference plane according to the rotation angle deviation, and / or control the second moving platform to translate the second product in the reference plane according to the distance deviation, so that the second product is aligned with the first product.

[0149] In an embodiment of the present disclosure, the control of the second moving platform to rotate the second product in the reference plane according to the rotation angle deviation comprises:

[0150] controlling the second moving platform to rotate the second product in the reference plane around a direction perpendicular to the reference plane according to the rotation angle deviation, so that the rotation angle deviation of the second product and the first product in the reference plane is less than or equal to a fourth threshold value.

[0151] In an embodiment of the present disclosure, the control of the second moving platform to translate the second product in the reference plane according to the distance deviation comprises:

[0152] controlling the second moving platform to translate the second product in the reference plane according to the distance deviation, so that the distance deviation of the second product and the first product in the reference plane is less than or equal to a fifth threshold value.

[0153] The embodiment also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method according to any embodiment of the present disclosure.

[0154] The above-described embodiments are mainly focused on the differences from other embodiments, but those skilled in the art should understand that the above-described embodiments can be used alone or in combination according to the needs.

[0155] The embodiments in the present specification are described in progressive manner, and the same or similar parts between the embodiments can be mutually referred to, and each embodiment mainly focuses on the differences from other embodiments, but those skilled in the art should understand that the above-described embodiments can be used alone or in combination according to the needs. In addition, for the device embodiments, since they are corresponding to the method embodiments, they are described more simply, and the related parts can be referred to the description of the corresponding parts of the method embodiments. The system embodiments described above are only schematic, and the modules described as separate components can or can not be physically separate.

[0156] The present application can be a device, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0157] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a ROM, a magnetic track storage and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0158] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0159] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing / processing device, partly on the user's computing / processing device, as a stand-alone software package, partly on the user's computing / processing device and partly on a remote computing / processing device or entirely on the remote computing / processing device or server. In the latter scenario, the remote computing / processing device can be connected to the user's computing / processing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing / processing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0160] The computer readable program instructions can also be loaded onto a computing / processing device, other programmable data processing apparatus, or other device to cause a series of operations to be performed on the computing / processing device, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computing / processing device, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0161] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all. The computer readable storage medium can also have instructions stored thereon or therein which may

[0162] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0163] The flow diagrams and block diagrams in the accompanying drawings show archi tectures, functional and operational architectures of possible implementations of systems, methods, and computer program products according to the present disclosure. In this regard, each block in the flow diagrams and block diagrams can represent a module, a segment, or a portion of instructions, which comprises one or more executable instructions for implementing the specified logical functions (acts). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in some cases, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0164] Having described various embodiments of the application, it is to be understood that the above description is meant to be illustrative only and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art, without departing from the scope and spirit of the described embodiments. The selection of terms to be used herein is intended to best explain the principles of the embodiments, the practical application, or technical improvement over technology found in the market, or to enable other of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the application is defined by the claims appended hereto.

Claims

1. A product assembly method, characterized in that, include: Control the first moving platform to drive the first product to rotate, so that the first included angle between the first product and the reference plane is less than or equal to the first threshold. The first product is a waveguide sheet; Control the second mobile platform to move the second product below the first product; The second moving platform is controlled to rotate the second product, such that the second included angle between the second product and the first product is less than or equal to a second threshold; the second product is an optical engine. The camera is controlled to capture images of the first product and the second product, thereby obtaining a first image; Based on the first image, control the second mobile platform to move the second product in the reference plane and / or rotate it so that the second product is aligned with the first product. The second mobile platform is controlled to move the second product toward the first product, so as to assemble the first product and the second product. The control of the first moving platform to rotate the first product, such that the first angle between the first product and the reference plane is less than or equal to a first threshold, includes: Measure the first coordinates of at least three first identification points in the first product in a set measurement coordinate system; wherein the first coordinate axis and the second coordinate axis of the measurement coordinate system are both parallel to the reference plane; The first moving platform is controlled to rotate the first product according to the first coordinate, so that the first included angle is less than or equal to the first threshold. The control of the second mobile platform to rotate the second product, such that the second included angle between the second product and the first product is less than or equal to a second threshold, includes: Measure the second coordinates of at least three first identification points in the first product in a set measurement coordinate system; wherein the first and second coordinate axes of the measurement coordinate system are both parallel to the reference plane; Measure the third coordinates of at least three second identification points in the second product in the measurement coordinate system; Based on the second coordinate and the third coordinate, the second moving platform is controlled to rotate the second product, such that the second included angle is less than or equal to the second threshold.

2. The method according to claim 1, characterized in that, Before the method of controlling the second mobile platform to rotate the second product, such that the second included angle between the second product and the first product is less than or equal to a second threshold, the method further includes: Measure the fourth coordinates of at least three second identification points in the second product in a set measurement coordinate system; wherein the first and second coordinate axes of the measurement coordinate system are both parallel to the reference plane; The second moving platform is controlled according to the fourth coordinate to rotate the second product, so that the third included angle between the second product and the reference plane is less than or equal to the third threshold.

3. The method according to claim 1, characterized in that, The step of controlling the second moving platform to translate and / or rotate the second product within the reference plane based on the first image, so that the second product is aligned with the first product, includes: Based on the first image, determine the rotational angle deviation and / or distance deviation of the first product and the second product in the reference plane; The second moving platform is controlled to rotate the second product within the reference plane according to the rotation angle deviation, and / or the second moving platform is controlled to translate the second product within the reference plane according to the distance deviation, so that the second product is aligned with the first product.

4. The method according to claim 3, characterized in that, The step of controlling the second moving platform to rotate the second product within the reference plane based on the rotation angle deviation includes: Based on the rotation angle deviation, the second moving platform is controlled to drive the second product to rotate in the reference plane and around a direction perpendicular to the reference plane, so that the rotation angle deviation between the second product and the first product in the reference plane is less than or equal to the fourth threshold.

5. The method according to claim 3, characterized in that, Controlling the second moving platform to move the second product within the reference plane based on the distance deviation includes: Based on the distance deviation, the second moving platform is controlled to move the second product in the reference plane, so that the distance deviation between the second product and the first product in the reference plane is less than or equal to the fifth threshold.

6. A product assembly apparatus, characterized in that, include: A first control module is used to control a first moving platform to rotate a first product, such that the first angle between the first product and the reference plane is less than or equal to a first threshold; the first product is a waveguide sheet. The sixth control module is used to control the second mobile platform to move the second product to below the first product; The second control module is used to control the second mobile platform to rotate the second product, such that the second included angle between the second product and the first product is less than or equal to a second threshold; the second product is an optical engine. The third control module is used to control the camera to take pictures of the first product and the second product to obtain the first image; The fourth control module is used to control the second moving platform to translate and / or rotate the second product in the reference plane according to the first image, so that the second product is aligned with the first product; The fifth control module is used to control the second mobile platform to move the second product toward the first product, so as to assemble the first product and the second product. The first control module is specifically used to: measure the first coordinates of at least three first identification points in the first product in a set measurement coordinate system; wherein the first coordinate axis and the second coordinate axis of the measurement coordinate system are both parallel to the reference plane; and control the first moving platform to rotate the first product according to the first coordinates, so that the first included angle is less than or equal to a first threshold. The second control module is specifically used to: measure the second coordinates of at least three first identification points in the first product in a set measurement coordinate system; wherein the first coordinate axis and the second coordinate axis of the measurement coordinate system are both parallel to the reference plane; measure the third coordinates of at least three second identification points in the second product in the measurement coordinate system; and control the second moving platform to rotate the second product according to the second coordinates and the third coordinates, so that the second included angle is less than or equal to the second threshold.

7. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 5.

Images