Component assembly method, terminal device, and computer-readable storage medium

Abstract

The present application discloses a component assembly method, terminal device, and computer-readable storage medium. The method includes: determining a first pose of a first component to be assembled and a bonding area position corresponding to the first component to be assembled based on a first image captured by a first image acquisition device; determining a second pose of a second component to be assembled based on a second image captured by a second image acquisition device; planning a motion path for a second robotic arm and a third robotic arm based on the first pose, the bonding area position, and the second pose; controlling the second robotic arm and the third robotic arm to move according to the motion path, and controlling the third robotic arm to perform an adhesive injection action after the movement is completed. This method solves the problem of low efficiency in component assembly and achieves the effect of improving component assembly efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of construction robots, and in particular to a component assembly method, a terminal device, and a computer-readable storage medium. Background Art

[0002] Currently, the assembly of building components based on 3D vision usually uses a 3D vision camera component to capture the pose images of the assembled components, adjusts the direction of the components to be assembled in real time based on the pose images, and then places the components to be assembled on the assembled components according to the assembly commands of the robotic arm.

[0003] When adjusting components for assembly, adhesive must be manually applied to the surface areas where bonding is required. Furthermore, due to component support limitations, assembly must be performed from bottom to top on a horizontal platform, following the robotic arm's planned path. Component shapes are typically limited to regular rectangles. Assembling irregularly shaped components requires repeated offline programming to adjust the robotic arm's motion path, resulting in inefficient assembly. Summary of the Invention

[0004] The present application provides a component assembly method, terminal device, and computer-readable storage medium to solve the problem of low component assembly efficiency caused by the need to strictly follow the assembly sequence or the need to repeatedly adjust the robotic arm offline during the assembly process. This improves the assembly efficiency of the components.

[0005] An embodiment of the present application provides a component assembly method, the component assembly method comprising:

[0006] Determining a first pose of a first component to be assembled and a position of a bonding area corresponding to the first component to be assembled based on the first image captured by the first image capturing device;

[0007] determining a second posture of a second component to be assembled based on a second image captured by the second image capture device;

[0008] planning motion paths of the second robotic arm and the third robotic arm according to the first posture, the bonding area position, and the second posture;

[0009] The second robotic arm and the third robotic arm are controlled to move according to the motion path, and the third robotic arm is controlled to perform an adhesive injection action after the movement is completed.

[0010] Optionally, the step of determining the first pose of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled based on the first image captured by the first image capture device includes:

[0011] Calculating position information of the first component to be assembled based on a position of the first component to be assembled relative to a visual reference system label in the first image;

[0012] Perform semantic segmentation on the first image and calculate the position of the bonding area.

[0013] Optionally, the step of planning the motion paths of the second robotic arm and the third robotic arm according to the first posture, the bonding area position and the second posture information includes:

[0014] adjusting a theoretical gripping mode of the second robotic arm according to the first posture information, the bonding area position, and the second posture information;

[0015] After determining the theoretical gripping method, the motion path of the second robotic arm is calculated.

[0016] Optionally, after the step of adjusting the theoretical gripping mode of the second robotic arm according to the first posture information, the bonding area position and the second posture information, the method further includes:

[0017] Determining whether the mechanical structure of the second robotic arm is applicable to the theoretical gripping method;

[0018] If not, control the first robotic arm to adjust the posture of the first component to be assembled based on the second posture, and execute the step of determining the first posture of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled based on the first image captured by the first image acquisition device.

[0019] Optionally, after determining the theoretical gripping mode, the step of calculating the motion path of the second robotic arm includes:

[0020] determining whether the second robotic arm can move according to the motion path;

[0021] If not, the first robotic arm is controlled to adjust its position, and the step of determining the first pose of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled is performed based on the first image captured by the first image capture device.

[0022] Optionally, the step of determining the first pose of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled based on the first image captured by the first image capture device includes:

[0023] Calculating the global coordinate positions of the first camera and the first component to be assembled according to the visual machine system label present in the first image;

[0024] Determining whether the first component to be assembled is located in a preset working area;

[0025] If not, control the first robotic arm to move to the preset working area.

[0026] Optionally, the step of determining the second posture of the second component to be assembled based on the second image captured by the second image capture device includes:

[0027] determining the global coordinates of the second robotic arm and a currently gripped shape based on label information of the visual reference system label in the second image;

[0028] The position and posture information of the second component to be assembled is determined according to the global coordinates and the current clamping shape.

[0029] Optionally, the step of controlling the second robotic arm and the third robotic arm to move according to the motion path, and controlling the third robotic arm to perform an adhesive injection action after the movement is completed includes:

[0030] Controlling the second robotic arm to move to a target position according to a motion path corresponding to the second robotic arm, and engaging with the first component to be assembled;

[0031] After the third robotic arm is controlled to move to a designated position according to a movement path corresponding to the third robotic arm, adhesive is added.

[0032] In addition, to achieve the above objectives, an embodiment of the present invention also provides a terminal device, including a memory, a processor, and a component assembly program stored in the memory and runnable on the processor. When the processor executes the component assembly program, the method described above is implemented.

[0033] In addition, to achieve the above-mentioned purpose, an embodiment of the present invention further provides a computer-readable storage medium, on which a component assembly program is stored. When the component assembly program is executed by a processor, the method described above is implemented.

[0034] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0035] The component assembly system includes a first robotic arm, a second robotic arm, and a third robotic arm. The first robotic arm is provided with a first image acquisition device, the second robotic arm is provided with a second image acquisition device, and the third robotic arm is used for injecting adhesive. Based on the first image captured by the first image acquisition device, the first pose of the first component to be assembled and the pose of the bonding area corresponding to the first component to be assembled are determined. Based on the second image captured by the second image acquisition device, the second pose of the second component to be assembled is determined. Based on the first pose, the position of the bonding area, and the second pose, the movement paths of the second and third robotic arms are planned, and the second and third robotic arms are controlled to move according to the planned movement paths. After movement to the desired position, the third robotic arm is controlled to perform the adhesive injection action to complete the assembly of the component. Since the assembly is completed by controlling the robotic arms, it is not necessary to strictly follow the assembly sequence. In addition, the component assembly system can automatically adjust the pose of the components, eliminating the need for manual adjustment after shutdown, which can greatly improve the efficiency of component assembly. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the first embodiment of the component assembly method of the present application;

[0037] Figure 2 This is a schematic diagram of the collaborative assembly of special-shaped components by multiple robotic arms;

[0038] Figure 3 This is a schematic diagram of the local operation of the multi-manipulator head;

[0039] Figure 4 This is a flow chart of Example 2 of the component assembly method of this application;

[0040] Figure 5 This is a schematic diagram of the terminal structure of the hardware operating environment involved in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to solve the problem of low efficiency during component assembly, the present application provides a component assembly method for controlling a component assembly system, wherein the component assembly system includes a first robotic arm, a second robotic arm, and a third robotic arm. A first image acquisition device is provided on the first robotic arm, a second image acquisition device is provided on the second robotic arm, and the third robotic arm is used to inject adhesive. The first image acquisition device captures a first image to determine the first posture of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled. The second image acquisition device captures a second image to determine the second posture of the second component to be assembled. Based on the first posture, the position of the bonding area, and the second posture, the movement paths of the second robotic arm and the third robotic arm are planned. The second robotic arm and the third robotic arm are controlled to move according to the movement path, and the third robotic arm is controlled to perform the adhesive injection action after the movement is completed. The robotic arms can autonomously complete the assembly of components, thereby improving the efficiency of component assembly.

[0042] To better understand the above technical solutions, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0043] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0044] Example 1

[0045] In this embodiment, a component assembly method is provided.

[0046] Reference Figure 1 The component assembly method of this embodiment includes the following steps:

[0047] Step S100: determining a first pose of a first component to be assembled and a position of a bonding area corresponding to the first component to be assembled based on a first image captured by the first image capturing device;

[0048] In this embodiment, the component assembly method is used to control the component assembly system. Figure 2 , the component assembly system is provided with a first robotic arm 1, a second robotic arm 2, and a third robotic arm 3. The first robotic arm is provided with a first camera 4, and the first camera 4 is used to shoot the component clamped by the first robotic arm 1 to generate a first image. The second robotic arm is provided with a second camera 5, and the second camera 5 is used to shoot the component clamped by the second robotic arm 2 to generate a second image. The third robotic arm 3 is used to inject adhesive. There is also a computer control unit 6 on the control component assembly system, which is used to control the component assembly system to be turned on or off, and the robotic arm and the camera are both connected to the computer control unit 6 to establish communication between the devices. 7 is a visual reference system label. When the first camera 4 shoots the first image, it will shoot the visual reference system label and calculate the coordinate position of the first component to be assembled based on the label.

[0049] As an optional implementation, after acquiring the first image, the pose information of the first component to be assembled is calculated based on the position of the first component to be assembled relative to the visual reference system label in the first image, and then the first image is semantically segmented to calculate the position of the bonding area.

[0050] Exemplarily, after the first camera fixed on the first robotic arm acquires the first image, it sends the first image to the computer control unit. The computer control unit uses the fully convolutional neural network technology to perform semantic segmentation on the first image to identify the surface to which the adhesive needs to be added, that is, the location of the bonding area. In addition, the first image contains a visual reference system tag, that is, the AprilTag tag. The computer control unit can determine the precise position of the first component to be assembled in the global coordinate system based on the tag information and the relative position of the first component to be assembled to the first camera. Since the first camera is fixedly mounted on the first robotic arm, the position of the clamping part from the first camera to the first robotic arm is fixed and known.

[0051] As another optional implementation, after acquiring the first image, the global coordinate positions of the first camera and the first component to be assembled are calculated based on the visual system label present in the first image, and it is determined whether the first component to be assembled is located in a preset working area. If not, the first robotic arm is controlled to move to the preset working area.

[0052] For example, the first, second, and third robotic arms 1, 2, and 3 are positioned to ensure overlapping working areas between the robotic arms and to allow for adequate space to prevent collisions during operation. When the robotic arms are in their working areas, the first camera 4 on the first robotic arm 1 and the second camera 5 on the second robotic arm 2 can capture the label area of ​​the visual reference system label 7.

[0053] Step S200: determining a second posture of a second component to be assembled based on a second image captured by the second image capture device;

[0054] In this embodiment, a second image acquisition device is provided on the second robotic arm and is used to capture the component held by the second robotic arm, i.e., the second component to be assembled. The captured component image also includes an AprilTag, and the position and orientation of the second component to be assembled can be determined using the AprilTag.

[0055] As an optional implementation, after collecting the second image, the global coordinates of the second robotic arm and the current clamped shape of the second component to be assembled are determined based on the label information of the visual reference system label in the second image, and the posture information of the second component to be assembled is determined based on the global coordinates and the current shape of the second component to be assembled.

[0056] For example, after acquiring the second image, the AprilTag information in the second image is obtained, and the precise position of the second irregularly shaped component to be assembled in the global coordinate system is determined based on the relative position of the captured second component to be assembled to the second camera. The computer control unit integrates and analyzes the pose information and geometric shape information of the second component to be assembled.

[0057] Step S300: planning the motion paths of the second robotic arm and the third robotic arm according to the first posture, the bonding area position, and the second posture;

[0058] Step S400: controlling the second robotic arm and the third robotic arm to move according to the movement path, and controlling the third robotic arm to perform an adhesive injection action after the movement is completed.

[0059] In this embodiment, after determining the first and second poses of the components to be assembled, the theoretical motion paths of the robotic arms are calculated. After the second and third robotic arms are moved into position according to the theoretical motion paths, the third robotic arm performs adhesive injection, completing the bonding of the first and second components to be assembled.

[0060] As an optional implementation, refer to Figure 3 The first robot arm gripper 8 grips the first component to be assembled 11, and the second robot arm gripper 9 grips the second component to be assembled 12. The adhesive extrusion head 10 of the third robot arm extrude adhesive.

[0061] For example, after the second and third robotic arms are controlled to move to their respective theoretical motion paths, the adhesive extruder 10 of the third robotic arm squeezes the adhesive onto the bonding area of ​​the first component to be assembled. The second robotic arm is then controlled to clamp the second component to be assembled and bond it to the first component to be assembled.

[0062] As another optional implementation, for special-shaped components, the position and posture of the special-shaped components to be assembled can be obtained through a computer and the AprilTag visual reference system, and a robotic arm can be used to autonomously adjust the position and posture of the special-shaped components and add adhesive, thereby realizing autonomous path planning of the robotic arm and completing the assembly of the special-shaped components.

[0063] Exemplarily, a unified global three-dimensional coordinate system is established for multiple robotic arms in advance. After acquiring the image of the special-shaped component to be assembled, the relative position between the first special-shaped component to be assembled and the second special-shaped component to be assembled, as well as the respective postures of the first special-shaped component to be assembled and the second special-shaped component to be assembled are determined based on the AprilTag visual reference system. The posture includes information such as the position and shape of the special-shaped component that can reflect the overall structure of the special-shaped component. Since the shape of the special-shaped component is irregular, it is necessary to determine the first posture of the first special-shaped component to be assembled clamped by the first robotic arm and the second posture of the second special-shaped component to be assembled clamped by the second robotic arm. After determining the postures of the two, it is determined whether the first posture and the second posture can be fitted together. If not, the component clamping method of the second robotic arm is adjusted until the unclamped surface of the first special-shaped component and the unclamped surface of the second special-shaped component can be fitted together.

[0064] In this embodiment, the component assembly system includes a first robotic arm, a second robotic arm, and a third robotic arm. The first robotic arm is equipped with a first image acquisition device, the second robotic arm is equipped with a second image acquisition device, and the third robotic arm is used to inject adhesive. The system also includes a computer control unit to facilitate information exchange between the first, second, and third robotic arms. Based on a first image captured by the first image acquisition device, a first pose of the first component to be assembled and the location of the corresponding adhesive area of ​​the first component to be assembled are determined. Based on a second image captured by the second image acquisition device, a second pose of the second component to be assembled is determined. Based on the first pose, second pose, and location of the adhesive area, motion paths of the second and third robotic arms are calculated. After the second and third robotic arms are controlled to move into position according to the corresponding motion paths, the third robotic arm performs adhesive injection, completing component assembly. Because the robotic arms are used to complete assembly, the assembly of building components no longer requires a horizontal platform and a bottom-up path. In addition, the present application uses a visual reference system to locate the relative position between the robotic arm and the component, and utilizes three-dimensional visual recognition and neural network technology to plan the movement path of the robotic arm, so that even when assembling special-shaped components, the bonding of the components can be completed accurately, thereby improving the efficiency and accuracy of component assembly.

[0065] Example 2

[0066] Based on the first embodiment, another embodiment of the present application is proposed, referring to Figure 4 The step of planning the motion paths of the second robotic arm and the third robotic arm according to the first posture, the bonding area position and the second posture information includes:

[0067] Step S310: adjusting the theoretical gripping mode of the second robotic arm according to the first posture information, the bonding area position, and the second posture information;

[0068] Step S320: After determining the theoretical gripping method, calculate the motion path of the second robotic arm.

[0069] In this embodiment, the bonding area of ​​the first component to be assembled is marked with a special mark. By identifying the special mark, the location of the bonding area can be determined. Based on the first pose, the location of the bonding area, and the second pose, the theoretical gripping method of the second robot arm can be calculated. After the theoretical gripping method is determined, the motion path of the second robot arm is calculated.

[0070] As an optional embodiment, after determining the theoretical clamping method of the second robotic arm based on the first posture, the bonding area position and the second posture, it is judged whether the mechanical structure of the second robotic arm is applicable to the theoretical clamping method. If not applicable, the first robotic arm is controlled to adjust the posture of the first component to be assembled based on the second posture, and execute the steps of determining the first posture of the first component to be assembled and the bonding area position corresponding to the first component to be assembled based on the first image captured by the first image acquisition device.

[0071] For example, due to the mechanical structure of the robotic arm, the calculated theoretical gripping method for the second robotic arm may not be applicable to the second robotic arm. Therefore, it is necessary to adjust the gripping method of the first robotic arm and adjust the position of the first component to be assembled. In addition, when the component to be assembled is a special-shaped component, the second robotic arm may not be able to adapt to the theoretical gripping method. After adjusting the position of the first component to be assembled, it is necessary to recalculate the theoretical position of the second component to be assembled until the second robotic arm can adapt to the theoretical gripping method of the second robotic arm, thereby completing the assembly of the components.

[0072] As another optional embodiment, after determining the theoretical gripping method and calculating the motion path of the second robotic arm, it is necessary to determine whether the second robotic arm can move according to the motion path. If the second robotic arm cannot move according to the motion path, the position of the first robotic arm is adjusted, and the steps of capturing the first image, determining the first pose of the first component to be assembled, and the position of the bonding area corresponding to the first component to be assembled are repeated.

[0073] For example, after calculating the motion path of the second robotic arm based on the position of the first component to be assembled, the second robotic arm may be unable to complete the corresponding movement due to its mechanical structure, resulting in assembly failure. The position of the first robotic arm needs to be readjusted, and after the position is adjusted, the image acquisition step is repeated, and this cycle continues until the second robotic arm can operate according to the predetermined motion path.

[0074] In this embodiment, since the first robotic arm can adjust its position and the posture of the component according to actual conditions, the component assembly system can automatically complete the assembly, greatly improving the applicability and efficiency of the assembly system.

[0075] Example 3

[0076] In an embodiment of the present application, a component assembly device is provided.

[0077] Reference Figure 5 , Figure 5 This is a schematic diagram of the terminal structure of the hardware operating environment involved in an embodiment of the present application.

[0078] like Figure 5 As shown, the control terminal may include: a processor 1001, such as a CPU, a network interface 1003, a memory 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The network interface 1003 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1004 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1004 may also be a storage device independent of the aforementioned processor 1001.

[0079] Those skilled in the art will understand that Figure 5 The terminal structure shown in the figure does not constitute a limitation to the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0080] like Figure 5 As shown, the memory 1004 as a computer storage medium may include an operating system, a network communication module, and a component assembly program.

[0081] exist Figure 5 In the hardware structure of the component assembly device shown, the processor 1001 can call the component assembly program stored in the memory 1004 and perform the following operations:

[0082] Determining a first pose of a first component to be assembled and a position of a bonding area corresponding to the first component to be assembled based on the first image captured by the first image capturing device;

[0083] determining a second posture of a second component to be assembled based on a second image captured by the second image capture device;

[0084] planning motion paths of the second robotic arm and the third robotic arm according to the first posture, the bonding area position, and the second posture;

[0085] The second robotic arm and the third robotic arm are controlled to move according to the motion path, and the third robotic arm is controlled to perform an adhesive injection action after the movement is completed.

[0086] Optionally, the processor 1001 may call a component assembly program stored in the memory 1004 and further perform the following operations:

[0087] Calculating position information of the first component to be assembled based on a position of the first component to be assembled relative to a visual reference system label in the first image;

[0088] Perform semantic segmentation on the first image and calculate the position of the bonding area.

[0089] Optionally, the processor 1001 may call a component assembly program stored in the memory 1004 and further perform the following operations:

[0090] adjusting a theoretical gripping mode of the second robotic arm according to the first posture information, the bonding area position, and the second posture information;

[0091] After determining the theoretical gripping method, the motion path of the second robotic arm is calculated.

[0092] Optionally, the processor 1001 may call a component assembly program stored in the memory 1004 and further perform the following operations:

[0093] Determining whether the mechanical structure of the second robotic arm is applicable to the theoretical gripping method;

[0094] If not, control the first robotic arm to adjust the posture of the first component to be assembled based on the second posture, and execute the step of determining the first posture of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled based on the first image captured by the first image acquisition device.

[0095] Optionally, the processor 1001 may call a component assembly program stored in the memory 1004 and further perform the following operations:

[0096] determining whether the second robotic arm can move according to the motion path;

[0097] If not, the first robotic arm is controlled to adjust its position, and the step of determining the first pose of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled is performed based on the first image captured by the first image capture device.

[0098] Optionally, the processor 1001 may call a component assembly program stored in the memory 1004 and further perform the following operations:

[0099] Calculating the global coordinate positions of the first camera and the first component to be assembled according to the visual machine system label present in the first image;

[0100] Determining whether the first component to be assembled is located in a preset working area;

[0101] If not, control the first robotic arm to move to the preset working area.

[0102] Optionally, the processor 1001 may call a component assembly program stored in the memory 1004 and further perform the following operations:

[0103] determining the global coordinates of the second robotic arm and a currently gripped shape based on label information of the visual reference system label in the second image;

[0104] The position and posture information of the second component to be assembled is determined according to the global coordinates and the current clamping shape.

[0105] Optionally, the processor 1001 may call a component assembly program stored in the memory 1004 and further perform the following operations:

[0106] Controlling the second robotic arm to move to a target position according to a motion path corresponding to the second robotic arm, and engaging with the first component to be assembled;

[0107] After the third robotic arm is controlled to move to a designated position according to a movement path corresponding to the third robotic arm, adhesive is added.

[0108] In addition, to achieve the above-mentioned purpose, an embodiment of the present invention also provides a terminal device, including a memory, a processor, and a component assembly program stored in the memory and runnable on the processor. When the processor executes the component assembly program, the component assembly method described above is implemented.

[0109] In addition, to achieve the above objectives, an embodiment of the present invention further provides a computer-readable storage medium, on which a component assembly program is stored. When the component assembly program is executed by a processor, the component assembly method described above is implemented.

[0110] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0111] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0114] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0115] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0116] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present application fall within the scope of the claims and their equivalents, the present application is intended to include such modifications and variations.

Claims

1. A component assembly method, characterized in that: Used to control a component assembly system, the component assembly system includes a first robotic arm, a second robotic arm and a third robotic arm, the first robotic arm is provided with a first image acquisition device, the second robotic arm is provided with a second image acquisition device, the third robotic arm is used for injecting adhesive, and the component assembly method includes the following steps: Determining a first pose of a first component to be assembled and a position of a bonding area corresponding to the first component to be assembled based on the first image captured by the first image capturing device; determining a second posture of a second component to be assembled based on a second image captured by the second image capture device; According to the first posture, the position of the bonding area and the second posture, the motion paths of the second robotic arm and the third robotic arm are planned; wherein, according to the first posture, the position of the bonding area and the second posture, the theoretical gripping mode of the second robotic arm is adjusted; it is determined whether the mechanical structure of the second robotic arm is applicable to the theoretical gripping mode; if not, the first robotic arm is controlled to adjust the posture of the first component to be assembled based on the second posture, and the step of determining the first posture of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled according to the first image captured by the first image acquisition device is executed; after determining the theoretical gripping mode, the motion path of the second robotic arm is calculated; The second robotic arm and the third robotic arm are controlled to move according to the motion path, and the third robotic arm is controlled to perform an adhesive injection action after the movement is completed.

2. The component assembly method according to claim 1, wherein: The step of determining the first pose of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled based on the first image captured by the first image capture device includes: Calculating position information of the first component to be assembled based on a position of the first component to be assembled relative to a visual reference system label in the first image; Perform semantic segmentation on the first image and calculate the position of the bonding area.

3. The component assembly method according to claim 1, wherein: After determining the theoretical gripping mode, the step of calculating the motion path of the second robotic arm includes: determining whether the second robotic arm can move according to the motion path; If not, the first robotic arm is controlled to adjust its position, and the step of determining the first pose of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled is performed based on the first image captured by the first image capture device.

4. The component assembly method according to claim 1, wherein: The step of determining the first pose of the first component to be assembled and the position of the bonding area corresponding to the first component to be assembled based on the first image captured by the first image capture device includes: Calculating the global coordinate positions of the first image acquisition device and the first component to be assembled according to the visual machine system label present in the first image; Determining whether the first component to be assembled is located in a preset working area; If not, control the first robotic arm to move to the preset working area.

5. The component assembly method according to claim 1, wherein: The step of determining the second posture of the second component to be assembled based on the second image captured by the second image capture device includes: determining the global coordinates of the second robotic arm and a currently gripped shape based on label information of the visual reference system label in the second image; The position and posture information of the second component to be assembled is determined according to the global coordinates and the current clamping shape.

6. The component assembly method according to claim 1, wherein: The step of controlling the second robotic arm and the third robotic arm to move according to the motion path, and controlling the third robotic arm to perform the adhesive injection action after the movement is completed includes: Controlling the second robotic arm to move to a target position according to a motion path corresponding to the second robotic arm, and engaging with the first component to be assembled; After the third robotic arm is controlled to move to a designated position according to a movement path corresponding to the third robotic arm, adhesive is added.

7. A terminal device, characterized in that: The method comprises a memory, a processor and a component assembly program stored in the memory and executable on the processor. When the processor executes the component assembly program, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a component assembly program, and when the component assembly program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images