Composite mobile robot operation method and system with autonomous switching of multiple end effectors

Abstract

A composite mobile robot operation method and system with autonomous switching of multiple end effectors, the operation method includes: 1. full scene map modeling; 2. replacing the glue gun to the end of the mechanical arm, the robot performs path planning and glue application on the position of the structural part to be assembled; 3. obtaining the grasping posture of the robot grasping the structural part to be assembled; 4. switching the end of the mechanical arm to a two-finger gripper, and taking the structural part to be assembled from the warehouse according to the grasping posture; 5. obtaining the posture of the position to be assembled of the structural part to be assembled, determining the assembly posture of the structural part to be assembled on the skin to be assembled, and bonding the structural part to be assembled to the skin to be assembled; 6. obtaining the posture of the screw hole after the structural part to be assembled and the skin to be assembled are bonded; replacing the screw gun to the end of the mechanical arm, and fixing the structural part to be assembled on the skin to be assembled. The present invention can autonomously determine the assembly position of the structural part to be assembled on the skin according to the type of the structural part to be assembled, and realizes the assembly of large components through a single robot.

Description

Technical Field

[0001] The present invention relates to the technical field of composite mobile robot assembly, and in particular to an operation method and system of a composite mobile robot with autonomous switching of multiple end effectors. Background Art

[0002] The efficient and high-quality manufacturing of large and complex components such as aircraft skins, ship cabins, and high-speed rail bodies is the foundation for the development of major equipment such as aerospace, marine ships, and rail transit. The above-mentioned large and complex components usually have the characteristics of oversized size, numerous processes, weak structural rigidity, and complex surfaces. Their manufacturing faces challenges such as large scale, multiple tasks, complex processes, and high precision. How to complete the manufacturing task of large and complex components efficiently and with high quality is a bottleneck problem that my country's major equipment manufacturing urgently needs to break through.

[0003] The assembly of aviation parts involves multiple different processes, such as handling, gluing, screw fixing, etc. However, these multiple different processes are generally completed by multiple robots, which has high equipment costs, high risks of collisions between multiple robots, and high equipment failure rates. Summary of the invention

[0004] The present invention provides an operation method and system of a composite mobile robot with autonomous switching of multiple end effectors, so as to solve the technical problems mentioned in the background technology.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] The present invention provides a composite mobile robot operation method with autonomous switching of multiple end effectors, comprising the following steps:

[0007] S1. Full scene map modeling for aviation parts manufacturing scene;

[0008] S2. Obtain the position of the position where the structural part to be assembled is to be glued; replace the glue gun to the end of the robotic arm, and the composite mobile robot plans the operation path for the position where the structural part to be assembled is to be glued and glues, thereby completing the gluing task;

[0009] S3, obtaining the grasping posture of the composite mobile robot grasping the structural part to be assembled;

[0010] S4, replace the two-finger gripper to the end of the robot arm, use the two-finger gripper and take out the structure to be assembled from the warehouse according to the grasping posture of the composite mobile robot to grasp the structure to be assembled;

[0011] S5, obtaining the posture of the assembly position of the structural part to be assembled, determining the assembly posture of the structural part to be assembled on the skin to be assembled, and bonding the structural part to be assembled to the skin to be assembled by applying glue according to the assembly posture;

[0012] S6. Obtain the position and posture of the screw holes after the structural part to be assembled and the skin to be assembled are bonded; replace the screw gun to the end of the robot arm, and fix the structural part to be assembled on the skin to be assembled by the screw gun according to the position and posture of the screw holes.

[0013] Furthermore, the S1 specifically includes the following steps:

[0014] S11, controlling the composite mobile robot to move in the aviation component manufacturing scene, completing full-scene map modeling through two single-line laser radars mounted on the front and rear of the composite mobile robot, and saving the full-scene map information;

[0015] S12, start the real-time obstacle avoidance mode to ensure the safety of the composite mobile robot during movement;

[0016] S13, according to the full-scene map constructed by the composite mobile robot, obtaining the coordinates of the skin to be assembled in the constructed full-scene map;

[0017] S14, the composite mobile robot plans a collision-free executable path according to the coordinates of the skin to be assembled obtained in S13 and the current coordinates;

[0018] S15. The composite mobile robot autonomously navigates and moves to the front of the skin to be assembled according to the planned collision-free executable path.

[0019] Furthermore, the S2 specifically includes the following steps:

[0020] S21, collecting a point cloud image of a position where glue is to be applied to the structure part to be assembled by using a structured light camera to obtain a first point cloud image, and preprocessing the first point cloud image;

[0021] S22, using the SAC-IC algorithm to register the processed first point cloud image with the area on the three-dimensional model of the skin to be assembled corresponding to the position where the structural part to be assembled is to be glued, so as to obtain the position and posture of the position where the structural part to be assembled is to be glued;

[0022] S23, coordinate transformation is performed on the position of the structure to be assembled where glue is to be applied, and the position of the structure to be assembled where glue is to be applied in the structured light camera coordinate system is transformed into the robot arm base coordinate system;

[0023] S24, the end of the mechanical arm on the composite mobile robot moves to the end position of the glue gun, and is connected to the quick-changing device at the end of the glue gun through the quick-changing device at the end of the mechanical arm, so as to fix the glue gun to the end of the mechanical arm;

[0024] S25, determining the main direction of the first point cloud image, and then setting a plurality of slice planes equidistantly along the main direction, pairing the closest points on the left and right sides of the slice planes, and making the line between the paired points intersect with the slice planes, and taking the intersection point as the truncation point;

[0025] S26, setting a screening threshold, treating the truncation point whose normal vector angle with other truncation points in its neighborhood is greater than the screening threshold as a control point, fitting a curve with every four control points, and finally connecting all the curves according to the selection of the control points to generate an initial Catmull–Rom curve;

[0026] S27, optimizing the initial Catmull–Rom curve, eliminating the gluing path points that will collide, and obtaining the final motion path of the robot arm; the final motion path of the robot arm is the gluing trajectory;

[0027] S28. The composite mobile robot applies glue to the locations where the assembled structural parts are to be glued, thereby completing the gluing task.

[0028] Furthermore, the S25 specifically includes the following steps:

[0029] S251, firstly, all points in the processed first point cloud image obtained in S21 are represented by a matrix, and the first point cloud image is decentralized, that is, the coordinate of each point is subtracted from the coordinate mean of all points, and the decentralized matrix for:

[0030] ;

[0031] Among them, the matrix Each row represents the spatial coordinates of a point. Represents the spatial coordinates of the nth point, each column represents a dimension;

[0032] S252, construct a covariance matrix C, the covariance matrix C is as follows:

[0033] ;

[0034] in, Represents covariance, and the calculation formula of covariance is as follows:

[0035] ;

[0036] in, Represents the x-axis coordinate value of the i-th point;

[0037] S253, calculating the eigenvalues ​​and eigenvectors of the covariance matrix C, wherein the eigenvector with the largest eigenvalue represents the main direction of the point cloud image;

[0038] S254, calculating the difference S between the maximum value and the minimum value of the first point cloud image in the main direction, setting the distance d between adjacent slice planes, and then calculating the number N of slice planes set in the main direction as:

[0039] ;

[0040] S255. Pair the closest points on the left and right sides of the slice plane, and let the line between the paired points intersect with the slice plane, and use the intersection point as the truncation point.

[0041] Furthermore, the mathematical expression of the initial Catmull–Rom curve in S26 is as follows:

[0042] ;

[0043] in, is the mathematical expression of the Catmull–Rom curve; is a parameter ranging from 0 to 1; , , and Represent four adjacent control points respectively.

[0044] Furthermore, the S3 specifically includes the following steps:

[0045] S31, according to the full-scene map constructed by the composite mobile robot, obtaining the coordinates of the storage location of the structural parts to be assembled in the constructed full-scene map; and sending the coordinates to the motion controller of the composite mobile robot;

[0046] S32, the composite mobile robot plans a collision-free executable path according to the coordinates of the storage location of the structural part to be assembled and the current coordinates;

[0047] S33, the composite mobile robot moves to the front of the storage location where the to-be-assembled structural part is located in the storage location according to the instruction of the motion controller and the collision-free executable path planned in S32;

[0048] S34, controlling the manipulator arm of the composite mobile robot to move to a specified position, so that the structural parts to be assembled in the storage location are within the field of view of the structured light camera on the manipulator arm, and turning on the structured light camera, using the structured light camera to collect a point cloud image of the structural parts to be assembled in the storage location, to obtain a second point cloud image; and preprocessing the second point cloud image;

[0049] S35, using the SAC-IC algorithm to register the preprocessed second point cloud image with the three-dimensional model of the structural part to be assembled, and obtaining a grasping posture of the composite mobile robot grasping the structural part to be assembled;

[0050] S36, coordinate conversion is performed on the grasping posture of the composite mobile robot grasping the structural part to be assembled, and the grasping posture of the robot grasping the structural part to be assembled in the structured light camera coordinate system is converted to the robot arm base coordinate system.

[0051] Furthermore, the S4 specifically includes the following steps:

[0052] S41, switching the end of the mechanical arm on the composite mobile robot to a two-finger gripper;

[0053] S42, taking out the structure to be assembled from the warehouse using a two-finger gripper and according to the grasping posture of the composite mobile robot to grasp the structure to be assembled;

[0054] S43, according to the full-scene map constructed by the composite mobile robot, obtaining the coordinates of the skin to be assembled in the constructed full-scene map;

[0055] S44, sending the coordinates in S43 to the motion controller of the composite mobile robot;

[0056] S45, the composite mobile robot plans a collision-free executable path according to the coordinates of the skin to be assembled and the current coordinates;

[0057] S46. The composite mobile robot autonomously navigates and moves to the front of the skin to be assembled according to the collision-free executable path planned in S45.

[0058] Furthermore, the S5 specifically includes the following steps:

[0059] S51, collecting a point cloud image of the to-be-assembled position of the to-be-assembled structural part by using a structured light camera to obtain a third point cloud image;

[0060] S52, preprocessing the third point cloud image, using the SAC-IC algorithm to register the preprocessed third point cloud image with the area on the three-dimensional model of the skin to be assembled corresponding to the assembly position of the structural part to be assembled, obtaining the posture of the assembly position of the structural part to be assembled, and determining the assembly posture of the structural part to be assembled on the skin to be assembled;

[0061] S53, performing coordinate transformation on the assembly posture of the structural part to be assembled on the skin to be assembled, and transforming the assembly posture of the structural part to be assembled on the skin to be assembled in the structured light camera coordinate system into the robot arm base coordinate system;

[0062] S54, obtaining the grasping posture of the composite mobile robot grasping the structure to be assembled in S3, and obtaining the posture of the end of the manipulator relative to the structure to be assembled in the camera coordinate system; obtaining the posture of the end of the manipulator relative to the assembly position of the structure to be assembled in the camera coordinate system in S53; correcting the posture of the end of the manipulator relative to the assembly position of the structure to be assembled according to the relationship between the structure to be assembled and its assembly position, so that the structure to be assembled coincides with its assembly position, thereby obtaining the final assembly posture of the composite mobile robot;

[0063] S55, bonding the structural parts to be assembled to the skin to be assembled by glue according to the final assembly posture of the composite mobile robot.

[0064] Furthermore, the S6 specifically includes the following steps:

[0065] S61, using a structured light camera to collect a point cloud image of the screw hole after the structural part to be assembled and the skin to be assembled are bonded to obtain a fourth point cloud image, and preprocessing the fourth point cloud image, using a SAC-IC algorithm to align the preprocessed fourth point cloud image with the three-dimensional model of the threaded hole position, to obtain the position and posture of the screw hole to be assembled;

[0066] S62, performing coordinate transformation on the position and posture of the screw hole to be assembled, and converting the position and posture of the screw hole to be assembled in the structured light camera coordinate system to the robot arm base coordinate system;

[0067] S63, replace the screw gun to the end of the robot arm, and fix the skin to be assembled on the structural part to be assembled by the screw gun according to the posture information of the screw hole to be assembled.

[0068] The second aspect of the present invention further provides a composite mobile robot system with multiple end effectors that can switch autonomously, and the composite mobile robot system adopts the above operation method to perform operations, and comprises:

[0069] The robotic arm has six joints, which enable the end of the arm to achieve six degrees of freedom in space. Torque sensors are set between each joint. The end of the robotic arm is equipped with a quick-change device and a structured light camera.

[0070] The top-mounted mechanism is installed at the bottom of the base of the robot arm, including a top-mounted shell, a binocular vision sensor and a solid-state laser radar sensor installed on the side of the top-mounted shell; the multi-end effector is mounted on the outer surface of the top-mounted shell; the multi-end effector is equipped with a glue gun, a two-finger gripper and a screw gun; the glue gun, the two-finger gripper and the screw gun are all equipped with an end quick-change device adapted to the end quick-change device of the robot arm;

[0071] The dual-steering wheel chassis drive mechanism is installed at the bottom of the top-mounted mechanism and is used to drive the composite mobile robot to move in all directions; the dual-steering wheel chassis drive mechanism has a built-in motion controller; the robotic arm, binocular vision sensor, and solid-state laser radar sensor are all electrically connected to the motion controller.

[0072] Beneficial effects of the present invention:

[0073] 1. The present invention uses a method combining positioning navigation with three-dimensional vision for perception; it solves the problem of limited field of view and difficulty in full scene perception in large and complex component manufacturing scenarios when only three-dimensional vision is used for perception. After positioning and moving through the mobile chassis, the composite mobile robot can perform interactive operations in multiple scenarios, avoiding the limitations of the working space of fixed robotic arms and the difficulty of operating a single mobile vehicle. At the same time, through three-dimensional vision, the accuracy of the operation is greatly improved, enabling the composite mobile robot to perform fine operations and meet high-quality manufacturing needs.

[0074] 2. In view of different structural parts and their different assembly positions on the skin, the present invention can intelligently match them and autonomously determine their assembly positions on the skin according to the types of structural parts to be assembled, thus realizing complete large-scale component assembly through a single robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 is a flow chart of the present invention;

[0076] Figure 2 A spatial layout diagram of the aviation manufacturing scene of the present invention;

[0077] Figure 3 is a structural diagram of the multi-end effector in the present invention;

[0078] Figure 4 Schematic diagram of the structure of the two-finger gripper in the present invention, wherein (a) is a schematic diagram of the structure of the two-finger gripper in an open state, and (b) is a schematic diagram of the structure of the two-finger gripper in a closed state;

[0079] Figure 5 Flowchart for point cloud image preprocessing;

[0080] Figure 6 It is the SAC-IC algorithm registration flow chart;

[0081] Figure 7 Algorithm flow chart for Catmull–Rom curve generation and Descartes algorithm optimization of robot gluing path.

[0082] Description of reference numerals:

[0083] 1. Glue gun; 2. Two-finger clamp; 21. Grasping clamp finger one; 22. Grasping clamp finger two; 3. Screw gun. DETAILED DESCRIPTION

[0084] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many other different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0085] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0086] It should be understood that the orientation or position relationship indicated by terms such as "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0087] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0088] Reference Figures 2 to 3 In one aspect, an embodiment of the present application provides a composite mobile robot system with multiple end effectors capable of autonomous switching, comprising:

[0089] The robotic arm has six joints, which enable the end of the arm to achieve six degrees of freedom in space. Torque sensors are set between each joint. The end of the robotic arm is equipped with a quick-change device and a structured light camera.

[0090] The top-mounted mechanism is installed at the bottom of the base of the robot arm, including a top-mounted shell, a binocular vision sensor and a solid-state laser radar sensor installed on the side of the top-mounted shell; the multi-end effector is mounted on the outer surface of the top-mounted shell; the multi-end effector is equipped with a glue gun 1, a two-finger gripper 2, a screw gun 3 and other equipment; the glue gun 1, the two-finger gripper 2, the screw gun 3 and other equipment are all equipped with an end quick-change device adapted to the end quick-change device of the robot arm, such as a glue gun end quick-change device, so as to achieve rapid switching;

[0091] The dual-steering wheel chassis drive mechanism is installed at the bottom of the top-mounted mechanism and is used to drive the composite mobile robot to move in all directions; the dual-steering wheel chassis drive mechanism has a built-in motion controller; the robotic arm, binocular vision sensor, and solid-state laser radar sensor are all electrically connected to the motion controller.

[0092] In some embodiments, the dual steering wheel chassis drive mechanism includes:

[0093] The chassis has mounting areas at the top and bottom for mounting other components;

[0094] A drive housing, surrounding and mounted on the chassis;

[0095] A dual steering wheel drive assembly includes two support frames, two steering wheels, two linear travel motors electrically connected to a motion controller, and a drive assembly electrically connected to the motion controller. The two steering wheels are installed at intervals on the bottom of the chassis through two support frames, respectively. The drive assembly is installed on the chassis through a drive connector. The drive assembly includes two linear drives. The motion controller drives the linear travel motors through the linear drives, so that the linear travel motors drive the steering wheels to move linearly.

[0096] A downward-looking PGV sensor, electrically connected to the motion controller and mounted on the bottom surface of the chassis through a downward-looking PGV connector;

[0097] Two linear laser radars are respectively mounted on the front and rear sides of the chassis through a front linear laser radar connector and a rear linear laser radar connector; both linear laser radars are electrically connected to the motion controller;

[0098] A battery compartment electrically connected to the motion controller, mounted on the chassis, for supplying power to the composite mobile robot;

[0099] A plurality of universal wheels are arranged at equal intervals on the bottom surface of the chassis through a plurality of universal wheel connectors;

[0100] The I / O panel electrically connected to the motion controller is arranged on the side of the drive housing, and multiple interfaces are integrated on the I / O panel for the composite mobile robot to transmit data.

[0101] The present invention discloses a composite mobile robot system with multi-end effector autonomous switching, which realizes the functions of intelligent gluing, handling, screw fixing, etc. of the robot, and completes the manufacturing tasks of large equipment such as aircraft skin, long stringer, corner piece, etc. The composite mobile robot adopts a three-dimensional vision + multi-end effector autonomous switching scheme, and adopts a three-dimensional vision algorithm to calculate the posture of the skin hole position, long stringer and corner piece, corrects the position and posture errors between the composite mobile operation robot and the components to be assembled, and uses the characteristics of strong real-time and high precision of visual processing to ensure that each end effector of the robot arm can accurately reach the assembly position; adopts a multi-end effector autonomous switching scheme, so that the composite mobile robot can autonomously switch different ends such as glue gun 1, screw gun 3, two-finger gripper 2 according to different tasks, intelligently complete the corresponding tasks, and use one robot to complete the tasks of multiple robots, which greatly reduces the hardware cost, and avoids the problem of multi-robot autonomous scheduling, greatly shortens the task completion time, and improves the efficiency of aviation component manufacturing.

[0102] Reference Figure 1 The second aspect of the embodiment of the present application provides a method for operating a composite mobile robot with autonomous switching of multiple end effectors, comprising the following steps:

[0103] S1. Full scene map modeling for aviation parts manufacturing scene;

[0104] S2, obtaining the position of the position where the structural part to be assembled is to be glued; replacing the glue gun 1 to the end of the robotic arm, the composite mobile robot plans the operation path for the position where the structural part to be assembled is to be glued and glues, thus completing the gluing task;

[0105] S3, obtaining the grasping posture of the composite mobile robot grasping the structural part to be assembled;

[0106] S4, replace the two-finger gripper 2 to the end of the robot arm, use the two-finger gripper 2 and take out the structure to be assembled from the warehouse according to the grasping posture of the composite mobile robot to grasp the structure to be assembled;

[0107] S5, obtaining the posture of the assembly position of the structural part to be assembled, determining the assembly posture of the structural part to be assembled on the skin to be assembled, and bonding the structural part to be assembled to the skin to be assembled by applying glue according to the assembly posture;

[0108] S6. Obtain the position of the screw holes after the structural part to be assembled and the skin to be assembled are bonded; replace the screw gun 3 to the end of the robot arm, and fix the structural part to be assembled on the skin to be assembled through the screw gun 3 according to the position of the screw holes.

[0109] In some embodiments, the S1 specifically includes the following steps:

[0110] S11. Control the composite mobile robot to move omnidirectionally in the aviation component manufacturing scene through the dual-steering wheel chassis drive mechanism, complete the full-scene map modeling through the two single-line laser radars loaded on the front and rear of the composite mobile robot, and save the full-scene map information; so as to realize the autonomous positioning navigation and real-time obstacle avoidance of the composite mobile robot; the dual-steering wheel chassis drive mechanism can realize the omnidirectional movement of the composite mobile robot, ensuring the accessibility of the full-scene position;

[0111] S12, start the real-time obstacle avoidance mode to ensure the safety of the composite mobile robot during movement;

[0112] S13, according to the full-scene map constructed by the composite mobile robot, obtaining the coordinates of the skin to be assembled in the constructed full-scene map; sending the coordinates to the motion controller; the composite mobile robot reads the coordinates in the current full-scene map;

[0113] S14, the composite mobile robot plans a collision-free executable path according to the coordinates of the skin to be assembled obtained in S13 and the current coordinates;

[0114] S15. The composite mobile robot autonomously navigates and moves to the front of the skin to be assembled according to the planned collision-free executable path.

[0115] In some embodiments, S2 specifically includes the following steps:

[0116] S21, control the manipulator arm of the composite mobile robot to move to a specified position, so that the position of the structure to be assembled to be glued is within the field of view of the structured light camera on the manipulator arm; turn on the structured light camera; collect the point cloud image of the position of the structure to be assembled to be glued by the structured light camera, obtain the first point cloud image, and pre-process the first point cloud image; the pre-processing process can refer to Figure 5 ;

[0117] S22. Use the SAC-IC algorithm to register the processed first point cloud image with the area on the three-dimensional model of the skin to be assembled corresponding to the position where the structural part to be assembled is to be glued (refer to the registration process for details). Figure 6 ), obtain the position and posture of the position where the structural part to be assembled is to be glued;

[0118] S23, coordinate transformation is performed on the position of the structure to be assembled where glue is to be applied, and the position of the structure to be assembled where glue is to be applied in the structured light camera coordinate system is transformed into the robot arm base coordinate system;

[0119] S24. According to the process and process requirements, the end of the mechanical arm on the composite mobile robot moves to the end position of the glue gun 1, and is connected to the quick-changing device at the end of the glue gun through the quick-changing device at the end of the mechanical arm, so as to fix the glue gun 1 to the end of the mechanical arm;

[0120] S25, determining the main direction of the first point cloud image, and then setting a plurality of slice planes equidistantly along the main direction, pairing the closest points on the left and right sides of the slice planes, and making the line between the paired points intersect with the slice planes, and taking the intersection point as the truncation point;

[0121] S26. Set the screening threshold, and regard the truncation point whose normal vector angle with other truncation points in its neighborhood is greater than the screening threshold (that is, the inner product is greater than the screening threshold) as the control point. In order to avoid too many control points causing the generated curve to be unstable, strange shapes or discontinuous parts to appear, and overfitting to these control points, resulting in excessive changes in the curve at some positions and loss of overall smoothness; fit a curve with every four control points, and finally connect all the curves according to the selection of control points to generate the initial Catmull–Rom curve; that is, select the first four points Fit a small curve, and select the control points for the next curve To fit, finally fit into a complete initial Catmull-Rom curve;

[0122] S27, optimizing the initial Catmull–Rom curve, eliminating the gluing path points that will collide, and obtaining the final motion path of the robot arm, i.e., the gluing trajectory;

[0123] Specifically, refer to Figure 7 , select control points according to requirements, interpolate points on the path between control points, generate the gluing path of the robot arm by the interpolation points, and record it as the predefined gluing path; extract path points from the predefined gluing path; configure the parameters of the Descartes algorithm according to the robot model and actual requirements; the configured parameters include: setting the sampling strategy to random sampling, and setting parameters such as sampling resolution and workspace restrictions to establish a sampler (i.e., Descartes algorithm); perform collision detection on all gluing path points, and eliminate the gluing path points that will collide; thus obtaining the final motion path of the robot arm;

[0124] S28. The composite mobile robot applies glue to the locations where the assembled structural parts are to be glued, thereby completing the gluing task.

[0125] In some embodiments, the S25 specifically includes the following steps:

[0126] S251, firstly, all points in the processed first point cloud image obtained in S21 are represented by a matrix, and the first point cloud image is decentralized, that is, the coordinate of each point is subtracted from the coordinate mean of all points, and the decentralized matrix for:

[0127] ;

[0128] Among them, the matrix Each row represents the spatial coordinates of a point. Represents the spatial coordinates of the nth point, each column represents a dimension;

[0129] S252, construct a covariance matrix C, the covariance matrix C is as follows:

[0130] ;

[0131] in, Represents covariance, and the calculation formula of covariance is as follows:

[0132] ;

[0133] in, Represents the x-axis coordinate value of the i-th point;

[0134] S253, calculating the eigenvalues ​​and eigenvectors of the covariance matrix C, wherein the eigenvector with the largest eigenvalue represents the main direction of the point cloud image;

[0135] S254, calculating the difference S between the maximum value and the minimum value of the first point cloud image in the main direction, setting the distance d between adjacent slice planes, and then calculating the number N of slice planes set in the main direction as:

[0136] ;

[0137] S255. Pair the closest points on the left and right sides of the slice plane, and let the line between the paired points intersect with the slice plane, and use the intersection point as the truncation point.

[0138] In some embodiments, the mathematical expression of the initial Catmull–Rom curve in S26 is as follows:

[0139] ;

[0140] in, is the mathematical expression of the Catmull–Rom curve; is a parameter ranging from 0 to 1; , , and Represent four adjacent control points respectively.

[0141] In some embodiments, S3 specifically includes the following steps:

[0142] S31, according to the full-scene map constructed by the composite mobile robot, obtaining the coordinates of the storage location where the structural parts to be assembled are located in the storage location in the constructed full-scene map; and sending the coordinates to the motion controller of the composite mobile robot;

[0143] S32, the composite mobile robot plans a collision-free executable path according to the coordinates of the storage location of the structural part to be assembled and the current coordinates;

[0144] S33, the composite mobile robot moves to the front of the storage location where the to-be-assembled structural part is located in the storage location according to the instruction of the motion controller and the collision-free executable path planned in S32;

[0145] S34, controlling the manipulator arm of the composite mobile robot to move to a specified position, so that the structural parts to be assembled in the storage location are within the field of view of the structured light camera on the manipulator arm, and turning on the structured light camera, using the structured light camera to collect a point cloud image of the structural parts to be assembled in the storage location, to obtain a second point cloud image; and preprocessing the second point cloud image;

[0146] S35, using the SAC-IC algorithm to register the preprocessed second point cloud image with the three-dimensional model of the structural part to be assembled, and obtaining a grasping posture of the composite mobile robot grasping the structural part to be assembled;

[0147] S36, coordinate conversion is performed on the grasping posture of the composite mobile robot grasping the structural part to be assembled, and the grasping posture of the robot grasping the structural part to be assembled in the structured light camera coordinate system is converted to the robot arm base coordinate system.

[0148] In some embodiments, the S4 specifically includes the following steps:

[0149] S41, switching the end of the mechanical arm on the composite mobile robot to a two-finger gripper 2;

[0150] Specifically, firstly, the mechanical arm of the composite mobile robot is controlled to move to the top of the multi-end effector stored at the tail of the top loading mechanism, and the mechanical arm is controlled to move to the original position of the end of the glue gun 1, and an electrical signal is sent to the end quick-change device of the composite mobile robot, and the steel ball is contracted by electromagnetic control to disconnect the end connection of the composite mobile robot. According to the process and process requirements, the end quick-change device of the composite mobile robot moves to the position of the two-finger clamp end quick-change device, and the fitting and line connection of the end quick-change device of the composite mobile robot are completed autonomously, and an electrical signal is sent to the end quick-change device of the composite mobile robot, and the steel ball is released by electromagnetic control to complete the end fastening connection, and the mechanical arm is controlled to take out the end of the two-finger clamp 2 from the top loading mechanism, and activate the clamp; thereby completing the replacement of the two-finger clamp 2;

[0151] S42, taking out the structure to be assembled from the warehouse using the two-finger gripper 2 and according to the grasping posture of the composite mobile robot to grasp the structure to be assembled;

[0152] The structure of the two-finger gripper 2 is shown in Figure 4 , wherein (a) is a schematic diagram of the structure of the two-finger gripper 2 in an open state, and (b) is a schematic diagram of the structure of the two-finger gripper 2 in a closed state; the two-finger gripper 2 includes a grasping gripper finger 1 21, a grasping gripper finger 22, and a driving member for driving the grasping gripper finger 1 21 and the grasping gripper finger 22 to open and close, and the grasping gripper finger 1 21 and the grasping gripper finger 22 adopt a conformal design to be suitable for grasping structural members such as long beams and corner pieces; the two-finger gripper 2 adopts a specific structural design, thereby realizing the grasping and perfect fit of long and thin flexible structural members, avoiding problems such as angle tilt and front and back swinging of structural members, and greatly improving assembly accuracy and grasping stability.

[0153] The driving process of the two-finger gripper 2 is as follows: the industrial computer sends instructions to control the two-finger gripper 2 through RS485 communication, sets a specified force or closing angle, and when the specified force or angle is reached, the structural part to be assembled is successfully grasped;

[0154] S43, according to the full-scene map constructed by the composite mobile robot, obtaining the coordinates of the skin to be assembled in the constructed full-scene map;

[0155] S44, sending the coordinates in S43 to the motion controller of the composite mobile robot;

[0156] S45, the composite mobile robot plans a collision-free executable path according to the coordinates of the skin to be assembled and the current coordinates;

[0157] S46. The composite mobile robot autonomously navigates and moves to the front of the skin to be assembled according to the collision-free executable path planned in S45.

[0158] In some embodiments, S5 specifically includes the following steps:

[0159] S51, controlling the manipulator arm of the composite mobile robot to move to a specified position, so that the to-be-assembled position of the structural part to be assembled is within the field of view of the structured light camera on the manipulator arm, turning on the structured light camera on the manipulator arm, and collecting a point cloud image of the to-be-assembled position of the structural part to be assembled through the structured light camera to obtain a third point cloud image;

[0160] S52, preprocessing the third point cloud image, using the SAC-IC algorithm to register the preprocessed third point cloud image with the area on the three-dimensional model of the skin to be assembled corresponding to the assembly position of the structural part to be assembled, obtaining the posture of the assembly position of the structural part to be assembled, and determining the assembly posture of the structural part to be assembled on the skin to be assembled;

[0161] S53, performing coordinate transformation on the assembly posture of the structural part to be assembled on the skin to be assembled, and transforming the assembly posture of the structural part to be assembled on the skin to be assembled in the structured light camera coordinate system into the robot arm base coordinate system;

[0162] S54, obtaining the grasping posture of the composite mobile robot grasping the structure to be assembled in S3, and obtaining the posture of the end of the manipulator relative to the structure to be assembled in the camera coordinate system; obtaining the posture of the end of the manipulator relative to the assembly position of the structure to be assembled in the camera coordinate system in S53; correcting the posture of the end of the manipulator relative to the assembly position of the structure to be assembled according to the relationship between the structure to be assembled and its assembly position, so that the structure to be assembled coincides with its assembly position (edge, screw hole, etc.), thereby obtaining the final assembly posture of the composite mobile robot;

[0163] S55, bonding the structural parts to be assembled to the skin to be assembled by glue according to the final assembly posture of the composite mobile robot.

[0164] In some embodiments, the S6 specifically includes the following steps:

[0165] S61, controlling the manipulator arm of the composite mobile robot to move to a specified posture, so that the positions of the screw holes to be fastened on the skin to be assembled and the structural part to be assembled are within the field of view of the structured light camera, turning on the structured light camera on the manipulator arm, using the structured light camera to collect point cloud images of the structural part to be assembled and the screw holes on the skin to be assembled, obtaining a fourth point cloud image, preprocessing the fourth point cloud image, using the SAC-IC algorithm to align the processed fourth point cloud image with the three-dimensional model of the threaded hole position, and obtaining the posture of the screw hole to be assembled;

[0166] S62, performing coordinate transformation on the position and posture of the screw hole to be assembled, and converting the position and posture of the screw hole to be assembled in the structured light camera coordinate system to the robot arm base coordinate system;

[0167] S63, replace the screw gun 3 to the end of the robot arm, and fix the skin to be assembled on the structural part to be assembled by the screw gun 3 according to the posture information of the screw hole to be assembled.

[0168] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention. In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A composite mobile robot operation method with autonomous switching of multiple end effectors, characterized in that: The steps include: S1. Full scene map modeling for aviation parts manufacturing scene; S2. Obtain the position of the position where the structural part to be assembled is to be glued; replace the glue gun to the end of the robotic arm, and the composite mobile robot plans the operation path for the position where the structural part to be assembled is to be glued and glues, thereby completing the gluing task; S3, obtaining the grasping posture of the composite mobile robot grasping the structural part to be assembled; S4, replace the two-finger gripper to the end of the robot arm, use the two-finger gripper and take out the structure to be assembled from the warehouse according to the grasping posture of the composite mobile robot to grasp the structure to be assembled; S5, obtaining the posture of the assembly position of the structural part to be assembled, determining the assembly posture of the structural part to be assembled on the skin to be assembled, and bonding the structural part to be assembled to the skin to be assembled by applying glue according to the assembly posture; S6, obtaining the position and posture of the screw holes after the structural parts to be assembled and the skin to be assembled are bonded; Replace the screw gun at the end of the robot arm, and fix the structural part to be assembled on the skin to be assembled by the screw gun according to the position of the screw hole; The multi-end effector is mounted on the outer side of the top-mounted shell; the multi-end effector is equipped with a glue gun, a two-finger gripper and a screw gun; the glue gun, the two-finger gripper and the screw gun are all equipped with an end quick-change device adapted to the end quick-change device of the robot arm; The S2 specifically includes the following steps: S21, collecting a point cloud image of a position where glue is to be applied to the structure part to be assembled by using a structured light camera to obtain a first point cloud image, and preprocessing the first point cloud image; S22, using the SAC-IC algorithm to register the processed first point cloud image with the area on the three-dimensional model of the skin to be assembled corresponding to the position where the structural part to be assembled is to be glued, so as to obtain the position and posture of the position where the structural part to be assembled is to be glued; S23, coordinate transformation is performed on the position of the structure to be assembled where glue is to be applied, and the position of the structure to be assembled where glue is to be applied in the structured light camera coordinate system is transformed into the robot arm base coordinate system; S24, the end of the mechanical arm on the composite mobile robot moves to the end position of the glue gun, and is connected to the quick-changing device at the end of the glue gun through the quick-changing device at the end of the mechanical arm, so as to fix the glue gun to the end of the mechanical arm; S25, determining the main direction of the first point cloud image, and then setting a plurality of slice planes equidistantly along the main direction, pairing the closest points on the left and right sides of the slice planes, and making the line between the paired points intersect with the slice planes, and taking the intersection point as the truncation point; S26, setting a screening threshold, treating the truncation point whose normal vector angle with other truncation points in its neighborhood is greater than the screening threshold as a control point, fitting a curve with every four control points, and finally connecting all the curves according to the selection of the control points to generate an initial Catmull–Rom curve; the initial Catmull–Rom curve is as follows: ； in, is the mathematical expression of the Catmull–Rom curve; is a parameter ranging from 0 to 1; , , and Represent four adjacent control points respectively; S27, optimizing the initial Catmull–Rom curve, eliminating the gluing path points that will collide, and obtaining the final motion path of the robot arm; the final motion path of the robot arm is the gluing trajectory; S28. The composite mobile robot applies glue to the locations where the assembled structural parts are to be glued, thereby completing the gluing task.

2. The composite mobile robot operation method according to claim 1, characterized in that: The S1 specifically includes the following steps: S11, controlling the composite mobile robot to move in the aviation component manufacturing scene, completing full-scene map modeling through two single-line laser radars mounted on the front and rear of the composite mobile robot, and saving the full-scene map information; S12, start the real-time obstacle avoidance mode to ensure the safety of the composite mobile robot during movement; S13, according to the full-scene map constructed by the composite mobile robot, obtaining the coordinates of the skin to be assembled in the constructed full-scene map; S14, the composite mobile robot plans a collision-free executable path according to the coordinates of the skin to be assembled obtained in S13 and the current coordinates; S15. The composite mobile robot autonomously navigates and moves to the front of the skin to be assembled according to the planned collision-free executable path.

3. The composite mobile robot operation method according to claim 1, characterized in that: The S25 specifically includes the following steps: S251, firstly, all points in the processed first point cloud image obtained in S21 are represented by a matrix, and the first point cloud image is decentralized, that is, the coordinate of each point is subtracted from the coordinate mean of all points, and the decentralized matrix for: ； Among them, the matrix Each row represents the spatial coordinates of a point. Represents the spatial coordinates of the nth point, each column represents a dimension; S252, construct a covariance matrix C, the covariance matrix C is as follows: ； in, Represents covariance, and the calculation formula of covariance is as follows: ； in, Represents the x-axis coordinate value of the i-th point; S253, calculating the eigenvalues ​​and eigenvectors of the covariance matrix C, wherein the eigenvector with the largest eigenvalue represents the main direction of the point cloud image; S254, calculating the difference S between the maximum value and the minimum value of the first point cloud image in the main direction, setting the distance d between adjacent slice planes, and calculating the number N of slice planes set in the main direction as: ； S255. Pair the closest points on the left and right sides of the slice plane, and let the line between the paired points intersect with the slice plane, and use the intersection point as the truncation point.

4. The composite mobile robot operation method according to claim 1, characterized in that: The S3 specifically includes the following steps: S31, according to the full-scene map constructed by the composite mobile robot, obtaining the coordinates of the storage location of the structural parts to be assembled in the constructed full-scene map; and sending the coordinates to the motion controller of the composite mobile robot; S32, the composite mobile robot plans a collision-free executable path according to the coordinates of the storage location of the structural part to be assembled and the current coordinates; S33, the composite mobile robot moves to the front of the storage location where the to-be-assembled structural part is located according to the instruction of the motion controller and the collision-free executable path planned in S32 by autonomous navigation; S34, controlling the manipulator arm of the composite mobile robot to move to a specified position, so that the structural parts to be assembled in the storage location are within the field of view of the structured light camera on the manipulator arm, and turning on the structured light camera, using the structured light camera to collect a point cloud image of the structural parts to be assembled in the storage location, to obtain a second point cloud image; and preprocessing the second point cloud image; S35, using the SAC-IC algorithm to register the preprocessed second point cloud image with the three-dimensional model of the structural part to be assembled, and obtaining a grasping posture of the composite mobile robot grasping the structural part to be assembled; S36, coordinate conversion is performed on the grasping posture of the composite mobile robot grasping the structural part to be assembled, and the grasping posture of the robot grasping the structural part to be assembled in the structured light camera coordinate system is converted to the robot arm base coordinate system.

5. The composite mobile robot operation method according to claim 1, characterized in that: The S4 specifically includes the following steps: S41, switching the end of the mechanical arm on the composite mobile robot to a two-finger gripper; S42, taking out the structure to be assembled from the warehouse using a two-finger gripper and according to the grasping posture of the composite mobile robot to grasp the structure to be assembled; S43, according to the full-scene map constructed by the composite mobile robot, obtaining the coordinates of the skin to be assembled in the constructed full-scene map; S44, sending the coordinates in S43 to the motion controller of the composite mobile robot; S45, the composite mobile robot plans a collision-free executable path according to the coordinates of the skin to be assembled and the current coordinates; S46. The composite mobile robot autonomously navigates and moves to the front of the skin to be assembled according to the collision-free executable path planned in S45.

6. The composite mobile robot operation method according to claim 1, characterized in that: The S5 specifically includes the following steps: S51, collecting a point cloud image of the to-be-assembled position of the to-be-assembled structural part by using a structured light camera to obtain a third point cloud image; S52, preprocessing the third point cloud image, using the SAC-IC algorithm to register the preprocessed third point cloud image with the area on the three-dimensional model of the skin to be assembled corresponding to the assembly position of the structural part to be assembled, obtaining the posture of the assembly position of the structural part to be assembled, and determining the assembly posture of the structural part to be assembled on the skin to be assembled; S53, performing coordinate transformation on the assembly posture of the structural part to be assembled on the skin to be assembled, and transforming the assembly posture of the structural part to be assembled on the skin to be assembled in the structured light camera coordinate system into the robot arm base coordinate system; S54, obtaining the grasping posture of the composite mobile robot grasping the structure to be assembled in S3, and obtaining the posture of the end of the manipulator relative to the structure to be assembled in the camera coordinate system; obtaining the posture of the end of the manipulator relative to the assembly position of the structure to be assembled in the camera coordinate system in S53; correcting the posture of the end of the manipulator relative to the assembly position of the structure to be assembled according to the relationship between the structure to be assembled and its assembly position, so that the structure to be assembled coincides with its assembly position, thereby obtaining the final assembly posture of the composite mobile robot; S55, bonding the structural parts to be assembled to the skin to be assembled by applying glue according to the final assembly posture of the composite mobile robot.

7. The composite mobile robot operation method according to claim 1, characterized in that: The S6 specifically includes the following steps: S61, using a structured light camera to collect a point cloud image of the screw hole after the structural part to be assembled and the skin to be assembled are bonded to obtain a fourth point cloud image, and preprocessing the fourth point cloud image, using a SAC-IC algorithm to align the preprocessed fourth point cloud image with the three-dimensional model of the threaded hole position, to obtain the position and posture of the screw hole to be assembled; S62, performing coordinate transformation on the position and posture of the screw hole to be assembled, and converting the position and posture of the screw hole to be assembled in the structured light camera coordinate system to the robot arm base coordinate system; S63, replace the screw gun to the end of the robot arm, and fix the skin to be assembled on the structural part to be assembled by the screw gun according to the posture information of the screw hole to be assembled.

8. A composite mobile robot system with multiple end effectors switching autonomously, characterized in that: The operation method according to any one of claims 1 to 7 is used to perform the operation, and the composite mobile robot system comprises: The robotic arm has six joints, each with a torque sensor. The end of the robotic arm is equipped with a quick-change device and a structured light camera. A top-mounted mechanism, mounted on the bottom of the base of the robotic arm, comprises a top-mounted housing, a binocular vision sensor and a solid-state laser radar sensor mounted on the side of the top-mounted housing; The dual-steering wheel chassis drive mechanism is installed at the bottom of the top-mounted mechanism and is used to drive the composite mobile robot to move in all directions; the dual-steering wheel chassis drive mechanism has a built-in motion controller; the robotic arm, binocular vision sensor, and solid-state laser radar sensor are all electrically connected to the motion controller.

Images