A mobile collaborative robot

Abstract

The application provides a mobile collaborative robot, which comprises a mobile robot, an integrated control system, a six-axis collaborative robot and a vision system; the integrated control system is arranged in the mobile robot and is used for controlling the mobile robot, the six-axis collaborative robot and the vision system to realize collaborative operation of the three; the end of the six-axis collaborative robot is arranged on the mobile robot and is used for work in a work space; the vision system is arranged at the other end of the six-axis collaborative robot and is used for providing visual assistance when the six-axis collaborative robot works; and the mobile robot is used for receiving a movement instruction of the integrated control system and executing. The expanded work range improves the automation degree of work, the vision system provides visual assistance when the six-axis collaborative robot works, visual sensing is more accurate, and the production work efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of mobile collaborative robot technology, and in particular to a mobile collaborative robot. Background Technology

[0002] Against the backdrop of "Industry 4.0" and "Internet Plus," smart logistics and smart manufacturing have been continuously developed, and my country's robotics industry has also achieved tremendous growth. However, existing robots, due to their fixed body position and limited range of motion due to the arm span, have restricted their operational range. Furthermore, their visual sensing is not precise enough, which is detrimental to production operations. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in the aforementioned technologies. Therefore, the object of this invention is to propose a mobile collaborative robot that expands the working range, improves the degree of automation, and provides visual assistance based on a vision system when the six-axis collaborative robot is performing operations. This results in more accurate visual sensing and improved production efficiency.

[0004] To achieve the above objectives, embodiments of the present invention propose a mobile collaborative robot, comprising: a mobile robot, an integrated control system, a six-axis collaborative robot, and a vision system; wherein,

[0005] The integrated control system is located inside the mobile robot and is used to control the mobile robot, the six-axis collaborative robot and the vision system to achieve coordinated operation of the three.

[0006] The end of the six-axis collaborative robot is mounted on the mobile robot and is used to perform operations within the workspace.

[0007] The vision system is located at the other end of the six-axis collaborative robot and is used to provide visual assistance when the six-axis collaborative robot is performing its work;

[0008] The mobile robot is used to receive and execute movement commands from the integrated control system.

[0009] According to some embodiments of the present invention, the mobile robot includes a mobile robot body and a motion control module: wherein,

[0010] The mobile robot body includes: a drive wheel assembly, movable casters, anti-collision strips, an inspection door, a left side door of the MC30, a left side door of the middle frame, a right side door of the MC30, and a right side door of the middle frame;

[0011] The mobile control module includes: a lidar unit, a power battery unit, a communication unit, a control unit and a power supply unit, a charging port, a mobile indicator light, and a depth camera;

[0012] The movement indicator light is used to display the working status of the mobile robot body;

[0013] The lidar unit is used to acquire lidar information of the mobile robot body during its movement.

[0014] Depth cameras are used to acquire scene images of the mobile robot as it moves.

[0015] According to some embodiments of the present invention, the integrated control system includes: a mobile robot control module, a robotic arm indicator light, an emergency stop switch, a robotic arm power control box, an application expansion box, a control panel assembly, and a teach pendant; wherein,

[0016] The mobile robot control module is used to generate control commands for the mobile robot based on lidar information and scene images;

[0017] The robotic arm indicator light is used to display the working status of the six-axis robotic arm collaborative robot.

[0018] The control panel components include a USB interface, a power button, a rotary power switch, a power off button, and an antenna.

[0019] According to some embodiments of the present invention, the teach pendant is used for mobile robot operation control, map building, task editing and distribution, and viewing status and information logs, warning logs and error logs.

[0020] According to some embodiments of the present invention, the six-axis collaborative robot is a six-axis collaborative robotic arm; the six-axis collaborative robotic arm includes a base, communication and power cables, J1-J2 modules, a lower arm, J3-J4 modules, an upper arm, J5-J6 modules, tool I / O, and a tool flange; wherein,

[0021] The communication and power cables are disposed on the side of the base;

[0022] The J1-J2 modules are mounted on the base;

[0023] The J1-J2 modules, lower arm, J3-J4 modules, upper arm, J5-J6 modules, tool IO, and tool flange are connected in sequence.

[0024] According to some embodiments of the present invention, it further includes: a display module, used to determine the distance between the end-load center of the six-axis collaborative robotic arm and the center of the mounting tool flange, query the relationship between the preset maximum load and the center distance, obtain the maximum allowable load and display it.

[0025] According to some embodiments of the present invention, the vision system includes:

[0026] The lidar sensing module is used to acquire point cloud data of goods placed in the area to be grasped;

[0027] The image acquisition module is used to acquire perceived images of goods placed in the area to be grasped;

[0028] The first determining module is used to perform contour recognition based on the perceived image to determine the first contour corresponding to each item;

[0029] The second determining module is used to perform contour recognition based on the point cloud data and the perceived image to determine the second contour corresponding to each item.

[0030] The matching module is used to match the first contour with the second contour to obtain a matching relationship, and send the matching relationship to the six-axis collaborative robot.

[0031] According to some embodiments of the present invention, the first determining module includes:

[0032] The preprocessing module is used to preprocess the perceived images;

[0033] The segmentation module is used to segment the preprocessed perceptual image to obtain the target perceptual image;

[0034] Connection module, used for:

[0035] Obtain the pixel values ​​of each pixel in the target perception image, group the pixels with the same pixel values ​​into a set, and connect all the pixels in the set to obtain the feature region; each set corresponds to a feature region;

[0036] The center of each feature region is determined, and using the center of the feature region as the pole of the polar coordinate system, the region is rotated at preset angular intervals to determine the coordinates of the edges of the feature region.

[0037] By connecting the coordinates, the first outline of the goods corresponding to the feature region is determined.

[0038] According to some embodiments of the present invention, the segmentation module includes:

[0039] The third determining module is used for:

[0040] The preprocessed perceptual image is input into a pre-trained convolutional neural network, which outputs a convolutional feature map.

[0041] The convolutional feature map is input into the Region Candidate Network (RPN), which outputs several segmented regions, and the segmented regions are subjected to Ro I pooling.

[0042] The local convolutional feature maps corresponding to the segmented regions after Ro I pooling are cropped using an interpolation algorithm to convert them into a fixed size. Then, convolution and max pooling are used to obtain the feature vector maps of the segmented regions.

[0043] R-CNN is applied to several feature vector maps to determine the feature vector corresponding to each feature vector map;

[0044] The matching module is used to match several feature vectors with preset feature vectors respectively. The segmentation region corresponding to the feature vector with the highest matching degree is taken as the region of interest. The image of the region of interest is used as the target perception image and extracted and processed.

[0045] According to some embodiments of the present invention, the vision system further includes:

[0046] The first generation module is used for:

[0047] Based on the matching relationship, determine the distance information and orientation angle information between each cargo and the six-axis collaborative robot;

[0048] The location code information of each cargo is determined based on distance and orientation angle information;

[0049] Motion planning information for a six-axis collaborative robot is generated based on the position encoding information.

[0050] The second generation module is used to generate motion path instructions based on the motion planning information;

[0051] The correction module is used for:

[0052] When the six-axis collaborative robot executes motion path instructions to the corresponding position, it acquires point cloud data of the goods to be grasped;

[0053] The first global point-pair feature of the point cloud data of the goods to be captured is determined, and the first hash table is built as the actual model with the feature as the key and the point pair as the value.

[0054] The actual model is matched with the theoretical model to determine the mapping relationship between the two; the theoretical model is generated by a second hash table based on the second global point-pair features of point cloud data of standard goods, with the features as keys and the point pairs as values.

[0055] The gripping parameters corresponding to the theoretical model are corrected according to the mapping relationship to obtain the corrected gripping parameters, which are then sent to the six-axis collaborative robot.

[0056] The six-axis collaborative robot grips the goods to be grasped based on the corrected gripping parameters.

[0057] This invention proposes a mobile collaborative robot that combines the wide-range mobility of an AMR (Automatic Mobile Robot) with the precise operation of a six-axis collaborative robot, greatly expanding its application scope. It can perform automated transfer on production lines, automated loading and unloading, workshop distribution, and automated storage and warehousing of spare parts and tools, and is widely used. This mobile collaborative robot uses laser SLAM technology for autonomous positioning and navigation, and can perform functions such as grasping and transporting items using a simple and easy-to-use interface, truly achieving "hands and feet working together" and improving production efficiency.

[0058] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0059] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0060] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0061] Figure 1 This is a schematic diagram of the structure of a mobile collaborative robot in one state according to an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the structure of a mobile collaborative robot in another state according to an embodiment of the present invention;

[0063] Figure 3 This is a top view of a motion control module according to an embodiment of the present invention (excluding the top plate and middle frame);

[0064] Figure 4 This is a schematic diagram of a mobile robot body according to an embodiment of the present invention;

[0065] Figure 5 This is a partial schematic diagram of an integrated control system according to an embodiment of the present invention;

[0066] Figure 6 This is a partial schematic diagram of an integrated control system according to an embodiment of the present invention;

[0067] Figure 7 This is a schematic diagram of the left side of a mobile robot body according to an embodiment of the present invention;

[0068] Figure 8 This is a schematic diagram of the right side of a mobile robot body according to an embodiment of the present invention;

[0069] Figure 9 This is a schematic diagram of a six-axis collaborative robot according to an embodiment of the present invention;

[0070] Figure 10 This is a schematic diagram of the working area of ​​a six-axis collaborative robot according to an embodiment of the present invention.

[0071] Figure label:

[0072] 1. LiDAR Unit; 2. Power Battery Unit; 3. Communication Unit; 4. Control Unit and Power Supply Unit; 5. Drive Wheel Assembly; 6. Movable Casters; 7. Anti-collision Strip; 8. Movement Indicator Light; 9. Depth Camera; 10. Inspection Door; 11. Robotic Arm Indicator Light; 12. Emergency Stop Switch; 13. Robotic Arm Power Control Box; 14. Application Expansion Box; 15. Charging Port; 16. Control Panel Assembly; 17. Teach Pendant; 18. MC30 Left Side Door; 19. MC30 Right Side Door; 20. Middle Frame Right Side Door; 21. Base; 22. Communication and Power Cables; 23. J1-J2 Modules; 24. Lower Arm; 25. J3-J4 Modules; 26. Upper Arm; 27. J5-J6 Modules; 28. Tool I / O; 29. ​​Tool Flange; 30. Detailed Implementation

[0073] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0074] According to the appendix Figure 1-10 A mobile collaborative robot proposed in this embodiment of the invention will be described.

[0075] To achieve the above objectives, embodiments of the present invention propose a mobile collaborative robot, comprising: a mobile robot, an integrated control system, a six-axis collaborative robot, and a vision system; wherein,

[0076] The integrated control system is located inside the mobile robot and is used to control the mobile robot, the six-axis collaborative robot and the vision system to achieve coordinated operation of the three.

[0077] The end of the six-axis collaborative robot is mounted on the mobile robot and is used to perform operations within the workspace.

[0078] The vision system is located at the other end of the six-axis collaborative robot and is used to provide visual assistance when the six-axis collaborative robot is performing its work;

[0079] The mobile robot is used to receive and execute movement commands from the integrated control system.

[0080] The beneficial effects of the above technical solutions are as follows: Combining the wide-range mobility of AMRs with the precise operation of six-axis collaborative robots, the application scope is greatly expanded. It can perform automated transfer on production lines, automated loading and unloading, workshop distribution, and automated storage and warehousing of spare parts and tools, and is widely used. One type of mobile collaborative robot uses laser SLAM technology to achieve autonomous positioning and navigation. It can perform functions such as grasping and transporting items using a simple and easy-to-use interface, truly achieving "hands and feet working together" and improving production efficiency.

[0081] According to some embodiments of the present invention, the mobile robot includes a mobile robot body and a motion control module: wherein,

[0082] The mobile robot body includes: drive wheel assembly 5, movable casters 6, anti-collision strip 7, inspection door 10, MC30 left side door 18, middle frame left side door 19, MC30 right side door 20, and middle frame right side door 21;

[0083] The mobile control module includes: a lidar unit 1, a power battery unit 2, a communication unit 3, a control unit and power supply unit 4, a charging port 15, a mobile indicator light 8, and a depth camera 9.

[0084] The movement indicator light 8 is used to display the working status of the mobile robot body;

[0085] The lidar unit 1 is used to acquire lidar information of the mobile robot body during its movement.

[0086] Depth camera 9 is used to acquire scene images of the mobile robot body during its movement.

[0087] According to some embodiments of the present invention, the integrated control system includes: a mobile robot control module, a robotic arm indicator light 11, an emergency stop switch 12, a robotic arm power control box 13, an application expansion box 14, a control panel assembly 16, and a teach pendant 17; wherein,

[0088] The mobile robot control module is used to generate control commands for the mobile robot based on lidar information and scene images;

[0089] The robotic arm indicator light is used to display the working status of the six-axis collaborative robot;

[0090] The control panel component 16 includes a USB interface, a power button, a rotary power switch, a power off button, and an antenna (as shown in the figure, arranged from left to right).

[0091] According to some embodiments of the present invention, the teach pendant 17 is used for mobile robot operation control, map building, task editing and distribution, and viewing status and information logs, warning logs and error logs.

[0092] According to some embodiments of the present invention, the six-axis collaborative robot is a six-axis collaborative robotic arm; the six-axis collaborative robotic arm includes a base 22, a communication and power cable 23, J1-J2 modules 24, a lower arm 25, J3-J4 modules 26, an upper arm 27, J5-J6 modules 28, a tool IO 29, and a tool flange 30; wherein,

[0093] The communication and power cable 23 is disposed on the side of the base 22;

[0094] The J1-J2 modules 24 are disposed on the base 22;

[0095] The J1-J2 module 24, lower arm 25, J3-J4 module 26, upper arm 27, J5-J6 module 28, tool IO29, and tool flange 30 are connected in sequence.

[0096] According to some embodiments of the present invention, it further includes: a display module, used to determine the distance between the end load center of the six-axis collaborative robot arm and the center of the mounting tool flange 30, query the relationship between the preset maximum load and the center distance, obtain the maximum allowable load and display it.

[0097] The beneficial effects of the above technical solution are: it facilitates obtaining the maximum load of the six-axis collaborative robot arm, and avoids exceeding the maximum load during use, thus protecting the six-axis collaborative robot arm.

[0098] like Figure 10 As shown, the workspace of a six-axis collaborative robot refers to the area within the specified range of the base joints. Avoid moving the tool into the cylindrical space as much as possible, as this will cause the joints to move too quickly while the tool moves slowly, resulting in low efficiency and unpredictable risks for the six-axis collaborative robot. The working range is 800-1000mm.

[0099] According to some embodiments of the present invention, the vision system includes:

[0100] The lidar sensing module is used to acquire point cloud data of goods placed in the area to be grasped;

[0101] The image acquisition module is used to acquire perceived images of goods placed in the area to be grasped;

[0102] The first determining module is used to perform contour recognition based on the perceived image to determine the first contour corresponding to each item;

[0103] The second determining module is used to perform contour recognition based on the point cloud data and the perceived image to determine the second contour corresponding to each item.

[0104] The matching module is used to match the first contour with the second contour to obtain a matching relationship, and send the matching relationship to the six-axis collaborative robot.

[0105] The working principle of the above technical solution is as follows: In this embodiment, the time axis of the LiDAR sensing module and the image acquisition module when acquiring data is corresponding, and the field of view of the LiDAR sensing module is corresponding to the field of view of the image acquisition module (to ensure the consistency of the acquired scene), thereby ensuring that the point cloud data of the goods in the area to be grasped and the perceived image of the goods in the area to be grasped have a high degree of matching.

[0106] In this embodiment, the extrinsic coordinate transformation matrix of the lidar sensing module and the image acquisition module is determined using existing extrinsic calibration methods. This facilitates determining the correlation between the coordinates of the data acquired by the two modules, and is beneficial for adjusting the field of view and comparing the data.

[0107] In this embodiment, the first contour is the contour corresponding to each item determined by the first determining module based on the contour recognition of the perceived image.

[0108] In this embodiment, the second contour is the contour corresponding to each item determined by the second determining module based on point cloud data contour recognition.

[0109] In this embodiment, the first contour and the second contour are matched to obtain a matching relationship, which means that the first contour and the second contour of the same object are matched.

[0110] The beneficial effects of the above technical solution are: it enables the association of the lidar sensing module and the image acquisition module, avoiding the inaccuracy of identification by a single device, improving the accuracy of the association between the first contour and the second contour, thereby achieving accurate identification of goods, and facilitating accurate operation of the six-axis collaborative robot based on the matching relationship.

[0111] In one embodiment, the second determining module is used to perform contour recognition based on the point cloud data and the perceived image to determine the second contour corresponding to each item, including:

[0112] The point cloud data is filtered based on the RANSAC algorithm, and the point cloud data related to the ground is filtered out and removed to obtain the target point cloud data.

[0113] The target point cloud data is converted into a sparse depth image;

[0114] Visual detection is performed on the perceived image based on a pre-trained YOLOv4 deep neural network model to determine the detection results;

[0115] Based on the detection results and the external parameter coordinate transformation matrix corresponding to the lidar sensing module and the image acquisition module, contour recognition is performed to determine the second contour corresponding to each cargo.

[0116] The working principle and beneficial effects of the above technical solution are as follows: The detection results include the division of the regions of each item in the perceived image. The second contour corresponding to each item is accurately determined.

[0117] According to some embodiments of the present invention, the first determining module includes:

[0118] The preprocessing module is used to preprocess the perceived images;

[0119] The segmentation module is used to segment the preprocessed perceptual image to obtain the target perceptual image;

[0120] Connection module, used for:

[0121] Obtain the pixel values ​​of each pixel in the target perception image, group the pixels with the same pixel values ​​into a set, and connect all the pixels in the set to obtain the feature region; each set corresponds to a feature region;

[0122] The center of each feature region is determined, and using the center of the feature region as the pole of the polar coordinate system, the region is rotated at preset angular intervals to determine the coordinates of the edges of the feature region.

[0123] By connecting the coordinates, the first outline of the goods corresponding to the feature region is determined.

[0124] The working principle of the above technical solution is as follows: In this embodiment, the preprocessing module preprocesses the perceived image, including: extracting an image as an effective pixel every few pixels according to certain rules, which can obtain an image with reduced data volume, thus reducing computational complexity and improving computational speed. It also includes selecting to use a linear filter or morphological filter to filter out isolated noise points in the image based on image quality, thereby improving image accuracy.

[0125] In this embodiment, the target perception image is the image of the goods to be grasped after removing the background of the area to be grasped (i.e., the image corresponding to the background), and what remains is only the image of the goods to be grasped, thus reducing the amount of image processing.

[0126] In this embodiment, the pixel values ​​of each pixel in the target perception image are obtained, pixels with the same pixel values ​​are grouped into a set, and the pixels within each set are connected to obtain a feature region; each set corresponds to one feature region; that is, connected component processing is implemented to facilitate the determination of feature regions. Each feature region corresponds to one item.

[0127] In this embodiment, the preset angle interval can be 60° or 30°. The smaller the preset angle interval, the more edge points of the determined feature region are, and the more accurate the first contour is obtained.

[0128] The beneficial effects of the above technical solution are as follows: Preprocessing the perceived image improves its accuracy while reducing data processing volume. Segmenting the target perceived image facilitates further identification of regions of interest and removal of regions of no interest, further reducing data processing volume. Based on connected component processing, feature regions are obtained, enabling the division of each item. By determining the center of each feature region and using the center as the pole of polar coordinates, rotation is performed at preset angular intervals to determine the coordinates of the edges of the feature regions, accurately defining the first contour of each item.

[0129] According to some embodiments of the present invention, the segmentation module includes:

[0130] The third determining module is used for:

[0131] The preprocessed perceptual image is input into a pre-trained convolutional neural network, which outputs a convolutional feature map.

[0132] The convolutional feature map is input into the Region Candidate Network (RPN), which outputs several segmented regions, and the segmented regions are subjected to Ro I pooling.

[0133] The local convolutional feature maps corresponding to the segmented regions after Ro I pooling are cropped using an interpolation algorithm to convert them into a fixed size. Then, convolution and max pooling are used to obtain the feature vector maps of the segmented regions.

[0134] R-CNN is applied to several feature vector maps to determine the feature vector corresponding to each feature vector map;

[0135] The matching module is used to match several feature vectors with preset feature vectors respectively. The segmentation region corresponding to the feature vector with the highest matching degree is taken as the region of interest. The image of the region of interest is used as the target perception image and extracted and processed.

[0136] The working principle of the above technical solution is as follows: In this embodiment, the Region Proposal Network (RPN) works by assuming an input image, after a series of convolutions in the backbone network, a feature map of size m*n is obtained, which divides the original image into m*n regions. The center of each region of the original image is represented by the coordinates of a pixel on this feature map. This results in several segmented regions.

[0137] In this embodiment, several feature vector maps are processed using R-CNN, including ReLU and two fully connected layer processing steps. R-CNN stands for Relation-CNN, and it was the first algorithm to successfully apply deep learning to object detection.

[0138] In this embodiment, the preset feature vector is the feature vector corresponding to the region of interest.

[0139] The beneficial effects of the above technical solution are as follows: Based on convolutional neural networks, region candidate networks (RPN), and R-CNN processing of several feature vector maps, the feature vector corresponding to each feature vector map is accurately determined; several feature vectors are matched with preset feature vectors respectively, and the segmentation region corresponding to the feature vector with the highest matching degree is accurately determined, thus obtaining the target perception image and extracting and processing it, reducing the amount of image processing and improving the processing speed.

[0140] According to some embodiments of the present invention, the vision system further includes:

[0141] The first generation module is used for:

[0142] Based on the matching relationship, determine the distance information and orientation angle information between each cargo and the six-axis collaborative robot;

[0143] The location code information of each cargo is determined based on distance and orientation angle information;

[0144] Motion planning information for a six-axis collaborative robot is generated based on the position encoding information.

[0145] The second generation module is used to generate motion path instructions based on the motion planning information;

[0146] The correction module is used for:

[0147] When the six-axis collaborative robot executes motion path instructions to the corresponding position, it acquires point cloud data of the goods to be grasped;

[0148] The first global point-pair feature of the point cloud data of the goods to be captured is determined, and the first hash table is built as the actual model with the feature as the key and the point pair as the value.

[0149] The actual model is matched with the theoretical model to determine the mapping relationship between the two; the theoretical model is generated by a second hash table based on the second global point-pair features of point cloud data of standard goods, with the features as keys and the point pairs as values.

[0150] The gripping parameters corresponding to the theoretical model are corrected according to the mapping relationship to obtain the corrected gripping parameters, which are then sent to the six-axis collaborative robot.

[0151] The six-axis collaborative robot grips the goods to be grasped based on the corrected gripping parameters.

[0152] The working principle of the above technical solution: In this embodiment, the location coding information is used to encode each item.

[0153] In this embodiment, the motion planning information is the order in which the six-axis collaborative robot picks up each item.

[0154] In this embodiment, generating motion path instructions based on the motion planning information includes: performing trajectory planning for the six-axis collaborative robot in Cartesian space based on the motion planning information, and determining the correspondence between the Cartesian space trajectory and the joint angles, angular velocities, and angular accelerations in each joint space of the six-axis collaborative robot;

[0155] Dynamic modeling is performed based on model parameters and dynamic analysis methods of a six-axis collaborative robot.

[0156] Based on trajectory planning and dynamic modeling, the output trajectory state of the six-axis collaborative robot at each moment is determined, and the joint angular position and joint angular velocity of each joint space are output.

[0157] Motion paths and trajectory commands are generated based on the joint angular positions and joint angular velocities of each joint space.

[0158] In this embodiment, point cloud data of the goods to be grabbed is acquired at the corresponding location; the corresponding location can be a preset distance directly above the goods.

[0159] In this embodiment, the principle of determining the first global point-pair features of the point cloud data of the goods to be grasped, and establishing a first hash table as the actual model using features as keys and point pairs as values, is based on point-pair feature processing. This is a 3D point cloud recognition algorithm. It does not use local features such as 3D feature descriptors, but instead employs global point-pair features. The algorithm can be divided into two parts: offline modeling and sampling matching. Offline modeling includes point-pair distances, the angle between point normal vectors and point-pair vectors. After sampling sufficient point-pair ppfs on the model surface, these features are written into a hash table, with features as keys and point pairs (sets) as values. Sampling matching determines the corresponding sampling points.

[0160] In this embodiment, the actual model is matched with the theoretical model to determine the mapping relationship between the two; the sampling points in the actual model are matched with the corresponding sampling points in the theoretical model to establish the matching relationship between the sampling points.

[0161] In this embodiment, standard goods are defined as goods whose placement posture, integrity, and shape are all preset standards.

[0162] The beneficial effects of the above technical solution are as follows: Based on the distance and orientation angle information between each cargo and the six-axis collaborative robot, the position coding information of each cargo is determined; motion planning information of the six-axis collaborative robot is generated according to the position coding information; motion path instructions are generated according to the motion planning information; the mapping relationship between the actual model and the theoretical model is determined by matching the two; the gripping parameters corresponding to the theoretical model are corrected based on the mapping relationship to obtain corrected gripping parameters; the six-axis collaborative robot grips the cargo to be gripped based on the corrected gripping parameters, which facilitates the determination of the gripping posture and gripping coordinates of the cargo to be gripped, and achieves accurate gripping.

[0163] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A mobile collaborative robot, characterized in that, include: Mobile robots, integrated control systems, six-axis collaborative robots, and vision systems; among them, The integrated control system is located inside the mobile robot and is used to control the mobile robot, the six-axis collaborative robot and the vision system to achieve coordinated operation of the three. The end of the six-axis collaborative robot is mounted on the mobile robot and is used to perform operations within the workspace. The vision system is located at the other end of the six-axis collaborative robot and is used to provide visual assistance when the six-axis collaborative robot is performing its work; The mobile robot is used to receive and execute movement commands from the integrated control system. The vision system includes: The lidar sensing module is used to acquire point cloud data of goods placed in the area to be grasped; The image acquisition module is used to acquire perceived images of goods placed in the area to be grasped; The first determining module is used to perform contour recognition based on the perceived image to determine the first contour corresponding to each item; The second determining module is used to perform contour recognition based on the point cloud data and the perceived image to determine the second contour corresponding to each item. The matching module is used to match the first contour with the second contour to obtain a matching relationship, and send the matching relationship to the six-axis collaborative robot; The first determining module includes: The preprocessing module is used to preprocess the perceived images; The segmentation module is used to segment the preprocessed perceptual image to obtain the target perceptual image; Connection module, used for: Obtain the pixel values ​​of each pixel in the target perception image, group the pixels with the same pixel values ​​into a set, and connect all the pixels in the set to obtain the feature region; each set corresponds to a feature region; The center of each feature region is determined, and using the center of the feature region as the pole of the polar coordinate system, the region is rotated at preset angular intervals to determine the coordinates of the edges of the feature region. Based on the connections between the coordinates, the first outline of the goods corresponding to the feature region is determined. The segmentation module includes: The third determining module is used for: The preprocessed perceptual image is input into a pre-trained convolutional neural network, which outputs... Convolutional feature maps; The convolutional feature map is input into the Region Candidate Network (RPN), which outputs several segmented regions, and the segmented regions are subjected to RoI pooling. The local convolutional feature maps corresponding to the segmented regions after RoI pooling are cropped using an interpolation algorithm to convert the local convolutional feature maps into a fixed size. Then, convolution and max pooling are used to obtain the feature vector maps of the segmented regions. R-CNN is applied to several feature vector maps to determine the feature vector corresponding to each feature vector map; The matching module is used to match several feature vectors with preset feature vectors respectively. The segmentation region corresponding to the feature vector with the highest matching degree is taken as the region of interest. The image of the region of interest is used as the target perception image and extracted and processed.

2. The mobile collaborative robot as described in claim 1, characterized in that, The mobile robot includes a mobile robot body and a mobile control module: wherein, The mobile robot body includes: a drive wheel assembly, movable casters, anti-collision strips, an inspection door, a left side door of the MC30, a left side door of the middle frame, a right side door of the MC30, and a right side door of the middle frame; The mobile control module includes: a lidar unit, a power battery unit, a communication unit, a control unit and a power supply unit, a charging port, a mobile indicator light, and a depth camera; The movement indicator light is used to display the working status of the mobile robot body; The lidar unit is used to acquire lidar information of the mobile robot body during its movement. Depth cameras are used to acquire scene images of the mobile robot as it moves.

3. The mobile collaborative robot as described in claim 1, characterized in that, The integrated control system includes: a mobile robot control module, robotic arm indicator lights, an emergency stop switch, a robotic arm power control box, an application expansion box, a control panel assembly, and a teach pendant; wherein, The mobile robot control module is used to generate control commands for the mobile robot based on lidar information and scene images; The robotic arm indicator light is used to display the working status of the six-axis collaborative robot; The control panel components include a USB interface, a power button, a rotary power switch, a power off button, and an antenna.

4. The mobile collaborative robot as described in claim 3, characterized in that, The teach pendant is used for mobile robot operation control, map building, task editing and distribution, and viewing status and information logs, warning logs, and error logs.

5. The mobile collaborative robot as described in claim 1, characterized in that, The six-axis collaborative robot is a six-axis collaborative robotic arm; the six-axis collaborative robotic arm includes a base, communication and power cables, J1-J2 modules, a lower arm, J3-J4 modules, an upper arm, J5-J6 modules, tool I / O, and a tool flange; wherein, The communication and power cables are disposed on the side of the base; The J1-J2 modules are mounted on the base; The J1-J2 modules, lower arm, J3-J4 modules, upper arm, J5-J6 modules, tool IO, and tool flange are connected in sequence.

6. The mobile collaborative robot as described in claim 5, characterized in that, Also includes: The display module is used to determine the distance between the end-load center of the six-axis collaborative robotic arm and the center of the mounting tool flange, query the relationship between the preset maximum load and the center distance, obtain the maximum allowable load, and display it.

7. The mobile collaborative robot as described in claim 1, characterized in that, The vision system also includes: The first generation module is used for: Based on the matching relationship, determine the distance information and orientation angle information between each cargo and the six-axis collaborative robot; The location code information of each cargo is determined based on distance and orientation angle information; Motion planning information for a six-axis collaborative robot is generated based on the position encoding information. The second generation module is used to generate motion path instructions based on the motion planning information; The correction module is used for: When the six-axis collaborative robot executes motion path instructions to the corresponding position, it acquires point cloud data of the goods to be grasped; The first global point-pair feature of the point cloud data of the goods to be captured is determined, and the first hash table is built as the actual model with the feature as the key and the point pair as the value. The actual model is matched with the theoretical model to determine the mapping relationship between the two; the theoretical model is generated by a second hash table based on the second global point-pair features of point cloud data of standard goods, with the features as keys and the point pairs as values. The gripping parameters corresponding to the theoretical model are corrected according to the mapping relationship to obtain the corrected gripping parameters, which are then sent to the six-axis collaborative robot. The six-axis collaborative robot grips the goods to be grasped based on the corrected gripping parameters.

Images