A swarm system and control method for air-suction dual-arm handling robots

Abstract

This invention discloses a suction-type dual-arm handling robot cluster system and control method, comprising: a scheduling system and several handling robots. The scheduling system can wirelessly communicate with the handling robots. Based on user instructions, the scheduling system determines the operating mode, selects the corresponding platform to establish communication, obtains delivery information through the platform, and dynamically allocates it to the handling robots. The handling robots can communicate with each other, forming a cluster system through information sharing, facilitating the viewing of each robot's specific information and unified management. The handling robots can identify and calculate the weight of the target item, and then, based on the weight, move the target item to the required delivery location. The suction method minimizes damage to the target item, protecting delicate items during transport.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of carrying robots, in particular to an air-suction double-arm carrying robot cluster system and a control method. BACKGROUND

[0002] With the continuous development of e-commerce, the logistics industry has developed rapidly. The work of the logistics industry is becoming more and more busy. In the carrying scene of more goods, manual distribution often cannot quickly and accurately distribute goods to the corresponding place, and the automation rate of distribution is low.

[0003] In the prior art, most of the robots are used for picking, sorting and delivering goods. However, the robot mainly uses mechanical hard clamping, or pushes the goods out, or pulls the goods through the mechanism, which is easy to cause the goods to deform or even be damaged, and increases the compensation risk. And most of the current carrying robots are single robots, each robot is relatively independent, which is not conducive to unified scheduling, is difficult to control as a whole, and is difficult to improve the overall operation efficiency, and the number of robots required for goods transportation is often large. SUMMARY

[0004] The purpose of the present application is to provide an air-suction double-arm carrying robot cluster system and a control method to solve one or more technical problems existing in the prior art, at least to provide a beneficial choice or to create conditions.

[0005] The solution to the technical problem of the present application is to provide an air-suction double-arm carrying robot cluster system and a control method.

[0006] In an embodiment of the first aspect of the present application, an air-suction double-arm carrying robot cluster system comprises a scheduling system and a plurality of carrying robots;

[0007] The scheduling system and the plurality of carrying robots are in wireless communication, the carrying robots are in wireless communication, the scheduling system is used to obtain a user instruction, determine a running mode according to the user instruction, and establish communication with a required platform to obtain distribution information, and dynamically distribute the distribution information to the plurality of carrying robots;

[0008] Correspondingly, the carrying robot is used to obtain current position information, move to a target object according to the current position information and corresponding distribution information, identify the weight of the target object, and correspondingly suck and carry the target object to a required delivery position according to the weight of the target object.

[0009] Further, the carrying robot comprises a storage rack, a moving device, an air-suction carrying device, an image recognition module and a positioning and navigation module.

[0010] The positioning navigation module is configured to acquire current position information, and control the mobile device to move to the target object and the required delivery location according to the current position information and corresponding delivery information.

[0011] The image recognition module is installed on the air suction conveying device, and is configured to recognize the weight of the target object according to the corresponding delivery information.

[0012] The air suction conveying device is located on one side of the storage rack, and is configured to suck and convey the target object and place it on the storage rack according to the weight and the corresponding delivery information.

[0013] Further, the air suction conveying device comprises a lifting platform, a rib plate, a first mechanical arm, a second mechanical arm and a suction disc.

[0014] The rib plate is installed on the lifting platform, the height of the lifting platform is consistent with the height of the storage rack, and the lifting platform controls the height of the rib plate according to the corresponding delivery information.

[0015] The first mechanical arm and the second mechanical arm are each provided with a suction disc, and the suction disc is configured to adjust the suction force to suck the target object according to the weight.

[0016] The first mechanical arm is installed on the top plate of the rib plate, the second mechanical arm is installed on the bottom plate of the rib plate, and the first mechanical arm and the second mechanical arm are configured to convey the target object and place it on the storage rack according to the corresponding delivery information.

[0017] Further, the mobile device comprises a Mecanum wheel, an encoder counter and a driving motor.

[0018] The driving motor is connected with the Mecanum wheel and the encoder counter respectively, the encoder counter is connected with the positioning navigation module, and the Mecanum wheel, the encoder counter and the driving motor are all arranged at the bottom of the storage rack, and the encoder counter is configured to acquire current speed information.

[0019] Further, the positioning navigation module comprises a ranging module, a laser radar module and an IMU module.

[0020] The ranging module is arranged around the bottom of the storage rack, the ranging module is connected with the driving motor, and is configured to detect the distance value from the obstacle to the storage rack to drive the driving motor to avoid obstacles.

[0021] The laser radar module and the IMU module are both arranged at the top of the storage rack, the laser radar module is configured to acquire radar scanning information, and the IMU module is configured to acquire current movement information.

[0022] The control method of the air suction type dual-arm carrying robot cluster system of the embodiment of the second aspect of the application applies the air suction type dual-arm carrying robot cluster system of the embodiment of the first aspect of the application, and the control method comprises the following steps of:

[0023] The scheduling system obtains a user instruction, determines a running mode according to the user instruction, and establishes communication with a required platform according to the running mode;

[0024] The scheduling system obtains distribution information through the required platform and dynamically distributes the distribution information to a plurality of carrying robots, wherein the distribution information comprises target position, type and map information of a target object in a warehouse;

[0025] The corresponding carrying robot obtains current position information, plans a first path according to the current position information, the map information and the target position, and moves to the target object according to the first path;

[0026] Image information of the target object is collected, the image information is recognized and processed to obtain size information of the target object, and the weight of the target object is determined according to the size information and the type;

[0027] According to the weight, the corresponding carrying robot sucks and carries the target object, again obtains current position information, plans a second path according to a required delivery position, and moves to the required delivery position according to the second path.

[0028] Further, the collection of the image information of the target object further comprises the following steps of:

[0029] An image recognition module is used to calibrate the first mechanical arm and the second mechanical arm respectively.

[0030] Further, the obtaining process of the current position information specifically comprises the following steps of:

[0031] Current movement information, current speed information and radar scanning information are obtained, a point cloud map is constructed, and a current pose is obtained;

[0032] The current movement information is taken as a prediction value, the radar scanning information and the current pose are taken as observation values, a Kalman filtering algorithm is used to fuse the current movement information, the current speed information and the radar scanning information, and the current pose is obtained;

[0033] An LAMA positioning algorithm is used to obtain positioning information of the corresponding carrying robot, and the current position information is obtained according to the current pose and the positioning information.

[0034] Further, the control method further comprises the following steps of:

[0035] The scheduling system obtains a verification instruction and encapsulates running adjustable parameters of the corresponding carrying robot;

[0036] The scheduling system obtains a cluster verification algorithm, initializes current position information of the corresponding carrying robot, adjusts the running adjustable parameter according to the cluster verification algorithm, and schedules a plurality of carrying robots.

[0037] Further, the dynamic distribution of the distribution information specifically includes:

[0038] The scheduling system obtains the current running time and the running state corresponding to the plurality of carrying robots;

[0039] According to the running mode, the current running time and the corresponding running state, the number of running carrying robots and the number of standby carrying robots are determined, and the distribution information is dynamically distributed to the running carrying robots.

[0040] The beneficial effects of the present application are that the scheduling system can wirelessly communicate with a plurality of carrying robots, the scheduling system determines the running mode according to the user instruction, selects the required platform corresponding to the mode to establish communication, obtains the distribution information through the required platform, and dynamically distributes it to a plurality of carrying robots. A plurality of carrying robots can communicate with each other, utilize information sharing between a plurality of carrying robots, form a cluster system, facilitate viewing of specific information of each carrying robot, and unified management. The carrying robot can identify and calculate the weight of the target object, so as to correspondingly carry the target object to the required delivery position according to the weight of the target object, through the suction mode, it is not easy to cause damage to the target object, and it can play a role in protecting precise objects during the carrying process. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a front view schematic diagram of part of the structure of an air-suction type double-arm carrying robot provided by an embodiment of the present application;

[0042] Figure 2 is a top view schematic diagram of part of the structure of an air-suction type double-arm carrying robot provided by another embodiment of the present application;

[0043] Figure 3 is a bottom view schematic diagram of part of the structure of an air-suction type double-arm carrying robot provided by another embodiment of the present application;

[0044] Figure 4 is a flowchart of a control method of an air-suction type double-arm carrying robot cluster system provided by an embodiment of the present application;

[0045] Figure 5 is a positioning simulation schematic diagram of a control method of an air-suction type double-arm carrying robot cluster system provided by an embodiment of the present application.

[0046] 100, shelf, 110, shelf layer, 120, vehicle-mounted display, 130, speaker, 140, wireless charging module, 200, suction type carrying device, 210, rib plate, 220, suction cup, 221, electric control air pump, 230, lifting platform, 240, first mechanical arm, 250, second mechanical arm, 300, image recognition module, 400, mobile device, 410, driving motor, 420, Mecanum wheel, 430, encoder counter, 500, distance measuring module, 510, laser radar module, 520, IMU module. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0048] In the description of the present application, it should be noted that, unless otherwise explicitly defined, the words such as setting, installation, connection, etc. should be understood broadly, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical scheme.

[0049] It should be noted that although the functional modules are divided in the system schematic diagram, in some cases, the module division in the system can be different. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0050] In some embodiments of the first aspect of the present application, with reference to Figure 1 and Figure 3 A suction type dual-arm carrying robot cluster system comprises a scheduling system and a plurality of carrying robots.

[0051] Each carrying robot is in wireless communication with the scheduling system, and each carrying robot can communicate with each other. A master-slave communication mode is adopted, and the mutual communication between each carrying robot is utilized to form a cluster system. In the prior art, most of the carrying robots are single robots, and each robot is relatively independent, which is not convenient for scheduling and overall control. In the present embodiment, each carrying robot shares information, which is convenient for checking the specific information of each carrying robot, unified management and improving the overall operation efficiency.

[0052] The user selects the operation mode through the scheduling system, the scheduling system determines the operation mode in response to the user's instruction, determines the required platform that needs to be connected through the operation mode, and the scheduling system communicates wirelessly with the required platform.

[0053] The scheduling system obtains the distribution information from the required platform, and dynamically distributes the distribution information, so that the carrying robot receives the corresponding distribution information.

[0054] After the corresponding carrying robot receives the corresponding distribution information, the current position information of the carrying robot is obtained. The corresponding carrying robot moves to the target object after obtaining the current position information and the corresponding distribution information, and performs image recognition detection on the target object to obtain the weight of the target object.

[0055] According to the weight of the target object, the corresponding carrying robot sucks the target object and carries the target object to realize the delivery of the target object to the required delivery position. Compared with the mechanical hard clamping method in the prior art, the carrying robot can identify and calculate the weight of the target object, so as to carry the target object to the required delivery position according to the weight of the target object. Through the sucking method, the target object is not easily damaged, and the precise object can be protected during the carrying process.

[0056] Reference Figures 1 to 3 In some embodiments of the present application, the carrying robot comprises a storage rack 100, a moving device 400, a control module, an image recognition module 300, a positioning navigation module and a suction type carrying device 200.

[0057] The storage rack 100 is provided with a plurality of storage layers 110, and the moving device 400 is arranged at the bottom of the storage rack 100 and used to drive the storage rack 100 to move. The control module is in wireless communication with the scheduling system.

[0058] The positioning navigation module is connected with the moving device 400 and the control module respectively, and is used to detect the current position to obtain the current position information. Through the current position information and the corresponding distribution information, the moving device 400 is driven to move and drive the storage rack 100 to walk to the target object, and the moving device 400 is driven to move and drive the storage rack 100 to carry the target object to the required delivery position.

[0059] The image recognition module 300 is connected with the control module, and the image recognition module 300 is arranged on the suction type carrying device 200. The image recognition module 300 collects image information of the target object, and identifies the image information to obtain the weight of the target object by using the corresponding distribution information.

[0060] The image recognition module 300 comprises an RGB camera.

[0061] The air suction conveying device 200 is located on one side of the storage shelf 100, and is connected with the control module. The air suction conveying device 200 sucks the target object according to the weight and corresponding delivery information, and conveys the target object to the corresponding storage layer 110 of the storage shelf 100, so as to deliver the target object to the required delivery position.

[0062] The pressure sensor is arranged on each storage layer 110, and is used to obtain the pressure value of each storage layer 110. The pressure sensor is connected with the control module, so that whether the pick-up is successful or the put-in is successful can be determined.

[0063] Referring to Figures 1 to 3 In some embodiments of the present application, the conveying robot further comprises a battery compartment, a wireless charging module 140, a vehicle-mounted display 120 and a loudspeaker 130.

[0064] The battery compartment is connected with the control module and the wireless charging module 140 respectively, and is arranged at the bottom of the storage shelf 100. The battery compartment is provided with a soft silica gel material for heat insulation and shock absorption and a temperature sensor. When the temperature of the battery compartment is abnormal, the control module controls the loudspeaker 130 to broadcast, and controls the positioning navigation module to upload the current position information to the dispatching system, so that the user can take reaction measures.

[0065] When the power of the conveying robot decreases to a set threshold value, an optimal route to the nearest ground charging pile is planned through the positioning navigation module.

[0066] The vehicle-mounted display 120 is connected with the control module. The vehicle-mounted display 120 is used to display the delivery task performed by the corresponding conveying robot, the running state, the power information of the battery compartment, whether each part is connected, etc., so that the user can more intuitively understand the vehicle condition.

[0067] The loudspeaker 130 is connected with the control module. The loudspeaker 130 is used for the delivery task performed by the corresponding conveying robot, to remind the user to avoid or take the object, etc.

[0068] Referring to Figures 1 to 3 In some embodiments of the present application, the air suction conveying device 200 comprises a rib plate 210, a suction cup 220, a lifting platform 230, a first mechanical arm 240 and a second mechanical arm 250.

[0069] The height of the lifting platform 230 is the same as the height of the storage shelf 100. The rib plate 210 is installed on the lifting platform 230 and located at the middle part of the lifting platform 230. The lifting platform 230 is connected with the control module. When the lifting platform 230 moves up and down, the rib plate 210 can be driven to rise or fall.

[0070] The lifting platform 230 comprises a stepping motor and a screw rod. The stepping motor is connected to the control module. One end of the fixed part of the screw rod is connected to the top of the storage rack 100, and the other end of the fixed part of the screw rod is connected to the bottom of the storage rack 100. The sliding part of the screw rod is connected to the rib plate 210.

[0071] The first mechanical arm 240 is provided with a suction cup 220. The first mechanical arm 240 is arranged on the top plate of the rib plate 210 and is connected to the control module. The first mechanical arm 240 sucks the target object through the suction cup 220 and places the target object on the corresponding storage layer 110 of the storage rack 100.

[0072] The second mechanical arm 250 is provided with a suction cup 220. The second mechanical arm 250 is arranged on the bottom plate of the rib plate 210 and is connected to the control module. The second mechanical arm 250 sucks the target object through the suction cup 220 and places the target object on the corresponding storage layer 110 of the storage rack 100.

[0073] The first mechanical arm 240 and the second mechanical arm 250 are both five-degree-of-freedom mechanical arms. The two mechanical arms are arranged on the rib plate 210 in a relative manner. The height of the rib plate 210 can be adjusted by the lifting platform 230, so that the height of the two mechanical arms can be adjusted. The two mechanical arms can suck objects in multiple ways, and can simplify the scheduling complexity of multiple mechanical arms and divide the object space simply, thereby improving the object taking efficiency of the mechanical arms.

[0074] The suction cup 220 is connected to the control module. The suction cup 220 can adjust the suction amount and control the suction force of the suction cup 220 according to the weight of the target object. The target object can be adsorbed on the suction cup 220 and then transported to the corresponding storage layer 110 of the storage rack 100 by the corresponding mechanical arm.

[0075] The suction cup 220 comprises an electrically controlled air pump 221 connected to the control module. The electrically controlled air pump 221 is used to suck the air in the suction cup 220 to adjust the suction amount and control the suction force of the suction cup 220.

[0076] In this embodiment, the position and quantity of the target object are determined according to the corresponding distribution information. It is determined whether there are several target objects in the current shelf in the warehouse. If yes, the target objects within the optimal distance range are selected according to the target position. The optimal height to which the rib plate 210 is adjusted by the lifting platform 230 is determined according to the target objects within the optimal distance range, and the corresponding mechanical arm is determined to be called. The optimal height can make the two mechanical arms easily and quickly obtain the corresponding target objects.

[0077] For one of the mechanical arms, the pose of the current mechanical arm and the target position are obtained, the target object is selected, and the optimal storage layer 110 for placing the target object is selected according to the target object, the pressure value of the storage layer 110 corresponding to the storage shelf 100, and the pose of the current mechanical arm, and the optimal object taking path is planned. The selection of the target object and the optimal storage layer 110 can facilitate the double mechanical arms to quickly obtain the corresponding target object, improve the object taking efficiency, and the plurality of storage layers 110 arranged on the storage shelf 100 can facilitate the mechanical arm to place the target object on the nearest storage layer 110.

[0078] With reference to Figures 1 to 3 In some embodiments of the present application, the mobile device 400 includes a driving motor 410, a Mecanum wheel 420, and a code counter 430.

[0079] The driving motor 410, the Mecanum wheel 420, and the code counter 430 are all installed at the bottom of the storage shelf 100. The driving motor 410 is connected with the code counter 430, and the driving motor 410 is connected with the Mecanum wheel 420. The code counter 430 obtains the current speed information of the carrying robot.

[0080] The Mecanum wheel 420 can rotate 360° in place and move up, down, left and right. Compared with the four-wheel differential steering or Ackerman steering structure of the carrying robot in the prior art, the Mecanum wheel 420 of the present application does not have requirements for the warehouse where the goods are located, which is beneficial to improve the space utilization rate of the warehouse area. The use of a driving motor 410 with large torque can realize the carrying of heavier goods, and the use of a code counter 430 can realize the detection of speed.

[0081] With reference to Figures 1 to 3 In some embodiments of the present application, the positioning and navigation module includes a ranging module 500, a laser radar module 510, and an IMU module 520.

[0082] The ranging module 500 is installed around the bottom of the storage shelf 100, and is connected with the control module. The ranging module 500 is used to perceive obstacles, obtain the distance value from the obstacles to the storage shelf 100, and send the distance value to the control module. According to the distance value, the dynamic window method is used to realize the obstacle avoidance control of the driving motor 410.

[0083] The laser radar module 510 is installed at the top of the storage shelf 100, and is connected with the control module. The laser radar module 510 is used to obtain radar scanning information, and facilitate the obtaining of the current position information of the robot and the planning of the path.

[0084] The IMU module 520 is mounted on the top of the shelf 100, and is connected with the control module to obtain current movement information, so as to obtain current position information of the robot and plan a path. The current movement information includes vehicle speed, yaw angle and the like.

[0085] With reference to Figure 4 In some embodiments of the second aspect of the application, a control method of an air suction type dual-arm carrying robot cluster system is applied to the air suction type dual-arm carrying robot cluster system of the first aspect of the application. The control method comprises the following steps:

[0086] In S100, the scheduling system obtains a user instruction, determines a running mode according to the user instruction, and establishes communication with a required platform according to the running mode.

[0087] In S200, the scheduling system obtains delivery information through the required platform, and dynamically distributes the delivery information to a plurality of carrying robots.

[0088] In S300, a corresponding carrying robot obtains current position information, plans a first path according to the current position information, map information and a target position, and moves to the target object according to the first path.

[0089] In S400, image information of the target object is collected, the image information is recognized and processed to obtain size information of the target object, and the weight of the target object is determined according to the size information and the type.

[0090] In S500, the corresponding carrying robot sucks and carries the target object according to the weight, obtains current position information again, plans a second path according to a required delivery position, and moves to the required delivery position according to the second path.

[0091] In this embodiment, the scheduling system is initialized, a user issues a user instruction, a running mode is selected, and the running mode of the robot cluster system is determined according to the user instruction.

[0092] According to the running mode, the scheduling system is connected to a required platform to obtain delivery information therefrom, and dynamically distributes corresponding delivery information to each carrying robot to form a carrying robot cluster.

[0093] With reference to Figure 5 In this embodiment, the running mode includes an express station sorting and storage mode and a hospital medicine delivery mode. When the express station sorting and storage mode is determined, the scheduling system is connected to an express information in-out warehouse platform to distribute corresponding delivery information to each carrying robot. When the hospital medicine delivery mode is determined, the scheduling system is connected to a hospital medicine delivery information platform to distribute corresponding delivery information to the carrying robots.

[0094] The distribution information includes target position of the target item, type of the target item, and map information of a warehouse where the target item is located.

[0095] For any one of the carrying robots, the corresponding distribution information is received, current position information is acquired, a first path to the target item is planned according to the current position information, the map information, and the target position, so as to move the carrying robot from the current position to the target item, that is, to the corresponding shelf in the warehouse, and the target item is placed on the shelf.

[0096] After the carrying robot reaches the target item, the image acquisition module acquires image information of the target item, recognizes the image information, obtains size information of the item, calculates the weight according to the size information and the type.

[0097] According to the weight, the suction cup is adjusted, and the target item is carried to a storage layer on the storage shelf by the mechanical arm. After the item is taken, the current position information is acquired again, a second path to the required delivery position is planned according to the current position information and the required delivery position, so as to move the carrying robot from the current position to the required delivery position.

[0098] The first path and the second path are planned by using a global navigation algorithm.

[0099] The carrying robots carry the target items in an orderly manner by the above method, form a robot cluster, and can complete heavy and repetitive important material carrying work. The safety of the transportation work is ensured, the whole process of the logistics transportation can be monitored, and the distribution information closed loop can be realized, so that the cost of the logistics transportation is effectively reduced, and the operating efficiency is improved.

[0100] In some embodiments of the second aspect of the application, in S200, the dynamic allocation of the scheduling system specifically includes:

[0101] In S210, the scheduling system acquires the current running time and the corresponding running state uploaded by each carrying robot.

[0102] In S220, according to the running mode, the current running time, and the corresponding running state, the number of standby carrying robots is determined, the remaining carrying robots are operated, and the distribution information is dynamically allocated to the operating carrying robots.

[0103] In this embodiment, each operating mode exhibits a certain regularity and minimal fluctuations. By analyzing the operating mode and the current operating time, it is determined whether the cluster system is in a peak working period. If not, based on the operating status of each handling robot, some handling robots are put into standby mode for charging and maintenance, while the remaining handling robots receive delivery information dynamically allocated by the scheduling system and continue operating to meet the hospital's specific drug delivery needs.

[0104] For example, during hospital working hours, approximately 60% of the medications need to be delivered in the morning. If the system is operating in hospital medication delivery mode and the current operating time is in the afternoon, then the cluster system is considered to be in a non-peak operating period. Based on the operating status of each handling robot, some robots that need charging or maintenance enter standby mode, while the remaining robots enter working mode, receiving delivery information dynamically assigned by the scheduling system and continuing to operate.

[0105] In some embodiments of the second aspect of the present invention, the process of obtaining current location information specifically includes:

[0106] The S600 acquires current movement information through the IMU module, current speed information through the code counter, and radar scan information through the LiDAR module. Based on the radar scan information, it constructs a point cloud map and matches the radar scan information with the point cloud map to obtain the initial pose.

[0107] S610 uses the current movement information as the predicted value and the radar scan information and current pose as the observed value. It uses the Kalman filter algorithm to fuse the current movement information, current speed information and radar scan information to obtain the current pose.

[0108] The S620 uses the LAMA positioning algorithm to obtain the positioning information of the corresponding handling robot, and obtains the current position information based on the current position and the positioning information.

[0109] In this embodiment, S610 specifically includes:

[0110] S611, perform initial localization on the point cloud map and assign values ​​to the following variables: Initial position; Initial velocity; Initial pose.

[0111] S612, Initialize the Kalman filter algorithm, state variables but

[0112]

[0113]

[0114]

[0115] ,in, Let Q be the variance, Q be the process noise, and R0 be the observation noise. The process noise Q and the observation noise R0 remain unchanged during the iteration process.

[0116] S613 performs inertial calculations, which include attitude calculation, velocity calculation, and position calculation.

[0117] Attitude calculation: in,

[0118] Speed ​​calculation: in,

[0119] Position calculation:

[0120] S614, update the predicted values, and execute the first two steps of the Kalman five-step process, namely...

[0121]

[0122]

[0123] Where F is the state transition matrix and B is the control matrix.

[0124] S615, when there are no observations, the posterior is updated. When there are no observations, the remaining three steps of the Kalman test are not required; the posterior equals the prior.

[0125]

[0126]

[0127]

[0128] S616, when there are observations, the measurement is updated, and the posterior state variable is:

[0129]

[0130]

[0131]

[0132] S617, when there are observations, calculate the posterior pose and update the posterior pose based on the posterior state variables:

[0133]

[0134]

[0135]

[0136]

[0137]

[0138] S618, the state variable is cleared to zero. The state variable has already been used for compensation, so it needs to be cleared.

[0139]

[0140] This is to keep the posterior variance constant.

[0141] S617, output pose, including the posterior pose. As the current pose.

[0142] Compared with existing single-sensor technologies, the use of multi-sensor information fusion technology can enhance system survivability, improve the reliability and robustness of the entire system, enhance data credibility, improve accuracy, expand the system's temporal and spatial coverage, and increase the system's real-time performance and information utilization in solving problems such as detection, tracking, and target recognition.

[0143] In some embodiments of the second aspect of the present invention, in S400, the process of determining the weight of the target article specifically includes:

[0144] S410, the image information is converted to grayscale to obtain a first image, and the first image is subjected to bilateral filtering to obtain a second image.

[0145] S420 uses the Canny operator to perform edge detection on the second image and uses the AprilTag visual reference system to determine the relative position of the target object, thereby obtaining the size information of the target object.

[0146] S430: Based on the size information, determine the volume of the target item, and based on the volume and type, calculate the weight of the target item.

[0147] In this embodiment, an RGB camera is used to capture images of the target object. The image information is in RGB format. Grayscale images have only one color channel, which only reflects the brightness of the pixel. The larger the value, the whiter the pixel, and the smaller the value, the darker the pixel. Converting the image to grayscale can reduce the amount of computation.

[0148] Bilateral filtering, while preserving the details of the target image, suppresses noise in the first image, improving the speed and accuracy of image processing. Based on Gaussian filtering, bilateral filtering considers the numerical differences between pixels, using a weighted average of the gray values ​​of surrounding pixels to replace the gray value of a single pixel. A bilateral filter centered at q can be expressed as:

[0149] Where y(p) is the noise pixel; F is the neighborhood of size (2r+1)*(2r+1) centered at q; W σs For the space kernel, W σr This is the range kernel. The spatial kernel and the range kernel are: |pq| 2 For the distance in the spatial domain, σ s This is a weighted value representing the intensity of the spatial domain's influence. |y(p)-y(q)| 2 σ represents the difference intensity of pixels. r This is a weighted value representing the intensity of numerical differences.

[0150] Edge detection algorithms facilitate the acquisition of the target object's side length, thereby helping to determine its volume. The control module adjusts the air intake based on the obtained weight and the set weight range, thus changing the suction power.

[0151] In some embodiments of the second aspect of the invention, S400 further includes:

[0152] S440, before acquiring image information, the image recognition module performs hand-eye calibration on the first and second robotic arms.

[0153] In this embodiment, the specific process of hand-eye calibration includes:

[0154] S441, Let G be the end effector, B be the calibration coordinate system, C be the camera coordinate system, and W be the robot coordinate system. The transformation relationship is as follows: in, To determine the pose of the calibration board relative to the robot, This represents the pose relationship between the end effector and the robot. The pose relationship between the RGB camera and the end effector is the desired quantity for hand-eye calibration. The pose relationship between the calibration board and the RGB camera is the extrinsic parameter obtained from the RGB camera calibration.

[0155] S442, Keep the calibration board stationary and take images of the calibration board from three different positions. Name the three positions as: initial position, first calibration position and second calibration position.

[0156] S443, obtain the pose matrices H0, H1, and H2 corresponding to the transformed end-effector pose. Based on the above pose matrices, obtain the transformation matrices corresponding to the two movements of the robotic arm. Where A1 is the first transformation matrix of the robotic arm from the initial position to the first target position, and A2 is the second transformation matrix of the robotic arm from the initial position to the second target position.

[0157] S444: Using RGB camera calibration, obtain the extrinsic parameter matrix M0 corresponding to the initial position, the extrinsic parameter matrix M1 corresponding to the first calibration position, and the extrinsic parameter matrix M2 corresponding to the second calibration position. Based on the above extrinsic parameter matrices, obtain the transformation matrix corresponding to the two movements of the RGB camera. Wherein, B1 is the first transformation matrix from the initial position to the first calibration of the RGB camera, and B2 is the second transformation matrix from the initial position to the second calibration of the RGB camera;

[0158] S445, based on the first transformation matrix A1 of the robotic arm, the second transformation matrix A1 of the robotic arm, the first transformation matrix B1 of the RGB camera, and the second transformation matrix B2 of the RGB camera, the hand-eye relationship matrix is ​​calculated. To enable control of the robotic arm.

[0159] In some embodiments of the second aspect of the present invention, the control method further includes:

[0160] S700: The scheduling system obtains verification instructions and encapsulates the corresponding adjustable operating parameters of the handling robot.

[0161] S710: The scheduling system obtains the cluster verification algorithm, initializes the current position information of the corresponding handling robot, and adjusts the encapsulated adjustable operating parameters according to the cluster verification algorithm to achieve the scheduling of several handling robots.

[0162] In this embodiment, the scheduling system receives the verification command issued by the user, enters the robot cluster experimental verification mode, and encapsulates the adjustable operating parameters of each handling robot into a class attribute. These adjustable operating parameters include: robot x and y axis velocities, robot angular velocity around the z-axis, robot linear acceleration in the x and y directions, angular acceleration around the z-axis, maximum angular velocity, maximum linear velocity, radar information transmission frequency, radar maximum and minimum sampling distances, minimum distance to obstacles, global map expansion radius, cost map expansion radius, and obstacle expansion distance, etc.

[0163] The scheduling system obtains the cluster verification algorithm based on the verification instructions. Each transport robot is initialized, including its current position. The cluster verification algorithm includes: encirclement algorithm verification and cluster formation movement algorithm verification. Based on the cluster verification algorithm, the system modifies the parameters in the class attributes of the class object, allowing users to find suitable parameters for adjustment to achieve the scheduling of several transport robots. This enables the use of the transport robot cluster system as a verification platform for the cluster algorithm, verifying its feasibility and facilitating the later deployment of the control algorithm to the transport robot cluster system for item handling.

[0164] Users can use the RVIZ visualization software in the Ubuntu system to observe the real-time location information of several transport robots.

[0165] The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A control method of an air-breathing dual-arm transport robot cluster system, characterized by, The system comprises a scheduling system and a plurality of carrying robots; The scheduling system is in wireless communication with the plurality of carrying robots, and the carrying robots are in wireless communication with each other; the scheduling system is configured to obtain a user instruction, determine an operation mode according to the user instruction, establish communication with a required platform, obtain delivery information, and dynamically distribute the delivery information to the plurality of carrying robots; Correspondingly, the carrying robots are configured to obtain current position information, move to a target object according to the current position information and corresponding delivery information, identify the weight of the target object, and correspondingly suck and carry the target object to a required delivery location according to the weight of the target object; the carrying robots comprise an image recognition module, which comprises an RGB camera; The control method comprises: The scheduling system obtains a user instruction, determines an operation mode according to the user instruction, and establishes communication with a required platform according to the operation mode; The scheduling system obtains delivery information through the required platform, and dynamically distributes the delivery information to the plurality of carrying robots, wherein the delivery information comprises target position information, type information, and map information of a warehouse where a target object is located; Correspondingly, the carrying robots obtain current position information, plan a first path according to the current position information, map information, and target position information, and move to the target object according to the first path; Image information of the target object is collected, the image information is identified to obtain size information of the target object, and the weight of the target object is determined according to the size information and the type; Correspondingly, the carrying robots suck and carry the target object according to the weight, obtain current position information again, plan a second path according to a required delivery location, and move to the required delivery location according to the second path; Before the image recognition module collects image information, the first mechanical arm and the second mechanical arm are calibrated by the image recognition module, and the process of the hand-eye calibration comprises: Let G be the end effector, B be the calibration coordinate system, C be the camera coordinate system, and W be the robot coordinate system. The transformation relationship is: wherein, is the pose of the calibration board relative to the robot, is the pose relationship of the end effector relative to the robot, is the pose relationship of the RGB camera relative to the end effector, is the pose relationship of the calibration board relative to the RGB camera; The calibration board is kept stationary, and images of the calibration board are collected by photographing the calibration board from three different positions, which are named as an initial position, a first calibration position, and a second calibration position, respectively; The corresponding pose matrix obtained by transforming the end pose According to the above pose matrix, a transformation matrix corresponding to the two motions of the robot arm is obtained , wherein, is a first transformation matrix of the robot arm from the initial position to the first calibration position, is a second transformation matrix of the robot arm from the initial position to the second calibration position; An initial position corresponding external parameter matrix is obtained by using RGB camera calibration A first calibration position corresponding external parameter matrix A second calibration position corresponding external parameter matrix A transformation matrix corresponding to two motions of the RGB camera is obtained according to the above external parameter matrices , Wherein, The first transformation matrix of the RGB camera from the initial position to the first calibration position, The second transformation matrix of the RGB camera from the initial position to the second calibration position; a first transformation matrix of the robot arm a second transformation matrix of the robot arm a first transformation matrix of the RGB camera a second transformation matrix of the RGB camera a hand-eye relationship matrix to control the robot arm.

2. The control method of claim 1, wherein The carrying robots comprise a storage rack, a moving device, a suction-type carrying device, and a positioning navigation module; The positioning navigation module is configured to obtain current position information, control the moving device to move to a target object and a required delivery location according to the current position information and corresponding delivery information; The image recognition module is configured to identify the weight of the target object according to the corresponding delivery information; The suction-type carrying device is located on one side of the storage rack and is configured to suck and carry the target object according to the weight and the corresponding delivery information, and place the target object on the storage rack correspondingly.

3. The control method of claim 2, wherein The suction-type carrying device comprises a lifting platform, a rib plate, a first mechanical arm, a second mechanical arm, and a suction disc; The rib plate is installed on the lifting platform, the height of the lifting platform is consistent with the height of the storage rack, and the lifting platform controls the height of the rib plate according to the corresponding delivery information; The first mechanical arm and the second mechanical arm are each provided with a suction disc, and the suction disc is configured to adjust suction force to suck the target object according to the weight. The first mechanical arm is installed on the top plate of the rib plate, and the second mechanical arm is installed on the bottom plate of the rib plate, and the first mechanical arm and the second mechanical arm are used to carry the target object according to the corresponding distribution information, and place the target object on the shelf.

4. The control method of claim 2, wherein The mobile device comprises a Mecanum wheel, an encoder counter and a driving motor; The driving motor is connected with the Mecanum wheel and the encoder counter respectively, the encoder counter is connected with the positioning navigation module, and the Mecanum wheel, the encoder counter and the driving motor are arranged at the bottom of the shelf, and the encoder counter is used to obtain the current speed information.

5. The control method of claim 4, wherein The positioning navigation module comprises a ranging module, a laser radar module and an IMU module; The ranging module is arranged around the bottom of the shelf, and the ranging module is connected with the driving motor and used to detect the distance value from the obstacle to the shelf to drive the driving motor to avoid obstacles; The laser radar module and the IMU module are arranged at the top of the shelf, the laser radar module is used to obtain radar scanning information, and the IMU module is used to obtain current movement information.

6. The control method of the air-breathing dual-arm transport robot cluster system according to claim 1, wherein The process of obtaining the current position information specifically comprises: obtaining current movement information, current speed information and radar scanning information, constructing a point cloud map and obtaining a current pose; using Kalman filtering algorithm, fusing the current movement information, the current speed information and the radar scanning information to obtain the current pose; using LAMA positioning algorithm to obtain the positioning information of the corresponding carrying robot, and obtaining the current position information according to the current pose and the positioning information. 7.The control method of the air-breathing dual-arm transport robot cluster system of claim 1, wherein, Further comprising: The scheduling system obtains the verification instruction, encapsulates the running adjustable parameters of the corresponding carrying robot; The scheduling system obtains the cluster verification algorithm, initializes the current position information of the corresponding carrying robot, adjusts the running adjustable parameters according to the cluster verification algorithm, and schedules a plurality of carrying robots. 8.The control method of the air-breathing dual-arm transport robot cluster system according to claim 1, wherein, The dynamic allocation of the distribution information specifically comprises: The scheduling system obtains the current running time and the running state corresponding to the plurality of carrying robots; According to the running mode, the current running time and the corresponding running state, the number of running carrying robots and the number of standby carrying robots are determined, and the distribution information is dynamically allocated to the running carrying robots.

Images