An online accurate luggage pick-up system balancing dynamic and cooperative motion

Abstract

The application provides an online accurate luggage pickup system balancing and cooperating with movement, and relates to the technical field of luggage pickup, and the system comprises the following steps: when luggage reaches a preset luggage detection position, size information and a rotation angle of the luggage are acquired; according to the size information of the luggage and a distance between an edge of a luggage conveying device dropping the luggage and an origin of a visual coordinate system, a pose matrix of a target vertex of the luggage in the visual coordinate is determined; in addition, according to the size information of the luggage and the rotation angle, a pose matrix of the luggage in a mechanical arm tray coordinate system is determined; then, according to a conversion relationship between the visual coordinate system and the mechanical arm coordinate system and the two determined pose matrices, a downward turning angle of the mechanical arm tray is determined, and a pose of the mechanical arm tray is obtained; and the application can improve the accuracy and stability of picking up the luggage.

Description

Technical Field

[0001] This invention relates to the field of baggage retrieval technology, and in particular to an online precision baggage retrieval system based on dynamic balance and coordinated movement. Background Technology

[0002] In the civil aviation sector, baggage handling is crucial to the overall operation of an airport. Airports are busy locations with a large number of flights and passengers passing through every day. An efficient baggage handling system can ensure that baggage is processed and transported at the right time and place, thereby avoiding delays and chaos. To improve the efficiency of baggage handling, airport baggage handling has shifted from traditional manual handling to automated handling. In existing technologies, automated baggage handling systems typically simply move baggage from the baggage conveyor belt to the baggage cart without considering parameters such as the weight and size of the baggage. When picking up baggage, the accuracy and stability of the picking are low, resulting in large errors in the subsequent stacking of baggage and irregular stacking patterns. Summary of the Invention

[0003] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0004] According to a first aspect of this application, an online precision baggage retrieval system based on dynamic balance and coordinated motion is provided. The system includes: a robotic arm tray, a storage medium, and a processor; wherein the robotic arm tray is used to retrieve baggage when it falls from a baggage conveyor; the storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to perform the following steps:

[0005] S100, in response to the baggage arriving at the preset baggage detection position, obtains the baggage length bag.h, width bag.w, and the baggage rotation angle bag.rz along the Z-axis in the preset visual coordinate system QT;

[0006] S200, based on bag.h, bag.w, bag.rz and the distance belt.h / 2 from the edge of the baggage dropped from the baggage conveyor in QT to the origin of QT, determine the pose matrix ZA of the target vertex A of the baggage in QT;

[0007] S300, based on bag.h, bag.w, bag.rz, the dimensions of the robotic arm tray, and the way the robotic arm tray is picked up, determine the pose matrix ZB of the target contact point B on the robotic arm tray in the robotic arm tray coordinate system QR; where B is the point on the robotic arm tray plane that coincides with A when picking up luggage;

[0008] S400, based on the preset transformation relationship M, ZA, and ZB between the visual coordinate system and the robotic arm pallet coordinate system, determine the downward tilt angle θ of the robotic arm pallet;

[0009] S500 determines the pose of the robotic arm tray based on ZA and θ.

[0010] The present invention has at least the following beneficial effects:

[0011] The present invention provides an online precision baggage retrieval system based on dynamic balance and coordinated motion. When baggage arrives at a preset baggage detection position, the system acquires the baggage's size information and rotation angle. Based on the baggage's size information and the distance from the edge of the baggage falling from the baggage conveyor to the origin of the visual coordinate system in a preset visual coordinate system, the system determines the pose matrix of the baggage's target vertex in the visual coordinate system. Furthermore, based on the baggage's size information and rotation angle, the system determines the baggage's pose matrix in the robotic arm tray coordinate system. Then, based on the transformation relationship between the visual coordinate system and the robotic arm coordinate system, and the two determined pose matrices, the system determines the downward tilt angle of the robotic arm tray, thereby obtaining the pose of the robotic arm tray. This ensures that when the robotic arm tray retrieves the baggage in this pose, the target vertex of the baggage contacts the target contact point. Due to the tilt angle, the baggage gradually contacts the robotic arm tray during retrieval, thereby reducing shaking during retrieval and further improving the accuracy and stability of baggage retrieval. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart of the processor execution steps of the online precision baggage retrieval system for dynamic balance and coordinated motion provided in Embodiment 1 of the present invention;

[0014] Figure 2 This is a schematic diagram of the coordinate system provided in Embodiment 1 of the present invention;

[0015] Figure 3 This is a schematic diagram of the robotic arm tray tilting down according to Embodiment 1 of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] It should be noted that, based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Furthermore, this device and / or practice the method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.

[0018] Example 1:

[0019] The following will refer to Figure 1 The flowchart shown is a process flow diagram of the processor execution steps of an online precision baggage retrieval system based on dynamic balance and cooperative motion, which introduces an online precision baggage retrieval system based on dynamic balance and cooperative motion.

[0020] This online precision baggage retrieval system based on dynamic balancing and coordinated motion includes: a robotic arm tray, a storage medium, and a processor; wherein, the robotic arm tray is used to retrieve baggage when it falls from the baggage conveyor; the storage medium stores at least one instruction or at least one program segment, which is loaded and executed by the processor to achieve the following steps:

[0021] S100, in response to the baggage arriving at the preset baggage detection position, obtains the baggage length bag.h, width bag.w, and the baggage rotation angle bag.rz along the Z-axis in the preset visual coordinate system QT.

[0022] In this embodiment, it should be noted that the luggage in the first embodiment is a suitcase-style luggage with a relatively regular shape; the luggage is transported from a luggage conveyor device, which can be a conveyor belt, and a vision device, such as a camera, is set above the luggage conveyor device; the vision device corresponds to a preset visual coordinate system QT; the luggage detection position is a preset position or a luggage detection point, which is the center point of the luggage when it arrives at the detection position, and the pose of the luggage detection point can be obtained by the vision device based on the visual calibration point.

[0023] like Figure 2As shown, "bag" represents luggage, and "hand" represents the robotic arm's tray. When the luggage reaches the luggage detection position on the luggage conveyor, the vision device can acquire the luggage's length, width, and rotation angle. At this time, the pose matrix of the luggage at the detection position can also be acquired.

[0024]

[0025] Where bag.x and bag.y are the X-axis and Y-axis coordinates of the center point of the luggage in the visual coordinate system, respectively.

[0026] S200, based on bag.h, bag.w, bag.rz and the distance belt.h / 2 from the edge of the baggage dropped from the baggage conveyor in QT to the origin of QT, determine the pose matrix ZA of the target vertex A of the baggage in QT.

[0027] Furthermore, ZA can be obtained through the following steps:

[0028] S210, Obtain the pose matrix of the luggage when its center point C is located at the edge of the luggage falling from the luggage conveyor. in, The X-axis coordinate of the baggage is the same as the X-axis coordinate when the baggage arrives at the baggage inspection position, the Y-axis coordinate is -belt.h / 2; the rotation angle along the Z-axis is bag.rz.

[0029] In this embodiment, it can be understood that the baggage conveyor transports baggage in a straight line. Therefore, when the baggage reaches the baggage release point, the baggage's pose matrix can be determined by the distance belt.h from the edge of the baggage falling from the baggage to the origin of QT, and... To obtain, that is:

[0030]

[0031] The baggage release point is the center point of the baggage just as it begins to fall. Assuming the baggage has uniform mass, this point is when the baggage is about to fall off the edge of the conveyor belt. The pose of this point is obtained by translating the baggage detection point along the Y-axis.

[0032] After obtaining the pose of the luggage, it is necessary to further determine the pose of the target vertex A of the luggage in QT; the target vertex can be understood as the vertex that is farthest from the visual coordinate system when the luggage is picked up.

[0033] S220, obtain the pose matrix PA of A in the preset separation point coordinate system; where the X-axis of QE is parallel to the width direction of the luggage, and the Y-axis of QE is parallel to the length direction of the luggage; the coordinates of A in PA are determined according to bag.rz, bag.h and bag.w, and the rotation angle of the robotic arm tray along the Z-axis in PA is determined according to the receiving method of the robotic arm tray and bag.rz.

[0034] In this embodiment, for ease of calculation, a separation point coordinate system is preset. The X-axis of the separation point coordinate system is parallel to the width direction of the luggage, and the Y-axis is parallel to the length direction of the luggage. Therefore, the pose matrix PA of the luggage in the separation point coordinate system can be obtained relatively easily.

[0035]

[0036] Specifically, the X-axis coordinate is determined as +bag.w / 2 or -bag.w / 2 based on bag.rz, and touch.rz is determined as -90°, 90°, or 0° based on the horizontal and vertical connections and bag.rz. It can be understood that touch.rz is 0 when connected vertically, and touch.rz is -90° or 90° when connected horizontally.

[0037] S230, according to PA, determined

[0038] In this implementation, according to With PA, we can obtain ZA;

[0039] ;

[0040] It should be noted that the specific values ​​in the above pose matrix can be determined based on the receiving method and the corresponding rotation angle, which will not be elaborated here.

[0041] S300, based on bag.h, bag.w, bag.rz, the dimensions of the robotic arm tray, and the way the robotic arm tray is picked up, determine the pose matrix ZB of the target contact point B on the robotic arm tray in the robotic arm tray coordinate system QR; where B is the point on the robotic arm tray plane that coincides with A when picking up luggage.

[0042] In this embodiment, it can be understood that when the robotic arm tray picks up luggage, the target contact point on the robotic arm tray comes into contact with the target vertex of the luggage. Therefore, a coordinate system QR for the robotic arm tray can be established, and the pose matrix ZB of the target contact point on the robotic arm tray in QR can be determined.

[0043] Furthermore, step S300 may include the following steps:

[0044] S310, if the robotic arm tray is picked up in the preset horizontal connection mode, and hand.w > bag.h, then the initial X-axis coordinate of the first contact point B1 in the QR is determined to be hang_left_touch_x = hand.w / 2 - bag.h, and the initial X-axis coordinate of the second contact point B2 in the QR is determined to be hang_right_touch_x = hand.w / 2; where hand.w is the width of the robotic arm tray; in the horizontal connection mode, the length direction of the robotic arm tray is perpendicular to the length direction of the luggage.

[0045] S320, if the robotic arm tray is picked up in the preset horizontal way and hand.w≤bag.h, then determine the initial X-axis coordinate of B1 in QR hang_left_touch_x=-bag.h / 2, and determine the initial X-axis coordinate of B2 in QR hang_right_touch_x=bag.h / 2;

[0046] S330, if the robotic arm pallet is picked up in the preset horizontal connection mode, and hand.h / 2-bag.w≤-hand.h / 2, then the Y-axis coordinates of B1 and B2 in QR are determined to be -hand.h / 2; if the robotic arm pallet is picked up in the preset horizontal connection mode, and -hand.h / 2<hand.h / 2-bag.w<hand.h / 2, then the Y-axis coordinates of MB1 and MB2 in QR are determined to be hand.h / 2-bag.w;

[0047] S340, if the robotic arm tray is picked up in a preset horizontal manner, and hand.h / 2 - bag.w ≥ hand.h / 2, then the Y-axis coordinates of B1 and B2 in QR are determined to be hand.h / 2.

[0048] In this embodiment, the pose matrix within the robotic arm tray before it tilts down is as follows:

[0049] Among them, hand.rz is determined based on the horizontal and vertical connections and bag.rz.

[0050] At this point, it is necessary to further determine the X-axis coordinate ht.x and Y-axis coordinate ht.y, which can be used to calculate the luggage dimensions in the QR code:

[0051] If the receiving method is horizontal connection, the length along the X-axis of the robotic arm tray (bag_along_hand_x) is: bag.h; the length along the Y-axis of the robotic arm tray (bag_along_hand_y) is: bag.w.

[0052] If the receiving method is vertical, then the length along the X-axis of the robotic arm tray (bag_along_hand_x) is: bag.w; and the length along the Y-axis of the robotic arm tray (bag_along_hand_y) is: bag.h.

[0053] If the robotic arm tray is picked up in a preset horizontal manner, then the X-axis coordinates and Y-axis coordinates of the first contact point and the second contact point in the QR are determined through the above steps S310-S340, that is, the X-axis coordinates and Y-axis coordinates of the left contact point and the right contact point in the QR.

[0054] Furthermore, step S300 may also include the following steps:

[0055] S350, if the robotic arm tray is picked up in the preset vertical orientation and hand.w > bag.w, then determine the initial X-axis coordinate of B1 in the QR: hang_left_touch_x = hand.w / 2 - bag.w, and determine the initial X-axis coordinate of B2 in the QR: hang_right_touch_x = hand.w / 2; in the vertical orientation, the length direction of the robotic arm tray is parallel to the length direction of the luggage.

[0056] S360, if the robotic arm tray is picked up in the preset vertical orientation and hand.w ≤ bag.w, then determine the initial X-axis coordinate of B1 in QR: hang_left_touch_x = -bag.w / 2, and determine the initial X-axis coordinate of B2 in QR: hang_right_touch_x = bag.w / 2.

[0057] S370, if the robotic arm pallet is picked up in the preset vertical orientation, and hand.h / 2 - bag.h ≤ -hand.h / 2, then the Y-axis coordinates of B1 and B2 in the QR are determined to be -hand.h / 2; if the robotic arm pallet is picked up in the preset vertical orientation, and -hand.h / 2 < hand.h / 2 - bag.h < hand.h / 2, then the Y-axis coordinates of B1 and B2 in the QR are determined to be hand.h / 2 - bag.h.

[0058] S380, if the robotic arm tray is picked up in the preset vertical way, and hand.h / 2-bag.h≥hand.h / 2, then the Y-axis coordinates of B1 and B2 in QR are determined to be hand.h / 2.

[0059] In this embodiment, if the robotic arm tray is picked up in a preset vertical manner, the X-axis coordinates and Y-axis coordinates of the first contact point and the second contact point in the QR are determined through the above steps S350-S380, that is, the X-axis coordinates and Y-axis coordinates of the left contact point and the right contact point in the QR.

[0060] It should be noted that when determining the pose of the robotic arm's pallet, only one contact point is needed; therefore, the target contact point needs to be determined.

[0061] Furthermore, after step S380, the method further includes the following steps:

[0062] S390, based on hang_left_touch_x and hang_right_touch_x, determine the target X-axis coordinate of B1 in QR: hang_left_touch_x' = hang_left_touch_x + x_offset and the target X-axis coordinate of B2 in QR: hang_right_touch_x' = hang_right_touch_x + x_offset; where x_offset is a preset offset distance.

[0063] In this embodiment, after the robotic arm tray receives the luggage, it needs to place the luggage on the luggage cart. In order to place the luggage compactly and neatly, the X-axis coordinates of the first contact point and the second contact point determined in the above steps need to be offset so that the luggage subsequently received is offset on the robotic arm tray. When placing the luggage, it can play a pushing role, thereby making the luggage more compact and neat.

[0064] S391, if bag.rz > 0, then B2 is determined to be B; otherwise, B1 is determined to be B; thus, ZB is obtained.

[0065] In this embodiment, it can be understood that the contact point that is far away from the baggage conveyor is determined as the target contact point through step S391.

[0066] S400 determines the downward tilt angle θ of the robotic arm pallet based on the preset transformation relationship M, ZA, and ZB between the visual coordinate system and the robotic arm pallet coordinate system.

[0067] Furthermore, step S400 may include the following steps:

[0068] S410, determine the vertical line between the drop surface of the luggage and the robotic arm tray;

[0069] S420, a parallel line that is at a preset distance from the perpendicular and parallel to the perpendicular is defined as the axis; wherein, the axis is located below the luggage;

[0070] S430, rotate the robotic arm tray around the axis, solve ZB=M×ZA, and obtain θ.

[0071] In this embodiment, as Figure 3 As shown, since the baggage conveyor has a certain thickness, and the robotic arm tray needs to be placed below the baggage conveyor when picking up baggage, there will be a certain distance between the target contact point and the target vertex, and the baggage will shake when picking up baggage. Therefore, the robotic arm tray needs to be flipped up so that the target contact point contacts the target vertex to prevent the baggage from shaking.

[0072] Specifically, after determining the axis, the pose matrix of the robotic arm tray can be rotated so that ZB = M × ZA, thereby solving for the rotation angle and the pose matrix of the robotic arm tray after rotation. This allows the robotic arm tray to pick up luggage according to the determined pose matrix, avoiding luggage shaking and improving the accuracy and stability of luggage picking.

[0073] S500 determines the pose of the robotic arm tray based on ZA and θ.

[0074] In this embodiment, after determining ZA, θ, and the axis coordinates, the pose of the robotic arm tray can be determined.

[0075] Furthermore, after step S500, the method may further include the following steps:

[0076] S600: If the robotic arm tray comes into contact with the luggage, the robotic arm tray is controlled to move along the direction of the luggage movement while the robotic arm tray is controlled to flip up evenly.

[0077] Furthermore, the speed at which the robotic arm tray moves along the direction of the luggage movement is the same as the speed at which the luggage is conveyed on the luggage conveyor; the rate of change of the angle at which the robotic arm tray flips upward uniformly is ω=θ / t; where t is the time taken for the luggage to come into contact with the robotic arm tray and fall from the luggage conveyor.

[0078] In this embodiment, after the target contact point and the target vertex make actual contact, it indicates that the luggage has made contact with the robotic arm tray. At this time, in order to further reduce the shaking of the luggage, as the luggage moves, the robotic arm tray is controlled to move along the direction of the luggage's movement while the robotic arm tray is controlled to flip upwards uniformly. The rate of change of the angle of uniform upward flipping of the robotic arm tray is ω=θ / t; t can be obtained according to the luggage's conveying speed and size information.

[0079] In this embodiment, when the luggage arrives at the preset luggage detection position, the size information and rotation angle of the luggage are acquired. Based on the size information of the luggage and the distance between the edge of the luggage falling from the luggage conveyor and the origin of the visual coordinate system in the preset visual coordinate system, the pose matrix of the target vertex of the luggage in the visual coordinate system is determined. In addition, based on the size information and rotation angle of the luggage, the pose matrix of the luggage in the coordinate system of the robotic arm tray is determined. Then, based on the transformation relationship between the visual coordinate system and the robotic arm coordinate system and the two determined pose matrices, the downward tilt angle of the robotic arm tray is determined, thereby obtaining the pose of the robotic arm tray. As a result, when the robotic arm tray picks up the luggage in this pose, the target vertex of the luggage contacts the target contact point. Due to the setting of the downward tilt angle, the luggage gradually contacts the robotic arm tray during the luggage picking process, thereby reducing the shaking of the luggage during the picking process and further improving the accuracy and stability of the luggage picking.

[0080] Example 2:

[0081] The luggage in the above embodiment one is a regular-shaped box-type luggage. For luggage with irregular shapes, it is usually placed in a luggage tray for transport. However, the luggage tray is typically larger than the luggage. After placing the luggage in the tray, it may be positioned at one end. If the robotic arm used to retrieve the luggage is positioned in a fixed location, the shift in the center of gravity of the luggage may cause it to fall off the tray, resulting in damage. To avoid this situation, the following method is provided:

[0082] Q100, in response to the arrival of the tray baggage at the preset baggage detection position, acquires the pose information of the baggage tray, the weight of the baggage tray, the center of gravity coordinates of the baggage tray, the pose information of the baggage inside the baggage tray, and the center of gravity coordinates of the baggage inside the baggage tray.

[0083] In this embodiment, the type of luggage can be identified by a vision device. When a tray of luggage is identified and arrives at the luggage detection position, the device can obtain the posture information of the luggage tray, the weight of the luggage tray, the center of gravity coordinates of the luggage tray, the posture information of the luggage inside the luggage tray, and the center of gravity coordinates of the luggage inside the luggage tray.

[0084] Q200, based on the pose information of the luggage tray and the receiving method of the robotic arm tray, determine the initial coordinates G1 of the center of the robotic arm tray when the robotic arm tray receives the luggage.

[0085] Furthermore, Q200 may include the following steps:

[0086] Q210, obtain the length bag1.h, width bag1.w, and rotation angle bag1.rz of the luggage tray along the Z-axis in the preset visual coordinate system QT.

[0087] Q220, based on bag1.h, bag1.w, bag1.rz and the distance belt.h / 2 from the edge of the luggage tray that fell from the luggage conveyor in QT to the origin of QT, determine the pose matrix YA of the target vertex MA of the luggage tray in QT.

[0088] Q230, based on bag1.h, bag1.w, bag1.rz, the dimensions of the robotic arm tray, and the way the robotic arm tray is picked up, determine the pose matrix YB of the target contact point MB on the robotic arm tray in the robotic arm tray coordinate system QR; where MB is the point on the robotic arm tray plane that coincides with MA when picking up luggage.

[0089] Q240, based on the conversion relationship M, YA, and YB between QT and QR, determine the downward tilt angle θ of the robotic arm tray.

[0090] Q250, based on YA and θ, determine the pose of the robotic arm tray, and then obtain G1.

[0091] In Example 1, it was introduced how to determine the pose of the robotic arm tray when the luggage is regular luggage. In this example, the luggage tray can be regarded as the luggage in Example 1, and then the pose of the robotic arm tray can be determined, thus obtaining G1. It should be noted that the method in this example is the same as the method in Example 1, and will not be described again here.

[0092] Furthermore, YA can be obtained through the following steps:

[0093] Q221, Obtain the pose matrix of the luggage tray when its center is located at the edge of the luggage falling from the luggage conveyor. in, The X-axis coordinate of the corresponding luggage tray is the same as the X-axis coordinate when the luggage tray reaches the luggage detection position, the Y-axis coordinate is -belt.h / 2; the rotation angle along the Z-axis is bag1.rz.

[0094] Q222, obtain the pose matrix PA1 of MA in the preset separation point coordinate system QE; where the X-axis of QE is parallel to the width direction of the luggage tray, and the Y-axis of QE is parallel to the length direction of the luggage tray; the coordinates of MA in PA1 are determined according to bag1.rz, bag1.h and bag1.w, and the rotation angle of the robotic arm tray along the Z-axis in PA1 is determined according to the receiving method of the robotic arm tray and bag1.rz.

[0095] Q223, according to PA1, confirmed

[0096] It should be noted that the method for determining YA in this embodiment is the same as the method for determining ZA in Embodiment 1. The difference is that the object in this embodiment is a luggage tray, while the object in Embodiment 1 is regular luggage.

[0097] Furthermore, step Q230 may include the following steps:

[0098] Q231, if the robotic arm tray is picked up in the preset horizontal connection mode, and hand.w > bag1.h, then determine the initial X-axis coordinate of the third contact point MB1 in QR: hang_left_touch_x = hand.w / 2 - bag1.h, and determine the initial X-axis coordinate of the fourth contact point MB2 in QR: hang_right_touch_x = hand.w / 2; where hand.w is the width of the robotic arm tray; in the horizontal connection mode, the length direction of the robotic arm tray is perpendicular to the length direction of the luggage tray.

[0099] Q232, if the robotic arm tray is picked up in the preset horizontal mode and hand.w≤bag1.h, then determine the initial X-axis coordinate of MB1 in QR: hang_left_touch_x=-bag1.h / 2, and determine the initial X-axis coordinate of MB2 in QR: hang_right_touch_x=bag1.h / 2.

[0100] Q233, if the robotic arm pallet is picked up in the preset horizontal connection mode, and hand.h / 2 - bag1.w ≤ -hand.h / 2, then the Y-axis coordinates of MB1 and MB2 in QR are determined to be -hand.h / 2; if the robotic arm pallet is picked up in the preset horizontal connection mode, and -hand.h / 2 < hand.h / 2 - bag1.w < hand.h / 2, then the Y-axis coordinates of MB1 and MB2 in QR are determined to be hand.h / 2 - bag1.w.

[0101] Q234, if the robotic arm's pallet receiving method is the preset horizontal connection mode, and hand.h / 2 - bag1.w ≥ hand.h / 2, then the Y-axis coordinates of MB1 and MB2 in QR are determined to be hand.h / 2.

[0102] Q235, if the robotic arm tray is picked up in the preset vertical mode and hand.w > bag1.w, then determine the initial X-axis coordinate of MB1 in QR: hang_left_touch_x = hand.w / 2 - bag1.w, and determine the initial X-axis coordinate of MB2 in QR: hang_right_touch_x = hand.w / 2; in the vertical mode, the length direction of the robotic arm tray is parallel to the length direction of the luggage tray.

[0103] Q236, if the robotic arm tray is picked up in the preset vertical mode and hand.w≤bag1.w, then determine the initial X-axis coordinate of MB1 in QR: hang_left_touch_x=-bag1.w / 2, and determine the initial X-axis coordinate of MB2 in QR: hang_right_touch_x=bag1.w / 2.

[0104] Q237, if the robotic arm pallet is picked up in the preset vertical connection mode, and hand.h / 2 - bag1.h < -hand.h / 2, then the Y-axis coordinates of MB1 and MB2 in QR are determined to be -hand.h / 2; if the robotic arm pallet is picked up in the preset vertical connection mode, and -hand.h / 2 < hand.h / 2 - bag1.h < hand.h / 2, then the Y-axis coordinates of MB1 and MB2 in QR are determined to be hand.h / 2 - bag1.h.

[0105] Q238, if the robotic arm tray is picked up in the preset vertical mode, and hand.h / 2 - bag1.h ≥ hand.h / 2, then the Y-axis coordinates of MB1 and MB2 in QR are determined to be hand.h / 2.

[0106] In this embodiment, the coordinates of the third and fourth contact points of the luggage tray can be determined through the above steps Q231-Q238. The method of determination is the same as the method of determining the coordinates of the first and second contact points in Embodiment 1, and will not be repeated here.

[0107] Furthermore, after step Q238, the method further includes the following steps:

[0108] Q21. Based on hang_left_touch_x and hang_right_touch_x, determine the target X-axis coordinate of B1 in QR: hang_left_touch_x' = hang_left_touch_x + x_offset, and the target X-axis coordinate of B2 in QR: hang_right_touch_x' = hang_right_touch_x + x_offset; where x_offset is a preset offset distance.

[0109] Q22, if bag1.rz > 0, then determine B2 as B; otherwise, determine B1 as B; thus obtaining YB.

[0110] In this embodiment, in order to stack the palletized luggage compactly and neatly, it is also necessary to offset the position of the palletized luggage. The offset method is the same as that in Embodiment 1, and will not be described in detail here.

[0111] Q300 determines the weight of the luggage inside the luggage tray based on the total weight of the luggage on the tray and the weight of the luggage tray itself.

[0112] In this embodiment, the total weight of the pallet luggage can be obtained by weighing it using the luggage conveying device. Since the weight of the luggage pallet is known, the weight of the luggage inside the luggage pallet can be obtained by subtracting the weight of the luggage pallet from the total weight of the pallet luggage.

[0113] Q400, based on the position information of the luggage tray, the weight of the luggage tray, the position information of the luggage inside the luggage tray, and the weight of the luggage inside the luggage tray, determine the coordinates G2 of the combined center of gravity of the luggage on the tray.

[0114] In this embodiment, it should be noted that after obtaining the pose information of the luggage tray, the weight of the luggage tray, the pose information of the luggage in the luggage tray, and the weight of the luggage in the luggage tray, those skilled in the art can use existing combined center of gravity determination methods to determine the coordinates G2 of the combined center of gravity of the luggage in the tray according to actual needs, which will not be elaborated here.

[0115] Q500, if the robotic arm tray is picked up in the preset horizontal connection mode, then the offset between the center of the luggage tray and the combined center of gravity of the luggage is determined according to G2 and the center coordinate G3 of the luggage tray.

[0116] In this embodiment, if the robotic arm tray is picked up in a preset horizontal manner, since the posture of the robotic arm determined in the above steps is based on the luggage tray and the position of the combined center of gravity is not considered, when the position of the combined center of gravity is far from the position of the center of the robotic arm tray, the luggage on the tray may fall off.

[0117] Furthermore, step Q500 includes the following steps:

[0118] Q510, Obtain the X-axis coordinate difference ΔX = G between G2 and G3. 3，x -G 2，x and the difference in Y-axis coordinates ΔY = G 3，y -G 2，y Among them, G 3，x and G 2，x These are the X-axis coordinates in G3 and G2, respectively; G 3，y and G 2，y These are the Y-axis coordinates in G3 and G2, respectively.

[0119] Q520 defines ΔX and ΔY as the offsets between the center of the robotic arm's pallet and the combined center of gravity of the palletized luggage.

[0120] In this embodiment, the offset of the combined center of gravity from the center of the luggage tray on the X-axis and the offset on the Y-axis can be determined through the above steps.

[0121] The Q600 adjusts the receiving position of the robotic arm tray based on the offset.

[0122] Furthermore, step Q600 includes the following steps:

[0123] Q610, obtain G1 = (G 1，x G 1，y ); where G 1，x and G 1，y These are the X-axis and Y-axis coordinates of the center of the robotic arm tray when it receives luggage from the tray.

[0124] Q620, based on G1, ΔX, and ΔY, determine the target coordinates G1' = (G... 1，x +ΔX, G 1，y +ΔY).

[0125] Q630, move the center of the robotic arm tray to the position corresponding to G1'.

[0126] In this embodiment, by taking the above steps, the center of the robotic arm tray can be adjusted to be close to or coincide with the center of gravity of the assembly, thereby preventing the tray luggage from falling off.

[0127] Furthermore, after step Q600, the method further includes the following steps:

[0128] Q700, if the robotic arm tray is picked up in the preset vertical way, then the vertical distance d between the combined center of gravity of the tray luggage and the vertical line of the luggage tray along the width direction is determined according to G2.

[0129] Q710, if d > d', then issue an alarm message; otherwise, use the robotic arm to retrieve the tray luggage; where d' is the preset safe distance; the alarm message is used to alert the user that the tray luggage may fall.

[0130] In this embodiment, if the robotic arm tray is picked up in the preset vertical method, there is no room for the robotic arm tray to be adjusted forward or backward. Therefore, it is necessary to determine the relationship between the vertical distance d and d' between the combined center of gravity of the tray luggage and the vertical line of the tray along the width direction. If d > d', it means that the position of the combined center of gravity is far off, and the tray luggage may fall. Therefore, an alarm message is issued.

[0131] In this embodiment, when the pallet luggage reaches the preset detection position, relevant information about the pallet and the luggage inside the pallet is acquired. This allows the determination of the center of gravity coordinates of the pallet and the luggage inside the pallet. Based on the size information of the pallet and the luggage inside the pallet, the combined center of gravity of the pallet luggage is determined. According to the offset between the combined center of gravity and the center of the luggage pallet, the receiving position of the robotic arm pallet is adjusted. The combined center of gravity can represent the actual position of the center of gravity of the pallet luggage as a whole. Adjusting the receiving position of the robotic arm pallet according to the combined center of gravity can make the center of the robotic arm pallet coincide with or be close to the position of the combined center of gravity, thereby preventing the pallet luggage from falling during receiving and preventing damage to the luggage.

[0132] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0133] Embodiments of the present invention also provide a non-transitory computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method in the method embodiments, wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiments.

[0134] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0135] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0136] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0137] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0138] Embodiments of the present invention also provide an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0139] The electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments in this application.

[0140] Electronic devices are manifested in the form of general-purpose computing devices. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, and a bus connecting different system components (including memory and processor).

[0141] The memory stores program code that can be executed by the processor, causing the processor to perform the steps in the various embodiments described in this specification.

[0142] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM).

[0143] The memory may also include programs / utilities having a set (at least one) of program modules, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0144] A bus can represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus that uses any of the various bus structures.

[0145] The electronic device can also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output (I / O) interfaces. Furthermore, the electronic device can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0146] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0147] Embodiments of the present invention also provide a computer program product including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps of the methods described above in various exemplary embodiments of the present invention.

[0148] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and are not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention.

Claims

1. A dynamic balance and coordinated motion online precision baggage retrieval system, characterized in that, The system includes: a robotic arm tray, a storage medium, and a processor; wherein, the robotic arm tray is used to catch luggage when it falls from the luggage conveyor; the storage medium stores at least one instruction or at least one program segment, which is loaded and executed by the processor to achieve the following steps: S100, in response to the baggage arriving at the preset baggage detection position, obtains the baggage length bag.h, width bag.w, and the baggage rotation angle bag.rz along the Z-axis in the preset visual coordinate system QT; S200, based on bag.h, bag.w, bag.rz and the distance belt.h / 2 from the edge of the baggage dropped from the baggage conveyor in QT to the origin of QT, determine the pose matrix ZA of the target vertex A of the baggage in QT; S300, based on bag.h, bag.w, bag.rz, the dimensions of the robotic arm tray, and the way the robotic arm tray is picked up, determine the pose matrix ZB of the target contact point B on the robotic arm tray in the robotic arm tray coordinate system QR; where B is the point on the robotic arm tray plane that coincides with A when picking up luggage; S400, based on the preset transformation relationship M, ZA and ZB between the visual coordinate system and the robotic arm pallet coordinate system, determine the downward tilt angle θ of the robotic arm pallet; S500 determines the pose of the robotic arm tray based on ZA and θ; ZA is obtained through the following steps: S210, Obtain the pose matrix of the luggage when its center point C is located at the edge of the luggage falling from the luggage conveyor. ;in, The X-axis coordinate of the baggage is the same as the X-axis coordinate when the baggage arrives at the baggage inspection position, the Y-axis coordinate is -belt.h / 2; the rotation angle along the Z-axis is bag.rz; S220, obtain the pose matrix PA of A in the preset separation point coordinate system QE; wherein, the X-axis of QE is parallel to the width direction of the luggage, and the Y-axis of QE is parallel to the length direction of the luggage; the coordinates of A in PA are determined according to bag.rz, bag.h, and bag.w, and the rotation angle of the robotic arm tray along the Z-axis in PA is determined according to the receiving method of the robotic arm tray and bag.rz; the receiving method includes horizontal receiving method and vertical receiving method. In the horizontal receiving method, the length direction of the robotic arm tray is perpendicular to the length direction of the luggage, and in the vertical receiving method, the length direction of the robotic arm tray is parallel to the length direction of the luggage; S230, according to PA, determine .

2. The online precision baggage retrieval system based on dynamic balance and coordinated motion according to claim 1, characterized in that, Step S300 includes the following steps: S310, if the robotic arm tray is picked up in a preset horizontal manner and hand.w > bag.h, then determine the initial X-axis coordinate of the first contact point B1 in the QR: hang_left_touch_x = hand.w / 2 − bag.h, and determine the initial X-axis coordinate of the second contact point B2 in the QR: hang_right_touch_x = hand.w / 2; where hand.w is the width of the robotic arm tray; S320, if the robotic arm pallet is picked up in the preset horizontal way and hand.w≤bag.h, then determine the initial X-axis coordinate of B1 in QR hang_left_touch_x=−bag.h / 2, and determine the initial X-axis coordinate of B2 in QR hang_right_touch_x=bag.h / 2; S330, if the robotic arm pallet is picked up in the preset horizontal connection mode, and hand.h / 2-bag.w≤−hand.h / 2, then the Y-axis coordinates of B1 and B2 in QR are determined to be -hand.h / 2; if the robotic arm pallet is picked up in the preset horizontal connection mode, and −hand.h / 2<hand.h / 2-bag.w<hand.h / 2, then the Y-axis coordinates of B1 and B2 in QR are determined to be hand.h / 2-bag.w; hand.h is the length of the robotic arm pallet; S340, if the robotic arm tray is picked up in a preset horizontal manner, and hand.h / 2 - bag.w ≥ hand.h / 2, then the Y-axis coordinates of B1 and B2 in QR are determined to be hand.h / 2.

3. The online precision baggage retrieval system based on dynamic balance and coordinated motion according to claim 2, characterized in that, Step S300 also includes the following steps: S350, if the robotic arm tray is picked up in the preset vertical way, and hand.w > bag.w, then determine the initial X-axis coordinate of B1 in QR hang_left_touch_x = hand.w / 2 − bag.w, and determine the initial X-axis coordinate of B2 in QR hang_right_touch_x = hand.w / 2. S360, if the robotic arm tray is picked up in the preset vertical way, and hand.w≤bag.w, then determine the initial X-axis coordinate of B1 in QR hang_left_touch_x=−bag.w / 2, and determine the initial X-axis coordinate of B2 in QR hang_right_touch_x=bag.w / 2; S370, if the robotic arm pallet is picked up in the preset vertical orientation, and hand.h / 2 - bag.h ≤ −hand.h / 2, then the Y-axis coordinates of B1 and B2 in QR are determined to be -hand.h / 2; if the robotic arm pallet is picked up in the preset vertical orientation, and −hand.h / 2 < hand.h / 2 - bag.h < hand.h / 2, then the Y-axis coordinates of B1 and B2 in QR are determined to be hand.h / 2 - bag.h. S380, if the robotic arm tray is picked up in the preset vertical way, and hand.h / 2-bag.h≥hand.h / 2, then the Y-axis coordinates of B1 and B2 in QR are determined to be hand.h / 2.

4. The online precision baggage retrieval system based on dynamic balance and coordinated motion according to claim 3, characterized in that, Following step S380, the following steps are also included: S390, based on hang_left_touch_x and hang_right_touch_x, determine the target X-axis coordinate of B1 in QR: hang_left_touch_x' = hang_left_touch_x + x_offset and the target X-axis coordinate of B2 in QR: hang_right_touch_x' = hang_right_touch_x + x_offset; where x_offset is a preset offset distance; S391, if bag.rz > 0, then B2 is determined to be B; otherwise, B1 is determined to be B; thus, ZB is obtained.

5. The online precision baggage retrieval system based on dynamic balance and coordinated motion according to claim 4, characterized in that, Step S400 includes the following steps: S410, determine the vertical line between the drop surface of the luggage and the robotic arm tray; S420, a parallel line that is at a preset distance from the perpendicular and parallel to the perpendicular is defined as the axis; wherein, the axis is located below the luggage; S430, rotate the robotic arm tray around the axis, solve ZB=M×ZA, and obtain θ.

6. The online precision baggage retrieval system based on dynamic balance and coordinated motion according to claim 1, characterized in that, Following step S500, the following steps are also included: S600: If the robotic arm tray comes into contact with the luggage, the robotic arm tray is controlled to move along the direction of the luggage movement while the robotic arm tray is controlled to flip up evenly.

7. The online precision baggage retrieval system based on dynamic balance and coordinated motion according to claim 6, characterized in that, The robotic arm tray moves at the same speed as the luggage in the direction of movement as the luggage is conveyed on the luggage conveyor; the rate of change of the angle of uniform upward tilt of the robotic arm tray is ω=θ / t; where t is the time from when the luggage comes into contact with the robotic arm tray until it falls off the luggage conveyor.

Images