Robot, robotic system and method of manipulating a container fitting

Abstract

Disclosed herein is a robotic system comprises a robot for manipulating a container fitting and a mapping sensor positioned in the container environment remote from the robot. The robot comprises a mobile base, a vision module, a first robotic arm, and a second robotic arm. The vision module is coupled to the mobile base and is being configured to identify the container fitting. The first robotic arm is coupled to the mobile base and comprises a first gripper. The second robotic arm is coupled to the mobile base and is configured to comprise a second gripper. The first gripper and the second gripper are cooperatively actuatable to couple the container fitting to a pair of adjacent containers. The first gripper and the second gripper are cooperatively actuatable to decouple the container fitting from the pair of adjacent containers.

Description

ROBOT, ROBOTIC SYSTEM AND METHOD OF MANIPULATING A CONTAINER FITTINGCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of priority to Singapore patent application nos. 10202401063U and 10202401065V which were filed on 12 April 2024, the contents of which are hereby incorporated by reference in its entirety for all purposes.TECHNICAL FIELD

[0002] This application relates to the field of fitting manipulation, and in particular, a robot, a robot system and a method of manipulating a container fitting.BACKGROUND

[0003] The handling of containers and their fittings in port operations is predominantly a manual process that requires significant time and labor. The securing and removal of fittings, which interlock containers for stability during transport, involve repetitive and physically demanding tasks performed at multiple points on each container. This not only slows down overall logistics efficiency but also contributes to worker fatigue and potential safety risks. Therefore, there is a need for a more automated and efficient solution to streamline container handling operations, reduce manual workload, and enhance productivity in port logistics.SUMMARY

[0004] According to an aspect, disclosed herein is a robot for manipulating a container fitting. The robot comprises a mobile base; a vision module coupled to the mobile base, the vision module being configured to identify the container fitting; a first robotic aim coupled tothe mobile base, the first robotic arm comprising a first gripper; and a second robotic arm coupled to the mobile base, the second robotic arm comprising a second gripper; wherein the first gripper and the second gripper are cooperatively actuatable to couple the container fitting to a pair of adjacent containers, wherein the first gripper and the second gripper are cooperatively actuatable to decouple the container fitting from the pair of adjacent containers.

[0005] According to another aspect, disclosed herein is a robotic system for a container environment. The robotic system comprises the robot as described above; a mapping sensor positioned in the container environment remote from the robot, the mapping sensor being configured to provide a navigation information to the robot.

[0006] According to another aspect, disclosed herein is a method of manipulating a container fitting. The method comprises identifying the container fitting using the vision module of the robot as described above; based on a motion planning library, controlling at least one of the first robotic arm and the second robotic arm of the robot, to manipulate the container fitting.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Various embodiments of the present disclosure are described below with reference to the following drawings:

[0008] FIG. 1 is a schematic diagram of a robotic system for manipulating a container fitting according to various embodiments;

[0009] FIG. 2 is an isometric view of a robotic system according to one embodiment of the present disclosure;

[0010] FIG. 3 is an image showing a first robotic arm and a second robotic arm according to another embodiment of the present disclosure;[0011J FIG. 4 is a perspective view of a first gripper according to another embodiment of the present disclosure;

[0012] FIG. 5 is a perspective view of the a second gripper according to yet another embodiment of the present disclosure;

[0013] FIG. 6 is an isometric view of the first gripper according to an embodiment of the present disclosure;

[0014] FIG. 7A is a front view of the first gripper of FIG. 6;

[0015] FIG. 7B is a side view of the first gripper of FIG. 6;

[0016] FIG. 7C is a sectional side view of the first gripper of FIG. 6, showing view A-A;

[0017] FIG. 8 A is an isometric view of a second gripper according to another embodiment of the present disclosure, in which the hooks are extended to an open position;

[0018] FIG. 8B is an isometric view of the second gripper of FIG. 9A, in which the hooks are retracted to a close position;

[0019] FIG. 9A is a top view of a lattice palm segment according to another embodiment of the present disclosure;

[0020] FIG. 9B is a side view of the lattice palm of FIG. 10A;

[0021] FIG. 9C is an isometric view of the lattice palm of FIG. 10A;

[0022] FIG. 9D is a back view of the lattice palm of FIG. 10A;

[0023] FIG. 9E is an isometric cross-sectional view of the lattice palm of FIG. 10A;

[0024] FIG. 10A is an image illustrating the reference dimension of a lattice cell structure;

[0025] FIG. 10B is an image illustrating an uncompressed lattice cell structure;

[0026] FIG. 10C is a simulation image of a compressed lattice cell structure under a compressive force of 960N;

[0027] FIG. 10D is an image of an Imada tensile machine used in the compression test of the lattice palm;

[0028] FIG. 11 A is a perspective view of an assembly of a sleeved sensor and a lattice palm according to another embodiment of the present disclosure;

[0029] FIG. 1 IB is an exploded of the assembly of FIG. 11 A;

[0030] FIG. 11C is a side view of the assembly of FIG. 11 A;

[0031] FIG. 1 ID is a top view of the assembly of FIG. 11 A;

[0032] FIG. 12 is a schematic diagram of a sensor reading circuitry;

[0033] FIG. 13 is a schematic diagram illustrating the control and feedback loop employed by the gripper of FIG. 7 according to an embodiment of the present disclosure;

[0034] FIG. 14A is a plot of stress (in Mega-Pascals or MPa) as a function of strain illustrating a lattice cell structure under compression in an experiment and numerical simulation respectively, according to various embodiments;

[0035] FIG. 14B is a plot of stress (in Mega-Pascals or MPa) as a function of strain illustrating a lattice cell structure under shear in a numerical simulation, according to various embodiments;

[0036] FIG. 15 is an isometric view of the gripper according to another embodiment of the present disclosure;

[0037] FIG. 16A is a top view of the second gripper of FIG. 15;

[0038] FIG. 16B is a side view of the second gripper of FIG. 15;

[0039] FIG. 16C is a top view of the second gripper of FIG. 15;

[0040] FIG. 17 is a diagram illustrating a representative set of commercial cones and fittings;

[0041] FIGS. 18A-18C are perspective views of ade-coning process of semi-automatic twist locks, SOI and S03;

[0042] FIGS. 19A-19D are perspective views of a coning process of semi-automatic twist locks, SOI and S03;

[0043] FIGS. 20A-20C are perspective views of a de-coning process of self-hanging stackers, C04;

[0044] FIGS. 21A-21C are perspective views of a coning process of self-hanging stackers, C04;

[0045] FIGS. 22A-22C are perspective views of a de-coning process of self-hanging stackers, C12;

[0046] FIGS. 23A-23C are perspective views of a coning process of self -hanging stackers, C12;

[0047] FIGS. 24A-24D are perspective views of a de-coning process of fully automatic twist locks, F01;

[0048] FIGS. 25A-25D are perspective views of a coning process of fully automatic twist locks, F01;

[0049] FIGS. 26A-26C are perspective views of a de-coning process of hanging stacker cones, C02;

[0050] FIGS. 27A-27D are perspective views of a coning process of hanging stacker cones, C02;

[0051] FIGS. 28A-28C are perspective views of a de-coning process of hanging stacker cones, C01;

[0052] FIGS. 29A-29D are perspective views of a coning process of hanging stacker cones, C01;

[0053] FIG. 30A is an image showing a side view of a robot next to a container according to another embodiment of the present disclosure;

[0054] FIG. 30B is an image showing a front view of the grippers manipulating a fitting according to another embodiment of the present disclosure;

[0055] FIG. 31 is a schematic diagram showing the classification process of cones and corner blocks;

[0056] FIG. 32 is a schematic diagram illustrating the motion pathway of a robot during a de-coning operation;

[0057] FIG. 33 is a schematic diagram illustrating the motion pathway of a robot during a coning operation;

[0058] FIG. 34 is an image showing a mapping sensor including a camera and Li-DAR at a wharf, port or container environment;

[0059] FIG. 35 is an image showing the 3-Dimensional (3D) mapping of an image captured by the camera;

[0060] FIG. 36 is a schematic diagram illustrating the navigation pathway of a robot based on an algorithm according to an embodiment of the present disclosure;

[0061] FIG. 37 is an image showing a perspective view of a robot beside a cone storage rack according to an embodiment of the present disclosure; and

[0062] FIG. 38 is an image showing another perspective view of the robot in FIG. 37.DETAILED DESCRIPTION

[0063] The following detailed description is made with reference to the accompanying drawings, showing details and embodiments of the present disclosure for the purposes of illustration. Features that are described in the context of an embodiment may correspondingly be applicable to the same or similar features in the other embodiments, even if not explicitly described in these other embodiments. Additions and / or combinations and / or alternatives as described for a feature in the context of an embodiment may correspondingly be applicable to the same or similar feature in the other embodiments.

[0064] In the context of various embodiments, the articles “a”, “an” and “the” as used with regard to a feature or element include a reference to one or more of the features or elements.

[0065] In the context of various embodiments, the term “about” or “approximately” as applied to a numeric value encompasses the exact value and a reasonable variance as generally understood in the relevant technical field, e.g., within 10% of the specified value.

[0066] As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0067] The term “pose” may include a position and an orientation of an object or part of an object. The term “position” may refer to a location or coordinate (for example, X-coordinate, Y coordinate, Z coordinate) of an object or part of an object in a space or a frame. The term “orientation” may refer to a facing or angle (for example, an X-direction vector, a Y-direction vector, a Z-direction vector) of an object or part of an object in a space or a frame.

[0068] The term “container fitting”, “container cone”, “fitting”, “cone”, “container connector” may be used interchangeably to refer to a connecting device for coupling two adjacent shipping containers. The container fitting may provide a detachable coupling between two shipping containers disposed vertically.

[0069] According to various embodiments of the disclosure, FIG. 1 is a schematic diagram of a robotic system 50 for manipulating a container fitting 70. The robotic system 50 comprises a robot 100 for manipulating the container fitting 70 and a mapping sensor 60 positioned in the container environment remote from the robot 100. The robot 100 comprises a mobile base 105, a vision module 120, a first robotic arm 200, and a second robotic arm 300. The vision module 120 is coupled to the mobile base 105 and is being configured to identify the container fitting 70. The first robotic arm 200 is coupled to the mobile base 105 and comprises a first gripper 210. The second robotic arm 300 is coupled to the mobile base 105 and is configured to comprise a second gripper 310. The first gripper 210 and the second gripper 310 are cooperatively actuatable to couple the container fitting 70 to a pair of adjacent containers 72. The first gripper 210 and the second gripper 310 are cooperatively actuatable to decouple thecontainer fitting 70 from the pair of adjacent containers 72. In some embodiments, the robot 100 may further comprise a navigation module 130 that receives navigation instructions from the mapping sensor 60. In some other embodiments, as seen in FIG. 2, the robot 100 may further comprise a storage 140 disposed on the mobile base 105. The storage 140 may be used to hold the container fitting 70.

[0070] FIRST GRIPPER (T-GRIPPER)

[0071] According to FIG. 3 and FIG. 4, various embodiments of the first gripper 210 comprise a first gripping member 220 and a second gripping member 230. The second gripping member 230 is disposed spaced apart from the first gripping member 220. The first gripping member 220 comprises a first lattice palm 222. The second gripping member 230 comprises a second lattice palm 232. The first lattice palm 222 and the second lattice palm 232 may face one another to define a gripping space 410 therebetween. The first lattice palm 222 defines a first gripping surface 224, and the second lattice palm 232 defines a second gripping surface 234.

[0072] FIG. 6 is a close-up isometric view of FIG. 4 illustrating the first gripper 210 according to various embodiments of the present disclosure. In the present disclosure, as shown in FIGs. 6, the first gripper 210 may define reference axes for ease of understanding. For example, the first gripper 210 may define a gripping axis 410 along the lengths of the first gripping member 220 and the second gripping member 230. The first gripping member 220 and the second gripping member 230 are displaceable relative to each other along the gripping axis 410 to vary a gripping space 400. Each of the first gripping surface 224 and the second gripping surface 234 may be conformable to a container fitting along the gripping axis 410. The first gripping member 220 and the second gripping member 230 are disposed on a gripper base 250, the gripper base 250 is connected to a gripper holding platform 260.[0073 J In various embodiments, each of the first lattice palm 222 and the second lattice palm 232 may have a shear stiffness along a shear axis 412 higher than a compressive stiffness along the gripping axis 410. The shear axis 412 may be orthogonal to the gripping axis 410. In other words, the first lattice palm 222 and the second lattice palm 232 may be stiffer or more resistant towards deformation along the shear' axis in relative to deformation along the gripping axis 410.

[0074] Further, each of the first gripping member 220 and the second gripping member 230 comprises a pair of opposing extensions 420 extending along a width axis 414. The width axis 414 is orthogonal to the gripping axis 410. Each of the pair of opposing extensions 220 comprises at least one hook 430, the at least one hook 430 is extendable to an open position and retractable to a close position. The at least one hook 430 is biased towards the open position. Each of the at least one hook 430 is independently extendable and retractable. In some embodiments, the at least one hook 430 may be lockable at the close position.

[0075] FIGS. 7A-7C illustrates the various views of the first gripper 210 in FIG. 6. FIG. 7C shows a detailed view of the pair of opposing extensions 420 and the at least one hook 430 in the open position. In some embodiments, the motion of the at least one hook 430 of the pair of opposing extensions 420 between the open position and the close position is controlled by a passive mechanism having an open state as default. The open state corresponds to the open position of the at least one hook 430. When the at least one hook 430 catches onto features in the fittings, a trigger in the passive mechanism is pressed. Alternatively, the pair of opposing extensions 420 may be actuated by a spring or a roller.

[0076] FIGS. 8A and 8B illustrate the first gripper 210 in the open position and the close position respectively. In FIG. 8A, the at least one hook 430 is fully extended along the gripper axis 410, while in FIG. 8B, the at least one hook 430 are rotated orthogonally to the gripper axis 410 and pressed into a respective slot 432 disposed on the pair of opposing extensions 420. FIGS. 8A and 8B highlight the two passive states of the first gripper 210. FIG. 8A shows thedefault state of the first gripper 210 (open state), in this state, the first gripper 210 is capable to trigger the hidden locks by performing pulling motions. FIG. 8B shows the full hidden state when both of the at least one hook 430 are pressed into their respective slot 432.

[0077] The first gripper 210 triggers hidden features in common locks such as stacker fittings, mid-locks and semi-automated twist locks. The first gripper 210 can securely grasp the knob member of the fittings to enable twisting, pulling, and turning during coning and deconing operations.

[0078] LATTICE PALM

[0079] FIGS. 9A-9E illustrate various views of a lattice palm 150 w'ith a lattice point identifying a unit cell of the structure. The lattice palm 150 comprises a plurality of polymer units. The three-dimensional unit cells are connected in three orthogonal plans to form a repeated structure. The plurality of polymer units is arranged into a known geometry to complete the lattice palm 150 used in de-conning and conning activities. The populated geometry can be printed using Fusion Deposition Manufacturing (FDM) printing technologies (e.g. Prusa i3Mk3 3D printers, etc.) and Thermoplastic Polyurethan (TPU) filaments (e.g. Ninjaflex 85A) at 230°C. The first 222 and the second 232 lattice palm have a first 224 and a second gripping surface 234, respectively, that contacts with and conforms to various surface geometries of the container fitting 70. The lattice palm 150 has a sufficiently small compression modulus for ease of conformation to various knob geometries when the first gripper 210 is in a closed state. Concurrently, the lattice palm 150 also has a sufficiently high shear modulus to provide sufficient resistance to transmit required forces and torque to the fittings. The high shear modulus prevents the lattice palm 150 from deforming or slipping when the first gripper 210 is applying a torque or a shearing force on the container fitting 70 when the first gripper210 is in a closed state.

[0080] FIG. 9B shows a side view of the lattice palm 150 with an irregular thickness, starting from 20mm at an edge and 25mm at the opposite edge. FIG. 9D shows a back view of the lattice palm 150 in which each of the plurality of polymer units 152 has a radius of 2.5mm in a segment dimensioned at 60mm x 35mm (L x H).

[0081] An initial assumption to predict the required encompassing gripping force is illustrated in Table 1 :

[0082] Table 1: Assumed variables[OO83J The formula to estimate the gripping force required is as follows:F = W(G + AR)(Jf) (1)

[0084] Where F = Compression force (N), W = mass of corner cast in kg, G = gravity, AR = linear acceleration of robot arm end effector in ms"2, and Jf = jaw factor of the first gripper 210. To estimate the linear acceleration of the first robotic arm 200, a conversion of radial acceleration to linear acceleration iththe formula as:AR = ar sin 9 (2)

[0085] Where a - angular acceleration of the first robotic arm 200 end effector in degrees2, and r = is the offset radius of the first robotic arm 200 from the gripper base 250, and 9 = final angle of end effector. By estimating the required gripping force, an estimation of compression modulus can be expressed as:

[0086] Where MC = modulus of compression, and At = total area of the lattice palm 150 experiencing the compression force.

[0087] The lattice palm 150 chosen is designed using a minimal surface formula of SchawrzP , where it is described that an estimation of a repeating minimal surface with torus 3 can be written as: sp(x, y,z) cos(x) + cos(y) + cos(z) = 0 (4)

[0088] FIG. 10A and FIG. 10B shows the isometric and front view of a lattice cell structure prior to compression. FIG. 10C shows the compressed lattice cell structure under an applied compressive force of 960N during a compressive test. The test setup comprises an Imada tensile tester as seen in FIG. 10D, and a unit lattice sample of 4cm x 4cm x 4cm (LxWxH). The unit lattice sample is subject to compressive forces of up to 500N and compressed until failure or when the emergency machine stops due to excessive compressive force. The data gathered from the experiments are plotted against simulation data to compare the properties of the lattices. The simulation data is generated using Abaqus software, with a range of input compressive forces from 560N (assumed an input pressure of 60 PSI / 0.4MPa) up to 960N (assumed input pressure of 90PSI / 0.6MPa), using Ninjaflex 85 A material properties as shown in Table 2.

[0089] Table 2: Material properties of Ninja Tek TPU 85A

[0090] The maximum compressive force and modulus obtained from the experiments are listed in Table 3:

[0091] Table 3: Maximum compressive force and displacement recorded from the experiment

[0092] FIG. 14A compares the experiment data against the simulation data of stress-strain behavior of a lattice cell, and FIG. 14B shows the strcss-strain curve for a simulated shear loading of the lattice structure. The stress-strain curve is obtained by converted a force by displacement graph from the Imada tensile machine. FIG. 14A shows the predicted stress-strain behavior of the lattice unit against the experiment results under compressive load. It is observed that the lattice unit can exert up to 0.17 MPa at 50% strain. FIG. 14B plots the simulated stress strain behavior of shear loading on the lattice unit. It is observed that the shear modulus is 2.4MPa at 10% strain. These results confirm that the lattice palm 150 displays two crucial functions: (1) The unit has low enough compressive modulus to easily conform around objects when the gripper closes and grasps around cones, and (2) the unit simultaneously is capable of restricting shear deformations (shear modulus roughly and order of magnitude larger than compressive modulus) present during conning and dc-conning operations where gripping and twisting are necessary. Hence the lattice palm 150 can enable a close secure grasp but will not deform when the first gripper 210 applies a torque to the fitting 70.

[0093] SECOND GRIPPER (OFFSET GRIPPER)

[0094] Referring again to FIG. 3 and FIG. 5, some embodiments of the second gripper 310 comprise a first holding arm 320 and a second holding arm 330. To aid understanding, an insertion axis 416 is defined along the lengths of the first holding arm 320 and the second holding arm 330. The second gripper 310 may further describe a holding axis 418 that is oriented orthogonally to the insertion axis 416. The first holding arm 320 and the second holding arm 330 extend along the inserting axis 416, the second holding arm 330 is spaced apart from the first holding arm 320 along the holding axis 418 to define a holding space 402. Likethe first gripper 210, the first holding arm 320 and the second holding arm 330 are disposed on the gripper base 250, the gripper base 250 is connected to the gripper holding platform 260.

[0095] FIGS. 15, 16A to 16C illustrates various views of the second gripper 310. Each of the first holding arm 320 and the second holding arm 330 may further comprise at least one hook 450. The at least one hook 450 is extendable to an open position and retractable to a close position. The at least one hook 450 is biased towards the open position (as shown in FIG. 1 ). In various embodiments, the at least one hook is lockable at the close position. Each of the at least one hook 450 is independently extendable and retractable.

[0096] The second gripper 310 is designed to hold the neck or collar 74 of a wide range of fittings 70. Its main function is to fit the collar of various cones to hold and lock them in position while deconning and conning. The second gripper 310 is also outfitted with the at least one hook 450, located at the tips of a pair of extensions 440, placed strategically to trigger hidden cone triggers or locks. The at least one hook 450 functions as a secondary triggering mechanism to trigger hidden features. A cone holder 340 located at the back of the gripper is used to engage triggers that are found in some twist locks.

[0097] SLEEVE SENSOR

[0098] According to embodiments of the present disclosure, the first gripping member 220 and the second gripping member 230 each includes a sensor. The first gripper 210 includes a first sleeve sensor 226 at least partially enclosing the first gripping surface 224. The sensor may be a Force Resistive Sensor (FRS). The first sleeve sensor 226 is configured to measure a first force applied to the first gripping surface 224. The second gripper 310 includes a second sleeve sensor 236 at least partially enclosing the second gripping surface 224. The second sleeve sensor configured to measure a second force applied to the second gripping surface 224. Each of the first force and the second force comprises a compressive force along the gripping axis 410 and a shear force along the shear axis 412. The inclusion of a sensor allows the first 220and the second gripping members 230 to form a control and feedback system. The sensors may be made with conductive materials (e.g. conductive TPU filament, procured from NinjaTek). An embodiment of the FRSs as shown in FIGS. 1 IB and 11C illustrates a sleeve 162 that covers the lattice palm 150.

[0099] FIG. 12 shows the experimental set up in which a multimeter is connected to a known resistor attached to the sleeve 162 to measure the voltage (V) change as the force (N) applied to the lattice palm 150 increases. The increase in shear force causes an increase in resistance ( ), therefore causing a reduction in V reading. An example of the voltage against force readings is shown in table 4:

[0100] Table 4: Minimum and Maximum reading of voltage and force of the force sensor

[0101] According to another embodiment of the present disclosure, the sensor may be a lattice sensor 164, as shown in FIGS. 11C and 11D. The lattice sensor 164 works as a flexible resistor whose resistivity changes due to compressive, or shear forces applied to the lattice palm 150. Changes in resistivity provide feedback on the forces exerted by or applied to the first gripper 210. A similar test was conducted to the lattice sensor 164 as to the sleeve sensor 160. Table 5 shows the recorded voltage against force reading:

[0102] Table 5: Minimum and Maximum reading of voltage and force of the lattice sensor164[00103 J FIG. 13 is a schematic diagram illustrating the control and feedback loop employed by the grippers. When the sensor receives feedback, it evaluates the voltage reading against a predetermined threshold to determine whether an object has been grasped. This decision is then sent as an input to the user control system, which subsequently regulates the fluid control system to adjust the gripper's position between open and closed.

[0104] When the voltage is lower than the threshold value, the system determines that the applied force is insufficient, and the object has not been grasped. An input is then sent to the user control system, followed by the fluid control system, to adjust the gripper to a close position for a secure grasp.

[0105] When the voltage is higher than the threshold value, the system determines that sufficient force has been applied, and the object has been successfully grasped. An input is then sent to the user control system, followed by the fluid control system, to maintain or adjust the gripper's position accordingly. If excessive force is detected, the system may regulate the fluid control mechanism to slightly release pressure, preventing potential damage to the object or the gripper.

[0106] EXAMPLE STRATEGIES FOR VARIOUS TYPES OF LOCKS

[0107] A set of strategies or sequential steps is formulated to handle fittings 70 for either coning or de-coning operations. The strategies involve the use of the first gripper 210 (e.g. T- Gripper) and the second gripper 310 (e.g. offset-gripper) positioned to individually or collaboratively perform a set of actions interacting with key cone 76 or fitting elements (knob, collar, trigger). The fittings 70 as shown in FIG. 17 are chosen as a representative set for a range of fully automatic twist locks, semi-automatic twist locks, mid-locks, and stacker cones as shown in FIGS. 18 to 29. A set of features are analyzed for each fitting 70 and generalized as 1) knob, the knob feature consists of pyramid like element having a tip pointed downwards that can be used to twist or rotate the fitting 70 held within the comer cast or storage systems, 2)collar, the collar feature consists of a longitudinal edge or a planar surface or a combination of that can be used as a support to hold the cone and 3) trigger, the trigger features are elements that are partially or fully spring-loaded levers or geometrical structures which hold the cone or fitting elements in lock or unlock positions. The mass of the fittings 70 ranges from 1 kg to 10 kg depending on design and geometrical construction of elements. The strategies are designed considering particular fitting elements, gripper geometry, and gripper’s functionalities.

[0108] FIGS. 18A-18C illustrate the steps for de-coning a family of semi-automatic twist locks, C01 and C03. First, the second gripper 310 grasp the collar 74 located between the corner cast 80 and the cone 76. The collar grasping configuration holds a supporting platform to grasp and support the collar 74 of fittings 70 against gravity. Next, the first gripper grasps the cone. Lastly, the first gripper rotates clockwise or counterclockwise to unlock the cone 76. Both the first gripper 210 and the second gripper 310 then moves away from the comer cast 80 to separate the fitting 70 from the corner cast 80.

[0109] FIGS. 19A-19D illustrate the steps for coning the abovementioned family of semiautomatic twist locks, C01 and C03. First, the first gripper 210 is rotated clockwise or counterclockwise to unlock the cone 76 and move the fitting 70 into the comer cast 80. Next, the first gripper’s 210 grasp on the cone is released to lock the cone. Lastly, the second gripper 310 is shifted away from the collar 74 to release the second gripper’ s 310 grasp, thereby locking the fitting 70 onto the comer cast 80.

[0110] FIGS. 20A-20C illustrate the steps for de-coning a type of self-hanging stacker, C04. First, the second gripper grasps the collar 74 located between the corner cast 80 and the cone 76. Next, the at least one hook 430 of the pair of opposing extensions 420 of the first gripper 210 catches onto the fitting 70 and unlocks the trigger. Lastly, the first gripper 210 rotates clockwise or counterclockwise to unlock the cone 76. Both the first gripper 210 and the secondgripper 310 are pulled in a downwards motion and separate the fitting 70 from the corner cast 80.

[0111] FIGS. 21A-21C illustrate the steps for coning self-hanging stackers, C04. First, the second gripper 310 rotates clockwise or counterclockwise to grasp the collar 74. Concurrently, the trigger of the cone 76 is unlocked by the first gripper 210 and both the first gripper 210 and the second gripper 310 move the fitting 70 towards the corner cast 80. Next, the first gripper 210 releases and unlocks from the cone 76 by moving away from the corner cast 80. Lastly, the second gripper 310 is shifted away from the collar 74 to release the second gripper’ s 310 grasp. The fitting 70 is inserted into the comer cast 80.

[0112] FIGS. 22A-22C illustrate the steps for de-coning a type of self-hanging stacker, C12. First, the firstgripper 210 is in an open position, the at least one hook 430 of the pair of opposing extensions 420 catches onto the fitting 70 and the passive mechanism is triggered. Subsequently, the first gripper 210 moves into the close position to grasp onto the cone 76. The first gripping member 220 and the second gripping member 230 move closer to one another. The first gripping surface 224 and the second gripping surface 234 interacts with the cone 76, which is disposed within the gripping space 400. Upon the first gripper’s 210 engagement with the fitting 70, the first gripper 210 is rotated clockwise or counterclockwise while being pulled in a downwards motion to unlock the cone 76. The fitting 70 is thereby separated from the corner cast 80.

[0113] FIGS. 23A-23C illustrate the steps for coning self-hanging stackers, C12. Similarly, the first gripper 210 grasps the cone 76 and rotates clockwise or counterclockwise while moving concurrently in an upward motion to fit the cone 76 into the comer cast 80. Next, upon locking the fitting 70 into the comer cast 80, the first gripper 210 releases its grasp from the cone 76 as the first gripping member 210 and the second gripping member 310 moves away from one another.[00114J FIGS. 24A-24D illustrate the steps for de-coning a type of fully automatic twist locks, F01. First, the second gripper 310 grasps the collar 74 of the fitting located in between the corner cast 80 and the cone 76. Next, the at least one hook 430 of the pair of extensions 440 of the first gripper 210 catches onto the cone 76 and unlocks the trigger. Then, both the first 210 and the second grippers 310 are rotated clockwise or counterclockwise to unlock the cone 76. Lastly, both the first 210 and the second grippers 310 move in a downwards motion to remove the fitting 70 from the corner cast.

[0115] FIGS. 25A-25D illustrate the steps for coning fully automatic twist locks, F01. First, the second gripper 310 grasps the collar 74 of the fitting 70 while the first gripper 210 grasps the cone 76, then engages and unlocks the trigger. This places the cone 76 in an open or unlatched position. Both the first 210 and the second grippers 310 then move in an upward motion to bring the fitting 70 closer to the comer cast 80. Next, both the first 210 and the second grippers 310 are rotated clockwise or counterclockwise to place the fitting 70 into the corner cast 80. Then, the first gripper 210 releases the trigger by shifting upwards, the cone 76 switches to a close position and latches to the corner cast 80. Lastly, the second gripper 310 releases its grasp on the collar 74 by moving away from the fitting 70. The fitting 70 is locked into the corner cast 80.

[0116] FIGS. 26A-26C illustrate the steps for de-coning a type of hanging stacker cone, C02. First, the second gripper 310 grasps the collar 74 of the fitting 70, and brings the fitting 70 towards the corner cast 80. Next, at least one integrated flap in the second gripper 310 engages the trigger in the cone 76 and the trigger is unlocked when the second gripper 310 moves away from the cone 76. This changes the cone 76 into an open position. Lastly, the second gripper 310 moves in a downwards motion to remove the fitting 70 from the corner cast80.[00117J FIGS. 27A-27D illustrate the steps for coning the hanging stack cone, C02. First, the second gripper 310 grasps the collar 74 of the fitting 70, and brings the fitting 70 towards the corner cast 80. Next, the second gripper 310 rotates clockwise or counterclockwise to lock the cone 76 into the comer cast 80. Lastly, the second gripper 310 releases its grasp on the collar 74 by moving away from the fitting 70. The fitting 70 is locked into the corner cast 80.[001 18] FIGS. 28A-28C illustrate the steps for de-coning a type of stacker cone, C01 . First, the second gripper 310 grasps the collar 74 of the fitting 70. Next, the second gripper 310 rotates clockwise or counterclockwise to unlock the cone 76, the cone 76 changes from a close position to an open position. Last, the second gripper 310 pulls in a downward motion to release and separate the cone 76 from the corner cast 80.[001 19] FIGS. 29A-29D illustrate the steps for coning the stacker cone, C02. First, the second gripper 310 grasps the collar 74 of the fitting 70, and brings the fitting 70 towards the comer cast 80. Next, as the cone 76 is in an open position, the second gripper 310 rotates clockwise or counterclockwise to change the cone 76 into a close position and locks it to the corner cast 80. Last, the second gripper 310 releases its grasp on the collar 74 by moving away from the fitting 70. The fitting 70 is locked into the corner cast 80.

[0120] WORKFLOW

[0121] FIG. 30A shows a side view of the robot 100 next to a container’s 90 edge, whereby a fitting 100 is engaged with a comer cast 80 of the container 90. FIG. 30B shows a front view of the robot 100 whereby the first gripper 210 is grasping the cone 76 while the second gripper 310 is grasping the collar 74 of the fitting 70. In some embodiments of the present disclosure, the vision module 120 is configured to determine at least one of a fitting type, a position, and an orientation, of the container fitting 70. As shown in a classification approach shown in FIG. 31, the vision module 120 detects the location of fittings 70 on the container 90 through an RGB-D image. A bounding box model is used to process the RGB-D image captured, where arectangular box aligns with the image axes and a mask outlines the shape of the fitting inside the bounding box. The processed image then undergoes 3D image extraction to obtain a reconstructed 3D representation by retrieving depth and structure from the 2D image. The 3D models of the cones and comer blocks (corner casts 74) are pre-built in a database. When the subsystem camera captures the cone or comer block image, an machine learning or artificial intelligence model first finds the corner cast then the cone, then the 3D point cloud of the cone is matched to the database, and a registration process is performed to find a six-degree-of- freedom (6D) pose in space and output the cone type. The 6D pose then defines the fitting’s position and orientation in 3D space. The cone’s type and geometry are also classified accordingly for the robot to plan end-effector poses and the first and the second gripper’s trajectories.

[0122] DECONING AND CONING WORKFLOW

[0123] The present application may disclose a method of manipulating a container fitting 70. The method includes identifying the container fitting 70 by using the vision module 120 of the robot 100 and controlling at least one of the first robotic arm 200 and the second robotic arm 300 of the robot 100, based on a motion planning library, to manipulate the container fitting 70. The manipulation of the container fitting 70 includes coning or de-coning the container fitting 70 from a pair of adjacent containers 72. The identification of the container fitting 70 includes determining at least one of a fitting type, a position, and an orientation of the container fitting 70.

[0124] Upon classification of the fittings 70 and corner blocks, the robot 100 may tap into the motion planning library that includes a suite of integrated algorithms for advanced multiarm coordinated manipulation capabilities in a tightly constrained and changing spatial environment. The algorithms have a library of the specific de-coning or coning motion flows for the vastly different cone types, which will be extracted based on the cunent job and conetype. The algorithms then capture surrounding obstacles in real time using 3D vision sensors. Using obstacle detection, the algorithms then plan the motions for the multi manipulator arms that avoid collision with the surrounding objects in the environment to perform the de-coning or coning task, as well as passing or getting the cones into or from the onboard magazine. Fittings that are light and do not involve any triggers could be handled by only one ami. FIG. 32 illustrates a manipulation motion involving the first robotic arm 200 and the second robotic arm 300 for de-coning tasks. The workflow system includes 4 work packages (WP), WP 1 to WP 4. WP 1 controls the positioning of the first 200 and the second robotic arms 300. WP 2 identifies the target cone and its 3D position. WP 4 identifies the target comer cast. Based on the output from WP 4, it is determined whether a double-arm operation (involving both the first 200 and the second robotic arm 300) or a single-arm operation (using either the first 200 or the second robotic arm 300) is required. This result, along with the target cone’s identification from WP 2, is then provided as input to WP 1. Using the unsynchronized obstacle-avoidance planner from the algorithm library, the first 200 and the second robotic arms 300 may execute a scries of motions, including approaching the cone 76 and performing the de-coning routine after receiving the target cone’s 6D pose input from WP 2. Such a decision-making process is based on changes in the operational environment and local sensory input, instead of a global planning framework. Subsequently, the unsynchronized or synchronized obstacle-avoidance planner may control the first 200 and the second robotic arms’ 300 movement to intermediary points, to approach magazines and to slot into cones. The first 200 and the second robotic arms 300 work in harmony to avoid obstacles.

[0125] FIG. 33 illustrates a manipulation motion involving the first robotic arm 200 and the second robotic arm 300 for de-coning tasks. In the coning workflow, WP 4 identifies the target corner cast and determines whether a double-arm operation (involving both the first 200 and the second robotic aim 300) or a single-arm operation (using either the first 200 or the secondrobotic arm 300) is required. Simultaneously, WP 1 identifies the target cone and provides this information as input for further decisions regarding the positioning of the first 200 and the second robotic arms 300. The first 210 and the second grippers 310 are then controlled to approach the magazine and extract the cone. Next, they move to intermediary points, with motion guided by the unsynchronized or synchronized obstacle-avoidance planner. Finally, WP 2 identifies the corner cast’s position and provides this data, enabling the first 200 and the second robotic arms 300 to locate and approach the corner cast before initiating the coning routine.

[0126] ROBOT NAVIGATION

[0127] The present application also discloses a robotic system 50 for a container environment. The robotic system 50 includes the robot 100, and a mapping sensor 60 positioned in the container environment remote from the robot 100. The mapping sensor 60 is configured to provide a navigation information (e.g. the goal position on a container) to the robot 100. The robot 100 is configured to navigate the container environment responsive to receiving the navigation information from the mapping sensor 60. In some embodiments, the camera in the mapping sensor 60 searches for the x and y coordinates of the corner cast in 2D space, while the robot’s 100 on-board camera identifies the x, y, z in 3D space.

[0128] During a de-coning operation, once the robot 60 reaches the target location and the type of the container fitting 70 as identified by the mapping sensor 60, the first 200 and the second robotic arms 300 then use the appropriate trajectories to grasp the target cone 76 and proceed with the sequence of motions to remove or attach it to the comer cast 80.

[0129] On the other hand, during a coning operation, once the robot 100 reaches the target location provided by the mapping sensor 60, the robot 100 uses the appropriate coordinated trajectories to insert the fitting 70 in the target corner cast 80. The robot 100 then proceeds tothe next coning target location. Once all target locations have been coned, the robot 100 navigates to a resting location.

[0130] RACK

[0131] The robotic system 50 may further include a cone storage unit array 500 for holding a plurality of container fittings 70. After a de-coning operation, the first 200 and the second robotic arms 300 bring the cone 76 into the storage 140 on board for temporary storage. The robot 100 then proceeds to the next de-coning target location. Once all target locations have been de-coned, the unit navigates to the cone storage unit array 500 and unloads all the onboard cones into the cone storage unit array 500. Alternatively, after a coning operation, the robot 100 navigates to the cone storage unit array 500 to retrieve all the cones required for the coning operation. FIGS. 37 and 38 show the robot 100 situated beside the cone storage unit array 500, the robot 100 carrying cones 70 in the storage 140. This retrieval could also be done using other conveyor-based mechanisms to the storage 140 on the robot 100. The cone storage unit array 500 sorts and stores the bulk number of fittings by type. An optional docking mechanism allows for communication between a robot’s 100 storage 140 and the rack, facilitating the exchange of fittings. The rack is powered by a 24v battery and Wi-Fi microcontrollers, coordinating various operations such as gantry positioning, and sorting-robot actions.

[0132] The navigation module 130 on the robot 100 includes a suite of integrated navigation algorithms for the robot 100 to reach the target cone or corner cast positions based on the goal positions provided by the mapping sensor 60 as shown in FIG. 34. The algorithms are for advanced mobile robot capabilities in a spatially constrained and dynamic wharf-side environment. The algorithms comprise mapping, localization, object (goal and obstacle) detection, collision avoidance, path planning and motion control modules for a non-holonomic mobile robot. The mapping sensor 60 includes a camera and Li-DAR. FIG. 36 schematically illustrates the navigation pathway of the robot 100 during operation. Firstly, a 3D map of theenvironment is obtained offline with static obstacles only (no trailers or moving objects). The map is filtered and processed for the ground plane removal.

[0133] During the online execution phase, the robot 100 uses the 3D map for selflocalization and navigation. Specifically, the robot 100 uses the localization module to estimate its pose in the map and plan a path to the desired goal location. Once the planned path is obtained, the controller module provides motion commands to the robot 100 to closely follow the waypoints of the path. The goal locations are dependent on the sub-tasks that are needed to be executed for coning and de-coning. These sub-tasks include motions to or fro between the cone storage unit array 500 and comer cast locations, and motions to or fro between the comer cast 80 locations or the cone storage unit array 500 and docking station. Executing these subtasks requires identifying the locations of the corner cast 80, the cone storage unit array 500 and docking station in the 3D map’s frame of reference. Since the latter two goal locations (cone storage unit array 500 and docking station) are static, they are reliably detected in realtime.

[0134] However, the corner cast locations are dynamic and can change due to the variability in parking locations of the trailer and positioning of the containers on the trailer. The mapping sensor including the overhead camera and Li-DAR sensor fusion based comer cast detection module provides estimates of the 2D pixel coordinates of the target corner casts. The 2D pixel coordinates are then transformed to 3D coordinates by using inverse perspective transformation to obtain coordinates in the map coordinate frame of reference. The proposed suite of algorithms provides capabilities for the robot 100 to navigate in environments alongside human drivers, crane and ground operators. The objective is to provide autonomy solutions for precise, agile, reliable and safe navigation capabilities for the robot 100 to operate in highly constrained and dynamic environments to ensure accurate localization and efficient movement towards specified goal point.[OO135J All examples described herein, whether of methods, materials, or products, are presented for the purpose of illustration and to aid understanding and are not intended to be limiting or exhaustive. Modifications may be made by one of ordinary skill in the art without departing from the scope of the invention as claimed.

Claims

CLAIMS1. A robot for manipulating a container fitting, comprising: a mobile base; a vision module coupled to the mobile base, the vision module being configured to identify the container fitting; a first robotic arm coupled to the mobile base, the first robotic arm comprising a first gripper; and a second robotic arm coupled to the mobile base, the second robotic arm comprising a second gripper; wherein the first gripper and the second gripper are cooperatively actuatable to couple the container fitting to a pair of adjacent containers, wherein the first gripper and the second gripper are cooperatively actuatable to decouple the container fitting from the pair of adjacent containers.2, The robot as recited in claim 1, wherein the first gripper comprising: a first gripping member comprising a first lattice palm; a second gripping member disposed spaced apart from the first gripping member, the second gripping member comprising a second lattice palm, wherein the first lattice palm and the second lattice palm faces one another to define a gripping space therebetween, wherein the first lattice palm defining a first gripping surface and the second lattice palm defining a second gripping surface; wherein the first gripping member and the second gripping member are displaceable relative to each other along a gripping axis to vary the gripping space.

3. The robot as recited in claim 2, wherein each of the first gripping surface and the second gripping surface is conformable to the container fitting along the gripping axis.

4. The robot as recited in any one of the above claims, wherein each of the first lattice palm and the second lattice palm comprises a plurality of polymer units.

5. The robot as recited in any one of the above claims, wherein each of the first lattice palm and the second lattice palm has a shear stiffness along a shear axis higher than a compressive stiffness along the gripping axis, wherein the shear axis is orthogonal to the gripping axis.

6. The robot as recited in claim 5, further comprising: a first sleeve sensor at least partially enclosing the first gripping surface, the first sleeve sensor being configured to measure a first force applied to the first gripping surface; and a second sleeve sensor at least partially enclosing the second gripping surface, the second sleeve sensor being configured to measure a second force applied to the second gripping surface.

7. The robot as recited in claim 6, wherein each of the first force and the second force comprises: a compressive force along the gripping axis and a shear force along the shear axis.

8. The robot as recited in any one of the above claims, wherein each of the first gripping member and the second gripping member further comprising: a pair of opposing extensions extending along a width axis, the width axis orthogonal to the gripping axis, wherein each of the pair of opposing extensions comprises at least one hook, the at least one hook being extendable to an open position and retractable to a close position.

9. The robot as recited in claim 8, wherein the at least one hook is biased towards the open position.

10. The robot as recited in any one of claims 8 to 9, wherein each of the at least one hook is independently extendable and retractable.

11. The robot as recited in any one of the above claims, wherein the second gripper comprising: a first holding arm extending along an inserting axis; a second holding arm extending along the inserting axis, the second holding arm spaced apart from the first holding arm along a holding axis to define a holding space.

12. . The robot as recited in claim 11, wherein each of the first holding arm and the second holding arm further comprising at least one hook, the at least one hook being extendable to an open position and retractable to a close position.

13. The robot as recited in claim 12, wherein the at least one hook is biased towards the open position.

14. The robot as recited in any one of claims 11 to 13, wherein each of the at least one hook is independently extendable and retractable.

15. The robot as recited in any one of the above claims, further comprising a navigation module for navigating the robot.

16. The robot as recited in any one of the above claims, further comprising a storage disposed on the mobile base, the storage for holding the container fitting.

17. The robot as recited in any one of the above claims, wherein the container fitting comprises at least one of: a semi-automatic twist lock, a self-hanging stacker, a fully automatic twist lock, a hanging stacker cone, and a stacker cone.

18. The robot as recited in any one of the above claims, wherein the vision module is configured to determine at least one of: a fitting type, a position, and an orientation, of the container fitting.

19. A robotic system for a container environment, comprising: the robot as recited in any one of the above claims; a mapping sensor positioned in the container environment remote from the robot, the mapping sensor being configured to provide a navigation information to the robot.

20. The robotic system as recited in claim 19, wherein the robot is configured to navigate the container environment responsive to receiving the navigation information from the mapping sensor.

21. The robotic system as recited in any one of claims 19 to 20, further comprising a cone storage unit array for holding a plurality of container fittings.

22. A method of manipulating a container fitting, the method comprising: identifying the container fitting using the vision module of the robot as recited in any one of claims 1 to 18; based on a motion planning library, controlling at least one of the first robotic arm and the second robotic arm of the robot, to manipulate the container fitting.

23. The method as recited in claim 22, wherein to manipulate the container fitting comprises coning or de-coning the container fitting from a pair of adjacent containers.

24. The method as recited in any one of claims 22 to 23, wherein identifying the container fitting comprises determining at least one of: a fitting type, a position, and an orientation, of the container fitting.

Images