Cargo unloading automation method and computing device for performing same
The automation of cargo unloading using image recognition and robotic arm devices addresses the inefficiencies in the logistics industry by determining optimal unloading sequences and modes, enhancing efficiency and safety in cargo handling.
Patent Information
- Application Number
- PCT/KR2023/006953
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-11-28
AI Technical Summary
The logistics industry faces inefficiencies in cargo unloading, relying heavily on manual labor due to the lack of automation, particularly in unloading cargo from mainline freight trucks, which is labor-intensive and contributes to worker avoidance of this task.
A method and system for automating cargo unloading using a computing device and robotic arm devices capable of multi-axis rotation and movement, equipped with conveyor belts, suction means, and clamp means, which recognize and unload cargo based on image analysis, determining the unloading order and mode through boundary and depth information, and handle caution labels for safe operation.
The system enables efficient and safe automation of cargo unloading, reducing labor requirements and improving operational efficiency by determining optimal unloading modes and sequences based on cargo location, size, and type, while ensuring compliance with caution labels.
Smart Images

Figure KR2023006953_28112024_PF_FP_ABST
Abstract
Description
Method for automating cargo unloading and computing device for performing the same
[0001] An embodiment of the present invention relates to automated cargo unloading technology. The present invention was derived from research conducted as part of the Ministry of Trade, Industry and Energy's Robot Industry Core Technology Development Project - Inter-Ministry Collaborative Robot Product Technology (Project Identification Number: 1415168943, Subproject Number: 20009109, Research Project Title: Development of a Robot-Used Trunk Freight Logistics Vehicle Unloading System, Supervising Organization: STC Engineering Co., Ltd., Research Period: May 1, 2020 - December 31, 2024).
[0002] Over the past three years, parcel volume has increased annually by 10-13%, but the average delivery cost has been declining by 2-3% annually. This makes automation crucial for maintaining corporate competitiveness in the logistics industry. While a significant portion of the parcel delivery process is currently automated within logistics centers using equipment like belt sorters, the unloading process, which involves unloading various parcels from vehicle loading bays, remains a manual process.
[0003] To handle the nationwide daily delivery demand, trunk cargo trucks at hub terminals typically carry approximately 2,000 pieces of cargo per truck, requiring two workers to unload all of them within an hour. This is the most demanding task at a parcel hub terminal, making it a highly undesirable task for workers. Therefore, the development of a robotic system capable of automating trunk cargo truck unloading is essential.
[0004] The purpose of the present invention is to provide an automated cargo unloading method capable of recognizing cargo in a loading box and automatically unloading it, and a computing device for performing the same.
[0005] Meanwhile, the technical tasks to be achieved in the present invention are not limited to the technical tasks mentioned above, and other technical tasks not mentioned can be clearly understood by a person having ordinary knowledge in the technical field to which the present invention belongs from the description below.
[0006] A method for automatically unloading cargo according to one embodiment of the present invention is a method performed in a cargo unloading automation machine including one or more processors and a memory storing one or more programs executed by the one or more processors, the method comprising: obtaining a loaded cargo image photographing the inside of a loading box in which cargo is loaded; recognizing each loaded cargo based on boundary information and depth information of each object in the loaded cargo image; generating loaded cargo-related information including at least one of a position, a size, a type of each recognized cargo, and a loading pattern of the cargo within the loading box; and determining at least one of an unloading order of the cargo within the loading box and an unloading mode of the cargo based on the loaded cargo-related information.
[0007] The above cargo unloading automation method may further include a step of checking whether a caution label is attached to the cargo through the loaded cargo image; and, if the caution label is attached, a step of analyzing the caution label to check the type of caution.
[0008] The above cargo unloading automation method may further include a step of assigning a cargo caution classification index to the corresponding cargo according to the type of the caution; and a step of unloading the corresponding cargo from the loading box based on the cargo caution classification index.
[0009] The above cargo unloading automation machine comprises a pair of robotic arm devices, each including an arm unit that is configured to enable multi-axis rotation and forward and backward movement, and a hand unit that is connected to the arm unit and contacts the cargo to unload the cargo, and each hand unit may include an unloading conveyor belt that is configured to sweep the cargo inside the loading box.
[0010] The step of determining at least one of the above-mentioned cargo unloading order and the cargo unloading mode may include the step of checking whether the space around the cargo located at the topmost part of the loading box is empty beyond a preset space; and the step of determining the unloading mode of the cargo located at the topmost part as a down sweep mode when the upper part is empty beyond a preset space based on the cargo located at the topmost part.
[0011] The above down sweep mode can sweep the cargo downward by rotating the unloading conveyor belt toward the cargo unloading automation machine while positioning the unloading conveyor belt so as to face the cargo from above the cargo.
[0012] The step of determining at least one of the above-mentioned cargo unloading order and the cargo unloading mode may include the step of checking whether the space around the cargo located at the topmost part of the loading box is empty beyond a preset space; and the step of determining the unloading mode of the cargo located at the topmost part as a side sweep mode when both sides are empty beyond a preset space based on the cargo located at the topmost part.
[0013] The above-mentioned side sweep mode can be configured to sweep down the cargo by rotating the pair of the unloading conveyor belts toward the cargo unloading automation machine while positioning the loaded cargo between the pair of the unloading conveyor belts and arranging the pair of the unloading conveyor belts to face the cargo on both sides of the cargo.
[0014] Each of the above hand units further includes a suction means for suctioning cargo inside the loading compartment, and the step of determining at least one of the cargo unloading order and the cargo unloading mode may include the step of checking whether the space surrounding the cargo located at the uppermost part of the loading compartment is empty beyond a preset space; and the step of determining the cargo unloading mode as the suction mode depending on the type of the cargo located at the uppermost part, if the space surrounding the cargo is not empty beyond a preset space.
[0015] Each of the above hand units further includes a clamping means capable of picking up cargo inside the loading box, and the step of determining at least one of the cargo unloading order and the cargo unloading mode may include the step of checking whether the space surrounding the cargo located at the uppermost part of the loading box is empty beyond a preset space; and the step of determining the cargo unloading mode as a clamping mode depending on the type of the cargo located at the uppermost part when the space surrounding the cargo is not empty beyond a preset space.
[0016] According to one embodiment of the present disclosure, a computing device comprises one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein the one or more programs include: a command for obtaining a loaded cargo image photographing the inside of a cargo box in which cargo is loaded; a command for recognizing each loaded cargo based on boundary information and depth information of each object in the loaded cargo image; a command for generating loaded cargo-related information including at least one of a position, a size, a type of each recognized cargo, and a loading pattern of the cargo within the cargo box; and a command for determining at least one of an unloading order of cargo within the cargo box and an unloading mode of the cargo based on the loaded cargo-related information.
[0017] According to an embodiment of the present invention, a cargo unloading automation machine is provided with a pair of robot arm devices capable of multi-axis rotation and movement, and the robot arm devices are provided with an unloading conveyor belt, an adsorption means, and a clamp means, so that the machine operates in various unloading modes such as a down sweep mode, a side sweep mode, an adsorption mode, and a clamping mode depending on the position of the loaded cargo, the loading pattern of the cargo, and the type of the cargo, thereby enabling the loaded cargo to be unloaded easily and quickly.
[0018] Meanwhile, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.
[0019] Figures 1 and 2 are front perspective views and back perspective views showing an automated cargo unloading machine according to one embodiment of the present invention.
[0020] FIG. 3 is a drawing showing a state in which a cargo unloading automation machine according to one embodiment of the present invention enters the interior of a loading box and unloads cargo.
[0021] Figure 4 is a perspective view showing a robot arm device according to one embodiment of the present invention.
[0022] FIG. 5 is a drawing showing each axis part of the arm unit of a robot arm device according to one embodiment of the present invention.
[0023] Figures 6 and 7 are perspective views showing one side and the other side of a hand unit according to one embodiment of the present invention.
[0024] Figure 8 is a drawing showing a state in which the suction pad part moves forward in one embodiment of the present invention.
[0025] Figure 9 is a drawing showing a state in which a clamping part moves forward in one embodiment of the present invention.
[0026] Figures 10 to 16 are drawings showing the operation of a cargo unloading automation machine according to one embodiment of the present invention to unload loaded cargo from a loading box.
[0027] FIG. 17 is a drawing showing a process of determining an unloading mode in an automated cargo unloading method according to one embodiment of the present invention.
[0028] Figure 18 is a drawing showing a state in which a caution label is attached to cargo in one embodiment of the present invention.
[0029] Figure 19 is a flowchart showing a cargo unloading method using a cargo unloading automation machine according to one embodiment of the present invention.
[0030] FIG. 20 is a drawing showing a state in which a cargo unloading order and a cargo unloading mode are determined based on loaded cargo-related information according to one embodiment of the present invention.
[0031] FIG. 21 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments.
[0032] Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. The embodiments of the present invention may be modified in various ways, and the scope of the present invention should not be construed as being limited to the embodiments described below. These embodiments are provided to more fully explain the present invention to those of ordinary skill in the art. Accordingly, the shapes of elements in the drawings have been exaggerated for clarity.
[0033] In order to clearly solve the problem to be solved by the present invention, the composition of the invention is described in detail with reference to the attached drawings based on a preferred embodiment of the present invention, and when assigning reference numbers to components in the drawings, the same reference numbers are assigned to the same components even if they are in different drawings, and it is made clear in advance that components in other drawings may be cited when necessary when describing the drawings.
[0034] Meanwhile, directional terms such as upper, lower, one side, and the other side are used in reference to the orientation of the disclosed drawings. Since components of embodiments of the present invention can be positioned in various orientations, directional terms are used for illustrative purposes and are not intended to be limiting.
[0035] FIG. 1 and FIG. 2 are front and rear perspective views showing an automated cargo unloading machine according to one embodiment of the present invention, and FIG. 3 is a drawing showing a state in which an automated cargo unloading machine according to one embodiment of the present invention enters the interior of a loading box and unloads cargo.
[0036] FIG. 1 is a front perspective view showing an automated cargo unloading machine according to one embodiment of the present invention, and FIG. 2 is a rear perspective view showing an automated cargo unloading machine according to one embodiment of the present invention.
[0037] Referring to FIGS. 1 to 3, the cargo unloading automation machine (100) may include a robot arm device (102), a conveyor device (104), a track device (106), and a main body frame (108).
[0038] The cargo unloading automation machine (100) is for unloading cargo (hereinafter, referred to as loaded cargo) loaded in a loading box (50) or transporting it after unloading. Here, the loading box (50) may be a loading box mounted on a freight vehicle (e.g., a trunk freight car, etc.) or may be a container. The cargo unloading automation machine (100) may unload the loaded cargo in the loading box (50) at the entrance of the loading box (50), or, as illustrated in FIG. 3, may enter the interior of the loading box (50) and unload the loaded cargo in the loading box.
[0039] In addition, the cargo unloading automation machine (100) may include various sensor equipment (e.g., vision sensor or Lidar or vision and Lidar fusion sensor, scanner, etc.) to recognize the cargo loading pattern, cargo loading location, cargo type, etc. and determine the cargo unloading mode accordingly.
[0040] The robot arm device (102) may be provided on both sides of the main body frame (108). That is, the robot arm devices (102) may be provided as a pair on both sides of the main body frame (108). The robot arm devices (102) may be provided to operate as a pair like human arms to unload various types of cargo, such as boxes, plastic pouches, and sacks, that are piled in bulk form in a loading compartment. The robot arm device (102) may unload the loaded cargo in the loading compartment toward the conveyor device (104).
[0041] A conveyor device (104) may be mounted on the main body frame (108). The conveyor device (104) may be provided along the length direction of the robot arm device (102) on the main body frame (108). The conveyor device (104) may serve to transport the loaded cargo unloaded by the robot arm device (102) to the rear of the main body frame (108).
[0042] The conveyor device (104) can operate in conjunction with the robot arm device (102). The front end of the conveyor device (104) can be provided so that its height can be adjusted according to the operation of the robot arm device (102). At this time, the front end of the conveyor device (104) can be adjusted in height according to the height of the loaded cargo to be unloaded by the robot arm device (102).
[0043] The track device (106) may be provided at the lower part of the main body frame (108). The track device (106) may serve to move the cargo unloading automation machine (100). That is, the track device (106) may move the cargo unloading automation machine (100) to the entrance of the loading compartment or allow it to enter the interior of the loading compartment.
[0044] The main body frame (108) can serve to support the cargo unloading automation machine (100). The main body frame (108) can be provided in various forms that can support the cargo unloading automation machine (100), and its form is not limited thereto. The main body frame (108) can be provided to support the robot arm device (102). The main body frame (108) can be provided to support the conveyor device (104). In addition, the main body frame (108) can be connected to the track device (106) at the upper portion of the track device (106).
[0045] The main body frame (108) may be equipped with a power unit (108a) that provides power to the cargo unloading automation machine (100) and includes circuit devices that control the operation of the cargo unloading automation machine (100). In addition, the main body frame (108) may be equipped with a hydraulic tank (108b) for providing hydraulic pressure to at least one of the robot arm device (102) and the track device (106). In addition, the main body frame (108) may be equipped with at least one monitor (108c) for checking the operation of the cargo unloading automation machine (100).
[0046] FIG. 4 is a perspective view showing a robot arm device (102) according to one embodiment of the present invention, and FIG. 5 is a drawing showing each axis part of the arm unit of the robot arm device (102) according to one embodiment of the present invention.
[0047] Referring to FIGS. 4 and 5, the robotic arm device (102) may include an arm unit (111) and a hand unit (113).
[0048] The robot arm device (102) can be operated in an unloading mode determined based on one or more of the loading pattern of cargo in the loading box, the loading position of the cargo, and the type of cargo. The unloading mode is an operation mode for unloading cargo in the loading box, and for example, the unloading mode can include a down sweep mode, a side sweep mode, a suction mode, and a clamping mode.
[0049] The arm unit (111) may be mounted on both sides of the main body frame (108). The arm unit (111) may be provided to enable multi-axis rotation. In addition, the arm unit (111) may be provided to enable forward and backward movement. The arm unit (111) may be provided to perform multi-axis rotation and forward or backward movement simultaneously.
[0050] In an exemplary embodiment, the arm unit (111) may be configured to have six degrees of freedom. The arm unit (111) may include a first axis portion (111-1), a second axis portion (111-2), a third axis portion (111-3), a fourth axis portion (111-4), a fifth axis portion (111-5), and a sixth axis portion (111-6).
[0051] The first shaft part (111-1) may be mounted on the main body frame (108). The first shaft part (111-1) may be provided to be rotatable in a first direction (①). The second shaft part (111-2) may be provided to be connected to the first shaft part (111-1). The second shaft part (111-2) may be provided to be rotatable in a second direction (②). The third shaft part (111-3) may be provided to be connected to the second shaft part (111-2). The third shaft part (111-3) may be provided to be able to move forward and backward along the third direction (③).
[0052] The fourth axis part (111-4) may be provided in connection with the third axis part (111-3). The fourth axis part (111-4) may be provided to be rotatable in the fourth direction (④). The fifth axis part (111-5) may be provided in connection with the fourth axis part (111-4). The fifth axis part (111-5) may be provided to be rotatable in the fifth direction (⑤). The sixth axis part (111-6) may be provided in connection with the fifth axis part (111-5). The sixth axis part (111-6) may be provided to be rotatable in the sixth direction (⑥). Here, the first direction (①) to the sixth direction (⑥) may be different directions. In this case, the arm unit (111) can move forward and backward while rotating in the five axial directions.
[0053] The hand unit (113) may be provided connected to the end of the arm unit (111). The hand unit (113) is a part that comes into contact with the loaded cargo and is a part that unloads the loaded cargo toward the conveyor device (104). The hand unit (113) may be provided connected to the sixth axis portion (111-6). Since the hand unit (113) is connected to the arm unit (111), the hand unit (113) can move forward and backward while rotating in five axial directions together with the arm unit (111) according to the operation of the arm unit (111). In this way, since the robot arm device (102) is provided to have six degrees of freedom, it is possible to unload the loaded cargo by covering the entire range within the loading box.
[0054] Figures 6 and 7 are perspective views showing one side and the other side of a hand unit (113) according to one embodiment of the present invention.
[0055] FIG. 6 is a drawing showing one side of a hand unit (113) according to one embodiment of the present invention, and FIG. 7 is a drawing showing the other side of a hand unit (113) according to one embodiment of the present invention.
[0056] Referring to FIGS. 6 and 7, the hand unit (113) may include a conveyor belt (121), an adsorption means (123), and a clamp means (125).
[0057] An unloading conveyor belt (121) may be mounted on one side of the hand unit (113). When the robot arm device (102) is likened to a human arm, one side of the hand unit (113) may be likened to a palm. The unloading conveyor belt (121) may be provided along the longitudinal direction of the arm unit (111). The unloading conveyor belt (121) may be provided to rotate in a certain direction. For example, the unloading conveyor belt (121) may be provided in a loop shape to rotate circularly.
[0058] The unloading conveyor belt (121) may be arranged to rotate toward the main body frame (108) (i.e., inwardly). That is, the unloading conveyor belt (121) of each hand unit (113) in a pair of robot arm devices (102) may be arranged to rotate toward the main body frame (108).
[0059] As the pair of unloading conveyor belts (121) rotate toward the main body frame (108), the loaded cargoes that come into contact with the pair of unloading conveyor belts (121) between the pair of unloading conveyor belts (121) are swept toward the main body frame (108) and unloaded toward the conveyor device (104). That is, the unloading conveyor belt (121) is used in a sweep mode, and can be used when the unloading mode of the robot arm device (102) is a down sweep mode or a side sweep mode.
[0060] A rough pattern portion (121a) may be formed on the surface of the unloading conveyor belt (121) to increase friction with the loaded cargo when sweeping it out. The rough pattern portion (121a) may be provided in a form that protrudes from the surface of the unloading conveyor belt (121), but is not limited thereto. A plurality of rough pattern portions (121a) may be provided at regular intervals along the length direction of the unloading conveyor belt (121).
[0061] A suction means (123) and a clamp means (125) may be provided on the other side of the hand unit (113). When the robot arm device (102) is likened to a human arm, the other side of the hand unit (113) may be likened to the back of the hand. The suction means (123) and the clamp means (125) may be arranged vertically on the other side of the hand unit (113). The suction means (123) and the clamp means (125) may be provided along the longitudinal direction of the hand unit (113).
[0062] In an exemplary embodiment, the suction means (123) and the clamp means (125) may be mounted to the hand unit (113) via a hand bracket (127). Both sides of the hand bracket (127) may be fixed to both sides of the unloading conveyor belt (121). The hand bracket (127) may be provided between the unloading conveyor belt (121) and the suction means (123) and the clamp means (125).
[0063] The suction means (123) may serve to absorb and unload the loaded cargo within the loading compartment. That is, the suction means (123) may be used when the unloading mode of the robot arm device (102) is the suction mode. For example, the suction means (123) may be arranged to move forward and backward along the longitudinal direction of the hand unit (113). The suction means (123) may include a suction pad portion (123a) and a suction drive portion (123b).
[0064] The suction pad section (123a) is a section that absorbs the loaded cargo. A plurality of suction pad sections (123a) may be provided. One end of each of the plurality of suction pad sections (123a) may be fixed to a mounting plate (123a-2). The mounting plate (123a-2) may be connected to the suction drive section (123b). Here, although a plurality of suction pad sections (123a) are illustrated, the present invention is not limited thereto.
[0065] A suction cup (123a-1) for vacuum-absorbing a loaded cargo may be provided at the end of the suction pad portion (123a). The suction cup (123a-1) may be provided in a Java-like shape so as to be bendable in various directions, but its shape is not limited thereto.
[0066] The suction drive unit (123b) may be configured to move the suction pad unit (123a) forward and backward. In addition, the suction drive unit (123b) may include a vacuum generator so that the suction pad unit (123a) absorbs the loaded cargo.
[0067] Figure 8 is a drawing showing a state in which the suction pad portion (123a) moves forward in one embodiment of the present invention.
[0068] FIG. 8a) is a drawing showing a state before the suction pad part (123a) moves forward in one embodiment of the present invention, and FIG. 8b) is a drawing showing a state after the suction pad part (123a) moves forward in one embodiment of the present invention.
[0069] Referring to FIGS. 8a) and 8b), the suction drive unit (123b) can move the suction pad unit (123a) forward in the suction mode to suction the loaded cargo. The suction drive unit (123b) can move the suction pad unit (123a) that has unloaded the suctioned loaded cargo backward to return it to its original position.
[0070] Here, it is described that the suction pad part (123a) moves forward to absorb the loaded cargo, but it is not limited to this, and the hand unit (111) may move toward the loaded cargo while the suction pad part (123a) is fixed, and then the loaded cargo may be absorbed through the suction pad part (123a).
[0071] The clamping means (125) can serve to pick up and unload the loaded cargo within the loading compartment. That is, the clamping means (125) can be used when the unloading mode of the robot arm device (102) is the clamping mode. For example, the clamping means (125) can be arranged to move forward and backward along the longitudinal direction of the hand unit (113). The clamping means (125) can include a clamping unit (125a) and a clamp driving unit (125b).
[0072] The clamping portion (125a) is a portion that picks up the loaded cargo. At this time, the type of loaded cargo may be a sack. That is, the clamping portion (125a) may be provided to pick up the sack among the loaded cargo. For this purpose, the end of the clamping portion (125a) may be provided in the form of a pair of tongs. The clamp driving portion (125b) may be provided to move the clamping portion (125a) forward and backward.
[0073] Figure 9 is a drawing showing a state in which the clamping portion (125a) moves forward in one embodiment of the present invention.
[0074] FIG. 9a) is a drawing showing a state before the clamping part (125a) moves forward in one embodiment of the present invention, and FIG. 9b) is a drawing showing a state after the clamping part (125a) moves forward in one embodiment of the present invention.
[0075] Referring to FIGS. 9a) and 9b), the clamp driving unit (125b) can move the clamping unit (125a) forward in clamping mode to pick up the loaded cargo. The clamp driving unit (125b) can move the clamping unit (125a) that has unloaded the loaded cargo backward to return it to its original position.
[0076] Here, it is described that the clamping part (125a) moves forward to pick up the loaded cargo, but it is not limited to this, and it may be arranged so that the hand unit (111) moves toward the loaded cargo while the clamping part (125a) is fixed and then clamps the loaded cargo through the clamping part (125a).
[0077] FIGS. 10 to 16 are drawings showing the operation of an automated cargo unloading machine according to one embodiment of the present invention unloading loaded cargo from a loading box.
[0078] FIG. 10 is a drawing showing a recognition operation of a cargo unloading automation machine according to an embodiment of the present invention, FIG. 11 is a drawing showing a height adjustment operation of a cargo unloading automation machine according to an embodiment of the present invention, FIG. 12 is a drawing showing an upper operation of a cargo unloading automation machine according to an embodiment of the present invention, FIG. 13 is a drawing showing a down sweep mode operation of a cargo unloading automation machine according to an embodiment of the present invention, FIG. 14 is a drawing showing a side sweep mode operation of a cargo unloading automation machine according to an embodiment of the present invention, FIG. 15 is a drawing showing a clamping mode operation of a cargo unloading automation machine according to an embodiment of the present invention, and FIG. 16 is a drawing showing an adsorption mode operation of a cargo unloading automation machine according to an embodiment of the present invention.
[0079] Referring to FIG. 10, the automated cargo unloading machine (100) can recognize the distance to the cargo, the cargo loading pattern, the cargo loading location, and the cargo type, etc., through a vision sensor or the like. At this time, the automated cargo unloading machine (100) can move toward the cargo depending on the distance from the cargo. The automated cargo unloading machine (100) can sequentially unload the cargo depending on the height of the loaded cargo.
[0080] Referring to FIG. 11, the cargo unloading automation machine (100) can adjust the front end height of the conveyor device (104) according to the loading height of the cargo. The cargo unloading automation machine (100) can adjust the front end height of the conveyor device (104) based on the height of the cargo located at the top among the loaded cargo. That is, since the robot arm device (102) unloads the cargo located at the top among the loaded cargo, the front end height of the conveyor device (104) can be adjusted to a height that can safely receive the cargo located at the top among the loaded cargo.
[0081] Referring to FIG. 12, the cargo unloading automation machine (100) can position the robot arm device (102) above the cargo in order to unload the cargo located at the top among the loaded cargo. In an exemplary embodiment, if the cargo located at the top among the loaded cargo is in the shape of a box and an entry space for the robot arm device (102) is secured above the cargo located at the top, the robot arm device (102) can operate in a down sweep mode to unload the cargo located at the top. At this time, the robot arm device (102) can unload the cargo located at the outermost part among the cargo located at the top among the loaded cargo in the down sweep mode, but is not limited thereto.
[0082] Referring to FIG. 13, the robot arm device (102) is shown in a state in which the uppermost cargo among the loaded cargoes is unloaded toward the conveyor device (104) through the down sweep mode. Here, the down sweep mode may mean an operation mode in which the unloading conveyor belt (121) of the hand unit (113) is positioned to face the target cargo (i.e., the cargo to be unloaded) from above the target cargo (i.e., the surface of the unloading conveyor belt (121) faces the upper surface of the target cargo from above the target cargo) and the unloading conveyor belt (121) rotates toward the main body frame (108) to sweep down the target cargo. At this time, depending on the number of cargoes to be unloaded through the down sweep mode or the location of the cargoes, both of the pair of unloading conveyor belts (121) may operate in the down sweep mode, or only one of them may operate in the down sweep mode.
[0083] Referring to FIG. 14, a state in which the robot arm device (102) unloads loaded cargo through a side sweep mode is illustrated. In an exemplary embodiment, the robot arm device (102) may unload the outermost cargoes among the uppermost cargoes in the loaded cargoes in the down sweep mode, and then unload the cargoes located in the middle in the side sweep mode. However, the present invention is not limited thereto, and if the uppermost cargo among the loaded cargoes is box-shaped and an entry space for the robot arm device (102) is secured on the side of the uppermost cargo, the robot arm device (102) may operate in the side sweep mode to unload the cargo.
[0084] Here, the side sweep mode may mean an operation mode in which one or more target cargoes are positioned between a pair of unloading conveyor belts (121), and the pair of unloading conveyor belts (121) are arranged to face the target cargoes (i.e., cargo to be unloaded) on both sides of the target cargoes (i.e., a state in which the surface of the unloading conveyor belts (121) faces the side of the target cargoes on the side of the target cargoes) and the pair of unloading conveyor belts (121) are rotated toward the main body frame (108) to sweep the target cargoes downwards.
[0085] Specifically, the robot arm device (102) can cause the hand unit (113) to enter the side space of the loaded cargo ((a) of FIG. 14). Next, the robot arm device (102) can arrange a pair of unloading conveyor belts (121) of the hand unit (113) to face the side of the target cargo ((b) of FIG. 14). Next, the robot arm device (102) can rotate the pair of unloading conveyor belts (121) toward the main body frame (108) to unload the target cargo ((c) of FIG. 14). At this time, the cargo unloading automation machine (100) can continue to operate in a side sweep mode while bringing the pair of robot arm devices (102) inward as the loaded cargo is unloaded.
[0086] Referring to FIG. 15, a state in which the robot arm device (102) unloads a loaded cargo through a clamping mode is illustrated. In an exemplary embodiment, the cargo unloading automation machine (100) can operate the robot arm device (102) in the clamping mode depending on the type of cargo to be unloaded. For example, if the type of cargo to be unloaded is cargo that can be held through a clamp means (125), such as a sack or pouch, the cargo unloading automation machine (100) can operate the robot arm device (102) in the clamping mode.
[0087] Specifically, when the type of cargo is a sack ((a) of FIG. 15), the robot arm device (102) can move the hand unit (113) toward the target cargo and then grab the target cargo through the clamping portion (125a) of the clamping means (125) ((b) of FIG. 15). At this time, the clamping portion (125a) can also be moved forward depending on the distance from the target cargo. Next, the robot arm device (102) can pull the target cargo downward to unload the target cargo ((c) of FIG. 15).
[0088] Referring to FIG. 16, a state in which a robot arm device (102) unloads a loaded cargo through suction mode is illustrated. In an exemplary embodiment, the robot arm device (102) may unload cargo through suction mode depending on at least one of the loading height of the cargo and the type of the cargo. For example, when the target cargo is located at the bottom or top and is in the shape of a box, the robot arm device (102) may unload the target cargo through suction mode. FIG. 16 illustrates a state in which cargo is unloaded through suction mode when the target cargo is located at the bottom and is in the shape of a box.
[0089] Specifically, the robot arm device (102) can adsorb the target cargo through the adsorption pad portion (123a) of the adsorption means (123) (Fig. 16(a)). At this time, the adsorption pad portion (123a) can also be moved forward depending on the distance from the target cargo. Next, the robot arm device (102) can move the target cargo to the upper part of the conveyor device (104) while adsorbing the target cargo (Fig. 16(b)). Next, the robot arm device (102) can detach the target cargo from the adsorption pad portion (123a) and unload the target cargo onto the conveyor device (104) (Fig. 16(c)).
[0090] According to the disclosed embodiment, a cargo unloading automation machine (100) is provided with a pair of robot arm devices (102) capable of multi-axis rotation and movement, and the robot arm devices (102) are provided with an unloading conveyor belt (121), an adsorption means (123), and a clamp means (125), so that the machine operates in various unloading modes such as a down sweep mode, a side sweep mode, an adsorption mode, and a clamping mode depending on the position of the loaded cargo, the loading pattern of the cargo, and the type of the cargo, thereby enabling the loaded cargo to be unloaded easily and quickly.
[0091] Figure 17 is a diagram illustrating a process for determining an unloading mode in an automated cargo unloading method according to one embodiment of the present invention. While the illustrated flowchart depicts the method as divided into multiple steps, at least some of the steps may be performed in a different order, combined with other steps and performed together, omitted, divided into substeps, or performed with one or more additional steps not illustrated.
[0092] Referring to FIG. 17, the automated cargo unloading machine (100) can acquire an image (loaded cargo image) of cargo loaded inside a loading box (S 101). For example, the automated cargo unloading machine (100) can acquire an image of loaded cargo using a 3D depth camera, a vision sensor, a Lidar, or a combination thereof.
[0093] Next, the cargo unloading automation machine (100) can recognize each loaded cargo based on the edge information and depth information of each object in the loaded cargo image (S 103).
[0094] Here, since the cargo unloading automation machine (100) photographs the inside of the cargo box from the front of the cargo box, the depth information may refer to the distance between an object included in the loaded cargo image and the cargo unloading automation machine (100). The loaded cargo image may be photographed so as to obtain such depth information. In addition, boundary information of each object in the loaded cargo image can be obtained using an object segmentation model among artificial neural network models based on deep learning. The cargo unloading automation machine (100) may assign an index to each recognized cargo.
[0095] Next, the cargo unloading automation machine (100) can generate loaded cargo-related information including one or more of the location of each recognized cargo, the size of each cargo, the type of each cargo, and the loading pattern of the cargo in the loading box (S 105).
[0096] That is, the cargo unloading automation machine (100) can check the location of each cargo within the loading compartment, the size of each cargo, the type of each cargo, and the pattern in which the cargo within the loading compartment is loaded.
[0097] Inside the loading compartment, various types of cargo are loaded in various patterns (for example, boxes are loaded in an aligned manner, boxes are loaded in an aligned manner at the bottom and boxes are loaded unaligned at the top, sacks are loaded in an aligned manner, sacks are loaded in an aligned manner at the bottom and unaligned at the top, boxes are loaded in the bottom and sacks are loaded in an unaligned manner at the top, etc.). The cargo unloading automation machine (100) can analyze the loaded cargo image to generate information related to the loaded cargo.
[0098] In an exemplary embodiment, the cargo unloading automation machine (100) may use a classification model among artificial neural network models based on deep learning when generating information related to loaded cargo.
[0099] Next, the cargo unloading automation machine (100) can determine the cargo unloading order and cargo unloading mode based on information related to the loaded cargo (S 107).
[0100] That is, the cargo unloading automation machine (100) can determine the order in which to unload the cargo loaded in the loading compartment based on information related to the loaded cargo. In addition, the cargo unloading automation machine (100) can determine which unloading mode to use when unloading each cargo. The specific details of how the cargo unloading automation machine (100) determines the unloading order and unloading mode of the cargo based on the loaded cargo-related information will be described later.
[0101] Meanwhile, the cargo unloading automation machine (100) can recognize the caution label attached to the cargo through the loaded cargo image.
[0102] Figure 18 is a drawing illustrating a state in which a caution label is attached to cargo according to one embodiment of the present invention. As illustrated in Figure 18, the cargo may have a caution label attached, such as "Do not stand up or flip over" (A1) or "Handle with care (Never throw)" (A2).
[0103] Accordingly, the cargo unloading automation machine (100) can check whether a caution label is attached to the cargo in the loaded cargo image. If a caution label is attached to a given cargo, the cargo unloading automation machine (100) can analyze the caution label to check the type of caution (i.e., whether it is a caution label for loading and unloading or handling with care, etc.).
[0104] The automated cargo unloading machine (100) can classify each loaded cargo according to the type of precautions, and assign a cargo caution classification index to the cargo based on the classification. For example, the automated cargo unloading machine (100) can classify cargo without a caution label attached as general cargo, and classify cargo with a caution label attached as cargo requiring loading or unloading or cargo requiring handling with care, depending on the type of precautions.
[0105] When unloading loaded cargo, the cargo unloading automation machine (100) can check the cargo caution classification index of each cargo and unload each cargo in consideration of this. That is, when the cargo caution classification index of a given cargo is a cargo requiring loading or unloading, the cargo unloading automation machine (100) can unload the cargo carefully so that the cargo does not flip over. When the cargo caution classification index of a given cargo is a cargo requiring handling, the cargo unloading automation machine (100) can unload the cargo carefully so that an impact exceeding a certain size is not applied to the cargo.
[0106] Figure 19 is a flowchart illustrating a cargo unloading method using an automated cargo unloading machine according to one embodiment of the present invention. While the illustrated flowchart depicts the method as divided into multiple steps, at least some of the steps may be performed in reverse order, combined with other steps and performed together, omitted, divided into substeps, or performed with one or more additional steps not illustrated.
[0107] Referring to FIG. 19, the cargo unloading automation machine (100) recognizes the cargo located at the top of the cargo loaded in the loading compartment (S 201). Next, the cargo unloading automation machine (100) checks whether the space surrounding the cargo located at the top is empty beyond a preset space (S 203). Here, the preset space may refer to a space that allows the hand unit (113) of the robot arm device (102) to enter.
[0108] As a result of the verification in step S 203, if the space around the cargo located at the top is empty beyond the preset space, the cargo unloading automation machine (100) checks the location of the empty space based on the cargo located at the top (S 205).
[0109] As a result of the verification in step S 205, if the upper part is empty to a preset space based on the cargo located at the top, the cargo unloading automation machine (100) can determine the unloading mode of the cargo located at the top as the down sweep mode (S 207).
[0110] As a result of the verification in step S 205, if both sides are empty to a preset space based on the cargo located at the top, the cargo unloading automation machine (100) can determine the unloading mode of the cargo located at the top as the side sweep mode (S 209).
[0111] Meanwhile, if the space around the cargo located at the top is not empty beyond the preset space as a result of the verification in step S 203, the cargo unloading automation machine (100) can check the type of cargo located at the top (S 211).
[0112] As a result of the verification in step S 211, if the type of cargo located at the top is cargo that can be absorbed (for example, the type of cargo is a box or pouch, etc.), the cargo unloading automation machine (100) can determine the unloading mode of the cargo located at the top as the absorption mode (S 213).
[0113] As a result of the verification in step S 211, if the type of cargo located at the top is cargo that can be picked up by a gripper (for example, the type of cargo is a sack or a pouch, etc.), the cargo unloading automation machine (100) can determine the unloading mode of the cargo located at the top as the clamping mode (S 215).
[0114] FIG. 20 is a diagram showing a state in which a cargo unloading order and a cargo unloading mode are determined based on loaded cargo-related information according to one embodiment of the present invention.
[0115] Referring to Figure 20, among the cargo loaded inside the loading compartment, cargo located on both sides of the uppermost layer is unloaded using suction mode or clamping mode, and then cargo located in between is unloaded using side sweep mode. Then, cargo located on both sides of the layer below is unloaded using down sweep mode, and then cargo located in between is unloaded using side sweep mode. In this manner, cargo is unloaded to the layer located above the lowermost layer, and cargo on the lowermost layer can be unloaded using suction mode or clamping mode.
[0116] FIG. 21 is a block diagram illustrating a computing environment (10) including a computing device suitable for use in exemplary embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.
[0117] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be a cargo unloading automation machine (100). The computing device (12) may be a device for performing a cargo unloading automation method.
[0118] A computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) may cause the computing device (12) to operate according to the exemplary embodiments mentioned above. For example, the processor (14) may execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (14), may be configured to cause the computing device (12) to perform operations according to the exemplary embodiments.
[0119] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by the processor (14). In one embodiment, the computer-readable storage medium (16) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that can be accessed by the computing device (12) and store desired information, or a suitable combination thereof.
[0120] A communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and computer-readable storage media (16).
[0121] The computing device (12) may also include one or more input / output interfaces (22) that provide interfaces for one or more input / output devices (24) and one or more network communication interfaces (26). The input / output interfaces (22) and the network communication interfaces (26) are connected to the communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) via the input / output interfaces (22). Exemplary input / output devices (24) may include input devices such as pointing devices (such as a mouse or a trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and / or photographing devices, and / or output devices such as display devices, printers, speakers and / or network cards. The exemplary input / output devices (24) may be included within the computing device (12) as a component constituting the computing device (12), or may be connected to the computing device (12) as a separate device distinct from the computing device (12).
[0122] The detailed description above is illustrative of the present invention. Furthermore, the foregoing description illustrates preferred embodiments of the present invention, and the present invention can be used in various other combinations, modifications, and environments. In other words, changes or modifications may be made within the scope of the inventive concepts disclosed herein, the scope equivalent to the written disclosure, and / or the scope of technology or knowledge in the art. The written embodiments illustrate the best possible state for implementing the technical idea of the present invention, and various modifications required for specific applications and uses of the present invention are also possible. Therefore, the detailed description of the invention above is not intended to limit the present invention to the disclosed embodiments. Furthermore, the appended claims should be construed to include other embodiments.
Claims
1. One or more processors, and A method performed in a cargo unloading automation machine including a memory storing one or more programs executed by one or more processors, A step of acquiring a loaded cargo video by photographing the inside of a loading box loaded with cargo; A step of recognizing each loaded cargo based on boundary information and depth information of each object in the above loaded cargo image; A step of generating loaded cargo-related information including at least one of the location, size, type, and loading pattern of each recognized cargo within the loading box; and A method for automating cargo unloading, comprising a step of determining at least one of an unloading order and an unloading mode of cargo in the loading box based on the information related to the loaded cargo.
2. In claim 1, The above cargo unloading automation method is, A step of checking whether a caution label is attached to the cargo through the above-mentioned loaded cargo video; and A method for automatically unloading cargo, further comprising a step of analyzing the above caution label to determine the type of caution, if the above caution label is attached.
3. In claim 2, The above cargo unloading automation method is, A step of assigning a cargo caution classification index to the cargo according to the above-mentioned type of caution; and A cargo unloading automation method further comprising a step of unloading the cargo from the loading box based on the above cargo attention classification index.
4. In claim 1, The above cargo unloading automation machine is equipped with a pair of robotic arm devices, each including an arm unit that is provided to enable multi-axis rotation and forward and backward movement, and a hand unit that is connected to the arm unit and contacts the cargo to unload the cargo. A method for automatically unloading cargo, wherein each of the above hand units includes an unloading conveyor belt configured to sweep cargo inside the loading box.
5. In claim 4, The step of determining at least one of the above cargo unloading order and cargo unloading mode is: A step of checking whether the space around the cargo located at the top of the above loading box is empty beyond a preset space; and A cargo unloading automation method, comprising a step of determining the unloading mode of the cargo located at the top as a down sweep mode when the upper portion is empty beyond a preset space based on the cargo located at the top.
6. In claim 5, The above down sweep mode is a cargo unloading automation method in which the cargo is swept down by rotating the cargo unloading conveyor belt toward the cargo unloading automation machine while the cargo is placed so as to face the cargo from above the cargo.
7. In claim 4, The step of determining at least one of the above cargo unloading order and cargo unloading mode is: A step of checking whether the space around the cargo located at the top of the above loading box is empty beyond a preset space; and A method for automatically unloading cargo, comprising a step of determining the unloading mode of the cargo located at the uppermost portion as a side sweep mode when both sides are empty beyond a preset space based on the cargo located at the uppermost portion.
8. In claim 7, The above-mentioned side sweep mode is a cargo unloading automation method in which the loaded cargo is positioned between a pair of the unloading conveyor belts, and the pair of the unloading conveyor belts are arranged to face the cargo on both sides of the cargo, and the pair of the unloading conveyor belts are rotated toward the cargo unloading automation machine to sweep the cargo downward.
9. In claim 4, Each of the above hand units further includes a suction means for absorbing cargo inside the loading compartment, The step of determining at least one of the above cargo unloading order and cargo unloading mode is: A step of checking whether the space around the cargo located at the top of the above loading box is empty beyond a preset space; and A cargo unloading automation method, comprising a step of determining the cargo unloading mode as an absorption mode according to the type of cargo located at the uppermost part, when the surrounding space is not empty beyond a preset space.
10. In claim 4, Each of the above hand units further includes a clamp means for picking up cargo inside the loading compartment, The step of determining at least one of the above cargo unloading order and cargo unloading mode is: A step of checking whether the space around the cargo located at the top of the above loading box is empty beyond a preset space; and A method for automatically unloading cargo, comprising a step of determining the unloading mode of the cargo as a clamping mode according to the type of cargo located at the uppermost part, when the surrounding space is not empty beyond a preset space.
11. One or more processors; memory; and Contains one or more programs, The one or more programs are stored in the memory and configured to be executed by the one or more processors, One or more of the above programs, A command to obtain a video of the loaded cargo by photographing the inside of the cargo box where the cargo is loaded; A command to recognize each loaded cargo based on boundary information and depth information of each object in the above loaded cargo image; A command for generating loaded cargo-related information including at least one of the location, size, type, and loading pattern of each recognized cargo within the cargo compartment; and A computing device comprising a command for determining at least one of an unloading order of cargo in the loading box and an unloading mode of the cargo based on the information related to the loaded cargo.
Citation Information
Patent Citations
Pallet / corrugated board charging equipment
JP2001122437A
Transfer apparatus, and cargo extracting method
JP2016094280A
Method and system for pallet packing, and method for providing pallet packing information
KR1020140000644A
Goods loading system using 3D camera and bigdata system, and thereof method
KR1020170101555A
Cable hanger
KR1020240173522A
