Automatic cargo handling system

Abstract

According to the automatic goods loading and unloading system, goods (3) are stacked and loaded or unstacked and unloaded through a mechanical arm (22), a detection robot (1) comprises a support (11), and a first depth camera (12) achieved through a structured light depth measuring device and a second depth camera (13) achieved through a dToF depth measuring device are arranged on the support. The loading and unloading robot (2) comprises a conveying belt (21) and a mechanical arm (22). The loading and unloading robot serving as the hand and the detection robot serving as the eye can respectively have certain mobility so as to improve the working coverage range of the system, and then the safety eye capable of moving along with the hand is additionally provided for the detection robot, alarming and braking are carried out in time, and safety protection of the whole working area is achieved.

Description

Technical Field

[0001] This disclosure relates to the field of logistics, and more particularly to an automated cargo loading and unloading system. Background Technology

[0002] Palletizing and depalletizing are common logistics operations in industrial automation, widely used in warehousing, manufacturing, food, pharmaceutical and other fields, and rely on automated cargo loading and unloading systems, including robotic arms and vision systems, to achieve efficient operations.

[0003] In industrial automation scenarios, employees accidentally entering the robotic arm's operating area can lead to serious safety accidents. While the probability of such accidents can be reduced through management regulations and personnel training, it is still desirable to fundamentally prevent them from occurring through the system's own technical safeguards.

[0004] Therefore, an improved, automated, and efficient cargo loading and unloading solution is needed. Utility Model Content

[0005] One technical problem this disclosure aims to solve is to provide an automated cargo loading and unloading system. This system, by introducing cooperative detection robots and loading / unloading robots to improve the detection range of machine vision and the operability range of robotic arms, equips the detection robots with wide-angle dToF depth cameras to detect the presence of people in the work area and to issue timely alarms and brakes, thereby fundamentally eliminating the possibility of safety accidents.

[0006] According to one aspect of this disclosure, an automated cargo loading and unloading system is proposed for loading or unloading cargo by a robotic arm, characterized in that the detection robot includes a support frame, on which a first depth data measuring device and a second depth data measuring device are disposed; the loading and unloading robot includes a conveyor belt and a robotic arm, wherein the first depth data measuring device is a structured light depth data measuring device centrally disposed on the support frame, and the second depth data measuring device is a dToF depth data measuring device disposed on the side of the support frame.

[0007] Optionally, the automated cargo loading and unloading system further includes an alarm and braking device connected to the robotic arm, for issuing an alarm or / or braking the system in operation in response to the second depth data measuring device detecting the presence of a person in the field of view.

[0008] Optionally, the dToF depth data measuring device provided beside the support includes one dToF depth data measuring device provided on the side of the support near the robotic arm, or alternatively, the dToF depth data measuring device provided beside the support includes one dToF depth data measuring device provided on each side of the support.

[0009] Optionally, the first depth data measurement device includes: a structured light projection device for projecting structured light onto a first shooting area below the support; and first and second image sensors having a predetermined relative spatial relationship for imaging the first shooting area onto which the structured light is projected to obtain first and second two-dimensional images.

[0010] Optionally, the first depth data measurement device includes a visible light image sensor, used to image the first shooting area to obtain a visible light image when the structured light projection device does not project the structured light.

[0011] Optionally, the second depth data measurement device includes: a light source module for projecting invisible light onto a second imaging area; and a dToF sensor for generating a sensing signal characterizing distance based on the reception time of the reflected light from the projected invisible light.

[0012] Optionally, the dToF sensor includes: an area array sensor composed of multiple avalanche photoelectric sensors, and the light source module includes: a light-emitting device for generating a laser beam; and a diffuser arranged in the propagation path of the beam to convert the laser beam into a wide-angle area array light.

[0013] Optionally, the first shooting area and the second shooting area do not overlap, and / or the invisible structured light projected by the first depth data measuring device and the invisible light projected by the second depth data measuring device have different wavelengths.

[0014] Optionally, the inspection robot includes two slide rails connected to the support, and the support is a gantry frame disposed between the two slide rails.

[0015] Optionally, the movable mechanisms include a parking area for transport vehicles, and the movable mechanisms are arranged along the length of the transport vehicle.

[0016] Optionally, the robotic arm includes a picking device for picking up goods and is equipped with a calibration disk.

[0017] Optionally, the robotic arm is mounted on one end of the conveyor belt near the support.

[0018] Optionally, the end of the conveyor belt is arranged on a slide rail on which the support can move, and the support serves as a blocking device for the flexible conveyor belt.

[0019] Optionally, one end of the flexible conveyor belt is connected to an automated shipping and / or receiving system.

[0020] This disclosure significantly improves the system's coverage by providing mobility to both the loading / unloading robot (acting as a "hand") and the inspection robot (acting as an "eye"). Furthermore, by providing the inspection robot with a "safety eye" that can move with the "hand" and provide timely alarms and braking, it achieves safety protection across the entire work area. Attached Figure Description

[0021] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments thereof taken in conjunction with the accompanying drawings, wherein like reference numerals generally denote like parts.

[0022] Figure 1 A schematic diagram of the composition of an automated cargo loading and unloading system according to an embodiment of the present invention is shown.

[0023] Figure 2 A top view schematic diagram of an automated cargo loading and unloading system according to an embodiment of the present invention operating at different positions is shown.

[0024] Figure 3 An example is shown where the automated cargo handling system issues an alarm and applies an emergency brake when someone intrudes.

[0025] Figure 4 A schematic diagram of the structured light depth data measurement device used in an automated cargo loading and unloading system is shown.

[0026] Figure 5 A schematic diagram of the dToF depth data measurement device used in an automated cargo loading and unloading system is shown. Detailed Implementation

[0027] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0028] Palletizing and depalletizing are common logistics operations in industrial automation, widely used in warehousing, manufacturing, food, pharmaceutical and other fields, and rely on automated cargo loading and unloading systems, including robotic arms and vision systems, to achieve efficient operations.

[0029] In industrial automation scenarios, employees accidentally entering the robotic arm's operating area can lead to serious safety accidents. While the probability of such accidents can be reduced through management regulations and personnel training, it is still desirable to fundamentally prevent them from occurring through the system's own technical safeguards.

[0030] To this end, this disclosure proposes an automated cargo loading and unloading system. This system introduces cooperative detection robots and loading / unloading robots to improve the detection range of machine vision and the operability range of robotic arms. The detection robots are equipped with a wide-angle dToF depth measurement device to detect whether there are people in the work area and to issue alarms and brake the system in a timely manner, thereby fundamentally eliminating the possibility of safety accidents.

[0031] Figure 1 A schematic diagram of the composition of an automated cargo loading and unloading system according to an embodiment of the present invention is shown.

[0032] As shown in the figure, the automated cargo loading and unloading system includes a detection robot 1 and a loading / unloading robot 2. Here, the detection robot 1 is implemented as a first depth camera 12 (also called a first depth data measuring device 12) mounted on the gantry 11, serving as the system's "eye" to determine the position of objects within its field of view. The loading / unloading robot 2 is implemented as a robotic arm 22 that continuously picks up goods from the conveyor belt 21 for loading (or, in the unloading process, a robotic arm 22 that continuously picks up goods from the cargo box to provide to the flexible conveyor belt 21), serving as the system's "hand," used to place goods in the appropriate position in the vehicle's cargo box, or pick up goods from the appropriate position obtained from the vehicle, under the guidance of the "eye."

[0033] In addition to the first depth camera 12, the detection robot 1 also includes a second depth camera 13 (also referred to as a second depth data measurement device 13) mounted on a support 11 (illustrated as a gantry 11, but in other embodiments it may have other forms besides a gantry). The second depth camera 13 is mounted close to the robotic arm and has a wide field of view, used for safety monitoring of the robotic arm's operating range. The second depth camera 13 is connected to an alarm and braking device, which can issue an alarm or even brake the system (especially the robotic arm 22) when the second depth camera 13 detects the presence of a person in its field of view, thereby avoiding safety accidents caused by personnel accidentally entering the system's working area.

[0034] In one embodiment, the detection robot 1 may be equipped with only one second depth camera 13 for safety monitoring within the robotic arm's operating range. In another embodiment, such as Figure 1 As shown, the inspection robot 1 may be equipped with only two second depth cameras. In addition to the second depth camera 13 for safety monitoring within the operating range of the robotic arm, it may also include a second depth camera 13' for safety monitoring of the working area on the other side of the gantry 11. In other embodiments, more second depth cameras may be installed in more locations. In this disclosure, the second depth camera may be implemented as a dToF (direct time-of-flight) depth camera as described below.

[0035] For ease of explanation, the following description will focus on the loading process. In this case, the automatic cargo loading and unloading system of this utility model can be implemented as an automatic cargo loading system, and the loading / unloading robot 2 can be implemented as a loading robot, used to transfer goods from the warehouse and / or the automatic dispatch system to the vehicle compartment. That is, the goods are stacked and loaded onto the transport vehicle 4 by the robotic arm of the loading / unloading robot 2. However, those skilled in the art should understand that in other embodiments, this system with the same configuration can also be implemented as an unloading system, and the loading / unloading robot 2 can be implemented as a unloading robot, used to transfer goods from the compartment to the warehouse and / or the automatic receiving system. That is, the goods 3 are unloaded from the transport vehicle by the robotic arm 22 and sent to the inlet via the reverse transmission of the conveyor belt 21.

[0036] exist Figure 1 In this embodiment, the inspection robot 1 is implemented as a mobile inspection robot, with mobility provided by a gantry 11 capable of moving on slide rails (e.g., the two slide rails 14 and 15 shown in the figure), and inspection capability provided by a first depth data measuring device 12 (hereinafter also referred to as a first depth camera 12) mounted on the gantry 11. Safety protection capability (requiring the participation of alarm and braking devices) is provided by a second depth data measuring device 13 (hereinafter also referred to as a second depth camera 13), also mounted on the gantry 11. In this case, the inspection robot 1 is mobile, acting as a limited "eye" with limited mobility, its movement form and range defined by the slide rails and the gantry. The two slide rails 14 and 15 and the gantry 11 define the work area for loading and unloading. Figure 1 In this example, the work area is the parking area for vehicle 4. In other embodiments, other cargo facilities besides vehicles may also be placed in the work area.

[0037] The loading robot 2 can be implemented as a robotic arm 22 positioned on one side near the conveyor belt 21. The conveyor belt 21 can be implemented as a flexible conveyor belt, as shown, thereby providing a degree of mobility, and the robotic arm 22 provides for picking up and releasing goods within a specific range. In other words, the loading robot 2 is a "hand" whose form and range of movement are defined by the flexible conveyor belt, and within the range defined by the flexible conveyor belt, the robotic arm 22 performs the picking and releasing.

[0038] Conveyor belt systems are a crucial component of manufacturing, used to transport materials along a predetermined path to a designated location. Currently, many factories' flexible manufacturing systems utilize computers and robotic arms as auxiliary equipment for material handling and storage, while using conveyor belts as the primary transport equipment. Therefore, system efficiency largely depends on the flexibility of the conveyor belt system. In conventional conveyor belt systems, the connections between each conveyor belt are fixed, meaning their relative positions cannot be changed. In contrast, "flexible" conveyor belts can refer to those that provide a degree of mobility at one or both ends (loading and unloading ends).

[0039] To accommodate the mobility of the inspection robot 1, this disclosure uses a flexible conveyor belt 21 to provide the loading robot 2 with mobility compatible with that of the inspection robot 1. Specifically, it provides mobility at least to the side where the robotic arm 22 is located, allowing the robotic arm 22 to remain relatively fixed to the depth camera 12 (this relative fixation has some redundancy, therefore a calibration disk is required for calibration). Specifically, although not explicitly stated in... Figure 1 The text shows ( ) Figure 1 The flexible conveyor belt 21 shown can be considered as part of a complete flexible conveyor belt (i.e., the flexible conveyor belt section near the robotic arm 22), but one end of the flexible conveyor belt can be connected to a shipping system. This shipping system can be manual or automated, used to supply goods stored in a warehouse or factory to the flexible conveyor belt, which then transports the goods to the other end of the flexible conveyor belt, i.e., the end near the robotic arm 22, via the movement of the conveyor belt. Due to the limited field of view of the depth camera 12 and the limited range of motion of the robotic arm 22, when facing situations such as... Figure 1 When the transport vehicle 4 with a long carriage is shown, in order to fill the carriage with goods, the gantry crane needs to stop at three positions shown in the figure (A, B, and C) (i.e., positions A', B', and C' on slide rail 15, which can correspond to the obstructed positions A, B, and C on slide rail 14). Accordingly, the bottom of the flexible conveyor belt 21 is equipped with rollers, which can extend and retract within a certain degree of flexibility, so that the robotic arm 22 can be positioned near positions A, B, and C respectively, thereby realizing the stacking of goods at the front, middle, and rear of the carriage (see...). Figure 2 ).

[0040] To stack goods at different locations within the carriage, the ends of the gantry 11 and the flexible conveyor belt 21 equipped with robotic arms 22 can be controlled to move to corresponding positions (e.g., when the gantry 11 moves to position A, the end of the flexible conveyor belt 21 equipped with robotic arms 22 moves closer to position A; when the gantry 11 moves to position B, the end of the flexible conveyor belt 21 equipped with robotic arms 22 moves closer to position B, etc.). However, since the inspection robot 1 and the loading robot 2 are not fixed to each other and reach their designated positions through independent movement, the relative positions between them will change due to the introduction of motor-driven mechanical relative motion. In other words, the coordinated movement of the gantry 11 and the flexible conveyor belt 21 under mechanical control will still introduce deviations, such as a lateral deviation of a few centimeters. These deviations are sometimes unacceptable for the reliable picking and releasing of the robotic arm 22, therefore a mechanism to eliminate these deviations is needed.

[0041] Therefore, upon reaching each new position (e.g., any position in ABC), hand-eye calibration can be re-performed so that the depth data detected by camera 12 and the motion data of robotic arm 22 are again coordinated in the same xyz coordinate system. To this end, this disclosure directly incorporates a calibration disk on robotic arm 22, particularly on the picking device of robotic arm 22, such as... Figure 1 The object 23 shown has a chessboard pattern, thereby enabling rapid calibration between the detection robot 1 and the loading robot 2.

[0042] To illustrate the mobility of the automated cargo loading and unloading system of this disclosure, Figure 2 A top view schematic diagram of an automated cargo loading and unloading system according to an embodiment of the present invention operating at different positions is shown. Although in Figure 1 The diagram shows a gantry 11 sliding on two slide rails 14 and 15, but it should be understood that the mobility of the detection robot 1 can also be achieved in other ways. For example, it can be implemented as a single slide rail 14 and a support 11 capable of sliding on the slide rail 14 (e.g., Figure 2 As shown), it can even be implemented as a base that can move freely (e.g., a caster base that can move in all directions and does not require a slide rail to limit the range of motion) and a bracket fixed on it.

[0043] In the case where the movable mechanism of the detection robot is used to define the movement path of the depth camera 12, for example in Figure 1 Double-rail gantry and Figure 2In an embodiment of the single-rail support, the movable mechanism (implemented as the rail in the figure) can be arranged along the length of the transport vehicle 4, and the length of the movable mechanism itself is longer than the field of view coverage of the first depth camera 12 in that length direction. In other words, the lengths of rails 14 and 15 should be longer than the field of view length indicated by the dashed box in the figure, for example, they can match the length of the longest model of truck that the system needs to load and unload. Additionally, although not shown in the figure, blocking devices can be installed on both sides of each rail to prevent the gantry or support 11 from sliding out.

[0044] The following will combine Figure 1 and Figure 2 The composition and operation of the automated cargo loading and unloading system are described.

[0045] First, transport vehicle 4 enters the automated cargo loading and unloading system and parks in the appropriate position. That is, the direction of travel of transport vehicle 4 is aligned with... Figure 1 The slide rails are aligned and located within the volume covered by the gantry frame 11 and slide rails 14 and 15.

[0046] Because the carriage 42 of vehicle 4 is relatively long, the field of view of the first depth camera 12 (such as...) Figure 1 As shown by the dotted line in the image, it cannot cover the entire carriage area at once, and the operating range of the robotic arm 22 is also limited. Therefore, the support 11 where the first depth camera 12 is located first moves to position A (…). Figure 1 and Figure 2 This can be considered as position A in different embodiments. Position A can be located at the front of carriage 42, thereby allowing the depth measurement field of view of camera 12 to be as follows: Figure 1 and Figure 2 As shown by the dotted line on the left, this covers the front of the cargo compartment of the large truck 4. At this time, the flexible conveyor belt 21 also moves forward, causing the robotic arm 22 located on one side of the conveyor belt 21 to also come to the vicinity of the front of the cargo compartment. Since the discharge port of the flexible conveyor belt 21 is furthest from the loading port indicated by the arrow at this time, the flexible conveyor belt 21 is stretched to a relatively straight state.

[0047] After the support 11 and the flexible conveyor belt 21 move to the predetermined position, the first depth camera 12 can be quickly "hand-eye calibrated" based on the calibration disk 23 on the robotic arm 22, thereby unifying the first depth camera 12 and the robotic arm base in the same coordinate system. In the rapid calibration, since the relative position of the first depth camera 12 and the calibration disk 23 can be obtained through the calibration operation itself, and the relative position of the picking end of the robotic arm 22 and the base can be accurately known through the multi-axis motion of the robotic arm 22, the relative position information between the calibration disk 23 and the picking end can be eliminated by solving equations simultaneously. Thus, the relative position of the first depth camera 12 and the robotic arm base can be directly obtained based on the calibration result of the first depth camera 12 relative to the calibration disk 23 and the position information of the picking end relative to the base. For example, the deviation compared to the default relative position can be obtained and the deviation can be compensated, thereby realizing the rapid calibration between the first depth camera 12 and the robotic arm 22 base. In this way, under the guidance of the first depth camera 12, the robotic arm 22 can pick up the goods 3 continuously conveyed on the flexible conveyor belt 21 and place them at a predetermined position at the front of the carriage 42.

[0048] When Figure 2 As shown in the gray area on the left, when the first depth camera 12 determines that the front of the carriage 42 is full of goods, it can control the support 11 to move to a new position, such as... Figure 2 Position B, shown in the middle, can be located in the middle of carriage 42, thereby allowing the depth measurement field of view of the first depth camera 12 to be as follows: Figure 2 The dotted line in the middle indicates the central area of ​​the cargo compartment 42 of the large truck 4. At this time, the flexible conveyor belt 21 also moves forward, causing the robotic arm 22 located on one side of the conveyor belt 21 to also come to the vicinity of the middle of the cargo compartment. Since the distance between the discharge port of the flexible conveyor belt 21 and the loading port indicated by the arrow becomes closer at this time, the bending degree of the flexible conveyor belt 21 increases.

[0049] When Figure 2 As shown in the gray area in the middle, when the depth camera 12 determines that the middle of the carriage 42 is full of goods, it can control the support 11 to move to a new position, such as... Figure 2 Position C is shown on the right. Position C can be located at the rear of carriage 42, thereby allowing the depth measurement field of view of the first depth camera 12 to be as follows: Figure 2 As shown by the dotted line, this covers the rear area of ​​the cargo compartment 42 of the large truck 4. At this time, the flexible conveyor belt 21 also moves forward, causing the robotic arm 22 located on one side of the conveyor belt 21 to also come to the vicinity of the rear of the cargo compartment. Since the distance between the discharge port of the flexible conveyor belt 21 and the loading port indicated by the arrow becomes closer at this time, the degree of bending of the flexible conveyor belt 21 is the greatest.

[0050] When Figure 2As shown in the gray area on the right, when the first depth camera 12 determines that the rear of the cargo compartment 42 is full, the system can determine that the current vehicle is fully loaded. At this time, vehicle 4 can leave to allow subsequent vehicles to be loaded.

[0051] It should be noted that when the feed port position is fixed, the flexible conveyor belt 21 still maintains the same conveyor belt length, and can achieve different discharge port positions by bending itself, and can convey the same number of goods 3 at the same time.

[0052] The end of the flexible conveyor belt 21 equipped with the robotic arm 22 can be positioned on the slide rail 14. However, positioning it on the slide rail 14 does not mean that the flexible conveyor belt 21 needs to slide on the slide rail 14. Figure 1 As shown, the bottom of the flexible conveyor belt 21 can be equipped with wheels to provide mobility, and its lateral movement can be restricted by the slide rail 14 by being physically located on top of it. That is, since the slide rail 14 is located between the two wheels on the bottom of the flexible conveyor belt 21, it can control the lateral movement of the flexible conveyor belt 21. Furthermore, the bracket 11 mounted on the slide rail 14 can be used as a blocking device for the flexible conveyor belt 21, for example, causing the flexible conveyor belt 21 to be linked with the bracket 11 to reach the vicinity of the front, middle and rear of the carriage.

[0053] As mentioned earlier, in addition to the first depth camera 12 installed in the middle of the support 11 for visual guidance of the robotic arm's palletizing and depalletizing, the inspection robot is also equipped with a second depth camera 13 (and in some cases, a second depth camera 13') for safety assurance. Figure 2 In the diagram, the field of view of the second depth camera 13 is shown within a red dashed box. Compared to the first depth camera 12, which is used to guide the precise operations of the robotic arm, the second depth camera 13 typically requires a larger field of view and, since it only needs to detect personnel approaching or entering the robotic arm's working area, a low-precision but high-speed response is required. In this disclosure, a wide-angle dToF depth camera is used to achieve the required functionality of the second depth camera 13. Because the second depth camera 13 is also mounted on the gantry, it can also provide full-area safety coverage as the gantry position changes.

[0054] Figure 3An example of an automated cargo handling system issuing alarms and emergency braking upon personnel intrusion is shown. As shown on the left, a person (shown only as head and shoulders) enters the field of view of the second depth camera 13. Since the person is still some distance from the robotic arm's operating area, when the presence of a person is detected by the imaging of the second depth camera 13, the alarm and braking device can simply issue an alarm to warn the person to leave the operating system. The alarm can be a voice alarm, such as "Work area, leave immediately," a visual alarm, such as a flashing red light, or a combination of various alarm methods.

[0055] And if that person is like Figure 3 As shown on the right, further approaching the robotic arm's operating area, due to the greater risk of safety accidents, the second depth camera 13 identifies that someone is approaching or in the robotic arm's operating area. The alarm and braking device can then brake the operating system, for example, causing the robotic arm to stop in an emergency.

[0056] In this invention, a depth data measurement device capable of actively projecting structured light is used as the first depth camera 12 to acquire depth information of the field of view area. Figure 4 A front view of a structured light depth data measurement device for an automated cargo loading and unloading system according to an embodiment of the present invention is shown. Figure 1 and Figure 2 In the example above, the light inlet / outlet window shown in the front view can be positioned downwards or diagonally downwards to capture images of the target area of ​​the carriage (e.g., the area indicated by the dashed box).

[0057] As shown in the figure, the first depth camera 12 includes a structured light projection device 121 for projecting structured light into a working area (e.g., inside a vehicle). To distinguish it from ambient light, infrared or ultraviolet structured light is preferably projected. Here, the structured light can be speckle structured light or stripe structured light. The first depth camera 12 can use an image sensor to image the projected structured light and calculate the depth information of the shooting area from the captured two-dimensional image. Figure 4 In the example, the first depth camera 12 adopts a binocular imaging structure, that is, it is equipped with two image sensors, a first image sensor 122 and a second image sensor 123. The two image sensors have a predetermined relative spatial position relationship, and each of them images the target area on which the structured light is projected to obtain first and second two-dimensional images. Subsequently, the three-dimensional distribution depth information of the goods within the field of view can be obtained by pixel window matching and triangulation calculation, that is, the height z value of each point on the xy distribution plane of the structured light in the measured space (target area) is calculated, and the three-dimensional information of the current goods in the target area is obtained thereby.

[0058] In some embodiments, the first depth camera 12 may further include a visible light image sensor 124, preferably an RGB sensor. The RGB sensor 124 can capture images at times other than when the structured light is projected to obtain visible light images of the target area, which can be combined with previously acquired depth information to obtain more accurate cargo volume and location information.

[0059] Although Figure 4 The diagram illustrates a depth data measurement device (3D camera) based on binocular imaging with visible light imaging capability. However, it should be understood that in other embodiments, depth imaging can also be achieved based on a comparison of a monocular image with a reference image, and the visible light imaging capability is also optional.

[0060] After the first depth camera 12 captures the images required for calculating depth / 3D information, a computing device can be used to determine the 3D distribution depth information of goods within the target area based on the first and second 2D images (and optionally, visible light images). The computing device can be located within the first depth camera 12, in which case the first depth camera 12, in addition to its measurement function, also has a depth data generation function. In other embodiments, the computing device can be located outside the first depth camera 12, for example, within a separately configured control device of the automated cargo handling system. Regardless of the configuration, the first depth camera 12 needs to communicate with the outside (e.g., wired or wireless communication) to send out the captured image information and / or depth calculation results for controlling the operation of the robotic arm 22.

[0061] In some embodiments, for the operation of the robotic arm 22, the field of view of the first depth camera 12 can cover not only the area of ​​the carriage that needs to be stacked, but also at least a portion of the area of ​​the flexible conveyor belt 21 close to the robotic arm 22, thereby enabling the measurement of the three-dimensional information of the goods to be picked up by the robotic arm 22 from the conveyor belt 21, thereby providing visual guidance for the entire process of picking up, moving and releasing goods by the robotic arm 22.

[0062] In other embodiments, the field of view of the first depth camera 12 may only cover the area of ​​the carriage that needs to be palletized. In other words, the first depth camera 12 only provides visual guidance on where the robotic arm 22 should place the currently picked-up goods, without participating in guiding the robotic arm 22 to pick up the goods 3 from the conveyor belt 21. In this case, the robotic arm 22 can rely on its own vision system (for example, a lightweight depth camera can also be installed on the robotic arm 22) or on the reasonable design of the conveying and picking system to achieve accurate picking of the goods 3. When the robotic arm 22 is equipped with its own depth camera, the robotic arm 22 can determine the height and position of the goods and pick them up based on its own visual capabilities. However, when relying on the reasonable design of the conveying and picking system to achieve accurate picking, the robotic arm 22 does not actually "see" the exact position of the goods on the conveyor belt, but simply picks up from a fixed position each time (i.e., the robotic arm 22 defaults to being able to pick up the goods to be grabbed from the same position). This situation is particularly suitable for application scenarios where all the goods to be loaded on the vehicle have the same outer packaging. When the shape of the loaded goods changes, it is usually necessary to adapt by reprogramming the picking position of the robotic arm 22 or by replacing the picking head.

[0063] To this end, the system may include a control device for determining the coordinates of the robotic arm 22 when releasing goods within the vehicle, based on the current depth information inside the vehicle, such as the xyz values ​​of the robotic arm 22's manipulator when releasing the goods. If, for example, there is a deviation between the goods picking orientation and the release orientation (e.g., the vehicle is parked crookedly), the robotic arm 22's manipulator can also rotate at a certain angle after picking up the goods and before releasing them, so that the goods 3 are placed in the vehicle with the desired orientation. Furthermore, if the first depth camera 12 provides visual guidance for the robotic arm 22's goods picking, the system may also include a second control device for determining the coordinates of the robotic arm 22 when picking up the goods, based on the three-dimensional depth information distributed on the conveyor belt 21, such as aligning the picking suction cup with the center of the goods' upper surface.

[0064] In one embodiment, the aforementioned control device and the second control device can be a single control device combined together, for example, implemented by a separate control device in the system other than the first depth camera 12 and the robotic arm 22, along with the aforementioned computing device. In this case, the first depth camera 12 and the robotic arm 22 need to communicate with the control device separately. In another embodiment, the aforementioned control device and the second control device can be located inside the robotic arm 22. When the aforementioned computing device is located inside the measuring device, the picking and releasing control of the robotic arm 22 based on the measurement results of the measuring device can be achieved through direct communication between the first depth camera 12 and the robotic arm 22.

[0065] Regardless of the implementation of the computing and control devices, in this invention, the depth information obtained by the depth data measurement device (i.e., the first depth camera 12) is used to provide the robotic arm 22 with at least one coordinate (the coordinate for releasing the cargo, or preferably, two coordinates for picking up and releasing), thereby enabling the robotic arm 22 to place the cargo 3 at the desired position inside the carriage 42.

[0066] In a preferred embodiment, the first depth camera 12 can also determine the stacking pattern of the cargo 3 in the carriage 42 based on the acquired size information of the carriage 42 and the size information of the cargo 3.

[0067] In this invention, a dToF depth data measurement device is used as a second depth camera 12 to acquire depth information of the second shooting area. Here, the second shooting area may correspond to the field of view of the second depth camera 12, and typically coincides with a security area where intrusion needs to be prevented. Figure 5 A schematic diagram of the dToF depth data measurement device used in an automated cargo handling system is shown. As shown, the second depth camera may include: a light source module 131 for projecting invisible light into a second imaging area; and a dToF sensor 132 for generating a sensing signal characterizing distance based on the reception time of the returned light from the projected invisible light. Here, the light source module 131 may be used to project an area array of light into the second imaging area and may include a light-emitting device for generating a laser beam; and a diffuser arranged in the propagation path of the beam to convert the laser beam into a wide-angle area array of light. Accordingly, the dToF sensor 132 may also be an area array sensor, such as an area array sensor composed of multiple avalanche photoelectric sensors. Similar to the first depth camera 12, the second depth camera 13 may also include a computing device for generating depth information based on the sensing signal and thereby identifying personnel, or it may send the acquired sensing signal to an external computing device for calculation. In either case, when the sensing signal indicates that someone has entered, an alarm and braking device can be triggered with the appropriate alarm or braking operation. In one embodiment, head and shoulder recognition can be used to identify an intruder, while in another embodiment, movement can be detected to identify an intruder or the intrusion of other objects.

[0068] Furthermore, since the first and second depth cameras may need to operate simultaneously, it is necessary to ensure that the two depth cameras do not interfere with each other during operation. In some implementations, the first and second depth cameras can be configured such that their first and second imaging areas do not overlap, in which case the invisible light projected by both can be indistinguishable in wavelength. However, in more cases, the imaging areas of the two cameras overlap. In this case, it is necessary to limit the invisible structured light projected by the first depth camera to have different wavelengths than the invisible light projected by the second depth camera. For example, they can each project infrared light of different wavelengths, and their respective sensors can obtain the reflected light of their respective projected invisible light through appropriate filters.

[0069] Furthermore, although in Figure 1 and Figure 2 The image shows cargo 3 in the form of a cargo box, but it should be understood that the automatic cargo loading and unloading system of this invention can also be used for loading and unloading other forms of cargo, such as bagged cargo. Furthermore, although in Figure 1 and Figure 2 The illustration shows goods 3 being conveyed at intervals on conveyor belt 21, but in other embodiments, goods, such as bagged goods, can also be conveyed continuously on conveyor belt 21 with almost no intervals. Accordingly, robotic arm 22 can be implemented as a vacuum suction cup, gripper, special-shaped pickup head, or any combination thereof, to pick up one or more goods at a time.

[0070] The automated cargo loading and unloading system according to this utility model has been described in detail above with reference to the accompanying drawings. This disclosure significantly improves the system's coverage by providing mobility to both the loading / unloading robot (acting as a "hand") and the inspection robot (acting as an "eye"), and further achieves full-work-area safety protection by providing the inspection robot with a "safety eye" that can move with the "hand".

[0071] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An automatic cargo handling system for stacking or unstacking cargo (3) by means of a robot arm (22), characterized in that, The inspection robot (1) includes a support (11), on which a first depth data measuring device (12) and a second depth data measuring device (13) are provided. The loading and unloading robot (2) includes a conveyor belt (21) and a robotic arm (22). The first depth data measuring device (12) is a structured light depth data measuring device centrally located on the support, and the second depth data measuring device (13) is a dToF depth data measuring device located on the side of the support.

2. The automated freight handling system of claim 1, wherein, The automated cargo loading and unloading system also includes: An alarm and braking device connected to the robotic arm is used to issue an alarm or / or brake the operating system in response to the second depth data measuring device detecting the presence of a person in the field of view.

3. The automated freight handling system of claim 1, wherein, The dToF depth data measurement device installed beside the support includes one dToF depth data measurement device installed on the side of the support near the robotic arm, or The dToF depth data measuring device installed beside the support includes one dToF depth data measuring device on each side of the support.

4. The automated freight handling system of claim 1, wherein, The first depth data measurement device includes: Structured light projection device for projecting invisible structured light into a first shooting area below the support; and First and second image sensors having a predetermined relative spatial relationship are used to image the first imaging area on which the structured light is projected to obtain first and second two-dimensional images.

5. The automated freight handling system of claim 4, wherein, The first depth data measurement device includes: A visible light image sensor is used to image the first shooting area to obtain a visible light image when the structured light projection device does not project the structured light.

6. The automated freight handling system of claim 1, wherein, The second depth data measurement device includes: The light source module is used to project invisible light into the second shooting area; The dToF sensor generates a sensing signal characterizing distance based on the reception time of the reflected light from the projected invisible light.

7. The automated freight handling system of claim 6, wherein, The dToF sensor includes: A planar array sensor composed of multiple avalanche photoelectric sensors is used, and The light source module includes: Light-emitting device for generating laser beam; and A diffuser is arranged in the propagation path of the laser beam to convert the laser beam into a wide-angle area array beam.

8. The automated freight handling system of claim 6, wherein, The first depth data measurement device includes: Structured light projection device for projecting invisible structured light into a first shooting area below the support; and First and second image sensors, having a predetermined relative spatial relationship, are used to image the first imaging area onto which the structured light is projected to obtain first and second two-dimensional images, respectively: The first shooting area and the second shooting area do not overlap, and / or The invisible structured light projected by the first depth data measuring device and the invisible light projected by the second depth data measuring device have different wavelengths.

9. The automated cargo loading and unloading system as described in claim 1, characterized in that, The inspection robot (1) is a mobile inspection robot and includes slide rails (14, 15) connected to the support (11), and the support is a gantry frame set on the slide rails (14, 15).

10. The automated freight handling system of claim 9, wherein, The slide rail is used to define the parking area of ​​the transport vehicle (4), and the slide rail is arranged along the length of the transport vehicle (4).

11. The automated freight handling system of claim 1, wherein, The robotic arm (22) includes a picking device for picking up goods (3) and is equipped with a calibration disk (23).

12. The automated freight handling system of claim 1, wherein, The conveyor belt (21) is a flexible conveyor belt (21), and the robotic arm (22) is mounted on one end of the flexible conveyor belt (21) near the support (11).

13. The automated freight handling system of claim 12, wherein, The end of the flexible conveyor belt (21) is arranged on a slide rail (15) on which the support (11) can move, and the support (11) serves as a blocking device for the flexible conveyor belt (21).

14. The automated freight handling system of claim 1, wherein, One end of the conveyor belt (21) is connected to an automated shipping and / or receiving system.

Images