Lifting device system for waste collection containers, and method for operating such a system
Patent Information
- Application Number
- EP2023728580
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-05-30
- Filing Date
- 2023-05-30
- Publication Date
- 2025-04-09
AI Technical Summary
Traditional methods for handling and emptying waste collection containers are inefficient, time-consuming, and prone to errors, requiring manual intervention and precise control, which is not effectively automated.
A lifting device system equipped with a 3D stereo camera, display device, and data processing unit that allows for automatic detection and handling of waste containers by capturing and processing 3D coordinates of load-carrying devices, enabling precise positioning and operation of the lifting device without manual control.
Enables efficient, precise, and automated detection and handling of waste containers, reducing manual intervention and improving the accuracy of the lifting and emptying process with an error margin of less than 2 cm, allowing for real-time adjustments and handling of multiple container types.
Smart Images
Figure 1.1
Abstract
Description
[0001] Lifting device system for waste collection containers and method for operating such a
[0002] Technical area
[0003] The invention relates to a lifting device system for lifting and emptying a waste collection container to be emptied and to a method for operating such a lifting device system.
[0004] Technical background
[0005] A lifting device on a truck for lifting and emptying waste collection containers is known from WO2012032438. The lifting device has a coupling tool for coupling to a waste collection container and is provided with a rotating platform and a support mounted thereon for a telescopically extendable boom. The support has nested C-profiles that are telescopically movable vertically. The boom is attached to the free end region of the support so that it can pivot vertically about a pivot axis. Furthermore, a pivot cylinder is provided between the boom and the support to bring the boom into a specific pivot angle relative to the horizontal. A similar lifting device is also known from WO2015150956.
[0006] Such lifting devices are typically mounted on a truck, which also has a loading area for the waste. A user drives the truck to the waste collection containers to be emptied, lifts them with the lifting device, and empties them into the loading area. For this purpose, the lifting device has a control device that allows the user to move the coupling tool according to the waste collection container. To precisely control the movement of the coupling tool, the user usually has to get out of the truck's driver's cab.
[0007] DE20312139 U1 discloses a control device for a loading arm assembly of a vehicle. The control device is equipped with a program control system so that, after picking up a waste container, the relative movement of the loading assembly relative to the vehicle is controlled by force, based on the respective position of the arm parts of the loading arm assembly relative to each other. The control device can be designed to include a "memory" in the form of a storage device that registers the position of the loading arm assembly in space at the start of a picking process, so that after emptying an empty waste container, it can be returned to the same position on the ground.
[0008] However, in many logistics processes, especially in waste disposal, it is desirable to detect and handle waste containers efficiently and accurately. Conventional approaches often require manual intervention and are time-consuming, error-prone, and inefficient. Therefore, there is a need for an improved system that enables automated detection and handling of waste containers.
[0009] Description of the invention
[0010] An object of the invention is to provide a lifting device system and a method for operating a lifting device system which simplifies the operation of a lifting device for lifting and emptying a waste collection container by the user.
[0011] This object is achieved by methods for operating a lifting device system having the features of claim 1 and a lifting device system having the features of claim 11.
[0012] The method is used to operate a lifting device system for lifting and emptying a waste collection container to be emptied from an arrangement of several waste collection containers, wherein the waste collection container has a load-carrying device. The lifting device system comprises a lifting device with a movable boom with a coupling tool for coupling to the load-carrying device of the waste collection container to be emptied, a 3D stereo camera device for recording a local overall view of the arrangement of several waste collection containers, a display device for presenting the local overall view of the arrangement of several waste collection containers to a user of the lifting device system,and a data processing device for controlling the lifting device system. The method comprises the following steps: (a) recording the local overall view of the arrangement of several waste collection containers by the 3D stereo camera device and identifying the load-carrying device of each individual waste collection container; (b) displaying the local overall view of the arrangement of several waste collection containers on the display device and marking the load-carrying device of each individual waste collection container such that the user of the lifting device system can identify and select the waste collection container to be emptied on the display device; (c) controlling the lifting device to the load-carrying device of the selected waste collection container by the data processing device, which is configured such thatthat (i) the 3D coordinates of the load-carrying device of the waste collection container to be emptied are calculated in a coordinate system of the 3D stereo camera device using the 3D stereo camera device, that (ii) the 3D coordinates of the load-carrying device are transformed from the coordinate system of the 3D stereo camera device into a coordinate system of the lifting device, that (iii) the coupling tool is moved to the waste collection container to be emptied using the boom based on the transformed 3D coordinates of the load-carrying device and couples the load-carrying device of the waste collection container to be emptied, that (iv) the waste collection container to be emptied is lifted and emptied at a desired location, and that (v) the emptied waste collection container is moved back to its original location and the load-carrying device is uncoupled.
[0013] The use of the 3D stereo camera system enables precise detection of the load-handling device of a waste collection container. This allows the position of individual waste collection containers or the position of a coupling element arranged on the waste collection container to be recorded in three dimensions. The data processing system can then calculate and execute the control of the lifting device to the waste collection container to be emptied, essentially based on the images from the 3D stereo camera system. The user of the lifting device system can easily select the waste collection container to be emptied on the display device. This container is then automatically lifted, emptied, and returned to its original location by the lifting device system, without any further manual control.
[0014] Preferred embodiments of the invention are set out in the dependent claims.
[0015] In some embodiments, the data processing device can use a convolutional neural network (CNN) to identify the load-handling device. This network is trained with image data of various arrangements of multiple waste collection containers based on characteristic features and shapes of the load-handling device. In conjunction with the machine processing of the image data and machine learning, the precise detection of the load-handling device can be significantly increased, which in turn significantly improves the calculation of the 3D coordinates. The 3D coordinates could be determined with an error of less than 2 cm. Furthermore, the system can be used to recognize different types of load-handling device and / or waste collection container.
[0016] The data processing device can also be configured to recognize different types of load-handling device and / or waste collection container using the Convolutional Neural Network (CNN).
[0017] In some embodiments, the calculation and transformation of the 3D coordinates of the load-carrying device of the waste collection container to be emptied can be performed in real time to continuously check and correct them if necessary. Real-time calculation of the 3D coordinates enables further improvement in the positioning of the coupling tool of the lifting device.
[0018] In some embodiments, the 3D coordinates of the load-handling device can be calculated by evaluating the horizontal disparity of the load-handling device. In other words, the horizontal disparity of the detected waste container receiving device is calculated using the images from the 3D stereo camera device to determine the spatial depth of the object. A correlation method is used to analyze the differences between the two stereo camera images. The determined disparity enables the calculation of the 3D coordinates of the waste container in the camera's coordinate system.
[0019] To effectively handle the waste container, the 3D coordinates of the load-handling device are transformed into the coordinate system of the crane control system or the lifting device. A rotation matrix (R) and a translation vector (t) are used to transfer the point into the coordinate system of the lifting device. The transformed 3D coordinates are then transmitted to the control system of the lifting device.
[0020] In some embodiments, the load-carrying device can be arranged on a top surface of the waste collection container, preferably protruding above the top surface, and coupled from above using a coupling tool. Such an arrangement facilitates the identification of the load-carrying device.
[0021] In some embodiments, the lifting device system can further comprise a sensor system, preferably a camera device, which is arranged on the boom or on the coupling tool and by means of which the relative position of the load-carrying device with respect to the coupling tool can be detected from above for fine adjustment of the coupling tool. With the sensor system, a coupling tool guided from above to the load-carrying device can be precisely positioned. The sensor system or the camera device can be aligned such that it can detect the load-carrying device of the waste collection container to be emptied and thus bring the coupling tool to the correct location for coupling with the waste collection container to be emptied. The sensor system can be used for fine adjustment of the coupling tool with respect to the waste collection container to be emptied.
[0022] In some embodiments, the display device may be a touchscreen and the user may select the waste collection container to be emptied by tapping.
[0023] In some embodiments, the arrangement of multiple waste collection containers can be illuminated with lighting. This is particularly advantageous if the waste collection container needs to be emptied in the dark (e.g., in winter).
[0024] In some embodiments, the coupling with the waste collection container to be emptied and / or the emptying of the waste collection container to be emptied can be carried out automatically or by actuating an activation field on the display device.
[0025] In some embodiments, the lifting system may be mounted on a truck, the 3D stereo camera device may be permanently mounted on the truck, e.g., a roof of a driver's cab, and the display device may be mounted in a driver's cab of the truck. The lifting system may include two 3D stereo camera devices arranged on opposite sides of the truck, allowing the truck to approach an array of multiple waste collection containers from either side.
[0026] In some embodiments, the desired discharge location may be a cargo area on the truck.
[0027] The invention further relates to a lifting device system for lifting and emptying a waste collection container to be emptied from an arrangement of several waste collection containers, wherein the waste collection container has a load-carrying device. The lifting device system comprises a lifting device with a movable boom with a coupling tool for coupling to the load-carrying device of the waste collection container to be emptied, a 3D stereo camera device for recording a local overall view of the arrangement of several waste collection containers, a display device for presenting the local overall view of the arrangement of several waste collection containers to a user of the lifting device system, and a data processing device for controlling the lifting device system. The lifting device system and the data processing device are configured such thatthat (i) the local overall view of the arrangement of several waste collection containers can be recorded by the 3D stereo camera device and the load-carrying device of each individual waste collection container can be identified, that (ii) the local overall view of the arrangement of several waste collection containers with the marked load-carrying device of each individual waste collection container can be displayed on the display device in such a way that the user of the lifting device system can select the waste collection container to be emptied on the display device, that (ii) the 3D coordinates of the load-carrying device of the waste collection container to be emptied can be calculated in a coordinate system of the 3D stereo camera device by means of the 3D stereo camera device and the 3D coordinates of the load-carrying device can be transformed from the coordinate system of the 3D stereo camera device into a coordinate system of the lifting device,that (iii) the coupling tool can be moved to the waste collection container to be emptied by means of the boom based on the transformed 3D coordinates of the load-carrying device, and the load-carrying device of the waste collection container to be emptied can be coupled in order to lift the waste collection container to be emptied, empty it at a desired location, and return the emptied waste collection container to the original location and decouple it.
[0028] In some embodiments, the data processing device may be configured to identify the load-handling device using a convolutional neural network (CNN) trained with image data of various arrangements of multiple waste collection containers based on characteristic features and shapes of the load-handling device.
[0029] In some embodiments, the data processing device may be configured such that the calculation and transformation of the 3D coordinates of the load-carrying device of the waste collection container to be emptied are carried out in real time in order to continuously check them and correct them if necessary.
[0030] In some embodiments, the data processing device can be configured such that the 3D coordinates of the load-carrying device are calculated by evaluating the horizontal disparity of the load-carrying device. In some embodiments, the load-carrying device can be arranged on a top side of the waste collection container, preferably protruding above the top side, and coupled from above using a coupling tool. Such an arrangement facilitates the identification of the load-carrying device.
[0031] In some embodiments, the lifting device system may further comprise a sensor system, preferably a camera device, which is arranged on the boom or on the coupling tool and by means of which the relative position of the load-bearing device with respect to the coupling tool can be detected from above for fine adjustment of the coupling tool.
[0032] In some embodiments, the display device may be a touchscreen and the waste collection container to be emptied may be selectable by tapping.
[0033] In some embodiments, the lifting device system may include lighting with which the arrangement of a plurality of waste collection containers can be illuminated.
[0034] In some embodiments, the 3D stereo camera device may be rotatably mounted.
[0035] In some embodiments, the lifting device system can be mounted on a truck. The truck can further include a loading space into which the waste collection container to be emptied can be emptied.
[0036] In some embodiments, the display device may be mounted in a driver's cab of the truck.
[0037] In some embodiments, the 3D stereo camera device can be mounted on a roof of the driver's cab, so that the overall view of the array of multiple waste collection containers on the display device approximately corresponds to the user's view of the array of multiple waste collection containers from the driver's cab. The lifting device system can have two 3D stereo camera devices arranged on opposite sides of the truck, so that the truck can approach an array of multiple waste collection containers from either side.
[0038] The invention further relates to a truck with a lifting device system as described above.
[0039] Short explanation of the figures
[0040] The invention will be explained in more detail below using exemplary embodiments in conjunction with the drawing(s). In the drawing(s): Fig. 1 shows a truck with a lifting device system ready for lifting and emptying a waste collection container;
[0041] Fig. 2 shows a 3D stereo camera device of the lifting device system;
[0042] Fig. 3 shows a display device of the lifting device system with a representation of an arrangement of several waste collection containers;
[0043] Fig. 4 a display device when selecting a container to be emptied
[0044] waste collection container;
[0045] Fig. 5 an extended lifting device shortly before coupling with a
[0046] waste collection containers;
[0047] Fig. 6 an extended lifting device shortly before emptying the coupled waste collection container.
[0048] Ways to implement the invention
[0049] Fig. 1 shows a truck 1 with a lifting device system. The truck
[0050] 1 comprises a driver's cab 2 and a loading area 3. A lifting device 4 is arranged between the driver's cab 2 and the loading area 3. The lifting device 4 comprises a vertically extendable support 42 with a telescopic boom 40 (see Fig. 6). At the end of the boom 40, a coupling tool 41 is arranged for coupling to a load-carrying device 51 of a waste collection container 6 to be emptied.
[0051] In Fig. 1, the truck is ready to lift and empty the waste collection container 6 to be emptied by moving in front of the arrangement of several waste collection containers 5 and extending the side supports of the truck 1.
[0052] In addition to the lifting device, the lifting device system further comprises a 3D stereo camera device 10, a display device 11, a sensor system 12, preferably a sensor system e.g. in the form of a (simple) camera device, and a data processing device.
[0053] Fig. 2 shows the 3D stereo camera device 10 of the lifting device system. In the illustrated embodiment, this is rotatably mounted on a roof of the driver's cab.
[0054] 2. The 3D stereo camera device 10 can be aligned accordingly to the arrangement of the multiple waste collection containers 5. By mounting the 3D stereo camera device 10 on the roof of the driver's cab 2, it captures an overall view of the arrangement of multiple waste collection containers 5, as seen by the driver or user 7 of the lifting system. This facilitates the subsequent selection of the waste collection container 6 to be emptied.
[0055] The 3D stereo camera device 10 allows the precise determination of the location of the individual waste collection containers, so that the data processing device can calculate and execute the control of the lifting device for the waste collection container to be emptied based on the location of the individual waste collection containers in the arrangement of multiple waste collection containers, essentially based on the images from the 3D stereo camera device. The sensor system 12 or the camera device on the coupling tool 41 (see Fig. 5) then serves to finely adjust or finely position the coupling tool 41 on the load-carrying device 51 of the waste collection container 6 to be emptied.
[0056] Fig. 3 shows a display device 11 of the lifting device system with a representation of an arrangement of multiple waste collection containers 6. The data processing device detects the individual waste collection containers from the arrangement of multiple waste collection containers 6 and sets a marker 13 for their selection. A type 16 of the waste collection container 6 to be emptied can also be selected via the display device 11. Activation fields for coupling 14 and emptying 15 of the waste collection container 6 to be emptied are also provided.
[0057] Fig. 4 shows a display device 11 configured as a touchscreen for selecting a waste collection container 6 to be emptied. The user 7 can tap the waste collection container 6 to be emptied or its marker 13 on the touchscreen and thus start the process for automatically lifting and emptying the waste collection container 6 to be emptied. The coupling of the coupling tool 41 to the waste collection container 6 to be emptied and / or the emptying of the waste collection container 6 to be emptied can occur automatically or be started by the activation fields for coupling 14 or for emptying 15.
[0058] Fig. 5 shows the lifting device 4 shortly before coupling the coupling tool 41 to the load-carrying device 51 of the selected waste container 6 to be emptied. For this purpose, in the embodiment shown, the camera device 12 can be attached to the coupling tool 41, which is directed downward toward the load-carrying device 51 of the waste container 6 to be emptied. In this way, the data processing device can automatically position the coupling tool 41 precisely above the waste container 6 to be emptied and couple it to it. The actual coupling can also be performed automatically or by the user 7 by actuating the coupling activation field 14.Based on the images of the 3D stereo camera device 10, a positioning of the coupling tool 41 with a maximum error of 2 cm can be calculated and carried out, while the fine adjustment of the coupling tool 41 can be carried out using the images of the camera device 12.
[0059] Fig. 6 shows the lifting device 4 shortly before emptying the coupled waste collection container 6. The waste collection container 6 to be emptied is automatically lifted above the loading space 3 of the truck 1 by the lifting device 4. Emptying can also occur automatically or be initiated by the user 7 pressing the emptying activation field 15. After emptying, the emptied waste collection container is automatically returned to its original location.
[0060] The system can be equipped with additional sensors that can detect obstacles that enter the range of action of the lifting device and interrupt the movement of the lifting device.
[0061] The entire process, including the recording and display of the arrangement of several waste collection containers 5, the selection of the waste collection container 6 to be emptied and the control of the lifting device 4 for lifting and emptying the waste collection container 6 to be emptied, as well as the returning of the emptied waste collection container, is calculated and controlled by the data processing device and can be carried out automatically.
[0062] List of designations
[0063] 1 truck
[0064] 2 driver's cab
[0065] 3 cargo space
[0066] 4 Lifting device
[0067] 5 Arrangement of several waste collection containers
[0068] 6 waste collection containers to be emptied
[0069] 7 users
[0070] 10 3D stereo camera setup
[0071] 11 Display device 12 Sensors, camera device
[0072] 13 Marking for waste collection containers
[0073] 14 Activation field for pairing
[0074] 15 Activation field for emptying 16 Selection for waste collection container type
[0075] 40 booms
[0076] 41 Clutch tool
[0077] 42 Boom support
[0078] 51 Load-bearing device
Claims
Patent claims 1. A method for operating a lifting device system for lifting and emptying a waste collection container (6) to be emptied from an arrangement of several waste collection containers (5); wherein the waste collection container (5) has a load-handling device (51); wherein the lifting device system comprises a lifting device (4) with a movable boom (40) with a coupling tool (41) for coupling to the load-handling device (51) of the waste collection container (6) to be emptied, a 3D stereo camera device (10) for recording a local overview view of the arrangement of several waste collection containers (5), a display device (11) for displaying the local overview view of the arrangement of several waste collection containers (5) to a user (7) of the lifting device system, and a data processing device for controlling the lifting device system;wherein the method comprises the following steps: a) recording the local overall view of the arrangement of multiple waste containers (5) by the 3D stereo camera device (10) and identifying the load-handling device (51) of each individual waste container (5); b) displaying the local overall view of the arrangement of multiple waste containers (5) on the display device (11) and marking the load-handling device (51) of each individual waste container (5) such that the user (7) of the lifting device system can select the waste container (6) to be emptied on the display device (11); c) controlling the lifting device (4) to the load-handling device (51) of the selected waste container (5) by the data processing device, which is configured such that; (i) using the 3D stereo camera device (10) the 3D coordinates of the load-bearing device (51) of the waste collection container (5) to be emptied are calculated in a coordinate system of the 3D stereo camera device, that (ii) the 3D coordinates of the load handling device (51) are transformed from the coordinate system of the 3D stereo camera device (10) into a coordinate system of the lifting device (4) such that (iii) the coupling tool (41) is moved to the waste container to be emptied by means of the boom (40) using the transformed 3D coordinates of the load-handling device (51) and couples the load-handling device (51) of the waste container to be emptied, so that (iv) the waste collection container (6) to be emptied is lifted and emptied at a desired location, and that (v) the emptied waste container is moved back to its original location and the load-bearing device (51) is uncoupled.
2. Method according to claim 1, wherein the data processing device for identifying the load-handling device (51) uses a Convolutional Neural Network (CNN) which is trained with image data of different arrangements of several waste collection containers (5) based on characteristic features and shapes of the load-handling device (51).
3. Method according to claim 2, wherein the data processing device is configured such that it can recognize different types of load handling device (51) and / or waste collection container (5) by means of the Convolutional Neural Network (CNN).
4. Method according to one of the preceding claims, wherein the calculation and transformation of the 3D coordinates of the load-bearing device (51) of the waste collection container (5) to be emptied is carried out in real time in order to continuously check and, if necessary, correct them.
5. Method according to one of the preceding claims, wherein the 3D coordinates of the load-bearing device (51) are calculated by evaluating the horizontal disparity of the load-bearing device (51).
6. Method according to any one of the preceding claims, wherein the load-bearing device (51) is arranged on the top side of the waste collection container, preferably projecting beyond the top side, and is connected by The coupling tool (41) is coupled from above.
7. Method according to any one of the preceding claims, wherein the The lifting device system further comprises a sensor system (12), preferably a camera system, which is arranged on the boom (40) or on the coupling tool (41), and by means of which the relative position of the load-handling device (51) relative to the coupling tool (41) is detected from above for fine adjustment of the coupling tool (41).
8. Method according to one of the preceding claims, wherein the coupling with the waste collection container (6) to be emptied and / or the emptying of the waste collection container (6) to be emptied is carried out automatically or by actuating an activation field (14, 15) on the display device (11).
9. Method according to one of the preceding claims, wherein the lifting device system is mounted on a truck (1), the 3D stereo camera device (10) is fixedly mounted on the truck (1) and the display device (11) is mounted in a driver's cab (2) of the truck (1).
10. Method according to claim 9, wherein the desired location is a cargo space (3) on the truck (1).
11. Lifting device system for lifting and emptying a waste collection container (6) to be emptied from an arrangement of several waste collection containers (5) according to the method according to one of claims 1 to 10.
12. Lifting device system according to claim 11, wherein the lifting device system is arranged on a truck (1) and the truck (1) has a cargo space (3) into which the waste collection container (6) to be emptied can be emptied.
13. Lifting device system according to claim 12, wherein the display device (11) is mounted in a driver's cab (2) of the truck (1).
14. Lifting device system according to one of claims 12 to 13, wherein the 3D- stereo camera device (10) is mounted on the roof of the driver's cab (2), so that the overall view of the arrangement of several waste collection containers (5) on the display device (11) corresponds approximately to the view of the user (7) from the driver's cab (2) of the arrangement of several waste collection containers (6). Truck (1) with a lifting device system according to one of claims 11 to 14.