System for monitoring a travel area for transport vehicles in a facility

Abstract

The invention relates to a system (100) for monitoring a travel area for transport vehicles (12) in a facility (10), in particular a packaging machine facility, comprising at least one camera unit (18) designed to capture an image of a predetermined section of the travel area, a traficable floor (16) which constitutes the travel area and is provided on its upper surface with a two-dimensional pattern, and a control unit (14) which is operatively coupled to the camera unit (18), wherein the control unit (14) is designed to identify the two-dimensional pattern on the traficable floor (16) in the image of the travel area captured by the camera unit (18) and to infer the presence of objects (O, V) in the travel area. The invention also relates to a facility (10) having a system of this type and to a method for monitoring the travel area in the facility (10).

Description

[0001] 221381PDE / FEHH 1

[0002] System for monitoring a movement area for transport vehicles in a plant

[0003] Description

[0004] The present invention relates to a system for monitoring a movement area for transport vehicles in a plant, in particular a packaging machine plant, as well as such a plant and a method for monitoring a movement area for transport vehicles in such a plant.

[0005] In systems with multiple stations, and especially in packaging machine systems, the purpose is generally to supply objects to the individual stations in a predetermined sequence, where they are processed or handled accordingly. In the specific case of a packaging machine system, this can include the precise repositioning of product packaging, such as cartons, delivered by a production machine on a conveyor belt, and this is done using so-called robotic lines.

[0006] In this process, open primary packaging, such as tray-shaped containers, which are erected and three-dimensionally fixed from flat cardboard blanks upstream but already within the machine, travel through the packaging machine on a container conveyor, usually parallel to the product conveyor. Several transfer robots are typically arranged in series as handling units, each picking up one or more products from the product conveyor and transferring them into the primary packaging, such as the tray, on the container conveyor.

[0007] Further downstream, these primary packages are often transformed into secondary packages, such as open-top cartons, in one or more layers, as shown in 221381PDE / FEHH 2

[0008] This is also usually done using transfer robots. Further downstream, such secondary packaging is combined with tertiary packaging, for example, stacked on pallets, usually also by robots. The necessary handling operations are carried out at the individual workstations within the packaging line, such as tray separators, blank separators, carton erectors, carton closers, tray fillers, carton fillers, palletizers, etc.

[0009] The transport system used for this purpose can include, in addition to the aforementioned product belt and / or container belt, i.e. conveyor belts, which usually extend over sections of the entire length of the packaging line, also rail-bound sleds, whereby these rails, which can be designed as physical or virtual non-visible guiding devices, usually also extend along the entire length of the packaging line.

[0010] Although in the latter solution the carriages can already be moved independently of each other along the guide rail, such transport systems are still comparatively inflexible, since these carriages, for example, cannot overtake each other, cannot deviate from the guide rails and are also relatively expensive to purchase, so that a limited number of carriages are used and thus the carriages do not yet offer any significant buffering capacity for, for example, packaging placed on them.

[0011] Especially when the underlying work order changes, not just in the form of a change to the product being processed, but a fundamental change to the processing or handling tasks with regard to the product and / or packaging process, such known transport systems are unsuitable, as they require a complex modification of the conveyor belts or guide rails, or even a complete rebuild within the packaging machines. Additionally, these transport systems are not suitable for spontaneously diverting defective products or repackaged goods from the system at various points, unless they were specifically designed for such operational processes from the outset, 221381PDE / FEHH 3, which also entails considerable additional effort.

[0012] It should be noted that although the scenario described above mainly refers to a packaging machine system, similar requirements and tasks also arise in other types of systems with multiple stations, in which products are processed in different ways at individual stations and must therefore be transported between individual stations during the processing process.

[0013] For such tasks, some production plants and logistics facilities have meanwhile started using so-called driverless transport systems, in which driverless transport vehicles, which are often also referred to as AGVs or Automated Guided Vehicles, transport objects or products, such as components or assemblies, from a warehouse to a workstation or between workstations, often also outside the secured machine frames of the workstations.

[0014] These AGVs are typically large and heavy, capable of carrying loads of 50 kg or more. Most importantly, they are usually autonomous vehicles equipped with a wide range of sensors and their own on-board control system, often including on-board navigation with position determination based on the environment via on-board cameras, as well as collision avoidance systems. This makes them expensive and complex to maintain. Furthermore, such systems typically include a central control unit, such as a control center, with which the AGVs communicate wirelessly and from which they receive, for example, work orders or information about their own positions within the facility.

[0015] Most of these AGVs, however, independently determine their current spatial orientation using sensors and reference points and transmit this information to the central control system, which usually specifies transport orders with a start and destination point as well as a start and destination time, but often handles the navigation between them. 221381PDE / FEHH 4

[0016] The onboard control of the AGVs is left to the user, as is real-time collision prevention based on locally acquired information about the environment of the respective vehicles.

[0017] It is essential to ensure that the relevant drivable surface is in the expected condition, meaning that, for example, no unexpected objects or obstacles are present in a travel area designated and approved for the automated guided vehicles (AGVs). These could include people or vehicles, or even dirt or debris on the surface, all of which could negatively impact the AGVs. While the AGVs themselves, using their collision avoidance systems and potentially other sensors, might be able to detect obstacles on the surface and take appropriate action, in many specific applications it can be advantageous to centrally monitor the travel area and centrally process and manage the results, as well as potentially...will be made available to the affected transport vehicles.

[0018] Accordingly, there is a need for a system to monitor a movement area for transport vehicles in a corresponding facility, by means of which it can be ensured that a drivable surface intended for use by driverless transport vehicles is in fact in the expected condition and, if necessary, appropriate measures can be initiated in the event of a deviation from this.

[0019] To achieve this functionality, a system for monitoring a movement area for transport vehicles in a plant, in particular a packaging machine plant, is proposed according to the invention, comprising at least one camera unit which is configured to take an image of a predetermined section of the movement area, a drivable surface which forms the movement area and which is provided on its upper surface with a two-dimensional pattern, and a control unit operationally coupled to the camera unit, wherein the control unit is configured to identify the two-dimensional pattern on the drivable surface in the image of the movement area taken by the camera unit and to deduce from this the presence of objects in the movement area.In this case, at least one camera unit can be permanently installed in the system itself, i.e., not physically assigned to a transport vehicle or similar in a mobile manner.

[0020] Accordingly, the system according to the invention is based on determining the presence of objects on the known two-dimensional pattern on the drivable surface, instead of attempting to identify the objects themselves using a significantly more complex image recognition process. For example, by comparing the image data captured by the at least one camera unit with the data available to the control unit and stored, for example, in a memory unit, with regard to the locations of known objects, dirt on the drivable surface, unexpected obstacles, vehicles, or people can be detected.

[0021] Based on this detection of obstacles or objects on the drivable surface, various reactions can be initiated. In a simple case, the system's driverless transport vehicles could be instructed to bypass the affected areas. Alternatively, messages could be sent to a human operator, who could then manually assess the situation and, if necessary, remove obstacles or debris. It is evident that this not only improves the operational safety of the system but also enhances its overall efficiency, as functional problems can be avoided from the outset, and collisions with objects or similar hazards can be prevented.

[0022] In particular, the evaluation of the image data recorded by at least one camera unit is based on detecting obstacles in the corresponding 221381PDE / FEHH 6

[0023] Camera images can be differentiated by identifying the aforementioned two-dimensional pattern. For example, all objects located in front of this pattern in the camera images can be located and classified as obstacles or unknown objects, perhaps using the map data already mentioned. By selecting a suitable two-dimensional pattern, it can be ensured that it is not reproducible in different recording environments and directions, or that it is unique under typical recording conditions.

[0024] Depending on the previously mentioned detection of known and unknown objects, the system can also issue a permit for individual driverless transport vehicles to access a specific area, or, for example, individual sections of the system can be closed to traffic. Furthermore, other reactions to detected obstacles are conceivable, such as issuing speed limits in certain sections of the system.

[0025] As already indicated several times, the control unit in the system according to the invention can have access to a map of the movement area in which the locations of known objects are stored. Such a map can accordingly be updated when additional objects are detected and / or, depending on the positions of the detected objects, individual sub-areas of the movement area can be handled differently, for example, blocked off or subjected to speed limits.

[0026] According to the invention, the two-dimensional pattern on the drivable surface can be selected such that the pattern of the sections of the movement area monitored by each of the camera units allows for the unambiguous identification of positions thereon, and / or the two-dimensional pattern on the drivable surface can be formed by a binary-coded, continuous code in two dimensions or a periodic, pseudo-random pattern in two dimensions. 221381PDE / FEHH 7

[0027] By performing a free-space search on the section of the movement area covered by each of the camera units, an improved result and increased safety can be achieved compared to an object search against an unknown background, as particularly robust pattern recognition algorithms can be used.

[0028] Furthermore, it would be conceivable that the drivable surface comprises multiple roadway modules, each with identical two-dimensional patterns on its upper surface. These patterns allow for the unambiguous determination of a position within the local coordinate system of the respective roadway module. This simplifies the pattern itself, as only the surface of a corresponding roadway module needs to be uniquely filled with a suitable pattern. Transitions between corresponding roadway modules can also be determined separately, which can therefore be done at a lower resolution.

[0029] Furthermore, the system according to the invention can comprise a plurality of camera units and a single control unit, which is coupled to all of the camera units, wherein the camera units are preferably arranged such that they collectively cover the entire range of movement of the system. Thus, in this variant, the principle of sensor fusion can be applied, and the corresponding data from the individual camera units can be collectively evaluated by the single control unit and used to create a complete overview of the system.

[0030] As already indicated several times, the control device can also be configured to only permit entry into a section of the travel area if no unknown or unidentifiable objects are present within it. This can be achieved, in particular, by the central control unit transmitting appropriate data to the individual automated guided vehicles. 221381PDE / FEHH 8

[0031] According to a further aspect, the present invention relates to a system, in particular a packaging machine system, comprising a plurality of stations for handling objects, at least one driverless transport vehicle, a central control unit which is communicatively coupled with the at least one transport vehicle and optionally the stations and is configured to coordinate the operation of the at least one transport vehicle and optionally the stations, and a system according to the invention of the type described above. In this system, at least some of the stations can be configured to receive objects from or transfer them to the at least one transport vehicle.

[0032] Furthermore, it should be noted that the control unit of the system for monitoring the movement area can be integrated or operationally coupled with the central control unit of the plant, so that direct integration of the system into the control of the overall plant can be achieved.

[0033] Furthermore, it should be noted that at least one transport vehicle may be equipped to determine its own position within the movement area using suitable localization means, in particular by means of a corresponding camera unit, which is designed to capture at least the drivable surface in order to determine its current position within the facility. It follows that the two-dimensional pattern on the drivable surface can play a central role not only for the camera unit of the monitoring system, but can also be recorded and evaluated by the individual transport vehicles for self-localization.

[0034] Particular attention should be drawn to the aforementioned version of a corresponding drivable surface, which is constructed from a plurality of driving surface modules, each featuring identical two-dimensional patterns on its upper surface. In such a system, when the pattern is evaluated, the position of the transport vehicle is initially known only within the local coordinate system 221381PDE / FEHH 9 of the driving surface module it is currently traversing. Additionally, transitions between individual driving surface modules can be traced based on the pattern, enabling a transfer to a global coordinate system for the entire system.

[0035] However, in such configurations, situations can arise where the vehicle in question is initially unaware of which of the track modules it is currently on, either during commissioning or in the event of malfunctions. This is where the system's camera unit for monitoring the movement area offers a further advantage: it can be used to locate the driverless transport vehicles, at least to the extent of determining which track module they are currently on. This does not require particularly high precision or resolution, as only a rough assignment of the vehicle to the track module it is currently on is needed.

[0036] In this context, at least one transport vehicle can be configured to perform a predetermined movement pattern, for example, a low-amplitude oscillating movement around its current position, upon instruction from the central control unit, in order to enable identification and, if necessary, localization by the system for monitoring the movement area. This ensures that, by detecting and recognizing this characteristic movement, the currently addressed driverless transport vehicle is uniquely identified and can be located within the system. Accordingly, this further development of a system according to the invention represents a way to support driverless transport vehicles in their self-localization, particularly in applications where a plurality of track modules with identical two-dimensional patterns on their surface are installed.It is understood that the corresponding detection of a moving vehicle cannot be accomplished by taking a single picture; rather, the camera unit must either take a number of pictures at short intervals or continuously provide image data from which temporal changes in the corresponding recorded state of the system can be derived.

[0037] Furthermore, the central control unit can be configured, as already indicated above, to adapt the operation of at least one transport vehicle, taking into account the objects detected by the system for monitoring the movement area, for example with regard to defining currently blocked areas or specifying speed limits in potentially problematic sections.

[0038] Additionally, the system according to the invention can comprise at least one further vehicle which is configured for autonomous movement within the system, independent of the central control unit. Examples include cleaning robots and similar external systems that move independently of the central control unit within the system and whose operation therefore cannot be coordinated by it. Since the camera unit of the system according to the invention also monitors these further vehicles, and the central control unit can adjust the operation of the at least one transport vehicle communicating with it accordingly, simultaneous operation of vehicles that communicate with and are coordinated by the central control unit, and of those that move completely autonomously within the system, is made possible without the risk of collisions or other disruptions.

[0039] Finally, the present invention relates to a method for monitoring a movement area for transport vehicles in a system of the type described above, comprising capturing an image of a predetermined section of the movement area and identifying the two-dimensional pattern on the drivable surface and deducing the presence of objects in the movement area. It is understood that the image capture can be performed at predetermined times, at a predetermined frequency, or continuously, depending on the current operating state of the system and the parameters to be monitored. 221381PDE / FEHH 11

[0040] Further features and advantages of the present invention will become even clearer from the following description of an embodiment thereof, when viewed together with the accompanying figures. These show in detail:

[0041] Figure 1 shows a schematic top view of a system according to the invention, in which a system according to the invention is used; and

[0042] Figure 2 shows an alternative schematic representation of a camera unit and a drivable surface of the system from Figure 1.

[0043] Figure 1 shows a purely schematic top view of a system according to the invention, which is generally designated by reference numeral 10, and which comprises an indicated system 100 according to the invention as well as a plurality of stations M1 to M4 for handling objects. The system 10 can, for example, be a packaging machine system, wherein at least some of the stations M1 to M4 can be configured to receive objects from or transfer them to driverless transport vehicles 12.

[0044] The transport vehicles 12, of which only one is shown in Figure 1 for clarity, form part of the system 10 according to the invention and are generally designed to transport objects between stations M1 to M4 of the system or other transfer points. For this purpose, they are equipped with additional components that enable driverless operation, such as a vehicle control unit 12b for controlling the vehicle's functions, a communication unit (not shown) for establishing a wireless communication link with a central control unit or monitoring station 14, a drive system controlled by the vehicle control unit 12b, and sensor and safety systems for preventing collisions and similar incidents with stationary objects, people, or other vehicles. 221381PDE / FEHH 12

[0045] The central control unit 14 of plant 10 is designed to coordinate the operation of the transport vehicles by means of work instructions and the like, and, if necessary, depending on the design of plant 10 and the intended work processes, also the operation of stations M1 to M4 for improved integration of the components of plant 10.

[0046] In this system, the transport vehicles 12 move on a drivable surface 16, which also forms part of the system 100 according to the invention and which is composed of a plurality of driving surface modules 16a, of which only a single one is shown in outline by way of example in Figure 1, and which are provided on their upper surface with an identical two-dimensional pattern, which can, for example, be printed onto the upper surface of the driving surface modules 16a with a suitable dye. The corresponding drivable surface 16 with the plurality of driving surface modules 16a and the two-dimensional pattern arranged on its upper surface is also shown again in a different view in Figure 2, whereby it should further be noted that different dimensions of driving surface modules can be provided, which, however, can have identical two-dimensional patterns.

[0047] In order to determine their position within the system 10 with high precision in order to process work and transport orders appropriately, the transport vehicles 12 are each equipped with camera units 12a, which are arranged to capture a predetermined section of the drivable surface 16. This predetermined section is accordingly in a fixed spatial relationship to the body of the corresponding transport vehicle 12, and by suitable image analysis of the two-dimensional pattern provided on the top surface of the corresponding driving surface module 16, both the current position and an angle in a local reference system of the currently traversed driving surface module 16 can be derived.

[0048] The necessary image recognition and processing can, for example, be carried out by the vehicle control unit 12b or another control unit coupled to it using known algorithms (221381PDE / FEHH 13). In this context, it should also be noted that transitions from one of the road surface modules 16a to an adjacent one can be recognized and processed accordingly during the acquisition of the two-dimensional pattern, so that a transition to the local coordinate system of the now traversed road surface module 16a is subsequently performed.

[0049] Furthermore, it should be noted that both Figure 1 and Figure 2 show a camera unit 18 belonging to the system 100, which captures an image of a predetermined section of the movement area on the drivable surface 16. The system 100 may include several camera units 18 to cover a larger part or the entire movement area. All of the camera units are coupled to the central control unit 14, which also acts as the control unit of the system 100, in addition to its function as the control center of the system 10.

[0050] By suitable image processing, the two-dimensional pattern on the drivable surface 16 can be identified as indicated in Figure 2, and the presence of objects O and contaminants V can be deduced from this, since these are superimposed on the pattern in the recorded images. Furthermore, a comparison can be made with a map of the system 10 available to the control unit 14, which contains known objects, to check whether the objects O detected in the image data are possibly previously unknown obstacles that would have to be driven around by the vehicles 12, or contaminants V that might require a reduced maximum speed in their vicinity.

Claims

221381PDE / FEHH 14 Claims 1. System (100) for monitoring a movement area for transport vehicles (12) in a plant (10), in particular a packaging machine plant, comprising: - at least one camera unit (18) which is configured to take an image of a predetermined section of the movement area; - a drivable surface (16) which forms the movement area and which has a two-dimensional pattern on its upper surface; and - a control unit (14) operationally coupled to the camera unit (18); wherein the control unit (14) is configured to identify the two-dimensional pattern on the drivable surface (16) in the image of the movement area recorded by the camera unit (18) and to derive from this the presence of objects (O, V) in the movement area.

2. System (100) according to claim 1, wherein the control unit (14) has access to a map of the movement area in which locations of known objects are stored.

3. System (100) according to one of the preceding claims, wherein the pattern of the section of the movement area monitored by each of the camera units (18) enables a unique identification of positions therein.

4. System (100) according to one of the preceding claims, wherein the two-dimensional pattern on the drivable surface (16) is formed by a binary-coded continuous code in two dimensions or a periodic, pseudo-random pattern in two dimensions. 221381PDE / FEHH 15 5. System (100) according to one of the preceding claims, wherein the drivable surface (16) comprises a plurality of driving surface modules (16a) which are provided on their upper side with identical two-dimensional patterns.

6. System (100) according to one of the preceding claims, comprising a plurality of camera units (18) and a single control unit (14) which is coupled to all of the camera units (18), wherein the camera units (18) are preferably arranged such that they collectively cover the entire range of movement.

7. System (100) according to one of the preceding claims, wherein the control device (14) is further configured to allow passage through a section of the movement area only if there are no unknown or unidentifiable objects in it.

8. Plant (10), in particular packaging machinery plant, comprising: - a plurality of stations (M1-M4) for handling objects; - at least one driverless transport vehicle (12); - a central control unit (14) which is communicatively coupled with the at least one transport vehicle (12) and, if applicable, the stations (M1-M4) and is configured to coordinate the operation of the at least one transport vehicle (12) and, if applicable, the stations (M1-M4); and - a system (100) according to any one of the preceding claims.

9. System (10) according to claim 8, wherein at least some of the stations (M1-M4) are equipped to receive objects from or transfer them to the at least one transport vehicle (12). 221381PDE / FEHH 16 10. System (10) according to claim 8 or 9, wherein the control unit (14) of the system (100) for monitoring the movement area is integrated or operationally coupled with the central control unit (14) of the system (10).

11. Plant (10) according to one of claims 8 to 10, wherein the at least one transport vehicle (12) is equipped to determine its own position within the movement area by means of suitable localization means, in particular by means of a corresponding camera unit (12a) which is equipped to detect at least the drivable surface (16) in order to determine its current position within the plant.

12. System (10) according to one of claims 8 to 11, wherein the at least one transport vehicle (12) is configured to perform a predetermined movement pattern, for example a low-amplitude quivering movement around its current position, upon instruction from the central control unit (14) in order to enable identification and, if necessary, localization by the system (100) for monitoring the movement area.

13. System (10) according to one of claims 8 to 12, wherein the central control unit (14) is configured to adapt the operation of the at least one transport vehicle (12) taking into account the objects (O, V) detected by the system (100) for monitoring the movement area.

14. System (10) according to one of claims 8 to 13, further comprising at least one further vehicle which is equipped for autonomous movement within the system (10) independent of the central control unit. 221381PDE / FEHH 17 15. Method for monitoring a movement area for transport vehicles (12) in a system (10) according to one of claims 8 to 13, comprising: - Capturing an image of a predetermined section of the movement area; and - Identifying the two-dimensional pattern on the drivable surface (16) and deducing the presence of objects (0, V) in the movement area.

Images