Monitoring apparatus for a storage area, intralogistics arrangement having the monitoring apparatus, and monitoring method
Patent Information
- Application Number
- EP2022830894
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-02-08
- Filing Date
- 2022-12-27
- Publication Date
- 2024-12-18
Smart Images
Figure 1.1
Abstract
Description
[0001] Description
[0002] Title for a with the as well as
[0003] State of the art
[0004] The invention relates to a monitoring device for a warehouse area. Furthermore, the invention also relates to an intralogistics arrangement comprising the monitoring device, and a monitoring method, in particular comprising the monitoring device and / or the intralogistics arrangement.
[0005] Intralogistics describes the logistical flow of materials and goods that takes place within a company's premises. Intralogistics is particularly important for lean manufacturing, as it requires the uninterrupted supply of raw materials to the production lines and, at the same time, the uninterrupted removal of the finished products. For an uninterrupted supply of raw materials, sufficient quantities must be available. However, economic considerations come into play here, as, in principle, a buffer stock of any size would be desirable for uninterrupted supply, but this also represents dead capital and becomes redundant, especially during a product change on the production lines. Therefore, it is desirable to keep buffer storage as small as possible.
[0006] From another technical field, DE 10 2016 223 859 A1 is known, which relates to a camera for monitoring a surveillance area. The camera is designed to record surveillance images of a surveillance area and mask people in the surveillance images to support data protection. Furthermore, the camera is designed to output only output images with masked people.
[0007] Disclosure of the invention
[0008] The subject matter of the invention is a monitoring device having the features of claim 1, an intralogistics arrangement having the features of claim 10 and a monitoring method having the features of claim 15. Preferred or advantageous embodiments of the invention emerge from the subclaims, the following description and the attached figures.
[0009] The subject matter of the invention is a monitoring device that is suitable and / or designed for monitoring a storage area. The storage area can be an open-plan area and / or a closed area, in particular a hall area. The storage area can be contiguous or distributed.
[0010] The monitoring device has at least one camera for recording a camera image. It can also be provided that the monitoring device has multiple cameras, each of which performs the same function as the camera, but in different sections of the storage area. The camera can be designed as a low-cost grayscale or color camera or color depth camera (time-of-flight, active stereo camera with pattern projector, stripe projector, laser scanner, or similar). In particular, the camera is designed to recognize depth information in the camera image or to record the information for depth detection so that depth detection can be performed subsequently.
[0011] The camera image can depict the storage area with a plurality of lanes with storage locations. The lanes are arranged parallel to one another, with the storage locations lined up along the lanes. The lanes extend parallel to a floor of the storage area. Storage goods can be arranged on each of the storage locations. In particular, the lanes, together with the storage locations and the storage goods, form a buffer storage for the stored goods. The storage area can be provided with several such buffer storage areas or other storage areas with lanes with storage locations. It is possible for the different storage areas to be monitored by different cameras.
[0012] Preferably, the camera is oriented relative to the storage area such that the camera's field of view is directed from above or substantially from above, in particular vertically from above or substantially vertically from above, onto the storage area with the tracks. Alternatively, the camera can also be directed laterally or obliquely from above onto the storage area with the tracks. In particular, a viewing direction of the camera encloses an angle between 75° and 90°, preferably between 85° and 90°, with a plane of a floor of the storage area.
[0013] The stored goods are particularly designed to be stackable. Thus, one or more stored goods can be arranged at a storage location, stacked one above the other and / or forming a column.
[0014] The monitoring device has an evaluation device for evaluating the camera image. The evaluation device is designed, in particular, as a digital data processing device, such as a computer, a server, or a cloud. The evaluation device is based, in particular, on digital image processing. The evaluation device is designed, using programming and / or circuitry, to determine the position of the stored goods relative to the storage locations.
[0015] The advantage of the invention is that the monitoring device in the storage area provides real-time information on the storage area's stock levels and where any required stock can be found. This prevents the system from searching for a stock item in a lane with empty storage locations. At the same time, it supports the retrieval of stock items from locations where the storage locations are full. The monitoring device thus represents a key element in an improved intralogistics solution.
[0016] In a preferred development of the invention, the evaluation device has a learning module, wherein the learning module is designed in terms of programming and / or circuitry to learn the tracks and the storage locations and, optionally, additionally, their 6DoF poses as target position data, a target fill level and / or a target stack height of the storage locations in the storage area based on a learning camera image from the camera with filled storage locations. The 6DoF pose comprises, in particular, all six degrees of freedom (6D0F: 6 degrees of freedom, translation in X, Y, Z and rotation about the X, Y, Z axes). The learning module can analyze the learning camera image using digital image processing and can thus, on the one hand, separate the different stored goods from one another so that it can determine the storage locations and the tracks.Furthermore, a 3D analysis of the teach-in camera image can be used to determine the height of the stacked goods at a storage location and save it as the target stack height. The teach-in module can be implemented as a software module or a hardware module. The teach-in of the lanes and storage locations in the storage area can be automated, as shown; alternatively, the geometry can also be entered as a model into a model module of the evaluation device, e.g., by an integrator.
[0017] In a preferred development of the invention, the evaluation device is designed to determine the degree of area overlap between the stored goods and the storage locations. If the stored goods are completely and correctly positioned in a storage location, the degree of area overlap is 100%. If the stored goods are located between two storage locations, the degree of area overlap can be, for example, 50% in each case. The percentage of overlap with the corresponding reference cell as a storage location in the storage area, in particular in the trolley grid, defines a fill level (FRTC) for the storage location. By evaluating the degree of area overlap of the stored goods, the stored goods can be assigned to one of the storage locations. Alternatively or additionally, a stored goods center of gravity and a storage location center of gravity are determined. For example, the centers of gravity can be determined as area centers.Furthermore, a center of gravity distance between the center of gravity of the stored goods and the storage space center of gravity is determined. Depending on the center of gravity distance, the stored goods are assigned to the storage space whose center of gravity distance is the smallest. With these variants, it is possible to assign the stored goods to a storage space in a defined manner. In a preferred development of the invention, the evaluation device has an evaluation module which is designed using programming and / or circuitry to determine a displacement of the stored goods along the tracks and / or a rotation of the stored goods, in particular about a vertical axis, based on the camera image and the known tracks and storage spaces. The evaluation module can thus orient itself on the learned and / or maintained geometry of the storage area and can, for example, use digital image processing to determine a corresponding displacement and / or rotation, in particular about the vertical axis of the stored goods.
[0018] In a preferred development of the invention, the evaluation device and / or the evaluation module is designed to determine a tilt of the stored goods. In conjunction with the previous development, in particular a 6DoF pose actual position is determined. With knowledge of the tilt and / or the 6DoF pose actual position, the intralogistics arrangement described below, in particular the
[0019] The guidance system can then generate adapted pickup instructions. From the detected 6DoF pose, the relative deviation from the learned target positions can then be determined, which can serve as a position correction for navigation during the pickup instructions.
[0020] The evaluation device and / or the evaluation module is particularly preferably designed to detect the stack height of the stored goods. The stack height of the stored goods can be used to estimate how full the respective storage location is. With knowledge of the stack height, the intralogistics arrangement described below, in particular the control device, can generate adapted pick-up instructions. Optionally, the evaluation device is designed for depth detection in the camera image. The stack height can be determined either only in the color or black and white image, or in particular via depth detection, in which the height of the stack of stored goods is also detected via depth detection. If the evaluation in the depth image is omitted, e.g.Because only a normal color or black and white camera is used, the translation in X and Y and the rotation around the Z axis can still be determined and the translation in Z as the distance of the stack top surface to the camera can also be estimated by scaling the detected stack contour with the help of the camera calibration.
[0021] Particularly preferably, it can be provided that the evaluation device and / or the evaluation module is designed to recognize the type of material and / or type of stored goods, in particular the type of product in the stored goods. This information can also be used to form the collection instructions.
[0022] In a preferred embodiment of the invention, the training module and / or the evaluation module, or a combination of training module and evaluation module, is embodied as a neural network. The neural network allows the detection of traces with storage locations as well as the detection of stored goods to be initially trained using training images of the storage area. In a second phase, the trained neural network can be used to evaluate the camera image.
[0023] In a preferred implementation of the invention, the neural network is embodied as a DCNN (deep convolutional neural network). Particularly preferably, the neural network and / or the DCNN are embodied as a Mask-R-CNN. In particular, the implementation initially involves object detection and the determination of bounding boxes around the detected objects, followed by a semantic segmentation of the detected objects in each bounding box. This results in the outer contour of each detected item of storage, in particular in the form of a stacked object. The outer contour can be used to determine the degree of surface overlap in comparison with the outer contour of the storage locations. These architectures or types of neural networks can be used particularly efficiently and accurately in such image analysis methods.
[0024] In a preferred implementation of the invention, the stored goods are each configured as a small load carrier. In particular, the stored goods and / or the small load carrier are configured as a box, wherein a stack of boxes can be arranged in the storage locations. Optionally, the stored goods and / or the small load carrier can be positioned on a trolley in the storage locations to simplify manual movement of the stored goods. Even after manual movement and / or rearrangement of the stored goods in the storage area, their position in the storage area can be determined again via the monitoring device.
[0025] Optionally, an identification for the type of stored goods is arranged on the stored goods at a position visible to the camera, at least on the topmost stored goods. For example, the identification can be applied to a lid of the stored goods. The monitoring device is designed to determine the type of stored goods via the evaluation unit and / or the evaluation module. In this way, the output of the monitoring device includes not only the position of the stored goods in the storage area, but also the type of stored goods. There is the restriction that only visible stored goods in a stack in the storage locations can be recorded by the camera.
[0026] A further subject of the invention is an intralogistics arrangement with a storage area, wherein the storage area has a plurality of lanes with storage locations. The intralogistics arrangement has the monitoring device as described above, wherein the monitoring device is designed to determine a position of the stored goods relative to the storage locations. Thus, the intralogistics arrangement has knowledge of the position of the storage locations and / or lanes as well as the position of the stored goods.
[0027] In a preferred embodiment of the invention, the monitoring device is designed to monitor restricted areas and / or delivery areas in the warehouse area. In particular, the monitoring device can be designed to monitor whether these restricted areas and / or delivery areas are actually kept clear or whether they are occupied contrary to the logistics regulations. This provides added value to the intralogistics arrangement and / or the monitoring device.
[0028] A possible further development envisages that the surveillance device is designed to detect and anonymize persons in the storage area. In principle, it is assumed that the camera images are only used for analysis to detect fill level and are then discarded. In the event that the camera images are archived for documentation purposes, the surveillance device can advantageously anonymize the persons in the storage area, allowing the camera images to be archived in compliance with data protection law.
[0029] In a preferred embodiment of the invention, the intralogistics arrangement comprises a control device, wherein the control device is configured to transmit pickup instructions to automated vehicles and / or pickup personnel based on the position of the stored goods. The control device is thus responsible for the material flow, wherein, based on the position of the stored goods relative to the storage locations and thus also the occupancy of the lanes, automated vehicles and / or pickup personnel can be instructed accordingly to pick up the stored goods from the correct storage locations. In particular, by determining the displacement, rotation, and / or tilt of the stored goods, in particular the 6doF pose, it is achieved that the pickup instructions do not refer to the storage locations, but can be coordinated with the actual position of the stored goods.This means that collection instructions can also be created for misplaced, particularly shifted, twisted and / or tilted stored goods.
[0030] In a preferred development of the invention, the master control device is configured to receive storage requirements from production facilities and, additionally, to generate the pick-up instructions based on the storage requirements. Thus, the master control device receives a storage requirement from the production facilities, since, for example, they require new storage goods for further production. Based on the known storage requirements and the known positions, and optionally additionally based on knowledge of the displacement, rotation, tilt, storage goods type, and / or stack height, the master control device can generate the pick-up instructions.
[0031] A further subject matter of the invention relates to a monitoring method with the monitoring device as described above and / or with the intralogistics arrangement as described above, wherein in the method a camera image of a storage area with a plurality of lanes with storage locations is recorded and wherein the camera image is evaluated to determine the position of the stored goods.
[0032] A further subject matter of the invention relates to a computer program configured to perform the above-described method when executed on a computer, a device, in the cloud, and / or on the monitoring device and / or intralogistics arrangement as described above. A further, optional subject matter of the invention relates to a storage medium, in particular a non-volatile machine-readable storage medium, containing the computer program.
[0033] Further features, advantages, and effects of the invention will become apparent from the following description of preferred embodiments of the invention and the accompanying figures. These show:
[0034] Figure 1 is a schematic block diagram of an intralogistics arrangement as an embodiment of the invention;
[0035] Figure 2 shows a schematic learning camera image of the intralogistics arrangement in Figure 1;
[0036] Figure 3 is a schematic camera image of the intralogistics arrangement in Figure 1;
[0037] Figure 4 shows a schematic result of the intralogistics arrangement in Figure 1 based on the camera image in Figure 3.
[0038] Figure 1 shows a schematic block diagram of an intralogistics system 1 as an exemplary embodiment of the invention. The intralogistics system 1 is designed to organize a flow of goods and / or materials for a production facility 2. The production facility 2 can, for example, manufacture automotive products, electronic products, etc.
[0039] The intralogistics arrangement 1 comprises a monitoring device 3, wherein the monitoring device 3 is configured to monitor a storage area 4. The monitoring device 3 comprises at least one camera 5, wherein the camera 5 is directed toward the monitoring area 4. The camera 5 records a camera image or, during operation, several camera images in succession or a video stream of camera images.
[0040] In the storage area 4, for example, a buffer storage 6 is arranged, wherein the storage area 4 and / or the buffer storage 6 has a plurality of tracks 7 which extend parallel on the floor of the storage area 4 and / or the buffer storage 6. In the frontal view of the monitored area 4 shown, a plurality of storage locations 8 are arranged on the tracks 7, wherein in the exemplary embodiment shown, the storage locations 8 are partially occupied by a stored item 9. The stored item 9 is designed in particular as a stackable stored item, so that the storage locations 8 - to the extent that they are occupied - are occupied by at least one stored item 9 and partially by a stack of stored items 9.
[0041] The monitoring device 3 has an evaluation device 10, wherein the evaluation device 10 is configured to detect the position of the stored goods 9 relative to the storage locations 8. Figure 1 shows two empty storage locations 8 on the left side of the storage area 4 and / or the buffer storage 6. Next to them, occupied storage locations 9 are shown.
[0042] The storage goods 9 are designed, for example, as small load carriers, in particular as boxes, wherein the small load carriers can be stacked as storage goods 9 at the storage locations 8.
[0043] The camera 5 is directed from above onto the storage area 4, in particular onto the buffer storage 6, so that the camera 5 can capture the tracks 7 in longitudinal extension with the storage locations 8 and the stored goods 9 arranged thereon.
[0044] The camera image of the camera 5 is transmitted to the evaluation device 10, wherein the evaluation device 10 determines the position of the stored goods 9 relative to the storage locations 8 on the basis of the camera image using digital image processing and evaluation algorithms.
[0045] In this exemplary embodiment, the evaluation device comprises a neural network 12, wherein the neural network 12 can be divided, at least functionally, into a training module 13 and an evaluation module 14. In the training module 13, the recognition of the lanes 7, the storage locations 8, and the position of the stored goods 9 is trained in the neural network 12 based on training camera images. The evaluation module 14 uses the trained neural network 12 and can subsequently determine the lanes 7, the storage locations 8, and the position of the stored goods 9 based on current camera images of the surveillance area 4 and / or the buffer memory 6 of the camera 5.
[0046] The evaluation module 14 can, in particular, determine a displacement of the stored goods 9 along the tracks 7 and / or a rotation of the stored goods 9. Optionally, the evaluation module 14 can additionally determine a tilt of the stored goods 9. In particular, the monitoring device 3 can determine a 6DoF pose of the stored goods 9. Optionally, it is also possible for the monitoring device 3 to determine a stack height and a storage item type of the stored goods 9.
[0047] The determined position and, in particular, the precise location of the stored goods 9 can subsequently be used by the intralogistics system 1 to transmit a pickup of the stored goods 9 with corresponding pickup instructions to automated vehicles 16 and / or pickup personnel, so that the stored goods 9 can be picked up even if they are not precisely arranged in the storage locations 8, in particular if they are shifted, twisted, and / or tilted. The monitoring device 3 thus enables fully automated operation of the intralogistics system 1.
[0048] In particular, it is optionally possible for a target stack height and a 6DoF pose to be determined as the target position of the storage locations 8 in the storage area 4 and / or the buffer storage 6 via a or the teach-in camera image of the storage area 4 and / or the buffer storage 6, and for the evaluation module 14 to be able to determine the position, an actual stack height, and an actual 6DoF pose based on the current camera image and the known tracks 7 and storage locations 8, as well as optionally the target stack height and the target position as a 6DoF pose. The neural network 12 is designed as a DCNN and / or as a Mask-R-CNN. The intralogistics arrangement 1 has a master control device 15, wherein the master control device 15 is designed to control the material flow and / or goods flow between the production system 2 and the storage area 4.The control device 15 receives from the evaluation device 10 the data on the lanes 7, storage locations 8, and the positions of the stored goods 9, in particular their exact location, including the displacement, rotation, tilt, and / or actual 6DoF pose. Optionally, the stored goods 9 can be provided with an identification, which can be captured by the camera 5, so that the type of stored goods 9 from the respective stack at the storage location 8 is also captured and forwarded to the control device 15.
[0049] The production facility 2 communicates a storage requirement to the master control device 15, e.g., so that the production facility 2 can continue production. Based on the storage requirement and the data on the lanes 7, storage locations 8, the position of the storage item 9, and other determined information, such as the displacement, rotation, and / or tilt, in particular the actual 6DoF pose of the storage item 9, the master control device 15 develops pick-up instructions for transporting the storage item 9 to the production facility 2 and thus for organizing the material flow. The pick-up instructions can, for example, be transmitted to automatic vehicles 16, which autonomously transport the respective storage item 9 or the respective stack of storage items from the storage area 4 and / or the buffer storage 6 to the production facility 2 for further use. It is also possible for the production facility 2 to transmit a product pick-up requirement to the master control device 15.Knowing the free storage locations 8 in the storage area 4 and / or in the buffer storage 6 or in another buffer storage of the storage area 4, the control device 15 can send pick-up instructions to the automatic vehicles 16 so that they transport the products from the production plant 2 to the storage area 4, in particular to the buffer storage 6, in order to organize the flow of goods.
[0050] It is possible for the monitoring device 3 to have a plurality of such cameras 5, which monitor a plurality of buffer stores 6 in the storage area 4. Thus, it is conceivable for storage goods 9 to be stored in the buffer store 6 and for the finished products to be transported to a further storage area (not shown), wherein in the further storage area the position of the storage goods is also monitored via the monitoring device 3 and reported to the control device 15.
[0051] In a possible extension of the exemplary embodiment, the monitoring device 3, in particular the evaluation device 10, is designed to monitor restricted areas and / or delivery areas in the storage area 4 based on the camera image. In particular, it monitors whether these are kept clear—as required by a logistics regulation or set of rules. If necessary, corresponding data regarding the status of the restricted areas and / or delivery areas can be transmitted to the control device 15, so that if the restricted areas and / or delivery areas are blocked, appropriate countermeasures can also be taken automatically.
[0052] Furthermore, it is possible for the monitoring device 3, in particular the evaluation device 10, to be configured to detect persons in the storage area 4, in particular in the buffer storage 6. After the persons have been detected, they can be automatically anonymized in the camera image, for example, by masking or hiding them or by other measures. In this way, the camera image is anonymized and can also be permanently stored for documentation purposes without violating applicable data protection guidelines.
[0053] Figure 2 shows a top view of a teach-in camera image 17 as a camera image of the camera 5 of the storage area 4 and / or of the buffer storage 6 in duplicate. In the upper part, the storage area 4 or the buffer storage 6 is shown in a first analysis step, wherein the outer contours of the stored goods 9 are detected as objects by the neural network 12. In particular, the structure of the storage area 4 and / or the buffer storage 6 can be seen from the illustration, wherein storage locations 8, each containing a stack of the stored goods 9, are lined up along the lanes 7. In the teach-in camera image 17, all storage locations 8 are fully occupied. This teach-in camera image 17 or the data derived therefrom regarding the lanes 7, storage locations 8, and the target position of the stored goods 9 can be used as a reference for the further operation of the monitoring device 3.In the lower part of Figure 1, the learning camera image 17 is shown as a map of the depth image of the same area, wherein a degree of area overlap is shown for the storage locations 8, wherein the stored goods 9 in the target state have a degree of area overlap with the storage locations of 100%.
[0054] Figure 3 shows, in the same representation, a current camera image 18 of the same storage area 4 and / or buffer storage 6, with the storage locations 8 partially empty and partially overlapping. The evaluation device 10 and / or the evaluation module 14 can automatically detect this state of the storage area 4 and / or the buffer storage 6 and forward the resulting data, in particular regarding the position of the stored goods 9, to the control device 15.
[0055] Figure 4 shows, in a highly schematic representation, an evaluation of the camera image 18 of Figure 3, wherein the storage locations 8 are shown with the degree of area overlap of the stored goods 9, and wherein the degree of area overlap of the storage locations 8 is represented, in addition to the percentages, by shading the storage locations 8. The degree of area overlap can be used to determine the assignment of the stored goods 9 to the storage locations 8.
[0056] A further exemplary embodiment is described below: For forklift trucks as examples of the autonomous vehicles 16, there are forklift guidance systems and material flow guidance systems with fork-mounted camera systems for goods and pallet identification. These detect Euro pallets as stored goods 9 in shelf warehouses as buffer storage 6 from the side in a horizontal direction. The intralogistics arrangement 1 relates to a system which detects material warehouses as buffer storage 6, such as digital trolley supermarkets (lane supermarkets) with stacks of boxes as stored goods 9 on trolleys in intralogistics, from above and, if necessary, also from the side, in order to guide autonomous mobile platforms (automatic guided vehicles AGVs) as autonomous vehicles 16 for material transport. This allows stocks to be optimized and the buffer storage 6 to be reduced in size. RFID supermarkets can also be supplemented with such overview devices 10 as a basis for navigation control of AGVs.
[0057] The intralogistics arrangement 1 allows for visual localisation of material bins as storage goods 9 in intralogistics supermarkets as storage areas 4 and / or buffer storage 6 for fleet management and AGV guidance (English: Visual Localisation of Bins in Intralogistic Supermarkets for fleet management and AGV guidance):
[0058] For the flexible, automatic supply of production lines in manufacturing plants 2 in intralogistics, boxes / small load carriers (KLTs) or similar transport boxes, blisters, pallets, etc. (hereinafter referred to as boxes) stacked on trolleys (floor rollers) filled with material or workpieces as stored goods 9 are to be transported automatically between mobile transport platforms (Automated Guided Vehicle AGVs) and production lines (Point of Use PoU) via automated communication.
[0059] Overview cameras as cameras 5 above the supermarkets (sources / sinks) are intended to capture the material on the trolleys from above and / or from the side and transmit the position data as well as further information such as stack height, material type, etc. to the fleet management control system as control device 15 in order to efficiently manage the AGV fleet for material transport and to trigger required transport orders through pick-up instructions.
[0060] Five overview cameras above and beside the supermarkets detect the trolleys from above and from the side and determine their position and contours, as well as other information such as stack height, quantity of material, material type, etc. In addition, free delivery areas are identified and restricted zones (e.g., the required distance between trolleys and walls / doors) are monitored. This information (e.g., where which trolleys are located and in which lane?) is transmitted to the fleet management or AGV control system, which then transmits the transport orders to the AGVs or milk run train or the detailed logistics employee (point of use provider), etc., for removal or refilling of the individual supermarket lanes.
[0061] Possible benefits include:
[0062] Less accumulated inventory and reduced buffer space in the supermarket thanks to optimized AGV fleet management. Increased flexibility and higher reliability. Ratio through automation: Elimination of labor-intensive material tracking tasks such as scanning Kanban cards, RFID tags, Data Matrix codes, barcodes, etc. by employees. Error prevention through automation.
[0063] Overview cameras such as low-cost grayscale or color cameras or color depth cameras (time-of-flight, active stereo cameras with pattern projectors, stripe projectors, laser scanners, etc.) are mounted above and next to the supermarkets. Additionally or alternatively, such cameras can also be mounted on AGVs such as the ActiveShuttle, ActiveHandler, ActiveSupermarket, and ActiveAssist assembly stations from Bosch-Rexroth AG and Robert Bosch GmbH. Preferably, each color depth camera is already intrinsically calibrated, and the intrinsic calibration parameters are readable so that the color image can be mapped to the depth image. Otherwise, the cameras would have to be calibrated using common calibration procedures. Smart cameras with embedded image recognition on the chip can transmit the position data directly to the master control system without the need for an additional image recognition computer.Alternatively, many cameras without an evaluation chip can transmit the image data to just one or a few central high-performance image recognition computers with a graphics card for rapid evaluation in order to save hardware costs.
[0064] The transmission of image data, especially camera images, or directly the position data can be done via cable or wirelessly via WLAN or 5G to avoid the need to lay long camera cables, especially with data transfer protocols such as USB3.X, whose permissible cable lengths are limited.
[0065] About the image analysis process:
[0066] In a teach-in process, the trolley and / or storage goods positions (3D
[0067] Pose with stack height above the ground), for example, using image matching methods or a previously trained neural network, e.g., a Deep Convolutional Neural Network (DCNN), is detected in the camera image, the outer contour of each trolley and / or stored item 9 is extracted, and the camera image is thus divided into lanes and storage locations. The target positions and outer contours are saved as a reference (trolley / storage location grid as rectangles) - see Figure 2. In automatic mode, the actual positions and the overlap (fill level FRTC / area overlap level) to the corresponding reference rectangular cell in the trolley / storage location grid (rectangles) of the detected KLT stacks, particularly on the individual trolleys, are determined - see Figure 3.Using the intrinsic camera parameters and the distance measurement of the detected storage goods 9 in the depth image, the extracted crate contour can be projected onto the hall floor as a 3D world coordinate system. The plane of the empty hall floor in the 3D world coordinate system was determined and saved once in the depth image during teach-in mode. This 3D world coordinate system could also be defined, for example, by detecting a calibration plate in the color image mapped to the depth image or a 3D calibration body in the depth image. This allows the exact position of the trolleys and / or stored goods 9 in the 3D world coordinate system of the hall floor to be transmitted via the control device 15 to the AGVs or similar for precise removal.
[0068] Additionally, DCNN-based detection could automatically mask out people who may be visible in the color image to ensure person-based data protection. Since no images are saved in automatic mode, but only location data and similar information are output, data protection is guaranteed anyway.
[0069] Example process description:
[0070] Teach-in mode: DCNN-based segmentation of the KLT stacks as storage goods 9 on optional trolleys in the camera images of cameras 5: Extraction of the outer contours of the KLT / storage goods regions and saving the target reference positions. Automatic numbering in lane and position numbers (e.g.: 11 : Lane 1 , Pos 1) and saving as a reference for each camera 5. Generation of the crate dolly locations as a trolley / storage goods grid of the individually recognized cells (rectangles in Figure 2), e.g. with DCNN segmentation in the camera image of a fully stocked supermarket with trolley or storage goods 9 in the target position or drawing the encompassing rectangle around the supermarket area on the screen with the mouse.
[0071] Automatic mode: Image recording and evaluation of all cameras 5 and allocation of the position data to the corresponding tracks 7 and position numbers of the
[0072] Storage locations of the stored trolley / warehouse grid; transmission to the control system.
Claims
Claims 1. Monitoring device (3) for a storage area (4), comprising at least one camera (5) for recording a camera image (18), wherein the storage area or a section thereof with a plurality of tracks (7) with storage locations (8) can be depicted on the camera image (18), wherein a stored good (9) can be arranged and / or is arranged on the storage locations (8), comprising an evaluation device (10) for evaluating the camera image (18), wherein the evaluation device (10) is configured to determine a position of the stored good (9) relative to the storage locations (8).
2. Monitoring device (3) according to claim 1, characterized in that the evaluation device (10) has a learning module (13), wherein the learning module (13) is configured to learn the tracks (7) and the storage locations (8) in the storage area (5) on the basis of a learning camera image with filled storage locations (8).
3. Monitoring device (3) according to claim 1 or 2, characterized in that the evaluation device (10) is configured to determine a degree of area overlap of the stored goods (9) with the storage locations (8) and / or a center of gravity distance between a center of gravity of the stored goods and a center of gravity of a parking space in order to determine an assignment of the stored goods (9) relative to the storage locations (8).
4. Monitoring device (3) according to one of the preceding claims, characterized in that the evaluation device (10) has an evaluation module (14), wherein the evaluation module (14) is configured to determine a displacement of the stored goods (9) along the tracks (7) and / or a rotation of the stored goods (9).
5. Monitoring device (3) according to claim 4, characterized in that the evaluation module (14) is configured to determine a tilting of the stored goods (9).
6. Monitoring device (3) according to one of the preceding claims, characterized in that the evaluation module (14) is designed to detect a stack height of the stored goods (9).
7. Monitoring device (3) according to one of the preceding claims, characterized in that the learning module (13) and / or the evaluation module (14) comprises a neural network (12).
8. Monitoring device (3) according to claim 7, characterized in that the neural network (12) is configured as a DCNN and / or as a deep learning mask-region CNN model.
9. Monitoring device (3) according to one of the preceding claims, characterized in that the stored goods (9) are designed as or comprise a small load carrier.
10. Intralogistics arrangement (1) with a storage area (4), wherein the storage area (4) has a plurality of tracks (7) with storage locations (8), characterized by the monitoring device (3) according to one of the preceding claims, wherein the monitoring device (3) is configured to determine a position of the stored goods (9) relative to the storage locations (8).
11. Intralogistics arrangement (1) according to claim 10, characterized in that the monitoring device (3) is designed to monitor restricted areas and / or delivery areas in the storage area (4).
12. Intralogistics arrangement (1) according to one of the preceding claims 10 to 11 , characterized in that the monitoring device (3) is designed to recognize and anonymize persons in the storage area (4).
13. Intralogistics arrangement (1) according to one of claims 10 to 12, characterized in that it comprises a control device (15), wherein the control device (15) is configured to transmit collection instructions to automated vehicles (16) and / or collection personnel based on the position of the stored goods (9) relative to the storage locations (8).
14. Intralogistics arrangement (1) according to claim 13, characterized in that the control device (15) is configured to receive storage requirements from production facilities (2) and additionally to generate collection instructions based on the storage requirements.
15. Monitoring method, in particular with the monitoring device (3) according to one of claims 1 to 9 and / or the intralogistics arrangement (1) according to one of claims 10 to 14, wherein a camera image (18) of a storage area (4) with a plurality of tracks (7) with storage locations (8) is recorded and wherein the camera image (18) is evaluated to determine a position of the stored goods (9) relative to the storage locations (8).