Inspection of warehouses by drone

Abstract

The invention relates to an inspection system for inspecting warehouses in operation, said warehouses being fitted with a plurality of shelves separated by aisles, with a UAV, in particular a quadrocopter or a multicopter, which has an optical sensor, preferably a camera, wherein the UAV uses the optical sensor to record data of the supporting structure of the shelf of the shelf region to be inspected, and with a computer, preferably a tablet or laptop, which is configured to specify a route to the UAV by means of which the UAV autonomously flies over the shelf regions to be inspected, and is configured to receive the data from the UAV and display same to the inspector, within an inspection process of the shelf region to be inspected, for the purpose of documentation and / or commenting, wherein the results of the inspection are stored in a database.

Description

[0001] Inspection of warehouses using drones

[0002] The invention relates to a system for inspecting warehouses using UAVs (“unmanned aerial vehicles”) and a corresponding method.

[0003] Warehouses and their facilities must be inspected and monitored regularly, partly due to legal regulations. The associated shelf inspection is a time-consuming, regular process. Labor costs are rising, and customers demand reliable, high-quality service at a reasonable price with short inspection / downtimes. According to the European standard DIN EN 15635, automated and high-bay racking systems require a professional inspection every 12 months, covering at least 20% of the racking, on a rolling basis, so that the entire racking is inspected every 60 months.

[0004] Although automated and high-bay racking systems are less susceptible to damage at higher levels, they also require inspection, and the higher levels cannot be seen from the ground.

[0005] Therefore, industrial climbers are usually employed to inspect and document the systems during downtime.

[0006] However, this work is risky and requires long downtimes for the equipment.

[0007] US 2018 / 0120196 A1 describes the use of a drone with a detachable sensor module for examining structures using the sensor module attached to it.

[0008] DE 102014224 884 A1 discloses the monitoring of the position of pallets in the rack from the air using drones.

[0009] DE 102019219655 A1 describes the material measurement of products in the warehouse from the air using drones.

[0010] WO 2018 / 035482 A1 and WO 2020 / 002455 A1 describe the mapping and inventory of warehouses using drones from the air.

[0011] DE 20 2020 101 395 U1 generally describes the use of UAVs or drones to inspect warehouses.

[0012] In contrast, the object of the present invention is to provide a system and corresponding method with which the legal requirements for the inspection of storage racks can be met.

[0013] This object is achieved by the system recited in claim 1 and the method recited in claim 8. Advantageous embodiments emerge from the subclaims and the description.

[0014] According to the invention, it has been recognized that if an inspection system is used to inspect warehouses in operation which are equipped with a plurality of shelves separated by aisles, with a UAV, in particular a quadrocopter or multicopter, which has an optical sensor, preferably a camera, wherein the UAV records data of the shelf area to be inspected by means of the optical sensor, and with a computer, preferably a tablet or laptop, which is configured to specify a route for the UAV, by means of which the UAV autonomously flies over the shelf areas to be inspected, and is configured to receive the data from the UAV and to display it to the inspector for documentation and / or commenting within an inspection process of the shelf area to be inspected, wherein the results of the inspection are stored in a database, it is possibleOn the one hand, to minimize the time required and unproductive downtime of the equipment, and on the other hand, to comply with legal requirements for the inspection of storage racks. In some cases, the equipment can even continue to operate uninterrupted while the inspection is taking place.

[0015] In contrast to (goods) inventory (see above), shelf inspection in the sense of the present invention is understood to mean an examination and documentation of the supporting structures of the stored equipment, such as the shelves and their parts (see DIN EN 15635).

[0016] In other words, the invention provides a system and method for carrying out the inspection process using automated flight patterns according to a flight and inspection plan, locating the UAV, and connecting it to a software-supported inspection solution. The data from the inspection is stored in a database for digitization and report generation and, if necessary, used to create maintenance plans in addition to the actual inspection reports. The shelves, as defined above, are primarily inspected.

[0017] Preferably, the computer is configured to store metadata, in particular location of the shelf area during recording and / or error information of the inspected area, of the inspection with the results of the inspection in the database.

[0018] The computer can be usefully configured to allow text-based, audio-based, and / or graphical commentary. This allows the inspector to comment on the results of the visual inspection (using images and / or videos) and even forward them to a maintenance team with instructions if necessary.

[0019] The preferred method for an inspector to capture defect information for each shelf location would be an audio recording. An image and / or video is taken of each shelf location. All images and videos are tagged with metadata. The metadata preferably includes the location of the shelf area at the time of capture and defect information (whether pass or fail, including the type and severity of the defect).

[0020] The system can also be configured to allow simple inputs during the flight via the remote control computers, e.g., using voice commands for confirmation.

[0021] It is also conceivable to control the UAV via voice commands, e.g., "Take off drone, land drone, stop drone." The voice commands can be entered and transmitted via the remote control and / or the computer.

[0022] It is understood that the inspector evaluates the data from the UAV, while a pilot monitors the UAV or flies it if necessary. It is also possible for the inspector to also be the pilot. It is also clear that different inspectors and UAVs can be deployed simultaneously, especially in different spatially separated areas.

[0023] The data or image and / or video analysis can be carried out online (during operation) or offline (after the end of flight operations).

[0024] While it is preferred that the UAV flies autonomously, it may become necessary for the inspector to change the UAV's orientation and / or distance for a more accurate inspection. In this case, the computer can be configured to change the UAV's flight path according to the inspector's instructions, similar to a traditional remote control. The system therefore allows switching to manual mode if necessary.

[0025] It's also conceivable that the UAV adjusts its angle around the vertical axis (by yaw) during autonomous flight so that the uprights and rear areas of the shelf are also visible. Autonomous zooming and adjusting the lighting are also possible.

[0026] If the UAV is configured to perform localization based on sensor data from the UAV and, if applicable, a combination of the built-in optical system and, if applicable, external navigation aids and / or navigation systems or localization systems, safe, particularly autonomous, flight operations or flight paths can be enabled despite the adverse conditions in a rack warehouse with its many steel supports, etc.

[0027] The UAV can also be localized based on sensor data from the UAV (e.g., barometric sensor data) and a combination of the built-in vision system and external tracking systems such as beacons (e.g., ultrasound). This allows the vision system to detect crossbeams and support frames to determine a precise warehouse position. The correct level can be identified, for example, by counting the crossbeams (vertical). The horizontal pallet locations can, in turn, be identified, for example, by detecting the uprights and pallets (shelf supports).

[0028] For example, image mapping and mapping of a storage location can be achieved by extracting still images from a video recording. To do this, multiple images can be extracted from a video and stitched together, meaning they are combined in such a way that a single composite image of several overlapping images makes the entire storage location visible (without any overlaps).

[0029] To avoid the effort inspectors have to make, for example, when counting shelf positions, the UAV's localization can be integrated into the system.

[0030] To speed up the inspection, it may be useful to configure the UAV and / or computer to assist the inspector in detecting defects using image recognition and / or machine vision (AI). This can involve traditional image recognition with a database comparison of stored "optimal" images with the actual captured images. Additionally, a suitably configured AI could learn from the inspector's results and gradually provide more support, e.g., by graphically marking areas of the warehouse's load-bearing structures that deviate from the norm, such as the steel structure of the shelves.

[0031] In other words, the inspection results can be used as training data.

[0032] An interface can also be provided in the system that enables the comparison between user input and the defects detected by the AI ​​(based on a trained model running, for example, on a tablet).

[0033] An assistant class also helps the inspector to check the entries and improve the quality.

[0034] For example, vision recognition could be implemented in the system or computer and offline training procedures for machine vision could be implemented.

[0035] The vision system can thus actively support the inspector during the inspection process, e.g., by highlighting deviations.

[0036] The vision system can be based on image recognition algorithms and / or AI-supported.

[0037] For example, an AI-supported unit based on machine learning algorithms could be deployed. This AI would (later) support the inspector or pilot by providing indications of irregularities.

[0038] The training of the AI ​​then takes place offline, not during the inspection. After the inspection, the data containing the information or results (whether OK or not OK) and the area of ​​the defects marked are uploaded to a server for further training of the AI.

[0039] The constantly improved and updated KL model can be regularly updated on the computer, etc., in order to further improve the process.

[0040] For example, the system can incorporate image recognition into the computer and / or have an offline machine vision training process based on the inspection results. For example, a machine vision unit (Kl) can assist inspectors in detecting defects (e.g., missing bolts).

[0041] The invention also relates to a method for carrying out an inspection of warehouses in operation which are equipped with a plurality of shelves separated by aisles, in which data from shelf components to be inspected in shelf areas are recorded by a UAV using an optical sensor and a computer is used to specify a route for the UAV of the shelf areas to be inspected, which the UAV flies over autonomously in sequence, wherein the inspector receives data from the UAV on the computer and uses it for documentation and / or commenting within an inspection process of the shelf area to be inspected, wherein the results of the inspection are stored in a database.

[0042] The invention also includes the method by which technicians (inspectors) can interact with the UAV's automatic mode.

[0043] For example, to define the shelves or areas to be inspected, the inspector could use a computer (tablet) to scan a QR code attached to the shelf at the start of the inspection. The system also allows this information to be entered via user input if it is not already available or accessible. For example, the QR code is linked to all information about the shelving system (via a database). This allows tracking of which areas were last inspected and which shelf sections, for example, had damage. This information is also used to create the customer report. Alternatively, the QR code itself can contain the shelf information (e.g., number of storage locations, levels, and level height). This information is used to calculate the route and sent to the UAV.

[0044] The attached code represents a link to the database, which stores the specific rack information, such as the number of shelves, height and width of the shelves, uprights, trusses, etc., which is used to calculate the flight pattern. Waypoints for the UAV's autonomous flight are calculated from the rack data.

[0045] This data can also be loaded and saved in advance into the app if, for example, the customer does not have an internet connection.

[0046] If the shelf information is not stored in the database, this shelf data can also be entered manually for flight calculation. Manual entry can be made via the user interface. The user interface can be provided, for example, in the remote control. This can be designed as an app (program).

[0047] If shelf data is not known, a mapping flight can be carried out in advance to measure the shelf structures.

[0048] The UAV takes off from a starting position (e.g., docking station), flies autonomously to the first inspection position (e.g., top left shelf), and waits for user input from the inspector. If the inspector confirms "OK," the UAV can proceed with data collection (image capture), and the inspector can proceed with collecting user input on possible defects in the shelf structures, thus performing the inspection for the respective area and creating the documentation. The inspector then releases the UAV, and it can fly autonomously to the next shelf position.

[0049] If necessary, the inspector can manually control the UAV via the computer to take a closer look at the current shelf location (e.g., changing the camera angle or flying deeper into the shelf, etc.).

[0050] The process is repeated until the UAV has recorded the entire route or all the rack locations in the flight plan one after the other and the inspector has inspected, documented, or commented on them.

[0051] The data or results are stored in a database either continuously or after completion in order to digitize them and to create reports.

[0052] An unmanned aerial vehicle (UAV) is an aircraft that can be operated and navigated autonomously, without a crew on board, either by computer or remotely from the ground. Multicopters or quadrocopters, also known as drones, are particularly suitable for this purpose.

[0053] This makes it possible to inspect the facilities in a short time, despite their great height and size, without climbing equipment or a stage, etc.

[0054] The UAV may be a commercially available UAV capable of performing the functions described herein, including operating indoors. The UAV may be capable of autonomous flight, be sensing of its surroundings, and be programmable. The UAV may be coupled with one or more sensors or devices to assist in performing the functions described herein. The sensors or devices may include, but are not limited to, an optical code reader, an image capture device, a motion sensor, a range sensor, an infrared sensor, a thermal sensor, a sonar sensor, an ultrasonic sensor, and the like. The UAV may also include a processing device or onboard computing device and memory for storing instructions or data, as well as a communications capability for communicating with a separate computing device (e.g., the computer as a sort of command center) or other UAVs.

[0055] In other words, UAVs with cameras are deployed on the ground, which create images and / or video recordings either manually or remotely controlled by a computer or autonomously while flying through the facility.

[0056] The UAV can initially be controlled manually (e.g., by sight or via FPV, first-person view, possibly using a screen and / or data glasses). The trained computer can then take over control, e.g., by route tracking with obstacle detection supported by AI. The drone can also perform an autonomous inspection under computer control.

[0057] Preferably, a UAV is used to perform a visual inspection of the shelving areas. IR sensors for thermal analysis are also conceivable. Ultrasonic sensors for material testing and distance measurement are also possible.

[0058] Other sensors can also be used.

[0059] In addition to the actual shelf inspection, the UAV can also fly through the warehouse or shelf at regular intervals to detect any defects in the stored goods (if necessary with the help of the vision system, with or without AI support), such as broken wood on pallets, shifted pallets, or a film that is hanging down and could cause damage during automated operation.

[0060] In particular, such a UAV can constantly fly around the facility and thus quickly detect problems or potential dangers.

[0061] In addition to the camera, UAVs can also be equipped with spotlights and a wireless transmission system for the camera data. This allows images to be taken in the darkness of warehouses and simultaneously transmitted directly to a control center. Alternatively, or additionally, the images and data can be buffered locally on the drone, which can also serve as a redundancy.

[0062] It is also possible to control the UAV remotely (via an external data connection), allowing service technicians, for example, to centrally control the UAV located on-site at the customer's location from a service center. Technicians also receive a live video stream from the UAV. This allows the inspection process to be carried out entirely remotely, which may be advantageous in situations where time zones differ.

[0063] The invention is particularly suitable for high-bay warehouses, which can have considerable heights. The term "rack" as used herein refers to a rack or racking system, a large rack or racking system for use in warehouses, or other storage units suitable for storing and organizing items or storage bins in warehouses or distribution centers.

[0064] An optical, machine-readable label can be attached to each shelf. The optical, machine-readable label represents a unique identifier for each shelf. It can also contain information about the shelf, such as location, orientation, length, height, number of shelves, shelf size, etc. This data can also be accessed in the connected database via the identifier. Manual entry or modification is also possible.

[0065] The term "optically machine-readable label" or "optically machine-readable identifier" refers to a code printed on a label that can be read by an optical code reader or device. The code can be, among other things, a barcode, 2D or matrix barcode, quick response code (QR code), and the like.

[0066] The autonomous UAV is capable of navigating and flying a route independently without further instructions from a user. The UAV can be given a route for navigating through the warehouse or shelves. The route can be optimized based on obstacles and human traffic within the facility. Preferably, the system includes a route planning function that calculates an optimized flight path based on the inspection plan and transmits it to the UAV. The route planning function can be implemented as a computer module.

[0067] The autonomous UAV may be capable of localization. The autonomous UAV may also be capable of detecting various objects within the facility. For example, the UAV may detect shelves, conveyors, storage units, storage boxes, storage containers, equipment, personnel, and the like. The UAV may also be capable of detecting waypoint markers along its route. The waypoint markers (e.g., beacons) may be placed at a predetermined distance from each facility. The UAV may also be capable of detecting docking or charging stations and landmarks.

[0068] Although the autonomous UAV can be instructed to navigate an optimized route, the UAV can also be capable of making decisions independently. The UAV can be equipped with artificial intelligence that makes autonomous decisions to detect and avoid dynamic and static obstacles. The UAV can also dynamically optimize the predefined route based on the data collected during route navigation. The UAV can be programmed to dynamically tolerate certain exceptions during its autonomous navigation through the shelves. This allows, for example, a stacker crane to be avoided during operation.

[0069] What is important is that the localization is included in the metadata and that all rule areas to be inspected are processed.

[0070] To support the autonomous operation of the UAV, the UAV can be configured to autonomously navigate to a docking station when needed. A docking station can be referred to as a base station, home base, charging station, or similar. The UAV can autonomously navigate to a docking station to recharge batteries. If the UAV requires maintenance, it can also independently travel to the docking station. The UAV can autonomously determine that it requires maintenance based on data collected by the UAV. Maintenance of the UAV can relate to the batteries, sensors, mechanical or electrical components, firmware, and the like.

[0071] The autonomous UAV can communicate with the computer that acts as the command center.

[0072] The computer can communicate with multiple UAVs that navigate autonomously through the shelves or are in standby mode in a docking station.

[0073] Various variants for localization and navigation in the warehouse are conceivable.

[0074] In one embodiment, an ultra-wideband (UWB) system can be used by the UAV for positioning or localization. In such an embodiment, beacons for transmitting a signal can be arranged throughout the facility or at specific landmarks, and a reader or receiver can be connected or integrated with the UAV. Alternatively, the beacon can be coupled to the UAV for transmitting a signal, while readers or receivers can be arranged throughout the facility or at specific landmarks.

[0075] In another embodiment, the UAV may use an intelligent lighting system to detect landmarks and static objects within the indoor facility. In this embodiment, a light, such as a light-emitting diode (LED), may be placed at specific landmarks and static objects. The light may have a specific color or emit a specific wavelength that, when perceived by the UAV, enables identification of the UAV's location (by the UAV or the computer). In another embodiment, a differential global positioning system (DGPS) may be used by the UAV for localization and navigation. DGPS is an extension of the Global Positioning System that provides improved positioning accuracy, from the nominal GPS accuracy of 15 meters to within a few centimeters in certain implementations.

[0076] In another embodiment, precisely positioned markers are used to determine the location of the UAV. Precise markers can include, among other things, static storage units, static equipment, and static obstacles at specific locations within the facility. Their positions can be measured and stored as landmarks.

[0077] In another embodiment, sonar systems can be used to detect static and dynamic objects or obstacles. The UAV can be equipped with a sonar system that enables object detection and also detects objects through walls. The data acquired by the sonar system can be used by the UAV or the computer to dynamically reroute the UAV or cause the UAV to dynamically avoid obstacles.

[0078] In another embodiment, the UAV is equipped with thermal scanners to enable the UAV to detect people, devices (which generate heat when idle or in operation mode), or other heat-generating objects.

[0079] In another embodiment, LIDAR systems and methods are used by the UAV to survey the facility. LIDAR can be used to create an overview or map of the facility's interior by measuring distances to objects by illuminating the object with light energy (e.g., laser light). The LIDAR system can be coupled to the UAV, which is tasked with surveying and scanning the interior to create a map of the facility's interior, indicating the position of static objects such as storage units, equipment, lighting fixtures, wires, and the like. Scanning with a LIDAR system can create a 3D point cloud representing the exterior surface of objects within the facility. The 3D point cloud can be analyzed using a computer to create the map. Using these methods, LIDAR scanning can also detect small obstacles, such as wires.The created map can be used by the computer or the UAV to create the route or navigate. Alternatively, the UAV can be configured to actively use the LIDAR system in flight to dynamically detect obstacles during navigation and determine the distance to the obstacle.

[0080] In another embodiment, suitable 3D mapping technologies are used to create a map of the warehouse's interior. For example, rapid 3D mapping may be used, which utilizes stereo photography technology to create a 3D map using imagery captured by a UAV.

[0081] In one embodiment, one or more parts or components of the system can communicate with each other and exchange data over a network. The network can be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular telephone network, a wireless network, a WiFi network, a WiMax network, any other type of network, or a combination of two or more such networks.

[0082] The computer may include, but is not limited to, workstations, computers, general-purpose computers, Internet devices, handheld devices, wireless devices, portable devices, wearable computers, cellular or mobile phones, portable digital assistants (PDAs), smart phones, tablets, ultrabooks, netbooks, laptops, desktops, multiprocessor systems, network PCs, minicomputers, and the like.

[0083] The computer can be connected to the network via a wired or wireless connection. The computer can contain one or more applications, such as, but not limited to, the inspection documentation module described here.

[0084] The database component or module can be hosted locally on the computer, centrally on a connected server, in the cloud, or synchronized across several of the aforementioned platforms. The database component or module can be linked to a reporting module to generate legally required reports. A maintenance module can also be integrated, which automatically generates and sends maintenance instructions from the inspection results in the database.

[0085] The inspection system can include an inspection module in which the inspector documents the inspection results. The inspection module can exchange data with other modules and the UAV via interfaces.

[0086] An interface can exist to a database module in which the inspection specifications (customer, shelf, location, equipment, etc.) and the inspection results are stored, along with the metadata, images, videos, notes, comments, etc.

[0087] A further interface can be to a Kl module in which a neural network is provided as a Kl model to support the inspection process.

[0088] A further interface can be provided to a learning module that trains the AI ​​module or its AI model. This module is preferably fed with the results from the inspection module. The data from the results, etc., is in turn temporarily stored and exchanged via another database module, the so-called experience database.

[0089] The experience database module thus has an interface to the inspection module and learning module.

[0090] A further interface can be provided to a remote control module, which enables remote control of the UAV and also exchanges data with the inspection module and the UAV. This allows the UAV to be controlled directly from the inspection module or to exchange data with it and / or via the remote control module.

[0091] For this purpose, the computer can be equipped with appropriate modules and interfaces to implement the aforementioned functionalities. It goes without saying that, as mentioned above, the various modules and functions can be distributed geographically, with some being hosted locally, some in the cloud, and / or some being hosted remotely.

[0092] The described system and method generates a large amount of image and video data. Therefore, it is important, among other things, that the system is equipped with a software module that subsequently checks the video and / or image data to determine whether it contains faces. If so, the faces are pixelated to protect data privacy.

[0093] In summary, the UAV can be used to easily survey shelf areas of the facility that would otherwise be impossible or difficult to inspect, or would require extreme effort.

[0094] Further details of the invention emerge from the following description of embodiments with reference to the drawing, in which

[0095] Fig. 1 is a schematic perspective view of an automated inspection of a shelf area including a row of shelves and a storage and retrieval machine using a quadrocopter;

[0096] Fig. 2 is a schematic perspective enlarged detail of Figure 1 in the area of ​​an inspection station;

[0097] Fig. 3 is a side view of Figure 1 with flight route;

[0098] Fig. 4 is a schematic perspective enlarged detail of Figure 1 in the area of ​​the storage and retrieval machine with flight route;

[0099] Fig. 5 is a schematic view of a variant of the invention for remote inspection, and

[0100] Fig. 6 shows a schematic view of a modular structure of an inspection system according to the invention.

[0101] Figure 1 shows a row of shelves in a warehouse, designated as a whole by 1, including a storage and retrieval machine 2, which is deployed in a not-shown aisle of a high-bay warehouse. Row 1 of shelves is shown as a section of the larger warehouse, which includes several rows of shelves 1 and aisles, as well as storage and retrieval machines and conveyor technology, etc.

[0102] The storage and retrieval machine 2 runs on a rail 3 and includes a load-handling device 4 that can be moved vertically on a mast 5. A guide 6 is also arranged at the upper end of the mast 5. A control and supply unit 7 is arranged at the base of the mast 5.

[0103] Since the rack row 1 and the masts 5 of such storage and retrieval machines 2 can be extremely high, the supporting structures of the rack and the retrieval machines are inspected using an inspection system 13 comprising a quadrocopter 10 or drone, which is automatically controlled by a central controller from a landing pad 8 to fly over the rack row 1 and the storage and retrieval machine 2, and in particular its mast 5. In this case, the drone 10 is monitored by a pilot 14 using a remote control 15 or, if necessary, takes over control.

[0104] The drone 10 takes pictures and videos using a camera 11 that is carried along and records areas illuminated by a light 12.

[0105] It goes without saying that the other components of the storage and retrieval machine 2 or the contents of the shelf 1 itself can also be flown over and examined or recorded.

[0106] The control system is integrated into a tablet computer 9 on which the inspection system 13 is installed (see Figure 6).

[0107] The images and videos are transmitted wirelessly from the drone 10 to the tablet computer 9 so that they can be incorporated into the inspection system 13. The tablet computer 9 is configured to provide the drone 10 with a route 16 (see Figures 3 and 4), along which the drone 10 autonomously flies over the shelf areas or installations to be inspected.

[0108] The drone 10 navigates using localization based on sensor data and a combination of the built-in optical system (camera).

[0109] The drone's flight path can also be changed as instructed by the inspector or pilot.

[0110] The tablet computer 9 is configured to receive the data from the drone 10 and to display it to the inspector for documentation and / or commenting during an inspection process of the shelf area to be inspected, wherein the results of the inspection are stored in a database 17.

[0111] Computer 9 is configured to allow text-based, audio-based, and graphical commentary on the respective inspection area. The inspector can write in a text field, record a voice message using the microphone on tablet computer 9, or "draw" on an image or video.

[0112] The tablet computer 9 is also configured to store metadata, in particular the location of the shelf area during the recording and / or error information, of the inspection with the results of the inspection in the database 17.

[0113] The inspection system 13 includes an inspection module 18, in which the inspector documents the results of the inspection of the supporting structures of the rack and the operating devices. The inspection module 18 is capable of exchanging data with other modules and also with the drone 10 via interfaces. In addition to route calculation, the inspection module 18 also allows the actual data evaluation, data commenting, and documentation of the inspection. In addition, the inspector can also control the drone 10 if necessary to take a closer look at the area to be inspected. The drone 10 flies autonomously to the next inspection area as soon as the inspector gives permission after the evaluation is completed.

[0114] An interface exists to the database module 17, in which the inspection specifications (customer, shelf, location, equipment, etc.) and the inspection results are stored, together with the metadata, images, videos and notes or comments, etc. as mentioned above.

[0115] A further interface exists to a Kl module 19, in which a neural network is provided as a Kl model to support the inspection process.

[0116] Another interface exists to a learning module 20, which trains the AI ​​module 19 or its AI model. This module 20 is fed with the results from the inspection module 18. The data of the results, etc., are in turn temporarily stored and exchanged via another database module 21, the so-called experience database.

[0117] The experience database module thus has an interface to the inspection module 18 and the learning module 20.

[0118] A further interface exists to a remote control module 22, which enables remote control of the drone 10 and also exchanges data with the inspection module 18 and the drone 10. Thus, the drone 10 can be controlled directly from the inspection module 18 or exchange data with it and / or via the remote control module 22. Control can be carried out directly via the tablet computer 9 and / or the remote control 15.

[0119] The tablet computer 9 can be coupled to an FPV headset 14A of the pilot 14, so that the pilot can make all displays, inputs, etc., if necessary, via the FPV headset 14A and thus simultaneously act as an inspector.

[0120] Although the drone 10 must always be located locally in the warehouse, the remaining components such as the controller 9 or computer 9 and the inspection system 13 can be located remotely, i.e., connected externally via a suitable data connection 300. This makes it possible to centrally inspect several warehouses 100A, B, C, D at different locations from an inspection service center 200 (see Figure 5).

[0121] The process of an inspection can be as follows:

[0122] Before carrying out an inspection in the warehouse in the company, a

[0123] Specification of the areas of the shelving structures to be inspected, e.g., through a statutory inspection interval for the supporting structures in accordance with DIN EN 15635.

[0124] A route for drone 10 through the shelves etc. is then calculated from this.

[0125] For this purpose, the inspector can initiate the route calculation by specifying the shelves and storage and retrieval machines to be visited, which he can do by scanning applied QR codes with the tablet computer 9 within the inspection module 18.

[0126] The route function within the inspection module 18 can then calculate a route for the drone 10 and transmit it wirelessly to the drone, if necessary after approval by the inspector. The route, between start A and end B, therefore contains all areas to be inspected and intermediate stops or waypoints at which the inspector must approve the otherwise autonomous flight of the drone 10. The drone 10 navigates independently using localization.

[0127] The drone 10 flies this route autonomously and takes pictures, videos, etc. of the shelves (uprights, cross beams, support plates, supports, frames, diagonals, reinforcements, spacers, stops, push-through guards, etc.) in the areas and sends this data to the tablet computer 9 or the inspection system 13 on it.

[0128] The inspector evaluates the data and comments on it, if necessary, within the inspection module 18 and the resulting inspection results are stored and documented in the database 17.

[0129] The inspector is supported in evaluating the data. For this purpose, any deviations detected in the inspection module while viewing images, videos, or other sensor data using a vision system and / or an AI module are highlighted for the inspector. These functions run locally on the tablet computer 9.

[0130] This process is repeated until all areas to be examined have been flown over, inspected, and documented. A corresponding report can then be created and, if necessary, a maintenance order. The data resulting from the results is stored in the event database.

[0131] 21 and provided to the learning module 20 in order to continuously improve the Kl model.

Claims

Patent claims 1. Inspection system for inspecting warehouses in operation, which are equipped with a plurality of shelves separated by aisles, with a UAV, in particular a quadrocopter or multicopter, which has an optical sensor, preferably a camera, wherein the UAV uses the optical sensor to record data of the supporting structures of the shelf of the shelf areas to be inspected, and with a computer, preferably tablet or laptop, which is configured to specify a route for the UAV, by means of which the UAV autonomously flies over the shelf areas to be inspected, and is configured to receive the data from the UAV and to display it to the inspector for documentation and / or commentary within an inspection process of the shelf area to be inspected, wherein the results of the inspection are stored in a database.

2. Inspection system according to claim 1, characterized in that the computer is configured to store metadata, in particular location of the shelf area during recording and / or error information, of the inspection with the results of the inspection in the database.

3. Inspection system according to claim 1 or 2, characterized in that the computer is configured to allow text-based and / or audio-based and / or graphical commenting.

4. Inspection system according to one of the preceding claims, characterized in that the computer is configured to change the flight path of the UAV as specified by the inspector.

5. Inspection system according to one of the preceding claims, characterized in that the UAV is configured to perform localization on the basis of sensor data of the UAV and, if applicable, a combination of the built-in optical system and, if applicable, external navigation aids and / or navigation systems or localization systems.

6. Inspection system according to one of the preceding claims, characterized in that the UAV and / or the computer is arranged to To assist inspectors in detecting defects using image recognition and / or machine vision (AI).

7. Inspection system according to claim 6, characterized in that the system provides image recognition in the computer and / or has an offline training method for machine vision based on the inspection results.

8. A method for carrying out an inspection of warehouses in operation which are equipped with a plurality of shelves separated by aisles, in which data from the supporting structures of the shelf of the shelf areas to be inspected are recorded by a UAV using an optical sensor and a computer is used to specify a route for the UAV of the shelf areas to be inspected, which route the UAV flies over autonomously in sequence, the inspector receiving data from the UAV on the computer and using it for documentation and / or commenting on the shelf area to be inspected within an inspection process, the results of the inspection being stored in a database.

Images