Apparatus, method and system for operating industrial vehicle in warehols

Abstract

The invention relates to a device, a method and a system for operating industrial vehicles in a warehouse. In particular to a data processing device or a control unit for operating an industrial vehicle for transporting a plurality of goods objects in a warehouse. The data processing device includes an interface configured to acquire a plurality of sensor data of the industrial vehicle while the industrial vehicle is moving in the warehouse, where the plurality of sensor data includes a plurality of thermal imaging data of the industrial vehicle environment. Further, the data processing device includes a processor device configured to create or update a temperature map of the warehouse based on the plurality of thermal imaging data, where the temperature map indicates respective temperature values for a plurality of points of the warehouse. The data processing device or control unit may be part of the industrial vehicle or a separate server.

Description

Technical Field

[0001] The present invention relates to an apparatus, method and system for operating industrial vehicles, particularly forklifts, in a warehouse. Background Technology

[0002] Loading devices, such as mesh crates or pallets, especially European pallets, are typically used for transporting and storing products, goods, and materials. To handle such loading devices, such as in-plant logistics (i.e., the flow of materials within a company, such as in a warehouse), industrial vehicles, such as forklifts, are used. Warehouses are often very dynamic environments where the operation of moving industrial vehicles is typically manual, semi-automatic, or automatic.

[0003] Modern industrial vehicles are typically equipped with sensor units, such as imaging cameras, which are configured to capture the environment of the respective industrial vehicle. This allows for the acquisition of information about the environment (e.g., a warehouse) to manage the warehouse and control the industrial vehicle, particularly in automated or semi-automated operations, based on the captured sensor data.

[0004] For some time now, energy management, as part of building management, has become increasingly important for warehouses. Typically, the goal of energy management is to minimize energy use (e.g., heating or cooling), related emissions, and energy consumption within the warehouse. Summary of the Invention

[0005] In this context, the objective of the present invention is to provide an improved apparatus, method, and system for operating industrial vehicles in a warehouse.

[0006] According to a first aspect, the task is solved by a data processing apparatus for operating an industrial vehicle in a warehouse, the industrial vehicle being used to transport multiple goods objects within the warehouse. The data processing apparatus according to the first aspect includes an interface configured to acquire multiple sensor data of the industrial vehicle as it moves within the warehouse, wherein the multiple sensor data includes multiple thermal imaging data of the industrial vehicle's environment. Furthermore, the data processing apparatus according to the first aspect includes a processor configured to create and / or update a temperature map of the warehouse based on the multiple thermal imaging data, wherein the temperature map indicates respective temperature values ​​for multiple points within the warehouse.

[0007] According to one embodiment, the plurality of sensor data of the industrial vehicle also includes plurality of location data of the industrial vehicle, wherein the processor device is configured to create or update a temperature map of the warehouse based on the plurality of thermal imaging data and the plurality of location data.

[0008] In one embodiment, the processor device is further configured to generate control signals for the industrial vehicle based on the plurality of location data of the industrial vehicle, wherein the control signals are configured to move (i.e. control) the position of the industrial vehicle in the warehouse, for example, to perform a transportation task.

[0009] According to one embodiment, the processor device is further configured to identify cargo objects among the plurality of cargo objects based on sensor data using an object recognition mechanism, and to determine the temperature of each identified cargo object based on a temperature map.

[0010] In one embodiment, the object recognition mechanism includes an artificial neural network configured to identify cargo objects among the plurality of cargo objects and determine the temperature of each identified cargo object based on a temperature map.

[0011] According to one embodiment, the processor device is further configured to use a monitoring mechanism based on a temperature map to monitor whether the temperature at one or more of the plurality of points in the warehouse is within their respective predefined temperature ranges, such as below a maximum temperature threshold.

[0012] In one embodiment, the monitoring mechanism includes an artificial neural network configured to monitor whether the temperature at one or more of the plurality of points in the warehouse is within a predefined temperature range.

[0013] According to the second aspect, the aforementioned task is accomplished by an industrial vehicle, particularly a forklift, used for transporting multiple goods objects in a warehouse. This industrial vehicle includes a data processing device according to the first aspect and a thermal imaging acquisition device, particularly a thermal imaging camera, wherein the thermal imaging acquisition device is configured to acquire the multiple sensor data containing thermal imaging data of the industrial vehicle environment.

[0014] In one embodiment, the industrial vehicle according to the second aspect may further include a drive unit configured to move the industrial vehicle within a warehouse based on control signals.

[0015] According to one embodiment, the industrial vehicle may also include a load receiving device, particularly a fork, wherein the load receiving device, particularly the fork, is configured to receive the cargo object based on movement relative to a cargo object (e.g., a loading device) based on a control signal.

[0016] According to the third aspect, the aforementioned task is solved by a system for operating multiple industrial vehicles in a warehouse. The system according to the third aspect includes multiple industrial vehicles according to the second aspect and a data processing device according to the first aspect for operating the multiple industrial vehicles in the warehouse.

[0017] According to the fourth aspect, the aforementioned task is accomplished by a method for operating industrial vehicles to transport multiple goods objects in a warehouse. The method according to the fourth aspect includes the following steps:

[0018] Acquire multiple sensor data from the industrial vehicle as it moves through the warehouse, wherein the multiple sensor data includes multiple thermal imaging data of the industrial vehicle's environment; and

[0019] A temperature map of the warehouse is created and / or updated based on the multiple thermal imaging data, wherein the temperature map indicates the respective temperature values ​​for multiple points in the warehouse.

[0020] The method according to the fourth aspect can be executed by the data processing apparatus according to the first aspect. Therefore, further embodiments of the method according to the fourth aspect are derived from the embodiments of the data processing apparatus according to the first aspect described above and below. Attached Figure Description

[0021] Further advantages and details of the invention will be described in more detail by way of exemplary embodiments schematically shown in the accompanying drawings. Wherein:

[0022] Figure 1 A schematic diagram of an industrial vehicle for transporting goods in a warehouse, according to one embodiment, is shown.

[0023] Figure 2 A schematic diagram of the present invention system according to one embodiment is shown, the system having multiple industrial vehicles and a central data processing unit; and

[0024] Figure 3 A flowchart illustrating the method steps of an industrial vehicle for transporting goods in a warehouse, according to one embodiment. Detailed Implementation

[0025] Figure 1 This diagram illustrates an industrial vehicle 120a transporting goods 140 in an industrial environment (particularly a warehouse) according to one embodiment. The industrial vehicle 120a may specifically refer to a forklift 120a that operates intermittently autonomously, semi-autonomously, and / or manually (i.e., driven by an operator). The goods 140 may be, for example, goods 143, such as packaging boxes 143, arranged on their respective loading devices 141. The loading device 141 may be, for example, a pallet 141, particularly a European pallet 141 or a mesh box 141.

[0026] like Figure 1As shown, the industrial vehicle 120a may include a load receiving device in the form of a pair of forks 124a,b, configured to insert into corresponding recesses (particularly bag slots 141a,b) at the end face of the loading device 141 to receive the loading device 141 and the goods 143 disposed thereon. According to other embodiments, the load receiving device may also be formed as a plug for receiving film rolls or wire rolls, or as a hook-type loading device (e.g., similar to a garbage truck for receiving trash cans), or as a ball clamp or coil clamp for receiving paper rolls.

[0027] Figure 1 The industrial vehicle 120a shown also includes a drive unit 121, such as at least one motor 121, particularly a battery-powered electric motor 121, wherein the drive unit 121 is configured to move the industrial vehicle 120a and a pair of forks 124a,b relative to the cargo object 140, for example, changing the orientation and / or distance between the industrial vehicle 120a and the cargo object 140 (particularly the loading device 141), and / or raising or lowering the pair of forks 124a,b. For this purpose, as... Figure 1 As shown, the drive unit 121 can be suitably connected to the wheels 122a-d and / or a pair of forks 124a,b of the industrial vehicle 120a. The industrial vehicle 120a may also include a display and / or operation panel 125 for displaying information and / or operating the industrial vehicle 120a.

[0028] According to the present invention, an industrial vehicle 120a includes a sensor unit 130a comprising a thermal imaging acquisition unit 130a configured to acquire a large number of thermal images of the environment of the industrial vehicle 120a as it moves through a warehouse. Preferably, the thermal imaging acquisition unit 130a includes an infrared camera 130a. In addition to the thermal imaging acquisition unit, the sensor unit 130a may also include other sensors, particularly lidar sensors 130a and / or radar sensors 130a for acquiring information about the environment of the industrial vehicle 120a, such as for acquiring position data of the industrial vehicle 120a, i.e., data capable of determining the relative and / or absolute position of the industrial vehicle 120a within the warehouse.

[0029] like Figure 1As shown, sensor unit 130a (in the form of thermal imaging acquisition unit 130a) is preferably mounted on industrial vehicle 120a such that the field of view 131a of thermal imaging acquisition unit 130a is substantially along the forward direction A of industrial vehicle 120a. Preferably, thermal imaging acquisition unit 130a can be mounted in a plane of symmetry between the two forks 124a,b. In addition to thermal imaging acquisition unit 130a having a field of view along the forward direction A of industrial vehicle 120a, industrial vehicle 120a may also include other sensor units, such as sensor units having a field of view along the reverse direction of industrial vehicle 120a, and / or sensor units having a field of view perpendicular to the forward direction A of industrial vehicle 120a.

[0030] According to the invention, the industrial vehicle 120a also includes a data processing device 123 (also referred to herein as a control unit 123), which may, for example, include one or more processors or microcontrollers 123a and suitable software, and is configured to control the industrial vehicle 120a at least partially automatically. Figure 1 As shown, the data processing device or control unit 123 also includes a communication interface 123b (particularly for acquiring thermal imaging data from the thermal imaging acquisition unit 130a) and a (particularly non-volatile) memory 123c. The memory 123c can be configured to store data and executable program code that, when executed by the processor 123a of the data processing device or control unit 123, causes the processor 123a to perform the functions, operations, and methods described below.

[0031] Figure 2 A schematic diagram of a system 100 according to the invention is shown for operating industrial vehicles 120a and one or more other industrial vehicles 120b, which can be operated intermittently in an automated, semi-automated and / or manual manner to perform the task of transporting goods objects 140 in a warehouse.

[0032] In addition to including multiple industrial vehicles 120a,b, the system 100 also includes a central data processing unit 110, such as a server 110, configured to communicate with each of the multiple industrial vehicles 120a,b, for example, to assign transportation tasks to each of the multiple industrial vehicles 120a,b to transport goods 140 in a warehouse. For this purpose, industrial vehicle 120a includes, for example, a communication interface 126 configured to communicate with a corresponding communication interface 113 of the central data processing unit 110, for example, via a wireless communication network 150 (e.g., a WLAN network or a motion communication network). Through this wireless communication network 150, industrial vehicle 120a and / or the central data processing unit 110 can also communicate with another external sensor unit 130b, such as another thermal imaging camera 130b installed in a warehouse aisle. Figure 2The central data processing unit 110 shown can be an industrial PC 110 or a cloud server, especially an edge cloud server 110.

[0033] like Figure 2 As shown, the central data processing unit 110 includes a communication interface 113, one or more processors 111, and (particularly non-volatile) memory 115. The memory 115 can be configured to store data and executable program code that, when executed by the processor 111 of the central data processing unit 110, causes the processor 111 to perform the functions, operations, and methods described below.

[0034] According to the present invention, a data processing apparatus (with Figure 1 The control unit 123 of the industrial vehicle 120a is in the form of or Figure 2 The central data processing unit (in the form of 110) is configured to acquire multiple sensor data of the industrial vehicle 120a while it is moving in the warehouse via respective interfaces 123b or 113, wherein the multiple sensor data includes multiple thermal imaging data of the environment of the industrial vehicle 120a. Furthermore, each processor unit (i.e. Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2 The processor 111 of the central data processing unit 110 is configured to create and / or update a temperature map of the warehouse based on the plurality of thermal imaging data, wherein the temperature map indicates the respective temperature values ​​(i.e., temperatures) for multiple points in the warehouse. In one embodiment, the temperature map is a three-dimensional temperature map that indicates the respective temperature values ​​for multiple points in the three-dimensional space of the warehouse.

[0035] As previously mentioned, the multiple sensor data of the industrial vehicle 120a may include, in addition to thermal imaging data, multiple position data of the industrial vehicle 120a. In this case, the processor device (i.e. Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2 The processor 111 of the central data processing unit 110 can be configured to create and / or update a temperature map of the warehouse based on the plurality of thermal imaging data and the plurality of location data of the industrial vehicle 120a. In one embodiment, the processor device (i.e. Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2 The processor 111 of the central data processing unit 110 can also be configured to generate control signals for the industrial vehicle 120a based on the plurality of location data of the industrial vehicle 120a, wherein the control signals are configured to cause the industrial vehicle 120a to move in the warehouse.

[0036] As mentioned earlier, in order to acquire position data, the sensor unit 130a of the industrial vehicle 120a may include, for example, a lidar sensor (especially a 3D lidar sensor) in addition to the thermal imaging camera 130a. Through proper calibration of the thermal imaging camera and lidar sensor, and the fusion of thermal imaging data and position data, the processor device (i.e....) Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2 The processor 111 of the central data processing unit 110 can create a temperature map in the form of a 3D point cloud with temperature values.

[0037] In other words, temperature values ​​acquired by a thermal imaging camera are assigned to a point in three-dimensional space, thereby generating a high-precision three-dimensional temperature map. The advantage of the movement of the industrial vehicle 120a and its sensor device 130a is that location data (i.e., points from the LiDAR sensor) can be supplemented into a high-resolution temperature map. According to one embodiment, the temperature map is not only created once but is continuously updated based on new data. For this purpose, an algorithm can be used to repeatedly add the current temperature value to the geometric points in the temperature map, which is formed as a point cloud, thereby updating the map. This creates a dynamic 3D temperature map that reflects the temperature changes over time at any location in the warehouse.

[0038] As those skilled in the art will recognize, temperature maps created based on thermal imaging and location data represent a kind of "digital twin" of a warehouse. Distribution maps in the form of such digital twins based on location data are known, for example in DE102022105079 and DE102021133614, the contents of which are incorporated herein by reference in their entirety for further details.

[0039] According to one embodiment, in addition to thermal imaging data and location data, other data collected by sensor device 130a can also be fed into the generation of the digital twin, such as air humidity, carbon dioxide concentration, noise level, etc. For example, sensor device 130a may also include one or more microphones (especially a microphone array, through which the direction of the acquired acoustic signal can be determined, and the distance to the source can be determined according to the arrangement). Furthermore, sensor device 130a can be configured to collect the strength of radio signals in the warehouse to determine which networks exist in the warehouse, their signal levels, and in which frequency band (in order to detect interference sources or switch to another frequency band or modulate the signal).

[0040] According to one embodiment, the processor device (i.e. Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2The processor 111 of the central data processing unit 110 can be configured to identify cargo objects 140 among the plurality of cargo objects 140 based on sensor data using an object recognition mechanism, and to determine the temperature of each identified cargo object 140 based on a temperature map. In such embodiments, the sensor data may specifically include image data of the cargo objects 140, which is acquired by an optical camera of the sensor unit 130a. In one embodiment, the object recognition mechanism may include or be implemented via an artificial neural network trained to identify cargo objects among the plurality of cargo objects 140 in the image data (e.g., determine the type of the respective cargo object 140), and to determine the temperature of each identified cargo object 140 based on a temperature map.

[0041] According to one embodiment, the processor device (i.e. Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2 The processor 111 of the central data processing unit 110 can also be configured to monitor, based on a temperature map, whether the temperature at one or more points in the warehouse is within its respective predefined temperature range, such as not exceeding a maximum temperature threshold. In one embodiment, the monitoring mechanism may include or be implemented via an artificial neural network configured to monitor whether the temperature at one or more points in the warehouse is within its respective predefined temperature range.

[0042] For example, a machine learning model (ML model), particularly a neural network, can be generated from training data. This model is trained to identify violations, such as certain parts of a warehouse exterior wall being too cold, or certain parts of a warehouse cold storage area being too hot. Depending on the application location, this may look different. In other words: according to one embodiment, a trained ML model (especially a neural network) can be used to infer and provide warnings to the user.

[0043] In another embodiment, the temperature of goods 140 in the warehouse can also be collected by a thermal imaging camera of sensor device 130a, for example, during the transportation of goods 140 by industrial vehicle 120a. Therefore, goods 140 can be monitored for temperature over a longer storage period. This can be achieved, for example, by an operator marking and setting thresholds via a motion application on a tablet / smartphone, or via AR / VR devices. The goods are then tracked throughout the storage process, and an alert can be issued if the threshold is exceeded. This can be addressed manually or automatically by connecting to a building management system (to lower the warehouse temperature).

[0044] As mentioned earlier, the processor device (i.e. Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2The processor 111 of the central data processing unit 110 can implement an artificial neural network trained to identify or classify cargo objects (e.g., categories: meat, fish, fruit, etc.) and to know the temperature values ​​that each cargo object category cannot exceed. In another embodiment, it can also be connected to an energy price monitoring system to enable refrigeration when electricity is cheap. Locations in the warehouse that can maintain low temperatures for extended periods can be used for temperature-sensitive goods or to schedule cargo transfers when electricity prices are high but there is still sufficient cold storage space.

[0045] As those skilled in the art will recognize, via a processor device (i.e. Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2 The temperature map created by the processor 111 of the central data processing unit 110 can make many processes in the warehouse more efficient, such as the placement of goods or the control of air conditioning and heating systems. For example, in the pharmaceutical industry, certain drugs and active ingredients must be produced and stored under controlled conditions to ensure their efficacy and safety. In the food industry, certain products must be produced and stored under controlled conditions to ensure their quality and shelf life. In the electronics industry, certain products must be produced and stored under controlled conditions to ensure their performance and reliability. In the automotive industry, certain components must be produced and stored under controlled conditions to ensure their performance and safety.

[0046] The embodiments described herein allow for real-time monitoring and analysis of temperatures (and possibly other physical quantities) at different points within a warehouse. As mentioned above, the monitoring results (in the form of a temperature map) can be integrated into a neural network to create more accurate warehouse environment predictions and optimize warehouse processes. For example, to avoid thermal bridging in a warehouse, the neural network can be trained to reduce heat sources. This can be achieved, for example, by altering warehouse storage methods, such as placing insulated materials near the exterior walls of the warehouse building while placing other goods requiring lower temperatures in the center of the warehouse. Furthermore, the temporal course of heat changes on warehouse surfaces can be determined, i.e., temperature changes on warehouse building components, goods, or machine surfaces can be detected. From this, faults or anomalies can be inferred, such as overheating of machine assemblies, building damage, or damaged packaging within the warehouse.

[0047] Furthermore, through the processor device (i.e. Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2The temperature map created by the processor 111 of the central data processing unit 110 can optimize the warehouse's insulation performance and the layout of climate units (i.e., heating or cooling equipment). The efficiency of warehouse energy management can be tracked very accurately over time. Furthermore, heat leaks leading to condensation or freezing are more easily detected. The cold chain for goods can be better controlled. This allows for traceability analysis after quality assurance by checking the cold chain. Damage to manufacturing machinery can be detected more easily. Deviations and deformations in the manufacturing process caused by temperature can be avoided. The 3D temperature map can be integrated into the warehouse's building information model (BIM).

[0048] According to another embodiment, the processor device (i.e. Figure 1 The control device 123a of the industrial vehicle 120a or Figure 2 The temperature map created by the processor 111 of the central data processing unit 110 can also be used by the warehouse building management system, which can also be implemented on the central data processing unit 110. Based on the temperature map, the warehouse building management system can operate, for example, as follows: the temperature in the warehouse lobby should be kept as constant as possible. If the lobby temperature rises, the lobby roller shutters are opened so that the air conditioning does not need to be run. If the set logic detects that the temperature rise in a certain area exceeds the maximum value, the roller shutters are automatically closed. This saves energy resources.

[0049] Figure 3 A flowchart illustrating the steps of an operational method 300 for an industrial vehicle 120a used to transport goods 140 in a warehouse is shown. The method includes step 301: acquiring multiple sensor data of the industrial vehicle 120a as it moves through the warehouse, wherein the multiple sensor data includes multiple thermal imaging data of the environment of the industrial vehicle 120a. Furthermore, the method 300 includes step 303: creating and / or updating a temperature map of the warehouse based on the multiple thermal imaging data, wherein the temperature map indicates respective temperature values ​​for multiple points within the warehouse.

[0050] The method 300 of the present invention can be executed by the data processing apparatus 123, 110 of the present invention. Therefore, further embodiments of the method 300 of the present invention are derived from the embodiments of the data processing apparatus 123, 110 of the present invention described above and below.

Claims

1. A data processing apparatus (123; 110) for operating an industrial vehicle (120a) for transporting multiple cargo objects (140) in a warehouse, wherein, The data processing device (123; 110) includes: Interfaces (123b; 113) configured to acquire multiple sensor data of the industrial vehicle (120a) while the industrial vehicle (120a) is moving in the warehouse, wherein the multiple sensor data includes multiple thermal imaging data of the environment of the industrial vehicle (120a); and A processor device (123a; 111) is configured to create and / or update a temperature map of the warehouse based on the plurality of thermal imaging data, wherein the temperature map indicates the respective temperature values ​​of a plurality of points in the warehouse.

2. The data processing apparatus (123; 110) according to claim 1, wherein, The plurality of sensor data of the industrial vehicle (120a) also includes plurality of location data of the industrial vehicle (120a), and wherein the processor device (123a; 111) is configured to create or update a temperature map of the warehouse based on the plurality of thermal imaging data and the plurality of location data.

3. The data processing apparatus (123; 110) according to claim 2, wherein, The processor device (123a; 111) is further configured to generate control signals for the industrial vehicle (120a) based on the plurality of location data of the industrial vehicle (120a), wherein the control signals are configured to cause the industrial vehicle (120a) to move within the warehouse.

4. The data processing apparatus (123; 110) according to any one of the preceding claims, wherein, The processor device (123a; 111) is further configured to identify cargo objects (140) among the plurality of cargo objects (140) based on the sensor data using an object recognition mechanism, and to determine the temperature of each identified cargo object (140) based on the temperature map.

5. The data processing apparatus (123; 110) according to claim 4, wherein, The object recognition mechanism includes an artificial neural network configured to identify cargo objects (140) among the plurality of cargo objects (140) and determine the temperature of each identified cargo object (140) based on the temperature map.

6. The data processing apparatus (123; 110) according to any one of the preceding claims, wherein, The processor device (123a; 111) is also configured to, based on the temperature map, use a monitoring mechanism to monitor whether the temperature at one or more of the plurality of points in the warehouse is within their respective predefined temperature ranges.

7. The data processing apparatus (123; 110) according to claim 6, wherein, The monitoring mechanism includes an artificial neural network configured to monitor whether the temperature at one or more of the plurality of points in the warehouse is within their respective predefined temperature ranges.

8. An industrial vehicle (120a), particularly a forklift (120a), for transporting multiple goods (140) in a warehouse, wherein, The industrial vehicle (120a) includes a data processing device (123) and a thermal imaging acquisition device (130a) according to any one of the preceding claims, wherein the thermal imaging acquisition device (130a) is configured to acquire multiple thermal imaging data of the environment of the industrial vehicle (120a).

9. A system (100) for operating multiple industrial vehicles (120a, b) in a warehouse, wherein, The system (100) includes: Multiple industrial vehicles (120a, b) according to claim 8; and The data processing apparatus (110) according to any one of claims 1 to 7 is used to operate multiple industrial vehicles (120a, b) in a warehouse.

10. A method (300) for operating an industrial vehicle (120a) for transporting multiple cargo objects (140) in a warehouse, wherein, The method (300) includes: While the industrial vehicle (120a) is moving within the warehouse, (301) multiple sensor data of the industrial vehicle (120a) are acquired, wherein the multiple sensor data includes multiple thermal imaging data of the environment of the industrial vehicle (120a); and Based on the multiple thermal imaging data, a temperature map of the warehouse is created and / or updated (303), wherein the temperature map indicates the respective temperature values ​​of multiple points in the warehouse.

Images