Method for generating a digital twin which images at least one industrial truck in an intralogistics environment in real time
The digital twin method addresses the limitations of existing intralogistics tracking by using real-time visual localization and unique identification to enhance safety and optimize logistics through continuous tracking and control.
Patent Information
- Application Number
- PCT/EP2026/063785
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-12
- Filing Date
- 2026-05-12
- Publication Date
- 2026-11-19
AI Technical Summary
Existing methods for collision avoidance and monitoring in intralogistics environments rely on manual control, rigid routes, or simple sensors with limited range, failing to ensure reliable, adaptive, and real-time detection and control of industrial trucks in dynamic and complex environments, leading to increased collision risks.
A method for generating a digital twin of industrial trucks using real-time visual localization, continuous optical data from sensors, and unique identification, enabling advanced applications like collision avoidance and logistics optimization through continuous tracking and control commands.
Enhances safety by providing real-time, reliable tracking and control of industrial trucks, reducing collision risks and optimizing logistics processes through accurate identification and adaptive control.
Smart Images

Figure EP2026063785_19112026_PF_FP_ABST
Abstract
Description
S13565WO SENTICS GMBH METHOD FOR PROCUREMENT OF A DIGITAL TWIN WHICH REPRESENTS AT LEAST A FLOOR CONVEYOR IN AN INTRALOGISTICS ENVIRONMENT IN REAL TIME TECHNICAL AREA
[0001] The invention relates generally to the field of intralogistics environments and applications related to the equipment used therein, particularly industrial trucks. Specifically, the invention relates to a method for generating a digital twin that represents at least one industrial truck in an intralogistics environment based on a real-time visual localization of the at least one industrial truck. Furthermore, the invention relates to a data processing system comprising means for executing the method. The invention also relates to a computer program product comprising instructions that, when the program is executed by a computer, cause the computer to execute the method. BACKGROUND
[0002] In intralogistics environments, industrial trucks, or in other words, material handling equipment or material handling devices, such as forklifts, are used to transport, load, or store goods or merchandise. Furthermore, the infrastructure of intralogistics environments typically includes other equipment such as automatic gates, freight elevators, conveyor belts, robot-controlled storage racking systems, and industrial machinery, for example, for automated production processes. Overall, the equipment in intralogistics environments is often integrated into complex traffic networks that include both stationary and mobile entities or components.
[0003] In intralogistics environments, various applications may be desirable, such as monitoring different equipment or processes within the intralogistics environment, or controlling industrial trucks, for example, to prevent collisions. For instance, complex traffic networks result in an increased risk of collisions between equipment and with personnel. Accidents involving industrial trucks (e.g., collisions, crushing, or falls) are among the most frequent workplace accidents. S13565WO SENTICS GMBH
[0004] Previous approaches to collision avoidance, monitoring, and other applications in intralogistics environments mostly rely on manual control, rigid routes, or simple sensors with limited range. These methods are often insufficient to ensure reliable, adaptive, and real-time detection and control of industrial trucks in dynamic and complex environments. SUMMARY
[0005] The present invention advantageously provides a method, a data processing system, and one or more computer program products, which can significantly increase safety in intralogistics environments.
[0006] The invention is defined in the independent claims. Advantageous further developments and / or embodiments are specified in the dependent claims and in the preceding and following description and disclosure.
[0007] A first aspect of the invention relates to a method for generating a digital twin which maps at least one industrial truck in an intralogistics environment based on a visual localization of the at least one industrial truck in real time, wherein the method comprises: - Continuous reception of optical data from the intralogistics environment in real time from at least one or more optical sensors in the intralogistics environment; - Continuous localization, based on the optical data, of at least one industrial truck in the intralogistics environment; - Determining a unique identification identifier for at least one continuously located industrial truck; and - Generating the digital twin based on the continuous localization of at least one industrial truck and its identification identifier.
[0008] Advantageously, this method provides a way to create a real-time digital twin of the intralogistics environment, including at least one industrial truck within it. The industrial truck is continuously tracked visually within the intralogistics environment using multiple sensors. In addition to the visual detection and tracking of the industrial truck, a unique identifier is determined for each industrial truck. The digital twin is then generated based on the continuous localization through optical data and the unique identifier. This ensures that the industrial truck(s) tracked are indeed the ones identified by S13565WO SENTICS GMBH, enabling the improvement of various applications based on the digital twin.
[0009] Through unambiguous assignment and real-time mapping, applications such as control and monitoring measures can be applied to the respective vehicle in a targeted and reliable manner. In a collision avoidance application, for example, this ensures that a control command for collision avoidance can be sent, based on the identification identifier, to the industrial truck that poses a collision risk with another piece of equipment or a person. However, the generated digital twin not only forms the basis for applications like collision avoidance, but can also be used for other advanced applications, such as the automated monitoring and / or control of industrial trucks and the optimization of logistics processes.
[0010] The digital twin advantageously maps the intralogistics environment and its industrial trucks in real time and is based on the visual localization of these trucks. The intralogistics environment can be equipped with a variety of optical sensors, particularly cameras. Such a sensor system, especially a camera system, can continuously capture visual data, particularly images, of the intralogistics environment in real time. While the optical data can be received sequentially in packets or frames, the reception is fundamentally continuous, meaning that optical data is continuously provided over time. The digital twin, or a data processing system on which the digital twin is created, provided, and / or executed, can perform a visual localization of the industrial trucks based on this optical data.The digital twin can also include people in the intralogistics environment and other equipment. Accordingly, the data processing system with the digital twin provides a real-time representation of the intralogistics environment with its moving or changing equipment and people.
[0011] The digital twin can comprise a map of the intralogistics environment, including its outlines, geometries, components, and so on. This map may have been created previously and / or based on optical data. Accordingly, the digital twin can depict the equipment and personnel, along with their positions and movements within the intralogistics environment. The provision of the digital twin can utilize one or more artificial intelligence-based algorithms. For example, the analysis of optical data can implement such algorithms to generate the digital twin, such as for identifying equipment and personnel, tracking the movement or positions of equipment and personnel, and similar functions. Crucially, in addition to locating the industrial trucks, the system also identifies them by determining their unique identifiers."Uniquely" in this context means that the digital twin can distinguish between all industrial trucks in the intralogistics environment based on their identification identifier. Each of several identification identifiers can be unique.
[0012] In principle, creating a digital twin can involve generating one or more digital objects or representations of real-world objects, such as industrial trucks, but also of people and other moving or changing objects within the intralogistics environment. The raw visual localization data can be used as measurement data to predict the state of the virtual object via the digital twin, particularly through models and / or filters within the digital twin. Even when no measurement data is available, for example, due to the brief obscuration of a real-world object, the state of the virtual object can still be predicted via the digital twin or its model. In this respect, the digital twin can also "see" or predict objects like industrial trucks, thus making them suitable for various applications that cannot be detected using optical data.
[0013] The procedure described in the first aspect can, in particular, be at least partially or fully computer-implemented. This means that at least one, several, or all steps of the procedure can be performed by a data processing system, which may include one or more computers or computing units. Such one or more computers can be based partially or entirely on a cloud architecture. In other words, one or more computers of the data processing system can be located in a cloud or remote data center, separate from the intralogistics environment. However, it is also possible for one or more computers to be located additionally or alternatively within the intralogistics environment. This allows for both decentralized and centralized execution of the computer-implemented process with respect to the intralogistics environment. Different steps can be performed by the same computer or by different computers.
[0014] In addition to computer-implemented steps, the procedure can also include steps performed by entities or units that are not, or not completely, executed by a computer. For example, in the context of the following application with a control command, the procedure can include transmitting one or more control commands to at least one industrial truck. This can be done using any wired or wireless communication technology such as WLAN, Bluetooth, and the like. One or more radio modules located within the intralogistics environment can be used.It is also possible that the procedure includes receiving at least one control command at the at least one industrial truck, in particular at a radio module of the industrial truck, and / or executing at least one control command, which in turn may be a computer-implemented step.
[0015] The at least one material handling vehicle can be a manually operated or human-driven vehicle. However, autonomously driven vehicles can also be used. In particular, material handling vehicles can be motorized vehicles such as forklifts, tractors, or other vehicles. Hand-held devices can also be used. The intralogistics environment can be set up specifically for transport, storage, and / or handling processes that take place within the intralogistics environment, and thus range, for example, from goods receipt through storage, order picking, internal transport, and inventory management to goods dispatch.
[0016] For example, the method can include controlling the processing rate of the optical data for continuously localizing the at least one industrial truck and / or generating the digital twin, depending on one or more parameters of the continuously localized industrial truck. The processing rate can be achieved via a streaming rate of the optical data, a receiving rate of the optical data, and / or processing by the data processing system, in particular one or more artificial intelligence models or machine learning models. For example, the processing can be configured such that only a certain proportion of the optical data is processed over time, while another proportion remains unprocessed. For example, only every second or, more generally, every Xth frame or every Xth image in the optical data from one or more optical sensors can be processed, instead of every frame or image.The unprocessed portion of the optical data can still be streamed and received, but it is not processed. One or more parameters can relate to driving parameters such as acceleration, speed, etc., but also to other parameters such as activity, standstill, or operation. Such parameters can be determined using sensor data from the industrial trucks, which can be transmitted from the trucks to the data processing system. As explained in more detail below, such sensor data can originate, for example, from an odometry sensor, accelerometer, rotary encoder, or similar sensor on the industrial truck. It is also possible to evaluate these parameters based on the optical data itself.By reducing the processing rate so that not every frame or image is evaluated to locate the industrial trucks and / or generate the digital twin, computing resources can be saved, especially when high-resolution processing is not necessary, particularly considering the resulting situation, such as a collision risk. For example, people can generally move slowly or slower than industrial trucks, at least as motorized industrial trucks or vehicles. Vehicles, in particular, are faster and require a higher processing rate at higher speeds, which can be expressed in frames per second (FPS) to ensure a fast response time from the digital twin to control interventions. However, if industrial trucks move slowly or...When the forklifts are moving or parked, the processing rate can be reduced to decrease the workload on the servers. When the forklifts are moving or accelerating, the processed frame rate per second can be dynamically increased or adjusted overall. This allows for efficient use of available computing resources.
[0017] Determining the unique identifier can include, in particular, determining an address, such as an IP (Internet Protocol) address and / or a MAC (Media Access Control) address, of at least one continuously located industrial truck. The unique identifier can be the IP address or the IP address itself. Using their address, the industrial trucks can be uniquely identified, and information or data can be exchanged with them. In particular, such information or data can include control commands. For example, corresponding control commands can be sent to the located industrial truck(s) via a wireless radio or communication network, whereby the control commands can be configured for purposes other than collision avoidance.Alternatively or additionally, the identification identifier could be a name and / or a number or sequence of numbers, or some other assignment. Instead of individual control, a message broker could be used, for example. A communication protocol such as MQTT (Message Queuing Telemetry Transport) could be used. In this case, it is sufficient to define a unique ID for each industrial truck or piece of equipment in the digital twin. A message is then sent on a specific topic. All equipment that subscribes to the topic receives the message; for example, all industrial trucks in intralogistics receive messages via this topic. The message in the topic can contain the respective unique individual IDs. For example, a control command can be issued to brake ID1. The industrial trucks know their own IDs.Thus, ID1 knows when it receives the message to brake. All other vehicles ignore the control command because it only affects ID1.
[0018] Furthermore, the method can include: providing at least one control command for transmission to at least one industrial truck based on its identification identifier, wherein the at least one industrial truck is configured to receive and execute the at least one control command. For example, the IP address or MAC address can be used as the address of the industrial truck to which the control command can be sent via the wireless network, or a message broker, as described above, can be used.
[0019] The procedure can further include: determining, based on the digital twin, an individual collision risk of the industrial truck with another piece of equipment or with at least one person in the intralogistics environment; and furthermore, determining, based on this at least one individual collision risk, at least one control command to control the at least one industrial truck to avoid a collision with the at least one other piece of equipment or the at least one person. Thus, the control command can, for example, be based on the determination of an individual collision risk for each industrial truck and be configured for collision avoidance.
[0020] For example, determining the unique identification identifier of the at least one continuously localized industrial truck may include: recognizing, based on optical data, a visual reference mark on the at least one continuously localized industrial truck; and determining, based on the recognized reference mark, the unique identification identifier of the at least one continuously localized industrial truck. Visual reference marks, or landmarks, are artificially designed markings that can be used as geometric or semantic reference points for optical processing systems. Examples of such reference marks include ArUco markers, AprilTag, and registration marks. Applied to industrial trucks, for example, printed or using adhesive film, they serve as visual reference points for determining position and / or orientation within the intralogistics environment. A recognition algorithm can be used to detect the visual reference marks, identify them, and assign them to the visually located industrial truck. By uniquely assigning the visual reference mark to the continuously located industrial truck, the industrial truck is unambiguously identified, unless it leaves the intralogistics environment or the area covered by the optical sensors.In principle, determining the unique identifier can be based on a single-stage or multi-stage process. First, for example, the forklift can be detected or located using a localization model, such as one based on artificial intelligence. In a further step, a reference marker can then be searched for in the image(s) in which the forklift has been located. If the reference marker is detected, the unique identifier can be assigned to the located forklift. This provides a unique identification of the forklift. The reference marker itself can contain the identifier or be associated with it, for example, in a database accessible to the data processing system. An additional advantage of some landmarks, such as AruCo, is that they have a clear direction.This means that the three-dimensional orientation of the landmark can be recognized from the image or optical data. This also allows the orientation and / or direction of travel of the forklift truck relative to the landmark to be determined. This can also be used for comparison with other information from the optical data and / or sensor data from the forklift trucks. This makes it possible to verify the reliability of the data determined from the optical data and / or sensor data of the forklift trucks with regard to the orientation and / or direction of travel, in order to, for example, increase the safety with which a certain control command is determined and applied.
[0021] Alternatively or additionally, determining the unique identifier of the at least one continuously located industrial truck can comprise: recognizing, based on optical data, a barcode on the at least one continuously located industrial truck; and determining, based on the recognized barcode, the unique identifier of the at least one continuously located industrial truck. Unlike visual reference marks, barcodes, or in other words, bar codes, are patterns in which information is encoded. Unlike visual reference marks, which typically serve for geometric reference and not for storing information, barcodes can, in particular, comprise linear patterns. Such patterns, unlike square reference marks, can be applied more easily or flexibly to industrial trucks, for example, on the outer sides of the cab pillars.It was found that, especially with industrial trucks such as forklifts, reach trucks, pallet trucks, etc., the available surfaces for attaching visual markers are often limited. Landmarks, in particular, are often essentially square and therefore cannot be attached to columns or elongated surfaces of the trucks, which presents a significant limitation. Furthermore, multiple barcodes, or a greater total number of barcodes, can be attached to industrial trucks compared to the possible number of reference markers, allowing the trucks to be identified from different perspectives for the creation of the digital twin. A recognition algorithm can be used to identify the barcodes and assign them to the visually located industrial truck.By uniquely assigning the barcode to the continuously tracked forklift, the forklift is unambiguously identified, unless it leaves the intralogistics environment or the area covered by the optical sensors. The determination of this unique identifier can be based on a single-stage or multi-stage process. First, for example, the forklift can be detected or located using a localization model, such as one based on artificial intelligence. In a further step, a barcode can then be searched for in the image(s) in which the forklift has been located. If the barcode is detected, the unique identifier can be assigned to the tracked forklift. This provides unambiguous identification of the forklift. For example, the barcode itself can contain the identification identifier as information encoded by the barcode, or it can contain information, such as a hyperlink, that points to a database of the data processing system in which the identification identifier is stored.
[0022] Determining the unique identification identifier of at least one continuously located industrial truck, based on the recognized barcode, can involve: determining the barcode's orientation within the optical data; and reading the barcode according to this orientation. Determining the barcode's orientation can include rotating at least the barcode and detecting vertical or horizontal lines, and / or rotating the barcode and detecting its aspect ratio. Unlike landmarks, barcodes must be read orthogonally to their lines in order to read the binary code, defined by thick and thin lines. During recognition, the orientation or the direction from which the barcode must be read may not be immediately clear. Therefore, the barcode's orientation can be determined in a single step.This can be achieved, for example, through visual edge detection. S13565WO SENTICS GMBH, for instance, can perform rotation and detection of horizontal or vertical lines, or rotation and analysis of the aspect ratio to maximize the length or minimize the height. Based on the detection of the vertical or horizontal lines or the aspect ratio of the barcode, the barcode's orientation can be determined for correct reading. Furthermore, perspective correction can be performed to compensate for image distortions caused by perspective effects. Subsequently, a section of the image can be generated from which a reading algorithm can read the binary code by detecting thin and thick lines, thus determining the unique identification identifier.
[0023] Alternatively or additionally, determining the unique identification identifier of at least one continuously located industrial truck can comprise: detecting, based on optical data, a light signal from an optical lighting system of the at least one continuously located industrial truck; and determining, based on the detected light signal, the unique identification identifier of the at least one continuously located industrial truck. The light signal can be encoded in the form of an individual light code for each industrial truck. The optical lighting system can be installed as an active lighting element for generating light signals on each of the industrial trucks in the intralogistics environment. The light signal of the industrial trucks can be detected in the optical data.
[0024] The light signal can be encoded by different colors, intensities, and / or durations. This allows for synchronization of visual recognition, for example at 10, 20, 25, or more frames per second, with the light signal on the forklift, by switching the light signal on and off. This enables the creation of highly individualized light codes, allowing for unique identification. Furthermore, by using a color space such as RGB and / or an infrared lighting system, identification can be achieved via a color sequence or by evaluating the color spectrum, for example, the proportions or intensity of RGB in a constant light signal.
[0025] Furthermore, the method can also include: providing at least one lighting command for transmission to the at least one continuously localized industrial truck, wherein the optical lighting system of the at least one industrial truck is configured to receive and execute the at least one lighting command to generate the light signal. For example, lighting commands can be provided and transmitted to all industrial trucks in the S13565WO SENTICS GMBH intralogistics environment, or more generally, to any trucks that can be controlled. This will ensure that each optical lighting system of each industrial truck generates its own light signal, which can then be evaluated based on the optical data to uniquely identify the industrial trucks using their individual light signal or light code.In other words, two-way communication is established between the industrial trucks and the data processing system, which is used for triggering visual light signals for identification, either by the system itself or by the digital twin. The light signal can then be issued when an unidentified industrial truck, or one without a determined identification code, is located in the intralogistics environment based on optical data. The industrial trucks can be queried via wireless communication, to which they may be connected. The data processing system can then, in the case of differing light signals,Light codes can cause all industrial trucks to generate a light signal, or alternatively, they can sequentially generate a light signal on each truck in turn, particularly if each truck produces the same light signal. For example, the existing lighting system, such as one or more headlights or taillights, can be controlled to generate the light signal. Alternatively, an additional optical lighting system could be installed on the industrial trucks for identification purposes only.
[0026] Furthermore, determining the unique identifier of at least one continuously tracked industrial truck can include: identifying radio data indicative of at least one radio signal from a radio module of the at least one continuously tracked industrial truck, received by a radio infrastructure within the intralogistics environment; and determining, based on this radio data, the unique identifier of the at least one continuously tracked industrial truck. The radio-based communication or network between the radio module and the radio infrastructure of the intralogistics environment, for example, comprising one or more radio transmitters, can be based on Bluetooth, RFID (Radio-Frequency Identification), Ultra-Wideband (UWB) technology, WLAN (Wireless Local Area Network), GPS (Global Positioning System), or similar technologies.The radio module can, for example, be implemented in a particularly simple form as a transponder on the industrial truck, such as an RFID or Bluetooth transponder. Using the radio data, S13565WO SENTICS GMBH can estimate the spatial position of the industrial truck via radio tracking, particularly through a suitable algorithm. The position of the industrial truck determined based on the radio data can be compared with the visual localization or the position based on visual localization. Accordingly, a unique identification identifier can be assigned to the industrial truck based on the radio data, which could, for example, contain the unique identification identifier or IP address. For example, triangulation can be used, in which the direction of the radio signal is determined from multiple locations, and the intersection of these directions indicates the position of the transponder and thus the forklift. Additionally or alternatively, time-of-flight measurement, signal strength evaluation, angle measurement, and / or time difference measurement can be used to determine the forklift's position. With time-of-flight measurement, the distance is calculated from the time difference between sending and receiving the signal. With signal strength evaluation, the distance of the transponder is estimated based on the received signal strength. With angle measurement, the angle of incidence of the signal is determined at multiple receivers of the radio infrastructure. With time difference measurement, the position is determined from the time differences with which the signal arrives at different receivers of the radio infrastructure.
[0027] Furthermore, determining the unique identifier of at least one continuously localized industrial truck can include: determining environmental sensor data that is indicative of environmental detection by an environmental sensor system of the at least one continuously localized industrial truck; and determining, based on the environmental sensor data, the unique identifier of the at least one continuously localized industrial truck. The environmental sensors of the industrial truck's environmental sensor system can, for example, include one or more cameras and / or lidar (light imaging, detection, and ranging) sensors. The data from the environmental sensor(s) can be used to determine the self-location of the industrial truck(s). This can be compared with the visual localization or the position based on the visual localization.Accordingly, the forklift truck can be assigned a unique identification identifier based on the environmental sensor data. For self-localization, a SLAM (Simultaneous Localization and Mapping) algorithm can be used, for example, which combines the environmental sensor data with complex calculations to determine its own position and, if necessary, its orientation in space.
[0028] Furthermore, determining the unique identifier of at least one continuously located industrial truck can include: determining sensor data that are indicative of the acceleration, speed, and / or steering angle of the at least one continuously located industrial truck; and determining, based on the sensor data and optical data, the unique identifier of the at least one continuously located industrial truck. The sensor data can be, for example, odometry data, yaw rate data, magnetic field sensor data, and / or acceleration data from appropriately configured sensors on the industrial truck. Thus, the position of the industrial trucks can be estimated over time based on the speed and / or steering angle. This allows for the determination of the truck's state, such as standstill, travel or movement, direction of travel, etc.and / or parameters such as speed can be used for identification by comparison with visual localization. It is also possible to read sensor data, such as from the seat sensor and / or the vehicle's ignition. This data can be used for comparison with the optical data and / or other sensor data, e.g., whether a forklift is actually stationary or moving. This can be checked when the seat sensor is inactive, i.e., when there is no driver in the seat or the ignition is switched off, and vice versa when the seat sensor is active or the ignition is switched on.
[0029] Alternatively or additionally, the procedure may further include: determining sensor data indicative of acceleration, velocity and / or steering angle of the at least one continuously localized industrial truck, environmental sensor data indicative of environmental detection by an environmental sensor system of the at least one continuously localized industrial truck, and / or radio data indicative of at least one radio signal from a radio module of the at least one continuously localized industrial truck, which has been received by a radio infrastructure of the intralogistics environment; and verifying, based on the optical data as well as the sensor data, the environmental sensor data and / or the radio data, the determined unique identification identifier of the at least one continuously localized industrial truck.The determined unique identifier cannot be based on sensor data, environmental sensor data, and / or radio data. Instead, the determined unique identifier can be based, for example, on the visual reference mark described herein, the barcode, and / or the light signal, or other methods. In the case of sensor data, the status and / or parameters can be used to continuously verify whether an assigned identification identifier is correct. The same can be done with radio data and environmental sensor data. For example, if an incorrect unique identifier was assigned to a forklift truck due to the faulty detection of a visual reference mark (S13565WO SENTICS GMBH), this can be verified by checking the vehicle's status using the sensor data.For example, if the forklift truck with an incorrectly assigned identification code visually indicates movement in the optical data and thus in the digital twin, but its sensor data indicates a standstill, then an incorrect visual assignment of the identification code can be detected and corrected by re-identifying the visual reference mark and determining the unique identification code based on the detected visual reference mark, especially from a better perspective or multiple perspectives in the optical data to improve the reliability of the assignment.
[0030] The at least one control command, as previously mentioned, can, for example, include reducing the travel speed and / or performing an evasive maneuver by the at least one industrial truck. Reducing the travel speed can be achieved, for example, by actively braking or stopping the at least one industrial truck. The evasive maneuver can be performed by a single change of direction or several consecutive changes of direction. The at least one industrial truck may be equipped with a control module that enables the reduction of travel speed and / or the evasive maneuver. Reducing the travel speed can be achieved, for example, by engaging a creep speed setting, particularly one provided by the manufacturer, or by commands via a CAN (Controller Area Network) bus of the industrial truck.The control command is executed by the industrial truck, in particular the control module, but is determined, in particular generated, by the data processing system, based on the digital real-time twin.
[0031] The procedure may also include, in particular: - Determining the current speed of at least one industrial truck; - Determining the distance between at least one industrial truck and at least one other piece of equipment or at least one person; and / or - Determining the state of at least one other piece of equipment; The determination of at least one control command is further based on the current driving speed, the distance, and / or the state of the vehicle. The current driving speed, distance, and / or state of at least one other piece of equipment can be visually detected and thus represented in the digital twin. This allows the driving speed, distance, and / or state of the vehicle to be incorporated into the determination of the individual collision risk, thereby increasing the accuracy of the calculation. Alternatively, the current driving speed, distance, and / or state can also be determined by appropriate sensors in the intralogistics environment and transmitted to the data processing system.For example, a sensor on the forklift, such as an odometry, rotation angle, and / or acceleration sensor, could measure the current travel speed and transmit it to the data processing system. Similarly, a distance sensor on the forklift could transmit its distance from other objects to the data processing system, and / or other equipment could transmit its status, such as whether a gate is open or closed, to the data processing system. It is also possible for both methods to be used: the digital twin and sensors in the intralogistics environment could determine the current travel speed, distance, and / or status. This would further increase the reliability of determining individual collision risks, as the respective current travel speeds, distances, and / or states could be compared against each other.
[0032] In general, the risk of collision increases as the distance between the forklift and a person decreases, so considering the distance allows for a reliable assessment of the collision risk. The risk of collision also increases with increasing speed, as the reaction time for the forklift operator becomes shorter, and there is less time for both the operator (braking and / or swerving) and the person (swiping) to react.
[0033] At least one other piece of equipment can be an infrastructure object within the intralogistics environment. Specifically, it could be an automatic gate, an automatic door, a freight elevator, or a machine. Its condition can be important insofar as it can influence the forklift's travel path. For example, automatic gates or doors may not be open sufficiently for a forklift to pass through. If the opening or closing status of the gate or door is not taken into account, or if, for example, the height or position of the freight elevator, or more generally, the condition of a machine, is not considered, this can lead to a collision with the other piece of equipment or increase the likelihood of a collision with a person due to possible manual evasive maneuvers or similar actions by the forklift operator.By taking such conditions into account when determining at least one control command, or in particular when determining the individual collision risk, it can be ensured that a suitable control command is provided to counteract a correspondingly high collision risk, for example by reducing the speed of the industrial truck before passing through a gate that is not yet sufficiently or fully open, in order to avoid collisions.
[0034] The procedure may also include, in particular: - Determine, based on the at least one individual collision risk, at least one further control command to control the at least one further piece of equipment to avoid collisions with the at least one industrial truck or the at least one person; and - Providing at least one additional control command for transmission to at least one additional piece of equipment, wherein at least one additional piece of equipment is configured to receive and export at least one additional control command. Accordingly, the digital twin also enables a reduction in the risk of collisions with infrastructure objects. A control command is provided that, in addition to controlling the industrial truck, also controls the infrastructure object in order to avoid a collision. For example, in the case of a closing automatic gate towards which an industrial truck is moving, it is possible that the automatic gate interrupts the closing process or, alternatively, that the automatic gate opens further and the industrial truck is slowed down.
[0035] The procedure may also include, in particular: - Determine, based on several individual collision risks and the associated positions of at least one industrial truck, at least one collision-prone area of the intralogistics environment; and - Providing preventive control commands to reduce driving speed and / or to perform an evasive maneuver by industrial trucks in the area where there is at least one collision risk. If a high, or even increased, individual collision risk is frequently identified at certain positions of the industrial truck(s) within the intralogistics environment, where a collision could or would have occurred had the control command not prevented it, these positions can be defined as collision-prone areas or grouped together as such. In such collision-prone areas, preventive control commands can be issued in addition to or as an alternative to determining the individual collision risk. These commands are executed by the industrial truck(s) crossing the collision-prone area. This creates an additional or alternative collision avoidance mechanism in particularly high-risk areas. It is also possible to define collision-prone areas in the digital twin manually or by other means, in which such preventive control commands are applied.Each time a forklift passes through a collision-prone area, the data processing system can register this using the digital twin and accordingly provide or instruct the preventive control command for transmission to the corresponding forklift.
[0036] Alternatively or additionally to the previous examples of control commands, at least one control command, or a further control command, can be configured to generate a visual, acoustic, and / or haptic signal to warn the operator of at least one industrial truck. Instead of or in addition to a control intervention in the industrial truck, this allows the operator to react individually to the hazardous or collision situation. Determining the at least one control command depends on the calculated individual collision risk. For example, there can be one or more threshold values for the individual collision risk, which are compared with the calculated individual collision risk in a specific situation represented by the digital twin. If a threshold value is reached or exceeded, the at least one control command can be determined.The control command can correlate in its extent, for example, the degree of speed reduction, with the determined individual collision risk. Alternatively, different threshold values can be used for various predefined control commands, which are then individually determined by comparing the collision risk with the threshold value. A control command to intervene in the forklift's steering or very strong interventions such as a stop can, for example, only be issued in cases of relatively high individual collision risks. Conversely, control commands to warn the driver, as a less severe measure, can be determined and issued even in cases of relatively low individual collision risks.
[0037] For example, at least one industrial truck can include at least one optical lighting module. This lighting module can be positioned within the field of vision of at least one driver, which can also include peripheral vision. The lighting module can be configured to illuminate in at least three different lighting modes depending on the individual collision risk, with each mode differing in color, intensity, and / or frequency. Of course, it is also possible for only one or two lighting modes to be provided. Furthermore, it is also possible for multiple lighting modules to be used. Accordingly, an intuitive optical warning system can be provided that uses different lighting modes for different collision risks. For example, red can be used for relatively higher collision risks than yellow.Alternatively, higher intensities or frequencies of illumination or flashing of the light module can be used to indicate different levels of collision risk and thus hazardous situations. It is also possible, for example, to display only the color red and not yellow and / or green.
[0038] Alternatively or additionally, the at least one lighting module can be configured as an information system to inform the driver about the functionality of the at least one industrial truck. This functionality can specifically relate to the individual collision risk, i.e., it can provide information on whether the at least one lighting module is functioning correctly in relation to the collision risk and / or whether it is implementing control commands, if necessary, thus enabling a control intervention. Such a visual indication from the lighting module can be displayed on the same or different light sources within the at least one lighting module, for example, on different LEDs than those used when the module illuminates to warn the driver based on the individual collision risk.This allows the operator to be informed if the forklift is unable to reduce its speed, or cannot reduce it at all, for example, because the forklift is outside of your reception range, such as a Wi-Fi coverage area, where it can receive the necessary control commands. It is also possible that if the forklift is connected to the Wi-Fi network but this connection is not recorded in the optical data, communication (such as messages or ping tests) can be used to check whether an individual forklift exists on the network. This identification may not yet have been performed, for example, by recognizing a landmark. Both of these conditions can be communicated to the operator via the indicator light on the light module to inform them that a speed reduction is not possible.
[0039] Alternatively or additionally, it is also possible that at least one lighting module is configured as a navigation system for the driver. In particular, different lighting modules can be arranged in different positions, e.g., on pillars in the driver's cab. This allows navigation signals from the digital twin to be sent as part of the control commands for collision avoidance and / or more generally, and then implemented accordingly by the driver. The same LEDs used for the warning and / or information function regarding functionality, or different LEDs, can be used for the navigation system.
[0040] For example, the at least one optical lighting module can comprise at least two light sources arranged at different positions within the driver's cab of the industrial truck. The control command can be determined such that the optical signal is generated at the light source in the position that indicates a direction related to the identified at least one individual collision risk. In other words, the control command is determined or generated such that the light source is activated that points the driver toward the direction of a hazardous situation or potential collision, for example, the position of a person behind a wall not visible to the driver, behind which the person is located and moving toward the industrial truck without the driver being able to see them. It is also possible for light sources to be arranged at three, four, or more different positions, particularly corners or sides, of the driver's cab.For example, it is possible that the light sources are arranged on different pillars in the driver's cab.
[0041] Alternatively or additionally, at least one industrial truck can include at least one vibration module. This vibration module can be located, for example, in the driver's seat and / or in the steering mechanism of the industrial truck. Accordingly, the control command can issue a haptic warning via vibration, for example, at the driver's seat or the steering mechanism, such as a steering wheel.
[0042] In particular, determining the individual collision risk of at least one industrial truck can be based on an evaluation of at least one or any several of the following influencing factors from the digital twin: - a position, a speed and / or a direction of movement of at least one industrial truck, at least one other piece of equipment and / or at least one person; - a distance between at least one industrial truck, at least one other piece of equipment and / or at least one person; - an infrastructure of the intralogistics environment in an area of at least one industrial truck, at least one other piece of equipment and / or at least one person; and - Perception conditions in an area of at least one industrial truck, at least one other piece of equipment, and / or at least one person. S13565WO SENTICS GMBH In principle, for the purpose of accident hazard detection or determining the individual collision risk of each piece of equipment, a situation analysis can be carried out for the location, area, or position of the equipment(s) and / or, if applicable, one or more persons located nearby. Various of the aforementioned influencing factors or parameters can be considered for determining the individual collision risk, which are evaluated accordingly, and on the basis of which a suitably appropriate control command can then be determined, provided that the data processing system deems intervention necessary or beneficial to avoid a collision.The influencing factors can be, for example, situational and / or geometric in their nature.
[0043] In addition to the positions of equipment and personnel, which can be used to determine the distance between them, and their speeds, directions of movement are also relevant influencing factors that can be provided by the digital twin and evaluated by the data processing system or taken into account by the process. If the directions of movement, which can be captured or provided as vectors by the digital twin, intersect, the distance can decrease. This typically results in a meeting point or potential accident or collision point. The risk increases the closer the intersection of the vectors is to one of the objects, i.e., equipment or personnel. If a forklift and a person have opposite direction vectors, they move away from each other, and the risk decreases.If the vectors are aligned and the forklift is moving towards the person, there is a higher risk of collision because the person cannot see the forklift. If the forklift is moving away from the person, the distance increases and the risk of collision decreases. If the vectors are aligned, the objects are moving towards each other and the risk of collision increases. However, in this case, there is a greater chance that both can see each other and react. If the objects are moving at right angles to each other, there is a higher risk of collision, especially in conjunction with infrastructure objects that obstruct the view, such as boxes or walls. If a forklift and people are moving side by side (so-called "followers"), there is a high risk of collision due to the forklift swinging out or being crushed between the wall and the forklift, or being run over if a person trips.If a person is moving in the same direction in front of the forklift (so-called spotter), there is a high risk of collision due to being run over if the person stumbles. S13565WO SENTICS GMBH
[0044] Even with a stationary industrial truck, a collision or accident risk can exist, which the system can take into account. This risk can even be greater than with moving industrial trucks, as a person cannot anticipate a sudden start. The corresponding risk depends significantly on whether the driver sees the person. By fusing sensors with odometry, acceleration, and / or steering angle or yaw angle data from the industrial truck, the data processing system can determine the individual collision risk with greater accuracy, and in particular, quantify it. For example, a person might be standing in front of the industrial truck, resulting in a low collision risk. However, the industrial truck, such as a forklift, might be loaded, thus restricting the driver's forward visibility.The driver's perception, as represented by the digital twin, is limited in this situation, resulting in a higher collision risk. If the person is behind the forklift, a lower collision risk is determined. However, if the person is to the side of the forklift, a higher collision risk can be determined, as this could cause the forklift to swing out on its rear axle. A high collision risk can also be determined if a person is positioned between a wall and the forklift. If the vehicle swings out, there is a significant risk of crushing the person.
[0045] Infrastructure, particularly infrastructure elements, in intralogistics or industrial environments can significantly influence the risk of collisions, a factor that can be taken into account by the procedure. For example, in the case of gates or entrances and / or passageways, people behind them may not be visible to the forklift operator. There is also the risk of being run over by high-speed doors. Furthermore, gates restrict space. When people and forklifts pass through them simultaneously, the distance between them is reduced to a minimum. People to the side of gates and entrances may then be invisible. Corridors, corners, walls, ramps, inclines, or varying floor heights and angles, and / or driving and walking areas, also prevent direct line of sight between people and forklift operators. Doors and exits leading onto a travel path can also increase the risk.Machines, objects, and other equipment can also restrict or completely obstruct the view between people and drivers. On ramps, there is a risk of falls or loads dropping from the lifting devices of industrial trucks, such as the forks of a forklift. Generally, there is a risk of loads slipping off the forks when driving up or down ramps. Raised edges or curbs can increase the risk of people tripping on the roadway or having their feet crushed. Bollards, barriers, and similar structures are installed to protect infrastructure, but they also increase the risk of people being crushed.
[0046] Furthermore, the senses of individuals and drivers can be impaired, increasing the risk of overlooking other objects and thus the risk of collision, which the procedure can take into account. For example, visibility and visual perception can be limited in darkness or poor lighting. This can be included in the determination of the collision risk. When a person or a forklift moves from a dark to a bright area or vice versa, visibility can be impaired due to delayed adaptation, which can also be considered when determining the collision risk. For instance, if a forklift travels from a poorly lit hall into a brightly lit outdoor area, the driver may experience glare, which can impair the visual perception of objects in the outdoor area.Conversely, visual perception is also limited when entering a poorly lit hall from a brightly lit outdoor area. Loud noise, and the potential need for hearing protection, can also impair auditory perception. This, too, can be modeled by the digital twin and thus recognized and taken into account when determining the collision risk. Furthermore, the driver's visibility of the immediate surroundings generally decreases with the size of the forklift, increasing the likelihood of overlooking people in the vicinity. Therefore, the size of the forklift can also be considered when assessing the collision risk.
[0047] In principle, one or more artificial intelligence-based algorithms can be used not only for providing or generating the digital twin, but also, alternatively or additionally, for determining the individual collision risk and / or determining at least one control command. For example, an algorithm can be trained to continuously determine the aforementioned influencing factors in the digital twin and, based on these factors, to continuously calculate the respective collision risk. Training data for any type of algorithm training, as described herein, can include, for example, optical data, odometry data, and other data from the equipment, etc. Synthetic data can also be generated and used for training.For example, renderings of virtual objects can be projected or mapped onto the real camera perspectives of the S13565WO SENTICS GMBH optical sensors to represent as many situations as possible with different industrial trucks, equipment, and environmental conditions. Entire scenarios or situational sequences, even virtual accidents, can be synthetically mapped to generate corresponding training data and train the AI accordingly. The synthetic data can be individually generated for each intralogistics environment based on its specific conditions, such as lighting, spatial characteristics, colors, and the geometries of the industrial trucks, etc., so that each training session can be individually tailored to each intralogistics environment to ensure optimal performance.The advantage here is that the synthetic data is automatically annotated for training during generation. This data is then also unambiguous and correct.
[0048] A second aspect of the invention relates to a data processing system which is set up to execute or carry out the method according to the first aspect of the invention.
[0049] The data processing system can comprise one or more computers or computing units. Such computers can be partially or fully based on a cloud architecture, meaning that the computer(s) of the data processing system are located in a cloud or data center remote from the intralogistics environment. However, it is also possible for one or more computers to be located additionally or alternatively within the intralogistics environment. This allows for both decentralized and centralized implementations of the data processing system and its processing within the intralogistics environment.
[0050] A third aspect of the invention relates to one or more computer program products, comprising instructions which, when executed by one or more computers, cause the execution of the method according to the first aspect of the invention.
[0051] The one or more computer program products can be executed or executable on the data processing system described herein and / or be stored therein. Each computer program product can be a computer program arrangement or a software arrangement comprising at least one computer program or at least one computer program code, as such, or a product such as a data storage device on which such a computer program arrangement or at least one computer program is stored.
[0052] A fourth aspect of the invention can relate to an intralogistics environment with a data processing system according to the second aspect of the invention. The S13565WO SENTICS GMBH intralogistics environment and the data processing system can be geographically separated and / or at least partially located in close proximity. For example, the data processing system can be based partially or completely on a cloud architecture.
[0053] The intralogistics environment can be equipped with a variety of optical sensors, particularly cameras. Such a sensor system, especially a camera system, can continuously capture optical data, particularly images, of the intralogistics environment in real time. The individual optical sensors can also cover overlapping monitoring or recording areas, allowing different optical sensors to capture at least part of the same area in the intralogistics environment from different perspectives or angles. The data processing system on which the digital twin is provided or executed can use this optical data to visually locate the equipment and personnel within the intralogistics environment.
[0054] For example, the intralogistics environment can comprise one or more halls or other industrial buildings in which the optical sensors are located. Furthermore, the intralogistics environment can have a wired and / or wireless communication interface for communicating with the data processing system. The control commands from the data processing system can be received and subsequently evaluated via this communication interface or, alternatively, directly via radio modules of the intralogistics equipment, such as forklifts.
[0055] Features, elements, functions and / or advantages of the method described herein may also be applied to the data processing system, computer program product and intralogistics environment described herein, and vice versa, and in any combination thereof. BRIEF DESCRIPTION OF THE FIGURES
[0056] Exemplary embodiments of the invention are described below with reference to the accompanying figures.
[0057] List of characters Fig. 1 schematically shows a digital twin of an intralogistics environment. Fig. 2 schematically shows the intralogistics environment from Fig. 1 and a Data processing system. Fig. 3 schematically shows a method for collision-avoiding control of equipment in an intralogistics environment. S13565WO SENTICS GMBH Fig. 4 schematically shows an example of a forklift truck in the intralogistics environment of Fig. 1. Fig. 5 schematically shows a method for generating the digital twin from Fig. 1.
[0058] Similar, similar-looking, identical or equivalent elements in the figures may be provided with similar or identical reference symbols. DETAILED DESCRIPTION OF EXECUTION FORMS
[0059] Fig. 1 schematically shows an intralogistics environment 10, which is shown here as an example section of a hall.
[0060] The intralogistics environment 10 features an optical system 50 (see Fig. 2) with several cameras 51, which are distributed throughout the intralogistics environment or the hall shown. The images from the cameras 51 are processed by a data processing system 100, which may be located partly within the intralogistics environment 10 and partly remotely, for example in a cloud environment and / or on a remote computer of the operator of the intralogistics environment 10.
[0061] Accordingly, Fig. 2, which schematically illustrates the intralogistics environment 10 from Fig. 1 and the data processing system 100, shows that at least a first computer 110, in particular in the form of a server, is arranged in the intralogistics environment 10. In contrast, at least one second computer 120, particularly in the form of a cloud architecture or infrastructure, and at least one third computer 130, particularly in the form of a computer at the operator of the intralogistics environment 10, are, by way of example, located outside the intralogistics environment 10. However, the at least one second computer 120 and / or the at least one third computer 130 can alternatively be located within the intralogistics environment 10.
[0062] As shown in Fig. 2, at least one computer 110 is connected to the optical system 50, enabling it to receive and process the images from the cameras 51. This allows the data processing system 100 to map the intralogistics environment 10, including the equipment 20, 30 and personnel 40 located therein, in real time. In particular, the continuous provision of images enables the data processing system 100 to generate a digital twin 1, which maps the intralogistics environment 10 based on the visual localization of the equipment 20, 30 and the person 40. Figure 1 shows two industrial trucks 20 as exemplary equipment 20 and a gate as an infrastructure object and exemplary additional equipment 30. Accordingly, the digital twin 1 represents a real-time copy of the intralogistics environment 10.
[0063] As shown in Fig. 2, the digital twin 1 can be displayed, for example, on at least one third computer 130 or its display, so that the operator of the intralogistics environment 10 or an authorized person can monitor the processes in the intralogistics environment 10 together with the industrial trucks moving in it, the other changing equipment 30 such as the gate, and the people 40 in it.
[0064] The equipment 20, 30 can in turn comprise control modules that are connected to the data processing system 100, exemplified in Fig. 2 by the at least one first computer 110. The examples of the industrial truck 20 and the gate 30 are accordingly a vehicle module 21 and a gate module 31, which can also be referred to as control modules. Via the connection to the modules 21, 31, the data processing system 100 can transmit control commands to the industrial trucks 20 and the gate 30. The connection can, in particular, be wireless, so that the at least one first computer 110 can provide the control commands for transmission via radio communication between corresponding radio modules of the equipment 20, 30 and the infrastructure of the intralogistics environment 10, which can then be transmitted. Fig.Figure 4 shows an example of an operating resource 20 in the form of a forklift truck 20, in particular a forklift, with a corresponding vehicle module 21 and a radio module 23.
[0065] Fig. 3 shows an exemplary method 200 for collision-avoiding control of the equipment 20, 30, in this case the industrial trucks 20 and the gate 30, in the intralogistics environment 10 using the digital twin 1.
[0066] In a first step 201 of the procedure 200, the digital twin 1 is provided, in particular created, which maps the intralogistics environment 10 in real time on the basis of the visual localization of the equipment 20, 30 and persons 40 by the optical system 50.
[0067] In a second step 202 of the procedure 200, an individual collision risk for each of the industrial trucks 20 in the intralogistics environment 10 is determined based on the digital twin 1. As shown in Fig. 1, it can be seen that the left industrial truck 20 in this example could potentially collide with the person 40, who might emerge from the door 12 behind the wall 11 of the intralogistics environment 10, which is not visible to the left industrial truck 20. Furthermore, it can be seen that the right industrial truck 20 could potentially collide with the insufficiently opened gate 30 because the driver's view of the right industrial truck 20 could be restricted due to the truck's load, here exemplified as a forklift.
[0068] To determine individual collision risks, various influencing factors can be evaluated based on the digital twin 1. Since the digital twin 1 is a real-time representation of the equipment 20, 30 and personnel 40 in the intralogistics environment 10, past and current real-time data are available. For example, the data processing system 100 can determine the directions of movement of the left-hand forklift 20 and the person 40 from the digital twin 1. If these paths intersect behind door 12, a higher individual collision risk can be determined.Further influencing factors such as the distance between the left forklift 20 and the person 40, their positions and speeds, the overall infrastructure, for example the door 12 and the wall 11 and thus the restricted or impossible view of the left forklift 20 and the person 40 can also be determined and taken into account on the basis of the digital twin 1.
[0069] To determine the individual collision risk, one or more AI (artificial intelligence) models or algorithms can be used, which can be accessed, for example, by at least one second computer 120. Such AI models can be trained to assess the multiple influencing factors for the respective situations of the industrial trucks 20 and to determine an individual collision risk from this.
[0070] It may also be provided that, in addition to the digital twin 1, further data are considered for determining the individual collision risk, or that the digital twin 1 is fed with further data. Such further data could, for example, be sensor data, such as odometry data, from the industrial trucks 20, so that the travel speed of the industrial trucks 20 is known not only from the optical data but also from the sensor data of the industrial trucks 20. Data from the equipment 30, such as actuator data or other equipment data, such as limit switch data for closing and opening the gate 30, can also be taken into account.Such additional data can be received by at least one first computer 110 and taken into account for the generation of the digital twin 1 or additionally taken into account for the determination of the individual collision risk of each of the industrial trucks 20.
[0071] In a third step 203 of the procedure 200, a control command is determined, which serves to control the respective industrial truck 20 in the example of Fig. 1. The determination is based on the individual collision risk, which was determined in step 202. For example, if there is a sufficiently high individual collision risk or a limit value for the collision risk is reached, a corresponding control command can be determined. It is also possible that the control command correlates with the determined individual collision risk, especially in its intensity, i.e., how strongly an industrial truck 20 is braked, whereby this can be stronger the higher the collision risk.
[0072] In a fourth step 204 of the process 200, the control commands thus determined for the two industrial trucks 20 in Fig. 1 are transmitted to the two industrial trucks 20 or their modules 21, 31 for execution. In a fifth step 205, the control commands are finally transmitted, and in a sixth step 206, the control commands are executed by the modules 21, 31. An actual-target comparison can be performed here, for example, between the optical data and the sensor data of the industrial trucks 20, to determine whether the control command has actually been executed, i.e., whether, for example, an industrial truck 20 has been brought to a standstill or slowed down to a lower speed.
[0073] For example, a control command can be executed for the left-hand industrial truck 20, which reduces its travel speed, stops the industrial truck 20 completely and / or causes it to perform an evasive maneuver, for example by causing the industrial truck 20 to move away from door 12.
[0074] For the right-hand forklift 20, a control command can be executed, for example, to reduce its travel speed or to stop the forklift 20 completely. Additionally or alternatively, a control command for gate 30 can also be determined, provided, transmitted, and executed. This command could, for example, cause gate 30 to open further.
[0075] Fig. 4 shows an example of a material handling vehicle 20, specifically a forklift truck. The material handling vehicle 20 is operated or driven by a human driver 3. In addition to or as an alternative to the vehicle module 21 for controlling the operation of the material handling vehicle 20, a further vehicle module 22 can be provided, or this further vehicle module 22 can be integrated with the vehicle module 21 in its functionality described below. The radio module 23 can receive a type of control command from the data processing system 100, which is configured to generate an optical, acoustic, and / or haptic signal to warn a driver 3 of the material handling vehicle 20, instead of or in addition to controlling the operation.For example, such a visual, acoustic, and / or haptic warning can be issued before the ferry's control intervention, thus preparing the driver (S13565WO SENTICS GMBH) for the intervention. Alternatively, the driver (S13565WO SENTICS GMBH) can be left to react to the warning manually. The decision as to whether a control intervention occurs or whether the driver is left to manual intervention can, for example, depend on a determined degree of individual collision risk.
[0076] The industrial truck 20 of Fig. 4 features, by way of example, an optical lighting module 24, which is arranged on the driver's cab 26 within the driver's field of vision 3. The lighting module 24 is configured to illuminate in one, two, three, or more different lighting modes depending on the individual collision risk. These different lighting modes can vary in their colors, intensities, and / or frequencies. This allows the vehicle module 22 to generate or control an optical warning signal for the driver 3 by executing the corresponding control command. Different levels of collision risk can be achieved by controlling the module with different colors, intensities, and / or flashing frequencies.
[0077] By way of example, the optical lighting module 24 comprises two or more, in particular four, light sources 25 or lighting elements, each of which can be arranged at different positions, for example, pillars of the driver's cab 26. This allows the control command to selectively determine where the lighting module 24 generates the optical signal in the driver's 3 field of vision. This makes it possible to indicate a direction in connection with the determined, at least one, individual collision risk. Thus, the light source 25 can be illuminated precisely where the individual collision risk arises or from which it is evident, for example, in Fig. 1 for the left-hand industrial truck 20, to the right of the driver 3 or industrial truck 20, so that the front right pillar or its light source 25 can illuminate.
[0078] Alternatively or additionally, the further vehicle module 22 can be configured as a vibration module, which can be located in a driver's seat 27 and / or in a steering mechanism 28 of the industrial truck 20. This allows warnings to be transmitted to the driver 3 via haptic signals. Additionally or alternatively, for example, one or more loudspeakers can also be provided on the industrial truck 20 to emit acoustic signals to warn the driver 3, for example in the form of warning tones and / or spoken text or announcements indicating the risk of collision, e.g. “Attention, person front right” in the example of the left-hand forklift in Fig. 1.S13565WO SENTICS GMBH
[0079] Fig. 5 schematically shows a method 300 for generating the digital twin 1 from Fig. 1, as it can be carried out before or together with the method 200, in particular before or together with step 201. For example, the provision of the digital twin 1 in step 201 can include the generation of the digital twin 1 by the method 300.
[0080] In a first step 301, the method 300 comprises the continuous reception of optical data of the intralogistics environment 10 in real time from the multiple optical sensors 51 in the intralogistics environment 10. The optical data can include images from the optical sensors 51, in particular cameras, of the different areas of the intralogistics environment as well as from different perspectives and viewpoints and can show the industrial trucks 20, the other equipment 30 such as the gate 30 in Fig. 1 and persons 40.
[0081] In a second step 302, the procedure 300 includes continuous localization, based on optical data, of the industrial trucks 20 in the intralogistics environment 10. In other words, the position of the industrial trucks 20 is continuously recorded using optical data.
[0082] In a third step 303, the procedure 300 includes determining a unique identification identifier for the continuously localized industrial trucks 20. This unique identification identifier can include an IP address by means of which the industrial truck 20 can be reached, and in particular controlled, via wireless radio communication.
[0083] For determining the unique identification identifier, a visual reference mark 29a or a barcode 29b can, for example, be recognized in the optical data on the industrial trucks 20, as shown in Fig. 4. These reference marks 29a or barcodes 29b can be or comprise the identification identifier, or the reference marks 29a or barcodes 29b can, for example, be assigned to the corresponding identification identifiers, e.g., IP addresses, in a database of the data processing system 100, so that these can be determined on the basis of the recognized reference marks 29a or barcodes 29b.
[0084] Alternatively or additionally to reference marks 29a and barcodes 29b, the industrial trucks 20 can each include one or more optical lighting systems 29c, wherein in Fig. 4 a lighting system 29c is arranged purely by way of example on the roof of the industrial truck 20 shown therein. The optical lighting system 29c can generate a light signal which can be captured in the optical data and can accordingly be recognized by the processing of the optical data. The light signal can be encoded by different luminous colors, luminous intensities and / or luminous duration, and by this encoding can be assigned a unique identification identifier, for example in a database of the data processing system 100.
[0085] Alternatively or additionally, the industrial trucks 20 can also be identified by their unique identification identifier using radio data from the radio module 23 of the industrial trucks 20, environmental sensor data from an environmental sensor system of the industrial trucks 20 and / or sensor data from the industrial trucks 20 which determine an acceleration, a speed and / or a steering angle.
[0086] In a fourth step, 304, the digital twin 1 is generated based on the continuous localization of the industrial trucks 20 and their identification identifiers. The digital twin 1 can also be generated continuously, which may include, for example, updating the digital twin 1 in real time based on the continuously received optical data. Accordingly, the process 300, with its steps 301, 302, and 304, can be repeated continuously and in real time. Step 303 can be performed once for each of the industrial trucks 20 or multiple times to verify that the correct, unique identification identifier has been assigned to each industrial truck 20.
[0087] Advantageously, the digital twin 1 thus knows not only the positions of the industrial trucks 20, but also their identification identifiers, in particular IP addresses. In the procedure 200, the control commands can therefore be transmitted unambiguously and specifically to and executed by those industrial trucks 20 that exhibit a corresponding individual collision risk in a given situation.
[0088] It should be further noted that "comprehensive" and "comprising" do not exclude other features or steps, and the indefinite articles "a" or "an" do not exclude a plurality. It should also be noted that features or steps described with reference to one of the above embodiments may also be used in combination with other features or steps of other embodiments described above. Reference numerals in the claims are not to be considered limitations.
[0089] As used herein, the expression "indicative of" can mean, for example, "reflective" and / or "comprehensive". Accordingly, a unit, element and / or step referred to here as "indicative of [...]" may be used synonymously or interchangeably with one, two, or all of the aforementioned units, elements and / or steps "comprehensive [...]" and the aforementioned unit, element and / or step "reflective [...]".
[0090] Furthermore, expressions such as "based on," "related," "associated," and similar expressions are not to be understood exclusively in relation to the units, elements, and / or steps to which they refer, unless otherwise specified. Instead, unless otherwise specified, these expressions are to be understood as meaning that, for example, a unit, element, or step to which one of these expressions or a similar expression refers, e.g., "based on" one or another unit, element, or step, does not preclude the possibility that the unit, element, or step in question may also be "based" on a different unit, element, or step than the one to which it refers.
[0091] Any designation of procedures, steps, and elements as first, second, etc., as indicated herein, serves only to make the procedures, their steps, and elements referable and distinguishable from one another. The designation of methods, steps, and elements in no way constitutes a limitation of the scope of this disclosure. For example, if a third step of a procedure is described in this disclosure, a first or second step of the procedure need not exist and certainly need not be performed before the third step, unless it is expressly stated that they are required per se or prior to the third step. Furthermore, the presentation of procedures or steps in a particular order is intended only to facilitate understanding of this disclosure and in no way constitutes a limitation of the scope of this disclosure.In general, the procedures and steps can be carried out in any conceivable order unless an expressly prescribed order is stated. In particular, the terms "first," "second," "third," or "a), "b)," "c)," and the like are used in the description and in the claims to distinguish similar elements and not necessarily to describe a sequential or chronological order. It is to be assumed that the terms used in this way are interchangeable under suitable circumstances and that the embodiments of the disclosure described herein may also function in orders other than those described or illustrated herein.
Claims
S13565WO SENTICS GMBH REQUIREMENTS 1. Method (300) for generating a digital twin (1) which maps at least one industrial truck (20) in an intralogistics environment (10) based on a visual localization of the at least one industrial truck (20) in real time, wherein the method (300) comprises: - continuous reception of optical data from the intralogistics environment (10) in real time from at least one or more optical sensors (51) in the intralogistics environment (io); - continuous localization, based on the optical data, of at least one industrial truck (20) in the intralogistics environment (10); - Determining a unique identification identifier of the at least one continuously located industrial truck (20); and - Generating, based on the continuous localization of at least one industrial truck (20) and its identification identifier, the digital twin (1).
2. Method (300) according to claim 1, wherein a processing rate of the optical data for continuously localizing the at least one industrial truck (20) and / or for generating the digital twin (1) is controlled depending on one or more parameters of the at least one continuously localized industrial truck (20).
3. Method (300) according to claim 1 or 2, wherein the method (300) further comprises: - Providing at least one control command for transmission to the at least one industrial truck (20) by reference to its address, wherein the at least one industrial truck (20) is equipped to receive and export the at least one control command.
4. Method (300) according to one of the preceding claims, comprising determining the unique identification identifier of the at least one continuously localized industrial truck (20): - Detect, based on the optical data, a visual reference mark (29a) on the at least one continuously localized industrial truck (20); and S13565WO SENTICS GMBH Determine, based on the detected reference mark (29a), the unique identification identifier of the at least one continuously localized industrial truck (20).
5. Method (300) according to one of the preceding claims, comprising determining the unique identification identifier of the at least one continuously localized industrial truck (20): - Recognizing, based on optical data, a barcode (29b) on at least one continuously localized industrial truck (20); and - Determine, based on the detected barcode (29b), the unique identification identifier of at least one continuously located industrial truck (20).
6. Method (300) according to claim 5, wherein determining, based on the recognized barcode (29b), the unique identification identifier of the at least one continuously localized industrial truck (20) comprises: - Determining the orientation of the barcode (29b) in the optical data; and - Reading the barcode (29b) according to the determined orientation, where determining the orientation of the barcode (29b) (i) rotating at least the barcode (29b) and detecting vertical or horizontal lines, and / or (ii) rotating the barcode (29b) and recognizing the aspect ratios of the barcode (29b) includes.
7. Method (300) according to one of the preceding claims, comprising determining the unique identification identifier of the at least one continuously localized industrial truck (20): - Detect, based on the optical data, a light signal at an optical lighting system (29c) of at least one continuously localized industrial truck (20); and - Determining, based on the detected light signal, the unique identification identifier of the at least one continuously localized industrial truck (20). S13565WO SENTICS GMBH 8. Method (300) according to claim 7, wherein the light signal is encoded by different luminous colors, luminous intensities and / or luminous durations.
9. Method (300) according to claim 7 or 8, wherein the method (300) further comprises: - Providing at least one lighting command for transmission to the at least one continuously localized industrial truck (20), wherein the optical lighting system (29c) of the at least one industrial truck (20) is configured to receive and execute the at least one lighting command to generate the light signal.
10. Method (300) according to one of the preceding claims, comprising determining the unique identification identifier of the at least one continuously localized industrial truck (20): - Determining radio data indicative of at least one radio signal from a radio module (23) of the at least one continuously localized industrial truck (20) that has been received by a radio infrastructure of the intralogistics environment (10); and - Determine, based on the radio data, the unique identification identifier of at least one continuously located industrial truck (20).
11. Method (300) according to one of the preceding claims, comprising determining the unique identification identifier of the at least one continuously localized industrial truck (20): - Determining environmental sensor data that are indicative of environmental detection by an environmental sensor system of the at least one continuously localized industrial truck (20); and - Determine, based on the environmental sensor data, the unique identification identifier of at least one continuously located industrial truck (20).
12. Method (300) according to one of the preceding claims, comprising determining the unique identification identifier of the at least one continuously localized industrial truck (20): - Determining sensor data indicative of acceleration, velocity and / or steering angle of the at least one continuously localized industrial truck (20); and S13565WO SENTICS GMBH Determining, based on the sensor data and the optical data, the unique identification identifier of the at least one continuously localized industrial truck (20).
13. Method (300) according to any one of the preceding claims, wherein the method (300) further comprises: - Determining o Sensor data indicative of acceleration, velocity and / or steering angle of the at least one continuously localized industrial truck (20), o Environmental sensor data indicative of environmental detection by an environmental sensor system of the at least one continuously localized industrial truck (20), and / or o Radio data indicative of at least one radio signal from a radio module (23) of the at least one continuously localized industrial truck (20) which has been received at a radio infrastructure of the intralogistics environment (10); and - Verify, based on the optical data as well as the sensor data, the environmental sensor data and / or the radio data, the determined unique identification identifier of the at least one continuously located industrial truck (20).
14. Data processing system (100) which is set up to carry out the method (300) according to any of the preceding claims.
15. One or more computer program products comprising instructions which, when executed by one or more computers (110, 120, 130), cause the execution of the method (300) according to any one of claims 1 to 13.
