Object handling in the absolute coordinate system

Abstract

A method for controlling the handling of another object (90), the other object (90) being handled at a site (10) by at least two different workstations (51, 52) and the other object (90) being moved between at least two different workstations, determining a site floor plan of the site (10), the site floor plan indicating at least two workstations at predefined positions in the site floor plan, determining the trajectory of a first object (90) moving in the site (10) in the absolute coordinate system of the site, inferring the absolute positions of at least two workstations (51, 52) in the absolute coordinate system from the determined trajectory of the first object, and using the absolute positions of the at least two workstations (51, 52) to control the handling of at least one other object handled by the at least two workstations.

Description

Technical Field

[0001] The present application relates to a method for controlling the handling of an object, the object being handled at a site by at least two different workstations and the object being moved between at least two workstations. Furthermore, a corresponding object control entity, a computer program, and a carrier including the computer program are provided. Background Art

[0002] Indoor positioning systems are used for various tasks in the industrial field, such as high-value asset or product tracking, or ensuring safety by tracking vehicles and humans moving close to each other. These positioning systems may be based on WiFi, Bluetooth, UWB, or 3GPP (LTE, 5G) radio technologies, but most of them require an infrastructure equipped with anchor devices having a known absolute position and tag devices having an unknown position to be measured and used in industrial control command generation. In such a system, the goal is to calculate the absolute position of the tag device in order to generate meaningful control commands. This is accomplished by using some radio communication and measurements, such as directly measuring the distance between the anchor and the tag device (e.g., UWB-based systems) or measuring the Received Signal Strength Indicator (RSSI) (e.g., Bluetooth-based solutions).

[0003] Mapping of routes and device positioning are mainly known to be used by autonomous vehicles to navigate through a factory using special guides that are predefined dedicated trajectories for them to follow.

[0004] Industrial control systems can use location information at different levels of control logic. For example, in the case where a robotic arm is instructed to pick up a moving object, the map location will be used to define the high-level command. Here, the accuracy of the target point to be reached from the current arm position can directly affect the feasibility of the task. On the other hand, the internal command logic of the robotic arm can be deployed through communication between a dedicated arm controller unit and the arm joints. A feasible trajectory of the arm endpoint will be calculated to reach the target, and it will be described as a series of precise high-frequency (e.g., every 40 ms) joint acceleration commands.

[0005] In addition, continuous monitoring can provide feedback for various use cases, where "location" itself will create the necessary insights, such as object-object relationship information, 3D trajectories in space, occupancy information of workstations, autonomous vehicle route planning, etc. In most of these use cases, spatial resolution and the matching of the location with the absolute site map and with production line components (e.g., workstations and site-specific equipment) are necessary.

[0006] With the advancement of Industry 4.0 solutions, flexible manufacturing sites are becoming a more efficient way to use these sites, where production line components are frequently reorganized according to a newly planned layout that dynamically changes. In today's reorganization process, precise maps are used to create the new planned layout, and production lines are built based on cartography with the precision necessary for their operation. There are some cases where the relative positions of production line components are important, but the placement of production lines within the site can tolerate a greater mismatch without losing functionality.

[0007] Although many solutions use ML (Machine Learning) to create outdoor or indoor maps of the environment or perform classical map matching to improve the accuracy of position measurements, here we are focusing on more specific industrial problems.

[0008] As Figure 1 shown, indoor localization systems use anchor devices with known positions to calculate the position of the tag device 30. For industrial sites, the installation of a permanent system of anchor points 20 will use an absolute site coordinate system. The position of the tag device will be calculated with high precision in this coordinate system.

[0009] On the other hand, use cases based on control are usually defined locally with respect to equipment (such as robotic arms). This means that creating a new site configuration according to a given site plan will only require considering local precision (e.g., within the conveyor belt area), and the factory can remain fully functional without considering the overall construction precision (e.g., the placement of two independent conveyor belts relative to each other). As Figure 2 shown, this can frequently result in a mismatch between the floor plan 10 and the physical positions of the equipment ensemble. For example, a single production line 12 with components or workstations 51 and 52 (such as robotic arms) and a conveyor belt 60 can consider the internal placement with high precision, but each production line can be placed together offset by a few decimeters from their corresponding site plan positions. This is shown in Figure 2 . If a mismatch exists between the cloud map and the physical placement of production line components 71 to 74, a false view will be created, and localization problems or manufacturing faults will be falsely reported.

[0010] The mismatch between the map 10 and the physical positions 71 to 74 prevents the direct use of position measurements as an input to control functions - even if the localization accuracy itself is suitable for the control function. Triggering control commands based on position and motion measurements that include system-level deviations can lead to malfunctions, false alarms, missed alarm situations, and will introduce overall safety hazards.

[0011] With the advent of flexible installation, all components reorganized according to a new floor plan will repeatedly become uncertain parts of an analysis system that relies on indoor positioning. For a complete or partial reconstruction of a flexible site, precise reconstruction and global calibration can be expensive and time-consuming, resulting in extended production downtimes. Summary of the Invention

[0012] Accordingly, there is a need to overcome the problems mentioned above and provide the possibility of precisely positioning and controlling objects moving in a site with high precision.

[0013] This need is met by the features of the independent claims.

[0014] Further aspects are described in the dependent claims.

[0015] According to a first aspect, a method for controlling the handling of additional objects is provided, the additional objects being handled at a site by at least two different workstations and moved between the at least two workstations. An object control entity controls the handling including determining a floor plan of the site, wherein the floor plan indicates at least two workstations at predefined positions in the floor plan. Further, a trajectory of a first object moving in the site is determined in an absolute coordinate system of the site. Additionally, the absolute positions of the at least two workstations are inferred from the determined trajectory of the first object in the absolute coordinate system and the absolute positions of the at least two workstations are used to control the handling of at least one additional object handled by the at least two workstations.

[0016] Furthermore, a corresponding object control entity is provided, the corresponding object control entity being configured to control the handling of objects, wherein the object control entity includes a memory and at least one processing unit, wherein the memory contains instructions executable by the at least one processing unit. The object control entity is operative to work as discussed above or as discussed in further detail below.

[0017] Alternatively, an object control entity is provided that is configured to control the handling of objects by at least two different workstations at a site, where the objects are moved between the at least two different workstations, and where the control entity includes a first module configured to determine a site floor plan of the site, where the site floor plan indicates at least two workstations at predefined locations in the site floor plan. A second module is provided that is configured to determine the trajectory of a first object moving in the site in an absolute coordinate system, and a third module is provided that is configured to infer the absolute positions of at least two workstations in the absolute coordinate system based on the determined trajectory of the first object. The control entity includes a fourth module configured to use the determined absolute positions of the at least two workstations to control the handling of at least one additional object handled by the at least two workstations.

[0018] Furthermore, a computer program including program code is provided, where the execution of the program code causes at least one processing unit to perform the method as discussed above or as further explained in detail below.

[0019] In addition, a carrier including the computer program is provided, where the carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

[0020] It is to be understood that the features mentioned above and the features yet to be explained below can be used not only in the indicated respective combinations, but also in other combinations or independently, without departing from the scope of the invention. Unless explicitly mentioned otherwise, the features of the aspects mentioned above and the embodiments described below can be combined with each other in other embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The foregoing and additional features and effects of the application will become apparent from the following detailed description when read in conjunction with the drawings, where like reference numerals refer to like elements.

[0022] Figure 1 A schematic view of an industrial site with a localization system known in the art is shown.

[0023] Figure 2 A schematic view of a site floor plan is shown, where the absolute position of an object is incorrectly indicated due to the low accuracy of the site floor plan.

[0024] Figure 3 A schematic architectural view of an object control entity configured to use a modified site floor plan in an absolute coordinate system to control the handling of objects is shown.

[0025] Figure 4Shows a more detailed view of the system, where tags attached to different components send location information to an object control entity that determines a modified site floor plan using an absolute coordinate system for controlling the object.

[0026] Figure 5 Shows a more detailed schematic view of an industrial site, where the position of a moving object is correctly indicated based on the modified site floor plan.

[0027] Figure 6 Shows a schematic example view of a flowchart of a method executed at an object control entity when disposing of an object based on an absolute position.

[0028] Figure 7 Shows an example schematic representation of an object control entity configured to dispose of a moving object in a site based on an absolute coordinate system.

[0029] Figure 8 Shows Figure 7 Another example schematic representation of the object control entity shown in Detailed Description of the Invention

[0030] Hereinafter, embodiments of the invention will be described in detail with reference to the drawings. It is to be understood that the following description of the embodiments is not to be taken in a limiting sense. The scope of the invention is not intended to be limited by the embodiments described hereinafter or by the drawings, and the embodiments described hereinafter or the drawings are merely illustrative.

[0031] The drawings are to be regarded as schematic representations and the elements illustrated in the drawings are not necessarily shown to scale. Instead, the various elements are represented such that their general function becomes apparent to those skilled in the art. Any connection or coupling between the functional blocks, devices, components of the physical or functional units shown in the drawings and described hereinafter can also be achieved by indirect connection or coupling. The coupling between components can be established by wired or wireless connections. The functional blocks can be implemented in hardware, software, firmware, or a combination thereof.

[0032] Hereinafter, a solution is proposed that solves a specific industrial problem. An entity and method for automatic reverse map matching are provided to enable precise positioning systems to be integrated into industrial site monitoring and control functions. The entity and method rely on a site map. Once a mismatch occurs between the site map and the actual physical layout of a production line including workstations and conveyors, compatibility issues between the positioning system and the control function will occur. Hereinafter, a mechanism for automatically inferring the correction of such a mismatch is proposed.

[0033] The proposed method and apparatus use the localization of a site and sensor measurement traces as inputs to a pattern recognition model. The model may have been trained to automatically discover mismatches between a site floor plan or map information, also referred to hereinafter as a floor plan, and the physical placement of production line elements such as workstations and conveyors. This dynamically inferred parameter is then used to correct control messages based on absolute position. Once a match is available, the solution enables use cases, typically defined in a local coordinate system such as relative to a robotic workstation, to directly or via a common cloud service use measurements from an absolute precision localization system for an entire site or building. However, it should be understood that the method and apparatus may be used outside of a cloud environment, where steps are performed in a single entity or a group of connected entities.

[0034] In the art, navigation algorithms that use satellite-based systems for outdoor navigation are known. Here, a simple map matching problem occurs where the receiver of satellite signals is randomly placed somewhere next to a given route on a map with uncertain measurements. In this case, given the history and properties of map elements such as roads, sidewalks, buildings, a map matching algorithm finds the most appropriate place on the map for the signal receiver device. It matches the uncertain position to increasingly certain map elements. In the case of an industrial site, mainly in indoor locations, the situation is reversed, where the localization of the receiver device has higher precision, but the site map itself contains greater uncertainty.

[0035] In a factory setup such as Figure 1 shown, different time scales for fixed and non-fixed positions can be identified. Workstations such as Figure 2 workstations 51 or 52 can be considered fixed, while tags attached to personal objects will represent a higher availability of position during the map matching process. The trace of an object is defined by characterizing the set of positions and sensor measurements of the tag over space and time. Tags are attached to humans or objects.

[0036] Figure 4 An example schematic representation of a site is shown, where the physical infrastructure is provided and stored at the manufacturing site shown in the Figure 4 lower part, while in an embodiment of the cloud infrastructure shown in the Figure 4 upper part, the infrastructure performs absolute industrial site localization measurements, map matching, and enhanced localization. In Figure 4The physical system shown below the dashed line in [Figure] includes tags 30, 35, or 40 that are positioned by means of distance measurements to an anchor point 20. Tags such as tag 35 can be attached to a person, tags can be attached to an object passing through a production line as shown by tag 30, or tags can be attached to a production line element as shown by tag 40. The anchor point 20 has a known position, and the anchor point performs distance measurements via the tags and communicates with an entity 100 that is configured to control the handling of the object, where the object itself is not shown in [Figure], but the object itself can be any object passing through the production site. The anchor point 20 sends the raw measurement data to the entity 100 and receives tag commands and anchor commands that are forwarded to the tags. Figure 4 The industrial site localization module 101 shown in [Figure] controls the physical localization infrastructure and creates the absolute tag positions in real time. The reverse map matching module 102 receives these positions and can be used for direct map matching of production line equipment if the direct position of a production line element with an attached tag such as tag 40 is available. If they are not available, the position stream obtained via tag 30 together with the position stream of the tag attached to a person (tag 35) may result in unknown device positions such as the positions of workstations 51 and 52 shown in [Figure] and the production line route in the absolute industrial site map generated as a modified site floor plan. The localization intelligence module 103 can then use the correct position information and can provide the positions of the involved entities in the absolute coordinate system to the modified site floor plan, and the information is transmitted to the monitoring and control function 104, which transmits control commands to workstations 51 and 52 to control the operation of the workstations. Figure 4 The industrial site localization module 101 shown in [Figure] controls the physical localization infrastructure and creates the absolute tag positions in real time. The reverse map matching module 102 receives these positions and can be used for direct map matching of production line equipment if the direct position of a production line element with an attached tag such as tag 40 is available. If they are not available, the position stream obtained via tag 30 together with the position stream of the tag attached to a person (tag 35) may result in unknown device positions such as the positions of workstations 51 and 52 shown in [Figure] and the production line route in the absolute industrial site map generated as a modified site floor plan. The localization intelligence module 103 can then use the correct position information and can provide the positions of the involved entities in the absolute coordinate system to the modified site floor plan, and the information is transmitted to the monitoring and control function 104, which transmits control commands to workstations 51 and 52 to control the operation of the workstations. Figure 2 The localization intelligence module 103 can then use the correct position information and can provide the positions of the involved entities in the absolute coordinate system to the modified site floor plan, and the information is transmitted to the monitoring and control function 104, which transmits control commands to workstations 51 and 52 to control the operation of the workstations.

[0037] The analysis database 105 can collect the results of the modified site floor plan and can collect parts of the production line route. The modeling of the modified map by different elements of the production line can use a hierarchical segmentation of different functional units such as regions, production lines, and production line segments as discussed below in relation to Figure 5 In addition, the active elements of the production site can be continuously updated based on the current descriptors of the industrial site.

[0038] Traces collected by the anchor points are defined by characterizing the positions of tags over space and time and a set of sensor measurements. The collection time required to create the trajectories that are input to the reverse map matching can depend on the quality of the regular data set. A longer measurement period can reduce the overall uncertainty, and the tag positioning system can provide the necessary information even from a shorter measurement time. The distribution of the recorded movement types should be covered by representations of different scenarios. This can mean the following: A longer measurement period will only reduce the uncertainty in various scenarios if those scenarios are represented in the measurement and the distribution is not concentrated on the repetition of a restricted mode of operation of a complex system. Therefore, it is preferred to cover the diversity of possible scenarios in these records.

[0039] In one scenario, a production line such as Figure 5 the production line 13 shown in Figure 4 has local tags such as tag 40 that directly measure the position of the conveyor or production line, and this position can be used to calculate the differential translation and rotation of an object for matching the site plan with the absolute site coordinate system. Since there is still some uncertainty in the determined position, additional optimization steps can be included to minimize the overall error. The distance measurement between the anchor point and the tag device can include errors. Therefore, it may be necessary to further reduce the uncertainty by considering multiple anchor point measurements and discarding outliers or by considering other known fixed distances, for example, between two tag devices.

[0040] In an alternative step of the processing method, the entire production workshop is divided into smaller parts with fixed relative positions to utilize the additional information that they are fixed to each other or production line segments such as Figure 5 segments 61, 62, and 63 in

[0041] Once these segments 61 to 63 are identified, they can be processed separately using an error minimization algorithm.

[0042] 1. In the initial stage, the known workstation types are recorded by trace features, or this information is used from the known available information. The control of the device handling is performed at the workstation. The scope of the task can range from robot arm movement to manual handling or painting work. Depending on the manufacturing process, the information collected here creates a multi-dimensional trace that includes the movement and other sensor information during the time the object spends at the station. These traces can be classified and the workstations of a given type can be identified.

[0043] 2. Additionally, when an object is being handled by a workstation and other elements such as Figure 4 and Figure 5 the elements shown in are present on the site, data from tags attached to the object and / or to the personnel working on the site are used to record the trajectory of the object.

[0044] 3. In a third step, traces of workstations from the factory are identified by analyzing the movement of the trajectory and optionally by analyzing the patterns of other sensors of the object. Here, a clustering algorithm can be used, applying the clustering algorithm on the multi-dimensional time series trajectory or on different time series segments collected as in step 2.

[0045] 4. In an optional step, a hierarchical clustering method is used to identify higher-level sequences of workstation traces in the record. Hierarchical clustering can be performed in various dimensions. One option would be to create a hierarchy of units following the manufacturing process. For example, low-level movement patterns would constitute workstation patterns, a given series of workstation patterns would constitute a production line pattern of a given type, and so on.

[0046] 5. In a further step, a match is performed between the workstation trace patterns or their optional sequences and a known site floor plan of the production site.

[0047] 6. Here, coordinate system correction in the floor plan is performed by creating the differential translations and rotations necessary for matching the floor plan with the inverse map of the absolute site coordinate system. In this step, a modified floor plan is generated from the floor plan including all positions in the absolute coordinate system. Accordingly, entity 100 has the absolute position of the moving object as determined by the trajectory and the modified floor plan now available, the modified floor plan having the positions of the workstations and the object in the absolute coordinate system.

[0048] 7. Without redefining the implemented control process, the calculated correction itself can be adapted to each use case in the manufacturing control process to achieve the corresponding necessary positioning accuracy. During the handling of an object in the manufacturing control process, various levels of positioning accuracy are required. If the accuracy reaches a given level, there is no need to redefine the control process itself, but for example, parameters used in their algorithmic coding can be modified by including the calculated positioning mismatch correction values. This can be different for each of the use cases or processes involved.

[0049] In steps 3 and 5 mentioned above, since both the known site floor plan and the trace or trajectory measurements are usually noisy and may have additional local and global distortions such as distortions in the absolute site coordinate system, optimization methods can be performed. Here, it is possible to apply numerical methods like gradient descent via simulated annealing to find the global minimum of different functions.

[0050] Step 7 can be solved dynamically and can be used as a dynamic cloud service. This step may be required for fully integrating the localization system into the manufacturing process with the same ease of use.

[0051] In an Industry 4.0 setting where the mobility time scale of fixed workstations is significantly reduced, the method can be repeated for each new piece of equipment in the site. Contrary to the one-way operation mode required in traditional settings where the setup of the site is not changed, this can be considered a continuous operation mode of reverse map matching.

[0052] In a dynamic factory setting, precise planning is often replaced by loosely located functional areas. In such a scenario, the proposed method can be used to find the site floor plan of its active components such as workstations by matching the trajectories of the components with the known trace features.

[0053] In one use case, it is possible to consider the localization within a single robot cell and only use relative positions such as the center of the robot arm. Here, every other aspect discussed above remains the same, but higher precision may be required for the smaller space to be matched. Here, different workstations are related to different joints of the robot arm.

[0054] Figure 6 is shown Figure 4 A simplified flowchart of the two main components shown in shows where different steps are performed. The industrial site localization module 101 uses the gateway for command and measurement communication with the anchors. Accordingly, the gateway receives distance measurements from the anchors and then processes the raw measurements, and also uses, for example, the known positions of the anchors to convert the measured tag-anchor distances into the absolute positions of the tags. In the reverse map matching module 102, the absolute tag positions are used to match the equipment on the production line with the absolute site coordinate system. Based on the history and based on the trajectory collection, the analyzer can determine when an object is being handled by a workstation or when an object is being handled by a conveyor. In terms of the machine learning framework, typically the system and method are trained, for example, with measurements, and once ready, the resulting model is used for automatic inference. In the case of mismatch inference, the measurement traces are the input, and the inference uses the trained model to determine the correction values for the mismatched localization to be used in a given control process. The overall model training logic is detailed in steps 1 - 6 of the factory floor plan matching details.

[0055] Figure 5 Shows a schematic view of the site after the site floor plan has been corrected to a modified site floor plan, where the positions of objects such as positions 81, 82, 83 or 84 are correctly indicated as being located on the conveyor belt. The positions of workstations 51 and 52 as well as the position of object 90 are all provided in an absolute coordinate system. Accordingly, it is possible to control the handling of object 90 by conveyor 60 or workstations 51 or 52 using the modified site floor plan with the provided absolute coordinate system. Figure 5 Furthermore, different parts 61 to 63 of conveyor 60 are shown.

[0056] Figure 6 Outlines some of the steps performed at the object control entity. In step S61, the site floor plan is determined. This site floor plan is not provided in an absolute coordinate system such that it may not be used to control objects handled on the site. In step S52, at least the position tracked by a tag attached to the object is used to determine the trajectory of the first object. From the determined trajectory, it is possible to infer the absolute position of the workstation where the object has been manipulated or handled in some way. In this context, prior knowledge of the types of workstations provided on the site can be used. For example, it may be known that a workstation performs a pure translational movement or it may be known that a workstation is handling an object in some way by clamping the object and moving it to a different position. These movements can be easily identified in the trajectory of the object provided with high precision. In step S63, the absolute position of the workstation is determined based on the detection of certain patterns in the movement of the object. Since the geometry of the workstation is precisely known, it is possible to determine the absolute position of the workstation based on the evaluation of the position and the positions in the vicinity of the workstation.

[0057] In step S64, when the absolute position of the workstation is known, it can be used to control the handling of the same or additional objects handled at the corresponding workstation.

[0058] Figure 7A schematic architectural view of an object control entity that performs the above-discussed handling of an object based on the absolute position of a workstation is shown. Entity 100 includes an interface or input-output 110 that provides for the transfer of user data or control messages to other entities. Interface 110 is particularly configured to receive the trajectory of the object, and after the absolute position and the modified site floor plan have been calculated, interface 110 can output the control commands required to handle the object to the workstation. Entity 100 further includes a processing unit 120 that is responsible for the operation of entity 100. Processing unit 120 may include one or more processors and may execute instructions stored on one or more memories 130, where the memories may include read-only memory, random access memory, mass storage devices, hard disks, and the like. The memories may further include appropriate program code to be executed by processing unit 120, so that it will implement the above-described functionality related to entity 100. Entity 100 may be implemented as a single standalone entity, or entity 100 may be implemented in a cloud environment, where the processing unit, the memories, and the interface are distributed in the cloud environment.

[0059] Figure 8 Another schematic architectural view of entity 300 is shown, which is configured to control the handling of an object based on the absolute position of a workstation and based on a modified site floor plan. Module 300 includes a first module 310 that is configured to determine a site floor plan that includes the positions of the workstation and other equipment on the site. Further, a module 320 is provided that is configured to determine the trajectory of an object in an absolute coordinate system based on raw measurements received from the moving object. A third module 330 is provided that is configured to determine the absolute position of the workstation based on an analysis of the trajectory of the object. When the absolute position of the workstation is known, these absolute positions can be used by module 340 to handle the object by the workstation and to control the handling by using control commands.

[0060] From the above, some general conclusions can be drawn.

[0061] When the absolute positions of at least two workstations are known, it is possible to match the predefined positions of the workstations in the site floor plan with the absolute positions and to determine a modified site floor plan of the site based on the match. This modified site floor plan then includes the absolute positions of at least two workstations in an absolute coordinate system. The modified site floor plan and the absolute positions of the components on the site can be used to control the handling of one or several additional objects.

[0062] A conveyor 60 can be provided to move an object from one workstation in the workstations to another workstation in at least two workstations. When determining the trajectory of the first object, an absolute conveyor trajectory including the position of the object moving between two workstations in the absolute coordinate system can be determined. When determining the trajectory, an absolute workstation trajectory including the position of the object during manipulation at the corresponding workstation in the absolute coordinate system can be determined for each workstation in the workstations. The conveyor trajectory and the absolute workstation trajectory can then be used to control the handling of at least one additional object.

[0063] In addition to the absolute workstation trajectory, the absolute site plan can further include the absolute conveyor trajectory.

[0064] The trajectory of the first object can be determined based on a localization tag attached to the object. However, other options for precise determination of the position can be used.

[0065] The absolute workstation trajectory can be determined by analyzing the determined trajectory of the first object and by inferring the movement caused by the workstations in the trajectory of the first object. Each workstation has its own specific movement when handling the object. One workstation can be a test workstation where the object is located at a defined fixed position for a predetermined amount of time; where another workstation can be a workstation where parts are added to the object or where parts are removed from the object such that a specific movement is performed with the object. Since these movements are known from the trajectory in the 3D environment, it is possible to clearly separate the trajectory initiated by the manipulation of the workstation from other trajectories.

[0066] In addition, a control command for at least one additional object can be determined, and the control command can be used at one workstation in the workstations to manipulate at least one additional object. In addition, the control command can be transmitted to the workstation for the manipulation of additional objects to be handled by the corresponding workstation.

[0067] At least two workstations 51 and 52 can be part of a production line of the site, where the trajectory of the object is analyzed and different line segments are determined within the production line, where the relative distance between different segments is determined in the absolute coordinate system, and the relative distance between different segments is used in controlling the handling of the object. For example, in Figure 4 the embodiment shown, the production line can be divided into different line segments such as the first segment 61, where the object located on the conveyor belt is conveyed in one direction. Another segment can be a segment 62 perpendicular to the first segment, and another line segment can be a segment 63 that is perpendicular to the segment 62 and parallel to the segment 61. The relative distance between different segments can be determined in the absolute coordinate system and the relative distance between different segments can be used in the control.

[0068] Accordingly, for such asFigure 5 It is possible to determine the conveyor trajectory and the absolute workstation trajectory for each line segment of the different segments 61 to 63 shown.

[0069] In addition, it is possible to determine the trajectory of a user moving on the site. The determined trajectory of the user can also be considered to determine the absolute position of the workstation.

[0070] For the determination of the absolute positions of at least two workstations, a clustering mechanism can be used and applied to the determined trajectory of the first object.

[0071] In addition, an optimization process can be used, which finds a minimum or maximum value in order to obtain the absolute workstation trajectory. Since each of the determined positions of the first object is determined with a certain precision, the determination of the absolute position can include an optimization algorithm that finds an absolute minimum or maximum value in order to determine the absolute workstation trajectory.

[0072] The solutions discussed above provide a mechanism that is capable of filling the gap between industrial location-based control and precision positioning systems. Both systems rely on an industrial site map or a site floor plan. Once there is a mismatch between the site floor plan and the true physical layout of the workstations or production lines, the mismatch causes the positioning system to be incompatible with the control function. The solutions discussed above automatically correct this mismatch and determine the modified site floor plan.

[0073] To solve this inverse map matching requirement and to integrate the solution into industrial site monitoring and real-time manufacturing control functions, a system of physical components and cloud components can be used.

[0074] The solutions above automatically and dynamically generate the necessary corrections to the position to avoid the expensive and time-consuming process of repeatedly required site calibration.

[0075] The method identifies the exact positions in the absolute coordinate system of the feature positions and matches them with the coordinates (i.e., map coordinates) used by the entity controlling the different components.

[0076] In the case where a floor plan is available, for each production line element, a correction is calculated from the input of the sensor and absolute position measurement traces of the people and objects to be tracked.

[0077] Machine learning algorithms can be used for pattern recognition for the construction of an analysis knowledge base, and the analysis knowledge base can be dynamically used to calculate the corresponding and inverse map matching correction factors for the local control function.

[0078] Numerical optimization methods can be used to reduce uncertainty and utilize more advanced factory floor plan units in the matching algorithm. The above solutions can also be used in flexible industrial sites, where continuous monitoring of the floor plan or site plan can be used for different use cases.

[0079] In a flexible site, new installations of production lines become possible. Additionally, reorganization of the site can be achieved such that the modified floor plan is used to control the movement of objects. The solutions discussed above provide the following options:

[0080] The correction of a mismatched floor plan that would otherwise render the localization system unusable can now be used in solutions that rely on tasks defined on the map, such as geofence alerts and operation mode setting for certain geographical locations. The solution provides high precision in the precise localization system and allows for the integration of the absolute localization system into control functions, where commands and algorithms can rely on a near-real-time measurement stream with dynamic correction.

[0081] Continuous manufacturing using an accurate map of the production line becomes possible without additional, repeated calibration of the localization system or the production line. The modified floor plan can be used to map new equipment and can be used in the manufacturing process. No additional accuracy requirements are needed for the absolute position of the production line. Accordingly, a cost-effective way for mapping production equipment is provided without additional effort. The method can use either direct measurements of the workstations (if available) or indirect position information inferred from the trajectories determined for objects moving in the site. Since this accurate floor plan is not required upfront in generating the accurate floor plan, flexibility is increased.

Claims

1. A method for controlling the handling of additional objects, the additional objects being handled at a site (10) by at least two different workstations (51, 52) and the additional objects being moved between the at least two different workstations, - determining a site floor plan of the site (10), the site floor plan indicating at least two workstations at predefined positions in the site floor plan, - determining a trajectory of a first object (90) moving in the site (10) in an absolute coordinate system of the site, - inferring absolute positions of the at least two workstations (51, 52) in the absolute coordinate system from the determined trajectory of the first object, - using the absolute positions of the at least two workstations (51, 52) to control the handling of at least one additional object handled by the at least two workstations; Among them, providing a conveyor (60) to move the first object from one workstation (51) of the at least two workstations to another workstation (52) of the at least two workstations, wherein determining the trajectory includes determining an absolute conveyor trajectory including positions of the first object moving between the at least two workstations in the absolute coordinate system and includes determining an absolute workstation trajectory including positions of the first object (90) in the absolute coordinate system during manipulation at a corresponding workstation for each of the at least two workstations, and wherein the absolute conveyor trajectory and the absolute workstation trajectory are used to control the handling of the at least one additional object.

2. The method according to claim 1, further comprising: - matching the predefined positions of the at least two workstations (51, 52) in the site floor plan with the absolute positions of the at least two workstations, - determining a modified site floor plan of the site (10) based on the matching, the modified site floor plan including the absolute positions of the at least two workstations in the absolute coordinate system, - using the modified site floor plan to control the handling of at least one additional object.

3. The method according to claim 1, wherein Determining an absolute site floor plan including the absolute conveyor trajectory.

4. The method according to any one of the preceding claims, wherein, Determining the trajectory of the first object (90) based on a localization tag (30) attached to the first object.

5. The method according to any one of claims 1 to 3, wherein, Determining the absolute workstation trajectory by analyzing the determined trajectory of the first object (90) and by inferring workstation-induced movements in the trajectory of the first object.

6. The method according to any one of claims 1 to 3, wherein, Using the absolute position includes - determining a control command for the at least one additional object, the control command being used at one of the at least two workstations to manipulate the at least one additional object, - transmitting the control command to the one workstation.

7. The method according to any one of claims 1 to 3, wherein The at least two workstations (51, 52) are part of a production line, wherein the trajectory of the first object is analyzed and different line segments are determined within the production line, wherein the relative distances between the different line segments are determined in the absolute coordinate system and the relative distances between the different line segments are used in the control of the handling.

8. The method according to claim 7, wherein, The absolute conveyor trajectory and the absolute workstation trajectory are determined separately for each line segment.

9. The method according to any one of claims 1 to 3, further determining a trajectory of a user moving at the site, wherein, The absolute positions of the at least two workstations are determined taking into account the determined trajectory of the user.

10. The method according to any one of claims 1 to 3, wherein, In order to determine the absolute positions of the at least two workstations (51, 52), a clustering mechanism is used on the determined trajectory of the first object.

11. The method according to any one of claims 1 to 3, wherein, An optimization process of finding a minimum or a maximum value is used when determining the absolute workstation trajectory.

12. An object control entity (100) configured to control the handling of additional objects, the additional objects being handled at a site by at least two different workstations and being moved between the at least two different workstations, the object control entity including a memory (130) and at least one processing unit (120), the memory containing instructions executable by the at least one processing unit, wherein, The object control entity is operative to: - determine a floor plan of the site (10), the floor plan indicating at least two workstations at predefined positions in the floor plan, - determine the trajectory of a first object (90) moving in the site (10) in the absolute coordinate system of the site, - infer the absolute positions of the at least two workstations (51, 52) in the absolute coordinate system from the determined trajectory of the first object, - the absolute positions of the at least two workstations (51, 52) are used to control the handling of at least one additional object handled by the at least two workstations; wherein a conveyor (60) is provided to move the first object from one of the at least two workstations (51) to the other of the at least two workstations (52), wherein the object control entity is operative to determine an absolute conveyor trajectory including the position of the first object moving between the at least two workstations in the absolute coordinate system for determining the trajectory, and is operative to determine an absolute workstation trajectory including the position of the first object (90) in the absolute coordinate system during the manipulation at the corresponding workstation for each of the at least two workstations, and is operative to use the absolute conveyor trajectory and the absolute workstation trajectory to control the handling of the at least one additional object.

13. The object control entity according to claim 12, further operative to: - match the predefined positions of the at least two workstations (51, 52) in the floor plan with the absolute positions of the at least two workstations, - determine a modified floor plan of the site (10) based on the matching, the modified floor plan including the absolute positions of the at least two workstations in the absolute coordinate system, - use the modified floor plan to control the handling of at least one additional object.

14. The object control entity according to claim 12, further operative to determine an absolute floor plan including the absolute conveyor trajectory.

15. The object control entity according to any one of claims 12 to 14, further operative to determine the trajectory of the first object (90) based on a local tag (30) attached to the first object.

16. The object control entity according to any one of claims 12 to 14, further operative to determine the absolute workstation trajectory by analyzing the determined trajectory of the first object (90) and by inferring movements in the trajectory of the first object caused by workstations.

17. The object control entity according to any one of claims 12 to 14, further operative to determine a control command for the at least one further object when using the absolute position, the control command being used at one of the at least two workstations to manipulate the at least one further object, and operative to transmit the control command to the one workstation.

18. The object control entity according to any one of claims 12 to 14, wherein, The at least two workstations (51, 52) are part of a production line, the object control entity being operative to analyze the trajectory of the first object, operative to determine different line segments within the production line, and operative to determine the relative distances between the different line segments in the absolute coordinate system used in the control of the handling.

19. The object control entity according to claim 18, further operative to determine the absolute conveyor trajectory and the absolute workstation trajectory separately for each line segment.

20. The object control entity according to any one of claims 12 to 14, further operative to determine the trajectory of a user moving at the site and operative to determine the absolute positions of the at least two workstations taking into account the determined trajectory of the user.

21. The object control entity according to any one of claims 12 to 14, further operative to use a clustering mechanism on the determined trajectory of the first object in order to determine the absolute positions of the at least two workstations.

22. The object control entity according to any one of claims 12 to 14, further operative to use an optimization process in which a minimum or a maximum is found in order to determine the absolute workstation trajectory.

23. A computer program product, comprising program code to be executed by at least one processing unit of an object control entity, wherein, The execution of the program code causes the at least one processing unit to execute the method according to any one of claims 1 to 11.

24. A carrier, comprising the computer program product as described in claim 23, wherein, The carrier is a computer-readable storage medium.

Images