SYSTEM FOR SUPPORTING TEAMWORK USING AUGMENTED REALITY

Description

TECHNISCHES GEBIET

[0001] This description relates to a system and a method for dynamically supporting a team of people when operating (e.g. for setting up or resetting) machines with the aid of Augmented-Reality- (AR) systems. HINTERGRUND

[0002] Augmented-Reality - also as Mixed Reality or holographic computing - can be understood as an overlay of real environment and virtual elements. Such an overlay is possible, for example, using a stereoscopic head-up display (HUD). Such displays can be worn like glasses, with the two lenses serving as transparent head-up displays, which enable the stereoscopic (3D) representation of elements in the field of view of the display, and are therefore also referred to as AR glasses. A commercially available system with a head-up display and integrated processor (CPU) and graphics processor (GPU) and sensors for determining the position of the AR glasses in space is, for example, the HoloLens ®< from Microsoft. Such devices are referred to below as AR terminals.

[0003] The use of AR systems in production and service is well known, for example, in the automotive industry (see, for example, EP 1 295 099 B1). AR systems can support an inspector, for example, in quality control. The AR system can display information in the user's field of vision (e.g., the inspector during quality control), thus expanding their perception. Image, video, or text data, in particular, can be displayed as information. By overlaying the virtual and real environments, the user can easily perform target-actual comparisons.

[0004] The inventors have set themselves the task of better utilizing the potential of modern AR systems in production, for example in situations where several people are to carry out a process on a machine together (as a team), and to dynamically generate the work instructions and adapt them in a step-by-step and dynamic manner.

[0005] US 2016 / 049 006 A1 discloses a system and a method for data acquisition in space and for creating augmented reality content based on a physical object acquired thereby.

[0006] CN 204 406 329 U discloses a system that, after scanning the packaging of a product, displays this product in augmented reality.

[0007] EP 3 086 193 A1 discloses a wearable augmented reality device that displays additional information in the field of vision of a user while working with a tool.

[0008] EP 3 076 253 A1 discloses another augmented reality system that superimposes additional information from an automation system of an industrial plant or part thereof onto an image of an industrial plant.

[0009] US 2014 / 240 349 A discloses a method for presenting parts required to perform a task in an augmented reality interface.

[0010] US 2014 / 204 121 A1 discloses a method for assisting a worker during work on oil production facilities. According to the method, the worker's current location and the work currently being performed are detected. The worker is then presented with appropriate metadata.

[0011] US 2017 / 061 212 A1 discloses a system with detectors that detect a user's movement and the work he or she is performing, and a display that projects work-related information into the user's field of vision. ZUSAMMENFASSUNG

[0012] The stated object is achieved by the system according to claim 1 and the method according to claim 17. Various embodiments and further developments are the subject of the dependent claims. The invention is defined by the claims. Embodiments and configurations not covered by the claims are presented to illustrate the subject matter of the claimed invention and to facilitate its understanding.

[0013] A computer-based system to support teamwork using Augmented Reality (AR). According to one embodiment, the system comprises a production server configured to store the following: the three-dimensional geometry of at least part of a machine and a plurality of tasks, each task being assigned a position in space at which the respective task is to be completed, as well as position-related information relating to the completion of the task. The system further comprises a plurality of user terminals connected to the production server via a computer network, each user terminal being assigned to a member of a team. The production server is further configured to assign a task to each member of the team, the position-related information assigned to the respective task being displayed on the user terminal of the respective person.

[0014] Furthermore, a computer-aided method for supporting teamwork is described. According to one embodiment, the method comprises storing the following data on a production server: the three-dimensional geometry of at least part of a machine and a plurality of tasks, wherein each task is assigned a position in space at which the respective task is to be completed, as well as position-related information relating to the completion of the task. The method further comprises logging on to the production server a plurality of user terminals, which are connected to the production server via a computer network. Each user terminal is assigned to a person in a team. The method further comprises assigning a task from the plurality of tasks to each person in the team, wherein the position-related information assigned to the respective task is displayed on the user terminal of the person in question.

[0015] In one embodiment, the respective user terminal can display the position-related information associated with the respective task at the position in space associated with the task (e.g. by means of Augmented-Reality Technology). A person can also be assigned a group of tasks (from the plurality of tasks), with the user terminal allowing the person to select a task from the group of tasks. The user terminal of the person in question can then display the position-related information associated with a task selected by the person.

[0016] The assignment of a task or group of tasks to a person (to whom a user terminal is assigned) can be dynamic and adapted to the process to be performed by the team. Task assignments can be dynamically updated when the number of participating user terminals changes, and the assignment of tasks can depend on a person's skill level, which represents the respective person's authority or capabilities. KURZE BESCHREIBUNG DER ABBILDUNGEN

[0017] The invention is explained in more detail below using the examples shown in the figures. The illustrations are not necessarily to scale, and the invention is not limited to the aspects shown. Rather, emphasis is placed on illustrating the principles underlying the invention. Regarding the figures: Figur 1 schematically illustrates an example of a system to support a team of (any number of) operators ( Operators ) using networked AR terminals. Figur 2 is an exemplary representation of a machine with position- and context-related representation of information about an operating or display element of the machine. Figur 3 shows, using a schematic sketch, the division of a production line into several system modules and several positions where tasks ( Tasks ) need to be done. Figur 4 shows, using a schematic sketch, the sum of all possible tasks (Tasks T11, T21...) required for the dynamic generation of a process, with the assignment of individual tasks to corresponding positions on the production line. Figur 5 shows the assignment of tasks to different task clusters using a schematic sketch. Figur 6 is a diagram illustrating the automated organization of the activity of a team of operators ( Operators ) when processing tasks. Figur 7 illustrates, using three schematic diagrams (ac), the procedure for assigning a task to an operator who has recently joined a team. Figur 8 uses a schematic diagram to illustrate the procedure for completing a specific task by the respective operator. Figur 9 uses a schematic diagram to illustrate the procedure when an operator leaves the team. DETAILLIERTE BESCHREIBUNG

[0018] Fig. 1 shows schematically a first example of a system for supporting a team of operators ( Operators ) when operating production machines using networked terminals such as Augmented-Reality- End devices (also called AR terminals). The process performed by the team can, for example, be the setup or conversion of one or more machines on a production line. In many areas of industry, packaging machines, for example, must be frequently converted to produce packaged products with different pack sizes or with different packaging variants. Just one example is the automated packaging of pharmaceuticals, where a variety of different packaging is required for many different countries and products. Regular changeover of the packaging machines is necessary, and the efficiency of this changeover process can be an important factor.

[0019] In the example shown, the system can be operated in two modes. The first mode is Authoring- mode and the second operating mode with Client- mode. The system comprises one or more production servers 11, 12, etc. (e.g., one server for each production line), which can be connected to a central server 10 in a computer network. In smaller production plants, the functions of the central server and the production server 11 can be concentrated in one (production) server. In the present example, the production server 11 is assigned to the production line 31. A plurality of user terminals, such as AR terminals 41 and (optionally) an administrator terminal 40 (e.g., a personal computer, a tablet PC, or another AR terminal), can be connected to the production server 11 via the computer network, for example, via a wireless local area network ( wireless lokal area network, WLAN). The production servers 11, 12 do not have to run on standalone (computer) hardware, but the respective server services can also be executed in a user terminal (e.g., an AR terminal or another mobile device) or the administrator terminal 40.

[0020] AR terminals can use stereoscopic head-up displays (HUDs or HMDs, head-mounted displays ) with and without sensors for spatial detection of the environment. Commercially available systems include Google Glass ®,< , Epson Moverio ®< (without sensors for spatial detection of the environment) and Microsoft HoloLens ®< (with sensors for spatial detection of the environment). However, conventional mobile devices such as a tablet PC with a built-in camera can also be used as AR terminals. The built-in camera can then be used to read so-called AR markers ( marker-based Augmented Reality ) can be used, which can be arranged at specific positions on a production line. If the AR device recognizes an AR marker (e.g. a barcode or a QR code), information relevant to the respective position can be displayed on the AR terminal. HUDs or HMDs have the advantage that the person concerned has their hands free to carry out the necessary work at the respective position. In addition, conventional mobile devices such as a tablet PC with a built-in camera (although these are often not referred to as AR terminals) can also be used. These can then work with text or audio instructions, for example.

[0021] The local production server 11 for the production line 31 stores production-specific data ( Fig. 1 , reference numeral 15) with respect to the production line 31 and can dynamically make this available to the AR terminals 41 and the administrator terminal 40 as needed. This does not happen once at the beginning of a process, but can be adapted to the current situation during a process in step with the process. Adjustments can or must be made, for example, if one or more operators ( Operators ) leave the system (log out) or join (log in). The system must then (using computer algorithms) dynamically calculate the time for each operator, e.g., taking into account a " Skill-Levels " the operator, a current list of tasks ( Tasks ) and made available to the respective operator. This also happens - for each task - after a task has been completed by an operator. This ensures that all tasks are processed, no task is assigned more than once, and each operator is only offered or assigned tasks that are permissible with regard to their operator status. "Offered" in this context means that an operator, depending on their skill level (e.g. the "professional" skill level), is offered not just a specific task, but a selection of clusters and tasks (groups of tasks), for example, taking into account the positions of the other operators. An operator can therefore (within certain limits) select which specific task they will perform next.

[0022] The assignments of skill levels to the respective operators can be stored, for example, in a database on the production server 11 or the central server. An operator can authenticate themselves at a user terminal using a code or password, for example, and the system can thus identify users based on the networked user terminals at which they are logged in.

[0023] The central server 10 has access to the production-specific data of several (or all) production lines and can update them regularly. It can also be designed to perform (e.g., comparative) data analyses ( Fig. 1 , reference numeral 101). The central server 10 can be connected to the local production server 11 via the local computer network. If the central server 10 is located at a different location, it can also be connected to the local production server 11 via the public Internet (e.g., via a virtual private network, VPN). The mentioned data analyses can, for example, be real-time analyses ( Realtime Analytics ), benchmarking methods, etc.

[0024] The production line 31 comprises at least one machine 310 with several operating elements 312 and / or displays 311 (see also Fig. 2 ). In this example, the control elements 312 have displays 311. AR terminals with sensors for spatial detection of the environment can be used as AR terminals. This enables markerless AR ( markerless augmented reality ) and precise tracking of the position of the respective AR terminals. In authoring mode, with the help of at least one AR terminal 41 and the sensors contained therein (and the software running on the terminal), the geometry of the environment (and thus also of the machine 310) is recorded three-dimensionally and a corresponding 3D model is created and stored in the local production server 11. During this process, various sensors and, for example, algorithms for simultaneous localization and (3D) map creation ( Simultaneous Localization and Mapping, SLAM) are used. Suitable sensors include cameras, TOF cameras ( Time-of Flight- cameras) , inertial measurement units ( Inertial Measurement Units, IMUs), etc. The methods and algorithms for data acquisition and building a 3D model are well known and are supported by commercially available devices such as the HoloLens ®< from Microsoft, Tango from Google, Intel RealSense ™< , etc.

[0025] Fig. 2 schematically illustrates a part of a 3D model of the machine 310, wherein two control elements 312 with displays 311 are shown as examples. The control elements can generally be referred to as a human-machine interface ( Human-Machine Interface, HMI). In authoring mode, positions (e.g., in the form of coordinates) can also be defined in space, at which position-related information 410 relating to the task to be completed at the respective position can later be dynamically assigned and displayed in client mode. The position-related information 410 can thus represent a task in a form perceivable by a human (e.g., text, numerical values, graphics, etc.). For example, a position can be "marked" next to each human-machine interface (control element 312 / display 311), at which information relating to the respective human-machine interface can be displayed in client mode. This position-related information can be overlaid by the AR terminals 41 in the real environment. In the present example, a special marking (e.g., with a QR code) of the position is not necessary.In so-called markerless AR systems, the desired position is recognized based on current sensor information provided by the AR terminal and a previously created (or otherwise provided) 3D model of the environment. However, markers can also be used in simpler systems. In another example, markerless and marker-based AR terminals and non-AR-capable terminals can be combined in one system. It should be noted that the control elements 312 with displays 311 are merely an example of positions on the machine where tasks may be performed. Other examples include the installation or attachment of machine parts, the replacement of wear parts, etc.

[0026] In the Fig. 2 The example shown is in the client mode of the operator ( Operator ) carrying an AR terminal, the position-related information 410 is displayed at the point in space that was previously defined in authoring mode. In the present example, the position-related information 410 is displayed at a point that is spatially in the immediate vicinity of the control element 312 (labeled "C1.1"). The information 410 is displayed by the AR terminal of the respective operator as soon as the corresponding point in space is in the field of view of the AR terminal 41, which in turn depends on the position (location and orientation) of the AR terminal 41 in space. The information 410 can refer to a location at the respective position (e.g., at the location of the control element C1.1, see Fig. 2 ) task to be performed by the operator ( Task ). This means that each task is assigned a specific position in the room.

[0027] The information 410 displayed at a specific position can, for example, be a numerical value that the operator should set on the control element 312 and / or check on the associated display 311. When the operator has completed the task, ie, has set the desired value on the control element 312 (or checked the set value), he or she can, for example, confirm the completion of the task with a gesture in the field of view of the AR terminal 41 and the current work step ( Fig. 2 , step C1.1). The AR terminal 41 then offers the operator several locations (positions in the room) to choose from or a specific location (if only one is available) and then guides the operator to the location where the next task, i.e. the next work step (e.g. at the next control element / the next HMI), is to be completed. Upon completion of a task, its successful completion can also be verified and documented, e.g. by taking a photo of the control element 312 or display element 311 using the AR terminal. For analysis purposes (cf. Fig. 1 , Analyses 101), additional data can also be recorded, such as the name, time, or the amount of time a person needed to complete a specific task. The times can be stored on the production server either personally or anonymously. The recording of personal data may even be mandatory, for example, in the pharmaceutical sector with regard to safety audits (GMP, Good Manufacturing Practice ) . In this step, the system can record the tasks that have already been completed and will no longer offer or assign them to any other operator when dynamically creating the task lists.

[0028] Fig. 1 and 2 show the structure of the overall system and the view of an operator through an AR terminal 41. In the following Fig. 3-9 The functionality of the system is illustrated, whereby the individual functions can be implemented partly in the production server 11 and partly in the AR terminals 41 or the administrator terminal 40. As already explained above, in authoring mode, one or more system modules of a production line 31 (cf. Fig. 1 ) are geometrically recorded and several positions or areas ( Spots ) where a team of operators (Operators) a lot of tasks have to be done. In the schematic representation according to Fig. 3 These positions are labeled P11, P12, P13, P21, etc., which form the set of all possible positions. In the present example, certain work areas in a production line (e.g., different parts of a machine 310) are defined as plant modules, and each defined position can be assigned to a module. A (plant) module corresponds to a machine or a part of a machine in the respective production line. In the representation according to Fig. 3 Production line 31 (or part of it) is divided into three modules M1, M2, and M3. Module M1 comprises positions P11, P12, and P13, module M2 comprises positions P21 and P22, and module M3 comprises positions P31, P32, and P33. Each of the positions P11, P12, P13, P21, P22, P31, P32, and P33 can, if necessary for the process under consideration, be dynamically assigned a specific task to be performed by an operator on the team when creating the work document for the respective process (e.g., converting a machine). The assignment of positions to modules is not mandatory and may depend on the specific conditions of the production line and the exact requirements of the product being processed (e.g., packaged) during a particular pass on the production line. Each of the positions P11, P12, P13, P21, P22, P31, P32 and P33 can be defined, for example, by three spatial coordinates.Each position can also be assigned an orientation (for the purpose of 3D representation of the position-related information assigned to a position regarding the task to be performed at that position).

[0029] In diagram A from Fig. 4 is the allocation of tasks ( Tasks ) to the positions P11, P12, P13, P21, P22, P31, P32 and P33. The individual tasks can be grouped into clusters C1, C2, etc. (groups), whereby the tasks assigned to a specific cluster can, for example, be dependent on each other (and therefore, for example, must be processed one after the other). Furthermore, tasks that must be completed at positions that are very close to each other can be assigned to a cluster. A cluster can - but does not have to - include those tasks that are assigned to the positions of a module. In the example from Fig. 4 (Diagram A) Tasks T11, T12, and T13 are assigned to positions P11, P12, and P13 in module M1; tasks T14 and T21 are assigned to positions P21 and P22 in module M2. As can be seen, the number of tasks in a module is not equal to the number of tasks in a cluster. In this example, tasks T11, T12, T13, and T14 are assigned to cluster CL1, and task T21 (along with other tasks) is assigned to cluster CL2. As already mentioned, modules represent a specific area or part (the three-dimensional geometry) of a machine in a production line, whereas clusters group tasks according to their logical sequence or relatedness. Diagram B from Fig. 4 shows a detail from Fig. 2 . Position P21 indicates the location of the control element, and the displayed information 410 relates to the task assigned to position P21. As mentioned, there is no one-to-one mapping between system modules and clusters. For example, as shown in diagram C of the Fig. 4 shown, two system modules M1 and M2 can also be arranged partially one above the other. Due to the spatial proximity alone, it can be sensible to have the tasks at positions P21 and P13, P22 and P14 and P23 carried out one after the other by the same person (or the same people). In this case, the right-hand part of module M1, which is located above module M2, is assigned to the same cluster C2 as module M2. Only the left-hand part of module M1 is assigned to cluster C1. The assignment between tasks T11, T12, T13 and T14 and positions P11, P12, P13, P21 and the associated position-related information can be stored, for example, in a database of the production server 11 of the production line in question.

[0030] Fig. 5 shows an example of a system description using clusters and the distribution of tasks across multiple clusters. In this example, four tasks (T11, T12, T13, T14) are assigned to the first cluster CL1, two tasks (T21, T22) to the second cluster, and three tasks (T31, T32, T33) to the third cluster. The overall system (in relation to a production line 31) can essentially be defined by three dynamically managed lists (see Fig. 6 ), namely a list L1 of unfinished tasks that have not yet been assigned to any operator in a team, a list L2 of "active" tasks, each assigned to an operator, and a list L3 of completed tasks. The L1 list is a task list that is dynamically generated at the beginning of the process depending on the process result to be achieved (process parameters necessary to achieve the desired result) and variable framework conditions (e.g., number and skill level of the operators). This list can also be automatically adjusted during the process. The L1 list can also be dependent on process parameters of a previously performed process. For example, certain tasks can be omitted if the relevant settings / configurations on the production line have already been made for the previous process and do not need to be changed for the current process.

[0031] Lists L1, L2, and L3 can also be stored in a database on the production server 11 and managed dynamically by it. In this context, "dynamic" means that the list of tasks to be completed and the assignment of tasks to the operators in a team are not created statically at the beginning of the process (i.e., before the operators begin work), but can be changed / adapted as the work is being performed. The number of operators in the team can change during the process (while the team is executing the process), and the skill levels of the operators in the team can change (e.g., if an operator with the skill level "Trainee" is replaced by an operator with the skill level "Beginner"). This dynamic, which would be impossible with "manual" planning, can significantly improve the efficiency of process execution.

[0032] Fig. 6 illustrates the description of a production line with eleven unfinished tasks (List L1: Tasks T12, T13 and T14 from Cluster C1, Tasks T24 and T25 from Cluster C2, Tasks T32, T33 and T34 from Cluster C3, and Tasks T41, T42, and T43 from Cluster C4), three active tasks (List L2: Tasks / Operator T11 / O1, T23 / O2 and Task T31 / O3) and five completed tasks (List L3: Tasks T11, T21, T22, T23 and T31). The operators O1, O2 and O3 can be differentiated based on the AR terminals they use. At this point, it should be mentioned again that the list L1 of unfinished tasks is not a fixed list, but can be created dynamically at the beginning of the process. A number of factors influence the creation of the list. At the beginning of a process, for example, an operator (e.g.with skill level "Supervisor") decide whether or not to consider production line settings known from the previous process when compiling list L1 of tasks to be completed. Based on the list L1 generated in this way, lists L2 (active tasks) are created dynamically for each operator 01, 02, etc. This takes into account whether a task T1, T2, etc. can be assigned to an operator 01, 02, etc., i.e., whether they are authorized to perform certain tasks and whether these tasks are assigned to them as a selection from list L1 or in a fixed, dynamically generated order.

[0033] It should be noted here that, for example, when converting a production line (conversion process), not all possible settings / adjustments on the production line always need to be performed / checked. In some situations, this is desirable. However, it may be more efficient to only perform / check settings / adjustments on the production line at each position where the desired settings / adjustments differ from the last conversion process.

[0034] Each AR terminal has a unique identifier, for example the MAC address ( Media-Access-Control- Address) of the respective AR terminal in the computer network. However, any other unique identifiers can be used to identify an operator's AR terminals. The identifier can be linked to personal data and / or data regarding the completion of tasks (determined times, documentation, etc.) of the respective operator.

[0035] In many production plants, machines must be operated by several people (referred to here as operators) who must work as a team (for example, when converting a machine for production or packaging or handling a different product). The members of a team must be well coordinated in order to work together effectively. In practice, for each machine (or for a group of machines on a production line), schedules (e.g. in paper form) are created for the people in a team, and each team member works according to their own schedule. The team size is fixed for a specific process (e.g. converting a machine) and cannot simply be changed once work has started. The schedules are static. If a team member is absent (or if a shift change is taking place, for example), this can significantly delay the entire process.Even adding another person to a team cannot significantly speed up the entire process (the sum of the tasks) because this would disrupt established schedules.

[0036] The system described here enables flexible adjustment of team sizes using AR terminals (see Fig. 1 ) used by the team members (the operators). The following describes a method that allows, with the help of the networked AR terminals 41, to dynamically adjust the size of a team of, for example, differently trained operators (i.e., with different skill levels, e.g., "Trainee," "Beginner," "Advanced," "Professional," "Supervisor," etc.) and operators equipped with different terminals, as well as the associated allocation of tasks to the team members. The example from Fig. 7 shows a situation in which the team is expanded by an operator (Operator O5) (a new person joins the team). To do this, the AR terminal of Operator O5 registers with the production server 11. This registration can occur automatically (e.g., after connecting the AR terminal of Operator O5 to a specific WLAN access point), or can be triggered manually by Operator O5 (e.g., by entering a password) (e.g., using his AR terminal 41 or an administrator terminal 40). After registration, Operator O5 is added as a free operator to the list L0 of free operators (and, if applicable, taking into account the skill level) (i.e., those operators / team members who have not yet been assigned or offered a task). This situation is shown in diagram A from Fig. 7 shown.

[0037] Diagram B from Fig. 7 illustrates how a decision can be made automatically as to which unfinished task (from one of the clusters) is assigned to which free operator. An operator with a low skill level (e.g., a "beginner") can be dynamically assigned a task list by the system, which they must complete in a predefined order. The system will not assign tasks that have already been completed, nor will it assign tasks that are unsuitable for the operator's skill level. Furthermore, the system can preferentially assign tasks to the ("beginner") operator that are to be performed in the freest possible part of the production line. For this purpose, the system can use a suitable algorithm to determine the best starting position for a "beginner" operator and initially generate a list of tasks that are offered to them – keeping pace with the constantly changing situation during the process.In a situation where an operator with a higher skill level (e.g., "professional"), who can choose from a variety of clusters and tasks offered by the system, precedes a "beginner" operator and "gets in the way" of the "beginner" operator (preventing the "beginner" operator from performing a specific task), the system can dynamically generate a new task list (e.g., optimized according to certain criteria) for the "beginner" operator (autopilot mode). This process runs constantly in the background, keeping pace with the process, and ensures optimal distribution of operators on the production line. Depending on an operator's skill level, the operator can be assigned only one task at a time or a group of tasks from which the operator can choose (using their user terminal).Once an operator has confirmed a specific task (using his user terminal), the user terminal can virtually display the associated (position-related) information at the position in the room assigned to the respective task, which is intended to support or guide the operator in carrying out the task.

[0038] In the Fig. 7 In the example shown in Diagram B, the clusters are sorted in descending order according to the number of unfinished tasks (first criterion). The number of active tasks (with ascending sorting) can be used as the second criterion for dynamic generation. This means that a free operator (here, operator O5) (e.g., with skill level "Beginner") is assigned a task from the cluster that contains the most unfinished tasks. If this selection criterion does not lead to a clear result (e.g., because the number of unfinished tasks is the same for two clusters), operator O5 is assigned a task from the cluster that contains the most unfinished tasks and the fewest active tasks. Alternatively or additionally, other selection criteria can be considered.For example, the current location of all operators, which is known to the system (and regularly updated), can be taken into account as a criterion in that one or more tasks are assigned or offered to the (free) operator who is closest to the position assigned to the task. This approach makes it possible to shorten the routes taken by the operators. It is also possible to avoid certain assignments between tasks and operators, for example, if a particular assignment would result in too many operators being in a certain spatial area at the same time. As mentioned, the position of the individual operators can be tracked, for example, using the AR terminals 41, and this information can also be taken into account when dynamically adapting the task lists L1-L3. It goes without saying that for small production lines (e.g., only a single machine), grouping the tasks into clusters may not be necessary.The criteria for assigning tasks to operators can then be based on the resulting paths of the operators or similar criteria. Even random assignment would be possible if the application allows it.

[0039] In this example, cluster C4 includes three unfinished tasks and no active tasks (because, for example, a task has just been completed, see also Fig. 8 ), cluster C2 has one pending task and one active task, cluster C3 has no pending task and one active task, and cluster C1 has no pending task and two active tasks. Consequently, the free operator O5 is offered - among others - a task (e.g., task T41) from cluster C4. This assignment is shown in diagram C from Fig. 7 shown schematically. The task is deleted from the list L1 of unfinished tasks and added to the list L2 of active tasks. The operator O5 is deleted from the list L0 of free operators (see Fig. 6 ). Within the clusters, the order of tasks can be fixed. The order is defined in authoring mode if a required order exists (e.g., first remove a part, then install a new part). If a fixed order of tasks (or a subset of tasks) is not specified by the production line, they can be selected according to certain selection criteria (e.g., so that spatial overlaps of the work areas (i.e., collisions) of the operators at the positions assigned to the tasks are avoided). In one variant, several tasks, for which the order of completion is not important, can be offered to an operator at the same time. In this case, the order of completion is left to the operator. If he does not complete all tasks (e.g.,because he leaves the process, for example during a shift change), the tasks not completed by this operator are taken into account by the system when creating the task list of the other and, if applicable, new operators.

[0040] Fig. 8 schematically illustrates a situation in which an operator has confirmed the completion of a task (a task) or the completion of the last task in a group of tasks (if the operator has been assigned several tasks). This confirmation is usually made by the operator, e.g. manually on their AR terminal (e.g. by pressing a virtual button in the field of view of the AR terminal, by pressing a real button on the AR terminal, using a Bluetooth controller, voice control, etc.). In this case, the active task (e.g. Task 41) is deleted from the L2 list of active tasks and added to the L3 list of completed tasks, and the operator in question (e.g. Operator O5) is added back to the L0 list of free operators. There are situations in which an active operator leaves (or has to leave) their workstation without completing (or being able to complete) the task assigned to them.When leaving the workstation, the AR terminal of the operator in question is logged off from server 11 (. unregister ) . This logout can occur automatically (e.g., if the WLAN connection between the AR terminal and the access point is interrupted) or manually by the operator or another person (e.g., via the Administrator Terminal 40 or an AR terminal). This situation is shown in diagram A. Fig. 9 shown. In the example shown, operator O5 has deregistered his AR terminal from server 11. The active, unfinished task (e.g., task T41) and the corresponding operator O5 are removed from the L2 list of active tasks, and the task is added back to the L1 list of unfinished tasks and can, for example, be assigned to another operator (see Fig. 7 , Diagram C). If the AR terminal of a free operator (e.g., O4) is deregistered, it is removed from the list L0 of free operators.

[0041] In the following, some aspects of the embodiments described here are summarized (cf. Fig. 1-9 ). However, it should be noted that this is not a complete list, but only an example. A system for supporting teamwork using Augmented Reality (AR). According to the examples described here, the system allocates a production server (see Fig. 1 ) which is designed to store the three-dimensional geometry of at least part of a machine (cf. Fig. 2 ). Furthermore, a large number of tasks are stored, whereby each task can be assigned to a position in space at which the respective task is to be completed, as well as position-related information regarding the completion of the task (cf. Fig. 4 , diagrams A and B). The system also has a variety of AR terminals (see Fig. 1 ) that are connected to the production server via a computer network. Each AR terminal can be assigned to a member of a team. The production server is designed to dynamically generate a "recipe" for implementing the process (and adapt it if necessary), assigning each member of the team a specific task (e.g., taking into account the person's skill level) or offering a group (cluster) of tasks (see Fig. 7 ), by displaying the corresponding position-related information on the AR terminal of the person concerned at the position in the room assigned to the respective task (cf. Fig. 4 , Diagram B). Depending on the implementation, the AR terminals can be used to capture the position of each operator in real-time (i.e., during a process) and store it (e.g., together with the associated times and active tasks). This data can be used for later data analysis or for real-time data analysis ( Realtime Analytics ) can be used (cf. Fig. 1 ). For example, using this data, workflows can be (partially) automatically optimized (e.g., the assignment of tasks to clusters or the order of tasks in a cluster or the avoidance of collisions between two or more operators).

[0042] The production server can be further configured to mark each member of the team to whom a task has been assigned or offered as busy and to mark the assigned task as active. These markings are achieved, for example, by adding the person and task to the L2 list of active tasks (see Fig. 6 ). Furthermore, the production server can mark a task as completed if the (busy) person assigned to this task confirms completion of the task via their AR terminal. This marking occurs, for example, by adding the task to the L3 list of completed tasks (see Fig. 6 ).

[0043] The active marking of a task is removed (without marking the task as completed) if the AR terminal of the person assigned to the task is logged off from the production server without the person first confirming completion of the task. The marking can be removed, for example, by adding the task in question back to the L1 list of unfinished tasks (see Fig. 6 ). A person whose AR terminal has been logged off the production server is no longer available for assignment of further tasks.

[0044] The production server can assign people who are not currently assigned a task (List L0 of free operators, see Fig. 7 ), as long as new (unfinished) tasks are assigned, as long as there are tasks that are neither marked as active nor as completed, and the assignment does not result in any collisions (overlapping workspaces of two operators due to spatially close tasks). An additional person with an additional AR terminal can be added to the team by registering the additional AR terminal with the production server (see Fig. 7 , Diagram A). In the examples described here, the members of the team are identified or differentiated by the production server using an identifier of the AR terminals assigned to them.

[0045] Each task of a process (e.g., the retooling of a production line or part of it) can be assigned to a cluster, and each task can be marked as either pending, active, or completed (see Fig. 5 und 6 ). When assigning tasks to people, the production server can prioritize. For example, tasks in the cluster that contains the most unfinished tasks and the least active tasks can be considered first, provided that no collisions with other operators are expected. To this end, the task lists of each operator can be updated in step with the process, taking into account defined constraints – such as the skill level required for a specific task or task dependencies ( Tasks ) (logical dependencies or spatial dependencies) between each other - are generated or updated.

Claims

1. Computer-assisted system for supporting teamwork, comprising: a production server (11) configured to store: the three-dimensional geometry of at least one part of a machine in a production line and a plurality of tasks (T12, T22, ...), each task being assigned a position (P11, P12, ...) in space, at which the relevant task (T12, T22, ...) is to be performed, and positioned-based information regarding the completion of the task; a plurality of user terminals (41) connected to the production server (11) via a computer network, each user terminal (41) being assigned to a person (O3, O4, O5, ...) in a team, wherein the production server (11) is further configured to generate dynamically and to adapt dynamically and in real time a task list (LI) for a process to be handled by the team, and each person (O3, O4, O5, ...) in the team is allocated an individual task (T11, T12, ...) or a group of tasks from this list (LI), the position-based information (410) assigned to each task (T11, T12, ...) being to be displayed on the user terminal (41) of the relevant person (O3, O4, O5, ...) , wherein the tasks are allocated dynamically as a function of the number of persons (O3, O4, O5, ...) and as a function of process parameters of a previously performed process and are updated if the number of persons (O3, O4, O5, ...) changes, wherein tasks which are logically and / or spatially dependent on one another are combined into groups and / or have to be processed consecutively, and wherein the specified process includes setting up or retooling one or more machines in the production line.

2. System according to claim 1, wherein the user terminal (41) is configured to display the position-based information (410) assigned to the relevant task (T11, T12, ...) in the position (P11, P12, ...) in space assigned to the task.

3. System according to either claim 1 or claim 2, wherein the production server (11) is further configured to allocate each person an individual task or a group of tasks, a skill level of the relevant person being taken into account in the allocation.

4. System according to any of claims 1 to 3, wherein the production server (11) is further configured to mark each person (O5, ...) in the team who has been allocated a task (T41) as busy and to mark the allocated task as active.

5. System according to claim 4, wherein the production server (11) is further configured to mark a task (T41) as complete when the person (O5) who is allocated the task (T41) confirms completion of the task (T41) via his user terminal (41).

6. System according to claim 5, wherein the production server (11) is further configured to remove the marking of a task as active again, without marking it as complete, if the user terminal (41) of the person (O5) who is allocated the task (T41) is signed out from the production server (11) without the person (O5) having confirmed completion of the task (T41).

7. System according to either claim 5 or claim 6, wherein the production server (11) is further configured to allocate people (O2, O4, ...) who are currently not allocated a task a new task from the plurality of tasks for as long as the plurality of tasks (T11, T12, ...) contains tasks which are marked neither as active nor as complete.

8. System according to any of claims 1 to 7, wherein the production server (11) is further configured to add an additional person (O5) with an additional user terminal (41) to the team in that the additional user terminal (41) is registered by the production server.

9. System according to any of claims 1 to 8, wherein the production server (11) is further configured to identify the persons (O3, O4, O5, ...) in the team using identifiers of the user terminals (41) assigned to them.

10. System according to any of claims 1 to 9, wherein each of the tasks (T11, T12, ...) is assigned to a group and each task in a group is marked as incomplete, as active or as complete.

11. System according to claim 10, wherein the production server (11) is further configured, when allocating people tasks, initially to take into account tasks in the group comprising most incomplete tasks and fewest active tasks.

12. System according to any of claims 1 to 11, wherein the user terminals (41) are augmented reality (AR) terminals, which each have a stereoscopic heads-up display and one or more sensors which capture the geometry of the environment in three dimensions.

13. System according to claim 12, wherein at least some of the AR terminals (41) and / or production servers (11) are configured to create from the sensor data, in an authoring mode, a 3D model of the environment which also includes the positions (P11, P12, ...) assigned to the tasks (T12, T22, ...).

14. System according to any of claims 1 to 13, wherein the production server (11) is further configured initially to create, for a process to be handled by the team, a list (LI) of tasks required for performing the process, wherein the process parameters of a previously performed process are taken into account in creating the list (LI).

15. System according to claim 14, wherein the list (LI) of tasks is continuously dynamically adapted while taking into account the already completed tasks (T12, T22, ...) and the potentially changing number of persons (O3, O4, O5, ...) and their skill levels.

16. Computer-assisted method for supporting teamwork, comprising: storing the following data on a production server (11): the three-dimensional geometry of at least one part of a machine in a production line and a plurality of tasks (T12, T22, ...), each task being assigned a position (P11, P12, ...) in space, at which the relevant task (T12, T22, ...) is to be performed, and positioned-based information regarding the completion of the task; signing a plurality of user terminals (41), connected to the production server (11) via a computer network, in to the production server (11), each user terminal (41) being assigned to a person (O3, O4, O5, ...) in a team, generating dynamically and adapting dynamically and in real time a task list (LI) for a process to be handled by the team, including allocating each person (O3, O4, O5, ...) in the team an individual task (T11, T12, ...) or a group of tasks from this list (LI), the position-based information (410) assigned to each task (T11, T12, ...) being displayed on the user terminal (41) of the relevant person (O3, O4, O5, ...) , wherein the tasks are allocated dynamically as a function of the number of persons (O3, O4, O5, ...) and as a function of process parameters of a previously performed process and are updated if the number of persons (O3, O4, O5, ...) changes, wherein tasks which are logically and / or spatially dependent on one another are combined into groups and / or have to be processed consecutively, and wherein the specified process includes setting up or retooling one or more machines in the production line.

17. Method according to claim 16, wherein the user terminal (41) displays the position-based information (410) assigned to the relevant task (T11, T12, ...) in the position (P11, P12, ...) in space assigned to the task.

18. Method according to either claim 16 or claim 17, wherein each user terminal (41) displays the task allocated to the person (O3, O4, O5) assigned to the user terminal (41) and, when the user has confirmed the task, displays the position-based information (410) assigned to the relevant task (T11, T12, ...) in the position (P11, P12, ...) in space assigned to the task.

19. Method according to any of claims 16 to 18, wherein allocating a task (T11, T12, ...) from the plurality of tasks comprises: allocating a group of tasks (T11, T12, ...) from the plurality of tasks to one or more persons (O3, O4, O5, ...) in the team, the user terminal (41) of the person (O3, O4, O5, ...) making it possible to select a task from the group of tasks (T11, T12, ...); and displaying the position-based information (410) assigned to the task (T11, T12, ...) selected by the person (O3, O4, O5, ...) on the user terminal (41) of the relevant person (O3, O4, O5, ...).

20. Method according to any of claims 16 to 19, wherein the tasks (T11, T12, ...), the positions in space assigned to the tasks and the associated position-based information (410) are stored in a database of the production server (11).