AUTOMATED TASK ASSISTANCE SYSTEM

Abstract

A computer system is provided to assist a user in performing a task within a technical system. The computer system includes a user intent detection module configured to detect the user's intentions when interacting with the technical system, a user request clarification module configured to determine chat texts to assist the user in the task, and a task assistance module.The task assistance module is configured to receive information from the user, in which the information relates to one or more natural language chat messages and / or an observation of the user's interaction with the technical system, to determine, by applying the user intent detection module, a task execution process based on the information, and to interact, by applying the user request clarification module, with the user via chat messages in accordance with the process determined to assist the user in the task.

Description

Title of Invention: AUTOMATED TASK ASSISTANCE SYSTEM Field

[0001] The présent disclosure relates to Systems for assisting a user with performing a task within a technical System. More specifically, the disclosure concems a System and method that utilizes user intention détection, user request clarification, and task assistance modules to facilitate user interaction with technical Systems, such as computer programs or computer-controlled machines. Background

[0002] In recent years, there has been an increasing demand for Systems that assist users in performing tasks within technical Systems. This demand arises from the growing complexity of technical Systems and the need for users to efficiently and effectively internet with these Systems to complété tasks. Users often face challenges in understanding and navigating the processes necessary to complété tasks within technical Systems, leading to frustration, wasted time, and reduced productivity.

[0003] Traditional methods for assisting users with tasks in technical Systems are based on manually studying documentation of the technical System, such as user manuals, implémentation guides, maintenance guides, or the like. These documents often provide step-by-step instructions for completing tasks within the technical System. However, these methods hâve several drawbacks, including the need for users to locate and reference the relevant documentation, the potential for outdated or incomplète information, and the difficulty in understanding complex instructions written in technical language.

[0004] Hence, there is a need for an improved System of assisting users with performing tasks within technical Systems. Summary

[0005] In this context, a computing System for assisting a user with performing a task within a technical System is provided. The computing System comprises a user intention détection module configured to detect intentions of the user when interacting with the technical System, a user request clarification module configured to détermine chat texts for assisting the user with the task, and a task assistance module. The task assistance module is configured to receive information from the user, wherein the information relates to one or more natural language chat messages and / or an observation of interaction of the user with the technical System, détermine, by applying the user intention détection module, a procès s for performing the task based on the information, and internet, by applying the user request clarification module, with the user via chat messages according to the determined process to assist the user with the task.

[0006] In embodiments, the System further comprises a first memory configured to store sets of process information, wherein the sets of process information relate to processes for completing tasks, wherein a set of process information is associated with a process identifier. In these embodiments, the task assistance module is, for interacting with the user via chat messages, further configured to provide the user request clarification module with a process identifier of the determined process and the information from the user, to receive a chat text for communicating with the user from the user request clarification module, and to présent the user with one or more chat messages according to the received chat text.In these embodiments, the user request clarification module is further configured to receive the process identifier of the determined process and the information from the user from the task assistance module, to receive at least one set of process information according to the process identifier, to generate a chat text through prompt engineering according to the at least one set of process information of the determined process and the information from the user, and to provide the chat text to the task assistance module. .

[0007] In some embodiments, the set of process information further comprises at least one or more of input data, target web page or program context, one or more steps to be performed for completing the task, human readable description of the task and / or the one or more steps, and a title. In further embodiments, the user request clarification module uses generative artificial intelligence to generate the chat text.

[0008] In embodiments, the System further comprises a second memory configured to store embeddings of the sets of process information associated with respective process identifiers. In these embodiments, the task assistance module is, for determining the process for performing the task, further configured to provide the user intention détection module with the information, and to receive a process identifier of the process for performing the task from the user intention détection module. In these embodiments, the user intention détection module is configured to receive the information from the task assistance module, to détermine a process mapping to the information based on the information and the embeddings stored in the second memory, and to provide the process identifier of the determined process to the task assistance module.

[0009] In some embodiments, the user intention détection module uses generative artificial intelligence to détermine the process mapping to the information.

[0010] In some embodiments, the System comprises a third memory configured to store a technical documentation of the technical System, wherein the technical documentation is in natural language format and defines processes for completing tasks, and an information extraction module. In these embodiments, the information extraction module is configured to obtain the technical documentation from the third memory, to extract a processing block related to a task from technical documentation, to parse the processing block to a format defining detailed information about the respective process, to generate an embedding related to the processing block based on at least part of the detailed information, to store the embedding with a respective process identifier in the second memory, to generate a set of process information of the task based on at least part of the detailed information, and to store the set of process information with the respective process identifier in the first memory.

[0011] In some embodiments, the technical System is a computer executing programs, wherein the technical documentation is an implémentation or administration guide to one or more of the programs or wherein the technical System is a computer-controlled machine, wherein the technical documentation is a maintenance or administration guide to controlling the machine. In further embodiments, the information extraction module uses generative artificial intelligence to extract at least one processing block and / or to parse the at least one processing block.

[0012] In some embodiments, the System further comprises a fourth memory configured to store action templates, wherein the action templates comprise computer-readable instructions for processes associated with respective identifiers for the processes, and a task managing module. In these embodiments, the user request clarification module is further configured to extract one or more values of input parameters from the information, wherein the input parameters are required for completing the determined process, and to provide the one or more values of the input parameters in a structured form to the task assistance module.In these embodiments, the task assistance module is, for interacting with the user via chat messages, further configured to provide the process identifier of the determined process and the one or more values of the input parameters to the task managing module, to receive an exécutable script for automatically performing the process from the task managing module, and to provide, within the chat messages, the user with the exécutable script. In these embodiments, the task managing module is configured to receive the process identifier of the determined process and the one or more values of the input parameters from the task assistance module, to retrieve an action template for the determined process according to the process identifier from the fourth memory, to create an exécutable script based on the action template and the one or more values of the input parameters, and to provide the exécutable script to the task assistance module. .

[0013] In some embodiments, the format of the exécutable script is adjusted according to a computing device operating System of the user and / or the technical System.

[0014] In some embodiments, the System further comprises a template recording module. In these embodiments, the user request clarification module is further configured to détermine whether an action template for the determined process is stored in the fourth memory, and, in response to no action template for the determined process being stored in the fourth memory, to include in the chat text a request for recording interactions with the technical System from the user when performing the determined process for the task. In these embodiments, the template recording module is, in response to the user acknowledging the recording, configured to record interactions of the user with the technical System for performing the task, to create a new action template for the determined process by converting the interactions to machine-readable instructions for the determined process, and to store the new action template in the fourth memory.

[0015] In some embodiments, the System further comprises a fifth memory configured to store a prompt history of the one or more natural language chat messages from the user and the chat messages provided to the user. In these embodiments, the task assistance module is further configured to store the one or more natural language chat messages chat from the user and the chat messages provided to the user in the fifth memory, and to obtain the one or more natural language chat messages chat and the chat messages stored in the fifth memory for providing them to the user intention détection module and / or the user request clarification module.

[0016] In some embodiments, the chat messages provide a step-by-step solution of the process required for completing the task. In further embodiments, the task assistance module is connected to an add-on of a messaging System over which the chat messages are provided or the task assistance module is a browser extension connected to a web page over which the chat messages are provided.

[0017] The foregoing paragraphs hâve been provided by way of general introduction and are not intended to limit the scope of the foliowing daims. The described embodiments, together with further advantages, will be best understood by reference to the following detailed description taken in conjunction with the accompanying drawings. Brief description of the drawings

[0018] The foregoing and further objects, features and advantages of the présent subject matter will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings, wherein like numerals are used to represent like éléments, in which:

[0019] [Fig-1] is a schematic overview of a computing System for assisting a user with performing a task within a technical System comprising a task assistance module and its interaction with a user intention détection module and a user request clarification module according to the disclosure.

[0020] [Fig.2] présents the computing System for assisting a user with performing a task within a technical System further comprising a first memory according to an embodiment.

[0021] [Fig.3] présents the computing System for assisting a user with performing a task within a technical System further comprising a second memory according to an embodiment.

[0022] [Fig.4] présents the computing System for assisting a user with performing a task within a technical System further comprising a third memory and an information extraction module according to an embodiment.

[0023] [Fig.5] présents the computing System for assisting a user with performing a task within a technical System further comprising a fourth memory and a task managing module according to an embodiment.

[0024] [Fig.6] présents the computing System for assisting a user with performing a task within a technical System further comprising a template recording module according to an embodiment.

[0025] [Fig.7] présents the computing System for assisting a user with performing a task within a technical System further comprising a fifth memory according to an embodiment.

[0026] [Fig.8] shows an example of chat messages of the computing System for assisting a user with performing a task within a technical System according to an embodiment.

[0027] [Fig.9] is diagrammatic représentation of a computing System, which may implement ail or part of the functionalities described herein. Detailed description

[0028] The présent disclosure relates to a computing System for assisting a user with performing a task within a technical System. The computing System described herein may be implemented by a single computer, server, or the like, which comprises ail components required to be présent for executing the processes. Altematively, the computing System may be distributed among several processing Systems, such as computers, servers, and the like, located at one géographie area. Additionally or altematively, some components of the computing System described herein may be located remotely from the others, e.g., in the cloud. Hence, different setups of the computing System are conceivable as long as the components described herein are présent and connected to each other as required.

[0029] These modules described herein can be written in any suitable programming language and can utilize varions machine learning libraries and frameworks for implementing the modules described herein. The modules interact with each other and with the user through operating Systems, using standard input and output methods provided by the operating Systems. In a networked configuration, some or ail of the modules may be implemented on different distributed processing Systems, communicating with each other over the network. The processing Systems may be general-purpose Systems, or at least some of them or the overall computing System may be specialized Systems designed specifically for assisting users with performing tasks within technical Systems. Regardless of the spécifie configuration, the computing System described herein provides a versatile and powerful platform for implementing the inventive concepts according to this disclosure.

[0030] Before turning to the description of the Figures, some background information and information on the processes performed by the computing System described herein is provided. Technical Systems are more and more controlled and maintained by the aid of complex computer programs. Moreover, classical technical Systems also become more and more complex and performing tasks for interacting with (or herein also denoted as "within") the technical Systems becomes more and more difficult, too. Therefore, many technical Systems, from complex technical machines in production environments to professional applications in computing environments, corne with large documentations, such as user manuals, implémentation guides, maintenance guides, and the like. The administration and maintenance are complex, not user friendly, and redesigns of technical Systems are always faced with high costs of training staff and administrators.And even if staff and administrators are well trained to perform tasks within the technical Systems, they still need time to perform the tasks step by step as described in the documentation. .

[0031] The solutions presented herein improve and facilitate the performing of tasks within technical Systems. For example, administrators are enabled to perform complex or massive administration tasks in several technical Systems without having to navigate through ail the administration pages, application programming interfaces, manuals etc. of each of the several technical Systems. Administrators with limited functional knowledge are similarly enabled to manage the complex configuration using natural language as administrators with high knowledge supported to speed up their tasks since the described System may propose the next most probable tasks based on their previous actions. The solution thereby helps reduce, e.g., the on-boarding time of new staff, attract new staff, remove tool complexity from a user's perspective, and overall reduces the time required to perform tasks.

[0032] Robotic procès s automation (RPA) is one approach to automatize and support the performance of tasks in technical Systems. Combined with natural language processing (NLP) technologies, this can help to automatize tasks. However, if a classical approach of machine learning is used, massive training and manual extraction of processes would be required. Moreover, RPA and NLP cannot help to provide a user with tasks suggestions.

[0033] The herein described Systems of assisting a user with performing a task within a technical System overcome the issues in the art. By providing a task assistance module, a user intention détection module, and a user request clarification module that interact with each other, the described Systems are capable of understanding and completing the intentions of the users assisted with NLP. In some embodiments, the intentions of the user are matched with processes automatically extracted from technical documentations of the technical System. In some embodiments, the computing Systems can create and explain automated process, build a knowledge base of processes / implémentation scripts, run the process through script or browser extension, and support the user further in performing the required tasks.

[0034] In some embodiments, the Systems for assisting a user with performing a task within a technical System comprises some or ail of the following procedures. Preprocessing: Read and extract data / processes to be performed from technical documentation. Create embeddings and initial sets of processes related to the task to be performed (resulting in a knowledge base of action templates). User interaction: There may be a context-based (pro-active) mode and / or a dialog-based (passive) mode. In the context-based mode, artificial intelligence (AI) identifies the current page, the user is interacting with in a spécifie manner, i.e., flow, and proposes the most probable task the user intends to perform. The AI engages discussions with the user to collect ail required input to perform the processes of the task the user intends to perform. In the dialog-based mode, the user requests support from the System (again supported by AI) regarding settings of the technical System.The AI engages discussions with the user if more information is required, which may be decided based on the technical documentation of the technical System. Once the AI in both modes has enough information, AI describes a comprehensive step-by-step solution (sum-up of technical documents) for performing the task. AI may also push a script to reproduce the solution automatically. If no script is stored in the computing System, AI may also help a user to record a new script for future performance of the task. .

[0035] Now turning to the description of [Fig.l], which is a schematic overview of a computing System 10 comprising a task assistance module 11 and its interaction with a user intention détection 12 module and a user request clarification module 13 according to the disclosure. The computing System 10 assists a user 14 with performing a task within a technical System. Technical Systems may comprise different kind of technical machines, such as machines or robots in a production environment, technical machines in energy production, such as wind turbines or solar plants, computer or computing Systems, which implement programs / applications, IT product Systems, or the like. The tasks can also be any task to be performed within the technical System, e.g., to control, amend, improve, maintain, implement etc. the technical System.

[0036] Example tasks may be helping the user 14 with adding another user, which is allowed to control a machine, in the computer application for controlling the machine, helping the user 14 with starting a cleaning program of a machine in a production environment, or helping the user 14 with adding new features to a computer program implemented on a computer.

[0037] The user intention détection module 12 is configured to detect intentions of the user 14 when interacting with the technical System, e.g., with computing applications related to the technical System. For example, if the technical System is a computer executing programs, the user 14 may interact with the computer by, e.g., searching for a spécifie way to add another user or an entity to a database on the computer, add a new program, modify a program, delete another user, entity, or program, or the like. In another example, if the technical System is a production machine, the user 14 may interact with a control field of the production machine and, e.g., search for a spécifie way to start a cleaning program, add users allowed to control the machine, modify programs of the machine, or the like. The user intention détection module 12 may also détermine, based on the detected intention, a process that is required to perform the task intended by the user 14.

[0038] The user intention détection module 12 may be based on a machine learning model, which is trained to détermine intentions from chat messages received from users and to détermine the process that is required to perform the task. The user intention détection module 12 may in these examples be based on a generative artificial intelligence tool, e.g. a large language model (LLM) chatbot. Non-limiting examples of such LLM chatbots are ChatGPT from OpenAI, Copilot from Microsoft, Gemini from Google AI, and LLaMA from Meta AI.

[0039] The user request clarification module 13 is configured to détermine chat texts for assisting the user 14 with the task. Based on the chat messages from the user 14 and the process determined by the user intention module 12, the user request clarification module 13 may also détermine a text that can be provided to the user 14 for assisting the user 14 with performing the intended task.

[0040] The user request clarification module 13 may be based on a machine learning model, which is trained to détermine intentions from chat messages received from users. The user request clarification module 13 may in these examples be based on a generative artificial intelligence tool, e.g. a large language model (LLM) chatbot. Non-limiting examples of such LLM chatbots are ChatGPT from OpenAI, Copilot from Microsoft, Gemini from Google AI, and LLaMA from Meta AI.

[0041] The task assistance module 11 is the main component interconnecting the user 14 with the user intention détection module 12 and the user request clarification module 13. The procedures performed in the most basic embodiment are the following as depicted with the arrows in [Fig. 1].

[0042] Starting with arrow 101, task assistance module 11 receives information from the user 14, wherein the information relates to one or more natural language chat messages and / or an observation of interaction of the user with the technical System. For example, the user 14 may explicitly request in a chat, e.g., in a chat application or on a website relating to the technical System, to receive help for adding a new user for controlling the technical System. In this example, the information comprises the text of the chat messages. Additionally or alternatively, the user 14 may click on several webpages, start one or more programs, search in menus of webpages or programs because the user 14 tries to find the appropriate way for performing the task alone. In this example, the information comprises the observation of the interaction of the user with the webpages / programs related to the technical System.

[0043] The tasks assistance module 11 then proceeds to détermine a process for performing the task based on the information. Therefore, the user intention détection module 12 is involved. With arrow 102, the tasks assistance module 11 provides the user intention détection module 12 with the information received from the user 14. The information may be preprocessed and transformed in a form that is processable by the user intention détection module 12. For example, the sequence of clicks by the user 14 on the webpage of the technical System may be recorded and submitted as an ordered list of parts of the webpage, sub-webpages, menus, and the like, to the user intention détection module 12. In another example, the one or more natural language chat messages may also be preprocessed, e.g., by extracting only parts of the messages, reformatting the text stream, correcting spelling errors, and the like. It is noted that preprocessing is not necessary but an optional feature.

[0044] The user intention détection module 12 thereafter detects an intention of the user 14, i.e., the user intention détection module 12 détermines what the user 14 aims to do with / within the technical System. Therefore, the user intention détection module 12 may, e.g., interact with a database of user intents as described with respect to [Fig.3] below. The resuit of the détermination of the user intent is a (most likely) task that the user 14 wants to perform. A task relates to a process that is to be performed for completing the task. Hence, in arrow 103, the user intention détection module 12 provides back to the task assistance module 11 an indication of a process required to complété the task. In embodiments, the indication may be a unique identifier for the process.

[0045] Thereafter, the user request clarification module 13 is involved so that the tasks assistance module 11 can interact with the user 14 via chat messages according to the determined process to assist the user with the task. Therefore, in arrow 104, the task assistance module 11 provides the user request clarification module 13 with the information and the determined process. The user request clarification module 13 then détermines chat texts based on the information and the determined process. As described with respect to [Fig.2] below, the user request clarification module 13 may therefore interact with a database of sets of process information 15.

[0046] In arrow 105, the user request clarification module 13 retums an appropriate chat text to provide to the user to the task assistance module 11. The chat text may, e.g., comprise further questions to the user, may comprise a step-by-step description of the process(es) to be performed, and the like. In arrow 106, the task assistance module 11 then interacts with the user 14 via chat messages, which comprise the chat texts of the user request clarification module 13, according to the process, which was determined by the user intention détection module 12, to assist the user with the task. The most basic procedures of the System for assisting a user with performing a task within a technical System as described with respect to [Fig.l] can be enhanced as is described with respect to Figs. 2 to 7. It is apparent to the skilled person that the modules and procedures described therein can be combined or replaced by other procedures if appropriate for the technical System.

[0047] [Fig.2] présents the computing System 10 of assisting a user with performing a task within a technical System further comprising a first memory 15 according to an embodiment. The first memory 15 is configured to store sets of process information. The sets of process information relate to processes for completing tasks. For example, a set of process information is related to one process and comprises ail data that is required to perform the task. In one embodiment, the process information may comprise at least one or more of required input data, target web page or program context, one or more steps to be performed for completing the task, human readable description of the task and / or the one or more process steps, and a title of the task.

[0048] For example, a set of process information for performing the task of adding a user for being allowed to interact with the technical System may be in the form of a template in textual form that reads as follows: "You are a supporting the création of a user. You must request mandatory fields: Firstname, Lastname. Gender, Birthdate. You can also invite the user to provide optional fields: Phone number, Address. Once ail is clarified, you retum a message followed by “I”, then a JSON describing data: {Firstname, Lastname, Birthdate, ...}". In an alternative example, a set of process information for performing the task of adding a user for being allowed to interact with the technical System has a structured form (e.g., JSON) and reads as follows: "{"MandatoryData": ["Firstname", "Lastname", "Gender", "Birthdate"]", "OptionalData": ["Phone number", "Address"], "Return data": ["MandatoryData", "OptionalData"]}". The skilled person will be aware of different formats for storing the sets of process information in the first memory 15.

[0049] In another example, a set of process information for performing the task of changing what is displayed on a technical machine as customized / custom logo may be in the form of a template in a textual form that reads as follows: " You are a supporting the update of the customized / custom logo. You must request a picture, a title and alternative text. Once ail is clarified, you return a message including the foliowing steps to be performed: 1 - Connect to TS terminal; 2 - Go to TS Main > Logo > Update; 3 - Complété the form; 4 - Check and validate. The message is foliowed by “I”, then a JSON describing data: {title:" String", altText:"String", image:"url"}". Hence, the set of process information also comprises a step-by-step solution for the user 14 to perform the task.

[0050] Each set of process information is further associated with a (unique) process identifier. For example, the process identifier for adding a user may be "ADD_USER" and for updating a customized / custom logo "UPD_LOGO". Alternatively, the process identifier may also be a number, such as "123456" for adding a user and "123460" for updating a customized / custom logo. Process identifiers may take any suitable form such that the processes they are referring to can be distinguished.

[0051] The procedures performed in this embodiment are the foliowing as depicted with the arrows in [Fig.2]. The procedures of arrows 101, 102, and 103 are the same as in [Fig.l]. With arrow 201, which corresponds but is not identical to arrow 104 of [Fig.l], the task assistance module 11 provides the user request clarification module 13 with a process identifier of the determined process (which may be provided by the user intention détection module 12 or determined by the tasks assistance module 11 based on other information provided by the user intention détection module 12 in arrow 103) and the information from the user 14 (which was received with arrow 101).

[0052] The user request clarification module 13 then receives at least one set of process information from the first memory 15. Therefore, as shown with arrow 202, the user request clarification module 13 may query the first memory 15 for at least one set of process information by providing the process identifier. The first memory 15 retums the at least one set of process information to the user request clarification module 13. The user request clarification module 13 generates a chat text through prompt engineering according to, i.e., based on, the at least one set of process information of the determined process and the information from the user. The user request clarification module 13 may use generative artificial intelligence, e.g., an LLM chatbot as explained above, to generate the chat text.

[0053] In arrow 203, the task assistance module 11 then receives the chat text for communicating with the user from the user request clarification module 13 and présents, in arrow 204, the user with one or more chat messages according to the received chat text.

[0054] For example, if the detected user intention is to add a user to the technical System, which may hâve the process identifier "ADD_USER" as explained above, the user request clarification module 13 may retrieve from the first memory 15 the set of process information "You are a supporting the création of a user. You must request mandatory fields: Firstname, Lastname. Gender, Birthdate. You can also invite the user to provide optional fields: Phone number, Address. Once ail is clarified, you return a message followedby “I”, then a JSON describing data: {Firstname, Lastname}". This set of process information may be provided to an LLM chatbot with the information received from the user 14. The information may, e.g., be "Hi, I want to add the new user Adam Smith to the System." and "Adam Smith is born on 05 September 1975.". The LLM chatbot then returns "Nice! Here is the process to add Adam Smith as new user: l{Firstname:"Adam", Lastname:"Smith", Birthdate:" 1975-09-05"}".The user 14 may, in this example, only be provided with the first part "Nice! Here is the process to add Adam Smith as new user:" and the last part may be used by the task assistance module 11 to retrieve, e.g., an exécutable script as described with respect to [Fig.5] below to provide to the user. Alternatively, the last part may be used by the task assistance module 11 to retrieve a respective page of a documentation that explains the process to complété the intended task, or the user request clarification module 13 may append further information from the set of process information, possibly not provided to the LLM chatbot, e.g., a description of steps to be performed adapted according to the information comprised by the last part. .

[0055] [Fig.3] présents the computing System 10 of assisting a user with performing a task within a technical System further comprising a second memory 16 according to an embodiment. The second memory 16 is configured to store embeddings of the sets of process information associated with respective process identifiers. Embeddings are représentations of values or objects like text, images, and audio that are designed to be consumed by machine learning models and semantic search algorithms. Hence, the second memory 16 stores vectors that reflect user intentions and map them to tasks or processes for performing the tasks, respectively.

[0056] In [Fig.3], the arrow 101 corresponds to that of [Fig.l] and the arrows 201, 202, 203, and 204 correspond to those of [Fig.2]. In arrow 301, the task assistance module 11 provides the user intention détection module 12 with the information provided by the user 14. The user intention détection module 12 then détermines a process mapping to the information based on the information and the embeddings stored in the second memory 16. In other words, the user intention détection module 12 transforms the information from the user 14 in an embedding (i.e., vector représentation), which is then, e.g., fed to a machine leaming model, e.g., in some embodiments a generative artificial intelligence, to détermine the process mapping to the information., which détermines the most pertinent intent, i.e., task to be performed, relating to the embeddings stored in the second memory 16, as is shown with arrow 302.

[0057] The user intention détection module 12 then extracts the process identifier(s) of the process(es) relevant for completing the task based on the embeddings stored in the second memory 16. In arrow 303, the user intention détection module 12 provides the process identifier of the determined process to the task assistance module 11.

[0058] [Fig.4] présents the computing System 10 of assisting a user with performing a task within a technical System further comprising a third memory 17 and an information extraction module 18 according to an embodiment. The third memory 17 is configured to store a technical documentation 19 of the technical System. The technical documentation may be in natural language format and define processes for completing tasks. For example, if the technical System is a computer executing programs, the technical documentation may be an implémentation or administration guide to one or more of the programs. In another example, if the technical System is a computer-controlled machine, the technical documentation may be a maintenance or administration guide to controlling the machine. The information extraction module 18 may be based on machine leaming and may, in some embodiments, use generative artificial intelligence.

[0059] In [Fig.4], the arrow 101 corresponds to that of [Fig.l], the arrows 201, 202, 203, and 204 correspond to those of [Fig.2], and the arrows 301, 302, and 303 correspond to those of [Fig.3]. In this example, the procedures illustrated with arrows 401, 402, 403, and 404 are pre-processing procedures for assisting the user 14 with tasks. Hence, these procedures are performed before the task assistance module 11 is applied.

[0060] In arrow 401, a technical documentation 19, such as an implémentation, administration, or maintenance guide, is stored in the third memory 17. The technical documentation 19 may, e.g., be in pdf format. The information extraction module 18 obtains the technical documentation 19 from the third memory 17, which is depicted with arrow 402. The information extraction module 18 then extracts a processing block related to a task from the technical documentation 19. Extraction of processing block may comprise finding titles, subtitles, section, pseudo-codes, or otherwise structured information in the technical documentation 19 by the information extraction module 18, e.g., by machine leaming models supporting natural language processing or by generative artificial intelligence.

[0061] The information extraction module 18 further parses the processing block to a format defining detailed information about the respective process. For example, the information extraction module 18 generates the process identifier, extracts input requirements for the process, extracts a target page or context of the process, summarizes the procedures to be performed in a human readable way, describes keywords, activities, interactions with the technical System etc. of users that search for the respective task, and generates a title of the process for the task.

[0062] The information extraction module 18 further generates an embedding related to the processing block based on at least part of the detailed information. For example, the information extraction module 18 may generate the embedding based on the description of keywords, activities, interactions, and the like. In arrow 403, the information extraction module 18 stores the embedding with a respective process identifier in the second memory 16.

[0063] The information extraction module 18 further generates a set of process information of the task based on at least part of the detailed information. For example, the information extraction module 18 may generate the set of process information by applying prompt engineering for an LLM chatbot using a template. In arrow 404, the information extraction module 18 stores the set of process information with the respective process identifier in the first memory 15.

[0064] [Fig.5] présents the computing System 10 of assisting a user with performing a task within a technical System further comprising a fourth memory 20 and a task managing module 21 according to an embodiment. The fourth memory 20 is configured to store action templates, wherein the action templates comprise computer-readable instructions for processes associated with respective identifiers for the processes. The task assistance module 11 requests the task managing module 21 to provide an exécutable script for automatically performing the process to the user 14.

[0065] The procedures according to this embodiment are again basically similar to those of the ones described before, i.e., in [Fig.5], the arrow 101 corresponds to that of [Fig.l], the arrows 201 and 202 correspond to those of [Fig.2], and the arrows 301, 302, and 303 correspond to those of [Fig.3]. It is noted that although not shown the procedures described with respect to [Fig.4] can also be applied in combinations with those of [Fig.5].

[0066] In this example, the user request clarification module 13 further extracts values of input parameters from the information, wherein the input parameters are required to complété the determined process. For example, if a first name, a last name and a birthdate are required input parameters, the respective values may be provided by the user 14 in the information, e.g., "Adam Smith, 05 September 1975". In arrow 501, the user request clarification module 13 provides the one or more values of the input parameters in a structured form to the task assistance module 11. For example, the structured form may then be "{Firstname:"Adam", Lastname:"Smith", Birthdate:"1975-09-05"}".

[0067] In arrow 502, the task assistance module 11 provides the process identifier of the determined process and the one or more values of the input parameters to the task managing module 21. The task managing module 21 then retrieves an action template for the determined process according to the process identifier from the fourth memory 20 as is depicted with arrow 503. Using this action template and the one or more values of the input parameters, the task managing module 21 créâtes an exécutable script. The format of the exécutable script may in some embodiments be adjusted by the task managing module 21 according to a computing device operating System of the user 14 and / or the technical System.

[0068] In arrow 504, the task assistance module 11 received the exécutable script from the task managing module 21, which is then provided, within the chat messages, to the user 14 in arrow 505. The exécutable script may, in some embodiments, be an rpa-file, an exe-file, or the like.

[0069] [Fig.6] is a further development of the computing System 10 of [Fig.5] and présents the computing System 10 for assisting a user 14 with performing a task within a technical System further comprising a template recording module 22. The procedures according to this embodiment are again basically similar to those of the ones described before, i.e., in [Fig.6], the arrow 101 corresponds to that of [Fig.l], the arrows 201 and 202 correspond to those of [Fig.2], the arrows 301, 302, and 303 correspond to those of [Fig.3], and the arrows 502, 503, and 504 correspond to those of [Fig.5]. It is noted that although not shown the procedures described with respect to [Fig.4] can also be applied in combinations with those of [Fig.5].

[0070] In this example, the user request clarification module 13 further détermines whether an action template for the determined process is stored in the fourth memory 20 as shown with arrow 601. In response to no action template for the determined process being stored in the fourth memory 20, the user request clarification module 13 further includes in the chat text, which is transmitted in arrow 602 to the task assistance module 11, a request for recording interactions with the technical System from the user 14 when performing the determined process for the task. For example, the chat text may - additionally to an indication of the steps to be performed - send a request to the user 14 such as: "Do you want to record this process for future uses? If OK, click on the button below to start the recording.". The chat text with the request is additionally provided to the user 14 in arrow 603.

[0071] If the user 14 clicks OK or otherwise confirms the request, the template recording module 22 is initiated and records interactions of the user 14 with the technical System for performing the task, which is depicted with arrow 604. The template recording module 22 créâtes a new action template for the determined process by converting the interactions to machine-readable instructions for the determined process. Finally, the template recording module 22 stores the new action template in the fourth memory 20.

[0072] [Fig.7] présents the computing System 10 for assisting a user 14 with performing a task within a technical System further comprising a fifth memory 23 according to an embodiment. The fifth memory 23 stores a prompt history of the one or more natural language chat messages from the user 14 and the chat messages provided to the user 14. In this example, the task assistance module 11 stores the one or more natural language chat messages from the user 14 and the chat messages provided to the user 14 in the fifth memory 23 and obtains the one or more natural language chat messages from the user 14 and the chat messages provided to the user 14 stored in the fifth memory 23 for providing them to the user intention détection module 12 and / or the user request clarification module 13. This interaction of the task assistance module 11 is depicted with arrow 701. It is noted that - although the arrows 101 to 106 reflect procedures described with respect to [Fig.l], the example of [Fig.7] is also combinable with the embodiments of Figs. 2 to 6.For example, data stored in the prompt history of the fifth memory 23 can be used by the user intention détection module 12 to détermine the intent of the user 14 and / or by the user request clarification module 13 to generate the chat texts for provision to the user 14. In this example, the user intention détection module 12 and / or the user request clarification module 13 are stateless, i.e., the respective module is not aware of the State of the communication with the user 14. As apparent, if both modules were stateful, i.e., aware of the State of the communication with the user 14, the fifth memory 23 storing the prompt history may not be required. .

[0073] Moreover, the example of [Fig.7] also shows an optional component, namely, a messaging System 24 to which the task assistance module 11 is connected for chatting with the user 14. Messaging Systems according to this disclosure may be, e.g., MSTeams, Slack, Messenger, WhatsApp, and the like. The task assistance module 11 (or even the whole computing System 10) may altematively be directly implemented as add-on within the messaging System 24. Further altematively, the task assistance module 11 (or even the whole computing System 10) may be a browser extension connected to a web page over which the chat messages are provided.

[0074] [Fig.8] shows an example of chat messages of the computing System 10 for assisting a user 14 with performing a task within a technical System according to an embodiment. In this example, the user 14 (right hand side chat messages) may hâve connected to a chatbot provided by the task assistance module 11 via a messaging System 24. The chatbot asks in message 81 the user 14 directly, what the intent of the user 14 is. Hence, we herein consider the pro-active mode as explained above. The user 14 then indicates in message 82 what the user 14 wants to do. In this example, the user 14 wants to create a user for a software application called "TS control", e.g., for controlling a technical System. As explained with respect to Figs. 1 to 7, the user intention détection module 12 may then be involved to détermine the task the user intents to perform, which is in this example the process with process identifier ADD_USER (as explained above).

[0075] The chatbot then responds in message 83 with a request for additional information, which relates to minimum and optional input parameters for the process determined. The request may be based on information provided by the user request clarifying module 13 for the process with process identifier ADD_USER. Hence, the chatbot asks for providing a first name, a last name, a date of birth and optionally also address and phone number. The data is provided by the user 14 in message 84.

[0076] The computing System 10, in particular, the task assistance module 11 controlling the chatbot, now has sufficient information and provides the user 14 in message 85 with a step-by-step solution for providing the task. Additionally, the chatbot may provide the user 14 in message 86 with an exécutable script to automatically perform the task for the user 14.

[0077] [Fig.9] is a diagrammatic représentation of internai components of a computing System 90 implementing the functionality of one or more of the components as described herein. The computing System 90 includes at least one processor 91, a user interface 92, a network interface 93 and a main memory 96, that communicate with each other via a bus 95. Optionally, the computing System 90 may further include a static memory 97 and a disk-drive unit (not shown) that also communicate with each via the bus 95. A video display, an alpha-numeric input device and a cursor control device may be provided as examples of user interface 92. Furthermore, the computing System 90 may also comprise one or more graphies processing units (GPU) 94.

[0078] The GPUs 94 may also comprise a plurality of GPU cores or streaming multiprocessors, which comprise many different components, such as at least one register, at least one cache and / or shared memory, and a plurality of ALUs, FPUs, tensor processing unit (TPU) or tensor cores, and / or other optional processing units. GPUs can perform multiple simultaneous computations, thereby enabling the distributing of training processes and speeding up machine leaming operations.

[0079] The main memory 96 may be a random-access memory (RAM) and / or any further volatile memory. The main memory 96 may store program code 98a and may also store additional program data 98b required for providing the functionalities described herein. Moreover, the main memory 96 may also include a cache 99.

[0080] According to an aspect, a computer program comprising instructions is provided. These instructions, when the program is executed by a computer, cause the computer to carry out the methods described herein. The program code embodied in any of the Systems described herein is capable of being individually or collectively distributed as a program product in a variety of different forms. In particular, the program code may be distributed using a computer readable storage medium having computer readable program instructions thereon for causing a processor to carry out aspects of the embodiments described herein.

[0081] Computer readable storage media, which are inherently non-transitory, may include volatile and non-volatile, and removable and non-removable tangible media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Computer readable storage media may further include random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid State memory technology, portable compact dise read-only memory (CD-ROM), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and which can be read by a computer.

[0082] A computer readable storage medium should not be construed as transitory signais per se (e.g., radio waves or other propagating electromagnetic waves, electromagnetic waves propagating through a transmission media such as a waveguide, or electrical signais transmitted through a wire). Computer readable program instructions may be downloaded to a computer, another type of programmable data processing apparatus, or another device from a computer readable storage medium or to an external computer or external storage device via a network.

[0083] It should be appreciated that while particular embodiments and variations hâve been described herein, further modifications and alternatives will be apparent to persons skilled in the relevant arts. In particular, the examples are offered by way of illustrating the principles, and to provide a number of spécifie methods and arrangements for putting those principles into effect.

[0084] In certain embodiments, the functions and / or acts specified in the flowcharts, sequence diagrams, and / or block diagrams may be re-ordered, processed serially, and / or processed concurrently without departing from the scope of the disclosure. Moreover, any of the flowcharts, sequence diagrams, and / or block diagrams may include more or fewer blocks than those illustrated consistent with embodiments of the disclosure.

[0085] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments of the disclosure. It will be further understood that the terms "comprise" and / or "comprising," when used in this spécification, specify the presence of stated features, integers, processes, operations, éléments, and / or components, but do not preclude the presence or addition of one or more other features, integers, processes, operations, éléments, components, and / or groups thereof. Furthermore, to the extent that the terms "include", "having", "has", "with", "comprised of", or variants thereof are used in either the detailed description or the daims, such terms are intended to be inclusive in a manner similar to the term "comprising".

[0086] While a description of various embodiments has illustrated the method and while these embodiments hâve been described in considérable detail, it is not the intention of the applicants to restrict or in any way limit the scope of the appended daims to such detail. Additional advantages and modifications will readily appear to those skilled in the art. The disclosure in its broader aspects is therefore not limited to the spécifie details, représentative apparatus and method, and illustrative examples shown and described. Accordingly, the described embodiments should be understood as being provided by way of example, for the purpose of teaching the general features and principles, but should not be understood as limiting the scope, which is as defined in the appended daims.

Claims

1.

2. Demands A computer system to assist a user in performing a task within a technical system, the computer system comprising: a user intent detection module configured to detect user intentions when interacting with the technical system; a user request clarification module configured to determine chat texts to assist the user in the task; and a task assistance module configured to: • receive information from the user, in which the information relates to one or more natural language chat messages and / or an observation of the user's interaction with the technical system; • determine, by applying the user intent detection module, a process to execute the task based on the information; and • interact, by applying the user request clarification module, with the user via chat messages in accordance with the determined process to assist the user in the task. The system according to claim 1 further comprises: a first memory configured to store sets of process information, wherein the sets of process information relate to task execution processes, in which a set of process information is associated with a process identifier; in which the task assistance module is configured to interact with the user via chat messages, and is also configured to: • provide the user request clarification module with a process identifier for the determined process and the user's information; • receive a chat text from the user request clarification module to communicate with the user; and • present the user with one or more chat messages in accordance with the received chat text; in which the user request clarification module is further configured to: • receive the process ID of the determined process and the user information from the task assistance module; • receive at least one set of process information in accordance with the process ID; • generate a chat text via rapid engineering in accordance with at least one set of process information from the determined process and the user information; and • provide the chat text to the task assistance module.

3. The system according to claim 2, wherein the process information set further comprises at least one or more input data, target web page or program context, one or more steps to be executed to perform the task, a human-readable description of the task and / or said one or more steps, and a title.

4. The system according to claim 2 or claim 3, wherein the user request clarification module uses generative artificial intelligence to generate the chat text.

5. The system according to any one of claims 2 to 4 further comprises: a second memory configured to store integrations of the information sets associated with the respective process identifiers; wherein the task assistance module is, for the purpose of determining the process to execute the task, further configured to: • provide information to the user intent detection module; and • receive a process identifier from the process to execute the task, from the user intent detection module; and in which the user intent detection module is configured to: • receive information from the task assistance module; • determine a process match with information based on information and integrations stored in the second memory; and • provide the process ID of the determined process to the task assistance module.

6. The system according to claim 5, wherein the user intent detection module uses generative artificial intelligence to determine the match of the process with the information.

7. The system according to claim 5 or claim 6 further comprises: a third memory configured to store technical documentation of the technical system, in which the technical documentation is in natural language and defines processes for performing tasks, and an information extraction module configured to: • obtain the technical documentation in a third memory; • extract a processing block related to a task from the technical documentation; • parse the processing block in a format defining detailed information about the respective process; • generate an integration related to a processing block based on at least a portion of the detailed information; • store the integration with a respective process identifier in the second memory; • generate a set of information about the task process based on a portion of the detailed information;and • store all information about the process with the respective process identifier in the first memory.;

8. The system according to claim 7, wherein the technical system is a computer executing programs, in which Technical documentation is a guide for the implementation or administration of one or more programs, or in which the technical system is a computer-controlled machine, in which the technical documentation is a maintenance or administration guide for controlling the machine.

9. The system according to claim 7 or claim 8, wherein the information extraction module uses generative artificial intelligence to extract at least one processing block and / or to analyze said at least one processing block.

10. The system according to any one of the preceding claims further comprises: a fourth memory configured to store action patterns, wherein the action patterns comprise computer-readable instructions for processes associated with respective identifiers for the processes, and a task management module; wherein the user request clarification module is further configured to: • extract one or more input parameter values ​​from the information, wherein the input parameters are required to perform the determined process; and • provide said one or more input parameter values ​​in a structured form to the task assistance module;in which the task assistance module is, for interacting with the user via chat messages, further configured to: • provide the process ID of the determined process and said one or more values ​​of the input parameters to the task management module; • receive an executable script to automatically execute the process from the task management module; and • provide, in chat messages, the executable script to the user; • in which the task management module is configured to:; • receive the process ID of the determined process and said or one more values ​​of the input parameters, from the task helper module; • retrieve an action pattern for the determined process according to the process ID in the fourth memory; • create an executable script based on an action pattern and said one or more values ​​of the input parameters; and • provide the executable script to the task helper module.

11. The system according to claim 10, wherein the format of the executable script is adjusted in accordance with a user's computer device operating system and / or technical system.

12. The system according to claim 10 or claim 11 further comprises: a pattern recording module; wherein the user request clarification module is further configured to: • determine whether an action pattern for the specified process is stored in the fourth memory; and • in response to no action pattern for the specified process being stored in the fourth memory, include in the chat text a request to record the user's interactions with the technical system when it executes the specified process for the task; wherein the pattern recording module is, in response to the user acknowledging the recording, configured to: • record the user's interactions with the technical system to execute the task;• create a new action model for the determined process by converting the interactions into machine-readable instructions for the determined process; • store the new action model in the fourth memory.

13. The system according to any one of the preceding claims comprises: a fifth memory configured to store a brief history of one or more of the user's natural language chat messages and the chat messages provided to the user; wherein the task assistance module is further configured to: • store one or more of the user's natural language chat messages and the chat messages provided to the user in the fifth memory; • retrieve said one or more natural language chat messages and the chat messages stored in the fifth memory to provide them to the user intent detection module and / or the user request clarification module

14. The system according to any one of the preceding claims, wherein chat messages provide a step-by-step solution to the process required to perform the task.

15. The system according to any one of the preceding claims, wherein the task assistance module is connected to or an add-on of a messaging system on which chat messages are provided, or the task assistance module is a browser extension connected to a web page on which chat messages are provided.