Method and assistance system for supporting a user in operating a device

Abstract

The invention relates to a method and an assistance system for supporting a user in operating a device. In order to train the assistance system (A), a plurality of first operating sequences (BS1) of different first users (U1) of the device (DEV) are recorded. A quality indication (Q) is assigned to each of the first operating sequences (BS1). Furthermore, a pre-trained large language model (LLM) is fine-tuned on the basis of the first operating sequences (BS1) and the assigned quality indications (Q) such that a first response text (A1), derived from a first operating sequence (BS1), reproduces, at least on average, a quality indication (Q) assigned to said first operating sequence (BS1). During ongoing operation of the device (DEV), second operating sequences (BS2) of a second user (U2) are then recorded in a time-resolved manner. The second operating sequences (BS2) are fed into the fine-tuned large language model (LLM) which generates second response texts (A2) on the basis thereof. Depending on the second response texts (A2), a user interface (IO) is prompted to reduce or increase a number of outputs (H) provided to support the second user (U2) in operating the device (DEV).

Description

[0001] Description

[0002] Method and assistance system for user support when operating a device

[0003] The correct and efficient operation of a device often requires specific knowledge and skills from the user. This is particularly true for complex devices, such as control units for machine tools, production plants, turbines, logistics systems, or design systems, as well as graphic editing systems, video editing systems, or other highly structured control or design systems. In these cases, users are often confronted with complex user interfaces that have a steep learning curve.

[0004] To support users, assistance systems are often provided that offer help to the user during operation, e.g. in the form of tooltips or when a help button is pressed.

[0005] However, the assistance systems themselves often require a certain level of user experience and are frequently difficult for inexperienced users to understand. On the other hand, experienced users can usually do without much assistance and are often hindered by too many prompts and security checks.

[0006] The object of the present invention is to provide a method and an assistance system for supporting a user in operating a device, which allows more efficient control of the device by different users.

[0007] This problem is solved by a method with the features of claim 1, by an assistance system with the features of claim 13, by a computer program product with the features of claim 14, and by a computer-readable storage medium with the features of claim 15.

[0008] To support users in operating a device, a large number of initial operating sequences from different first users of the device are recorded with time resolution. Each initial operating sequence is assigned a quality rating specifying the operating quality. Furthermore, a large language model, such as GPT-4, GPT-3, BLOOM, LLaMA, T5-11B, PaLM-E, Gemini Pro, or Mixtral 8x7b, is provided, which is pre-trained to generate related response texts from input texts. Such large language models are often abbreviated as LLM (Large Language Model) and have been generally available or usable for some time. In particular, a so-called generative pre-trained transformer, abbreviated GPT (Generative pre-trained Transformer), can be used as the large language model.The pre-trained large language model is retrained using the time-resolved first operating sequences and the associated quality information so that each first response text derived from a first operating sequence fed in as input text reproduces a quality information assigned to this first operating sequence, at least on average.

[0009] During operation, the device captures second user's input sequences with time resolution. These time-resolved second input sequences are fed into the retrained large-scale language model, which then generates second response texts. Depending on these generated response texts, a user interface is triggered to decrease or increase a number of outputs intended to support the second user in operating the device, particularly help texts, warning signals, safety prompts, or operating suggestions. These outputs can be triggered during operation to continuously guide and / or support the user.

[0010] The first and second operating sequences may each include one or more operating actions, such as control commands, keyboard inputs, mouse clicks, mouse movements, voice inputs, touchpad inputs, joystick inputs, or the operation of switches, levers, rotary knobs, or other controls.

[0011] To carry out the method according to the invention, an assistance system, a computer program product and a computer-readable, preferably non-volatile storage medium are provided.

[0012] The inventive method, the inventive assistance system and the inventive computer program product can be implemented or executed in particular by means of one or more computers, one or more processors, application-specific integrated circuits (ASIC), digital signal processors (DSP), a cloud infrastructure and / or so-called "Field Programmable Gate Arrays" (FPGA).

[0013] A particular advantage of the invention is that users with varying levels of experience can receive user-specific and situation-specific support when operating a device. Specifically, unnecessary assistance can be suppressed for expert users, while additional assistance can be generated for inexperienced users. In this way, the operation of the device can generally be significantly improved.

[0014] Furthermore, inexperienced users become familiar with the operation of the device more quickly. The method according to the invention can automatically adapt to the user's increasing operating expertise.

[0015] Advantageous embodiments and further developments of the invention are specified in the dependent claims.

[0016] According to an advantageous embodiment of the invention, the second response texts can be at least partially output to the user via the user interface. Insofar as the large language model is retrained to reproduce quality information in its response texts, second response texts can be output to the user, in particular, as informative feedback regarding the current service quality during the operation of the device.

[0017] According to a further advantageous embodiment of the invention, the second operating sequences can be modified depending on the second response texts. The device can then be controlled using the modified second operating sequences. In this way, mouse movements identified as unsafe can be smoothed out, or individual operating actions within an operating sequence can be suppressed, depending on the second response texts, at least until confirmation of an additional security prompt. Alternatively or additionally, the device's implementation of the second operating sequences can be controlled depending on the second response texts. This can improve the detection or interpretation of unsafe operating sequences in many cases.

[0018] Furthermore, for initial operating sequences

[0019] - the duration of an operating action and / or the duration between operating actions of a respective first operating sequence are measured,

[0020] - Repetitions and / or reversals of operating actions from the respective first operating sequence are counted, and / or

[0021] - The performance and / or error rate of the device controlled by the respective first operating sequence can be determined. Based on this, a corresponding quality rating can be generated and assigned to the respective first operating sequence. Performance can be measured as efficiency, speed, effectiveness, success, emissions, resource consumption, or any other metric that quantifies the fulfillment of the device's intended task. In particular, operating sequences can be assigned a comparatively lower operating quality if they are relatively slow, involve long pauses, many repetitions, or many reversals, or if they result in a relatively high error rate or relatively low performance.Accordingly, operating sequences can be assigned a comparatively higher operating quality if they run relatively quickly, have short pauses, few repetitions or few retractions, or if they lead to a relatively low error rate or a relatively high performance.

[0022] According to a further advantageous embodiment of the invention, during retraining the large language model, the degree of agreement between a derived first response text and its corresponding quality rating can be determined and quantified as a corresponding agreement value. Accordingly, the large language model can be retrained to increase the agreement values, at least on average. Alternatively or additionally, during retraining the large language model, the degree of deviation between a derived first response text and its corresponding quality rating can be determined and quantified as a corresponding deviation value. Accordingly, the large language model can be retrained to decrease the deviation values, at least on average.A respective agreement or deviation value can be used as a measure of how accurately a given initial response text reproduces a given quality specification.

[0023] In particular, to determine the respective degree of agreement, a prompt to output a value indicating the degree of agreement can be fed into the large language model as input text. The respective agreement value can then be derived from the output. In this way, the ability of large, pre-trained language models to answer questions about textual relationships relatively accurately can be used to generate training data for further training. According to a further advantageous embodiment of the invention, the large language model can be provided with a user ID of each of the first and second users as input text, respectively, along with the first and second user's operating sequences. In this way, the large language model can learn to distinguish between different users and recognize them based on their operating sequences.

[0024] Furthermore, the large language model can also be supplied with a currently recorded operating state of the device in the form of an input text via a respective first and / or second operating sequence.

[0025] In this way, the large language model can learn dependencies and / or correlations between operating states and user sequences. Such additional information, and / or the ability to infer current operating states from user sequences, can typically significantly improve the predictive quality of the large language model.

[0026] According to a further advantageous embodiment of the invention, the quality indicator assigned to a first user's operating sequence can include information about that first user's operating expertise. This information can, in particular, specify the first user's training level and / or the number of operating hours or years of service. Alternatively or additionally, operating expertise can also be derived from the measurements performed on the operating sequences as described above. In this way, the large language model can learn to distinguish between experienced and inexperienced users based on their operating sequences.

[0027] Furthermore, the large language model can be retrained to generate information about a similarly functioning operating sequence with higher user-friendliness from an initial operating sequence. Since the large language model is already being retrained to predict a quality indicator specifying user-friendliness, this indicator can be used by the large language model to select, from among several similarly functioning operating sequences, the one with the highest or particularly high user-friendliness. During device operation, information about a similarly functioning operating sequence with higher user-friendliness, generated from a second operating sequence, can then be output to the user via the user interface, for example, as an operating suggestion or improvement recommendation. Alternatively or additionally, such a similarly functioning operating sequence can be output, at least partially, directly to the device.According to a further advantageous embodiment of the invention, user input text from a second user can be read through the user interface. This user input text can then be fed into the large language model to train it through prompt-based learning, in particular through so-called few-shot learning, during the device's operation. Prompt-based learning allows the internal state of the large language model to be modified by targeted user input in order to adapt or control the large language model in a context-specific manner. For example, a particularly good example of an advantageous operating action, information on whether a current operating suggestion is advantageous or not, and / or an instruction that GitHub should preferably be used can be entered as user input text at the prompt.

[0028] According to a further advantageous embodiment of the invention, second operating sequences of several second users can be recorded. A quality rating can then be derived from the second response texts of each second user and assigned to that user. Furthermore, device error rates can be recorded on a user-specific basis, and a statistical correlation between the recorded error rates and the corresponding quality ratings can be determined. The determined correlation can then be compared with a predefined threshold, whereby if the predefined threshold is not met, the device, the user interface, or a software version of the device can be marked as error-prone and / or a function of the device can be restricted.The above procedure is based on the understanding that if error rates correlate only weakly with user-friendliness, this often indicates problems with the user interface design or other program or device issues. In particular, in cases where experienced and inexperienced users frequently make the same or similar operating errors, a corresponding comment can be automatically added or a device function can be restricted.

[0029] An embodiment of the invention is explained in more detail below with reference to the drawing. The drawings illustrate, in schematic form:

[0030] Figure 1 shows a training of a learning-based assistance system to support users in operating a device, and Figure 2 shows an application of the trained assistance system.

[0031] Insofar as the same or corresponding reference symbols are used in different figures, these reference symbols denote the same or corresponding entities, which may be described, implemented or designed in particular as in connection with the figure in question.

[0032] Figure 1 illustrates the training of a learning-based assistance system A, designed to support users in operating a device DEV. The device DEV can be, in particular, a control unit for controlling manufacturing plants, robots, turbines, logistics systems, energy networks, machine tools, or other machines. Furthermore, the device DEV can be a graphic editing system, a video editing system, or another structure-rich control system, design system, or work tool.

[0033] The assistance system A is intended to support and / or improve the operation of the DEV device by different users in a user- and situation-specific manner.

[0034] The assistance system A has one or more PROC processors for executing process steps according to the invention, as well as one or more MEM storage devices for storing data to be processed by the assistance system A.

[0035] Assistance system A also includes a CTL controller to which the DEV device is connected via a device interface of the CTL controller. The CTL controller continuously acquires status data ST of the DEV device, which specifies its current operating state. The status data ST can include, in particular, measurement data, sensor data, environmental data, or other data generated during or influencing the operation of the DEV device, especially data on power, energy consumption, temperature, pressure, actuator positions, valve positions, forces, and / or emissions of the DEV device.

[0036] To control the DEV device, the CTL controller outputs control signals CS to the DEV device.

[0037] Furthermore, the assistance system A has a user interface (IO) for interacting with users of the device DEV. For reading operating sequences and other user actions, the IO user interface can include a keyboard, mouse, touchpad, touchscreen, joystick, camera, gesture control, microphone, and / or voice control. Output of operating-related information to a given user can be provided via a screen, speaker, haptic feedback, and / or an optical or acoustic signal generator of the IO user interface.

[0038] To train the assistance system A, user actions from a large number of different users are evaluated. Each user operating the device DEV during the training is referred to below as the first user U1.

[0039] The first user U1 interacts with the device DEV via the assistance system A. For this interaction, a multitude of operating sequences BS1 performed by this first user U1 are recorded in temporal resolution via the user interface IO, along with a user ID1 of the first user U1. Each operating sequence BS1 can include, in particular, one or more operating actions by the first user U1, such as control commands, keyboard inputs, mouse clicks, mouse movements, voice inputs, touchpad inputs, joystick inputs, or the operation of switches, levers, rotary knobs, or other controls.

[0040] In time-resolved data acquisition, the temporal progression of operating sequences BS1 is measured, specifically the durations of operating actions and / or the time intervals between successive operating actions. The operating sequences BS1 can each be coded or represented by an operating sequence identifier that specifies the operating actions contained within and their timing.

[0041] The operating sequences BS1 are fed from the user interface IO into the controller CTL and into a quality assessor QEV of the assistance system A. The controller CTL derives control signals CS from the fed operating sequences BS1, which are transmitted to the device DEV to control it. In parallel, the controller CTL continuously acquires current status data ST of the device DEV. The acquired status data ST specifies the current operating state of the device DEV, which is controlled by the fed operating sequences BS1. The acquired current status data ST is fed by the controller CTL into the quality assessor QEV.

[0042] The quality evaluator QEV serves the purpose of evaluating the input operating sequences BS1 with regard to their operating quality. In addition to each operating sequence BS1 itself, the operating state of the device DEV induced by the operating sequence is also taken into account. The induced operating state is derived by the quality evaluator QEV from the input state data ST. In particular, the current performance and / or error rate of the device DEV controlled by the respective operating sequence BS1 is derived from the state data ST.

[0043] When assessing the operating quality of the respective operating sequence BS1, the quality assessor QEV takes into account, in particular, the durations of operating actions contained in the respective operating sequence BS1, the durations between successive operating actions, repetitions of operating actions, reversals of operating actions, as well as the current performance and the current error rate of the device DEV.

[0044] The quality evaluator QEV assigns a comparatively lower user quality to the respective operating sequence BS1 if it runs relatively slowly, has long pauses, many repetitions or many retractions, or if it leads to a relatively high error rate or relatively low performance. Conversely, a comparatively higher user quality is assigned to the respective operating sequence BS1 if it runs relatively quickly, has short pauses, few repetitions or few retractions, or if it leads to a relatively low error rate or relatively high performance.

[0045] The quality evaluator QEV quantifies the operating quality determined for each operating sequence BS1 by assigning a quality rating Q to the respective evaluated operating sequence BS1. The quality rating Q can specify the operating quality using one or more numerical values ​​and / or in text form. The latter can include, in particular, a description of aspects of the operating quality, such as "Slow operation," "Erratic operation," "Program response time is too long," "Device shows poor performance," or "Device shows a high error rate."

[0046] Alternatively or additionally, the quality rating Q can also include the operating expertise of the respective first user U1. This operating expertise can be derived from the above quality assessment of operating sequences BS1 and / or from other available information about user U1.

[0047] The operating sequences BS1 derived from the multitude of first users U1

[0048] Quality specifications Q are used to retrain a pre-trained large language model LLM of the assistance system A to reproduce corresponding quality specifications based on input operating sequences.

[0049] Such a large language model is often referred to as a "Large Language Model" and abbreviated as LLM. Through extensive pre-training with an immense amount of text, a large language model acquires the ability to generate related and, in many cases, relatively correct response texts, especially in natural language, based on input texts.

[0050] Training is generally understood as the optimization of a mapping of input data from a machine learning model, in this case the large language model LLM, to its output data. This mapping is optimized according to one or more predefined criteria during one or more training phases. For language models, for example, the highest possible response accuracy can be used as a criterion. Through training, the network structures of neurons in a neural network and / or the weights of connections between neurons can be adjusted or optimized so that the predefined criteria are met as effectively as possible. Training can thus be understood as an optimization problem. A multitude of efficient optimization methods are available for such optimization problems in the field of machine learning.

[0051] A large language model (LLM) can preferably be a so-called generative pre-trained transformer (GPT), particularly one based on an artificial neural network. Examples include models like GPT-4, GPT-3, BLOOM, LLaMA, T5-11 B, PaLM-E, Gemini Pro, or Mixtral 8x7b. Ideally, a large language model (LLM) should be chosen that has also been pre-trained with content from one or more software repositories, such as GitHub. In this way, the large language model (LLM) typically acquires the ability to generate, annotate, or make statements about software.

[0052] To retrain the large language model LLM, the user interface IO feeds it the operating sequences BS1 of the respective first user U1 in the form of operating sequence specifications, along with the user ID1, as input text. In addition, the current state data ST is fed into the large language model LLM in the form of input text. Retraining a large language model is often also referred to as fine-tuning. A corresponding Wikipedia article (https: / / en.wikipedia.org / wiki / Fine-tuning_(deep_learning), accessed on April 17, 2024) provides an overview and references to relevant training methods.

[0053] From each input operating sequence BS1 and the corresponding input texts ID1 and ST, the large language model LLM generates an initial response text A1. One goal of the retraining is that each response text generated from an operating sequence describes or otherwise reproduces the operating quality of that sequence as accurately as possible.

[0054] For this purpose, the first response text A1, generated based on a given operating sequence BS1, is fed by the large language model LLM to a comparison unit CP of the assistance system A. Simultaneously, the quality indicator Q associated with the respective operating sequence BS1 is fed into the comparison unit CP by the quality evaluator QEV. The comparison unit CP then determines the degree of agreement between the first response text A1 and the associated quality indicator Q and quantifies this degree of agreement using a agreement value C.

[0055] The degree of agreement can be determined, for example, by the semantic similarity between the first response text A1 and the quality indicator Q. A variety of efficient semantic analysis or semantic correlation methods are available for determining such a degree of agreement and, in particular, such semantic similarity between text-based statements.

[0056] Alternatively or additionally, the large language model LLM itself can be used to determine the respective degree of agreement between the respective first response text A1 and the respective assigned quality rating Q.

[0057] For this purpose, the associated quality indicator Q is fed into the large language model LLM as additional input text. Furthermore, a request RAC, which requests the output of a statement regarding the degree of agreement between the quality indicator Q and the first response text A1, is fed into the large language model LLM as further input text. The request RAC could, for example, contain the text "To what extent does your last answer correspond in content with the quality indicator?" The resulting statement AC regarding the degree of agreement, which is then transmitted by the large language model LLM, is then sent to an evaluation unit EV of the assistance system A. The evaluation unit EV ultimately derives a quantifiable agreement value C from the statement AC.

[0058] The agreement values ​​C determined by the comparison unit CP and / or the evaluation unit EV are fed back to the large language model LLM, as indicated by dotted arrows in Figure 1. Depending on this, model parameters of the large language model LLM that are to be optimized during retraining are adjusted within an optimization procedure such that the agreement values ​​C are maximized, at least on average, during the course of the optimization procedure.

[0059] This retraining enables the large language model LLM to generate response texts that accurately describe or otherwise reproduce the user experience of a given user sequence. Such response texts can include descriptions of various aspects of user experience, such as "Slow operation," "Erratic operation," "Program response takes too long," "Device shows poor performance," or "Device shows a high error rate."

[0060] In this context, it should be noted that a large language model can often make certain predictions about command sequences even without further training, provided it has been pre-trained with content from software repositories and can often abstract relatively well. However, further training generally significantly improves the accuracy of generated response texts. Optionally, user manuals for the DEV device, user manuals for similar devices, or user manuals for a large number of other devices can also be used to retrain the large language model LLM.

[0061] Advantageously, the large language model LLM can also be retrained to derive information about a similarly functioning operating sequence with higher user-friendliness from an input operating sequence and output this as a response text. Since the large language model LLM is already being retrained to predict the user-friendliness of an operating sequence, the predicted user-friendliness can be used to select the operating sequence with the highest or particularly high user-friendliness from among several similarly functioning sequences. In many cases, a large language model pre-trained using large software repositories can already recognize the similarity of operating sequences with acceptable accuracy. Alternatively or additionally, the large language model LLM can be retrained using user manuals in this regard.

[0062] Figure 2 illustrates an application of the assistance system A after the large language model LLM has been retrained as described above. The trained assistance system A is intended to support different users in operating the device DEV. Each user to be supported by the trained assistance system A is referred to below as the second user U2.

[0063] The second user, U2, interacts with the device DEV via the assistance system A. For this second user, operating sequences BS2 are recorded in temporal resolution using the user interface IO, along with a user ID2 of the second user. The recording of operating sequences BS2 and user ID2 is carried out in the same manner as the recording of operating sequences BS1 and user ID1 described above.

[0064] The user interface IO includes a help system HS for the preferably interactive support of user U2. The help system HS can, in particular, output help information or other operation-related and / or situation-specific output, for example in the form of help texts, warning signals, safety queries or operating suggestions, to the second user U2.

[0065] The operating sequences BS2 and the user ID2 of the second user U2 are fed into the retrained large language model LLM as input text from the user interface IO during the operation of the device DEV. Furthermore, status data ST, acquired by the controller CTL and specifying the current operating state of the device DEV, is fed into the retrained large language model LLM in the form of input text.

[0066] From each input sequence BS2 and the corresponding input texts ID2 and ST, the retrained large language model LLM generates a second response text A2. Due to the training of the large language model LLM described above, the second response text A2 is expected to describe the user experience of the input sequence BS2 relatively accurately. If the large language model LLM has also acquired the ability—as described above—to additionally derive information BQ2 about a user sequence with higher user experience that is equivalent to BS2, this information BQ2 can preferably be transmitted by the retrained large language model LLM to the user interface IO. Alternatively or additionally, the information BQ2 can also be output as part of the second response text A2.

[0067] The second response text A2, derived from the respective operating sequence BS2, is transmitted by the retrained large language model LLM to an input processing unit (IOP) of the assistance system A, into which the respective operating sequence BS2 is also fed. The input processing unit IOP modifies the operating sequence BS2 depending on the corresponding second response text A2. For example, a second response text such as "the first action was undone by the second action" can cause the first two operating actions to be removed from the operating sequence BS2. Similarly, a second response text such as "the mouse movements appear unsteady" can cause the mouse movements in the operating sequence BS2 to be smoothed. In this way, the detection or interpretation of unsteady operating sequences can be improved in many cases.

[0068] The operating sequences BS2', modified by the input processing unit (IOP), are transmitted from the IOP to the control unit (CTL). The CTL then derives control signals CS from the modified operating sequences BS2', which are subsequently transmitted to the DEV device for control.

[0069] Furthermore, the second response texts A2 from the retrained large language model LLM are also transmitted to the user interface IO and, in particular, fed into the help system HS. Depending on the second response texts A2, the help system HS is prompted to decrease or increase a number of outputs H intended to support the second user U2. These outputs H can include, in particular, user-relevant and / or situation-specific help information, help texts, warning signals, safety prompts, or operating suggestions.

[0070] The help system HS can recognize text components in the second response texts A2, such as "Slow operation," "Erratic operation," "Program response takes too long," "Device shows poor performance," or "Device shows high error rate," and consequently increase the number of outputs H or expand their content. Specifically, additional help texts, warning signals, security prompts, or operating suggestions can be displayed. Conversely, the help system HS can recognize text components with the opposite meaning in the second response texts A2 and consequently decrease the number of outputs H or shorten their content. Specifically, intended help texts, warning signals, security prompts, or operating suggestions can be suppressed.In this way, additional assistance can be generated for inexperienced users and unnecessary assistance can be suppressed for experts, so that the level of support can be automatically adapted to the respective user expertise.

[0071] Furthermore, the user interface IO can also be used to output second response texts A2, or parts thereof, as informative feedback regarding the current user experience quality to the second user U2 during operation. Similarly, information BQ2 about equivalent operating sequences with higher user experience quality can be output to the second user U2 as operating suggestions during operation.

[0072] Furthermore, by means of the retrained large language model LLM and a correlation device COR of the assistance system A, conclusions can be automatically drawn about the quality of the user guidance, the program quality and / or the device quality.

[0073] For this purpose, operating sequences BS2 from several second users U2 are recorded and fed to the retrained large language model LLM, each assigned to the corresponding user ID2 and the corresponding state data ST. From the input texts BS2, ID2, and ST, the retrained large language model LLM derives second response texts A2, as described above, which are fed into the correlation unit COR, each assigned to the corresponding user ID2 and the corresponding state data ST.

[0074] The correlation unit COR derives a quality rating Q, quantifying the operational quality of the relevant operating sequence BS2, from a respective second response text A2 and assigns this quality rating Q to the corresponding user ID2. Furthermore, the correlation unit COR determines a current error rate ER of the device DEV based on the corresponding status data ST and also assigns this error rate ER to the corresponding user ID2 and thus to the corresponding quality rating Q.

[0075] The correlation unit COR then correlates the error rates ER across all users U2 with the associated quality ratings Q and calculates a statistical correlation factor CC that quantifies the correlation. This CC is then compared by the COR to a predefined threshold TH of, for example, 0.1. If the threshold TH is not met, the COR generates and outputs a message MS. The message MS is therefore generated in cases where the error rates correlate only weakly with user service quality, or where experienced users make a similar number of errors as less experienced users. This often indicates problems with the design of the user interface IO or other program or device issues.In such cases, the message MS can be used to mark the user interface IO or a software version of the device DEV as fault-prone and / or to restrict a function of the device DEV.

Claims

Patent claims 1. A computer-implemented method for user support in operating a device (DEV), wherein a) a large number of first operating sequences (BS1) from different first users (U1) of the device (DEV) are captured with time resolution, b) each first operating sequence (BS1) is assigned a quality indicator (Q) specifying an operating quality, c) a large language model (LLM) is provided that is pre-trained to generate related response texts from input texts, d) the pre-trained large language model (LLM) is retrained using the time-resolved first operating sequences (BS1) and the assigned quality indicators (Q) so that each first response text (A1) derived from a first operating sequence (BS1) fed in as input text reproduces a quality indicator (Q) assigned to that first operating sequence (BS1), at least on average.e) during the operation of the device (DEV), second operating sequences (BS2) of a second user (U2) are captured with time resolution, f) the time-resolved second operating sequences (BS2) are fed as input texts into the retrained large language model (LLM), which generates second response texts (A2) from them, and g) a user interface (IO) is caused, depending on the generated second response texts (A2), to decrease or increase a number of outputs (H) intended to support the second user (U2) in operating the device (DEV).

2. Method according to claim 1, characterized in that the second response texts (A2) are at least partially output to the user (U2) via the user interface (IO).

3. Method according to one of the preceding claims, characterized in that the second operating sequences (BS2) are modified depending on the second response texts (A2), and that the device (DEV) is controlled by means of the modified second operating sequences (BS2').

4. Method according to one of the preceding claims, characterized in that for the first operating sequences (BS1) - a duration of an operating action and / or a duration between operating actions of a respective first operating sequence (BS1) is measured, - Repetitions and / or retractions of operating actions of the respective first operating sequence (BS1) are counted, and / or - a performance and / or error rate of the device (DEV) controlled by the respective first operating sequence (BS1) is determined, and depending on this, a respective quality indication (Q) is generated and assigned to the respective first operating sequence (BS1).

5. Method according to one of the preceding claims, characterized in that, during retraining of the large language model (LLM), a degree of agreement between a derived first response text (A1) and the corresponding quality indicator (Q) is determined and quantified in the form of a respective agreement value (C), and that the large language model (LLM) is retrained to increase the agreement values ​​(C) at least on average.

6. Method according to claim 5, characterized in that, in order to determine the respective degree of agreement, a request (RAC) to output a statement (AC) about the respective degree of agreement is fed into the large language model (LLM) as input text, and that the respective agreement value (C) is derived from the output statement (AC).

7. Method according to one of the preceding claims, characterized in that the large language model (LLM) with the first operating sequences (BS1) of a respective first user (U 1 ) is supplied with a user identifier (ID1) of this first user (U 1 ) as input text, and / or that the large language model (LLM) with the second operating sequences (BS2) of a respective second user (U2) is supplied with a user identifier (ID2) of this second user (U2) as input text.

8. Method according to one of the preceding claims, characterized in that the large language model (LLM) with a respective first and / or second operating sequence (BS1 , BS2) is also supplied with a currently recorded operating state of the device (DEV) in the form of an input text.

9. Method according to one of the preceding claims, characterized in that the quality specification (Q) assigned to a first operating sequence (BS1) of a first user (U1) comprises a specification about the operating expertise of this first user (U1).

10. Method according to one of the preceding claims, characterized in that the large language model (LLM) is retrained to generate from a first operating sequence (BS1) a statement about an equivalent operating sequence (BQ1) with higher operating quality, and that during the operation of the device (DEV) a statement about an equivalent operating sequence (BQ2) with higher operating quality generated from a second operating sequence (BS2) is output to the user (U2) via the user interface (IO).

11. Method according to one of the preceding claims, characterized in that a user input text (UT) of the second user (U2) is read through the user interface, and that the user input text (UT) is fed into the large language model (LLM) in order to train it by prompt-based learning during the operation of the device (DEV).

12. Method according to one of the preceding claims, characterized in that second operating sequences (BS2) of several second users (U2) are recorded, that a respective quality rating (Q) is derived from the second response texts (A2) of each second user (U2) and assigned to the respective second user (U2), that current error rates (ER) of the device (DEV) are recorded user-specifically, that a statistical correlation (CC) of the recorded error rates (ER) with the corresponding quality ratings (Q) is determined, that the determined correlation (CC) is compared with a predetermined threshold (TH), and that if the predetermined threshold (TH) is undershot, the device (DEV), the user interface (IO) or a software version of the device (DEV) is marked as error-prone and / or a function of the device (DEV) is restricted.

13. Assistance system (A) for user support in operating a device (DEV), comprising means for performing the steps of a method according to one of the preceding claims.

14. Computer program product comprising instructions which, when the program is executed by a computer, cause an assistance system (A) according to claim 13 to execute a method according to any one of claims 1 to 12.

15. Computer-readable storage medium with a stored computer program product according to claim 14.

Images