Systems and methods for progressing a user in a clinical treatment programme

Abstract

Computer method of training a machine learning model (MLM) to predict whether interaction between a large language model (LLM) and a user will progress the user in a CTP. Training includes fitting the

Description

Field The present disclosure relates to large language models, and in particular, to the use of large language models in clinical treatment programmes. More specifically, the present disclosure relates to systems and methods that enable a user to engage with a clinical treatment programme via a large language model. Background Conversational systems, such as for example chatbots or voice-based agents, are widely used in various applications. More recently, conversational systems have begun to leverage capabilities of artificial intelligence models such as large language models to generate text and dialogue that are human-like. By leveraging these capabilities, conversational systems that deploy artificial intelligence models can advantageously be used in healthcare applications, such as to augment and / or provide clinical care. In particular, such conversational systems can provide easy access to clinical treatment programmes. A patient can engage with a clinical treatment programme at any time via these conversational systems, thereby reducing the barrier to accessing clinical therapy and / or clinical treatment. However, a major challenge associated with deploying artificial intelligence models in conversational systems that are used for clinical care is that large language models are prone to hallucinations. Put differently, large language models are prone to sometimes generate content that are irrelevant, nonsensical, inaccurate, or made-up. Furthermore, even if large language models generate content that are correct, it may be possible that the generated content is unsafe in a clinical setting (e.g., content that is detrimental to a user’s physical or mental health). This can affect the reliability of conversational systems that use large language models. Unreliable conversational systems in healthcare applications (e.g., to augment and / or provide clinical care) can have adverse effects on the health and safety of the patients. Providing inaccurate / unreliable content to patients can also inhibit the patients from initiating interactions with the conversational systems. To overcome these challenges, some existing approaches use data that is annotated and / or rated by humans to fine-tune large language models that are used in conversational systems for healthcare applications. For example, these existing approaches use human annotators to annotate outputs from conversational systems based on their preferences. More specifically, conversational systems that enable a user (e.g., a patient) to engage with a clinical treatment programme are configured to generate one or more outputs, for example in response to an input from the user. These outputs are annotated by human annotators based on their individual preferences. This annotated data is then used to fine-tune large language models which are then deployed in conversational systems for use in healthcare applications. There are several drawbacks that are associated with such existing approaches. Firstly, given that these approaches rely on annotation by human annotators, such approaches are labour-intensive. Secondly, such approaches rely firmly on the quality and fidelity of human preference data. For instance, different annotators may have different preferences to a specific output, thereby annotating the same output in a different manner. This can cause the annotated data to have low reliability. Therefore, the annotated data that is then used to finetune the large language models can often be noisy. Thirdly, simply using a human annotator’s overall preference to an output can be a poor target to improve the reliability of large language models for healthcare applications. This is because when these large language models are used in conversational systems for healthcare applications, in addition to providing a reliable output, the output from these conversational systems may need to possess certain desirable features, such as for example, the outputs may need to display warmth, empathy, etc. so as to keep a user engaged with the clinical treatment programme provided by the conversational system. Merely annotating overall preference for an output ignores such desirable features that the output should possess. Fourthly, even if human annotators were to annotate for various different criterions (e.g., one or more desirable features of the output, overall preference for the output, a combination thereof, and / or the like), fine-tuning the large language model based on multiple different annotations for different criterions can become computationally expensive. Fifthly, such approaches ignore temporal dependencies, for instance when an output may seem initially unappealing to a user, but can lead to improvement of a clinical state of the user in the future. Accordingly, there is a need for a reliable conversational system that overcomes these challenges so as to provide easy access to clinical care in a safe manner. Summary of Invention Aspects described herein provide systems and methods that provide a reliable conversational system that overcomes the challenges discussed above so as to provide easy access to clinical care in a safe manner. In particular, aspects described herein provide systems and methods that enable a user to engage with a clinical treatment programme via a large language model in a safe and reliable manner. “Conversational systems” are referred to herein as “dialogue systems”. The dialogue systems described herein use large language models to produce outputs that are provided to the users of the dialogue systems. The dialogue systems described herein can be configured to provide clinical care to users (e.g., patients). For example, these dialogue systems may provide a “clinical treatment programme” via large language models. “Clinical treatment programmes” as described herein are programmes (e.g., therapy programmes, therapeutic programmes, counselling programmes, diagnostic programmes, and / or the like) that are configured to transition a user from a less desirable clinical state to a more desirable clinical state. “Clinical treatment programmes” may be provided via one or more “interactions” between a user of a dialogue system and the dialogue system. For example, a clinical treatment programme may be provided via one or more outputs from a large language model that is included in a dialogue system. An “interaction” may be defined by a period of time, a set of predefined inputs-and-outputs ( / .e., inputs from the user and outputs from the dialogue system), or in some other appropriate way depending on the clinical treatment programme. Systems and methods described herein enable a large language model of a dialogue system to produce an output that is configured to “progress a user in a clinical treatment programme”. According to aspects described herein, “progress of a user in a clinical treatment programme” is a measurable metric. This measurable metric is also referred to herein as a “clinical benchmark”. As an example, the measurable metric may include one or more metrics that are representative of one or more of: • adherence to homework by the user that is provided in the clinical treatment programme, • whether the user has attended a clinical session, • whether the user is engaging with the clinical treatment programme, • the number of inputs from the user on the clinical treatment programme, • likelihood of adverse events experienced by the user, • reduction of clinical symptoms in the user, • recovery of the user (i.e., transition from unrecovered clinical state to fully recovered clinical state), etc. A metric and / or a clinical benchmark meeting a predetermined threshold may be representative of the “progress of a user in a clinical treatment programme”. Put differently, achieving a clinical benchmark ( / .e., the clinical benchmark meeting the predetermined threshold) can be representative of a user progressing in a clinical treatment programme. That is, achieving a clinical benchmark ( / .e., the clinical benchmark meeting the predetermined threshold) can be representative of whether one or more of the above metrics meets a predetermined threshold ( / .e., whether one or more of the above metrics is improved). When one or more of the clinical benchmark is below a predetermined threshold, it may be indicative of a user being in a “less desirable clinical state”. And, when one of more of the clinical benchmark meets or is above a predetermined threshold, it may be indicative of the user being in a “more desirable clinical state”. Accordingly, systems and methods described herein enable a large language model of a dialogue system to produce an output that is configured to achieve a clinical benchmark. The clinical benchmark may be achieved at a current time or at a future time so as to progress a user in a clinical treatment programme. For example, the clinical benchmark may be achieved during a current interaction between the dialogue system and a user. Alternatively, the clinical benchmark may be achieved during a future interaction between the dialogue system and a user. The clinical treatment programme may define a number of interactions that may be needed between a user of the dialogue system and the dialogue system to achieve the clinical benchmark in order to progress the user in a clinical treatment programme. In “progressing a user in a clinical treatment programme”, the output from the large language model improves a clinical state of a user, for example the output from the large language model may transition the user from a less desirable clinical state to a more desirable clinical state. Furthermore, such an output may meet one or more “desirable criterions”. “Desirable criterions” are features that humans would prefer in an output that would cause them to continue engaging with the dialogue system, for example by initiating an interaction with the dialogue system. Accordingly, producing an output that meet one or more “desirable criterions” may cause a user to continue engaging with a clinical treatment programme provided via a large language model of a dialogue system in a safe manner. Some example desirable criterions in an output include: • whether an output displays warmth, • whether the output displays cognitive empathy, • whether the user will continue to engage with the clinical treatment programme, • whether socratic questioning is being performed well, • whether the output is performing well in normalising a challenging situation, • whether the output has harmful or offensive language, • whether the output includes medical advice, etc. Therefore, systems and methods described herein enable a large language model of a dialogue system to produce an output that is configured to progress a user in a clinical treatment programme, for example by achieving a clinical benchmark. Such outputs may be configured to improve a clinical state of a user while simultaneously meeting one or more desirable criterions, such as for example criterions discussed above. The systems and methods described herein improve accuracy and / or reliability of outputs produced by the large language model, thereby providing a reliable dialogue system to users. According to a first aspect, there is provided a computer-implemented method. The computer-implemented method comprises training a machine learning model to predict whether an interaction between a large language model and a user would progress the user in a clinical treatment programme from a first point in the clinical treatment programme to a more advanced point in the clinical treatment programme. The interaction includes at least one output from the large language model. The training comprises fitting the machine learning model using a first plurality of interactions in a first dataset. The first dataset comprises the first plurality of interactions between the large language model and a plurality of users of the large language model. Each interaction of the first plurality of interactions includes at least one output from the large language model to a corresponding user of the plurality of users. The computer-implemented method further comprises validating the machine learning model using a second plurality of interactions between the large language model and a plurality of other users of the large language model to generate a second dataset. The validating comprises for each interaction of the second plurality of interactions: selecting, using the trained machine learning model, a candidate output from a plurality of candidate outputs that is most likely to achieve a clinical benchmark, and generating the second dataset based on the selected candidate output. The computer-implemented method further comprises fine-tuning the large language model using the second dataset to generate a fine-tuned large language model. After fine-tuning the large language model, the large language model is configured to provide, during an interaction with a first user, an output that is configured to progress the first user in the clinical treatment programme. One or more features disclosed above or a combination of features disclosed above mitigate the drawbacks (discussed in the background section) of large language models. For example, training a machine learning model to predict whether an output from a large language model would progress a user in a clinical treatment programme, and using a dataset generated from the validated machine learning model to fine-tune the large language model can improve the accuracy and / or reliability of outputs that are provided by the fine-tuned large language model. In particular, such outputs can be configured to progress a user in a clinical treatment programme such that the user transitions from a less desirable clinical state to a more desirable clinical state. When large language models are used in healthcare applications, such as for example in clinical settings, in addition to providing outputs that improve a clinical state of the user, it is desirable that these outputs meet one or more desirable criterions, such as for example the desirable criterions discussed above. Meeting these desirable criterions can allow a user to engage with the clinical treatment programme in a safe manner. Such engagement can facilitate the transition of the user from a less desirable clinical state to a more desirable clinical state. For example, an output that improves the clinical state of the user, but does not display warmth and empathy, or comprises harmful language, can cause a user to simply disengage from a clinical treatment programme, thereby leading to adverse effects on the clinical state of the user. Therefore, training the large language models to provide outputs that merely improve a clinical state may cause the large language models to provide inaccurate and / or unreliable outputs. This is because such outputs may not take into consideration the criterions that are desirable so as to continue to engage a user in the clinical treatment programme in order to progress the user in the clinical treatment programme. The approach described herein overcomes this challenge by training a machine learning model to predict whether an output from a large language model progresses a user in a clinical treatment programme. As noted above, outputs that progress the user in the clinical treatment programme may be configured to improve a clinical state of the user while simultaneously meeting one or more desirable criterions, such as for example desirable criterions discussed above. Furthermore, simply training a machine learning model to predict whether each of the desirable criterions are met can be computationally expensive. For instance, such a prediction may require solving a computationally expensive function, such as for example, a multidimensional optimization function. In contrast, the approach described herein is not computationally expensive. This is because the approach described herein trains a machine learning model to predict whether a user progresses in a clinical treatment programme, which as noted above is a measurable metric. Additionally, merely predicting whether a specific desirable criterion for an output is met during a current interaction may not account for whether the output would improve a clinical state of a user. For example, it may be possible that an output that does not display warmth at a current time, and is therefore unappealing to the user may improve the clinical state of the user at a future time. Whereas, an output that displays warmth at a current time may not improve the clinical state of the user at a future time. Put differently, an output that displays a desirable criterion at a current time may still be detrimental to the overall progress of the user in the clinical treatment programme. The approach described herein overcomes such situations by training a machine learning model to predict whether an output would progress a user in a clinical treatment programme. Put differently, the approach described herein trains a machine learning model to predict whether an output from a large language model would achieve a clinical benchmark. As noted above, achieving a clinical benchmark is indicative of whether one or more metrics, such as for example metrics discussed above, meet a predetermined threshold. This predetermined threshold can be met at a current time or a future time. Put differently, the machine learning model that is used to fine-tune the large language model predicts whether an output would meet a predetermined threshold at either a current time or a future time. Therefore, the approach described herein is a holistic approach that is not focussed on simply enabling a large language model to produce an output that meets a desirable criterion at a current time but rather on enabling the large language model to produce an output that would progress the user in the clinical treatment programme. The large language models are fine-tuned using a dataset that is generated using the validated machine learning model. This eliminates the need for manual intervention, such as, laborious generation of datasets by human annotators. Eliminating manual intervention can reduce noisy data in the datasets. In particular, there can be a significant variation in how human annotators may perceive a specific output. This variation in perception can cause the data generated by human annotators to be noisy and unreliable. Such a challenge can be overcome using the approach described herein. It should be readily understood that the output from the large language model may be an utterance, a video, an image, text, or any other resource. In some variations, selecting a candidate output can comprise selecting a candidate output that is most likely to achieve the clinical benchmark at a current time. For example, “at a current time” may mean during a single interaction. As noted above, an interaction may be defined by a period of time, a set of predefined questions-and-answers ( / .e., inputs from the users and outputs from the large language model), or in some other appropriate way depending on the clinical treatment programme. After the fine-tuning, the large language model may be configured such, during the interaction with the first user, that the large language model provides an output that is configured to progress the first user in the clinical treatment programme during that interaction. In some variations, selecting a candidate output can comprise selecting a candidate output that is most likely to achieve the clinical benchmark at a future time after a further plurality of interactions between the large language model and a user. Similarly, after the fine-tuning, the large language model may be configured such, during the interaction with the first user, that the large language model provides an output that is configured to progress the first user in the clinical treatment programme at a future time after a further plurality of interactions between the large language model and the first user. Validating the machine learning model may further comprise for each interaction of the second plurality of interactions, for a user input in that interaction, receiving a first candidate output from the large language model, and receiving a second candidate output from the large language model. The first candidate output is outputted from the large language model in response to the large language model receiving the user input a first time. The second candidate output is outputted from the large language model in response to the large language model receiving the user input a second time. Validating the machine learning model may further comprise for each interaction of the second plurality of interactions, selecting the first candidate output or the second candidate output as the candidate output. The plurality of candidate outputs include the first candidate output and the second candidate output. Generally, outputs from a large language model may not be deterministic in nature. Put differently, for a given interaction, the large language model may produce a first output a first time and a second output that is different from the first output a second time. That is, for a same given interaction, the non-deterministic nature of the large language model may cause the large language model to produce different outputs when the large language model is configured to produce outputs multiple times for that interaction. Validating the trained machine learning model by receiving multiple outputs for a given interaction, and selecting a best candidate output for that given interaction, and using this best candidate output to generate a dataset that is used to fine-tune the large language model can improve the accuracy and / or reliability of the outputs that are outputted by the fine-tuned large language model. In some variations, validating the trained machine learning model may comprise resampling the first candidate output and the second candidate output to select the best candidate output. In this first aspect, fine-tuning the large language model comprises training the large language model using supervised learning based on the dataset that is generated by the validated machine learning model. In comparison to other fine-tuning approaches (e.g., reinforcement learning), supervised learning can be computationally efficient. For example, supervised learning does not require large computational resources and can occur faster than other fine-tuning approaches (e.g., reinforcement learning). In particular, by generating a dataset using the validated machine learning model discussed herein and fine-tuning the large language model using supervised learning based on the generated dataset, can reduce latency for fine-tuning the large language model and can reduce the computational power that may be required to fine-tune the large language model. Typically, the large language model comprises weighted parameters that enable the large language model to provide outputs. For instance, weighted parameters of the large language model determine how inputs to the large language model are transformed so as to provide outputs. In this first aspect, fine-tuning the large language model may comprise modifying one or more weighted parameters of the large language model. For example, during the fine-tuning step, one or more weighted parameters of the large language model may be adjusted to optimise the performance of the large language model. By modifying the weighted parameters of the large language model during the fine-tuning, the large language model can be enabled to provide outputs with improved accuracy and / or reliability. In some variations, the machine learning model may comprise a binary classification random forest model. Binary classification random forest model can be more efficient for large datasets. Furthermore, such models may have high accuracy and may be more robust to noise in datasets. In other variations, the machine learning model may comprise a regression model. In some variations, training the machine learning model to predict whether the interaction between the large language model and the user would progress the user in a clinical treatment program comprises training the machine learning model to predict whether the interaction between the large language model and the first user would cause the first user to initiate a further plurality of interactions with the large language model. As noted above, the clinical benchmark meeting a predetermined threshold may be representative of whether the first user has progressed in the clinical treatment programme. For example, the clinical benchmark may be representative of whether one or more of: • adherence of the first user to the clinical treatment programme, • engagement of the first user to the clinical treatment programme, • interactions initiated by the first user with the large language model, • likelihood of adverse clinical events faced by the first user, • reduction of clinical symptoms of the first user, or • clinical recovery of the first user. According to a second aspect, there is provided a computer-implemented method. The computer-implemented method comprises training a machine learning model to predict whether an interaction between a large language model and a user would progress the user in a clinical treatment programme from a first point in the clinical treatment programme to a more advanced point in the clinical treatment programme. The interaction includes at least one output from the large language model. The training comprises fitting the machine learning model using a first plurality of interactions in a first dataset. The first dataset comprises the first plurality of interactions between the large language model and a plurality of users of the large language model. Each interaction of the first plurality of interaction includes at least one output from the large language model to a corresponding user of the plurality of users. The computer-implemented method further comprises fine-tuning the large language model using the trained machine learning model to generate a fine-tuned large language model. After fine-tuning the large language model, the large language model is configured to provide, during interaction with a first user, an output that is configured to progress the first user in the clinical treatment programme. As with the first aspect, training a machine learning model to predict whether an output from a large language model would progress a user in a clinical treatment programme, and fine-tuning the large language model using the trained machine learning model can improve the accuracy and / or reliability of outputs that are provided by the fine-tuned large language model. In particular, such outputs can be configured to progress a user in a clinical treatment programme such that the user transitions from a less desirable clinical state to a more desirable clinical state. As noted above, training large language models to provide outputs that merely improve a clinical state may cause the large language models to provide inaccurate and / or unreliable outputs because such outputs may not take into consideration the criterions that are desirable so as to continue to engage a user in the clinical treatment programme. One or more features or a combination of features above overcomes this challenge by training a machine learning model to predict whether an output from a large language model progresses a user in a clinical treatment programme. Furthermore, given that the progress of a user in a clinical treatment programme is a measurable metric, the prediction described herein does not require solving computationally expensive multidimensional optimization function to predict whether one or more desirable criterions are met. Additionally and as noted above, the approach described herein is a holistic approach that is not focussed on simply enabling a large language model to produce an output that meets a desirable criterion at a current time but rather on enabling the large language model to produce an output that would progress the user in the clinical treatment programme, thereby achieving a clinical benchmark at a current time or at a future time. The approach described herein also eliminates laborious generation of datasets by human annotators. It should be readily understood that the output from the large language model may be an utterance, a video, an image, text, or any other resource. As an example, in this second aspect, fine-tuning the large language model can comprise training the large language model using reinforcement learning. Unlike the first aspect, fine-tuning the large language model does not require generating a dataset. This is because reinforcement learning typically does not need a labelled dataset to fine-tune / train the large language model. Instead, reinforcement learning may comprise maximizing a reward function to fine-tune the large language model. For instance, the reward function may be the machine learning model. Fine-tuning using reinforcement learning may take longer than supervised learning and might need more computational resources than supervised learning. However, after fine-tuning using reinforcement learning, the output produced by the large language model may be even more accurate and / or even more reliable than outputs that are produced after fine-tuning the large language model using supervised learning. In some variations of the second aspect, fine-tuning the large language model may comprise modifying one or more weighted parameters of the large language model. For example, during the fine-tuning step, one or more weighted parameters of the large language model may be adjusted to optimise the performance of the large language model. By modifying the weighted parameters of the large language model during the fine-tuning, the large language model can be enabled to provide outputs with improved accuracy and / or reliability. As another example, in this second aspect, fine-tuning the large language model can comprise: during the interaction between the large language model and the first user, and in response to being provided with an input from the first user: receiving, from the large language model, a first output a first time; receiving, from the large language model, a second output a second time; and selecting, using the trained machine learning model, the first output or the second output as the output to be outputted by the large language model, the output being configured to progress the first user in the clinical treatment programme. Producing multiple outputs for a given interaction and selecting a best output can compensate for the non-deterministic nature of the large language models. In some variations, training the machine learning model to predict whether the interaction between the large language model and the user would progress the user in a clinical treatment program comprises training the machine learning model to predict whether the interaction between the large language model and the first user would cause the first user to initiate a further plurality of interactions with the large language model. As noted above, the clinical benchmark meeting a predetermined threshold may be representative of whether the first user has progressed in the clinical treatment programme. For example, the clinical benchmark may be representative of whether one or more of: • adherence of the first user to the clinical treatment programme, • engagement of the first user to the clinical treatment programme, • interactions initiated by the first user with the large language model, • likelihood of adverse clinical events faced by the first user, • reduction of clinical symptoms of the first user, or • clinical recovery of the first user. According to a third aspect, there is provided a computer-implemented method. The computer-implemented method comprises training a machine learning model to adaptively allocate a variant of an input prompt from a plurality of variants of the input prompt to a first large language model of a plurality of large language models. The first large language model interacts with a first user of a plurality of users. The input prompt is configured to guide each of the plurality of large language models to produce a corresponding output. Training the machine learning model comprises associating each variant of the plurality of variants of the input prompt to one or more other users of the plurality of users. Each other user interacts with a respective one of the plurality of large language models. Training the machine learning model also comprises assigning the associated variant to the respective one of the plurality of large language models interacting with the corresponding one or more other users, and determining a score for each associated variant of the plurality of variants based on feedback from the corresponding one or more other users. The score is indicative of a likelihood that a respective output produced by the respective one of the plurality of large language models responsive to that variant achieves a clinical benchmark. The computer-implemented method further comprises adaptively allocating, using the trained machine learning model, the variant of the input prompt to the first user. The allocated variant causes the first large language model to produce an output. The output is configured to progress the first user in a clinical treatment programme from a first point in the clinical treatment programme to a more advanced point in the clinical treatment programme. As noted above, large language models typically comprise weighted parameters. These weighted parameters influence the outputs that are produced by the large language models. Generally, weighted parameters are set when a large language model is being trained. These weighted parameters can be modified by fine-tuning the large language model. Therefore, one possible way to improve accuracy and / or reliability of outputs that are produced by a large language model is to modify the weighted parameters (e.g., by fine-tuning the large language model) of the large language model. To do so, typically, the large language model may have to be transitioned to an “offline” state (e.g., a state in which the large language model is not in use and a state in which a user cannot interact with the large language model). This can be challenging when the large language model is already deployed in a dialogue system and the dialogue system is actively in use. In addition to these weighted parameters, input prompts that guide large language models to produce outputs can also have an influence on the outputs that are produced by a large language model. The approach described herein leverages the capabilities of input prompts to improve accuracy and / or reliability of outputs produced by a large language model. Accordingly, the large language model need not be transitioned to an offline state. More specifically, the approach described herein can train a machine learning model to adaptively allocate a variant of an input prompt to a large language model that is interacting with a user such that this variant causes the large language model to produce an output that is configured to progress the user in a clinical treatment programme. In some variations, determining a score may comprise determining a score indicative of a likelihood that the respective output produced by the respective one of the plurality of large language model responsive to that variant achieves the clinical benchmark at a current time. For example, “at a current time” may mean during a single interaction. As noted above, an interaction may be defined by a period of time, a set of predefined questions-and-answers, or in some other appropriate way depending on the clinical treatment programme. After adaptively allocating the variant to the first user, the first large language model may be configured such, during the interaction with the first user, that the first large language model provides an output that is configured to progress the first user in the clinical treatment programme during that interaction. In some variations, determining a score comprises determining a score indicative of a likelihood that the respective output produced by the respective one of the plurality of large language model responsive to that variant achieves the clinical benchmark at a future time. Similarly, after adaptively allocating the variant to the first user, the first large language model may be configured such, during the interaction with the first user, that the first large language model provides an output that is configured to progress the first user in the clinical treatment programme at a future time after a further plurality of interactions between the first large language model and the first user. Associating each variant may comprise randomly allocating each variant to one or more other users of the plurality of users. Adaptively allocating the variant of the input prompt to the first user may comprise minimizing an expected cumulative regret associated with allocating that variant. Put differently, the claimed approach can adaptively allocate, based on balancing an explorationexploitation tradeoff, the variant of the input prompt to the first user. Said another way, the claimed approach can intelligently balance the need for continuing to determine scores for variants with uncertain values while at the same time choosing a variant with the best currently estimated score so as to allocate the best variant to the users. In some variations, adaptively allocating the variant may comprise responsive to determining that the score associated with the variant exceeds a first predetermined threshold, selecting the variant from the plurality of variants. Furthermore, the clinical benchmark meeting a second predetermined threshold may be representative of whether the first user has progressed in the clinical treatment programme. As noted above, the clinical benchmark may be representative of whether one or more of: • adherence of the first user to the clinical treatment programme, • engagement of the first user to the clinical treatment programme, • interactions initiated by the first user with the large language model, • likelihood of adverse clinical events faced by the first user, • reduction of clinical symptoms of the first user, or • clinical recovery of the first user. According to a fourth aspect, there is provided a computer-implemented method. The computer-implemented method comprises fine-tuning, using a first machine learning model, a large language model, and adaptively allocating, using a second machine learning model, a variant of an input prompt from a plurality of variants of the input prompt to the large language model. The first machine learning model is configured to predict whether an output from the large language model would achieve a first clinical benchmark. The second machine learning model being configured to select a variant of the input prompt that is most likely to cause the large language model to produce an output that achieves a second clinical benchmark. After the fine-tuning and the allocating, and during an interaction with a user, the large language model is configured to provide an output that is configured to progress the user in a clinical treatment programme from a first point in the clinical treatment programme to a more advanced point in the clinical treatment programme. One or more of these features or a combination of these features can improve the accuracy and / or reliability of outputs that are provided by the large language model. In particular, fine-tuning the large language model as described herein (e.g., using the machine learning model described herein) and adaptively allocating a variant of an input prompt as described herein can align the large language model such that the large language model produces outputs that are configured to achieve one or more clinical benchmarks. The clinical benchmark may be achieved at a current time (e.g., during a current interaction with a user) or at a future time (e.g., after a further plurality of interactions with a user). Accordingly, the systems and methods described herein circumvents the need for labour-intensive gathering of data (e.g., gathering annotated data from human annotators on their preference to an output) while still leading to improved alignment of the large language model to human preferences (e.g., producing outputs that meet one or more desirable criterions such as discussed above) and improved therapeutic interactions (e.g., interactions that can transition the user from an undesirable clinical state to a desirable clinical state) with the large language model. As noted above, systems and methods described herein enable a large language model of a dialogue system to produce an output that is configured to achieve a clinical benchmark. Clinical benchmark may be representative of behaviour of a user, such as for example, whether a user would continue to engage with the clinical treatment programme, whether the user is experiencing a reduction in clinical symptoms, whether the user is adhering to the clinical treatment programme, etc. Enabling a large language model to produce an output that affects the behaviour of the user can be advantageous. For example, such outputs may exhibit qualities of a trained professional (e.g., a trained therapist). In particular and as discussed above, such outputs may exhibit one or more desirable criterions such as for example, warmth, empathy, etc. while simultaneously improving the clinical state of the users. Brief Description of Drawings Embodiments will now be described, by way of example only and with reference to the accompanying drawings having like-reference numerals, in which: Figure 1 is a schematic illustration of a system, according to an example; Figure 2a is a schematic illustration of a clinical treatment application, according to a first example; Figure 2b is a schematic illustration of a clinical treatment application, according to a second example; Figure 2c is a schematic illustration of a clinical treatment application, according to a third example; Figure 3a shows a flow chart of a fine-tuning method for a large language model, according to a first example; Figure 3b shows a flow chart of a fine-tuning method for a large language model, according to a second example; Figure 4 depicts an example performance of an example machine learning model that is trained as described in Figure 3a; Figure 5 depicts that outputs selected during a validating step as described in Figure 3a have an improved score for clinical benchmark, according to an example; Figure 6a and Figure 6b depict that outputs from a large language model that is finetuned as described herein have an improved score for clinical benchmark, according to an example; Figure 7a depicts annotated data from all human annotators depicting their preferences to outputs from a fine-tuned large language model vs. outputs from large language model that is not fine-tuned; Figure 7b depicts annotated data from clinically trained human annotators depicting their preferences to outputs from a fine-tuned large language model vs. outputs from large language model that is not fine-tuned; Figure 8 shows a flow chart of an input prompt allocation method according to an example; Figure 9 depicts the score of two hypothetical variants of an input prompt, according to an example; Figure 10 depicts simulations of the method in Figure 8 on two variants of an input prompt, according to an example; Figure 11 is a flow diagram depicting the implementation of both the input prompt allocation method and the fine-tuning method, according to an example. Description Referring to Figures 1 to 11, the details of one or more aspects of systems and methods that enable a dialogue system to provide a clinical treatment programme to a user in a safe, reliable, and engaging manner are described in further detail below. Figure 1 is a schematic illustration of a system 100 that enables a dialogue system to provide a clinical treatment programme to a user in a safe, reliable, and engaging manner. As discussed above, dialogue systems as used herein, are conversational systems that are configured to provide a clinical treatment programme to a user via a large language model. Put differently, the dialogue systems described herein use a large language model. Typically, large language models are pre-trained on diverse datasets such as for example, text from the Internet, so as to perform a wide range of language tasks (e.g., generate human-like text, answer questions, compose emails, summarize passages, create content in various styles and formats, etc.). Non-limiting examples of a large language models include generative pretrained transformer (GPT) models (e.g., ChatGPT that was developed by OpenAI™). The dialogue systems described herein may be configured to facilitate a large language model to provide a clinical treatment programme to a user by way of producing outputs that are provided to the user, by initiating interactions with the user, by responding to interactions initiated by the user, a combination thereof, and / or the like. Accordingly, throughout this document, “an interaction between a large language model and a user” refers to an interaction between a dialogue system that uses the large language model and the user. Referring back to Figure 1, the system 100 can be configured to enable a large language model used by the dialogue system to produce an output that is configured to progress a user in a clinical treatment programme. The system 100 comprises a working memory 102, a processor 104, and a storage 106. The processor 104 is coupled to the storage 106 and accesses the working memory 102. The processor 104 may comprise logic circuitry that responds to and processes the instructions in code stored in the working memory 102. For instance, the processor 104 may be any suitable processing device that is configured to run and / or execute a set of instructions or code, and may include one or more data processors, image processors, graphics processing units, digital signal processors, and / or central processing units. The processor 104 may be, for example, a general purpose processor, a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), and / or the like. The underlying device technologies may be provided in a variety of component types (e.g., MOSFET technologies like complementary metal-oxide semiconductor (CMOS), bipolar technologies like emitter-coupled logic (ECL), polymer technologies (e.g., Silicon-conjugated polymer and metal-conjugated polymer-metal structures), mixed analog and digital, and / or the like. In this example, the processor 104 is configured to execute a clinical treatment application 108 that is represented as a software product and stored in the working memory 102. The clinical treatment application 108 will be described in more detail in relation to the figures below. Execution of the clinical treatment application 108 by the processor 104 will cause examples as described herein to be implemented. Figure 1 illustrates a single processor 104 solely for illustrative purposes. It should be readily understood that the clinical treatment application 108 may be executed across multiple processing components, which may be located remotely, for example, using cloud based processing. The processor 104 is also configured to communicate with the non-volatile storage 106. The storage 106 may contain data that is used by the clinical treatment application 108 when executed by the processor 104. The storage 106 may be a local memory that is contained in the same device as the processor 104. Alternatively however, the storage 106 may be wholly or partly located remotely, for example, using cloud based memory that can be accessed remotely by the processor 104 via a communication network. The clinical treatment application 108 may be stored in the storage 106. The clinical treatment application 108 may be placed in the working memory 102 when executed. The clinical treatment application 108 can be embedded in original equipment, or can be provided, as a whole or in part, after manufacture. For instance, the clinical treatment application 108 can be introduced, as a whole, as a computer program product, which may be in the form of a download, or can be introduced via a computer program storage medium, such as an optical disk. Alternatively, modifications to existing software can be made by an update, or plug-in, to provide features of the above described example. Figure 2a is a schematic illustration of a clinical treatment application 208a, according to a first example, that may be stored and executed by a system 100 as described in relation to Figure 1. The clinical treatment application 208a can be structurally and / or functionally identical to the clinical treatment application 108 in Figure 1. Execution of the clinical treatment application 108 will cause methods as described herein to be implemented. In this example, the clinical treatment application 208a comprises a fine-tuning module 212. The fine-tuning module 212 includes instructions and / or software code to implement one or more fine-tuning methods for a large language model as described herein. Figure 3a shows a flow chart of a fine-tuning method 322a for a large language model according to a first example, which may be performed by the clinical treatment application 108 when executed on the system 100 as discussed above. Figure 3b shows a flow chart of a fine-tuning method 322b for a large language model according to a second example, which may be performed by the clinical treatment application 108 when executed on the system 100 as discussed above. Referring to both Figure 3a and Figure 3b, at step 324a and at step 324b, the method 322a or method 322b comprises training a machine learning model to predict whether an interaction between a large language model and a user would progress the user in a clinical treatment programme. In particular, training the machine learning model can comprise training the machine learning model to predict whether an output from the large language model would achieve a clinical benchmark. The clinical benchmark can be one or more measurable metrics, such as for example, the metrics discussed above. The machine learning model may be trained to predict whether an output from the large language model would meet a predetermined threshold set for a clinical benchmark in the clinical treatment programme. As an illustrative example, the clinical benchmark may be engagement of a user with a clinical treatment programme that is provided by a dialogue system. The predetermined threshold may be return of the user to the clinical treatment programme on a subsequent day. Put differently, the machine learning model may be trained to predict whether an output from the large language model would engage a user with a clinical treatment programme by predicting whether the output from the large language model would cause the user to initiate an interaction with the dialogue system on a subsequent day. As another illustrative example, the clinical benchmark may be reduction of a clinical symptom in a user. The predetermined threshold may be a predetermined reduction and / or a predetermined increase in one or more physiological parameters (e.g., heart rate, blood pressure, body temperature, oxygen saturation, respiratory rate, muscle strength, metabolic rate, hormonal levels, a combination thereof, and / or the like). Put differently, the machine learning model may be trained to predict whether an output from the large language model would reduce clinical symptoms in a user by predicting whether an output from the large language model would cause the user to experience an increase and / or a decrease in one or more physiological parameters of the user. Whether or not the user experiences an increase and / or a decrease in one or more physiological parameters can be determined using physiological sensors such as for example, wearable devices, blood pressure cuff, glucometer, pulse oximeter, implantable medical devices, thermometer, a combination thereof, and / or the like. Referring to both Figure 3a and Figure 3b, training the machine learning model may comprise fitting the machine learning model using a dataset comprising one or more interactions between large language models and one or more users. Each interaction in the dataset includes at least one output that has been produced by a large language model during that interaction with a user. Such an output may include a medical intervention, such as for example, a cognitive behavioural therapy intervention. In this example, one portion of the dataset may be used to fit the machine learning model while the other portion of the dataset may be used to test the fitting. For instance, 80 percent of the dataset may be used to fit the machine learning model while 20 percent of dataset may be used to test the fitting. Referring to both Figure 3a and Figure 3b, each interaction in the portion of the dataset that is used to fit the machine learning model is transformed into a vector representation. In particular, each interaction is embedded into a vector representation. The vector embedding can translate semantic similarity in the interactions to proximity in a vector space. As an example, OpenAI™ text-embedding-ada-002 embedding may be used to transform an interaction in the dataset into a 1536 dimensional vector. Accordingly, the input that is used for training the machine learning model are interactions that are in the dataset. Although, in this example, interactions are used as an input for training the machine learning model, it should be readily understood that further inputs may be added for the purpose of training the machine learning model. Some non-limiting examples of further inputs include demographic information of the one or more users, clinical information of the one or more users (e.g., previous clinical diagnosis, previous treatment pathways, etc.), information relating to the interactions (e.g., length of the interactions, time of the interactions, location of the interactions, etc.), information from external sources (e.g., clinical information from clinicians or other resources), a combination thereof, and / or the like. Additionally or alternatively, further inputs may include inputs from other machine learning models that may be applied to the large language models, summary of previous interactions generated by the large language models, a representation of a user’s clinical problem and clinical progress in higher-dimensional space that takes into account history of interactions with the user, etc. Referring to Figure 3a, an example machine learning model in Figure 3a, that predicts whether an interaction between a large language model and a user would progress a user in a clinical treatment programme is a binary classification model. For instance, the machine learning model may be a binary classification random forest model, such as for example, XGBoost (Extreme Gradient Boosting), which is a granted-boosted decision tree machine learning model. This machine learning model may comprise multiple decision trees. For instance, the machine learning model may be trained by iteratively training an ensemble of decision trees. The decision trees are built in parallel. The error residual from each iteration is used to fit the machine learning model in the next iteration. The final prediction is the weighted sum of each of the decision tree predictions. An example objective function for the binary classification model may be a binary cross entropy loss function, such as equation 1 - 1 N LogisticLoss = - — >[y; log log (fa) +(1- yj log log (1 - y;)] TV Z—I i = l where N is the number of samples (e.g., number of interactions) that are used to train the machine learning model, y; is the class label (e.g., class 0 or class 1 for binary classification) for each sample, and fa is the predicted probability that the sample belongs to a positive class (e.g., class 1 for binary classification). As an illustrative example, if the machine learning model is trained to predict whether an output from the large language model would engage a user with a clinical treatment programme, then yt = 1 would represent outputs that would cause the user to initiate an interaction with the large language model on a subsequent day and y; = 0 would represent outputs that would not cause the user to initiate an interaction with the large language model on a subsequent day , and fa is the predicted probability that an interaction in the training dataset belongs to the class y; = 1. Referring to Figure 3a, another example machine learning model in Figure 3a, that predicts whether an interaction between a large language model and a user would progress a user in a clinical treatment programme is a regression model and the objective function that is used to train the model is mean-square error (MSE), such as equation 2 - N MSE=±y (yi-yd2 TV Z—I i = l where N is the number of samples (e.g., number of interactions) that are used to train the machine learning model, yt is the observed value of the clinical benchmark that is being predicted and y( is the predicted value of the clinical benchmark that is being predicted. Referring to Figure 3b, an example machine learning model in Figure 3b, that predicts whether an interaction between a large language model and a user would progress a user in a clinical treatment programme is a reinforcement learning model and the objective function is a reinforcement-learning derived objective function, such as for example, policy gradient loss, such as equation 3 - T Policy Gradient Loss = —ET~ng logne(at |st)At] t=o where E is an expectation over the dataset, T is a trajectory (i.e., a set of actions taken by the large language model in a sequence), 9 is the parameters of the large language model, a t is the action taken by the large language model at time t (e.g., this may be a token in an output wherein the output is T or this may be an entire output if multiple outputs are generated), s t is the state that the large language model is in (e.g., this may be the history of all outputs, but may be supplemented with inputs from other clinical models), At is the advantage. At in effect is the reward signal (e.g., which would be given by the machine learning model). At can be determined in any suitable way. For example, At can be defined as Q(s, a)-y(s), where Q is the value of taking action a in state s, and V is the expected value of the state s. Effectively, this may determine whether this action was better than the actions that are done on average in the given state. Referring to both Figure 3a and Figure 3b, in some variations, training may comprise optimising regularisation parameters before fitting the machine learning model. As an example, referring to Figure 3a, the XGBoost machine learning model discussed in Figure 3a uses regularisation parameters. These parameters may be optimised using hyper-parameter optimisation before fitting the machine learning model. Examples of optimised parameters include Max_depth, N_estimators, Gamma, Alpha, Colsample_bylevel, Colsample_bytreefuup, Early_stopping_rounds, and eta As noted above, the training in Figure 3a and Figure 3b comprises using a dataset with one or more interactions between large language models and one or more users. In examples in which the machine learning model is trained to predict whether an output from a large language model would achieve more than one clinical benchmark, the dataset is split equally for each benchmark. Consider as an example that the machine learning model is trained to predict whether an output from a large language model achieves two clinical benchmarks, a first clinical benchmark being engagement of a user with a clinical treatment programme provided by the large language model, and a second benchmark being reduction of clinical symptoms in the user. In this example, the dataset is split equally. That is, 50 percent of the dataset is used to predict whether an output achieves the first benchmark, and the other 50 percent of the dataset is used to predict whether an output achieves the second benchmark. For instance, if the dataset comprises 26,000 interactions, then 13,000 interactions are used to train the machine learning model to predict whether an output from a large language model would achieve the first benchmark, and the other 13,000 interactions are used to train the machine learning model to predict whether an output from a large language model would achieve the second benchmark. Furthermore and as discussed above, in Figure 3a and Figure 3b, 20 percent of the dataset may be withheld from the training process so as to test the trained machine learning model. Put differently, the dataset is split such that 80 percent of the dataset is used for training the machine learning model, while the remaining 20 percent of the dataset is used for testing the trained machine learning model. Accordingly, in some variations, after training the machine learning model, the performance of the trained machine learning model is tested using the withheld dataset. As an illustrative example, Figure 4 depicts an example performance of a binary classification machine learning model that is trained as described in step 324a of Figure 3a. Figure 4 plots the false positive rate against the true positive rate of the trained machine learning model for possible threshold values. In this figure, the testing shows that the trained machine learning model has achieved an accuracy of 58.44% with an area under the curve (e.g., a metric of how well the trained machine learning model performs if its thresholds are changed) of 0.63. A value of 0.5 would indicate change performance ( / .e., continue training to achieve higher performance), whereas a value of 1 would indicate perfect performance no matter the threshold. Referring only to Figure 3a, after training, at step 326, the method 322a comprises validating the trained machine learning model. Validating the trained machine learning model improves the accuracy and / or reliability of the output produced from the large language model. This is because outputs from large language models may not be deterministic. Therefore, for a given interaction, the large language model may generate a first output a first time and a second output that is different from the first output a second time. Therefore, to improve the accuracy and / or reliability of the output produced from the large language model, the trained machine learning model can be validated by producing outputs for a given interaction multiple times and selecting the best output from these produced outputs. In particular, the large language model can be fine-tuned using a dataset that is generated based on the selected best outputs. This is discussed in further detail below with reference to step 328a in Figure 3a. Put differently, a validating dataset with one or more interactions between large language models and one or more users can be used to validate the trained machine learning model. For each interaction in the validating dataset, the large language model can be configured to produce outputs multiple times. Each of these outputs for the given interaction can be resampled. A best output that yields the highest score can be selected. In this context, “score” is the probability that an output would achieve a clinical benchmark (e.g., engage a user such that the user initiates an interaction on a subsequent day, reduce clinical symptoms of a user such that the user experiences a predetermined increase and / or a predetermined decrease in one or more physiological parameters, etc.). Therefore, “highest score” indicates that the output has the highest probability of achieving the clinical benchmark. A dataset for fine-tuning a large language model can be generated based on the best outputs that are selected during this step 326. As an example, the dataset for validation ( / .e., validating dataset) may comprise 1000 interactions. For each interaction of the 1000 interactions, the large language model is configured to produce 10 outputs. These 10 outputs are resampled to select the best output with the highest score. The best outputs with the highest scores may be used to generate a dataset for fine-tuning a large language model. Figure 5 shows that validating the trained machine learning model can improve the score of an output produced by a large language model. In this figure the clinical benchmark is engagement of the user with the clinical treatment programme. Each bar in this figure indicates the number of samples for a given level of improvement (in 0.001 increments). In this example, the score is improved by 0.038 after validating the trained machine learning model, thereby implying that there is a 3.8 percent higher chance of a user initiating an interaction with the large language model a subsequent day if the validation step 326 were to be performed. In one example, 91.4% of the interactions were improved by validating the trained machine learning model. Referring to Figure 3a, the method 322a comprises at step 328a, fine-tuning a large language model using the validated (as in step 326) machine learning model. In particular, the fine-tuning in Figure 3a may comprise fine-tuning a large language model using the dataset that is generated during validation of the machine learning model (as in step 326). As noted above, the best outputs that were selected in step 326 ( / .e., outputs that have the highest probability of achieving the clinical benchmark) are used to generate a fine-tuning dataset in step 326. The fine-tuning dataset would include the interactions that were used for validating the machine learning model (validating dataset used in step 326) along with the best output that was selected for a given interaction during the validating step (outputs selected in step 326). In this example, the large language model is fine-tuned using the fine-tuning dataset in a supervised learning setting. As an example, OpenAI™’s gpt-3.5-turbo model can be finetuned using the fine-tuning dataset in a supervised learning setting. Fine-tuning using supervised learning can be computationally efficient. In particular, in comparison to the other fine-tuning approaches, fine-tuning using supervised training and based on the fine-tuning dataset that comprises best outputs that were selected in step 326, can facilitate a reduction in computational resources that may be required to fine-tune the large language model and can facilitate a reduction in computational time that may be required to fine-tune the large language model. In this example, fine-tuning the large language model comprises modifying one or more weighted parameters of the large language model. For example, during the step 328a, the one or more weighted parameters of the large language model is adjusted to optimise the performance of the large language model, thereby improving the accuracy and / or reliability of outputs from the large language model. Referring to Figure 3b, the method 322b (unlike method 322a) comprises at step 328b, fine-tuning a large language model using the trained (as in step 324b) machine learning model. As an example, fine-tuning the large language model comprises using reinforcement learning (e.g., reinforcement learning from human feedback). For example, the objective function shown in equation 3 that is fitted and tested as in step 324b, can be used to fine-tune the large language model. In particular, fine-tuning can comprise optimizing the policy gradient loss (that is fitted and tested as in step 324b) such that after the fine-tuning, for every interaction, the large language model is configured to produce an output that maximizes the expected reward. Fine-tuning using reinforcement learning (unlike the supervised learning in step 328a) may not be computationally efficient. However, fine-tuning using reinforcement learning may enable the large language models to provide outputs with further accuracy and / or further reliability. In this example, fine-tuning the large language model comprises modifying one or more weighted parameters of the large language model. For example, during the step 328b, the one or more weighted parameters of the large language model is adjusted to optimise the performance of the large language model, thereby improving the accuracy and / or reliability of outputs from the large language model. Referring to Figure 3b, in some variations, the step 328b may comprise after the fine-tuning, when the large language model is in use, causing the large language model to produce outputs for a given interaction multiple times. These multiple outputs are scored and the output with the highest score is selected as an output to be provided to the user. In this context, “score” is the probability that an output would achieve a clinical benchmark (e.g., engage a user such that the user initiates an interaction on a subsequent day, reduce clinical symptoms of a user such that the user experiences an increase and / or a decrease in physiological parameters, etc.). Therefore, “highest score” indicates that the output has the highest probability of achieving the clinical benchmark. Put differently, to compensate for the non- deterministic nature of the large language model, multiple outputs are produced for a given interaction when the large language model is in use, to further improve the accuracy and / or reliability of the output produced by the large language model. While Figure 3a and Figure 3b show two example methods for fine-tuning a large language model, it should be readily understood that, in some variations, the fine-tuning can be performed using a combination of one or more steps described in relation to method 322a in Figure 3a and method 322b in Figure 3b. As an example, the machine learning model may be trained as described in Figure 322a or Figure 322b, validated as described in Figure 322a, and fine-tuned using a combination of supervised learning and reinforcement learning. Figure 6a and Figure 6b depict that outputs from a large language model that is finetuned as described herein (e.g., using method 322a and / or method 322b) have an improved score for a clinical benchmark. In these figures, the clinical benchmark is engagement of the user with the clinical treatment programme. Figure 6a shows a sample of 1000 interactions. The outputs for each of the 1000 interactions are generated using a fine-tuned GPT-4 model (fine-tuned using supervised learning as described herein). Each bar indicates the number of samples for a given level of improvement (in 0.001 increments) in the fine-tuned model compared to the original GPT-4 model. Positive values indicate that the fine-tuned model scored higher. Figure 6b shows a sample of 1000 interactions. The outputs for each of the 1000 interactions are generated using a fine-tuned GPT-3.5 model (fine-tuned using supervised learning as described herein). Each bar indicates the number of samples for a given level of improvement (in 0.001 increments) in the fine-tuned model compared to the base-GPT-3.5 model. Positive values indicate that the fine-tuned model scored higher. As can be seen in Figures 6a and 6b, on an average, fine-tuning can significantly improve the scores of the outputs. As discussed above, the fine-tuning of the large language model as described herein enables the large language model to produce outputs that are configured to achieve a clinical benchmark. Fine-tuning based on a machine learning model that predicts whether an output achieves a clinical benchmark (e.g., predicts a behaviour of a user) can yield safer and better aligned large language models. Disclosed herein is data confirming that such fine-tuning as described herein yields safer and better aligned large language model. To confirm this, 1000 interactions that were used to validate the trained machine learning model (e.g., interactions used in step 326) were annotated by human annotators. In this example, 5 human annotators were used of which 2 of the human annotators were clinically trained annotators. Each human annotators chose between 111 pairs of outputs, one from GPT-3.5-model (that was not fine-tuned as described herein), and the other from a fine-tuned GPT 3.5-model as described herein. 60 of these pairs of outputs were rated by clinically trained human annotators. The human annotators were blinded about which model ( / .e., fine-tuned or not fine-tuned) produced the output. The human annotators were asked to choose which output they preferred while also choosing which output scored better on the following desirable criterions- • warmth • cognitive empathy • how engaging the output was • how well the output performed socratic questioning • how well the output performed with respect to normalising a challenging situation • whether the output contained harmful or offensive language • whether the output gave medical advice While the human annotators had to indicate an overall preference for an output, they did not have to necessarily indicate their score on the desirable criterions if the large language model did not respond adequately to the desirable criterions. There was an overall preference for the fine-tuned large language model. Additionally, the fine-tuned large language model was consistently preferred on desirable criterions such as warmth, normalising, and cognitive empathy, which cause the user to continue to engage with the clinical treatment programme. The fine-tuned language model was also considered as safe as the not fine-tuned large language model. Figure 7a depicts annotated data from all human annotators while Figure 7b depicts annotated data from clinically trained human annotators. Each bar in these figures indicate how many times each model was chosen as the winner for desirable criterions indicated above each subplot. Error bars are standard errors of the mean. *** indicates p <0.001, ** indicates p <0.01, * indicates p <0.05, no asterisk indicates a non-significant difference. As seen in these figures, both clinically trained human annotators and not clinically trained human annotators preferred large language models that are fine-tuned as described herein. Referring back to Figure 2b, Figure 2b is a schematic illustration of a clinical treatment application 208b, according to a second example, that may be stored and executed by a system 100 as described in relation to Figure 1. The clinical treatment application 208b can be structurally and / or functionally identical to the clinical treatment application 108 in Figure 1. Execution of the clinical treatment application 108 will cause methods as described herein to be implemented. In this example, the clinical treatment application 208b comprises an input prompt allocation module 214. The input prompt allocation module 214 includes instructions and / or software code to implement an input prompt allocation method as described herein. Figure 8 shows a flow chart of an input prompt allocation method 832 according to an example, that may be performed by the clinical treatment application 108 when executed on the system 100 as discussed above. The input prompt allocation method 832 is configured to adaptively allocate a variant of an input prompt to a user interacting with a large language model. As noted above, fine-tuning a large language model may need the large language model to be transitioned to an offline state. In contrast, the input prompt allocation method 832 can be performed when the large language model is in an online state and is in use. The input prompt allocation method 832 may iterate over one or more variants of an input prompt to identify a best possible variant that can enable a large language model to produce an output that is configured to progress a user in a clinical treatment programme. At step 834, the method 832 comprises associating each variant of a plurality of variants of an input prompt to one or more users of a plurality of users. Generally, an input prompt may comprise one or more instructions that are configured to guide a large language model to produce an output. In this example, such an input prompt may have one or more variants. Put differently, each variant of an input prompt may have at least slightly different instructions from each other variant of the input prompt so as to guide the large language model to produce an output. It is possible that at least some variants of the input prompt that are different from at least some other variants of the input prompt may cause the large language model to produce different outputs. At step 834, the method 832 comprises randomly allocating each variant of the input prompt to a user. In this manner, different variants of an input prompt may be associated with different users in step 834. At 836, the method 832 comprises assigning an associated variant of an input prompt to a large language model. A variant that has been associated with a user in step 832 is then assigned to a large language model in step 836. More specifically, dialogue systems as described herein uses large language models. In some variations, the large language models that are used in each dialogue system may be substantially same. However, it may be possible that some large language models that are used in the dialogue systems described herein may be different from some other large language models that are used in the dialogue systems described herein. For example, some weighted parameters of these large language models may be different. Accordingly, once a variant of an input prompt is associated with a user at step 832, the associated variant may be assigned to a large language model in step 836. For example, if the large language models are substantially same ( / .e., a substantially same large language model is used in each dialogue system), then at step 836 the associated variant may be simply assigned to this substantially same large language model. However, if some large language models are different from some other large language models, then at step 836 the associated variant may be assigned to a specific large language model. Such assigning may be at random. At step 838, the method comprises determining a score for an associated variant based on the feedback from corresponding users. The score is indicative of a likelihood that an output that is produced by the large language model that is assigned the associated variant achieves a clinical benchmark. For example, before associating a variant to a user ( / .e., before performing step 834) and / or before assigning the associated variant to a large language model ( / .e., before performing step 836), there is equal uncertainty associated with each variant on whether that variant would enable a large language model to produce an output that is configured to progress a user in a clinical treatment programme. That is, before associating a variant to a user and / or before assigning the associated variant to a large language model, there is equal uncertainty on the score for a variant of an input prompt. This is operationalized by assigning the value to each variant of an input prompt i as a Bernoulli probability, where 9t is representative of an outcome probability e.g., likelihood that an output that is produced by the large language model that is assigned the variant i would progress a user in a clinical treatment programme ( / .e., achieve a clinical benchmark). Since, initially there is uncertainty around the outcome probability, the prior belief about is expressed as a Beta distribution, for example as seen in equation 4 - 9^ = P(outcome = 1| input prompt variant = j) And as seen in equation 5 - 91 ~ Beta Expected value E (9^ = ----— «i+ Pi Figure 9 depicts the scores of two hypothetical variants of an input prompt, with expected values corresponding to p = 0.33 and p = 0.66 shown in dotted lines. As seen in this figure, input prompt variant 1 has a higher value of uncertainty than that of input prompt variant 2. After associating a variant to a user (e.g., randomly allocating a variant to a user as in step 834) and after assigning the associated variant to a large language model (e.g., after step 836), the large language model produces an output responsive to receiving the associated variant. The user provides feedback on this output. The value of 9t is updated based on the feedback from the user. At step 840, the method 832 comprises training a machine learning model to adaptively allocate a variant of an input prompt to a large language model that is interacting with a user. For example, the machine learning model may be an adaptive sampling model that allocates a variant of an input prompt to a user interacting with a large language model. The machine learning model is trained such that the machine learning model is configured to balance exploration-exploitation tradeoff. Put differently, the machine learning model is trained to balance the need for continuing to collect more information (e.g., continuing to collect scores) about variants with uncertain values, while at the same time selecting the variant that has a best estimated score at a current time so as to ensure that the best variant is allocated to the users most frequently. This allows the method 832 to ensure that the number of users that are allocated a variant with a low score is minimized (referred to in this context as “minimizing regret”). Put differently, the loss incurred from choosing low scoring variants is minimized. The adaptive sampling model is trained such that the expected cumulative regret R(T) is minimized. To do so, the allocation policy is set to be equal to the score of a variant with the best likelihood of achieving a clinical benchmark ( / .e., to the probability of a variant with the best value). This may be achieved by sampling from each score and maximising with respect to the samples. For example, by performing Thompson sampling. Sampling may asymptotically minimize regret. For instance, as seen in equation 6 - T E[R(t)] = E[^ (y* ~ rt )] t=i Where r* is the optimal outcome. As noted above, allocation can be according to the policy as seen in equation 7 - N n(i) = P(9i = mctXjdj « — 1(0^ = maxj9jn^} n = l At step 842, the method 832 comprises adaptively allocating a variant (e.g., according to equation 7) to a user interacting with a large language model. As is discussed above, adaptively allocating the variant comprises minimizing an expected cumulative regret that is associated with allocating the variant. The score associated with each variant may be updated in a periodic manner. For example, at the end of an update cycle, such as for example, at the end of each day, after a predetermined number of allocations, etc., feedback from users can be used to update the scores. Accordingly, feedback from n users may be collected to have the score based on the number of successes k as follow as seen in equation 8, to yield a new estimate of theta with an updated score: 0 Beta (f^update’ Pupdate^where (^update (X + k and Pupdate P T Over time, the confidence that a given variant of an input prompt is the best prompt can be monitored (e.g., using the sampling estimate discussed above) and a winning variant can be chosen when the confidence exceeds a set threshold. For example, the winning variant can be chosen as seen in equation 9 - Choose variant i as winning if P(di = maXjd^ >© The method described herein provides several advantages in comparison to existing approaches that allocate variant of prompts. Firstly, in comparison to methods such as A / B testing, this method 832 does not require human oversight. Secondly, in A / B testing, regret is not maximized and the time to reach an optimal solution is typically fixed. In contrast, in method 832 the time to reach a threshold for choosing a winning variant is dynamic and may depend on the feedback that is collected from the users. Put differently, the method 832 is an adaptive method. Therefore, the method 832 is more efficient that the existing methods. For example, consider a scenario with two variants of an input prompt. A first variant enables a large language model to produce an output that users prefer 90 percent of the time, and a second variant enables a large language model to produce an output that users prefer 95 percent of the time. A standard approach to compare the first variant and the second variant would be to A / B test the variant with users, collect feedback on whether the outputs that were produced responsive to that variant was preferred by the users or not (e.g., whether the users thought that these outputs had achieved a clinical benchmark). This standard approach would run the risk of producing outputs that are not preferred by users roughly 10% of the time. In contrast, the method described in Figure 8 gathers this feedback from the users while minimizing the number of users that are allocated the worst variant, thereby eventually converging on the better variant. This minimizes the fraction of time users receive outputs that they do not prefer. Figure 10 depicts simulations of the method in Figure 8 on two variants of an input prompt - one that enables a large language model to produce an output that users prefer 95 percent of the time, and the other that enables a large language model to produce an output that the users prefer 90 percent of the time. In this figure, about 50 users have been allocated with one of these two variants each day and the rate of feedback is about 30 percent. 1062a shows the comparison of cumulative regret over time (i.e., total number of users that received outputs that they did not prefer) for A / B testing vs. for the method in Figure 8. 1062b shows the proportion of times the variant that enables a more preferred output to be outputted was selected by the method in Figure 8 over time, compared to selecting a variant by chance (e.g., as in A / B testing). The method in Figure 8 minimizes cumulative regret, while still allowing exploration of different variants. This method identifies the variant that enables a more preferred output to be outputted about 80 percent of the time within 2 weeks. Referring back to Figure 2c, Figure 2c is a schematic illustration of a clinical treatment application 208c, according to a third example, that may be stored and executed by a system 100 as described in relation to Figure 1. The clinical treatment application 208c can be structurally and / or functionally identical to the clinical treatment application 108 in Figure 1. Execution of the clinical treatment application 108 will cause methods as described herein to be implemented. In this example, the clinical treatment application 208c comprises a fine-tuning module 212 and an input prompt allocation module 214. The fine-tuning module 212 includes instructions and / or software code to implement a fine-tuning method as described herein. For example, the fine-tuning method can be the fine-tuning method 322a described in Figure 3a. As another example, the fine-tuning method can be the fine-tuning method 322b described in Figure 3b. As yet another example, the fine-tuning method can be a combination of the fine-tuning method 322a described in Figure 3a and the fine-tuning method 322b described in Figure 3b. For instance, in this example, the fine-tuning method may comprise training the machine learning model using binary classification, validating the machine learning model, and fine-tuning the large language model using supervised learning. In this instance, afterfinetuning using supervised learning, the fine-tuning may further comprise further fine-tuning the large language model using reinforcement learning. The input prompt allocation module 214 includes instructions and / or software code to implement an input prompt allocation method as described herein (e.g., as described in Figure 8). Figure 11 is a flow diagram depicting the implementation of both the input prompt allocation method and the fine-tuning method, according to an example. As discussed, both the input prompt allocation method and the fine-tuning method may be performed by the clinical treatment application 108 when executed on the system 100 as discussed above. In Figure 11, a large language model 1183 that is to be used in a dialogue system is fine-tuned using the fine-tuning method 1122 as described herein. During this fine-tuning 1122, the large language model 1183 may be in an offline state. In one example, a training dataset comprising one or more interactions with one or more users can be used to train a machine learning model. For instance, the dataset may include previous interactions that the large language model 1183 may have had with one or more users. At 1124, the training dataset can be used to fit an objective function of a machine learning model. At 1126, the fitted machine learning model can be validated using a validating dataset. For instance, for a given interaction in the validating dataset, the large language model 1183 can be configured to produce outputs multiple times. The output that is most likely to achieve a clinical benchmark can be selected from these produced outputs. This best candidate output can be used to generate a fine-tuning dataset. At 1128, the large language model 1183 can be fine-tuned using the fine-tuning dataset to generate a fine-tuned large language model 1183a. For instance, the large language model 1183 may be fine-tuned using supervised learning. Additionally, a variant of an input prompt can be adaptively allocated to a user as described herein. The adaptive allocation occurs when the fine-tuned large language model 1183a is in an online state ( / .e., in use). In one example, an input prompt 1185 can comprise one or more variants 1185’. Initially, each variant can be randomly associated with a user. The associated variant can be assigned to the fine-tuned large language model 1183a. Each variant is initially assigned a score using Beta distribution. The score indicates a likelihood that the variant would cause the fine-tuned large language model 1183a to produce an output that achieves a clinical benchmark. This score is a Bernoulli probability. After the variants are allocated to an initial set of users, feedback from this initial set of users is used to update the score of the variants. A machine learning model that comprises an allocating policy for allocating input variants is trained. The training is such that the machine learning model is configured to balance exploration-exploitation tradeoff. The allocating policy is also updated in a periodic manner based on the feedback that is obtained from the users. The machine learning model is configured to adaptively allocate the variants to new users. The scores of the variants and the allocating policy are adaptively updated based on feedback from the new users, until a winning variant is determined. For instance, until the confidence value for the winning variant is above a set threshold. In this manner, a new user is allocated a variant 1185a such that when the new user interacts with the fine-tuned machine learning model 1183a. The fine-tine large language model 1183a and the variant 1185a can be configured to cause the fine-tuned large language model 1183a to produce an output that is configured to progress a user in a clinical treatment programme. Together, the fine-tuning method and the input prompt allocation method may act as complementary techniques such that they enable a user to transition to an improved clinical state. These methods can be used synergistically. For instance, the large language models can be fine-tuned in an offline state. The fine-tuned large language models can be further validated on real users with the input prompt allocation method that is implemented when the large language models are in online state and in use. In this manner, undesirable outputs from the large language models can be reduced. Taken together, both these methods can cause a user to continue to engage with a clinical treatment programme while simultaneously improving the clinical state of the user. In this manner, the large language models that are used in the dialogue systems may exhibit clinically desirable skills (e.g., therapeutic skills). Any system feature as described herein may also be provided as a method feature, and vice versa. As used herein, means plus function features may be expressed alternatively in terms of their corresponding structure. Any feature in one aspect may be applied to other aspects, in any appropriate combination. In particular, method aspects may be applied to system aspects, and vice versa. Furthermore, any, some and / or all features in one aspect can be applied to any, some and / or all features in any other aspect, in any appropriate combination. It should also be appreciated that particular combinations of the various features described and defined in any aspects can be implemented and / or supplied and / or used independently.

Claims

1. A computer-implemented method, comprising:training a machine learning model to predict whether an interaction between a large language model and a user would progress the user in a clinical treatment programme from a first point in the clinical treatment programme to a more advanced point in the clinical treatment programme, wherein the interaction includes at least one output from the large language model, the training comprising:fitting the machine learning model using a first plurality of interactions in a first dataset, the first dataset comprising the first plurality of interactions between the large language model and a plurality of users of the large language model, each interaction of the first plurality of interactions including at least one output from the large language model to a corresponding user of the plurality of users;validating, using a second plurality of interactions between the large language model and a plurality of other users of the large language model, the machine learning model to generate a second dataset, the validating comprising:for each interaction of the second plurality of interactions:selecting, using the trained machine learning model, a candidate output from a plurality of candidate outputs that is most likely to achieve a clinical benchmark; andgenerating the second dataset based on the selected candidate output; andfine-tuning the large language model using the second dataset to generate a fine-tuned large language model,wherein after fine-tuning the large language model, the large language model is configured to provide, during interaction with a first user, an output that is configured to progress the first user in the clinical treatment programme.

2. The computer-implemented method of claim 1, wherein selecting a candidate output comprises selecting a candidate output that is most likely to achieve the clinical benchmark at a current time.

3. The computer-implemented method of claim 1, wherein selecting a candidate output comprises selecting a candidate output that is most likely to achieve the clinical benchmark at a future time after a further plurality of interactions between the large language model and a user.

4. The computer-implemented method of any one of the preceding claims, wherein validating the machine learning model further comprises:for each interaction of the second plurality of interactions:for a user input in that interaction:receiving, from the large language model, a first candidate output, wherein the first candidate output is outputted from the large language model in response to the large language model receiving the user input a first time, andreceiving, from the large language model, a second candidate output, wherein the second candidate output is outputted from the large language model in response to the large language model receiving the user input a second time; andselecting the first candidate output or the second candidate output as the candidate output, wherein the plurality of candidate outputs include the first candidate output and the second candidate output.

5. The computer-implemented method of any of the preceding claims, wherein fine-tuning the large language model further comprises:training, using supervised training and based on the second dataset, the large language model.

6. The computer-implemented method of any of the preceding claims, wherein the large language model comprises weighted parameters that enable the large language model to provide outputs, and fine-tuning the large language model comprises modifying one or more weighted parameters of the large language model.

7. The computer-implemented method of any one of the preceding claims, wherein the machine learning model comprises a binary classification random forest model.

8. The computer-implemented method of any of the preceding claims, wherein the machine learning model comprises a regression model.

9. The computer-implemented method of any one of the preceding claims, wherein training the machine learning model to predict whether the interaction between the large language model and the user would progress the user in a clinical treatment program comprises training the machine learning model to predict whether the interaction between thelarge language model and the first user would cause the first user to initiate a further plurality of interactions with the large language model.

10. The computer-implemented method of any one of the preceding claims, wherein theclinical benchmark meeting a predetermined threshold is representative of whether the first user has progressed in the clinical treatment programme.

11. The computer-implemented method of claim A10, wherein the clinical benchmark is representative of whether one or more of:adherence of the first user to the clinical treatment programme, engagement of the first user to the clinical treatment programme, interactions initiated by the first user with the large language model, likelihood of adverse clinical events faced by the first user, reduction of clinical symptoms of the first user, or clinical recovery of the first user.

12. A computer-implemented method, comprising:training a machine learning model to predict whether an interaction between a large language model and a user would progress the user in a clinical treatment programme from a first point in the clinical treatment programme to a more advanced point in the clinical treatment programme, wherein the interaction includes at least one output from the large language model, the training comprising:fitting the machine learning model using a first plurality of interactions in a first dataset, the first dataset comprising the first plurality of interactions between the large language model and a plurality of users of the large language model, each interaction of the first plurality of interactions including at least one output from the large language model to a corresponding user of the plurality of users; andfine-tuning the large language model using the trained machine learning model to generate a fine-tuned large language model,wherein after fine-tuning the large language model, the large language model is configured to provide, during interaction with a first user, an output that is configured to progress the first user in the clinical treatment programme.

13. The computer-implemented method of claim 12, wherein fine-tuning the large language model further comprises:training, using reinforcement learning, the large language model.

14. The computer-implemented method of any of the preceding claims, wherein the large language model comprises weighted parameters that enable the large language model to provide outputs, and fine-tuning the large language model comprises modifying one or more weighted parameters of the large language model.

15. The computer-implemented method of claim 12, wherein fine-tuning the large language model further comprises:during the interaction between the large language model and the first user, and in response to being provided with an input from the first user:receiving, from the large language model, a first output a first time;receiving, from the large language model, a second output a second time; and selecting, using the trained machine learning model, the first output or the second output as the output to be outputted by the large language model, the output being configured to progress the first user in the clinical treatment programme.

16. The computer-implemented method of any one of the preceding claims, wherein training the machine learning model to predict whether the interaction between the large language model and the user would progress the user in a clinical treatment program comprises training the machine learning model to predict whether the interaction between the large language model and the first user would cause the first user to initiate a further plurality of interactions with the large language model.

17. The computer-implemented method of any one of the preceding claims, wherein theclinical benchmark meeting a predetermined threshold is representative of whether the first user has progressed in the clinical treatment programme.

18. The computer-implemented method of claim 17, wherein the clinical benchmark is representative of whether one or more of:adherence of the first user to the clinical treatment programme, engagement of the first user to the clinical treatment programme, interactions initiated by the first user with the large language model, likelihood of adverse clinical events faced by the first user, reduction of clinical symptoms of the first user, or clinical recovery of the first user.

19. A computer-implemented method, comprising:training a machine learning model to adaptively allocate a variant of an input prompt from a plurality of variants of the input prompt to a first large language model of a plurality of large language models, the first large language model interacting with a first user of a plurality of users, the input prompt being configured to guide each of the plurality of large language models to produce a corresponding output, the training the machine learning model comprising:associating each variant of the plurality of variants of the input prompt to one or more other users of the plurality of users, each other user interacting with a respective one of the plurality of large language models,assigning the associated variant to the respective one of the plurality of large language models interacting with the corresponding one or more other users, anddetermining a score for each associated variant of the plurality of variants based on feedback from the corresponding one or more other users, the score being indicative of a likelihood that a respective output produced by the respective one of the plurality of large language models responsive to that variant achieves a clinical benchmark; andadaptively allocating, using the trained machine learning model, the variant of the input prompt to the first user, wherein the allocated variant causes the first large language model to produce an output, the output being configured to progress the first user in a clinical treatment programme from a first point in the clinical treatment programme to a more advanced point in the clinical treatment programme.

20. The computer-implemented method of claim 19, wherein determining a score comprises determining a score indicative of a likelihood that the respective output produced by the respective one of the plurality of large language model responsive to that variant achieves the clinical benchmark at a current time.

21. The computer-implemented method of claim 19, wherein determining a score comprises determining a score indicative of a likelihood that the respective output produced by the respective one of the plurality of large language model responsive to that variant achieves the clinical benchmark at a future time.

22. The computer-implemented method of any one of the preceding claims, wherein associating each variant comprises randomly allocating each variant to one or more other users of the plurality of users.

23. The computer-implemented method of any of the preceding claims, wherein adaptively allocating the variant of the input prompt to the first user comprises minimizing an expected cumulative regret associated with allocating that variant.

24. The computer-implemented method of any one of the preceding claims, wherein adaptively allocating the variant comprises:responsive to determining that the score associated with the variant exceeds a first predetermined threshold, selecting the variant from the plurality of variants.

25. The computer-implemented method of any one of the preceding claims, wherein the clinical benchmark is representative of whether the progress of the first user in the clinical treatment programme meets a second predetermined threshold.

26. The computer-implemented method of claim 25, wherein the clinical benchmark is representative of whether one or more of:adherence of the first user to the clinical treatment programme, engagement of the first user to the clinical treatment programme, interactions initiated by the first user with the large language model, likelihood of adverse clinical events faced by the first user, reduction of clinical symptoms of the first user, or clinical recovery of the first user, meets the second predetermined threshold.

27. A computer-implemented method for automatically transmitting a treatment output to a user, the method comprising:fine-tuning, using a first machine learning model, a large language model, the first machine learning model being configured to predict whether an output from the large language model would achieve a first clinical benchmark;adaptively allocating, using a second machine learning model, a variant of an input prompt from a plurality of variants of the input prompt to the large language model, the second machine learning model being configured to select a variant of the input prompt that is most likely to cause the large language model to produce an output that achieves a second clinical benchmark,wherein after the fine-tuning and the allocating, and during an interaction with a user, the large language model is configured to provide an output that is configured to progress the user in a clinical treatment programme from a first point in the clinical treatment programme to a more advanced point in the clinical treatment programme.

Images