Medical trial outcome prediction confidence

Abstract

Disclosed is a method for generating a medical trial conclusion for a medical trial of a participant intervention, the medical trial conclusion depending on an outcome prediction of the medical trial. The method uses medical impact data for existing participants in the medical trial. The outcome prediction indicates an expected outcome of the medical trial in terms of the performance of the participant intervention. The expected outcome depends on the performance of the participant intervention for the existing participants. The method comprises, given the medical impact data for the existing participants, determining a counter prediction probability which is the probability of the medical impact data for further medical trial participants being adverse to the outcome prediction to an extent minimally sufficient for the outcome prediction to be confounded. The method further comprises generating, as at least part of the medical trial conclusion, a confidence in the outcome prediction in accordance with the determined counter prediction probability.

Description

[0001] MEDICAL TRIAL OUTCOME PREDICTION CONFIDENCE

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to a medical trial outcome prediction confidence. Aspects of the invention relate to methods for generating medical trial conclusions, medical trial conclusion generating systems, computer programs, non-transitory computer readable storage mediums and signals.

[0004] BACKGROUND

[0005] Conventional practice in the art of medical trials is to rigidly design the medical trial in advance and then rigidly complete the medical trial in its entirety (except for instance where safety concerns are identified as the medical trial progresses). In many cases however, the outcome of a medical trial might be predicted at least somewhat ahead of its completion. Nonetheless, particularly in view of the inadequacy if existing outcome prediction techniques, the existing rigid approach is followed.

[0006] Providing better outcome prediction capability and detail has many potential benefits. For instance, it could lead to the earlier and / or better treatment of patients, better design of future medical trials, prevention of the wasting of resources and / or more timely / justified decision making e g. as regards unblinding, medical trial performance analysis, and production related to treatments.

[0007] It is an aim of the present invention to address one or more of the disadvantages associated with the prior art.

[0008] SUMMARY OF THE INVENTION

[0009] According to an aspect of the invention there is provided a method optionally for generating a medical trial conclusion for a medical trial optionally of a participant intervention, the medical trial conclusion optionally depending on an outcome prediction of the medical trial, where: the method optionally uses medical impact data for existing participants in the medical trial, the medical impact data for the existing participants optionally corresponding to the medical trial in incomplete form, participant medical impact data for further participants in the medical trial optionally being required for correspondence with the medical trial in complete form; and the outcome prediction optionally indicates an expected outcome of the medical trial optionally in terms of the performance ofthe participant intervention, the expected outcome optionally depending on the performance of the participant intervention for the existing participants optionally when compared to an expected performance for the existing participants, optionally in order that a predetermined outcome threshold would be equalled with the inclusion of medical impact data for the further participants optionally corresponding with the medical trial in complete form and where optionally the expected performance accounts for a completion proportion ofthe medical trial optionally corresponding to the medical impact data for the existing participants, the method optionally comprising: optionally given the participant medical impact data for the existing participants, optionally determining a counter prediction probability which is optionally the probability of the participant medical impact data forthe further medical trial participants being adverse to the outcome prediction optionally to an extent minimally sufficient for the outcome prediction to be confounded; optionally generating, optionally as at least part of the medical trial conclusion, a confidence in the outcome prediction optionally in accordance with the determined counter prediction probability.

[0010] This may be advantageous in providing a reliable confidence associated with the outcome prediction. Specifically, it may be considered that the method behaves in an intelligent and adversarial manner, using the benefit of knowledge of the medical impact data for the existing participants and the pre-determined outcome threshold, to determine the most likely scenario leading to the outcome prediction proving incorrect (i.e. incorrect by the narrowest of margins, so as to maximally confound the outcome prediction). The medical trial conclusion may therefore be considered a reliable guide to a worst case scenario concerning the nature of the participant medical impact data for the existing participants in terms of the making of an accurate outcome prediction.

[0011] The participant intervention could be of any suitable medical type e.g. use and / or omission of a medical treatment or combination of treatments, a particular dosage regime of a medical treatment, a surgical intervention and / or a behavioural intervention.

[0012] The pre-determined outcome threshold might for instance be an efficacy probability threshold for a treatment being reached, but alternatives are also possible, e.g. proportion of participants encountering a particular side effect. Still further alternatives need not necessarily be directly relevant / determinative forthe trial outcome but may for instance be relevant to clinical care and / or a patients’ clinical state. For instance, blood test results could be the subject data of the participant medical impact data and could be assessed against a predetermined outcome threshold for blood pressure (e.g. change / stability / crossing a threshold / absolute value etc), with an outcome prediction being made in respect of the blood pressure data and a confidence in the outcome prediction determined. Consequently and as will be appreciated, whilst the expected outcome of the medical trial may correspond to a parameter that determines success or failure of the medical trial, this need not be the case.

[0013] The method may comprise collecting the medical impact data for the existing participants and / or generating the outcome prediction.

[0014] In some embodiments, the method comprises performing at least one of the following as part of the generated medical trial conclusion: i) Treating one or more patients in dependence on the confidence in the prediction; ii) Designing another medical trial in dependence on the confidence in the prediction; ill) Adapting the medical trial in dependence on the confidence in the prediction; iv) Halting the medical trial before its planned conclusion in dependence on the confidence in the prediction; v) Making a determination on the success or failure of the medical trial in dependence on the confidence in the prediction; vi) Determining whether or not to unblind the medical impact data of the existing participants at a given stage of the medical trial; vii) Determining whether or not to assess the medical trial at a given stage of the medical trial; viii) Labelling, optionally automatically, a medication for treatment of one or more patients in dependence on the confidence in the prediction; ix) Determine the timing and / or scale of production of a facilitating technology for the participant intervention in dependence on the confidence in the prediction.

[0015] Consequently, the enhanced understanding afforded by the method in terms of the confidence of the outcome prediction may be acted upon. To give an example of each of the options mentioned: a) Regarding i), it could be for instance that the confidence in an outcome prediction is suggestive of the treatment being sufficiently likely and / or unlikely to prove effective as to be sufficient for treatment decisions to be taken before the completion of the medical trial. The one or more patients may include participants in the medical trial. b) Regarding ii), it could be for instance that the confidence in an outcome prediction is suggestive of an intervention being sufficiently likely and / or unlikely to prove effective as to be sufficient for informing the design of another (future) medical trial, to increase the likelihood of that other medical trial giving useful / decisive results. For instance, a trial may be designed with a particular treatment or dosage and / or with trial participants recruited having particular traits. c) Regarding iii), it may be that the medical trial can be usefully adapted where the confidence in the outcome prediction contributes to or is sufficient to render this justified (e.g. more useful than completing the medical trial according to its existing design). d) Regarding iv), it may be appropriate to halt the medical trial before its completion where for instance the confidence in the outcome prediction suggests or contributes to suggesting that the prospects for a favourable outcome are sufficiently poor that there is considered insufficient justification for completing the medical trial. Confidence in an outcome prediction being suggestive of excessive risk or danger is another example of circumstances where halting the medical trial may be appropriate. e) Regarding v), it may be possible to pre-empt the completion of the medical trial where for instance the confidence in the outcome prediction is considered sufficient to justify this and that therefore the final outcome can be assumed known. f) Regarding vi), unblinding (especially more than is necessary) may be considered undesirable because it may increase the likelihood of the medical trial being compromised and / or reduce the perceived value of the medical trial. Nonetheless, unblinding may be desirable / necessary in order to analyse the performance of the intervention in particular ways. The confidence in the prediction may indicate or contribute to indicating whether or not unblinding at a given stage of the medical trial is justified. g) Regarding vii), the confidence in the outcome prediction may be indicative of the likelihood of being able to draw significant conclusions regarding the intervention and / or the medical trial if the medical trial is analysed. This may in turn inform whether or not to analyse the medical trial at a given stage of the medical trial. h) Regarding viii), the confidence in the outcome prediction may give sufficient confidence to justify production of a pharmaceutical or piece of treatment equipment desirable / required for the participant intervention.

[0016] In some embodiments the method comprises determining the outcome prediction by: comparing the medical impact data for the existing participants that are in a first arm of the medical trial with medical impact data for the existing participants that are in a second arm of the medical trial, where the participants in the first arm receive the participant intervention and the participants in the second arm receive a comparative approach in accordance with the medical trial, the comparison thereby giving a performance of the participant intervention as compared with the comparative approach for the existing participants; calculating an interim outcome prediction threshold indicating an expected outcome for the completion proportion of the medical trial corresponding to the medical impact data for the existing participants, in order that the pre-determined outcome threshold would be equalled with the inclusion of medical impact data for the further participants corresponding with the medical trial in complete form; comparing the performance of the participant intervention as compared with the comparative approach for the existing participants with the interim outcome threshold to determine any discrepancy between the two to thereby generate the outcome prediction.

[0017] As will be appreciated, the comparative approach will be different to the participant intervention in order that it allows for performance comparison. Like the participant intervention, it could be use and / or omission of a medical treatment or combination of treatments, a particular dosage regime of a medical treatment, a surgical intervention and / or a behavioural intervention. It might in particular be a control intervention (e.g. treatment with a placebo), no treatment and / or current best practice care / treatment.

[0018] Comparison of data associated with the first and second arms may provide a basis on which to generate an outcome prediction (and ultimately a conclusion to the medical trial) i.e. a point of comparison.

[0019] Taking a specific example, the outcome prediction calculated might be predicted efficacy probability for the participant intervention as compared with the comparative approach and the pre-determined outcome threshold might be a pre-determined efficacy probability threshold required for the medical trial to be considered successful (e.g. the participant intervention being ‘proved’ effective to the required standard of probability by comparison with the comparative approach). In this case, the interim outcome prediction threshold might be an interim efficacy probability threshold, this being the efficacy probability threshold expected to be reached at a given stage of the medical trial if the pre-determined efficacy probability threshold is to be met with participant medical impact data for the complete medical trial. Therefore, the interim efficacy probability threshold may provide a basis to evaluate the expected efficacy with respect to the pre-determined efficacy probability threshold in order to generate the outcome prediction. The outcome prediction may provide the basis from which to determine the counter prediction probability.

[0020] As will be appreciated, the interim efficacy probability threshold will change in dependence on the completed proportion (or stage) of the medical trial (i.e. the completeness of the participant medical impact data used / available) because where the data is less complete, less statistical confidence should be expected in it currently reflecting an ultimate trend. However, as more data is available, its statistical strength in predicting any ultimate trend will necessarily increase and so the interim efficacy probability threshold should tend towards agreement with the pre-determined efficacy probability threshold.

[0021] The above is just one example among many possible. To illustrate, it could be for instance that the outcome prediction calculated is a predicted side effect rate for the participant intervention as compared with the comparative approach and the pre-determined outcome threshold might be a pre-determined side effect rate threshold that must not be exceeded for the medical trial to be considered successful. In this case, the interim outcome prediction threshold might be an interim side effect rate threshold, this being the side effect rate threshold that should not be exceeded at a given completed proportion of the medical trial for there to be an expectation that the pre-determined side effect rate threshold will not be exceeded at the end of the medical trial.

[0022] In some embodiments, the comparing of the medical impact data for the existing participants in the first arm of the medical trial and the medical impact data for existing participants in the second arm of the medical trial is performed by calculating the z score between a distribution of the medical impact data for the existing participants in the first arm and a distribution of the medical impact data for the existing participants in the second arm.

[0023] This may provide a convenient mechanism for comparing the two sets of participant medical impact data formatted in a distribution / histogram. Specifically, it may provide a mechanism by which to characterise the difference between the two data sets.

[0024] In some embodiments, determining the counter prediction probability comprises: determining a confounding distribution required for the medical impact data for the further medical trial participants in order to have the minimum necessary effect such that the outcome prediction is confounded; and calculating the probability of the confounding distribution occurring given the medical impact data for the existing participants.

[0025] By calculating a confounding distribution for the medical impact data for the further medical participants, the probability ultimately determined may be more accurate (i.e. based on a more realistic data set of medical impact data forthe further medical participants). This confounding distribution may be considered the minimum in terms of impact to result in the pre-determined outcome threshold being crossed at the end of the medical trial by comparison with the outcome prediction relative to the interim outcome threshold, but to the minimum extent necessary in order that the outcome prediction is proven incorrect. That is, the crossing would result in an outcome prediction of success being confounded such that the medical trial is a narrow failure or an outcome prediction of failure being confounded such that the medical trial is a narrow success. In this context, narrow means by the smallest of margins possible.

[0026] In some embodiments, the determining of the confounding distribution is performed by calculating a z score representing the confounding distribution, the z score being that required between distributions of the medical impact data for the further participants in the first arm and the medical impact data forthe further participants in the second arm in order to have the minimum necessary effect such that the outcome prediction is confounded.

[0027] This amounts to calculating the mean and standard deviation for the confounding distribution required. In particular, the method may subtract, as a function of time, the z score between distributions for the medical impact data for the existing participants in the first arm and the medical impact data for the existing participants in the second arm from a z score between distributions for participant medical impact data forthe first arm and participant medical impact data for the second arm as if the medical trial were completed and the predetermined outcome threshold is just satisfied.

[0028] In some embodiments, the calculating of the counter prediction probability is performed by calculating an area under a curve for an overlap region of: a) the determined confounding distribution; and b) a distribution corresponding to the z score between the distribution of the medical impact data forthe existing participants in the first arm and the distribution of the medical impact data for the existing participants in the second arm.

[0029] This area may correspond to the probability that the confounding distribution (or something more extreme than the confounding distribution) will be observed with respect to the further participants, i.e. the probability that the outcome prediction at the time of calculation will turn out to be wrong.

[0030] In some embodiments, calculating the interim outcome prediction threshold is performed by multiplying a predetermined constant by the square root of a metric for the completion proportion of the medical trial corresponding to the medical impact data for the existing participants.

[0031] The metric may for instance correspond to a number between 0 and 1 in proportion to the number of participants so far recruited and for which medical impact data is available / used vs the number of total participants planned for the medical trial (i.e. existing participants and further participants). Calculating the interim outcome prediction threshold in this manner may produce an interim outcome prediction threshold that increases as the completion proportion / stage of the medical trial progresses, with a rate of increase that reduces over the course of the medical trial progression. This may be appropriate given that: as the medical trial progresses, the totality of the data available should be expected to offer ever more certainty as regards the relevant pre-determined outcome threshold being at least met, if indeed that threshold is to be at least met; this increase should not be linear, given that when only a small proportion of the total data ultimately expected is being used (e.g. near to the beginning of the medical trial), there should be a disproportionally small expectation in terms of what the outcome should be expected to indicate regarding whether or not the pre-determined outcome threshold is likely to be met. This is due to the significant scope for change resulting from the proportionately large volume of remaining data.

[0032] As noted previously, it may be that the outcome prediction calculated is predicted efficacy probability for the participant intervention as compared with the comparative approach and the pre-determined outcome threshold is a pre-determined efficacy probability threshold required for the medical trial to be considered successful. In this case, the constant might be selected in accordance with the required probability that the observed effect would have occurred by chance if, in reality, there is no real effect. This is known as the ‘p- value' and is often set at 0.05%. This would correspond to the value of the constant being approximately 1 .96 (though as will be appreciated, values only corresponding substantially to this value would in most cases yield useful values for the interim outcome prediction threshold). The constant 1 .96 is approximately the value of a z-score required between a distribution of the medical impact data for the participants in the first arm and a distribution of the medical impact data for the participants in the second arm, such that the area under the distribution to the right of this value corresponds to 0.05. That is, the probability of the outcome prediction being incorrect in this case (due to chance). As will be appreciated, this approach can be used to determine the constant in accordance with the p-value required / desired.

[0033] In some embodiments, the method is repeated.

[0034] The method may for instance be repeated periodically and / or when additional participant medical impact data is acquired / available. As will be appreciated, the additional participant medical impact data may correspond to additional data for one or more of the existing medical trial participants and / or data for a further participant or participants. With the additional participant medical impact data, the outcome prediction and / or the confidence therein may be updated, potentially thereby improving the quality of its information content and utility.

[0035] In some embodiments, the medical trial conclusion comprises a projection for the confidence in the outcome prediction expected at one or more completion proportions of the medical trial corresponding to additional medical impact data of the further participants having been used by extrapolating the calculated values for the confidence in the outcome prediction calculated in accordance with the method.

[0036] The completion proportion(s) of the medical trial may correspond to a completed / used proportion of the participant medical impact data ultimately planned to be collected / used in accordance with the medical trial. The projection may for instance take the form of a trace showing expected confidence at different future stages, or a prediction of a point at which a particular pre-defined confidence in the outcome prediction is projected to be achieved.

[0037] In some embodiments, the existing and further participants are blind or double blind randomised between the first and second arms and the method comprises unblinding of the participant medical impact data for performance of the method and where further the method steps performed to generate the confidence in the outcome prediction including the unblinding are performed by a computer system without allowing human access to the unblinded participant medical impact data.

[0038] The unblinded participant medical impact data may be used to perform generation of the outcome prediction and / or determination of the counter predication probability and / or generating the confidence in the outcome prediction.

[0039] The computer system may be of any suitable configuration. It may for instance comprise one or more computers and / or processors and / or servers for the performance of processing in the method. At least part of the method may be hosted / orchestrated by a cloud based platform. The computer system may include any suitable hardware devices for the collection / input of participant medical impact data e.g. computers, dedicated terminals, wearable electronic devices, portable electronic devices, diagnostic devices, monitoring devices and / or smartphones. The process of collection of participant medical impact data may be automated or semiautomated. The computer system may comprise one or more wired or wireless networks via which one or more entities / devices of the computer system may communicate (e.g. for the transmission of data and / or the results of processing of data). The communication networks and / or any of the entities / devices may be secured e.g. via encryption. The computer system may have access to a randomisation schedule detailing the arms to which each of the existing participants in the medical trial are assigned. This may have been provided to the computer system (e.g. uploaded) from a source which does not have access to the participant medical impact data.

[0040] According to an aspect of the invention there is provided a computer program that, when read by a computer, causes performance of the method previously described.

[0041] According to an aspect of the invention there is provided a non-transitory computer readable storage medium comprising computer readable instructions that, when read by a computer, cause performance of the method previously described.

[0042] According to an aspect of the invention there is provided a signal comprising computer readable instructions that, when read by a computer, cause performance of the method as previously described.

[0043] According to an aspect of the invention there is optionally provided a medical trial conclusion generating system optionally arranged to generate a medical trial conclusion optionally for a medical trial of a participant intervention, the medical trial conclusion optionally depending on an outcome prediction of the medical trial, where: the system optionally uses medical impact data optionally for existing participants in the medical trial, the medical impact data for the existing participants optionally corresponding to the medical trial in incomplete form, medical impact data for further participants in the medical trial optionally being required for correspondence with the medical trial in complete form; and the outcome prediction optionally indicates an expected outcome of the medical trial optionally in terms of the performance of the participant intervention, the expected outcome optionally depending on the performance of the participant intervention optionally for the existing participants optionally when compared to an expected performance for the existing participants optionally in order that a pre-determined outcome threshold would be equalled with the inclusion of medical impact data for the further participants optionally corresponding with the medical trial in complete form and optionally where the expected performance accounts for a completion proportion of the medical trial optionally corresponding to the medical impact data for the existing participants, the system optionally comprising: optionally an input means optionally arranged to receive the medical impact data for the existing participants in the medical trial; optionally a processing means optionally arranged to: optionally analyse the medical impact data for the existing participants in the medical trial optionally to determine a counter prediction probability which is optionally the probability of the medical impact data for the further medical trial participants being adverse to the outcome prediction optionally to an extent minimally sufficient for the outcome prediction to be confounded; and optionally generating, optionally as at least part of the medical trial conclusion, a confidence in the outcome prediction optionally in accordance with the determined counter prediction probability; and optionally an output means optionally arranged to output the confidence in the outcome prediction.

[0044] The medical trial conclusion generating system may additionally receive the outcome prediction of the medical trial via the input means or may generate this using the processing means. The medical trial conclusion generating system may also receive data indicating the planned extent of the medical trial (e.g. the number of further participants planned and / or the quantity of additional participant medical impact data planned to be collected / provided for the further participants) via the input means.

[0045] The medical trial conclusion generating system may be a computer system or computer implemented.

[0046] The input means may comprise one or more inputs, which may be electrical inputs, for receiving a signal or signals indicative of the medical impact data and / or the outcome prediction and / or the data indicating the planned extent of the medical trial. The processing means may comprise one or more processing devices such as electronic processing devices. The output means may comprise one or more outputs, which may be electrical outputs, for outputting a signal or signals indicative of the counter prediction probability and / or the confidence in the outcome prediction.

[0047] According to an aspect of the invention there is optionally provided a medical trial conclusion generating system optionally arranged to generate a medical trial conclusion optionally for a medical trial of a participant intervention, the medical trial conclusion optionally depending on an outcome prediction of the medical trial, where: the system optionally uses medical impact data optionally for existing participants in the medical trial, the medical impact data for the existing participants optionally corresponding to the medical trial in incomplete form, medical impact data for further participants in the medical trial optionally being required for correspondence with the medical trial in complete form; and the outcome prediction optionally indicates an expected outcome of the medical trial optionally in terms of the performance ofthe participant intervention, the expected outcome optionally depending on the performance of the participant intervention optionally for the existing participants optionally when compared to an expected performance forthe existing participants in orderthat a pre-determined outcome threshold would be equalled with the inclusion of medical impact data for the further participants corresponding with the medical trial in complete form and where the expected performance accounts for a completion proportion of the medical trial optionally corresponding to the medical impact data for the existing participants, the system optionally comprising: optionally a data gathering node optionally comprising collecting the medical impact data for the existing participants in the medical trial; optionally a computer implemented analysis node comprising: optionally analysing the medical impact data forthe existing participants in the medical trial optionally to determine a counter prediction probability which is optionally the probability of the medical impact data forthe further medical trial participants being adverse to the outcome prediction optionally to an extent minimally sufficient forthe outcome prediction to be confounded; and optionally generating, optionally as at least part ofthe medical trial conclusion, a confidence in the outcome prediction optionally in accordance with the determined counter prediction probability; and optionally an implementation node optionally comprising at least one of: i) Treating one or more patients in dependence on the confidence in the prediction; ii) Designing another medical trial in dependence on the confidence in the prediction; iii) Adapting the medical trial in dependence on the confidence in the prediction; iv) Halting the medical trial before its planned conclusion in dependence on the confidence in the prediction; v) Making a determination on the success or failure of the medical trial in dependence on the confidence in the prediction; vi) Determining whether or not to unblind the existing participant medical impact data at a given stage of the medical trial; vii) Determining whether or not to assess the medical trial at a given stage of the medical trial; viii) Labelling, optionally automatically, a medication for treatment of one or more patients in dependence on the confidence in the prediction; ix) Determine the timing and / or scale of production of a facilitating technology for the participant intervention in dependence on the confidence in the prediction.

[0048] The data gathering node may comprise collecting the outcome prediction of the medical trial or this may be generated in the analysis node. The data gathering node may comprise collecting data indicating the planned extent of the medical trial (e.g. the number of further participants planned and / or the quantity of additional participant medical impact data planned to be collected / provided for the further participants).

[0049] Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and / or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and / or features of any embodiment can be combined in any way and / or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and / or incorporate any feature of any other claim although not originally claimed in that manner.

[0050] BRIEF DESCRIPTION OF THE DRAWINGS

[0051] One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0052] Figure 1 shows a flow chart of a method for generating a medical trial conclusion according to an embodiment of the invention;

[0053] Figure 2 shows, according to an embodiment of the invention, a graph over information time of a z score between a distribution of first trial arm medical impact data for existing participants and second trial arm medical impact data for existing participants as well as an interim efficacy probability threshold trace over information time;

[0054] Figure 3 shows a graph including a confounding distribution according to an embodiment of the invention; Figure 4 shows a graph illustrating projected counter prediction probability extrapolated from counter prediction probability determined through information time according to an embodiment of the invention;

[0055] Figure 5 shows a schematic representation of a medical trial conclusion generating system according to an embodiment of the invention; and

[0056] Figure 6 shows a schematic representation of a medical trial conclusion generating system according to an embodiment of the invention.

[0057] DETAILED DESCRIPTION

[0058] A method for generating a medical trial conclusion is discussed below with reference to the Figures. Prior to the performance of the method, a medical trial is designed and at least partially performed. In the present case, an example medical trial is designed to analyse whether or not a higher dose of a cancer drug (a first intervention) is more effective at reducing the size of a particular type of cancer tumour than a lower dose of the drug (a comparative approach in this case a second intervention).

[0059] A pre-determined outcome threshold is defined which must be reached in order for the medical trial to be considered to have shown that the higher dose is more effective. Specifically, it must be shown that the higher dose has on average the effect of a greater reduction in tumour size than the lower dose with the probability of this effect having occurred at random (ratherthan it being a real effect inherent to the dosage administered) of less than 0.05%. This is known as a p-value of less than 0.05%. In this case the pre-determined outcome threshold is therefore a pre-determined efficacy probability threshold of below 0.05%.

[0060] The trial is designed to include 100 participants all having tumours of the relevant type. The participants will be recruited and treated over time. The participants will be divided into two arms, a first arm that will receive the higher dose of the drug and a second arm that will receive the lower dose. The medical trial will be conducted under double blind conditions. Consequently, neither the participants nor their clinicians will know which arm of the trial each participant is in. A blinding schedule, detailing which participants are in which arm of the trial will be kept securely by a computer system.

[0061] Performance monitoring of the first and second arms will be performed in the same manner. Specifically, the size of the relevant participant’s tumour will be measured prior to administering the relevant dosage of the drug. In six weeks time from the dosage being administered, the size of the relevant tumour is measured again. Any change in tumour size for the participant is recorded as a percentage of the original tumour size. This constitutes medical impact data for the relevant participant arising from their participation in the medical trial. This participant medical impact data is collected and input to the computer system by personnel different to those that have generated and uploaded the blinding schedule to the computer system. In the implementation of the present method, analysis of the participant medical impact data is conducted without the complete set of planned participant medical impact data being used. Typically this is because the medical trial is not yet complete (e.g. participant medical impact data has not yet been collected for all 100 planned participants and indeed at least some participants may not yet have been administered with the drug or even recruited), though in other cases it may be that whilst the medical trial is completed, not all participant medical impact data is yet available. It could also be that an incomplete medical trial is simulated by only using some of the participant medical impact data acquired in order to test / analyse the method described herein given that the outcome of the medical trial is in fact known. For the purposes of this document, references to ‘completion proportion’ should be interpreted to encompass both scenarios indicating the stage of the completion of the medical trial itself and scenarios indicating only the proportion of the participant medical impact data used in the method from a corpus of medical impact data that would correspond with the medical trial being complete (that is whether or not the medical trial is complete or further progressed in reality). Those of the participants for which their medical impact data is used in the method are for the purposes of this document referred to as existing participants, whilst participants that are planned in terms of meeting the medical trial design, but whose medical impact data is not used in the method (e.g. because it has not yet been collected) are for the purposes of this document referred to as further participants.

[0062] Potential benefits of conducting analysis without the complete set of planned participant medical impact data include making predictions concerning the outcome of the medical trial (e.g. whether or not the pre-determined efficacy probability threshold will be met). This may offer many potential benefits including for instance facilitating improved treatment options / decisions for patients and / or more effective designing and / or managing of the and / or other medical trials. The present method analyses the confidence in such outcome predictions. This may offer further benefits in terms of the accuracy, nature and / or timing of actions taken based on such predictions. The method may be expanded to include taking various steps dependent on the confidence in the prediction and / or predicting the quantity of participant medical impact data (e.g. corresponding to the completion proportion of the medical trial) that will be required such that the confidence in the outcome prediction will reach a particular level.

[0063] Referring now to Figure 1 , an embodiment of the method is described as implemented at least in part by a computer system. In a data acquisition step 10, the computer system acquires the medical impact data of the existing participants. In a blinding acquisition step 12, the computer system acquires the blinding schedule which allows each piece of the participant medical impact data acquired in the data acquisition step 10 to be assigned to the relevant first or second arm of the medical trial. In a deblinding step 14, the computer system uses the blinding schedule to deblind the participant medical impact data, assigning each piece of the participant medical impact data to the correct arm of the medical trial.

[0064] In a comparison step 16, the computer system begins the process of generating an outcome prediction for the medical trial using the participant medical impact data that it has, that is all the medical impact data for the existing participants. In the comparison step 16, the computer system analyses the performance of the higher dose of the drug as compared with the lower dose by comparing a distribution of proportional tumour size changes for the existing participants in the first arm with a distribution of proportional tumour size changes for the existing participants in the second arm. Specifically, a z score between these two distributions is calculated. The z score maps the distribution of differences between the participant medical impact data for the two arms to one point on a standard normal distribution. The z score effectively therefore provides a metric quantifying the difference in the two distributions and thereby the performance of the higher dose in comparison to the lower dose for the existing participants.

[0065] To evaluate the z score against the pre-determined efficacy probability threshold, the method includes generating an interim outcome prediction threshold at step 18. In this case, the interim outcome prediction threshold is an interim efficacy probability threshold. It is used in order to account for the completion proportion of the medical trial corresponding to the quantity of medical impact data used. That is, for the existing participants, as compared to the quantity of participant medical impact data that would correspond to use of all participant medical impact data for the medical trial if completed (existing and further participants). The interim efficacy probability threshold indicates an expected outcome for the given completion proportion of the medical trial if the pre-determined efficacy probability threshold is to be met with participant medical impact data for the complete medical trial. This approach is taken because as the proportion of the ultimately expected participant medical impact data actually used decreases, so its capacity to provide a given efficacy probability also decreases. This is due to the ever greater potential for participant medical impact data yet to be collected to undermine any trend previously suggested. Consequently, in order for the participant medical impact data of the existing participants to be considered “on track” for the medical trial to ultimately meet the pre-determined efficacy probability threshold, it need not reach the standard of the pre-determined efficacy probability threshold itself. Further, this relationship should be non-linear, with the uncertainty reducing more rapidly with additional participant medical impact data in the earlier stages than in the latter stages of the medical trial.

[0066] An example graph can be seen in Figure 2 showing traces (progressing as additional participant medical impact data is used) for the z score 19a between the distributions and for the interim efficacy probability threshold 19b.

[0067] In the present embodiment, the interim efficacy probability threshold is determined by multiplying a predetermined constant by the square root of a metric for the completion proportion of the medical trial (i.e. corresponding to the medical impact data for the existing participants as compared with the planned medical impact data for the combined existing and further participants). In this instance, the constant is 1.96, which corresponds to the p value of less than 0.05. The constant 1 .96 is approximately the value of a z-score required between a distribution of the medical impact data for the participants in the first arm and a distribution of the medical impact data for the participants in the second arm, such that the area under the distribution to the right of this value corresponds to 0.05. That is, the probability of the outcome prediction being incorrect.. The metric is a number between 0 and 1 proportional to the completion proportion. The interim efficacy probability threshold is thereby calculated as 1 .96 multiplied by the square root of the metric.

[0068] In order to calculate the interim efficacy probability threshold, the computer system acquires information indicating the completion proportion in a completion proportion acquisition step 20. As will be appreciated, in other embodiments, the computer system may alternatively infer the completion proportion in accordance with information it has acquired regarding the total number of planned participants and the quantity of participant medical impact data acquired in the data acquisition step 10.

[0069] In an outcome prediction step 22, the computer system compares the z score calculated in step 16 with the interim efficacy probability threshold generated in step 18. Where the z score is at or above the interim efficacy probability threshold, this indicates that the medical impact data for the existing participants suggests an expectation that the required pre-determined efficacy probability threshold would be met were the participant medical impact data for the complete medical trial to be used. That is, an outcome prediction that the p-value would be below 0.05 and so that the higher dose would be proved to be more effective than the lower dose at reducing tumour size in accordance with the medical trial design. Alternatively, where the z score is below the interim efficacy probability threshold, this indicates that the medical impact data for the existing participants suggests an expectation that the required pre-determined efficacy probability threshold would not be met were the participant medical impact data for the complete medical trial to be used. That is, an outcome prediction that the p-value would be at or above 0.05 and so that the higher dose would not be proved to be more effective than the lower dose at reducing tumour size in accordance with the medical trial design.

[0070] In a confounding distribution determination step 24, the computer system begins the process of determining a probability that the outcome prediction is proved wrong by the narrowest margin possible. Specifically, the computer system determines a distribution of medical impact data required for the further participants in order to have the minimum necessary effect to confound the outcome prediction.

[0071] The confounding distribution determination step 24 proceeds by calculating a z score representing the confounding distribution, the z score being that required between distributions of the medical impact data for the further participants in the first arm and the medical impact data for the further participants in the second arm in order to have the minimum necessary effect such that the outcome prediction is confounded. This is achieved by subtracting, as a function of time, the z score between distributions for the medical impact data for the existing participants in the first arm and the medical impact data for the existing participants in the second arm from a z score between distributions for participant medical impact data for the first arm and participant medical impact data for the second arm as if the medical trial were completed and the predetermined outcome threshold is satisfied by the narrowest of margins. Specifically, the relationship of equation 1 leads to equation 2 for the calculation of the z score of the confounding distribution where: Zois the z score between a distribution for the participant medical impact data for the first and second arms as if the medical trial is complete and the pre-determined outcome threshold is just satisfied, ZiiVeis the z score between distributions for the medical impact data for the existing participants in the first arm and the medical impact data for the existing participants in the second arm, ZfiiPis the z score representing the confounding distribution and t is metric for the completion proportion of the medical trial corresponding to the medical impact data for the existing participants. This may be the same metric as previously described for use in calculating the interim outcome prediction threshold.

[0072] 15 Equation 1 Equation 2

[0073] Figure 3 shows a graphical illustration including:

[0074] A distribution 26 corresponding to the z score between distributions for the medical impact data for the existing participants in the first arm and the medical impact data for the existing participants in the second arm;

[0075] A distribution 28 representing the z score between distributions of the medical impact data for the further participants in the first arm and the medical impact data forthe further participants in the second arm having the minimum necessary effect such that the outcome prediction is confounded;

[0076] The location 30 of the pre-determined efficacy probability threshold.

[0077] Determining the confounding distribution may inherently assume a ‘distribution shape’. One option is for this to conform to the shape of the distribution generated by the medical impact data for the existing participants. Another option is to select a predetermined distribution shape (e.g. corresponding to a Gaussian distribution).

[0078] In a counter prediction probability step 32, the computer system calculates the probability of the confounding distribution occurring given the medical impact data forthe existing participants. This is achieved by calculating an area under a curve for an overlap region of: a) the determined confounding distribution; and b) a distribution corresponding to the z score between the distribution of the medical impact data for the existing participants in the first arm and the distribution of the medical impact data for the existing participants in the second arm.

[0079] This area correspond to the probability that the confounding distribution.

[0080] This counter prediction probability corresponds directly to the confidence in the outcome prediction and is output by the computer system as at least part of the medical trial conclusion.

[0081] At least one or some of the steps 10, 12 and 20 may be considered to form a data gathering node 110 of a medical trial conclusion generating system 100 (see Figure 5). Additionally or alternatively, at least one of some of the steps 10, 12 and 20 may be collected by medical trial conclusion generating system 200 via an input means 210 thereof (see Figure 6).

[0082] At least one or some of the steps 14, 16, 18, 22, 24 and 26 may be considered to form computer implemented analysis node 120 of the medical trial conclusion generating system 100. ). Additionally or alternatively, at least one of some of the steps 14, 16, 18, 22, 24 and 26 may be performed by a processing means 220 of the medical trial conclusion generating system 200. The output of the counter prediction probability as at least part of the medical trial conclusion may be performed by an output means 230 of the medical trial conclusion generating system 200.

[0083] At this point the method may optionally include performing at least one of the following as part of the medical trial conclusion: i) Treating one or more patients in dependence on the confidence in the prediction; ii) Designing another medical trial in dependence on the confidence in the prediction; ill) Adapting the medical trial in dependence on the confidence in the prediction; iv) Halting the medical trial before its planned conclusion in dependence on the confidence in the prediction; v) Making a determination on the success or failure of the medical trial in dependence on the confidence in the prediction; vi) Determining whether or not to unblind the medical impact data of the existing participants at a given stage of the medical trial; vii) Determining whether or not to assess the medical trial at a given stage of the medical trial; viii) Labelling, optionally automatically, a medication for treatment of one or more patients in dependence on the confidence in the prediction. ix) Determine the timing and / or scale of production of a facilitating technology for the first intervention in dependence on the confidence in the prediction.

[0084] The performance of one or more such actions may be considered an implementation node 130 of the medical trial conclusion generating system 100.

[0085] A further alternative is that such actions are delayed until the confidence in the outcome prediction reaches a pre-determined level. Further, the completion proportion of the medical trial (i.e. the proportion of planned participants that have become existing participants rather than further participants) required in order to reach this confidence level can be predicted. This projection may be of assistance in being indicative of the completion proportion (and potentially time) at which a desired level of confidence will be reached in order to justify a particular action. If desired the projection may be made for a confidence level at which it is virtually impossible or actually impossible for participant medical impact data for the further participants to confound the predicted outcome. This may be useful in being indicative of the completion proportion (and potentially time) at which (among other possible actions) it would be appropriate to discontinue the medical trial, its continuation being futile given that the outcome is effectively known.

[0086] The projection is possible because the counter prediction probability is a function of information time (i.e. the completion proportion of the medical trial). Specifically, as more participant medical impact data is gathered / used the counter prediction probability will tend to fall in value. Consequently, it is possible to fit an empirical (or theoretical) model to the counter prediction probability, thereby extrapolating the previously calculated values for the counter prediction probability (and thereby the confidence in the outcome prediction) into the future.

[0087] To produce the projection, the method previously described is repeated (potentially multiple times e.g. as every new piece of participant medical impact data arrives). The resulting counter prediction probabilities can then be extrapolated as described. The computer system can also output the completion proportion where the resulting model trace coincides with a desired probability, thereby indicating the expected completion proportion of the medical trial at which the desired confidence in the outcome prediction will be reached.

[0088] Figure 4 shows a graphical illustration of a projected counter prediction probability 40 extrapolated from the counter prediction probability 42 determined through information time according to the method. Also shown is a pre-determined desired counter prediction probability 44. The coincidence 46 in information time of the projected counter prediction probability 40 with the pre-determined desired counter prediction probability 44 is highlighted at 46. Also highlighted is the coincidence 48 in information time of the projected counter prediction probability 40 with the start in information time of a collapse in the projected counter prediction probability 40 towards 0. The start of this collapse may be referred to as a ‘knee’ or discontinuity. It reflects a completion proportion of the medical trial at which the remaining participant medical impact data (i.e. of the few further participants remaining) is virtually or completely insufficient to allow for confounding of the outcome prediction. In many instances, projecting the completion proportion at which the start of this collapse occurs may be advantageous, as it may be considered a point at which discontinuing the medical trial is most clearly justified.

[0089] As will be appreciated, just as the outcome prediction and confidence in the outcome prediction may be updated (optionally continuously / in real-time as new participant medical impact data is made available) so may be the projection and / or any changes in the expected completion proportion of the medical trial at which the desired confidence in the outcome prediction will be reached.

[0090] As will be appreciated, the projection and / or output of expected completion proportion of the medical trial at which the desired confidence in the outcome prediction will be reached, may be performed regardless of whether or not there is any intention to take action and / or consider taking any action in dependence thereon.

[0091] It will be appreciated that various changes and modifications can be made to the present invention without departing from the scope of the present application.

Claims

CLAIMS1 . A method for generating a medical trial conclusion for a medical trial of a participant intervention, the medical trial conclusion depending on an outcome prediction of the medical trial, where: the method uses medical impact data for existing participants in the medical trial, the medical impact data for the existing participants corresponding to the medical trial in incomplete form, medical impact data for further participants in the medical trial being required for correspondence with the medical trial in complete form; and the outcome prediction indicates an expected outcome of the medical trial in terms of the performance of the participant intervention, the expected outcome depending on the performance of the participant intervention for the existing participants when compared to an expected performance for the existing participants in order that a pre-determined outcome threshold would be equalled with the inclusion of medical impact data for the further participants corresponding with the medical trial in complete form and where the expected performance accounts for a completion proportion of the medical trial corresponding to the medical impact data for the existing participants the method comprising: given the medical impact data for the existing participants, determining a counter prediction probability which is the probability of the medical impact data for the further medical trial participants being adverse to the outcome prediction to an extent minimally sufficient for the outcome prediction to be confounded; generating, as at least part of the medical trial conclusion, a confidence in the outcome prediction in accordance with the determined counter prediction probability.

2. The method according to claim 1 comprising performing at least one of the following as part of the generated medical trial conclusion: i) Treating one or more patients in dependence on the confidence in the prediction; ii) Designing another medical trial in dependence on the confidence in the prediction; iii) Adapting the medical trial in dependence on the confidence in the prediction; iv) Halting the medical trial before its planned conclusion in dependence on the confidence in the prediction; v) Making a determination on the success or failure of the medical trial in dependence on the confidence in the prediction; vi) Determining whether or not to unblind the medical impact data of the existing participants at a given stage of the medical trial; vii) Determining whether or not to assess the medical trial at a given stage of the medical trial; viii) Labelling, optionally automatically, a medication for treatment of one or more patients in dependence on the confidence in the prediction; ix) Determine the timing and / or scale of production of a facilitating technology for the participant intervention in dependence on the confidence in the prediction.

3. The method according to claim 1 or claim 2 comprising determining the outcome prediction by: comparing the medical impact data for the existing participants that are in a first arm of the medical trial with medical impact data for the existing participants that are in a second arm of the medical trial, where the participants in the first arm receive the participant intervention and the participants in the second arm receive a comparative approach in accordance with the medical trial, the comparison thereby giving a performance of the participant intervention as compared with the comparative approach for the existing participants; calculating an interim outcome prediction threshold indicating an expected outcome for the completion proportion of the medical trial corresponding to the medical impact data for the existing participants, in order that the pre-determined outcome threshold would be equalled with the inclusion of medical impact data for the further participants corresponding with the medical trial in complete form; comparing the performance of the participant intervention as compared with the comparative approach for the existing participants with the interim outcome threshold to determine any discrepancy between the two to thereby generate the outcome prediction.

4. The method according to claim 3 where the comparing of the medical impact data for the existing participants in the first arm of the medical trial and the medical impact data for existing participants in the second arm of the medical trial is performed by calculating the z score between a distribution of the medical impact data for the existing participants in the first arm and a distribution of the medical impact data for the existing participants in the second arm.

5. The method according to claims 4 where determining the counter prediction probability comprises: determining a confounding distribution required for the medical impact data for the further medical trial participants in order to have the minimum necessary effect such that the outcome prediction is confounded; and calculating the probability of the confounding distribution occurring given the medical impact data for the existing participants.

6. The method according to claim 5 where the determining of the confounding distribution is performed by calculating a z score representing the confounding distribution, the z score being that required between distributions of the medical impact data for the further participants in the first arm and the medical impact data for the further participants in the second arm in order to have the minimum necessary effect such that the outcome prediction is confounded.

7. The method according to claim 5 or claim 6 where the calculating of the counter prediction probability is performed by calculating an area under a curve for an overlap region of: a. the determined confounding distribution; and b. a distribution corresponding to the z score between the distribution of the medical impact data for the existing participants in the first arm and the distribution of the medical impact data for the existing participants in the second arm.

8. The method according to any of claims 3 to 7 where calculating the interim outcome prediction threshold is performed by multiplying a pre-determined constant by the square root of a metric for the completion proportion of the medical trial corresponding to the medical impact data for the existing participants.

9. The method according to any preceding claim comprising repeating the method.

10. The method according to claim 9 where the medical trial conclusion comprises a projection for the confidence in the outcome prediction expected at one or more completion proportions of the medical trial corresponding to additional medical impact data of the further participants having been used by extrapolating the calculated values for the confidence in the outcome prediction calculated in accordance with the method.11 . The method according to any of claims 3 to 10 where the existing and further participants are blind or double blind randomised between the first and second arms and the method comprises unblinding of the participant medical impact data for performance of the method and where further the method steps performed to generate the confidence in the outcome prediction including the unblinding are performed by a computer system without allowing human access to the unblinded participant medical impact data.

12. A computer program that, when read by a computer, causes performance of the method of any of claims 1 to 11.

13. A non-transitory computer readable storage medium comprising computer readable instructions that, when read by a computer, cause performance of the method of any of claims 1 to 11.

14. A signal comprising computer readable instructions that, when read by a computer, cause performance of the method of any of claims 1 to 11.

15. A medical trial conclusion generating system arranged to generate a medical trial conclusion for a medical trial of a participant intervention, the medical trial conclusion depending on an outcome prediction of the medical trial, where: the system uses medical impact data for existing participants in the medical trial, the medical impact data for the existing participants corresponding to the medical trial in incomplete form, medical impact data for further participants in the medical trial being required for correspondence with the medical trial in complete form; andthe outcome prediction indicates an expected outcome of the medical trial in terms of the performance of the participant intervention, the expected outcome depending on the performance of the participant intervention for the existing participants when compared to an expected performance for the existing participants in order that a pre-determined outcome threshold would be equalled with the inclusion of medical impact data for the further participants corresponding with the medical trial in complete form and where the expected performance accounts for a completion proportion of the medical trial corresponding to the medical impact data for the existing participants, the system comprising: an input means arranged to receive the medical impact data for the existing participants in the medical trial; a processing means arranged to: analyse the medical impact data for the existing participants in the medical trial to determine a counter prediction probability which is the probability of the medical impact data for the further medical trial participants being adverse to the outcome prediction to an extent minimally sufficient for the outcome prediction to be confounded; and generate, as at least part ofthe medical trial conclusion, a confidence in the outcome prediction in accordance with the determined counter prediction probability; and an output means arranged to output the confidence in the outcome prediction.

16. A medical trial analysis system arranged to generate a medical trial conclusion for a medical trial of a participant intervention, the medical trial conclusion depending on an outcome prediction of the medical trial, where: the system uses medical impact data for existing participants in the medical trial, the medical impact data for the existing participants corresponding to the medical trial in incomplete form, medical impact data for further participants in the medical trial being required for correspondence with the medical trial in complete form; and the outcome prediction indicates an expected outcome of the medical trial in terms of the performance of the participant intervention, the expected outcome depending on the performance of the participant intervention for the existing participants when compared to an expected performance for the existing participants in order that a pre-determined outcome threshold would be equalled with the inclusion of medical impact data for the further participants corresponding with the medical trial in complete form and where the expected performance accounts for a completion proportion of the medical trial corresponding to the medical impact data for the existing participants, the system comprising: a data gathering node comprising collecting the medical impact data for the existing participants in the medical trial; a computer implemented analysis node comprising:analysing the medical impact data for the existing participants in the medical trial to determine a counter prediction probability which is the probability of the medical impact data for the further medical trial participants being adverse to the outcome prediction to an extent minimally sufficient for the outcome prediction to be confounded; and generating, as at least part of the medical trial conclusion, a confidence in the outcome prediction in accordance with the determined counter prediction probability; and an implementation node comprising at least one of: i) Treating one or more patients in dependence on the confidence in the prediction; ii) Designing another medical trial in dependence on the confidence in the prediction; iii) Adapting the medical trial in dependence on the confidence in the prediction; iv) Halting the medical trial before its planned conclusion in dependence on the confidence in the prediction; v) Making a determination on the success or failure of the medical trial in dependence on the confidence in the prediction; vi) Determining whether or not to unblind the medical impact data of the existing participants at a given stage of the medical trial; vii) Determining whether or not to assess the medical trial at a given stage of the medical trial; viii) Labelling, optionally automatically, a medication for treatment of one or more patients in dependence on the confidence in the prediction; ix) Determine the timing and / or scale of production of a facilitating technology for the participant intervention in dependence on the confidence in the prediction.

Images