Method for assessing an automatically generated report of a medical procedure

Abstract

The invention relates to a method for assessing a report of a medical procedure, including the following steps: (E2) providing a set of information (D, CRMi) relating to the medical procedure; (E3) generating a report (CRSj), by means of a machine-learning algorithm (MLj) trained to generate a report using a training set (DSj) including sets of information (CEk) associated with predetermined reports (CSk); (E4) estimating a first proximity value (V1) between the set of information provided and at least one set of information of the training set; (E5) estimating a second proximity value (V2,j) between the generated report and at least one predetermined report of the training set; (E6) estimating a confidence index (Ij) on the basis of the first and second proximity values; (E8) transmitting the generated report and the confidence index to a computer terminal (2).

Description

Description Title of the invention: Method for evaluating an automatically generated medical report

[0001] technical field

[0002] The invention relates to the technical field of health, and more specifically to the production of medical reports using machine learning algorithms.

[0003] State of the art

[0004] Typically, during a patient's care pathway, one or more medical examinations are prescribed, such as a consultation with a specialist, a medical examination involving imaging, or a laboratory test. The practitioner conducting the examination generally writes a report or summary of the medical examination, possibly including observations or diagnostic conclusions. This report is then transmitted, either electronically via the patient's medical record or directly from the patient, to the next practitioner in the care pathway and / or to the physician coordinating the care pathway.

[0005] While this method does allow the various medical professionals to access the results of previous examinations and to effectively monitor the patient throughout the care pathway, it has the disadvantage of excluding the patient from the therapeutic strategy of the care pathway.

[0006] Indeed, a medical report generally contains complex medical terms that the patient may not be familiar with. Furthermore, the answers and conclusions provided by the practitioner in the report are often difficult for the patient to interpret, for example, because the practitioner uses clinical terms or acronyms, or because they refer to values ​​without providing an explanatory framework.

[0007] The patient may then lose confidence in the proposed treatment strategy, reducing their adherence to and participation in the care pathway, which can lead to treatment failure. Therefore, there is a need to simplify medical reports to better suit the patient's level of understanding.

[0008] In this context, machine learning algorithms, or artificial intelligence (AI), and in particular large language models (LLMs), have shown significant potential in simplifying or translating complex texts. They can be trained using a corpus of texts representing various medical reports, labeled by physicians, learned societies, or patient associations, to automatically generate a simplified report from a medical report written by a practitioner, which the practitioner can then distribute to the patient. This eliminates the need for the practitioner to write the report themselves, thus reducing their workload.

[0009] Despite the progress made in these algorithms, it remains difficult to produce consistently satisfactory results. One of the problems is the variability of medical reports. The texts provided to the algorithms may differ considerably from those in the training corpus. Furthermore, this training corpus may be incomplete or unbalanced, leading to a loss of information in the simplified report. Finally, LLMs (Low-Level Mental Health) can produce hallucinations, which may compromise the reliability of the simplified report. This results in a need for a high level of trust from practitioners in the reliability and relevance of the simplified reports generated by the algorithms, in order to distribute them to patients. This need is further complicated by the initial problem: practitioners do not have the time to verify each AI-generated simplified report to meet this requirement for a high level of trust.

[0010] These needs exist more generally in a context of automatic generation of medical reports from information relating to a medical act, whether it is an expert medical report or data from the patient's file, and whether the automatically generated report is a simplified report or an expert report.

[0011] Thus, there is a need for a process that automatically generates a report from a set of information relating to a medical procedure performed for a patient, and that allows the practitioner to quickly know whether to check this report or, on the contrary, whether to distribute it to the patient without verification.

[0012] The invention therefore falls within this context and seeks to resolve all the aforementioned drawbacks by addressing this need.

[0013] Presentation of the invention

[0014] The invention thus relates to a method for evaluating a report of a medical procedure performed by a healthcare professional for a patient, the report being generated automatically, the method being implemented by a computer system and comprising the following steps: a. providing a set of information relating to the medical procedure to the computer system; b. generating, from said set of information provided, a report, by means of at least one machine learning algorithm trained to generate a report of a medical procedure from a set of information relating to that medical procedure, said at least one algorithm having been trained using a training set comprising a plurality of information sets, each relating to a given medical procedure, and a plurality of predetermined reports each associated with one of said information sets; c.d. estimation of a first proximity value between said provided information set and at least one information set from the training set of said machine learning algorithm; d. estimation of a second proximity value between the report generated by the machine learning algorithm from said provided information set and the. less a predetermined report from the training set of said machine learning algorithm; e. estimation of a confidence index associated with the report generated by the machine learning algorithm from the first and second proximity values; f. transmission of at least the report generated by the machine learning algorithm and the confidence index to a computer terminal

[0015] The invention thus proposes that a set of information relating to a medical procedure performed by a healthcare professional on a patient be transmitted to a computer system, where it is automatically converted into a medical report. This set of information may be a medical report, referred to as an "expert" report, routinely written by the healthcare professional during or after the medical procedure, or a set of raw, processed, or enriched information relating to the medical procedure. In the invention, the conversion is performed by one or more machine learning algorithms implemented by the computer system and previously trained to generate a medical procedure report from a set of information relating to that procedure.This could be a simplified version of the expert report, or a simplified or expert report generated from data from the medical procedure and / or data from the patient's medical file.

[0016] Following this conversion, the computer system can then estimate two so-called proximity values. One indicates the proximity of the incident data set to the data sets in the training corpus, while the second indicates the proximity between the report inferred by the algorithm(s) and the reports in the training corpus that label the data sets in the training corpus. It is thus possible to calculate a confidence index by combining these proximity values, indicating to the practitioner whether the incident data set is close to the data sets in the training dataset, and whether the report generated by the algorithm(s) is also close to one or more versions labeling these data sets.

[0017] The practitioner can thus, based on a threshold they have set and / or that has been set by an association, such as a learned society or other professional organization, determine whether the generated text warrants revision or whether they can trust it and distribute it to the patient without review. The invention therefore reduces the practitioner's intervention time while increasing their level of confidence in the automatic report generation tool. Furthermore, the confidence rating can also be communicated to the patient to indicate whether they can trust the generated text or, conversely, encourage them to maintain a critical perspective on it.

[0018] Definitions

[0019] In the present invention, the term "computer system" means any set of Interconnected hardware and software components designed to process, store, and manipulate digital data in order to implement the process according to the invention. A computer system may include one or more processing devices, storage means, input / output interfaces, applications, programs or software, and communication means. The computer system may operate autonomously or as a network, and may be physically located on a single site or geographically distributed.

[0020] The computer system may include, but is not limited to, a computer terminal such as a desktop computer, a laptop computer, a server, a tablet computer, a smartphone, a cell phone and / or equipment of a computer network; a local computer network comprising several computer terminals and one or more servers interconnected to the terminals; or a "cloud" type computer infrastructure comprising computer servers accessible by means of wireless communication and providing services to a remote computer terminal, in particular of the SaaS (Software As A Service) type.

[0021] Where appropriate, one or more computer applications, computer programs or software, enabling the implementation of the process according to the invention, may be installed directly on a computer terminal that can be operated by the practitioner, and / or installed on a server on a local network so that they can be accessed by a computer terminal on that local network, or hosted on a computer server from which they are accessible through software installed on a computer terminal, such as an internet browser or an application programming interface.

[0022] Preferably, in the case of a local area network or "cloud" infrastructure computer system, each computer terminal of the computer system is equipped with a communication unit capable of exchanging data with a communication unit of a processing server capable of implementing all or part of the process according to the invention, in particular according to one or more wireless communication protocols.

[0023] If desired, the computer system can be connected to, or be part of, one or more hospital computer systems to receive data relating to the medical procedure via other elements of that system. This computer system could, for example, be a radiology information system (RIS), a laboratory management system (LMS), a results server, and / or a picture archiving and transmission system (PACS). Where applicable, the method according to the invention can be implemented using software programs or applications, some of which can be installed on a computer terminal that can be operated by the practitioner, particularly for the purpose of collecting or generating said set of information relating to the medical procedure.This set will thus be transmitted, once generated or collected on the computer terminal, to a processing server, on which is installed another part of the programs or software applications enabling the implementation of the process according to the invention.

[0024] Alternatively, the computer system may include one or more remote servers connected to a computer terminal that can be operated by the practitioner. The services of said processing server(s) may then be accessible via this computer terminal using cloud computing methods. If necessary, all the programs, software, or applications required to implement the method according to the invention may be installed on the processing server(s), either centrally or in a distributed manner. All or part of these programs, software, or applications may be accessed from the computer terminal via wireless communication protocols, such as through an internet browser and HTTPS and IP protocols, or through an application programming interface (API). The information related to the medical procedure will thus be collected or generated directly on the remote server.

[0025] In this case, it may be provided that the process according to the invention includes a prior step of identification and / or authentication of the person and / or application in charge of the collection or generation of said set of information, in particular through a strong authentication process implemented by the processing server or another server dedicated to this process.

[0026] In the present invention, the term "medical procedure information set" means any collection of data, structured or unstructured, relating to a medical procedure performed on or for a patient. This information may be generated manually and / or automatically following various types of medical interventions, by one or more healthcare professionals and / or by one or more computer programs, all at once or sequentially, and is not limited to a particular medical field.

[0027] This set of data may include, but is not limited to, an expert medical report written by a healthcare professional, raw data from medical examinations, laboratory test results, medical images and their interpretations, clinical notes, patient observations, physiological measurements, relevant medical history, prescription data, ongoing monitoring data, or any combination thereof. For example, the data set may include a detailed account of a medical consultation, or a data set including laboratory test results, necessary for preparing a medical report.

[0028] The said set may be enriched with contextual data or information, not necessarily related to the medical act, such as information relating to a previous medical act, external data from reference medical databases, epidemiological registers, population data, clinical, sociological, epidemiological, geographical or environmental context information.

[0029] In the present invention, "expert report" means a written or electronic document that records essential information relating to a medical procedure performed by a healthcare professional for a patient. This document can be generated following of various types of medical interventions and is not limited to a particular medical field.

[0030] An expert report may include, but is not limited to, the following: patient identification information; date and nature of the medical procedure; reason for consultation or intervention; relevant medical history; symptoms reported by the patient; clinical observations of the practitioner; results of physical examinations; details of procedures performed; results of additional examinations, including imaging or laboratory tests; diagnosis established or diagnostic hypotheses; treatment administered or prescribed; prognosis; follow-up plan or future recommendations; any samples taken; or any other type of information provided to the patient.

[0031] The expert report can be generated by various methods, including: manual writing by a healthcare professional, the handwritten document possibly being digitized and provided to an optical character recognition (OCR) solution; entry by a healthcare professional via word processing software on a computer terminal or via a dedicated application, local or accessible via a SaaS service; dictation by a healthcare professional and transcribed via specific software such as "speech recognition"; generation assisted by one or more machine learning algorithms from a prompt provided by a healthcare professional; automatic generation by one or more machine learning algorithms from recordings and / or results of the medical procedure.

[0032] The expert report can be transmitted to the computer server, for example, via a wireless communication protocol, or directly generated using software on a processing server accessible from a computer terminal operated by the healthcare professional or installed directly on that terminal. The initial report is thus available and accessible from other computer terminals authorized to connect to the processing server, for example, to allow another healthcare professional, such as another practitioner, the attending physician, or other stakeholders in the care pathway, to review the report.

[0033] A medical procedure may be performed during a general or specialist medical consultation or teleconsultation, or for investigative or diagnostic purposes. It may include taking a medical history, performing an inspection, palpating, auscultating, taking a sample, conducting a psychotherapy session, performing an examination involving medical imaging, or analyzing a sample and / or a medical image. A medical procedure may also be performed for therapeutic purposes, including an oral consultation to provide therapeutic advice, administering medication, or performing a surgical, radiological, or physical intervention. All or part of the medical procedure may be performed by general practitioners or physicians of various medical specialties, including surgeons, psychologists, cardiologists, oncologists, and radiologists; by midwives, dentists, nurses, medical secretaries, pharmacists, and other healthcare professionals. physiotherapists, medical biologists, laboratory technicians, nursing assistants, ambulance drivers.

[0034] In the present invention, a "simplified report" means a version of the expert report that has been modified to be more easily understood by non-experts, particularly patients. The simplification of the expert report may include: replacing complex medical or technical terms with more common synonyms; adding detailed explanations of medical terms and procedures; removing or summarizing non-essential or overly detailed information; and reorganizing the report structure to highlight information considered essential for the patient, such as test results, diagnoses, and treatment recommendations.

[0035] It is possible to configure the simplification of the expert report by the healthcare professional or the patient, particularly to adapt the simplified report to the patient's level of understanding. If necessary, the expert report delivery step may include a sub-step for providing conversion parameters, notably through an input interface, which will be supplied to the machine learning algorithm to generate the second simplified report. For example, a query or prompt, perhaps pre-written, could be provided to the machine learning algorithm. This query would request, based on the presence of a given section in the expert report, the generation of a simplified section of the same type in the simplified report.

[0036] In the present invention, a "machine learning algorithm," also called an artificial intelligence (AI) model or a machine learning system, is defined as a set of mathematical instructions and rules designed to analyze data, identify patterns, and make decisions or predictions with minimal human intervention. These algorithms are characterized by their ability to improve iteratively through a training process, during which they adjust their internal parameters based on input data and expected results to optimize their performance on a specific task.

[0037] In the context of the present invention, the report may be generated using one or more machine learning algorithms, implemented in parallel or sequentially, to which the aforementioned set of information relating to the medical procedure is provided as input. By way of non-limiting example, this or these machine learning algorithms may be chosen from: artificial neural networks, deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), large language models (LLMs), support vector machines (SVMs), random forests, or any appropriate combination of these models.

[0038] Preferably, the report can be generated using one or more algorithms. large language model machine learning, including a predetermined generative transformer (GPT), which can be a semantic auto-regression algorithm (GPT-3, GPT-4, Mistral Large or Small, LLaMa, LLaMa 2, Gemini, Claude), a transformer or only part of a transformer, such as an encoder (BERT).

[0039] The machine learning algorithm(s) can be trained using supervised training methods with the training set, including methods such as few-shot learning, fine-tuning, and transfer learning. Alternatively, other training methods can be used, or several training methods can be combined, including semi-supervised or even unsupervised training.

[0040] In one example of an embodiment of the invention, the machine learning algorithm(s) may have been previously trained to convert an expert report into a simplified report, from a training set comprising expert reports each associated with a simplified report.

[0041] Preferably, and particularly in the case of a Learning Life Cycle (LLC), the training set should include expert reports written by healthcare professionals. Each of these expert reports is then labeled by an expert, such as a physician or a member of a patient association, with a simplified report. The machine learning algorithm(s) are thus trained to iteratively generate simplified reports from the expert reports in the training set and to evaluate an error metric between the generated simplified report and the simplified report labeling the expert report. At each iteration, the machine learning algorithm(s) adjust their internal parameters, such as weights and neuronal biases, to minimize this error metric over the course of the iterations.

[0042] The training can be implemented using all or part of the expert and / or simplified reports from the training set. These reports can be divided into several blocks or sections, such as a title, patient information, reasons for consultation, clinical context, medical history, clinical examination, diagnosis, treatment plan, clinical recommendations, and a conclusion. If necessary, the training can be implemented using a query or prompt, possibly pre-written, which, depending on the presence of a given section, requests the generation of a simplified section of the same type. All the sections thus generated are then combined to form the simplified report.

[0043] In the present invention, "a proximity value" means a quantitative measure representing the degree of similarity or resemblance between two sets of information.

[0044] In cases where the information sets include text, this proximity value can, for example, be calculated using one or more text quality assessment metrics. These metrics could, for instance, allow for the evaluation of the error, the gap, the similarity or the correspondence between two texts, as used in the training methods of an LLM. This proximity value can thus evaluate the correspondence of n-grams, the precision, the recall, or the semantic similarity based on contextual representations, or any other measure of the distance between the vector representations of the texts, in particular in a high-dimensional semantic space, such as the Euclidean distance, the cosine similarity, the Levenshtein distance or the Manhattan distance.

[0045] For other types of information, said proximity value may be calculated from one or more statistical metrics, such as a Pearson correlation, a mean squared deviation, or any other metric capable of assessing the error, deviation, similarity or correspondence between two data sequences or two time series.

[0046] Methods of implementation

[0047] In one embodiment of the invention, the first proximity value estimation step includes a substep for estimating a proximity index between the provided information set and each information set in a plurality of information sets in the training set, the first proximity value being calculated from these proximity indices. According to these characteristics, the first proximity value thus indicates the similarity between the information set provided as input to the machine learning algorithm and some, or even all, of the information sets in the training set of that algorithm, such that the confidence index reflects the proximity of the incident information set to the information sets used to train the machine learning algorithm(s).The healthcare professional thus has, through the confidence index, information enabling them to gauge whether they should review the generated report because of a significant divergence with the training set or whether they can instead distribute the generated report to the patient, or even to other healthcare professionals, or other medical computer systems, such as a patient medical record management system or DMP.

[0048] Advantageously, the training dataset can be segmented into different sets of information, each labeled with a predetermined report. Each set is associated with a medical type, such as a medical specialty or pathology. If necessary, a proximity index can be estimated for each information set within the training dataset associated with the medical type corresponding to the provided information set. This medical type can be pre-selected by the healthcare professional who wrote or collected the provided information set, or it can be automatically inferred from the provided information set. This allows the machine learning algorithm to be trained and the confidence index calculated for a specific medical type.

[0049] Alternatively, a proximity index can be estimated for each set of information in the training set.

[0050] Advantageously, when the provided information set is an initial, or expert, report, and the machine learning algorithm has been trained to convert, from a training dataset containing expert reports each paired with a simplified report, an expert report into a simplified report, each proximity index is calculated according to at least one initial metric of semantic distance between two texts. This type of metric makes it possible to assess the similarity between two texts beyond simple lexical correspondences, particularly through metrics for evaluating machine translation or automatic summarization / synthesis.

[0051] Preferably, said semantic distance metric is chosen from one of the following translation or simplification evaluation metrics: BLUE, RED, METEOR, BERTScore, BLEURT, COMET, cosine similarity, Fl, SARI, BARTScore.

[0052] The BLEU metric (Bilingual Evaluation Understudy) is a metric for evaluating the quality of machine translation, measuring unigram, bigram, and even n-gram accuracy. The ROUGE metric (Recall-Oriented Understudy for Gisting Evaluation) is a metric for evaluating the quality of machine summarization, measuring both accuracy and recall of unigrams, bigrams, and even n-grams. The METEOR metric (Metric for Evaluation of Translation with Explicit Ordering) is a metric for evaluating the quality of machine translation, taking into account synonyms, stemming, and word order. The BERTScore metric measures the similarity of two texts by comparing their vector representations.The BLEURT (Bilingual Evaluation Understudy with Representations from Transformers) and COMET (Cross-lingual Optimized Metric for Evaluation of Translation) metrics are used to evaluate the quality of machine translation, based on a pre-trained model that takes into account meaning and semantic similarity. The cosine similarity metric evaluates the cosine between two vector representations of two texts. The SARI (System Output Against References and Against Inputs) metric evaluates the quality of a simplified text, based on the additions, deletions, and modifications introduced by the simplification process.The Fl metric is a measure of model accuracy that indicates the harmonic mean of accuracy and recall, thus taking into account both the proportion of correct positive predictions among all the model's positive predictions and the proportion of positive predictions that are actually positive. BARTScore is a text generation assessment measure that treats model evaluation as a text generation task, leveraging BART's pre-trained semantic embeddings (or word embeddings) to return a score that measures the reliability, accuracy, recall, or F-score response of the main text generation model.

[0053] It will be possible to combine several semantic distance metrics and to employ other text comparison metrics, used in translation, simplification, or text summarization contexts, without departing from the scope of the present invention. It should be noted that combining several metrics can offer a more robust evaluation, for example by identifying both synonyms and paraphrases.

[0054] In another example, where the said information set provided is a dataset from a clinical examination or laboratory analysis; and the machine learning algorithm has been trained to generate a medical report from a dataset from a clinical examination or laboratory analysis, from a training set comprising datasets from a clinical examination or laboratory analysis each labeled with a medical report, each proximity index is calculated according to different metrics adapted to the types, categories, and natures of the data in said datasets.

[0055] For example, in the case where the data sets contain qualitative or categorical data, such as a positive or negative result of a test, a classification or a grading on a given scale, each proximity index can be calculated from the Hamming distance between these qualitative or categorical data.

[0056] For example, in the case where the data sets contain quantitative data, such as a time sequence or numerical vectors, each proximity index can be calculated from a cosine similarity between these quantitative data.

[0057] For example, in the case where the datasets contain text-type data, such as a medical history, medical background, a description of symptoms, each proximity index can be calculated from a Levenshtein distance between these text-type data.

[0058] In one embodiment of the invention, the first proximity value is the maximum value of the proximity indices. This example allows for the rapid identification of the training dataset that most closely resembles the incident dataset. Alternatively, the first proximity value could be the average of the N highest proximity index values. In this example, the first proximity value reflects the closeness of the incident dataset to the entire training dataset, making these characteristics particularly useful in fields where the variability of the datasets is significant.

[0059] In one embodiment of the invention, the step of estimating the first proximity value may be implemented prior to the step of generating the report from the provided information set using the machine learning algorithm(s). If applicable, if said first proximity value is a value indicating an exact match between the provided information set and one of the information sets in the training set, the steps of generating the report from the provided information set using the machine learning algorithm(s) and estimating the second proximity value are not The implementation and the process include a step of selecting the training set report associated with the training set information that exactly matches the provided information set. This selected report is then provided to the computer terminal with an indication of exact correspondence.

[0060] In this example, we thus avoid using the machine learning algorithm(s) in the case where the set of information provided already exists in the training set.

[0061] Alternatively, it may be possible to implement the steps of estimating the first and second proximity values ​​after the generation of the report by the machine learning algorithm(s).

[0062] In one embodiment of the invention, the first proximity value corresponds to a given set of information from the training set, and the second proximity value is an estimated proximity index between the report generated by the machine learning algorithm and a report from the training set associated with said given set of information from the training set corresponding to the first proximity value. According to these characteristics, the second proximity value thus indicates the similarity between the automatically generated report and the training set report labeling the training set of information closest to the incident set of information.Therefore, the confidence index combining these two values ​​reflects both the similarity of the incident data set to the data sets used to train the machine learning algorithm(s) and the relevance of the automatically generated report to the training dataset. This further increases the healthcare professional's confidence and speed in judging whether to review the automatically generated report or whether it can instead be shared with the patient, other healthcare professionals, or other medical information systems.

[0063] Advantageously, the said proximity index is calculated according to at least a second metric of semantic distance between two texts.

[0064] Preferably, when the provided information set is an initial, or expert, report, and the machine learning algorithm has been trained to convert an expert report into a simplified report from a training set containing expert reports each associated with a simplified report, the second semantic distance metric is distinct from the first semantic distance metric. It can thus be assumed that the first proximity value reflects the similarity between the incidental expert report and the training set, while the second proximity value indicates the relevance of the transformation applied to this incidental expert report. If appropriate, the first distance metric could be a measure of semantic similarity, such as RED, while the second distance metric could be a measure evaluating the quality of a text simplification, such as SARI.

[0065] Alternatively, the second semantic distance metric could be the same metric as the first semantic distance metric.

[0066] In one embodiment of the invention, the step of estimating a confidence index associated with the report generated by the machine learning algorithm includes a substep of estimating a penalty from one of the first and second proximity values, the confidence index corresponding to the other of said proximity values ​​penalized by said penalty.

[0067] Introducing a penalty targeting one or the other of the proximity values ​​in the calculation of the confidence index allows for the introduction of tolerance or, on the contrary, a sanction in the discrepancies between the incident information set and the inferred report on the one hand and the training set on the other hand, depending on the sensitivity of the healthcare professional with regard to machine learning algorithms.

[0068] Advantageously, the penalty can be calculated from the first proximity value, for example, the first proximity value raised to a given power such as 2 or higher. Alternatively, or cumulatively, the penalty can be estimated by a combination of several first values ​​defined according to several metrics, such as RED-1 and BLUE-1.

[0069] Advantageously, the confidence index can be estimated by multiplying the penalty by the second proximity value.

[0070] Alternatively, the penalty could be applied to the second proximity value, in order to penalize errors in generating reports, or to the first and second proximity values.

[0071] In one alternative or cumulative embodiment of the invention, the confidence index can be determined from the first and second proximity values ​​and a weight determined from the machine learning algorithm used to generate the report and / or the training set. For example, the weight can be determined from the type and / or number of hyperparameters of the LLM used to generate the report and / or the number of reports present in the training set. This weight allows the confidence index to be adjusted according to the reliability of the algorithm used and its training.

[0072] In one embodiment of the invention, the method includes, prior to the transmission step, a step for comparing the confidence index to a predetermined threshold value, the transmission step being conditional upon the result of said comparison. The results are thus filtered so as to transmit only the reports generated by the machine learning algorithm(s) that are likely to be disseminated by the healthcare professional.

[0073] For example, the report will only be transmitted if the confidence index is greater than the said threshold value, for example set at 0.5.

[0074] In one embodiment of the invention, the generation step is a generation step, from said provided information set, of a plurality of reports, each being generated by means of a separate machine learning algorithm trained to generate a report of a medical act from a set of information relating to that medical act, each algorithm having been trained using a training set comprising a plurality of information sets, each relating to a given medical act, and a plurality of predetermined reports each associated with one of said information sets; and a second proximity value and a confidence index are estimated for each of said reports generated by the machine learning algorithms.

[0075] It can be anticipated that each of the machine learning algorithms will be trained using the same training set or a dedicated training set. If so, each proximity value will be estimated from the dedicated training set.

[0076] Advantageously, it can be stipulated that only the report associated with the highest confidence level is transmitted during the transmission stage. In this example, only the best result is submitted, in order to expedite the healthcare professional's decision-making and increase their level of confidence.

[0077] Alternatively, it could be stipulated that the N reports associated with the highest confidence scores are transmitted during the transmission step, or even that all reports whose confidence score exceeds the aforementioned threshold value are transmitted during the transmission step. In this alternative, the healthcare professional has the choice of which reports they wish to review and / or share with the patient, or even with other healthcare professionals or other medical information systems.

[0078] Alternatively, each machine learning algorithm could be trained to convert an expert report into a simplified report according to a given level of understanding, which can be pre-established, selected, or indicated by the healthcare professional or patient via a selection interface or prompt. In this variant, the process generates multiple reports of the same medical procedure at different levels of understanding, allowing the healthcare professional to select the simplified report they deem most relevant based on the patient's understanding and their care pathway.

[0079] In one embodiment of the invention, the method includes an evaluation step, on said computer terminal, of the report generated by the machine learning algorithm, and a transmission step, to the computer system, of said evaluation. This evaluation step allows for the implementation of reinforcement training of the machine learning algorithm(s) using human feedback (HFF). Reports rejected by the healthcare professional or patient can thus be penalized in a subsequent retraining step, or even all or part of the training can be reintroduced. of reports edited or corrected by the healthcare professional in the training game.

[0080] The invention also relates to a computer system arranged to implement the process according to the invention.

[0081] The invention also relates to a computer program comprising program code which is designed to implement the method according to the invention.

[0082] The invention also relates to a data carrier on which the computer program according to the invention is recorded.

[0083] Brief description of the figures

[0084] Other advantages and features of the present invention are now described by means of purely illustrative and in no way limiting examples of the scope of the invention, and from the accompanying figures, in which:

[0085] [Fig. 1] represents, schematically and partially, a computer system for the implementation of a process according to an example of an embodiment of the invention;

[0086] [Fig. 2] represents, schematically and partially, a method for evaluating a report of a medical procedure automatically generated according to an example of an embodiment of the invention, implemented using the computer system of [Fig. 1];

[0087] [Fig. 3] represents, schematically and partially, an incidental expert report provided in a step of the process of [Fig. 1];

[0088] [Fig. 4] schematically and partially represents part of an example training set for a machine learning algorithm used in the process of [Fig. 1]; and

[0089] [Fig. 5] represents, schematically and partially, two simplified accounts inferred in a step of the process of [Fig. 1],

[0090] In the description that follows, identical elements, by structure or by function, appearing on different figures retain, unless otherwise specified, the same references.

[0091] The processes described below can also be implemented by software programs executable by a computer system. Furthermore, their implementation can be achieved through distributed processing and / or parallel processing, particularly for processing multiple data points simultaneously.

[0092] The figures described in this document are intended to provide a general understanding of the invention in various embodiments. These figures are not intended to serve as a complete description of all the elements and features of the devices, processors, and systems necessary for the invention. Many other embodiments of the invention, or combinations thereof, may be apparent to those skilled in the art upon reading this description, by combining the disclosed embodiments. Other embodiments may be derived from the description, so that structural and logical substitutions and changes may be made without departing from the scope of the present invention.

[0093] Furthermore, the description and figures should be considered illustrative rather than restrictive, and the appended claims are intended to cover all modifications, improvements, and other embodiments of the invention. Therefore, the scope of the following claims should be determined by the broadest possible interpretation of the claims and their equivalents and should not be restricted or limited by the preceding description.

[0094] Description of the implementation methods

[0095] [Fig. 1] A computer system 1 for the implementation of a method for generating reports of a medical procedure according to an example of an embodiment of the invention.

[0096] The computer system 1 comprises a first computer terminal 2, a processing server 3 and a second computer terminal 4.

[0097] In the example described, the first computer terminal 2 is a desktop computer, part of a hospital information system 5, for example of a hospital, clinic, radiology unit, surgery unit, consulting office or analysis laboratory.

[0098] The processing server 3 is remote from the hospital information system. The first computer terminal 2, or the hospital information system 5, and the processing server 3 are each equipped with a wireless communication unit, so that the services offered by the processing server 3 are accessible by the first computer terminal 2, in particular via a wireless communication network 6 implementing HTTPS and IP type communication protocols.

[0099] As will be described later, a software application 31 is installed on the processing server 3 and can in particular be executed on the first computer terminal 2 through an internet browser, in the form of a SaaS service.

[0100] The second computer terminal 4 is, in the example described, a patient's smartphone that can receive, through a software application 41 installed on this smartphone, data transmitted by the processing server 3.

[0101] Alternatively, the first computer terminal 2, the hospital information system 5, and the processing server 3 could be hosted in the same infrastructure, with these different elements then being connected to each other by a wired local communication network and / or a wireless local communication network. Alternatively, the software application 31 could be installed and run locally on the first computer terminal 2.

[0102] In connection with [Fig. 2], we will now describe a process for evaluating a report of a medical procedure carried out by a healthcare professional for a patient, and implemented using computer system 1.

[0103] In stages not shown, the patient follows a path within the hospital or laboratory where the medical procedure is performed. This path This process may include a reception stage, one or more examination stages involving medical imaging, sampling, analysis, surgery, care, or interviews with medical staff, and a discharge stage. A data set D can thus be collected throughout the patient's journey by various components of the information system 5. These components may include medical staff, such as reception staff, paramedics, doctors, or nurses; medical equipment; or a related information system such as a radiology information system (RIS), a laboratory management system (LMS), a picture archiving and transmission system (PACS), or even external information systems, such as the IT departments of public organizations.

[0104] This dataset D may therefore include information from a computerized patient record, clinical data, and medical images acquired using one or more medical imaging modalities.

[0105] During one of the stages of the patient's journey, a component of the information system 5 initiates the implementation of the process according to the invention. In the example described, this is the practitioner who performed the medical procedure for the patient. Alternatively, the process may be initiated by a non-human component of the information system 5, in particular by a computer program, or the process may be implemented by several components of the information system 5, in particular by different members of the medical staff at different points in the patient's journey, both before and after the medical procedure.

[0106] In a first step 1, the practitioner connects via the first computer terminal 2 to the software application 31 of the remote server 3, through a strong authentication process implemented by another server dedicated to this process (not shown).

[0107] In this step El, according to the example described, the practitioner usually writes a medical report, called "expert," relating to the medical procedure performed for the patient, through an input interface of the software application 31.

[0108] Figure 3 shows a simplified example of a CRMi expert medical report written during a human papillomavirus (HPV) screening performed on a patient. This CRMi expert report contains a title indicating the reason for the consultation, a section relating to the diagnosis, and a section providing a cytopathological commentary on the screening results as well as an indication of the cytopathological classification used.

[0109] The invention is not limited to this type of medical report or medical procedure and may include, but is not limited to, other elements relating to the patient, their medical history and symptoms, the medical procedure and / or consultation, clinical observations, results of additional examinations, diagnosis and prognosis, treatment and / or follow-up plan; or any other type of information provided to the patient. It may also contain other documents, in the form of reports, examination results, images. or any document that may result from a medical act, examination, intervention, or consultation. Furthermore, the organization and structure of the report may differ from those shown in [Fig. 3].

[0110] Furthermore, the CRMi expert report may be generated by other healthcare professionals and in particular all or part of the CRMi expert report may be entered by other components, human, software or hardware, of the information system 5, during the patient's journey, before and / or during and / or after the medical procedure, in particular from the data set D.

[0111] It may also be possible to consider that the CRMi expert report be generated by other methods, and in particular by: uploading a scan of a handwritten report into application 31; transcribing a report dictated orally and recognized by a speech recognition algorithm; generation assisted by one or more machine learning algorithms from a prompt provided by a healthcare professional; automatic generation by one or more machine learning algorithms from recordings and / or results of the medical procedure.

[0112] In a second step E2, the CRMi expert report is transmitted to a data processing module of server 3; to be converted there in a third step E3 into a plurality of simplified CRSj reports.

[0113] In other variants not shown, it may be provided that all or part of the data D is transmitted directly to the data processing module of server 3, either by the practitioner via terminal 2, or directly by the information system 5. In this case, the objective of the data processing module of server 3 is to generate, from this data D, a report, which may be a simplified report or an expert report.

[0114] In the example described, the conversion is implemented by this data processing module using multiple machine learning algorithms (MLj) executed in parallel in sub-steps (E3j), to which the CRMi expert report is provided as input. Alternatively, each machine learning algorithm could be executed by one or more other servers, located remotely from server 3, to which server 3 connects. These servers could, for example, be organized into clusters, particularly within a cloud infrastructure.

[0115] In the example described, each MLj algorithm is a large language model or LLM, such as a GPT-4 type semantic autoregression algorithm and a Mistral Large type semantic autoregression algorithm.

[0116] We can expect to use other semantic autoregression algorithms, such as GPT-3, LLaMa, Gemini, or Claude, or even other encoder-type algorithms, such as BERT, or other types of machine learning algorithms capable of converting a text into a simpler text, such as artificial neural networks, deep neural networks (DNNs), convolutional neural networks (CNNs), and so on. Recurrent neural networks (RNNs), large language models (LLMs), support vector machines (SVMs), and random forests are all possibilities. It is also possible to generate each simplified CRSj report using several algorithms executed sequentially.

[0117] To implement the conversion in each E3j substep, the CRMi expert report is provided as input to each MLj algorithm along with a query or prompt, which may be pre-written or, alternatively, written by the practitioner. This query can define instructions requesting the generation of a simplified CRSj report, excluding any use of technical or medical vocabulary.

[0118] Each machine learning algorithm MLj was previously trained, in EOj steps, to convert an expert report into a simplified report, that is, a version of this expert report made understandable by non-experts by replacing complex medical or technical terms with more common synonyms; by adding detailed explanations of medical terms and procedures; by removing or summarizing non-essential or overly detailed information; or by reorganizing the structure of the report to highlight information considered essential for the patient, such as test results, diagnoses, and treatment recommendations.

[0119] To this end, a training set DSj was prepared beforehand for each machine learning algorithm MLj. Each training set DSj includes, on the one hand, expert reports CEk written by healthcare professionals. Each of these expert reports CEk is then labeled by an expert, such as a physician or a member of a patient association, with a simplified report CSk. Alternatively, each of the machine learning algorithms could be trained using the same training set.

[0120] Each DSj training set is segmented into different sets of expert CEk and simplified CSk reports, each set being associated with a medical type, such as a medical specialty, or a pathology.

[0121] We have thus represented in [Fig. 4] a simplified example of part of a DSj training game, comprising three expert reports CEi, CE2 and CE3, each labeled by a simplified report CSi, CS2 and CS3.

[0122] In each EOj step, each machine learning algorithm MLj is trained to iteratively generate a simplified report from each expert report CEk in the training set and a pre-written prompt. At each iteration, an error metric is determined based on a semantic distance between the generated simplified report and the simplified report CSk labeling the expert report CEk used to generate the simplified report. As a non-limiting example, this error metric could be a BLUE or RED metric.

[0123] The MLj machine learning algorithm can then adjust its internal parameters, such as weights, neuronal biases, to minimize this error metric over iterations, for example by gradient descent.

[0124] In an LLM context, it can be expected that machine learning algorithms MLj will be pre-trained on a large training set not specific to the medical field, then trained by a "few-shot learning" or "fine tuning" method using a training set DSj containing only a few pairs, such as a dozen, of expert report CEk - simplified report CSk.

[0125] Alternatively, other supervised training methods could be used, or several training methods could be combined, including supervised and semi-supervised, or even unsupervised, training methods.

[0126] Furthermore, it can be anticipated that the expert reports CEk from a training set DSj will be segmented during labeling into several sections, such as a title, patient information, reasons for consultation, medical history, clinical examination, diagnosis, treatment plan, and a conclusion. Each section can then be labeled with a section from a simplified report CSk. The training implemented in the EOj steps can thus be used to generate simplified sections from an expert report CEk. All the sections thus generated are combined to form a simplified report, which is then compared to the simplified report CSk, which labels the expert report CEk to adjust the MLj algorithm.

[0127] In a fourth step, E4, an initial proximity value Vi of the CRMi expert report with the DSj training sets is estimated. This Vi value reflects the novelty of the CRMi expert report with all the CEk expert reports in the DSj training set associated with the medical type corresponding to the CRMi report. This medical type can be pre-selected by the practitioner or automatically inferred from the CRMi expert report.

[0128] To this end, in substep E41, a proximity index h,k is estimated from the CRMi expert report and each of the CEk expert reports. This proximity index h,k is calculated according to a semantic distance metric between the CRMi and CEk reports, for example RED-1.

[0129] It may be possible to use other translation or simplification evaluation metrics, such as BLEU, METEOR, BERTScore, BLEURT, COMET, cosine similarity, Fl, SARI, or other measures of distance between vector representations of texts, or to combine several semantic distance or text comparison metrics.

[0130] In a sub-step E42, the first proximity value Vi is determined as the maximum value of the proximity indices h,k, and thus indicates, on the one hand, which expert report CEk of the training corpus most closely resembles the incident report CRMi, and on the other hand, the level of similarity between these two texts.

[0131] Alternatively, other methods for calculating the first value of could be considered. proximity Vi, such that the average value of the N highest values ​​of the proximity indices ,k.

[0132] Note that, in the examples of [Fig. 3] and [Fig. 4], the value Vi is 1, the incident report CRMi being identical to the CEi report of the training set DSj.

[0133] In a fifth step E5, for each simplified report CRSj, a second proximity value V2,j of this CRSj report with the DSj training sets is estimated. This value V2J reflects the relevance of the simplified report CRSj with respect to the simplified report CSk, which labels the expert report CEk of the DSj training set closest to the CRMi incident report.

[0134] This second proximity value V2,j is calculated according to a semantic distance metric between the reports CRSj and CSk, distinct from that used in substep E41, for example SARL. Alternatively, the semantic distance metrics used in steps E41 and E5 may be identical.

[0135] We have represented in [Fig. 5] two simplified reports CRSj generated from the expert report CRMi represented in [Fig. 3], by two different LLM MLj, namely GPT-4 for CRSi and MISTRAL Large for CRS2, trained using the training set DSj represented in [Fig. 4],

[0136] In this example, the second proximity value V2 between the simplified CRSi report and the CSi report is 0.97, while the second proximity value V24 between the simplified CRS2 report and the CSi report is 0.61.

[0137] In a sixth step E6, a confidence index lj is estimated for each of the simplified reports CRSj inferred by the algorithms MLj from the first proximity value and the second proximity value V2j associated with the simplified report CRSj.

[0138] In the example described, a penalty P nis calculated from the first Vi value raised to a given power, for example 2. The penalty P n is multiplied by the second value V2j to arrive at the confidence index lj.

[0139] The confidence index lj thus reflects both the similarity of the CRMi expert incident report to the CEk reports of the DSj training set and the relevance of simplifying this incident report with regard to this training set. Furthermore, the penalty P n allows for penalizing a discrepancy between the CRMi report and the DSj training game.

[0140] It should be noted that, in the example of [Fig. 3] to [Fig. 5], the confidence index li associated with the simplified report CRSi is 0.97, while the confidence index I2 associated with the simplified report CRS2 is 0.61.

[0141] In undescribed examples, the penalty P ncan be estimated using other methods, for example a combination of several initial values ​​defined according to several metrics, such as RED-1 and BLUE-1, or even the penalty P n focuses on the second proximity values ​​V2j, in order to penalize errors in generating simplified reports. Other combinations of the first and second values ​​VI and could be considered. V2j, and to introduce into this combination a weight determined from the machine learning algorithm MLj used for the generation of the simplified report CRSj and / or the training set DSj.

[0142] In a seventh step E7, each confidence index lj is compared to a predetermined threshold value TS, for example 0.5, or even 0.75.

[0143] In an eighth step E8, all simplified CRSj reports whose confidence index lj is greater than the threshold value TS are transmitted to the practitioner's computer terminal 2, along with their confidence index lj, to be displayed on a viewing, editing and validation interface of the software application 31. Simplified CRSj reports that failed in step E7 are not transmitted, in order to avoid wasting time for the practitioner and risking providing unreliable information to the patient.

[0144] Therefore, the practitioner can view each of the simplified CRSj reports generated from their CRMi report on their terminal 2. Furthermore, they can also view the confidence index lj of each of these simplified CRSj reports and evaluate both the similarity of their expert CRMi report to the DSj training sets of the MLj machine learning algorithms and the relevance of the CRMi report simplification in relation to this DSj training set.

[0145] The practitioner can, in particular, select the simplified CRSj report with the highest confidence level (lj) and decide, by comparing this confidence level (lj) to a second threshold value that they have set and / or that an association, such as a learned society or other professional organization, has recommended, whether this simplified CRSj report warrants revision or whether they can trust it and distribute it to the patient without review. They can also evaluate each of the simplified CRSj reports to decide which one can be distributed to the patient, or even to other healthcare professionals or other medical information systems, without in-depth review. They can also edit the selected CRSj report to improve it before distributing it to the patient.

[0146] It can be anticipated that the fourth step, E4, will be implemented before step E3, and that, depending on the first proximity value, Vi, steps E5 to E8 will be replaced by other steps. In particular, as in the examples described in [Fig. 3] and [Fig. 4], when the CRMi incident report is identical to one of the CEk reports in the DSj training set, the value Vi is 1. In this case, the simplified CRSj report can be directly selected from among the simplified CSk reports by selecting the one associated with the expert CEk report that is identical to the CRMi incident report. The confidence index lj can also be directly set to a maximum value of 1, so that only the selected simplified CRSj report and this confidence index lj of 1 are transmitted to terminal 2.

[0147] In undescribed variants, it may be stipulated that only the simplified CRSj report associated with the highest confidence index lj is transmitted during transmission step E8, or that only the N simplified CRSj reports associated with the indices of the highest confidence levels are transmitted during the E8 transmission step.

[0148] Alternatively, each of the machine learning algorithms (MLj) could be trained to convert an expert report into a simplified report according to a given level of understanding. This would indicate, for example, whether certain technical terms are permitted, or whether specific sections should be deleted, simplified, or retained as is. In this variant, the process generates several simplified reports (CRSj) of the same medical procedure at different levels of understanding, allowing the healthcare professional to select the simplified CRSj report with a sufficiently high confidence level (lj) that they deem most relevant to the patient's understanding and the care pathway they are following.

[0149] Alternatively, a single machine learning algorithm MLj could be used to convert the expert CRMi report into a simplified CRSj report. In this case, only one simplified CRSj report will be transmitted to the practitioner's terminal 2, provided that its confidence index lj is greater than the threshold value TS.

[0150] In a ninth step E9, the simplified CRSj reports rejected by the practitioner, as well as the modifications made to the simplified CRSj report selected by the practitioner, are transmitted to the remote server 3 in order to perform reinforcement retraining of the machine learning (MLj) algorithms using these different CRSj reports. It can thus be anticipated that, during this retraining, simplified reports generated by an MLj algorithm that closely resemble one of these modified or rejected CRSj reports will result in a penalty when the MLj algorithm parameters are adjusted.

[0151] In a tenth step, E10, the expert CRMi report and the simplified CRSj report selected by the practitioner, and possibly corrected, are transmitted to the patient's computer terminal 4. This transmission can be carried out by the transmission server 3, once these CRMi and CRSj reports have been retrieved by server 3, following the practitioner's validation of their distribution.

[0152] In examples of implementing step E10, the processing server 3 could generate a digital identifier, for example in the form of a QR code, which could be communicated to the patient, for example via the screen of the computer terminal 2 or printed on a prescription form for later scanning by the patient. Entering this QR code on their computer terminal, particularly by scanning it using the software application 41 installed on terminal 4, allows the patient to automatically download the CRMi and CRSj reports from the processing server 3 into the software application 41.

[0153] It should be noted that different aspects of the various interfaces that have been represented can be envisaged and / or other functionalities added, without going out of the scope of the present invention.

[0154] The preceding description clearly explains how the invention achieves its stated objectives, namely, to provide a method for exchanging medical reports between the various healthcare professionals involved in a patient's care pathway and the patient, according to appropriate levels of understanding. These objectives are achieved, in particular, by means of a processing server capable of generating simplified, abridged, and / or simplified reports from a medical report generated at a first computer terminal, and of transmitting all the reports to another computer terminal.

[0155] In any event, the invention cannot be limited to the embodiments specifically described in this document, and extends in particular to all equivalent means and to any technically operative combination of these means.

Claims

Demands

1. A method for evaluating a report of a medical procedure performed by a healthcare professional for a patient, the report being automatically generated, the method being implemented by a computer system (1) and comprising the following steps: a. (E2) providing a set of information (D, CRMi) relating to the medical procedure to the computer system; b. (E3) generating, from said provided set of information, a report (CRSj), by means of at least one machine learning algorithm (MLj) trained to generate a report of a medical procedure from a set of information relating to that medical procedure, said at least one algorithm having been trained using a training set (DSj) comprising a plurality of information sets (CEk), each relating to a given medical procedure, and a plurality of predetermined reports (CSk), each associated with one of said information sets; c.(E4) estimation of a first proximity value (Vi) between said provided information set and at least one information set from the training set of said machine learning algorithm; d. (E5) estimation of a second proximity value (V2) between the report generated by the machine learning algorithm from said provided information set and at least one predetermined report from the training set of said machine learning algorithm; e. (E6) estimation of a confidence index (lj) associated with the report generated by the machine learning algorithm from the first and second proximity values; f. (E8) transmission of at least the report generated by the machine learning algorithm and the confidence index to a computer terminal (2).

2. A method according to the preceding claim, characterized in that the step (E4) of estimating the first proximity value (Vi) comprises a substep (E41) of estimating a proximity index (h,k) between said provided information set (CRMi) and each information set (CEk) of a plurality of information sets of the training set (DSj), said first proximity value being calculated from said proximity indices.

3. A method according to the preceding claim, characterized in that said provided information set (CRMi) is a first report, referred to as an expert report; in that the machine learning algorithm (MLj) has been trained to convert, from of a training set (DSj) comprising expert reports (CEk) each associated with a simplified report (CSk), an expert report in a simplified report, and in that each proximity index ( ,k) is calculated according to at least a first metric of semantic distance between two texts.

4. A method according to the preceding claim, characterized in that said semantic distance metric is chosen from one of the following translation or simplification evaluation metrics: BLUE, RED, METEOR, BERTScore, BLEURT, COMET, cosine similarity, Fl, SARI, BARTScore.

5. A method according to any one of claims 2 to 4, characterized in that said first proximity value (Vi) is the maximum value of the proximity indices.

6. A method according to any one of the preceding claims, characterized in that said first proximity value (Vi) corresponds to a given information set (CEk) of the training set (DSj) and in that the second proximity value (V2j) is an estimated proximity index between the report (CRSj) generated by the machine learning algorithm (MLj) and a report (CSk) of the training set (DSj) associated with said given information set (CEk) of the training set corresponding to the first proximity value.

7. Method according to the preceding claim, characterized in that said proximity index (V2j) is calculated according to at least a second metric of semantic distance between two texts.

8. A method according to any one of the preceding claims, characterized in that the step (E6) of estimating a confidence index (lj) associated with the report generated (CRSj) by the machine learning algorithm (MLj) comprises a substep of estimating a penalty (P n ) from one of the first and second proximity values ​​(Vi, V2, the confidence index corresponding to the other of said proximity values ​​penalized by said penalty.

9. A method according to any one of the preceding claims, characterized in that it comprises, prior to the transmission step (E8), a step (E7) of comparing the confidence index (lj) to a predetermined threshold value (TS), the transmission step being conditioned on the result of said comparison.

10. A method according to any one of the preceding claims, characterized in that the generation step (E3) is a generation step, from said provided information set (CRMi), of a plurality of reports (CRSj), each being generated by means of a machine learning algorithm (MLj) distinct and trained to generate a report of a medical procedure from a set of information relating to that medical procedure, each algorithm having been trained using a training set (DSj) comprising a plurality of information sets (CEk), each relating to a given medical procedure, and a plurality of predetermined reports (CSk) each associated with one of said information sets; and in that a second proximity value (V2j) and a confidence index (lj) are estimated for each of said reports generated by the machine learning algorithms.

11. A method according to any one of the preceding claims, characterized in that it comprises a step of evaluating, on said computer terminal, the report generated by the machine learning algorithm, and a step (E9) of transmitting, to the computer system (1) said evaluation.

Images