Machine learning system for diagnosing breast lesions, adaptable to the clinician and based on assertiveness

Abstract

The present disclosure relates to the medical imaging field, specifically, associated computer systems for identifying breast lesions. The subject of the present disclosure is a machine learning system for diagnosing breast lesions (10), comprising a first database (11) a second database (12) and at least one server (20) interconnected with the first and second databases, that trains (110) and uses classifier agents (140) to determine a classification and respective segmentation associated with breast lesions through the determination of probable physical parameters with the determination of a probability of occurrence, based on digital images of a breast section (120). The computational determination of probable physical parameters is adapted based on demographic characteristics of the clinician (130), thus implementing a solution with varying degrees of assertiveness (150). This solution improves on known solutions by determining a probability degree of breast lesions existence through a highly adaptable computer system trained for this purpose.

Description

[0001] DESCRIPTION

[0002] MACHINE LEARNING SYSTEM FOR DIAGNOSING BREAST LESIONS,

[0003] ADAPTABLE TO THE CLINICIAN AND BASED ON ASSERTIVENESS

[0004] TECHNICAL FIELD

[0005] The present disclosure is in the field of medical imaging, specifically imaging and associated computer systems aimed at identifying breast lesions, thus identifying potential breast cancer lesion masses or potential breast cancer lesion microcalcifications.

[0006] BACKGROUND

[0007] Breast cancer is the most common cancer in women worldwide, with almost 1.7 million new cases diagnosed in 2012, representing 12% of new cancers and 25% of all 166 types of cancer in women, according to the article "The efficacy of using computer-aided detection (cad) for detection of breast cancer in mammography screening : a systematic review" , Henriksen, E. et al, Acta Radiologica, 2018.

[0008] Medical imaging diagnosis is a routine procedure carried out by radiologists to help diagnose or monitor a medical condition. Medical imaging diagnosis allows doctors to identify pathologies by decoding tissue characteristics by examining features in medical images. It plays a key role in modern medicine, particularly in the prevention and diagnosis of cancer, which is one of the leading causes of mortality worldwide. The Portuguese patent application with publication number PT 116801 discloses a computational method for the improved identi f ication of breast lesions from digital images obtained using di f ferent technologies .

[0009] The solution of the present disclosure improves on that solution in that it determines a degree of probability of the existence of breast lesions through a highly adaptable computer system trained for this purpose .

[0010] DISCLOSURE SUMMARY

[0011] The subj ect of the present disclosure is therefore a machine learning system for diagnosing breast lesions , adaptable to the clinician and based on assertiveness , using one or more computing processors .

[0012] The system can include : a first operable database to maintain a diversity of characteristics obtained from digital images of breast sections comprising annotated patterns , at least part of the annotated patterns being associated with breast lesions ; a second operable database to maintain a diversity of demographic characteristics of clinicians ; at least one server interconnected with the first database and the second database , the at least one server being configured to : train one or more classi fier computer agents on the basis of the annotated patterns in the respective first database ; computationally obtain one or more digital images of a breast section; obtain one or more demographic characteristics of a clinician from a computational interaction provided by the clinician and the diversity of demographic characteristics of clinicians present in the respective second database ; use one or more classi fier agents to computationally determine a classi fication and respective segmentation associated with one or more breast lesions by determining one or more probable physical parameters , the probable physical parameters comprising potentially existing physical parameters for which a probability of occurrence is determined; in which the computational determination of probable physical parameters is adapted based on the demographic characteristics of the clinician .

[0013] Using one or more classi fier computer agents trained on the basis of the annotated patterns in the first database not only improves the determination made by these classi fier agents , but also makes the probabi lities of occurrence of potentially existing physical parameters more reliable . In addition, it permits a component of assertiveness , in which the computational determination of probable physical parameters and the corresponding availability of this determination are adapted based on the demographic characteristics of the clinician who is interacting with the system. The solution of the present disclosure also enables the detection of breast lesions by adapting the tone of assertiveness . Demographic characteristics can include a clinician' s degree of experience , defined on the basis of a number of years of experience .

[0014] Also included in the present disclosure is a system composed of machine learning for diagnosing breast lesions , adaptable to the clinician and based on assertiveness and using one or more computer processors , and consisting of the machine learning system for diagnosing breast lesions of the present disclosure , and at least one piece o f imaging equipment .

[0015] DESCRIPTION OF THE FIGURES

[0016] Figure 1 - schematic representation of a system ( 10 ) according to the present disclosure , comprising a first database ( 11 ) , a second database ( 12 ) and at least one server interconnected with the first database and the second database ( 20 ) . The server is configured to train one or more classi fier computational agents based on the annotated patterns from the respective first database ( 110 ) , computationally obtain one or more digital images of a breast section ( 120 ) , obtain one or more demographic characteristics of a clinician from a computational interaction provided by the clinician and from the plurality of demographic characteristics of clinicians present in the respective second database ( 130 ) , and using the one or more classi fier agents to computationally determine a classi fication and a respective segmentation associated with one or more breast lesions by determining one or more probable physical parameters , the probable physical parameters comprising potentially existing physical parameters for which a probability of occurrence is determined ( 140 ) , wherein the computational determination of the probable physical parameters is adapted based on the demographic characteristics of the clinician ( 150 ) .

[0017] Figure 2 - digital image of a breast section, obtained by mammography imaging (MG) . A result obtained from the machine learning system for diagnosing breast lesions is shown, which indicates that three potential lesions have been identi fied, with the type of distribution of each one and some probable physical parameters and additional parameters . The accuracy and a classi fication according to the Breast Imaging Reporting and Da ta System (BIRADS ) are also presented .

[0018] Figure 3 - examples of di f ferent presentations of information resulting from the determinations of this disclosure system, with di f ferent levels of assertiveness .

[0019] DETAILED DESCRIPTION

[0020] Additional details of the clinician-adaptable , assertiveness-based machine learning system for diagnosing breast lesions of the present disclosure are described below .

[0021] The system can be configured for classi fication according to the Breast Imaging Reporting and Da ta System ('BIRADS ) .

[0022] The system can be configured so that the probable physical parameters include one or more of the following : the number of breast lesions identi fied, the severity associated with the breast lesions identi fied, the medical imaging technology of one or more digital images , the visual representation of the breast lesions identi fied .

[0023] The system can also be configured to computationally obtain more than one digital image of the same breast section, these digital images being obtained using di f ferent imaging technologies .

[0024] The system can also be configured so that the probable physical parameters and their probability are determined for each imaging technology . This allows a more refined determination of any discrepancies between di f ferent technologies , when di f ferent technologies are used .

[0025] The system can also be configured to provide a computational determination of additional physical parameters , in which the computational determination of additional physical parameters is adapted based on the demographic characteristics of the clinician, the additional physical parameters involving the sensitivity and speci ficity of one or more of the machine learning models used, and a patient ' s medical history associated with the digital images and that patient ' s family members .

[0026] The assertiveness component is thus improved such that it also includes additional physical parameters .

[0027] The system can be further configured so that the demographic characteristics of the clinician comprise a degree of experience of the clinician, defined on the basis of the number of years of experience of the clinician . Speci fically, the system can be further configured so that the demographic characteristics of the clinician comprise a degree of experience of the clinician and include the definition of ranges of clinician years of experience , where these ranges correspond to the experience categories of intern, j unior, intermediate , and senior . As mentioned, the computational determination of the probable physical parameters is adapted based on the demographic characteristics of the clinician .

[0028] The determination and potential subsequent presentation of the determination to the clinician could involve determining the parameters density composition, shape and margin of a breast les ion that may consist of a mass , as well as the parameters morphology / type and distribution of a breast lesion that might consist of calci fication . With this information, the system can adapt the determination and subsequent presentation of a communication based on an assertive tone for the detection, classi fication and segmentation of breast lesions .

[0029] These one or more digital images can be obtained from mammography, ultrasound and / or magnetic resonance imaging . The system is therefore configured accordingly, so that it can operate with digital images obtained from mammography, ultrasound and / or magnetic resonance imaging .

[0030] The system can be configured so that this segmentation is carried out using a geometric shape .

[0031] The geometric shape can consist of one or more polygons , eccentric sections or combinations thereof .

[0032] Additional details of the composite breast lesion identi fication system of the present disclosure are described below .

[0033] The system may comprise at least two pieces of equipment with di f ferent imaging technologies . Said equipment can be mammography imaging, ultrasound and / or magnetic resonance imaging.

[0034] EXAMPLES

[0035] Additional details of the systems in this disclosure are described below.

[0036] The machine learning system for diagnosing breast lesions in the present disclosure uses intelligent agents with interactivity capabilities and assertiveness-based communication for personalised medicine. Assertiveness is adapted based on the clinical experience of the clinician / doctor , where the agent adapts a more or less assertive tone, as well as behaving in a more or less reactive / proactive way, depending on the doctor's clinical experience .

[0037] Depending on the professional experience of the clinicians / doctors (i.e. interns, junior, intermediate and senior) , various machine learning / artif icial intelligence (Al) results will inform and explain the following to doctors: (1) number of findings detected; (2) patient severity for each breast and by medical imaging modality (e.g. mammography - MG; ultrasound - US; and magnetic resonance imaging - MRI) ; (3) visual representation of benign or malignant estimates; (4) visualisation of the models' sensitivity and specificity results; and (5) patient's personal and family histories.

[0038] As mentioned above, the subsequent presentation of the determination to the clinician may involve determining the parameters density composition, shape and margin of a breast lesion that may consist of a mass, as well as the parameters morphology / type and distribution of a breast lesion that could consist of calcification. With this information, the system can adapt the determination and subsequent presentation of a communication based on an assertive tone for the detection, classification and segmentation of breast lesions.

[0039] As an example, a less assertive recommendation will indicate that, according to the probable physical parameters determined, a diagnostic recommendation should be followed. An example would be the indication that "with these determinations, it seems that you should follow my recommendations". A more assertive recommendation will indicate that, according to the probable physical parameters determined, it is certain that a diagnostic recommendation should be followed. An example would be the statement "with these determinations, I am certain that you should follow my recommendations" .

[0040] Through an interface, the patient's classified data from the Al results is communicated by the intelligent agent. The intelligent agent communicates the results in a more assertive (declarative) or less assertive (suggestive) tone, depending on the professional experience of each clinician. Diagnostic data representing the severity of each patient's breast is retrieved. The intelligent agent guides the patient's particular diagnosis for each clinician, adapting the communication narrative via the interface.

[0041] The assertiveness-based agent uses recommendations to classify and segment: (1) the number of lesions detected; (2) the severity for each breast of the patient and by medical imaging modality; (3) a visual scale representing benign or malignant estimates; (4) provide visualisation of the sensitivity and speci ficity results of the models ; and ( 5 ) with clinical arguments from the patient , such as pathological covariates . To compare the assertiveness-based agent with a conventional agent , participants are given the information that the recommendations are generated by Al models , so that they can also provide some feedback on the model ' s performance . As mentioned, the subsequent presentation of the determination to the clinician may involve determining the parameters mentioned above .

[0042] Applied machine learning / Al models may be known from the state o f the art , being distinct at least insofar as they have been trained according to the solution of the present disclosure .

[0043] The agent has the ability to recommend classi fication and segmentation based on the Al model for 2D imaging modalities such as mammography and ultrasound, as well as Al models for 3D imaging modalities such as MRI . The intelligent Al agent provides the classi fication of the patient ' s breast severity via the BIRADS scale , the accuracy of the model for this classification and the segmentation of the lesion to explain the regions derived from this classi fication .

[0044] With this system, doctors can access a patient ' s information by selecting the ID from a list of patients . Once they have accessed the patient ' s information, doctors can select each respective image of the breast by dragging and dropping each image into the viewing windows . From here , doctors can manipulate the image using the toolbox . The system allows the doctor to accept or rej ect the Al agent ' s final recommendation, while clinicians can also request an explanation to support the final decision they take . A typical output from an Al model includes not only the predicted BIRADS classi fication, but also a probability distribution regarding all possible classi fication choices .

[0045] In this assertiveness-based condition, the agent transmits the clinical arguments along with the communication of BIRADS and accuracy, but this time by descriptive information, such as the parameters mentioned above . Therefore , the assertiveness-based agent is able to communicate conf idence quantitatively based on a sentence describing the clinical arguments . Speci fically, the image viewing window has been augmented with an additional bounding box or circular ellipse highlighting the lesion characteristics that will explain the final BIRADS classi fication .

[0046] Detailed patient augmentation for medical images in breast cancer diagnosis has been extended with a recommendation and consequent explanation based on assertiveness .

[0047] The system set out in the present disclosure has improved how to reliably determine and list the clinical arguments that are later interpretable by human clinicians through the classi fication and segmentation recommendations given by Al .

[0048] This system adapts its communication depending on the demographic characteristics of the doctor, personalising and customising the Al ' s response to clinicians , as described above . These clinical arguments correspond to the classification outputs of an Al model, which have been trained on multimodal clinical case data.

[0049] As an example, the machine learning system for diagnosing breast lesions that is adaptable to the clinician and based on assertiveness of the present disclosure is capable of determining and presenting a more suggestive or assertive communication tone, depending on specific demographic characteristics of clinicians.

[0050] Based on the generic phrase, which contains probable physical parameters:

[0051] > In this case, we have N findings. A finding is an oval-shaped mass with a circumscribed margin...

[0052] One finding will be a breast lesion.

[0053] In a Non-Assertive case, the same information will consist of:

[0054] > In this case, it seems there are N findings . One finding looks like an oval-shaped mass with a circumscribed margin. . . with these findings, it looks like you should follow my recommendations for BIRADS = 4. . .

[0055] In an Assertive case, the same information will consist of:

[0056] > In this case, we have N findings . One finding is an oval-shaped mass with a circumscribed margin. . . with these findings , I'm sure you should follow my recommendations for BIRADS = 4. . . As mentioned, the agents can thus determine probable physical parameters of the lesions, which could take the form of lexical variables of the lesion identified. If the imaging technology is mammography and the breast lesion is a mass, the lexical variables are as follows . If the imaging technology is mammography and the breast lesion is a calcification, the lexical variables are as follows. I f the imaging technology is ultrasound and the breast lesion consists of a mass , the lexical variables are as follows .

[0057] I f the imaging technology is magnetic resonance imaging and the breast lesion is a mass , the lexical variables are as follows .

[0058] The system, server, databases , agents and / or their elements include components for executing at least some of the feature examples and outcomes of the methods described, whether by means of hardware components ( such as memory and / or processor ) , software or any combination thereof . The system, server, databases , agents and / or their elements can include general controllers or speci fic controllers , such as a communications controller, which in turn can consist of or include a microcontroller, capable of receiving external data via inputs and providing triggers for outputs , based on configured rules and / or predefined programming .

[0059] An article for use with the system, server, databases , agents and / or elements thereof , such as a prerecorded storage device or other similar computer-readable medium, including program instructions recorded thereon, or a computer data signal carrying computer-readable program instructions may direct a device to facilitate the implementation of the methods described herein . It is understood that such apparatus and computer data signals are also within the scope of the present disclosure .

[0060] A " computer-readable medium" means any medium that can store instructions for use or execution by a computer or other computing device , including read-only memory (ROM) , erasable programmable read-only memory (EPROM) or flash memory, random access memory (RAM) , a portable floppy disc, a hard disk drive (HDD) , a solid state storage device ( e . g . NAND flash or synchronous dynamic RAM ( SDRAM) ) and / or an optical disc, such as a Compact Disc ( CD) , Digital Versatile Disc ( DVD) or Blu-Ray™ Disc .

[0061] As will be evident to an expert in the field, this disclosure should not be limited to the details described in this document , and various changes are possible which remain within the scope of this disclosure . Of course, the details presented above can be combined in different ways, but we will avoid repeating all these combinations here.

Claims

CLAIMS1 . Machine-learning system for diagnosing breast lesions , adaptable to the clinician and based on assertiveness , and using one or more computer processors , the system being characterised by comprising : a first operable database to maintain a plurality of characteristics obtained from digital images of breast sections comprising annotated patterns , at least part of the annotated patterns being associated with breast lesions ; a second operable database to maintain a plurality of demographic characteristics of clinicians ; at least one server interconnected with the first database and the second database , the at least one server being configured for : training one or more classi fier computer agents on the basis of the annotated patterns in the respective first database ; computationally obtaining one or more digital images of a breast section; obtain one or more demographic characteristics of a clinician from a computational interaction provided by the clinician and the plurality of demographic characteristics of clinicians recorded in the respective second database ; use one or more classi fier agents to computationally determine a classi fication and respective segmentation associated with one or more breast lesions by determining one or more probable physical parameters , the probable physical parameters comprising potentiallyexisting physical parameters for which a probability of occurrence is determined; in which the computational determination of probable physical parameters is adapted based on the demographic characteristics of the clinician .2 . System according to the previous claim characterised by being additionally configured for the classi fication to be according to the Breast Imaging Reporting and Da ta System ('BIRADS ) .3 . System according to any of the preceding claims characterised in that it is configured so that the probable physical parameters include one or more of the following : the number of breast lesions identi fied, the severity associated with the breast lesions identi fied, the medical imaging technology of one or more digital images , the visual representation of the breast lesions identi fied .4 . System according to any of the previous claims characterised by being configured to computationally obtain more than one digital image of the same breast section, said digital images being obtained using di f ferent imaging technologies .5 . System according to the previous claim characterised in that it is additionally configured so that probable physical parameters and their probability are determined for each imaging technology .6 . System according to any of the preceding claims characterised in that it is further configured to provide for a computational determination of additional physical parameters ,in which the computational determination of additional physical parameters is adapted based on the demographic characteristics of the clinician, the additional physical parameters comprising the sensitivity and speci ficity of one or more of the machine learning models used and the medical history of a patient associated with the digital images and of that patient ' s family members .7 . System according to any of the preceding claims characterised in that it is further configured so that the demographic characteristics of the clinician include a degree of experience of the clinician, defined on the basis of a number of years of experience of the clinician .8 . System according to any of the preceding claims characterised in that it is further configured for the demographic characteristics of the clinician to comprise a degree of experience of the clinician including the definition of ranges of years of experience of the clinician, said ranges corresponding to the experience categories of intern, j unior, intermediate and senior .9 . System according to any of the preceding claims characterised by one or more digital images being obtained from mammography, ultrasound and / or magnetic resonance imaging .10 . System according to any of the previous claims characterised by being configured so that said segmentation is carried out using a geometric shape .11 . System according to the previous claim in which the geometric shape consists of one or more polygons , eccentric sections or combinations thereof . 12 . Composite machine learning system for diagnosing breast lesions that is adaptable to the clinician and based on assertiveness and using one or more computer processors characterised by including the computer system of any of the previous claims and at least one imaging device .13 . Composite system according to the previous claim characterised by comprising at least two di f ferent imaging technology devices . 14 . Composite system according to any one of claims 12- 13 characterised by the said device i s mammography, ultrasound and / or magnetic resonance imaging device .

Images