Method and system for quantification of a presence of disease or condition in images provided to artificial intelligence models

Abstract

Scores are generated indicating a presence and quantification of a disease or condition made by an image classifier model by extracting activations from a global average pooling layer of the image classifier model. The activations are generated by the image classifier model in response to being provided as input an image and computing as output a classification of an image class for the image. Coefficients of a concept class associated with the image class are obtained from a trained linear classifier. A directional derivative is generated, as a score, by multiplying the coefficients and the activations to generate a dot product.

Description

Attorney Docket No.515324 (Client Ref.24049PCT) METHOD AND SYSTEM FOR QUANTIFICATION OF A PRESENCE OF DISEASE OR CONDITION IN IMAGES PROVIDED TO ARTIFICIAL INTELLIGENCE MODELS CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims benefit to Provisional Patent Application No.63 / 744,924, filed on January 14, 2025, and Provisional Patent Application No.63 / 571,566, filed on March 29, 2024, and is related to PCT Patent Application No. US 2025 / 014842, filed on February 6, 2025, which claims priority to Provisional Patent Application No.63 / 551,828, filed on February 9, 2024, all of which are hereby incorporated by reference herein. FIELD

[0002] The present disclosure relates to Artificial Intelligence (AI) and machine learning (ML), and in particular to a method, system, and computer program product and computer- readable medium for generating scores indicating a presence and quantification of a disease or condition made by AI models. BACKGROUND

[0003] As the global ageing population increases, there is an observable rise in cases of chronic diseases, including but not limited to diabetes and its associated complications (i.e. diabetic retinopathy), in addition to age-related conditions (i.e. macular degeneration). Early detection and proper diagnosing of such indications could aid in the prevention of blindness or the minimization of vision loss in patients. Early detection and proper diagnosis can also reduce costs associated with expensive treatment during advanced stages of diseases or conditions. Furthermore, accurate determination of disease grade and stage can aid in triaging patients, planning treatment strategy and timeline, and monitoring disease progression, regression, or recurrence. Current methods to grade diabetic retinopathy or macular edema in patients use various phenotypic features, indication-specific biomarkers using segmentation methods, or by directly measuring the patient’s retinal thickness obtained from imaging methods, such as with ocular coherence tomography (OCT) images, collectively to grade the disease. Grading diabetic retinopathy is performed through either retinal fundus images or OCT images. For fundus images, the presence of microaneurysms, hemorrhages, exudates, venous beading, and other like symptoms are used to grade diabetic retinopathy. Whereas forAttorney Docket No.515324 (Client Ref.24049PCT) retinal OCT images, the amount of edema in the retina is graded by measuring the retinal thickness, the thickness location, the volume of fluid collected within the cysts, and indication-specific biomarkers such as disorganization of retinal internal layers (DRIL). In some cases, clinicians may use three features for disease grading, including but not limited to cystoid macular edema, diffuse retinal thickening, and serous retinal detachment.

[0004] The recent expansion of Artificial Intelligence (AI) models has enabled for its implementation at the point of care settings in the detection, diagnosing and grading diseases. Based on the grading methodology, the algorithm can suggest whether or not the patient should be referred to a specialist. AI image segmentation models, for example in diabetic retinopathy, can segment out the cystic spaces and calculate the area of fluid accumulation in a two-dimensional image. When a stack of such OCT scans (also referred to as three- dimensional scans) is analyzed, the volume of the fluid contained in the retinal cysts can be estimated. While the volume of fluid contained in retinal cysts has been used to indicate the severity of a disease or condition it has proven to not be a reliable measure. With manual segmentation of these cystic spaces being a tedious task, AI model segmentation can be used to ease the approach. This AI model segmentation gives an approximate assessment of the imaged fluid depending on the efficiency of the algorithm implemented by the AI model but it is limited to the visible boundaries of the area of interest. Therefore, the size and shape of the retinal cysts segmented by the AI model can vary in its application plus it potentially excludes the surrounding regions to the area of interest, thus not capturing or analyzing the total disease present. AI model segmentation also fails to properly select and analyze small lesions or cysts due to the inherent nature of the AI model architecture which includes dimensionality reduction. By not taking into account small and very small cysts, the fluid volume of cysts actually present is underestimated. Early edema may have only pattern changes and some small cysts, which are out of the scope of capture of segmentation algorithms.

[0005] Therefore, segmentation cannot be taken as a precisely accurate measurement to detect small cysts and the changes in them. Further, a variance in the signal strength of the laser light when passed through a targeted, scanned area is often observed during OCT imaging as a result of increased light scattering across the scanned area (ex. retinal tissue). This effect can potentially obscure phenotypic features or details of the targeted area, thus leading to inaccuracies when training segmentation models or contribute to inaccurateAttorney Docket No.515324 (Client Ref.24049PCT) measurements of the scanned area (i.e. retinal thickness). Although current algorithms attempt to mitigate these inaccuracies, the use of interpolations or extrapolations by these segmentation methods may still not yield accurate results. Another disadvantage of the segmentation approach is that only the cystic areas and sub-retinal fluid can be segmented and approximately quantified because they have distinct visible boundaries, while segmentation of diffuse retinal thickness, changes in pattern, and its quantification is not currently feasible, because only the regions with visible boundaries can be segmented by algorithms. Further, the segmentation methods cannot directly segment out the scanned area or extract the volume of diffuse tissue thickness (i.e. retinal tissue) as these do not have any definite visible boundaries. Solutions which improve AI models early diagnosis prediction results and accuracy while detecting small changes in phenotypic characteristics of indications of a disease or condition without using multiple and complex models are needed to solve the problems associated with current solutions. SUMMARY

[0006] In a first aspect, an embodiment of the present disclosure provides a computer- implemented method for generating scores indicating a presence and quantification of a disease or condition made by an image classifier model implemented by one or more processors, the method comprising: extracting, by a computer system, activations from a global average pooling layer of the image classifier model, the activations generated by the image classifier model in response to being provided as input an image and computing as output a classification of an image class for the image; obtaining, by the computer system, coefficients of a concept class associated with the image class from a trained linear classifier; and generating, by the computer system, a score indicative of the presence or absence of the disease based on the coefficients and the activations. The score may be generated by generating a directional derivative, as the score, by multiplying the coefficients and the activations to generate a dot product.

[0007] In a second aspect, the present disclosure provides the method according to the first aspect, wherein the score indicates the presence and quantification of the disease or condition in the image when the score is positive, and wherein the score indicates a lack of presence of the disease or condition in the image when the score is zero or negative, or wherein the score indicates the presence of another disease or condition in the image when the score is zero or negative.Attorney Docket No.515324 (Client Ref.24049PCT)

[0008] In a third aspect, the present disclosure provides the method according to the first aspect or the second aspect, wherein the method further comprises generating, by the computer system, a recommended treatment plan based on the score being positive.

[0009] In a fourth aspect, the present disclosure provides the method according to any of the first to third aspects, wherein obtaining the coefficients of the concept class associated with the image includes: providing, by the computer system, concept activations of concept images of concept classes associated with the image class and random activations of random images associated with the image class as input to train the linear classifier that is trained to separate the concept activations from the random activations; and obtaining, by the computer system, the coefficients of the concept classes.

[0010] In a fifth aspect, the present disclosure provides the method according to any of the first to fourth aspects, the method further comprising generating, by the computer system, the concept images by extracting, from a training dataset of images, portions from each image of the training dataset of images based on domain knowledge provided by users, the concept images associated with the concept classes.

[0011] In a sixth aspect, the present disclosure provides the method according to any of the first to fifth aspects, the method further comprising obtaining, by the computer system, updated coefficients of the concept class associated with the image class by updating the concept images of the concept classes and the random images to update the training of the linear classifier.

[0012] In a seventh aspect, the present disclosure provides the method according to any of the first to sixth aspects, the method further comprising generating, by the computer system, an updated directional derivative, as an updated score, by multiplying the updated coefficients and the activations of a new image provided as input to the image classifier model to generate an updated dot product.

[0013] In an eighth aspect, the present disclosure provides the method according to any of the first to seventh aspects, wherein the method further comprises, prior to extracting, by the computer system, the activations from the global average pooling layer of the image classifier model, adding a new layer to the image classifier model with 128x7x7 kernels, or updating a first layer of the image classifier model to use an different kernel amount than a previously used kernel amount for the first layer, or deleting a layer of the image classifier model, orAttorney Docket No.515324 (Client Ref.24049PCT) replacing one or more layers of the image classifier model, or using averaged activations from a combination of modified image classifier models and / or image classifier models.

[0014] In a ninth aspect, the present disclosure provides the method according to any of the first to eighth aspects, wherein adding the new layer or updating the first layer is based on a type of machine used to generate the image or based on a type of the disease or condition without batch normalization and activation functions.

[0015] In a tenth aspect, the present disclosure provides the method according to any of the first to ninth aspects, wherein adding the new layer or updating the first layer is based on a type of machine used to generate the image or based on a type of the disease or condition with batch normalization and activation functions.

[0016] In an eleventh aspect, the present disclosure provides the method according to any of the first to tenth aspects, wherein the image classifier model includes one of a Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory Network (LSTM), Transformer, Autoencoder, Principal Component Analysis, Support Vector Machine (SVM), Gaussian Mixture Models (GMM), Hidden Markov Models (HMM), or Decision Trees.

[0017] In a twelfth aspect, the present disclosure provides the method according to any of the first to eleventh aspects, wherein the method further comprises: extracting, by the computer system, feature maps from one or more layers of the image classifier model, wherein the feature maps are generated by the image classifier model in response to being provided as the input the image and computing as the output the classification of the image class for the image; and generating, by the computer system, the activations by performing global average pooling or max pooling of the feature maps to generate a one dimensional vector.

[0018] In a thirteenth aspect, the present disclosure provides the method according to any of the first to twelfth aspects, wherein the image, prior to being provided as input to the image classifier model, is normalized or not normalized based on attributes of a machine or device used to generate the image.

[0019] In a fourteenth aspect, the present disclosure provides a computer system for generating scores indicating a presence and quantification of a disease or condition made byAttorney Docket No.515324 (Client Ref.24049PCT) an image classifier model, the computer system including one or more hardware processors which, alone or in combination, are configured to perform a method according to any of the first to thirteenth aspects.

[0020] In a fifteenth aspect, the present disclosure provides the computer system according to the fourteenth aspect, wherein the score indicates the presence and quantification of the disease or condition in the image when the score is positive, and wherein the score indicates a lack of presence of the disease or condition in the image when the score is zero or negative, or wherein the score indicates the presence of another disease or condition in the image when the score is zero or negative.

[0021] In a sixteenth aspect, the present disclosure provides the computer system according to the fourteenth or fifteenth aspects, wherein the method further comprises generating a recommended treatment plan based on the score being positive.

[0022] In a seventeenth aspect, the present disclosure provides the computer system according to any of the fourteenth to sixteenth aspects, wherein obtaining the coefficients of the concept class associated with the image includes: providing concept activations of concept images of concept classes associated with the image class and random activations of random images associated with the image class as input to train the linear classifier that is trained to separate the concept activations from the random activations; and obtaining the coefficients of the concept classes, wherein the method further comprises: generating the concept images by extracting, from a test dataset, other dataset, or from a training dataset of images, portions from each image of the training dataset of images based on domain knowledge provided by users, the concept images associated with the concept classes; and obtaining updated coefficients of the concept class associated with the image class by updating the concept images of the concept classes and the random images to update the training of the linear classifier.

[0023] In an eighteenth aspect, the present disclosure provides the computer system according to any of the fourteenth to seventeenth aspects, wherein the method further comprises generating an updated directional derivative, as an updated score, by multiplying the updated coefficients and the activations to generate an updated dot product.

[0024] In a nineteenth aspect, the present disclosure provides the computer system according to any of the fourteenth to eighteenth aspects, wherein the method furtherAttorney Docket No.515324 (Client Ref.24049PCT) comprises, prior to extracting the activations from the global average pooling layer of the image classifier model, adding a new layer to the image classifier model with 128x7x7 kernels, or updating a first layer of the image classifier model to use an different kernel amount than a previously used kernel amount for the first layer.

[0025] In a twentieth aspect, present disclosure provides a tangible, non-transitory computer-readable medium having instructions thereon which, upon being executed by one or more processors, provide for generating scores indicating a presence and quantification of a disease or condition made by an image classifier model, according to any of the first to nineteenth aspects and by execution of the following steps: extracting activations from a global average pooling layer of the image classifier model, the activations generated by the image classifier model in response to being provided as input an image and computing as output a classification of an image class for the image; obtaining coefficients of a concept class associated with the image class from a trained linear classifier; and generating a directional derivative, as a score, by multiplying the coefficients and the activations to generate a dot product.

[0026] In a twenty-first aspect, the present disclosure provides the tangible, non- transitory computer-readable medium according to the twentieth aspect, wherein the score indicates the presence and quantification of the disease or condition in the image when the score is positive, and wherein the score indicates a lack of presence of the disease or condition in the image when the score is zero or negative, or wherein the score indicates the presence of another disease or condition in the image when the score is zero or negative, and wherein the instructions, upon being executed by the one or more processors, are further configured to execute the following step: generate a recommended treatment plan based on the score being positive.

[0027] In a twenty-second aspect, the present disclosure provides the tangible, non- transitory computer-readable medium according to the twentieth or twenty-first aspects, wherein the instructions, upon being executed by the one or more processors, are further configured to execute the following step: generating a plurality of directional derivatives, as scores, by multiplying, for each directional derivative of the plurality of directional derivatives, a particular coefficient of the coefficients and the activations to generate an associated dot product, wherein each particular coefficient of the coefficients correspond to a different concept class.Attorney Docket No.515324 (Client Ref.24049PCT)

[0028] In a twenty-third aspect, the present disclosure provides the tangible, non- transitory computer-readable medium according to any of the twentieth to twenty-second aspects, wherein the instructions, upon being executed by the one or more processors, are further configured to execute the following step: providing the plurality of directional derivatives as input to a natural language machine learning model, or rule-based model that is trained to generate, as output, a report indicating the quantification of the disease or condition in the image derived from the plurality of directional derivatives. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present disclosure will be described in even greater detail below based on the exemplary figures. The disclosure is not limited to the exemplary embodiments. All features described and / or illustrated herein can be used alone or combined in different combinations in embodiments of the disclosure. The features and advantages of various embodiments of the present disclosure will become apparent by reading the following detailed description with reference to the attached drawings which illustrate the following:

[0030] FIG.1 illustrates an example process for generating concept images for each class, according to embodiments of the present disclosure;

[0031] FIG.2 illustrates examples of images (mammogram images) with a score for use in categorical grading like in Breast Imaging Reporting and Data System (BI-RADS) grading of mammogram images;

[0032] FIG.3 illustrates an example conventional process of segmentation algorithms outlining retinal layers in an OCT image, used for thickness measurements;

[0033] FIG.4 illustrates an example conventional process of segmentation algorithms outlining fluid filled cysts which serve as a demarcation to estimate fluid volume;

[0034] FIG.5 illustrates an example graph representing scores generated by features of the present disclosure over time as well as heatmaps for the associated scores in the graph over the period of several visits for treatment of Edema, according to embodiments of the present disclosure;Attorney Docket No.515324 (Client Ref.24049PCT)

[0035] FIG.6 illustrates example user interface presentations of scores generated by features of the present disclosure which are scaled for use in mammogram images, according to embodiments of the present disclosure;

[0036] FIG.7 illustrates an example process for obtaining concept activation vectors / coefficients for each concept class, according to embodiments of the present disclosure;

[0037] FIG.8 illustrates an example process for training a retinal OCT classifier model and extracting activations from the classifier model, according to embodiments of the present disclosure;

[0038] FIG.9 illustrates an example process for obtaining coefficients of a linear classifier as well as generating a score indicating a presence and quantification of a disease or condition using extracted activations of a newly provided image, according to embodiments of the present disclosure;

[0039] FIG.10 illustrates an example flowchart generating scores indicating a presence and quantification of a disease or condition made by an image classifier model, according to embodiments of the present disclosure;

[0040] FIG.11 illustrates example Optical coherence tomography (OCT) images with generated scores using features of the present disclosure which indicate the presence or lack of presence of a certain disease or condition, in FIG.11, the presence or lack of presence of Macular Edema (DME), according to embodiments of the present disclosure;

[0041] FIG.12 illustrates example OCT images with generated scores using features of the present disclosure which indicate the presence and quantity or lack of presence of a certain disease or condition (DME in FIG.12) before and after treatment, according to embodiments of the present disclosure; and

[0042] FIG.13 illustrates a simplified block diagram of one or more devices or systems for generating scores indicating a presence and quantification of a disease or condition made by an image classifier model, according to embodiments of the present disclosure.Attorney Docket No.515324 (Client Ref.24049PCT) DETAILED DESCRIPTION

[0043] Embodiments of the present disclosure provide a method and system for generating scores indicating a presence and quantification of a disease or condition made by an image classifier model. While the present disclosure is described primarily in connection with scores associated with an image analyzed by an image classifier model, as would be recognized by a person of ordinary skill in the art, the disclosure is not so limited and inventive features apply to other scenarios such as using, adjusting, or modifying scores generated or otherwise associated with other AI models, such as large language models (LLMs), or otherwise interacting with other AI or machine learning models / algorithms.

[0044] Artificial Intelligence (AI) imaging models learn the concept of the classes they are trained to classify or segment. The implementation of user or domain expert defined concepts can be applied to determine and confirm the significance of these concepts in underlying model predictions when a new image is presented to the model for classification or diagnosis. Such images harboring such concepts are converted into vectors by the applied model and then converted into Concept Activation Vectors (CAV). The technique of using testing with CAVs (TCAV), a score called TCAV score, uses directional derivatives to calculate the importance the user or domain expert defined concept is for the model to make its prediction. This approach demonstrates the sensitivity of the model prediction is to the specific concepts, as a reflection of the internal working model and the influence of the concept to the prediction. The TCAV score provides a global, quantitative explanation of the importance of that concept the model uses to classify the image for predictive purposes, versus the local explainability provided by class attribution methods. The calculation and determination of the CAV and TCAV scores is as described in Kim, Been, et al. "Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV)." International conference on machine learning. PMLR, 2018), is incorporated by reference in its entirety herein.

[0045] The embodiments of the present disclosure may improve the prediction or indication of a presence or lack of presence of a disease or condition in images analyzed by AI models without using multiple and complex models. The present disclosure describes systems and methods to score or grade the presence and / or extent of indications, and may use AI models in classification processes. In embodiments, the present disclosure may implement an enhanced quantification algorithm that may be adaptive to architectural changes in AIAttorney Docket No.515324 (Client Ref.24049PCT) models. Embodiments of the present disclosure may use filters or kernels, directional derivatives, Concept Activation Vectors (CAV), and coefficients of a linear classifier or a support vector machine (SVM) to quantify user defined concepts in an image. The concepts of embodiments of the present disclosure may pertain to concepts as phenotypic features of various indications, diseases, lesions, or biomarkers. Embodiments of the present disclosure may assign a score as a scaler, numerical, or a continuous value relating to the amount or magnitude of a concept that is present in an image. Embodiments of the present disclosure may allow for the detection of concepts within or around the areas of interest within an image. Embodiments may allow for the detection of concepts within or around the areas of interest within an image without image segmentation. In some embodiments, the present disclosure enables grading the presence or severity of a concept in an image. Embodiments of the present disclosure may detect changes in the score over a period of time, such as over the course of several weeks or months which correspond to applications of certain treatment strategies provided by a clinic or doctor. A change in a score may indicate a change in the indication-specific concept or the occurrence of indication-specific concept. In embodiments, the present disclosure may be a system comprised of an image, a model, filters or kernels, CAV, coefficient of the linear classifier or SVM, directional derivatives or dot products, decision boundary, and scoring criteria.

[0046] The images of an indication described and depicted herein may include diseases or conditions of the eye (i.e. Ophthalmology), which may include but are not limited to optical coherence tomography (OCT) images of the retina. Indications of the present disclosure may predict or diagnose and include, but are not limited to, edema or Diabetic Macular Edema (DME), Choroidal Neovascularization (CNV), Drusen, and other like indications. Indications that the present disclosure may predict or diagnose or otherwise classify as present or not present within an image include, but are not limited to, cancer, hyperplasia, pulmonary indications, indications affecting the breast, etc., and other like indications which may be used in other medicine domains or in other domains other than medicine or health domains. The features of the present disclosure may use concepts to classify the indications images as normal as well. The concepts detected and classified by the embodiments of the present disclosure of various indications of the eye may have a concept specific to the indication which the model relies upon to make its prediction. Such concepts may include, but are not limited to, macular edema, cyst like spaces, or a honeycomb appearance. For clarity and as anAttorney Docket No.515324 (Client Ref.24049PCT) example, a retinal OCT image predicted diagnosis as diabetic macular edema, would contain concepts such as cystic spaces or honeycomb appearances.

[0047] The present disclosure may use the value of a directional derivative to quantify the extent of a specific concept in an image. The magnitude of the directional derivative of the concept in an image may be assigned a scaler, numerical, or a continuous value (score) that may be quantified as the extent of the concept’s presence in the image. The coefficients used to generate the value / score may act as a scale / template or basis to measure the defined concept in an image. This scale / template can be modified, updated, or adjusted by updating the model’s architecture, such as by adding layers or modifying kernels, or by updating the coefficients by using new or different concept images. In embodiments, the extent the concept is present in an image may be indicated by the present disclosure as the concept’s directional derivative proximity to or distance from the decision boundary of the linear classifier or SVM. The concepts detected using the features of the present disclosure may include but are not limited to one or more indication-specific phenotypic characteristics, one or more biomarkers, or one or more unique characteristics of an indication displayed in an image. The features of the present disclosure enables systems to detect small changes in a concept. In embodiments, the present disclosure may implement small sized filters or kernels to detect or extract fine or in-depth details of concept features in an image. In some embodiments, the present disclosure may implement large sized filters or kernels to detect or extract concept features from a wider area in an image. The use of large filters or kernels may aid in capturing the context and expansion or reduction of a concept overtime. In embodiments, the number of filters used may be increased or decreased. Detecting small changes in a concept may enable the determination of treatment or care strategy, timeline, administration, early detection, response and monitorization of indication progression, regression, and recurrence.

[0048] AI models implemented with embodiments of the present disclosure can help in early diagnosis, patient triaging, and provide an easy, accurate quantification of an acute or chronic disease requiring continuous monitorization and interventions that would have a profound impact on patients with ophthalmologic indications, such as diabetes and age- related macular degeneration. Embodiments of the present disclosure can effectively detect small changes in the phenotypic characteristics of these indications which will better enable clinicians to make accurate plans of care, treatment, and monitorization of their patients. TheAttorney Docket No.515324 (Client Ref.24049PCT) features of the present disclosure can also be used by manufacturers of imaging machines, by institutions, clinics, and even individual practitioners, as the described algorithms can be made vendor neutral. The accurate detections or indications made by models of the present disclosure can also be used to design treatment strategies for patients or aid clinicians in diagnosing and treatment planning of certain cancers, disorders, or diseases in a patient. For example, clinicians and doctors may utilize different chemotherapy treatments or other treatment options (e.g. surgery) based on the predicted diagnosis for a patient.

[0049] The ability to quickly and accurately predict indications of a disease or a condition represents an important improvement in the technical field of disease classification, where acting quickly to diagnose or provide particular treatments is time-critical and often lifesaving. For example, another conventional AI model approach used by clinicians includes classifying OCT images of macular edema into 4 classes: (a) mild, (b) moderate, (c) severe, and (d) atrophic disease. Many images of each of these classes are used to train an AI deep learning classification model. Any new image of diabetic macular edema given to the AI model is classified into one of these 4 classes. A disadvantage with such an approach is that small changes in the amount of macular edema cannot be detected. Detecting a change in the grade or class of a given image would require a significant amount of change in the edema to adjust or translate one class into the next disease class designation as there are many intra- class variations in the images for each of the 4 classes. A method of scoring to quantify the amount of macular edema with the sensitivity to detect small changes in the edema that may occur during disease progression, treatment, or otherwise would be beneficial. Monitoring a patient based on a change of a score (ex. a continuous value) could be optimal and more preferred over monitoring for changes of the grade of a disease or indication. The scores generated according to embodiments of the present disclosure may also be more advantageous for tracking disease than using AI segmentation, as AI segmentation may not be as accurate or reproducible – especially in the presence of a disease or condition, and is limited to visible boundaries only. As described above, AI segmentation also does not capture pattern changes, whereas embodiments of the present disclosure can capture pattern changes via the generated scores. The generated scores of the present disclosure can also capture features from a surrounding zone of invisible spread. For example, the score generated according to embodiments of the present disclosure include a deep radiomics approach where features are obtained from activations of the last layers of a deep learning model, and contain more non-linear features captured from the image than with traditional radiomics.Attorney Docket No.515324 (Client Ref.24049PCT) Additionally, the output of the system of the present disclosure (the scores) provide for easier explainability as compared to the complicated analysis provided by radiomics approaches.

[0050] The systems and methods described herein involve the use of machine learning models, activation(s) extraction, and generating of scores which represent indications which are computationally complex and cannot be performed in the human mind. Additionally, computing activations, coefficients, and generating scores representing indications of biomarkers in images made by an AI model include computationally complex steps which cannot be performed in the human mind as determining how to generate scores made by an AI model is already a complex task.

[0051] FIG.1 illustrates an example process for generating concept images for each class, according to embodiments described herein. The process 100 of FIG.1 includes selecting concepts for each class of interest at 102. For example, an expert or person with particular domain knowledge may select concepts for each class of interest 102 for diseases or conditions which can be diagnosed from OCT images of a patient. To continue the example, assume four concept classes of interest: Diabetic Macular Edema (DME), Choroidal Neovascularization (CNV), Drusen, and normal. Domain knowledge may be used as user input to crop / extract from input images of each concept class (from 102) to generate a concept image at 104. For example, a doctor may be provided with a set of images associated with a concept class from 102 and ask to crop or extract the portion or area of the image that is most important to the image being classified as a certain concept (e.g. as DME or CNV, etc.,). The cropped / extracted portions of input images may be used to generate concept images for each class at 106.

[0052] In embodiments, generating the concept images for each class at 106 may include padding each image (e.g. with OCT image background) to match the size of the original image (104) or the input image size for each model that will execute the classifying, such as an image classifier model. In some instances, preserving the aspect ratio between the concept images may help in utilizing some concept classes. In some embodiments, synthetic concept images may be generated and used for obtaining coefficients of a concept class associated with an image class by training a linear classifier using the synthetic images. For example, generative adversarial networks (GANs), variational autoencoders (VAE), DALL-E, stable diffusion, or other suitable algorithms may be provided, as input, a training dataset of images, and generate the concept images for each class of interest. The generative models describedAttorney Docket No.515324 (Client Ref.24049PCT) above may be trained using concept images to generate any number of concept images for each class of interest. Using synthetically generated concept images can help increase the dataset size and add more variations as needed. The generative models may also be used to generate subsets of concept images. For example, in the case of DME there may be concept images of small cysts, very small cysts, large cysts, etc. These types of subsets can also be generated absent the use of synthetic concept images as well. In embodiments, the activations of synthetic concept images and the activations of synthetic random images may be provided as input to train a linear classifier that is trained to separate the concept activations from the random activations to obtain the coefficients of the concept class or for each concept class. In embodiments, the concept images may be generated by extracting from a test dataset, other dataset, or from a training dataset of images (if the cropped region is very small compared to the original image), portions from each image of the training dataset of images based on domain knowledge provided by users, where the concept images are associated with a concept class. For example, the training dataset may be used in scenarios where the cropped region is very small as compared to the whole or entire image.

[0053] FIG.2 illustrates examples of images (mammogram images) with a score (0-6) for use in categorical grading like in Breast Imaging Reporting and Data System (BI-RADS) grading of mammogram images For example, a score of 0 may indicate a lack of presence or detection of a disease or condition in the mammogram image, whereas a score of 6 may indicate the presence of a certain type of disease(s) or condition(s) such as breast cancer, cysts, or tumors. However, the score generated according to a BI-RADS grading does not capture nuances such as the continuous score of the present disclosure captures, as described herein.

[0054] FIG.3 illustrates an example conventional process of segmentation algorithms outlining retinal layers in an OCT image, used for thickness measurements. The OCT image depicted in FIG.3 includes several segmented layers determined by an AI segmentation model. AI image segmentation models, for example in diabetic retinopathy, can segment out the cystic spaces and calculate the area of fluid accumulation in a two-dimensional image, such as the image depicted in FIG.3. When a stack of such OCT scans (also referred to as three-dimensional scans) is analyzed, the volume of the fluid contained in the retinal cyst can be estimated. With manual segmentation of these cystic spaces being a tedious task, AI model segmentation can be used to ease the approach. This AI model segmentation gives anAttorney Docket No.515324 (Client Ref.24049PCT) approximate assessment of the imaged fluid depending on the efficiency of the algorithm implemented by the AI model but it is limited to the visible boundaries of the area of interest. Therefore, the size and shape of the retinal cysts segmented by the AI model can vary in its application and it potentially excludes the surrounding regions to the area of interest, and fails to capture or analyze the total disease present. Therefore, segmentation cannot be taken as a precisely accurate measurement to detect small changes in cysts included in OCT images or other images. Further, a variance in the signal strength of the laser light when passed through a targeted / scanned area is often observed during OCT imaging as a result of increased light scattering across the scanned area (ex. retinal tissue). This effect can potentially obscure phenotypic features or details of the targeted area, thus leading to inaccuracies when training segmentation models or contribute to inaccurate measurements of the scanned area (i.e. retinal thickness). Although current algorithms attempt to mitigate these inaccuracies, the use of interpolations or extrapolations by these segmentation methods may still not yield accurate results. Another disadvantage of the segmentation approach is that only the cystic areas and sub-retinal fluid can be segmented and approximately quantified, while segmentation of diffuse retinal thickness and its quantification is not currently feasible. Further, current segmentation methods cannot directly segment out the scanned area or extract the volume of diffuse tissue thickness (i.e. retinal tissue).

[0055] FIG.4 illustrates an example conventional process of segmentation algorithms outlining fluid filled cysts which serve as a demarcation to estimate fluid volume. For example, the OCT images depicted in FIG.4 which include colored outlines 400 and 402 of fluid filled cysts, are examples of how conventional segmentation algorithms outline the fluid filled cysts which are a demarcation that is used to estimate fluid volume. However, these outlines determined by segmentation algorithms and fluid estimates are inaccurate. For example, small cysts may be missed because of the inherent architecture of the segmentation algorithms. This fact, and even the inaccurate boundaries of the detected cysts, make the assessment of the total fluid collected in the cysts inaccurate. So, neither retinal thickness (as discussed above with reference to FIG.3) nor the volume of fluid in cysts provide an accurate metric for quantification of edema in an image such as an OCT image. Both conventional methods fail to capture the pattern and its changes in the retina. The scores generated by the embodiments of the present disclosure can capture the pattern and its extent based on the template provided by the coefficients of the linear classifier in the defined feature space, thereby providing a robust measure to quantify macular edema / disease.Attorney Docket No.515324 (Client Ref.24049PCT)

[0056] FIG.5 illustrates an example graph representing scores generated by features of the present disclosure over time as well as heatmaps for the associated scores in the graph over the period of several visits for treatment of Edema, according to embodiments of the present disclosure. For example, the blue line in the graph of Edema scores, which refer to the scores generated by the models of the present disclosure which indicates the presence or lack thereof of a disease or condition, may represent analysis of OCT image(s) from a patient prior to receive any treatment for DME. The green line for visit 2 may represent the scores generated from an analysis of OCT image(s) from the same patient after receiving one injection of anti-vascular endothelial growth factor (VEGF). The red line for visit 3 may represent the scores generated from an analysis of OCT image(s) from the same patient after receiving five injections of anti-VEGF. The heatmaps below the graph depict the concentration of the scores for each visit, which visualizes the location of edema severity at each visit. In embodiments, the score generated by the computer system implementing features of the present disclosure indicates a lack of presence of a disease or condition in an image when the score is very low (but still positive), zero, or negative. The score may also indicate the presence of another disease or condition in the image when the score is zero or negative and it may also still be quantitative.

[0057] FIG.6 illustrates example user interface presentations of scores generated by features of the present disclosure which are scaled for use in mammogram images, according to embodiments of the present disclosure. In embodiments, a computer system implementing the AI model(s) of the present disclosure may be configured to generate, transmit, and update a user interface enabling a user, such as a doctor, to provide images, such as mammogram images, as input to the model for generating a score. For example, on the left side of FIG.6 a doctor may upload an image, as depicted at 600, and the system may return a score (“Mammogram Score”) of -4.14 representing an absence of any diseases or conditions that the model has been trained to detect when analyzing a mammogram image. On the right side of FIG.6, the user may upload a different image, as depicted at 602, and the system may return a score (“Mammogram Score”) of 8.59 representing a presence or indication with quantification of a certain disease or condition that the model has been trained to detect when analyzing a mammogram image.

[0058] FIG.7 illustrates an example process 700 for obtaining concept activation vectors / coefficients for each concept class, according to embodiments of the presentAttorney Docket No.515324 (Client Ref.24049PCT) disclosure. The process 700 includes obtaining concept images of classes the model (AI model / image classifier model) was trained to predict at 702. In an exemplary embodiment, a way to obtain the activations of images of each concept class includes giving the linear classifier (718) the activations of the images of the concept class under consideration (704 or 712) and use the activations of the images belonging to the other (remaining) classes as random activations. The table below depicts an example of the concepts generated using the above described process. Concept Activations Random Activations DME CNV, Drusen & Normal CNV DME, Drusen & Normal Drusen DME, CNV & Normal Normal DMA, CNV & Drusen Table 1

[0059] The process 700 includes multiple iterations where one part of the process 700 iterates through one particular concept class under consideration (e.g.704-710) and the other part of the process 700 iterates through the other remaining concept classes (e.g.706 and 712-716). For example, if 704-710 were iterating for the concept class DME, the process 700 for 706 and 712-716 would iterate through the remaining concept classes CNV, Drusen, and Normal. This iterative process loop is repeated until each concept class is the one under consideration and the remaining classes are provided through 706 and 712-716. The process 700 includes providing the concept images of one class under consideration 704 (e.g. input is concept images for a particular class under consideration) to an AI model or image classifier model 706. The process 700 includes extracting activations 708 to generate or obtain concept activations 710 for the class under consideration 704. An example of the model 706 includes ResNet18®. As used herein, the term “activations” includes a general term that applies to outputs from any layer of a CNN model, like ResNet18®. The feature maps described herein may be referred to as activations that are of a vector size of 512x7x7. The feature maps from the last CNN layer are provided to the global average pooling (GAP) layer of the model 706Attorney Docket No.515324 (Client Ref.24049PCT) where the GAP layer converts the 512x7x7 vector to 512x1 vector. This output is also referred to as activations. The embodiments of the present disclosure include extracting activations from a last convolutional layer (before the GAP layer) for any new image given to the image classifier model (e.g. model 706) for prediction. In embodiments, extracting activations 708 from a GAP layer for concept images includes executing or invoking functions of open source deep learning frameworks (e.g. PyTorch) which are defined to extract activations (e.g. vectors) from a specified layer (the GAP layer in this scenario) of the model 706. This function may be invoked for concept images. The activations extracted from the concept images are referred to as the concept activations 710. The concept activations belonging to the classes other than the class under consideration become the random activations. These are then both used to train a linear classifier 718 that learns to separate the concept activations 710 from the random activations 716. For each class of concept images and their activations, the rest of the concept images and their respective activations become random activations. In embodiments, another function call may be invoked or executed to extract the feature maps from the model 706 such as PyTorch’s hook function.

[0060] As described above, concept images of the remaining classes (mix of concept images of the remaining classes) 712 (e.g. input is random images from the remaining concept classes) are provided to the same model 706 to extract activations 714 from this image set and generate or obtain random activations 716. In embodiments, the activations extracted 708 and 714 may be extracted from a last convolutional layer and / or a GAP layer which is just after the last convolutional layer of the model 706, from one or more convolutional layers (layers) of the model 706, derived from an averaging of the layers of the model 706, or some other mathematical operation, or each convolutional layer of the model 706 may be used for a particular class under consideration 704. In some embodiments, the computer system implementing the features of the present disclosure may back-propagate the gradients of the model 706 to feature maps of the last convolutional layer of the model 706 and implement global average pooling to obtain activations 708 and 714.

[0061] The process 700 includes training a linear classifier to separate concept activations 710 from random activations 716 at 718. Although FIG.7 depicts using a linear classifier to separate concept activations 710 from random activations 716 at 718 – other processes, methods, or models may be used including a Convolutional Neural Network (CNN), a Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), Elastic Net, or RidgeAttorney Docket No.515324 (Client Ref.24049PCT) regression, or other suitable techniques which can distinguish between positive and negative examples based on the extracted activations (710 and 716) to obtain the coefficients / weights (e.g.720). The trained linear classifier at 718 may be used to obtain coefficients of the concept for the class under consideration at 720 of the process 700. As described above, the process 700 is repeated or iterated through for all concepts under consideration, represented at 722 which results in obtaining coefficients for each concept class at 724 of FIG.7. In embodiments, the coefficients of the concept classes 724 are updated by updating or providing updated concept images of the concept classes to the trained linear classifier 718 as part of a feedback loop.

[0062] FIG.8 illustrates an example process 800 for training a retinal OCT classifier model and extracting activations from the classifier model, according to embodiments of the present disclosure. The process 800 of FIG.8 includes obtaining training images 802. Examples in the present disclosure utilize retinal OCT images or mammogram images as the training images for training a classifier model. Examples of the classifier (Classifier Model 806) may include ResNet18®, CNNs, Recurrent Neural Networks (RNNs), Long Short-Term Memory Networks (LSTMs), Transformers, Autoencoders, Principal Component Analysis (PCA), SVMs, Gaussian Mixture Models (GMMs), Hidden Markov Models (HMMs), or Decision Trees. The process 800 includes pre-processing the training images at 804. Pre- processing the training images at 804 may include removing noise or outliers from the training images or using previously annotated or labeled images to train the Classifier Model 806. Pre-processing the training images at 4804 may include resizing the images to a size appropriate for the given classifier model (for example resizing to 224x224 for ResNet18®), implementing random rotation, horizontal and vertical flips, contrast variations, or other pre- processing steps to augment the training images 802 to obtain a robust trained model 806.

[0063] The process 800 includes training the Classifier Model at 806 using the pre- processed training images from 804 to classify images and / or generate a score that indicates a presence or lack thereof of a condition or disease based on the images. The process 800 includes evaluating the output of the model at 806 and tuning the hyperparameters of the model at 808. For example, training the model 806 may involve multiple feedback loops to update the weights or hyperparameters of the model at 808 until a best trained model is obtained at 810. The process 800 of FIG.8 also includes using the best trained model 810 to classify a concept included in provided images at 812. Once the model has generated aAttorney Docket No.515324 (Client Ref.24049PCT) classification for the image, the activations may be extracted at 814. In some embodiments, the activations may be extracted 814 prior to batch normalization is executed for the model 810 or for the convolutional layers of the model 810. For example, feature maps are typically extracted after batch normalization, but they may be extracted before it as well. The output of each layer of the model (806) (also called activations) becomes the input to the next layer. Instead of using a single image as the input to the model during training, a small batch of images are used together. Therefore, each layer’s output is in the form of batches of activations. These activations are normalized by subtracting the batch mean and dividing by batch standard deviation. This helps to accelerate training of the deep neural networks like ResNet18. The feature maps output from the last convolutional layer also under-go this process of batch normalization before they are given to the Global Average Pooling layer. During the inference phase, when only a new single image is given to the model to predict, the learned features during model training are present in the batch normalization layer. So, feature maps before this layer do not have these parameters applied to them, and after going through batch normalization layer the feature maps have these parameters applied to them. Thus, there is a small difference between the two feature maps. At this point the activations may be extracted from the last convolutional layer for the purposes of the embodiments of the present disclosure, either before batch normalization or after batch normalization.

[0064] FIG.9 illustrates an example process 900 for obtaining coefficients of a linear classifier as well as generating a score indicating a presence and quantification of a disease or condition using extracted activations of a newly provided image, according to embodiments of the present disclosure. As described herein, a trained model 902 may be provided with a new image or input image 904 for classifying a concept class in the image 904 in order to obtain activations (512x1) from a GAP layer 906 of the trained model 902 for generating a score that indicates a presence or lack thereof of disease or condition. For example, the trained model 902 may be an image classifier model such as ResNet18®, the input image 904 may be an OCT image for a patient, and the activations extracted from the GAP layer 906 of the trained model 902 may be used to generate the score. The trained model 902 may be trained in accordance with the process described above with reference to FIG.8. FIG.9 depicts two branches or paths of the process 900: one branch or path 908 is directed to obtaining coefficients of the linear classifier (512x1) 910, similar to the process described with reference to FIG.7; while the other branch or path 912 is directed to obtaining the scores (directional derivatives / CAV 914) for the new image 904. Although FIG.9 depicts extractingAttorney Docket No.515324 (Client Ref.24049PCT) activations from a GAP layer 906 of the model 902 to obtain the new image activations 916, embodiments of the present disclosure are not limited to just this layer, and activations may be obtained from one or more convolutional layers of the model 902 that then undergo a global averaging or some other form of normalization or mathematical operation to obtain the activations 916 of the model 902. In some embodiments, feature maps may be extracted before or after batch normalization, and converted separately into a 512,1 vector using average pooling or max pooling and other pooling methods instead of extracting the average pooled feature maps from the GAP layer.

[0065] As described herein, embodiments of the present disclosure may apply images to a model, such as model 902, to quantify a magnitude of the directional derivatives (914) of a concept in relation to a decision boundary of a linear classifier or support vector machine (SVM) to generate a scaler, numerical, or continuous value that us used to score or grade such images. The model or models implemented by a computer system implementing the features of the present disclosure may determine CAVs by applying the activation vectors of an image 916 to the coefficient of the linear classifier or SVM 910. The implementation of the coefficient of linear classifier or SVM 910 may transform the activations 916 into a directional derivative 914. The transformation of the activations 916 into a directional derivative 914 may yield a dot product. For example, the dot product may serve as a projection of the vector onto a direction assigned by the coefficient of the linear classifier or SVM 910. The dot product may be assigned to a scaler, numerical, or a continuous value (score). The sign of the scaler, numerical, or continuous value of the dot product may indicate the side of a decision boundary of the linear classifier or SVM 918 the dot product falls. The sign of the scaler, numerical, or continuous value of the dot product may indicate the class of the concept. In embodiments, the magnitude of the scaler, numerical, or continuous value of the dot product may indicate how far from the decision boundary of the linear classifier or SVM 918 the dot product is. The magnitude 920 of the scaler, numerical, or continuous value of the dot product may indicate the extent the model 908 may be indicating the presence of the concept of the specific class. The greater the magnitude 920 of the scaler, numerical, or continuous value of the dot product may indicate the greater the distance of the dot product is from the decision boundary of the linear classifier or SVM 918.

[0066] The greater the distance of the dot product is from the decision boundary of the linear classifier or SVM 918 may indicate the severity or advanced stage of an indication of aAttorney Docket No.515324 (Client Ref.24049PCT) disease or condition. The smaller the distance of the dot product is from the decision boundary of the linear classifier or SVM 918 may indicate a lower stage or the lack of an indication of a disease or condition. In accordance with embodiments of the present disclosure, the magnitude 920 of the scaler, numerical, or continuous value of the dot product may be used as a score of the indication’s specific concept or the extent of the concept’s presence in the image 904. The greater the magnitude 920 the dot product may indicate the confidence of the model 902 in prediction of grade and diagnosis of a disease or condition present in the image 904. The greater confidence of the model 902 may indicate the greater extent the specific concept is within that image 904. The magnitude 920 of the scaler, numerical, or continuous value of the dot product may be used as a score to grade an indication or presence of a disease or condition within image 904. In embodiments, the magnitude 920 and the sign of the assigned score may be concept specific (e.g. to a concept class, such as a concept class for an image). The score may change between a positive to negative value, and it may change in quantity. A change of the score between a positive to negative value may indicate the presence or development of one or more new concepts identified or detected within the image 904. For example, when the model, the class, the defined concept with concept and random images, their activations, the linear classifier, its coefficients, are all constants – a variable is the new image given to the model. This results in the magnitude of the directional derivative that is calculated by the current system which quantifies the defined concept in the image.

[0067] In embodiments, the model(s) of the present disclosure may be trained to classify various classes of indication represented in images, such as image 904. The model may be trained by a user or domain expert using data from a dataset of multiple images. The multiple images used for training may contain the desired concept class, alternative classes, or random images such as in the description of FIG.7 when describing the training of the linear classifier 718. The user or domain expert may elect the concept class whose CAV and TCAV scores and dot products are relevant to the defined class the user or domain expert wishes to determine. In embodiments, the user or domain expert may use uniform or standard concept images. The user or domain expert may fine tune the prediction or classification of the model by changing or altering the concept images or increasing the number of concept images used with the linear classifier or SVM 918. In some embodiments, the model may be trained with synthetic concept images. Synthetic images used to train the model may be used solely or to augment the training of the concept and random classes as described with reference to FIG.7.Attorney Docket No.515324 (Client Ref.24049PCT) In embodiments, the synthetically generated images of concepts may be used to fine tune the linear classifier or SVM 918.

[0068] Features of the present disclosure may grade various indications of a presence or lack thereof of a disease or condition from images. Embodiments of the present disclosure may grade diagnoses of various text, symbols, or non-images. The images of an indication may include diseases of the eye (ex. Ophthalmologic indications), breast (ex. breast cancer), lungs (ex. COVID), or other organ systems. The images of an indication may be acquired from other imaging modalities that include but are not limited to ocular coherence tomography (OCT) images, fundus images, and ultrasound images. The images of an indication may include images that include but are not limited to images obtained from computed tomography (CT), magnetic resonance imaging (MRI), X-ray imaging (ex. mammogram), histopathology, non-medical imaging, or other like methodologies.

[0069] In embodiments, the Ophthalmologic indications the present disclosure may grade include, but are not limited to, edema or Diabetic Macular Edema (DME), edemas from central and branch vein occlusions, age-related macular degeneration including but not limited to Choroidal Neovascularization (CNV), Drusen, macular holes, macular traction, or other like indications, retinal atrophy, lesions of the retina, optic nerve diseases, such as papilledema, optic disc changes and other like indications. Indications of a presence or lack thereof of a disease or condition that the present disclosure may classify, or grade include, but are not limited to, cancer (ex. breast cancer), hyperplasia, hemorrhages, lung opacities (ex. COVID), and other like indications. The model(s) of the present disclosure may use concepts to classify or grade the indications in images as normal. The concepts detected and classified by the present disclosure of various indications of the eye may have a concept specific to the indication which the model may rely upon to make its prediction. For example, such concepts may include, but are not limited to, macular edema, cyst like spaces, or a honeycomb appearance. In scenarios where the present disclosure may use the concept of retinal edema, and that concept may include the fluid accumulation and the diffuse retinal thickness. For clarity and as an example, a retinal OCT image predicted diagnosis as diabetic macular edema, would contain concepts such as cystic spaces or honeycomb appearances or the pattern of increased diffuse retinal thickness.

[0070] As an example, features of the present disclosure which analyze OCT images of diabetic macular edema may indicate the magnitude of the scaler, numerical, or continuousAttorney Docket No.515324 (Client Ref.24049PCT) value of the dot product is greater with more edema concepts present and less with concept of mild or early staged edema. To continue the example, OCT images may also indicate the magnitude of the scaler, numerical, or continuous value of the dot product is lowest with normal images with little to no edema concepts in the images. Embodiments of the present disclosure may detect small changes in the magnitude of the scaler, numerical, or continuous value of the dot product. For example, the small changes in the magnitude of the scaler, numerical, or continuous value of the dot product may correlate to small changes in the quantity of a concept specific to an indication of a presence or disease or condition. For example, small changes in the magnitude of the scaler, numerical, or continuous value may correspond to a positive effect (in case of a reduction of score) from an applied treatment to the patient, or an increase (in case of a lack of response to treatment). In embodiments, the magnitude of the scaler, numerical, or continuous value of the dot product is a score that quantifies a concept specific to an indication.

[0071] In embodiments, the model(s) of the present disclosure may assign scores as a positive or negative value at or between 0 to 1.5 for “normal” images (lacking in an indication or presence of a disease or condition under consideration for the images). The model(s) of the present disclosure may assign scores at or slightly greater than 1.5 for a mild form or stage of an indication (denoting a detection of a presence of a disease or condition under consideration for the images). The model(s) of the present disclosure may assign scores greater than 1.5 or at increasing increments as the indication increases in severity. It should be noted that the values described above are merely examples and other values, scores, ranges, or grades may be used to denote different grades, stages, or forms of a disease or condition. The scores generated by embodiments of the present disclosure may be monitored to detect changes, even small changes, which may indicate a return of a disease or condition or an effect that a certain treatment is having on the disease or condition. For example, the scores may be used to monitor an indication’s progression, recurrence, regression, alleviation, and like changes. A change in the score may show a patient’s response to a treatment with an indication, and changes in the score may also indicate changes in the context and expansion or reduction of a concept or indication overtime. This normal value can be changed or fine- tuned by changing the model architecture, the concept and random images used, and even the entire model (which would require retraining).Attorney Docket No.515324 (Client Ref.24049PCT)

[0072] In embodiments, a score may be assigned to a specific location or area in an image. For example, the score may be presented or generated as a heat map of a specific location or area in the retina as depicted by the position of the scan. The heat map may allow the score and indication-specific concepts of the targeted area of an image to be visualized, reviewed, or assessed via an associated user interface or display by a user or doctor. The score may be generated and presented as a change map or the amount of change in a specific location or area in the image. This enables presentation of a change map or the amount of change occurring over a period at a specific location or area in the image. Combining heat maps from horizontal and vertical scans can also provide a better visualization of edema severity in the retina.

[0073] In embodiments, the model 902 may include machine learning models, classification models, ResNet18, ResNet50, or other versions of ResNet or modifications or other deep learning models such as VGG-16, mobileNet, Inception-V3, GoogLeNet etc., to model architecture listed herein, other Convolutional Neural Networks (CNN) models, transformers, language models, and other like models not listed herein. The architecture of such systems or models may be modified with implementation of varying numbers, batch normalization, activation functions, or other like modifications.

[0074] As described with reference to FIG.7, concept images 922 may be used by a model, such as model 902, to generate concept activations (512x1) 924 that are then used to train the linear classifier or SVM 918. In embodiments, the present disclosure implements an enhanced quantification algorithm that may be adaptive to architectural changes in the model. For example, the model 902 may be a Convolutional Neural Network (CNN) model. In embodiments, the present disclosure enables implementation of small sized filters or kernels to detect or extract fine or in-depth details of concept features in an image. For example, the model(s) implemented by the computer system of the present disclosure may extract concepts from a lesser number of pixels, such as 128 kernels of 3x3. The use of small sized filters or kernels may enable the detection of changes of concept, such as changes in the density of tissue. In embodiments, the present disclosure enables implementation of large sized filters or kernels to detect or extract concept features from a wider area in an image. For example, the model(s) implemented by the computer system of the present disclosure may extract concepts from a larger number of pixels, such as 128 kernels of 7x7. The use of large filters or kernels may aid in capturing the context and expansion or reduction of a concept over time.Attorney Docket No.515324 (Client Ref.24049PCT)

[0075] The number of filters or kernels may also vary depending on the use case. In embodiments, the user may increase or decrease the number of filters used by providing input to the computer system implementing the features of the present disclosure. In embodiments, the coefficients of the linear classifier 910 act as a template / scale / basis which is used to measure a concept in a given image, such as Edema in an OCT image for a patient. This scale is defined by the concept images, and the location to begin the measurement in a positive or negative direction is decided by the decision boundary. As such, the scale can be changed, modified, or adapted by changing the selected concept images 922 or by changing the model 902 – by changing its architecture such as by using an extra layer of 128x7x7 kernels thereby providing a tunable scale. As the linear classifier is trained 918 the coefficients 910 of the classifier 918 determine the decision boundary within a subset space of the feature / activation space of the model 902. This subset space is defined by the concept image activations 924. In embodiments, the scale can also be changed, modified, or adapted by changing the model’s 902 architecture by adding other layers in between some of the existing layers or adding multiple layers to modify the coefficients. The directional derivatives 914 project the activations 916 obtained from the image 904 onto a concept activation vector in a direction of the concept. For example, the greater the presence of Edema in an image, the greater the magnitude of the directional derivative 914. A positive sign of the score 920 / directional derivative 914 confirm that, for example, the score is on the Edema concept side of the decision boundary. The coefficients in 910 can act as a template / scale / basis which is used to measure a given concept present in a given image. In embodiments, the computer system implementing the present disclosure may use multiple coefficients, where each coefficient corresponds to a different concept, like Edema, CNV, and Drusen. By analyzing a given image, such as image 904, using multiple coefficients, the system is able to quantify how much of each concept is present in that image. The generated scores for each concept can be used, or provided as input, to a natural language model to generate a report for an image using a rule-based prompt. The quantification of each concept in the image could help doctors make decisions, monitor responses to treatments closely, and explain their decisions to patients (provide explainability for the AI model or machine learning model). For example, a positive sign of a score generated by the system of the present disclosure indicates a presence of a disease or condition, such as edema, whereas the quantification of the concept or disease or condition is denoted by the magnitude. An example of a rule-based prompt includes: positive scores for CNV, edema, and Drusen concepts indicate macular degeneration. Positive edema scores and negative CNV and Drusen scores indicate onlyAttorney Docket No.515324 (Client Ref.24049PCT) Macular edema. To continue the example, a positive edema score may indicate edema but this may also be associated with CNV. A negative CNV score and negative Drusen score is required to confirm the presence of pure edema without associated age related macular degeneration in this scenario. In embodiments, the scores generated by the system of the present disclosure may be scaled or tuned by modifying the architecture of the image classifier model 902. For example, prior to extracting activations from the global average pooling layer of the image classifier model 902, a new layer may be added to the image classifier model 902 that includes 128x7x7 kernels, or a first layer of the image classifier model 902 may be updated to use a different kernel amount than a previously used kernel amount, or a layer may be delated from the image classifier model 902, or one or more layers of the image classifier model 902 may be replaced. Averaged activations from a combination of modified image classifier models and / or original (unmodified) image classifier models may also be used. For example, two or more models with different additions or modifications of layers may have their activations extracted and these can be averaged or a calculated weighted average of these activations can be used as the image activations 916.

[0076] FIG.10 illustrates an example flowchart for generating scores indicating a presence and quantification of a disease or condition made by an image classifier model, according to embodiments described herein. FIG.10 includes an exemplary process 1000 which may be performed by an environment or architecture such as in FIGs.1-9 and 13, and by systems and components presented in or described with reference to of FIGs.1-9, and 13. However, it will be recognized that any of the following blocks may be performed in any suitable order and that the process 1000 may be performed in any environment or architecture and by any suitable computing device and / or controller.

[0077] At step 1002, the process 1000 includes extracting activations from a global average pooling layer of the image classifier model, the activations generated by the image classifier model in response to being provided as input an image and computing as output a classification of an image class for the image. As an example, the image classifier model may be ResNet18® which includes four layers. In such a scenario, a computer system implementing the features described herein, may be configured to extract four feature maps, each from a different convolutional layer of the image classifier model. An average feature map or weighted average feature map may be generated from the four feature maps in some instances. The image classifier model may include one of a Convolutional Neural NetworkAttorney Docket No.515324 (Client Ref.24049PCT) (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory Network (LSTM), Transformer, Autoencoder, Principal Component Analysis, Support Vector Machine (SVM), Gaussian Mixture Models (GMM), Hidden Markov Models (HMM), or Decision Trees. In embodiments, feature maps from each layer of the model may be extracted and converted into one dimensional vectors. In such a scenario, their dimensions must be matched for addition and averaging. For example, the vector from the last layer may be 512x1 and from a previous layer is 256x1. Some mathematics such as padding, truncation, interpolation, etc., may be used for this matching. As the vectors are weighted, the value of the important 512x1 vector may be kept as desired by applying these mathematical operations or methods.

[0078] At step 1004, the process 1000 includes obtaining coefficients of a concept class associated with the image class from a trained linear classifier. In embodiments, the computer system may provide the concept images of the concept classes associated with the image class and the random images associated with the image class as input to the model 702, to obtain the concept and random activations, which are given to a linear classifier that is trained to separate concept activations from random activations in order to obtain the coefficients of the concept classes. In some embodiments, the computer system may obtain the coefficients of the concept classes by providing the concept images of the concept classes associated with the image class and random images associated with the image class as the input to one or more of a support vector machine (SVM), a trained regression model, and / or a Ridge Regression technique. In embodiments, the concept images may be provided to the image classifier model for obtaining activations from a Global Average Pooling (GAP layer) that is after a list convolutional layer of the image classifier model. The activations of the concept images or concept activations are then provided as input to the linear classifier that learns to separate the concept activations from the random activations.

[0079] At step 1006, the process 1000 includes generating a directional derivative, as a score, by multiplying the coefficients and the activations to generate a dot product. In embodiments, the computer system implementing the features of the present disclosure may generate a recommended treatment plan based on the score being positive. In some embodiments, the activations are also each associated with a probability that corresponds to a given image class of a plurality of image classes. The system implementing the features of the present disclosure may generate a plurality of directional derivatives, as scores, by multiplying, for each directional derivative of the plurality of directional derivatives, aAttorney Docket No.515324 (Client Ref.24049PCT) particular set of coefficients out of all the sets of coefficients and the activations to generate an associated dot product, wherein each particular set of coefficients out of all the sets of coefficients correspond to a different concept class. For example, the particular set of coefficients may correspond to each linear classifier whereas all the sets of coefficients may refer to the coefficients of all linear classifiers. The computer system may also provide the plurality of directional derivatives as input to a natural language machine learning model that is trained to generate, as output, a report indicating the quantification of the disease or condition in the image derived from the plurality of directional derivatives. In embodiments, the coefficients may be normalized or modified prior to being multiplied with the activations to generate the dot product.

[0080] FIG.11 illustrates example Optical coherence tomography (OCT) images with generated scores using features of the present disclosure which indicate the presence or lack of presence of a certain disease or condition, in FIG.11, the presence or lack of presence of Macular Edema (DME), according to embodiments of the present disclosure. For example, at the top of FIG.11, the OCT images and generated scores 1100 are associated with an absence of a disease or condition (DME in this scenario) as indicated by scores of 0.29 and 0.0. However, at the bottom of FIG.11, the OCT images and generated scores 1102 are associated with an indication or presence of DME and its quantity as indicated by scores of 3.26 and 4.53, where the higher the score the more presence of the condition or disease is present in the associated image.

[0081] FIG.12 illustrates example OCT images with generated scores using features of the present disclosure which indicate the presence and quantity or lack of presence of a certain disease or condition (DME in FIG.12) before and after treatment, according to embodiments of the present disclosure. As depicted in FIG.12, prior to treatment the high score of 10.12 indicates a strong presence of DME detected in the associated OCT image. In comparison, post-treatment the lower score of 2.26 indicates a presence of DME but to a lower degree which indicates to clinicians and doctors that the treatment is reducing the DME present in the associated patient.

[0082] FIG.13 illustrates a simplified block diagram of one or more devices or systems for generating scores indicating a presence and quantification of a disease or condition made by an image classifier model according to embodiments of the present disclosure. FIG.13 is a block diagram of an exemplary system or device 1300 that can be representative of eachAttorney Docket No.515324 (Client Ref.24049PCT) computing system disclosed herein. The system 1300 includes a processor 1304, such as a central processing unit (CPU), and / or logic, that executes computer executable instructions for performing the functions, processes, and / or methods described herein. In some examples, the computer executable instructions are locally stored and accessed from a non-transitory computer readable medium, such as storage 1310, which may be a hard drive or flash drive. Read Only Memory (ROM) 1306 includes computer executable instructions for initializing the processor 1304, while the random-access memory (RAM) 1308 is the main memory for loading and processing instructions executed by the processor 1304.

[0083] The network interface 1312 may connect to a wired network or cellular network and to a local area network or wide area network. The system 1300 may also include a bus 1302 that connects the processor 1304, ROM 1306, RAM 1308, storage 1310, and / or the network interface 1312. The components within the system 1300 may use the bus 1302 to communicate with each other. The components within the system 1300 are merely exemplary and might not be inclusive of every component for embodiments described herein. For instance, in some examples, the system 1300 might not include a network interface 1312. In embodiments the system 1300 may include one or more components such as input-output devices which can enable electronic, optical, magnetic, and holographic, communication with ROM 1306, RAM 1308, and / or storage 1310. The input-output devices can enable wireless communication via WiFi®, Bluetooth®, cellular (e.g., LTE®, CDMA®, GSM®, WiMax®, NFC®), GPS, and the like. The input-output devices can include wired and / or wireless communication pathways and may be configured to present or otherwise display predictions, updated predictions, images, attention-guided images, or generated recommendations for treatment plans associated with updated diagnosis of a disease or condition.

[0084] While the disclosure has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. It will be understood that changes and modifications may be made by those of ordinary skill within the scope of the following claims. In particular, the present disclosure covers further embodiments with any combination of features from different embodiments described above and below. Additionally, statements made herein characterizing the disclosure refer to an embodiment of the disclosure and not necessarily all embodiments.Attorney Docket No.515324 (Client Ref.24049PCT)

[0085] The terms used in the claims should be construed to have the broadest reasonable interpretation consistent with the foregoing description. For example, the use of the article “a” or “the” in introducing an element should not be interpreted as being exclusive of a plurality of elements. Likewise, the recitation of “or” should be interpreted as being inclusive, such that the recitation of “A or B” is not exclusive of “A and B,” unless it is clear from the context or the foregoing description that only one of A and B is intended. Further, the recitation of “at least one of A, B and C” should be interpreted as one or more of a group of elements consisting of A, B and C, and should not be interpreted as requiring at least one of each of the listed elements A, B and C, regardless of whether A, B and C are related as categories or otherwise. Moreover, the recitation of “A, B and / or C” or “at least one of A, B or C” should be interpreted as including any singular entity from the listed elements, e.g., A, any subset from the listed elements, e.g., A and B, or the entire list of elements A, B and C.

Claims

Attorney Docket No.515324 (Client Ref.24049PCT) CLAIMS What is claimed is:

1. A computer-implemented method for generating scores indicating a presence and quantification of a disease or condition made by an image classifier model implemented by one or more processors, the method comprising: extracting, by a computer system, activations from a global average pooling layer of the image classifier model, the activations generated by the image classifier model in response to being provided as input an image and computing as output a classification of an image class for the image; obtaining, by the computer system, coefficients of a concept class associated with the image class from a trained linear classifier; and generating, by the computer system, a directional derivative, as a score, by multiplying the coefficients and the activations to generate a dot product.

2. The computer-implemented method according to claim 1, wherein the score indicates the presence and quantification of the disease or condition in the image when the score is positive, and wherein the score indicates a lack of presence of the disease or condition in the image when the score is zero or negative, or wherein the score indicates the presence of another disease or condition in the image when the score is zero or negative.

3. The computer-implemented method according to claim 2, further comprising generating, by the computer system, a recommended treatment plan based on the score being positive.

4. The computer-implemented method according to claim 1, wherein obtaining the coefficients of the concept class associated with the image includes: providing, by the computer system, concept activations of concept images of concept classes associated with the image class and random activations of random images associated with the image class as input to train the linear classifier that is trained to separate the concept activations from the random activations; and obtaining, by the computer system, the coefficients of the concept classes.

5. The computer-implemented method according to claim 4, further comprising generating, by the computer system, the concept images by extracting, from a test dataset,Attorney Docket No.515324 (Client Ref.24049PCT) other dataset, or from a training dataset of images, portions from each image of the training dataset of images based on domain knowledge provided by users, the concept images associated with the concept classes.

6. The computer-implemented method according to claim 5, further comprising obtaining, by the computer system, updated coefficients of the concept class associated with the image class by updating the concept images of the concept classes and the random images to update the training of the linear classifier.

7. The computer-implemented method according to claim 6, further comprising generating, by the computer system, an updated directional derivative, as an updated score, by multiplying the updated coefficients and the activations of a new image provided as input to the image classifier model to generate an updated dot product.

8. The computer-implemented method according to claim 1, further comprising, prior to extracting, by the computer system, the activations from the global average pooling layer of the image classifier model, adding a new layer to the image classifier model with 128x7x7 kernels, or updating a first layer of the image classifier model to use an different kernel amount than a previously used kernel amount for the first layer, or deleting a layer of the image classifier model, or replacing one or more layers of the image classifier model, or using averaged activations from a combination of modified image classifier models and / or image classifier models.

9. The computer-implemented method according to claim 8, wherein adding the new layer or updating the first layer is based on a type of machine used to generate the image or based on a type of the disease or condition without batch normalization and activation functions.

10. The computer-implemented method according to claim 9, wherein adding the new layer or updating the first layer is based on a type of machine used to generate the image or based on a type of the disease or condition with batch normalization and activation functions.

11. The computer-implemented method according to claim 1, wherein the image classifier model includes one of a Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory Network (LSTM), Transformer, Autoencoder, PrincipalAttorney Docket No.515324 (Client Ref.24049PCT) Component Analysis, Support Vector Machine (SVM), Gaussian Mixture Models (GMM), Hidden Markov Models (HMM), or Decision Trees.

12. The computer-implemented method according to claim 1, further comprising: extracting, by the computer system, feature maps from one or more layers of the image classifier model, wherein the feature maps are generated by the image classifier model in response to being provided as the input the image and computing as the output the classification of the image class for the image; and generating, by the computer system, the activations by performing global average pooling or max pooling of the feature maps to generate a one dimensional vector.

13. The computer-implemented method according to claim 1, wherein the image, prior to being provided as input to the image classifier model, is normalized or not normalized based on attributes of a machine or device used to generate the image.

14. A computer system for generating scores indicating a presence and quantification of a disease or condition made by an image classifier model, the computer system comprising one or more hardware processors which, alone or in combination, are configured to provide for execution of the following steps: extracting activations from a global average pooling layer of the image classifier model, the activations generated by the image classifier model in response to being provided as input an image and computing as output a classification of an image class for the image; obtaining coefficients of a concept class associated with the image class from a trained linear classifier; and generating a directional derivative, as a score, by multiplying the coefficients and the activations to generate a dot product.

15. The computer system according to claim 14, wherein the score indicates the presence and quantification of the disease or condition in the image when the score is positive, and wherein the score indicates a lack of presence of the disease or condition in the image when the score is zero or negative, or wherein the score indicates the presence of another disease or condition in the image when the score is zero or negative.

16. The computer system according to claim 15, further comprising generating a recommended treatment plan based on the score being positive.Attorney Docket No.515324 (Client Ref.24049PCT) 17. The computer system according to claim 14, wherein obtaining the coefficients of the concept class associated with the image includes: providing concept activations of concept images of concept classes associated with the image class and random activations of random images associated with the image class as input to train the linear classifier that is trained to separate the concept activations from the random activations; and obtaining the coefficients of the concept classes, wherein the steps further include: generating the concept images by extracting, from a test dataset, other dataset, or from a training dataset of images, portions from each image of the training dataset of images based on domain knowledge provided by users, the concept images associated with the concept classes; and obtaining updated coefficients of the concept class associated with the image class by updating the concept images of the concept classes and the random images to update the training of the linear classifier.

18. The computer system according to claim 17, further comprising generating an updated directional derivative, as an updated score, by multiplying the updated coefficients and the activations to generate an updated dot product.

19. The computer system according to claim 14, further comprising, prior to extracting the activations from the global average pooling layer of the image classifier model, adding a new layer to the image classifier model with 128x7x7 kernels, or updating a first layer of the image classifier model to use a different kernel amount than a previously used kernel amount for the first layer.

20. A tangible, non-transitory computer-readable medium having instructions thereon which, upon being executed by one or more processors, provide for generating scores indicating a presence and quantification of a disease or condition made by an image classifier model, by execution of the following steps: extracting activations from a global average pooling layer of the image classifier model, the activations generated by the image classifier model in response to being provided as input an image and computing as output a classification of an image class for the image; obtaining coefficients of a concept class associated with the image class from a trained linear classifier; andAttorney Docket No.515324 (Client Ref.24049PCT) generating a directional derivative, as a score, by multiplying the coefficients and the activations to generate a dot product.

21. The tangible, non-transitory computer-readable medium according to claim 20, wherein the score indicates the presence and quantification of the disease or condition in the image when the score is positive, and wherein the score indicates a lack of presence of the disease or condition in the image when the score is zero or negative, or wherein the score indicates the presence of another disease or condition in the image when the score is zero or negative, and wherein the instructions, upon being executed by the one or more processors, are further configured to execute the following step: generate a recommended treatment plan based on the score being positive.

22. The tangible, non-transitory computer-readable medium according to claim 20, wherein the instructions, upon being executed by the one or more processors, are further configured to execute the following step: generating a plurality of directional derivatives, as scores, by multiplying, for each directional derivative of the plurality of directional derivatives, a particular coefficient of the coefficients and the activations to generate an associated dot product, wherein each particular coefficient of the coefficients correspond to a different concept class.

23. The tangible, non-transitory computer-readable medium according to claim 22, wherein the instructions, upon being executed by the one or more processors, are further configured to execute the following step: providing the plurality of directional derivatives as input to a natural language machine learning model or rule-based model that is trained to generate, as output, a report indicating the quantification of the disease or condition in the image derived from the plurality of directional derivatives.

Images