Detecting basal cell carcinoma using reflectance confocal microscopy and dermoscopy images

Abstract

Presented herein are systems and methods for detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions. A computing system may identify, for a first subject at risk of BCC in a first lesion on a region of an epidermis: (i) a first biomedical image of an outer layer of the region on the epidermis of the first subject acquired in accordance with dermoscopy, and (ii) a plurality of second biomedical images of at least one inner layer in the region on the epidermis of the first subject acquired in accordance with reflection confocal microscopy (RCM). The computing system may apply the first biomedical image and the plurality of second biomedical images to a model architecture. The computing system may generate, based on applying to the model architecture, a score indicating a likelihood of BCC in the first lesion.

Description

DETECTING BASAL CELL CARCINOMA USING REFLECTANCE CONFOCAL MICROSCOPY AND DERMOSCOPY IMAGESCROSS REFERENCE TO RELATED APPLICATIONS[00011 The present applications claims priority to U.S. Provisional Patent Application No. 63 / 657,438, titled “Detecting Basal Cell Carcinoma Using Reflectance Confocal Microscopy and Dermoscopy Images,” filed June 7, 2024, which is incorporated herein by reference in its entirety.BACKGROUND

[0002] A computer system may apply a machine learning model on an input dataset to generate an output dataset.SUMMARY

[0003] Aspects of the present disclosure are directed to systems and methods for detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions. One or more processors may identify, for a first subject at risk of BCC in a first lesion on a region of an epidermis (i) a first biomedical image of an outer layer of the region on the epidermis of the first subject acquired in accordance with dermoscopy, and (ii) a plurality of second biomedical images of at least one inner layer in the region on the epidermis of the first subject acquired in accordance with reflection confocal microscopy (RCM). The one or more processors may apply the first biomedical image and the plurality of second biomedical images to a model architecture. The model architecture may be established using a plurality of examples. Each of the plurality of examples may identify (i) a third biomedical image of an outer layer of a region with a second lesion on an epidermis of a respective second subject acquired in accordance with dermoscopy, (ii) a plurality of fourth biomedical images of at least one inner layer in the region with the second lesion on the epidermis of the respective second subject acquired in accordance with RCM, and (iii) a label identifying of one of a presence or absence of BCC in the second lesion inthe region of the epidermis of the second subject. The one or more processors may generate, based on applying the first biomedical image and the plurality of second biomedical images to the model architecture, a score indicating a likelihood of BCC in the first lesion. The one or more processors may store, using one or more data structures, an association between the first subject and the score indicating the composite likelihood of BCC in the first lesion.

[0004] In some embodiments, the one or more processors may generate an output including information based on at least one of (i) the first biomedical image, (ii) the plurality of second biomedical images, or (iii) the association between the first subject and the score. The one or more processors may provide the output including the information for presentation. In some embodiments, the one or more processors may select, from a plurality of candidate therapies, a therapy to administer to the first lesion based on the score indicating the likelihood of BCC in the first lesion. The one or more processors may provide an output identifying the therapy selected for the first subject. In some embodiments, the first lesion may be administered with the therapy. The plurality of candidate therapies may include at least one of electrosurgery, Mohs surgery, excision surgery, radiotherapy, photodynamic therapy, cryosurgery, laser surgery, or a topical medication. In some embodiments, the one or more processors may provide an output indicating refraining of administration of therapy to the first lesion based on the score not satisfying a threshold.

[0005] In some embodiments, the one or more processors may determine a classification indicating one of presence or absence of BCC in the first lesion based on a comparison between the score and a threshold. The one or more processors may store the association between the first subject and the classification. In some embodiments, the one or more processors may apply the first biomedical image to a first machine learning (ML) model of the model architecture to determine a first likelihood of BCC in the first lesion. In some embodiments, the one or more processors may apply the plurality of second biomedical images to a second ML model of the model architecture to determine a second likelihood of BCC in the first lesion. The one or more processors may generate the score as a function of the first likelihood and the second likelihood.[0OO6J In some embodiments, the one or more processors may determine, by applying each second biomedical image of the plurality of second biomedical images to a ML model of the model architecture, a respective likelihood of BCC in the first lesion. The one or more processors may generate, based on the respective likelihood for each second biomedical image, a composite likelihood of BCC in the first lesion over the plurality of second biomedical images.[0007| In some embodiments, the BCC may include at least one of nodular BCC, superficial spreading BCC, sclerosing BCC, or pigmented BCC. The dermoscopy may include at least one of polarized light dermoscopy, contact dermoscopy, or non-contact dermoscopy. The RCM may include at least one of a handheld RCM (HH-RCM) or wide-probe RCM (WP- RCM). Each of the plurality of second biomedical images may correspond to a respective depth of a plurality of depths within the at least one inner layer in the region on the epidermis of the first subject. The outer layer and inner layer may correspond to at least one of stratum corneum, stratum granulosum, stratum basale, dermal-epidermal junction, or papillary dermis.

[0008] Aspects of the present disclosure are directed to systems and methods for training models to detect basal-cell carcinoma (BCC) in biomedical images of skin lesions. One or more processors may identify a training dataset comprising a plurality of examples, at least one example of the plurality of examples comprising (i) a first biomedical image of an outer layer of a region with a lesion on an epidermis of a subject acquired in accordance with dermoscopy, and(ii) a plurality of second biomedical images of at least one inner layer in the region on the epidermis of the subject acquired in accordance with reflection confocal microscopy (RCM), and(iii) a label identifying of one of a presence or absence of BCC in the lesion in the region of the epidermis of the subject. The one or more processors may apply the first biomedical image and the plurality of second biomedical images to a model architecture comprising a plurality of weights, to generate a score indicating a likelihood of BCC in the lesion. The one or more processors may determine at least one metric loss based on the score and the label. The one or more processors may update at least one of the plurality of weights of the model architecture using the at least one loss metric.

[0009] In some embodiments, the one or more processors may determine a classification indicating one of presence or absence of BCC in the lesion based on a comparison between the score and a threshold. The one or more processors may determine the at least one metric based on a comparison of the label and the classification. In some embodiments, the one or more processors may apply the first biomedical image to a first machine learning (ML) model of the model architecture to determine a first likelihood of BCC in the lesion. The one or more processors may apply the plurality of second biomedical images to a second ML model of the model architecture to determine a second likelihood of BCC in the lesion. The one or more processors may generate the score as a function of the first likelihood and the second likelihood.

[0010] In some embodiments, the one or more processors may update at least one of the plurality of weights arranged across (i) a first ML model to determine a first likelihood of BCC using the first biomedical images and (ii) a second ML model to determine a second likelihood of BCC using the plurality of second biomedical images. In some embodiments, the one or more processors may identify the training dataset comprising at least one example of the plurality of examples. The label may identify one of the following: verified BCC, suspected BCC, benign, or normal. In some embodiments, the BCC may include at least one of nodular BCC, superficial spreading BCC, sclerosing BCC, or pigmented BCC. The dermoscopy may include at least one of polarized light dermoscopy, contact dermoscopy, or non-contact dermoscopy. The RCM may include at least one of a handheld RCM (HH-RCM) or wide-probe RCM (WP-RCM). Each of the plurality of second biomedical images may correspond to a respective depth of a plurality of depths within the at least one inner layer in the region on the epidermis of the first subject. The outer layer and inner layer may correspond to at least one of stratum comeum, stratum granulosum, stratum basale, dermal-epidermal junction, or papillary dermis.BRIEF DESCRIPTION OF THE DRAWINGS|0011| The foregoing and other objects, aspects, features, and advantages of the disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0012] FIG. 1 depicts a block diagram of a computer-aided diagnosis (CAD) system for basal cell carcinoma (BCC), in accordance with an illustrative embodiment;

[0013] FIGs. 2A-2C depict graphs operating characteristics (ROC) curves for validating a network architecture of receiver used in the computer-aided diagnosis (CAD) system;|0014] FIGs. 3A and 3B depict graphs operating characteristics (ROC) curves for validating a network architecture of receiver used in the computer-aided diagnosis (CAD) system, relative to manual dermatologists;

[0015] FIG. 4 depicts a block diagram of a system for detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions, in accordance with an illustrative embodiment;

[0016] FIG. 5 depicts a block diagram of a process to train network architectures to detect BCC in the system, in accordance with an illustrative embodiment;

[0017] FIG. 6 depicts a block diagram of a process to apply network architectures to incoming datasets to detect BCC in the system, in accordance with an illustrative embodiment;

[0018] FIG. 7 depicts a block diagram of a process to generate outputs based on application on the network architectures in the system, in accordance with an illustrative embodiment;

[0019] FIG. 8 depicts a flow diagram of a method of training machine learning (ML) models to detect basal-cell carcinoma (BCC) in biomedical images of skin lesions, in accordance with an illustrative embodiment;

[0020] FIG. 9 depicts a flow diagram of a method of detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions, in accordance with an illustrative embodiment;

[0021] FIG. 10 depicts a block diagram of a server system and a client computer system, in accordance with one or more implementations.DETAILED DESCRIPTION

[0022] Following below are more detailed descriptions of various concepts related to, and embodiments of, systems and methods for detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions. It should be appreciated that various concepts introduced above and discussed in greater detail below may be implemented in any of numerous ways, as the disclosed concepts are not limited to any particular manner of implementation. Examples of specific implementations and applications are provided primarily for illustrative purposes.

[0023] Section A describes detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions using network architecture.

[0024] Section B describes a network environment and computing environment which may be useful for practicing various computing-related embodiments described herein.A. Systems and Methods for Detecting Basal-Cell Carcinoma (BCC) in Biomedical Images of Skin Lesions.|0025] Basal-cell carcinoma (BCC) may be a type of cancer afflicting the skin or epidermis of a subject, originating in the basal cells of the epidermis. BCC may be characterized as a raised area of skin or ulceration. The diagnosis of basal cell carcinoma improved with dermoscopy but can be sometimes challenging. For the non-pigmented or lightly pigmented BCC, BCC localized on the face or the nose, Reflectance Confocal Microscopy (RCM) may improve the diagnosis of equivocal BCC lesions over dermoscopy. However, there are a number of limitations. For one, the specificity may be low, and the RCM may be reader-dependent. For another, there is difficulty in applying artificial intelligence (Al) in skin imaging, due to a lack of qualified dermatologists to create standardized images. In addition, most of the study analyzed dermoscopy performance independently from RCM diagnosis performance. Furthermore, it may be challenging to distinguish BCC from other types of skin discoloration or lesions, because the outwardly visible characteristics of BCC are often common with skin discoloration and lesions. For instance, BCC can appear as a bump resembling a scar, mole, or eczema.

[0026] Regarding Al algorithms for diagnosing BCC with RCM, deep learning showed high performance based on RCM imaging for BCC diagnosis, similar to dermatologists using a small dataset. However, those experiences are performed under artificial conditions and are not really similar to the clinical practice. In clinic, RCM is not performed alone but rather after dermoscopy examination. Combining Al on dermoscopy and on RCM at a lesion-level has never been reported. However, this type of algorithm may be more closely aligned with the clinical practice. Utility of combining RCM with dermoscopy in a large cohort of equivocal lesions is to be explored. Using multimodal imaging for these difficult-to-diagnose lesions could be interesting to improve the diagnosis and management of these lesions.

[0027] A multi-center study was performed using images of lesions with a clinical and dermoscopy impression for equivocal BCC (rule-out BCC), for a total of 251 cases. Images with obvious BCC, clinically or dermoscopically, were excluded. Image types may include RCM images (e.g., mosaics and stacks) with a minimum of 1 mosaic (Vivascope 1500, Caliber I D., Rochester, NY) and 1 stack, while cases are imaged with HH-RCM (Vivascope 3000, Caliber I.D., Rochester, NY). There may be a clinical overview image. The image types may also include dermoscopy images, including one polarized image, or one non-polarized image.

[0028] Image acquisition may be performed in accordance with the standard of care. The WP-RCM or the HH-RCM may be utilized to capture images depending on the anatomical location of the lesion. Imaging within the lesion using WP-RCM imaging includes a minimum of one mosaic for each layer of the skin (stratum corneum, stratum granulosum, stratum basale, dermal-epidermal junction, and papillary dermis) and at least one stack. Lesions imaged with the HH-RCM were delineated with a paper ring to establish borders, as is standard practice. At least one stack is acquired starting from stratum corneum to papillary dermis. Each lesion was labeled as a stack level analysis with the identified diagnosis. The images were labeled among the five classes: basal cell carcinoma; suspicious of basal cell carcinoma; benign; normal; or bad quality (images have been removed), among others.

[0029] FIG. 1 depicts a block diagram of a computer-aided diagnosis (CAD) system 100 for basal cell carcinoma (BCC). In the context of basal cell carcinoma (BCC) diagnosis, the system is based on a multi-modal approach combining two types of medical imaging (RCM and dermoscopy) for BCC detection. In this approach, the issue of BCC detection is regarded as a binary classification that distinguishes BCC lesions from non-BCC lesions. For this, the system may include three main components: an RCM classification module, a dermoscopic classification module and a fusion module, as shown.

[0030] For building the RCM classification module, the individual images may be identified from each stack of acquisition. These images are fed to a CNN model, giving the probability of being a BCC for each image. These probabilities are then combined with a weighted median to obtain the probabilities on the stacks, which, in turn, are combined to obtain the probability of the full acquisition based on RCM imaging. In the same way, for the dermoscopic classification module, dermoscopic images are fed to a CNN model to obtain the probability of the lesion being BCC based on the dermoscopic imaging. Once both probabilities are obtained, a weighted mean is used in the fusion module to classify the lesion as BCC or non- BCC based on both RCM and dermoscopic imaging.

[0031] In the construction of the CAD system, an EfficientNetbO model may be used for RCM and an EfficientNetb2 model may be used for dermoscopy. Both models were pre-trained on the ImageNet dataset. Then, fine-tuned on our dataset by removing the classification layer and replacing it with two fully-connected layers and a classification layer. The models were assessed on various metrics, including receiver operating characteristics (ROC) and area under the ROC curve (AUC), among others, as seen in Tables 1-3 below.Table 1Table 2Table 3

[0032] In addition, FIGs. 2A-C depict graphs receiver operating characteristics (ROC) curves 200A-C for validating a network architecture of receiver used in the computer-aided diagnosis (CAD) system. FIGs. 3A and B depict graphs receiver operating characteristics (ROC) curves 300A and 300B for validating a network architecture of receiver used in the computer- aided diagnosis (CAD) system, relative to manual dermatologists.(0033] Referring now to FIG. 4, depicted is a block diagram of a system 400 for detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions. In overview, the system 400 may include at least one image processing system 405, at least one reflectance confocal microscopy (RCM) acquirer 410, at least one dermoscopy acquirer 415, and at least one client device 420, among others, communicatively coupled with one another via at least network425. The image processing system 405 may include at least one image collector 430, at least one model applier 435, at least one model trainer 440, at least one output evaluator 445, at least one network architecture 450, and at least one database 455, among others. The network architecture 450 may include at least one RCM classifier 460, at least one dermoscopy classifier 465, and at least one aggregator 470, among others. Each of the components in the system 100 as detailed herein, may be implemented using hardware (e.g., one or more processors coupled with memory) or a combination of hardware and software, as detailed herein in Section B.

[0034] In further detail, the image processing system 405 may (sometimes herein generally referred to as a computing system or a server) be any computing device, including one or more processors coupled with memory and software and capable of performing the various processes and tasks described herein. The image processing system 405 may be associated with an entity (e.g., a clinician or a vendor) to process or evaluate images. The image processing system 405 can be in communication with the RCM acquirer 410, the dermoscopy acquirer 415, and client device 420 via the network 425. The image processing system 405 may be situated, located, or otherwise associated with at least one server group. The server group may correspond to a data center, a branch office, or a site at which one or more servers corresponding to the image processing system 105 are situated.(0035] On the image processing system 405, the image collector 430 may receive images of skin lesions from the RCM acquirer 410 and dermoscopy acquirer 415. The model applier 435 may process the images obtained from the RCM acquirer 410 and dermoscopy acquirer 415 using the network architecture 450 to determine a likelihood that the skin lesion is BCC. The model trainer 440 may initialize, train, and establish the network architecture 450, including the RCM classifier 460, the dermoscopy classifier 465, and the aggregator 470 (e.g., using supervised, unsupervised, or weakly supervised learning techniques). The output evaluator 445 may generate information derived from the likelihood that the skin lesion is BCC. The database 455 may be used to store and maintain data in connection with operations of the image processing system 405 or any other components of the system 400. The database 455 may include a database management system (DBMS) to organize and control access to data.

[0036] The network architecture 450 (including its constituent components) may be any type of machine learning (ML) model or artificial intelligence (Al) model to determine a likelihood that a skin lesion in RCM and dermoscopy images corresponds to BCC. The RCM classifier 460 may be any type of machine learning (ML) algorithm or model to determine a likelihood that a skin lesion in RCM images corresponds to BCC. The dermoscopy classifier 465 may be any type of machine learning (ML) algorithm or model to determine a likelihood that a skin lesion in dermoscopy images corresponds to BCC. The aggregator 470 (sometimes herein referred to as a fuser) may determine a composite score indicating a likelihood that the skin lesion in RCM and dermoscopy images corresponds to BCC. The network architecture 450 may include, for example, a deep learning-based artificial neural network (ANN) (e.g., convolutional neural network or transformer), a clustering algorithm (e.g., k-nearest neighbors), a support vector machine (SVM), a naive Bayesian classifier, and a random forest classifier, among others. In general, the network architecture 450 may include a set of inputs and a set of outputs. The inputs and the outputs of the network architecture 450 may be related to each other via a set of parameters (sometimes referred to herein as weights or kernel parameters). The set of parameters may be arranged across the RCM classifier 460, the dermoscopy classifier 465, and the aggregator 470, in accordance with the ML model or Al algorithm.|0037| The RCM acquirer 410 (sometimes herein referred to as an RCM imaging device or an RCM imager) may be any device to acquire RCM images of one or more inner layers of epidermis of a subject. The RCM images may be acquired in accordance with RCM techniques. For example, the RCM technique may include handheld RCM (HH-RCM) or wide-probe RCM (WP-RCM), among others. RCM may be an imaging technique to illuminate a specific depth or point within the epidermis without reliance on invasive measures on the subject. The RCM acquirer 410 may be a device to perform the RCM imaging technique on the region of the epidermis to image one or more inner layers. The RCM acquirer 410 may be used on the same subject to analyze and evaluate the same region (e.g., substantially 90% the same) of epidermis as the dermoscopy acquirer 415 for BCC. The RCM acquirer 410 may generate one or moreimage files for the acquisition of the RCM images to provide to the image processing system 405.

[0038] The dermoscopy acquirer 415 (sometimes herein referred to as a dermoscopy imaging device, a dermoscopy imager, a dermatoscopy acquirer, a dermatoscopy imaging device, or a dermatoscopy imager) may be any device to acquire a dermoscopy image of an outer layer of epidermis of the subject. The dermoscopy acquirer 415 may be a device (e.g., a handheld device) to image an outer layer of the epidermis. The dermoscopy acquirer 415 may be used on the same subject to analyze and evaluate the same region (e.g., substantially 90% the same) of epidermis as the RCM acquirer 410 for BCC. The dermoscopy images may be acquired in accordance with dermoscopy imaging techniques, such as polarized light dermoscopy, non-polarized light dermoscopy, contact dermoscopy, or non-contact dermoscopy, among others. The dermoscopy acquirer 415 may generate one or more image files for the acquisition of the dermoscopy to provide to the image processing system 405.

[0039] The client device 420 (sometimes herein referred to as an operator device or a clinician device) can be any computing device comprising one or more processors coupled with memory and software and capable of providing or presenting information from the image processing system 405. The client device 420 can be associated with an entity (e.g., a clinician) examining the subject or biomedical images from the subject. The client device 420 can be in communication with the image processing system 405, the RCM acquirer 410, the dermoscopy acquirer 415 to exchange data. The client device 420 can include a display device to render, present, or otherwise display projection images acquired from the RCM acquirer 410 and the dermoscopy acquirer 415.

[0040] Referring now to FIG. 5, depicted is a block diagram of a process 500 to train network architectures to detect BCC in the system 400. The process 500 may include or correspond to operations in the system 100 to initialize, train, and establish the network architecture 450. Under the process 500, the model trainer 440 may initialize the network architecture 450, including the RCM classifier 460, the dermoscopy classifier 465, and theaggregator 470, among others. For instance, the model trainer 440 may set the values of the parameters in the network architecture 450 to defined or random values. To train the network architecture 450, the model trainer 440 may retrieve, obtain, or identify at least one training dataset 505 from the database 455. The training dataset 505 may identify or include a set of examples. In the training dataset 505, each example may identify or include a set of RCM images 510A-N (hereinafter generally referred to as RCM images 510, and sometimes referred to herein as stacks), at least one dermoscopy image 515, and at least one label 520, among others. The example may be associated with a given subject 525. The subject 525 may be a human or animal subject, among others.

[0041] The subject 525 may have, be at risk of, or be afflicted with BCC. The BCC may include nodular BCC, superficial spreading BCC, sclerosing BCC, or pigmented BCC, among others. As used herein, “basal-cell carcinoma” or “BCC” may refer to a type of skin cancer in basal cells in the lower layers of the epidermis. The BCC may originate from a basal cell of the epidermis, or other layer. BCC may correspond to a discolored or raised portion of skin, such as a small, shiny bump or a scaly flat patch, among others. BCC may be found on any part of the skin of the subject 525, such as a facial region, head region, neck region, or back region, among others.(0042] The subject 525 may have at least one skin lesion 530 in at least one epidermis region 535 to be evaluated for BCC. The skin lesion 530 may correspond to a portion of the epidermis of the subject 525 with an abnormal change in skin color, shape, size, texture, or appearance. The skin lesion 530 may be, for example, a macule, a papule, a nodule, plaque, vesicle, bulla, pustule, a wheal, scale, crust, erosion, ulcer, fissure, scar, or striae, among others. The epidermis region 535 may correspond to an area of epidermis including at least a portion of the skin lesion 530 on the subject 525. The epidermis region 535 may correspond to any portion of the skin of the subject 525, such as the head, face, ears, scalp, neck, shoulders, back, arm, or hand, among others. The epidermis region 535 may also correspond to the area of epidermis image in the RCM images 510 and the dermoscopy image 515. The epidermis region 535 may have one or more layers, from the most superficial to the deepest layers: stratum corneum,stratum granulosum, stratum basale, dermal-epidermal junction, or papillary dermis, among others. The skin lesion 530 in the epidermis region 535 may be associated with BCC.

[0043] In each example of the training dataset 505, the set of RCM images 510 (also referred to herein as biomedical images) may be of at least one inner layer of the epidermis region 535 with the skin lesion 530 on the subject 525 acquired in accordance with RCM (e.g., HH-RCM or WP-RCM). The inner layer of the epidermis region 535 may correspond to at least one of stratum comeum, stratum granulosum, stratum basale, dermal-epidermal junction, or papillary dermis, among others. In some embodiments, each RCM image 510 may correspond to a different, respective depth within the inner layers of the epidermis region 535. For example, the set of RCM images 510 may contain images from the stratum comeum going down to the papillary dermis at various depths. In some embodiments, each RCM image 510 may correspond to a different, respective magnification factor for the corresponding inner layer of the epidermis region 535.

[0044] In addition, the dermoscopy image 515 (also referred to herein as biomedical images) may be of at least one outer layer of the epidermis region 535 with the skin lesion 530 on the subject 525, acquired in accordance with dermoscopy (e.g., polarized light dermoscopy, non-polarized light dermoscopy, contact dermoscopy, or non-contact dermoscopy). The outer layer may correspond to any layer that is superficial relative to the inner layers as imaged using the RCM images 510. The outer layer for the dermoscopy image 515 may correspond, for example, to the stratum corneum. The epidermis region 535 depicted in the dermoscopy image 515 may substantially correspond (e.g., 80-90% overlap) to the epidermis region 535 depicted in the set of RCM images 510.

[0045] The label 520 may indicate or identify a presence or absence of BCC in the skin lesion 530 in the epidermis region 535 of the subject 525. The label 520 may be manually created, inputted, or otherwise generated by a clinician examining the skin lesion 530 on the epidermis region 535 of the subject 525. The label 520 may be associated with the set of RCM images 510 and the dermoscopy image 515. In some embodiments, the label 520 may indicateor identify one of the following: verified BCC, suspected BCC, benign, or normal for the skin lesion 530. From the training dataset 505, the image collector 430 may obtain, retrieve, or otherwise identify the set of RCM images 510 and the dermoscopy image 515. The image collector 430 may traverse through the set of examples of the training dataset 505 to identify the set of RCM images 510 and the dermoscopy image 515 from each example. With each identification, the image collector 430 may convey, send, or otherwise provide the set of RCM images 510 and the dermoscopy image 515 to the model applier 435 for additional processing.

[0046] The model applier 435 may feed, provide, or apply the set of RCM images 510 and the dermoscopy image 515 as input to the network architecture 450. In feeding, the model applier 435 may provide or apply the set of RCM images 510 as input to the RCM classifier 460 of the network architecture 450. The model applier 435 may process the set of RCM images 510 in accordance with the set of parameters of the RCM classifier 460. For each RCM image 510, the model applier 435 may calculate, determine, or otherwise generate at least one stack score 550A-N (hereinafter generally referred to as stack scores 550). The stack score 550 may identify or indicate a likelihood of BCC in the skin lesion 530 for the respective RCM image 510. The stack score 550 may be generated from a portion of the RCM classifier 460. The model applier 435 may apply the set of stack scores 550 to the remaining portion of the RCM classifier 460. From processing the set of stack scores 550, the model applier 435 may calculate, determine, or otherwise generate at least one RCM score 555A. The RCM score 555A may identify or indicate a composite likelihood of BCC in the skin lesion 530 across the set of RCM images 510.

[0047] In conjunction, the model applier 435 may provide or apply the dermoscopy image 515 as input to the dermoscopy classifier 465 of the network architecture 450. The model applier 435 may process the dermoscopy image 515 in accordance with the set of parameters of the dermoscopy classifier 465. From processing the dermoscopy image 515 using the dermoscopy classifier 465, the model applier 435 may calculate, determine, or otherwise generate at least one dermoscopy score 555B. The dermoscopy score 555B may identify or indicate a likelihood of BCC in the skin lesion 530 for the dermoscopy image 515. With thegeneration, the model applier 435 may apply, provide, or feed the RCM score 555A and the dermoscopy score 555B as input to the aggregator 470. In some embodiments, the model applier 435 may provide or apply the set of stack scores 550 along with the dermoscopy score 555B as input to the aggregator 470. From processing using the aggregator 470, the model applier 435 may calculate, determine, or otherwise generate at least one combined score 560. The combined score 560 may identify or indicate a likelihood of BCC in the skin lesion 530 of the epidermis region 535 for the subject 525. In some embodiments, the model applier 435 may generate the combined score 560 as a function of the RCM score 555A (or the set of stack scores 550) and the dermoscopy score 555B. The function may be used instead of the aggregator 470, and may include, for example, a sum, a weighted sum, or any defined combination of the input values.

[0048] The model trainer 440 may calculate, determine, or otherwise determine at least one loss metric 565 based on the combined score 560 and the label 520. The loss metric 565 may indicate a level of deviation of the combined score 560 generated using the network architecture 450 versus the label 520. To determine the loss metric 565, the model trainer 440 may compare the combined score 560 with the label 520. The loss metric 565 may be in accordance with any number of loss functions, such as a norm loss (e.g., LI or L2), mean absolute error (MAE), mean squared error (MSE), a quadratic loss, a cross-entropy loss, and a Huber loss, among others. In general, if the combined score 560 differs from the indication of the label 520 (e.g., when the combined score 560 indicates low likelihood and label 520 indicates identification or suspicion of BCC), the loss metric 565 may be higher. Otherwise, if the combined score 560 is the same as the indication of the label 520 (e.g., when the combined score 560 indicates low likelihood of BCC and label 520 indicates benign or normal), the loss metric 565 may be lower.10049] In some embodiments, the model trainer 440 may compare the combined score 560 with a threshold. The threshold may delineate, define, or otherwise identify a value for the combined score 560 at which to classify a presence or absence of the BCC in the skin lesion 530 in the epidermis region 535 of the subject 525. Values of the combined score 560 above the threshold may be considered high likelihood of BCC. Conversely, values of the combined score560 below the threshold may be considered low likelihood of BCC. If the combined score 560 satisfies (e.g., greater than or equal to) the threshold, the model trainer 440 may determine the classification to indicate presence of BCC. Otherwise, if the combined score 560 does not satisfy (e.g., less than) the threshold, the model trainer 440 may determine the classification to indicate the absence of BCC. With the determination, the model trainer 440 may compare the classification with the label 520. Based on the comparison, the model trainer 440 may generate the loss metric 565.

[0050] Using the loss metric 565, the model trainer 440 may modify, change, or otherwise update at least one of the parameters of the network architecture 450. In some embodiments, the model trainer 440 may update at least one of the parameters arranged across the RCM classifier 460, the dermoscopy classifier 465, or the aggregator 470 in the network architecture 450. The updating of weights of the network architecture 450 may be in accordance with an optimization function (also referred to herein as an objective function). The optimization function may define one or more rates or parameters at which the weights of the network architecture 450 are to be updated. The optimization function may be in accordance with stochastic gradient descent, and may include, for example, an adaptive moment estimation (Adam), an implicit update (ISGD), and an adaptive gradient algorithm (AdaGrad), among others. The updating of the weights of the network architecture 450 may be repeated until convergence. Upon completion of training, the model trainer 440 may store and maintain the set of weights of the network architecture 450 on the database 455 for inference from newly acquired RCM and dermoscopy images.(0051 ] Referring now to FIG. 6, depicted is a block diagram of a process 600 to apply network architectures to incoming datasets to detect BCC in the system 400. The process 600 may include or correspond to operations performed in the system 400 for inference of likelihood of BCC using newly acquired RCM and dermoscopy images. Under the process 600, the RCM acquirer 410 may produce, output, or otherwise generate a set of RCM images 610A-N (hereinafter generally referred to as RCM images 610) for at least one subject 625. The RCM images 610 may be of at least one inner layer of epidermis region 635 with the skin lesion 630on the subject 625 acquired in accordance with RCM (e.g., HH-RCM or WP-RCM). For each RCM image 610, the RCM acquirer 410 may acquire or scan a respective depth in the one or more inner layers of the epidermis region 635 of the subject 625. In some embodiments, the RCM acquirer 410 may acquire or scan at least one of the inner layers of the epidermis region 635 of the subject 625 at a respective magnification factor. With the generation, the RCM acquirer 410 may send, provide, or otherwise transmit the set of RCM images 610 to the image processing system 405.

[0052] In conjunction, the dermoscopy acquirer 415 may produce, output, or otherwise generate at least one dermoscopy image 615. The dermoscopy image 615 may be of at least one outer layer of the epidermis region 635 with the skin lesion 630 on the subject 625, acquired in accordance with dermoscopy (e.g., polarized light dermoscopy, non-polarized light dermoscopy, contact dermoscopy, or non-contact dermoscopy). The epidermis region 635 may correspond to any portion of the skin of the subject 625, such as the head, face, ears, scalp, neck, shoulders, back, arm, or hand, among others. The outer layer may correspond to any layer that is superficial relative to the inner layers as imaged using the RCM images 510. The outer layer for the dermoscopy image 515 may correspond, for example, to the stratum corneum. The inner layer for the set of RCM images 610 may correspond, for example, to at least one of stratum corneum, stratum granulosum, stratum basale, dermal-epidermal junction, or papillary dermis, among others. The acquisition of the dermoscopy image 615 may be within a time window (e.g., ranging from 30 seconds to 72 hours) of the acquisition of the set of RCM images 610. For example, during the same clinical meeting, a clinician examining the skin lesion 630 on the subject 625 can operate the RCM acquirer 410 to generate the RCM image 610 and then, minutes later, operate the dermoscopy acquirer 415 to generate the dermoscopy image 615. With the acquisition, the dermoscopy acquirer 415 may provide, send, or otherwise transmit the dermoscopy image 615 to the image processing system 405.

[0053] The image collector 430 may retrieve, receive, or otherwise identify the set of RCM images 610 and the dermoscopy image 615 for the subject 625. In some embodiments, the image collector 430 may identify the set of RCM images 610 from the RCM acquirer 410. Insome embodiments, the image collector 430 may identify the dermoscopy image 615 from the dermoscopy acquirer 415. In some embodiments, the image collector 430 may access the database 455 to retrieve or identify the set of RCM images 610 and the dermoscopy image 615. The set of RCM images 610 and the dermoscopy image 615 may have been previously acquired by the RCM acquirer 410 and the dermoscopy acquirer 415 respectively, and stored and maintained on the database 455. With the identification, the image collector 430 may convey, send, or otherwise provide the set of RCM images 510 and the dermoscopy image 515 to the model applier 435 for additional processing.

[0054] With the identification, the model applier 435 may feed or apply the set of RCM images 610 and the dermoscopy image 615 to the network architecture 450. In feeding, the model applier 435 may apply the set of RCM images 610 to the RCM classifier 460 of the network architecture 450. The model applier 435 may process the set of RCM images 610 in accordance with the set of parameters of the RCM classifier 460. For each RCM image 610, the model applier 435 may calculate, determine, or otherwise generate at least one stack score 650A-N (hereinafter generally referred to as stack scores 650). The stack score 650 may identify or indicate a likelihood of BCC in the skin lesion 630 for the respective RCM image 610. The stack score 650 may be generated from a portion of the RCM classifier 460. The model applier 435 may apply the set of stack scores 650 to the remaining portion of the RCM classifier 460. From processing the set of stack scores 650, the model applier 435 may calculate, determine, or otherwise generate at least one RCM score 655A. The RCM score 655A may identify or indicate a composite likelihood of BCC in the skin lesion 630 across the set of RCM images 610.

[0055] In conjunction, the model applier 435 may apply the dermoscopy image 615 to the dermoscopy classifier 465 of the network architecture 450. The model applier 435 may process the dermoscopy image 615 in accordance with the set of parameters of the dermoscopy classifier 465. From processing the dermoscopy image 615 using the dermoscopy classifier 465, the model applier 435 may calculate, determine, or otherwise generate at least one dermoscopy score 655B. The dermoscopy score 655B may identify or indicate a likelihood of BCC in theskin lesion 630 for the dermoscopy image 615. With the generation, the model applier 435 may apply or feed the RCM score 655A and the dermoscopy score 655B to the aggregator 470. In some embodiments, the model applier 435 may apply the set of stack scores 650 along with the dermoscopy score 655B to the aggregator 470. From processing using the aggregator 470, the model applier 435 may calculate, determine, or otherwise generate at least one combined score 660. The combined score 660 may identify or indicate a likelihood of BCC in the skin lesion 630 of the epidermis region 635 for the subject 625. In some embodiments, the model applier 435 may generate the combined score 660 as a function of the RCM score 655A (or the set of stack scores 650) and the dermoscopy score 655B. The function may be used instead of the aggregator 470, and may include, for example, a sum, a weighted sum, or any defined combination of the input values.(0056] Referring now to FIG. 7, depicted is a block diagram of a process 700 to generate outputs based on the application on the network architectures in the system 400. With the generation of the combined score 660, the output evaluator 445 may store and maintain an association between the subject 625 and the combined score 660. The association may be stored and maintained on the database 455 using one or more data structures (e g., linked list, array, matrix, binary tree, hash, heap, queue, or stack). In some embodiments, the output evaluator 445 may store and maintain the association of the subject 625 with one or more of the following data structures: the set of the RCM images 610, the dermoscopy image 615, the stack scores 650, the RCM score 655A, and the dermoscopy score 655B, among others.

[0057] The output evaluator 445 may identify, generate, or otherwise determine at least one classification 705 based on the combined score 660. To determine the classification 705, the output evaluator 445 may compare the combined score 660 with a threshold. The threshold may delineate, define, or otherwise identify a value for the combined score 660 at which to classify a presence or absence of the BCC in the skin lesion 630 in the epidermis region 635 of the subject 625. If the combined score 660 satisfies (e g., greater than or equal to) the threshold, the output evaluator 445 may determine the classification 705 to indicate presence of BCC skin lesion 630 in the epidermis region 635 of the subject 625. In some embodiments, the Otherwise, if thecombined score 660 does not satisfy (e.g., less than) the threshold, the output evaluator 445 may determine the classification 705 to indicate absence of BCC skin lesion 630 in the epidermis region 635 of the subject 625. The output evaluator 445 may store and maintain the association between the subject 625 and the classification 705 using the one or more data structures on the database 455.

[0058] In some embodiments, the output evaluator 445 may identify or select at least one therapy 710 to administer to the skin lesion 630 based on the combined score 660 (or the classification 705). The therapy may be administered to the skin lesion 630 to address BCC. The therapy 710 may be selected from a set of candidate therapies, such as electrosurgery, Mohs surgery, excision surgery, radiotherapy, photodynamic therapy, cryosurgery, laser surgery, or topical medications, or any combination thereof, among others. The output evaluator 445 may select the therapy 710, when the combined score 660 satisfies the threshold indicating the presence of BCC in the skin lesion 630. Each candidate therapy may correspond to a range of values for the combined score 660. In some embodiments, the output evaluator 445 may select at least one of the candidate therapies as the therapy 710 to administer to the skin lesion 630, when the classification 705 indicates the presence of BCC.

[0059] To select, the output evaluator 445 may compare the combined scores 660 with the range of values for each candidate therapy. The output evaluator 445 may select the candidate therapy as the therapy 710 corresponding to the range of values in which the combined score 660 resides. In some embodiments, the output evaluator 445 may identify or determine whether a therapy 710 for BCC is to be administered to the skin lesion 630 based on the combined score 660 (or the classification 705). When the combined score 660 satisfies the threshold (or the classification 705 indicates the presence of BCC), the output evaluator 445 may determine that therapy 710 is to be administered to the skin lesion 630 for BCC. In some embodiments, the output evaluator 445 may also identify the subject 625 as a candidate for administration of the therapy 710. On the other hand, when the combined score 660 does not satisfy the threshold (or the classification 705 indicates the absence of BCC), the output evaluator 445 may determine that therapy 710 is not to be administered to the skin lesion 630 forBCC. In some embodiments, the output evaluator 445 may also identify the subject 625 as a non-candidate for administration of the therapy 710.

[0060] The output evaluator 445 may write, produce, or otherwise generate at least one output 715 to include information. The information may be based on the association between the subject 625 and the combined score 660, the classification 705, the set of RCM images 610, and the dermoscopy image 615, among others. In some embodiments, the output evaluator 445 may generate the output 715 to identify the therapy 710 selected from the set of candidate therapies. In some embodiments, the output evaluator 445 may generate the output 715 to indicate whether the therapy 710 for BCC is to be administered to the skin lesion 630. For instance, when the combined score 660 satisfies the threshold or the classification 705 indicates the presence of BCC in the skin lesion 630, the output 715 may be generated to indicate that therapy 710 is to be administered to the skin lesion 630. The output 715 may also identify the subject 625 as the candidate for the administration of the therapy 710. Conversely, when the combined score 660 does not satisfy the threshold or the classification 705 indicates the absence of BCC in the skin lesion 630, the output 715 may be generated to indicate that therapy 710 is not to be administered to the skin lesion 630. The output 715 may also identify the subject 625 as the non-candidate for the administration of the therapy 710. With the generation, the output evaluator 445 may send, transmit, or otherwise provide the output 715 to the client device 420 for presentation.[00611 Upon receipt, the client device 420 may render, display, or otherwise present the information included in the output 715 on a display. For example, the client device 420 may present the set of RCM images 610 and the dermoscopy image 615, along with the combined score 660 and the classification 705 indicating whether the skin lesion 630 has BCC, in a graphical user interface on a display. Using the information of the output 715, presented through the client device 420, a user (e.g., a clinician examining the skin lesion 630 on the subject 625) of the client device 420 may decide whether to administer the therapy 710 and if so, which therapy to deliver, provide, or otherwise administer. When the output 715 indicates that therapy 710 is to be administered, the skin lesion 630 of the subject 625 (or the subject 625) may be administered (e.g., by the clinician) with the therapy 710 as identified in the output 715. Thetherapy 710 may be administered to address the BCC corresponding to the skin lesion 630 in the epidermis region 635 of the subject 625. Conversely, when the output 715 indicates that therapy 710 is not to be administered, the clinician may refrain from administering the therapy 710 to the skin lesion 630 of the subject 625 (or the subject 625).100621 In this manner, the image processing system 405 may generate the combined score 660 to indicate the likelihood of BCC in the skin lesion 630 for the subject 625, using the set of RCM images 610 acquired according to RCM and the dermoscopy image 615 acquired according to dermoscopy. From a computer functionality perspective, the training of the network architecture 450 using the RCM and dermoscopy imaging modalities may allow the RCM classifier 460 and the dermoscopy classifier 465 to pinpoint latent features in the image data correlated with the presence of the BCC. The use of two imaging modalities for the skin lesion 630 (e g., RCM and dermoscopy) may enable the network architecture 450 to capture subtle morphological variations across epidermal layers, such as the stratum corneum, stratum basale, and dermal-epidermal junction, in the image data. The network architecture 450 may thus be trained to extract features in an efficient and targeted manner, in determining the likelihood of BCC for the combined score 660. This layered approach can improve precision and accuracy but also reduces false positives and negatives. By leveraging additional features, the image processing system 405 may thus achieve higher performance, relative to approaches that use either RCM or dermoscopy imaging alone. The image processing system 405 may conserve consumption of computer resources (e.g., processor and memory) that would have otherwise been spent on establishing separate models for the different imaging modalities and providing ineffective and inaccurate results as with the other approaches.

[0063] From a clinical perspective, the information provided through the output 715 may enable the clinician examining the skin lesion 630 of the subject 625 to make a more accurate diagnosis and assessment as to which therapy is to be administered to best treat the BCC. The image processing system 405 may provide for higher sensitivity, particularly in skin discoloration or lesions that are challenging to diagnose using methods that rely on RCM or dermoscopy alone. By using the network architecture 450, the classification process may reducethe reliance on subjective, observer-dependent methods for diagnosing BCC, thus allowing for more consistent clinical decision making. The output 715 generated by the image processing system 405 may provide for informed decisions, including the selection of appropriate interventions from a range of candidate therapies. Overall, the image processing system 405 can provide for accurate and robust detection of BCC from image data to allow for improved clinical outcomes.

[0064] Referring now to FIG. 8, depicted is a flow diagram of a method 800 of training machine learning (ML) models to detect basal-cell carcinoma (BCC) in biomedical images of skin lesions. The method 800 may be implemented using or performed by any of the components described herein, such as the image processing system 405 and the system 1000. Under the method 800, a computing system may identify a dataset from training data (805). The computing system may apply a network architecture to the dataset (810). The computing system may generate a score based on the application (815). The computing system may determine a loss metric (820). The computing system may update the network architecture using the loss metric (825).

[0065] Referring now to FIG. 9, depicted is a flow diagram of a method 900 of detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions. The method 900 may be implemented using or performed by any of the components described herein, such as the image processing system 405 and the system 1000. Under the method 900, a computing system may identify a dataset from image acquirers (905). The computing system may apply a network architecture to the dataset (910). The computing system may generate a score based on the application (915). The computing system may determine a classification (920). The computing system may provide an output (925).B. Computing and Network Environment

[0066] Various operations described herein can be implemented on computer systems.FIG. 10 shows a simplified block diagram of a representative server system 1000, clientcomputing system 1014, and network 1026 usable to implement certain embodiments of the present disclosure. In various embodiments, server system 1000 or similar systems can implement services or servers described herein or portions thereof. Client computing system 1014 or similar systems can implement clients described herein. The system 1000 described herein can be similar to the server system 1000. Server system 1000 can have a modular design that incorporates a number of modules 1002 (e.g., blades in a blade server embodiment); while two modules 1002 are shown, any number can be provided. Each module 1002 can include processing unit(s) 1004 and local storage 1006.

[0067] Processing unit(s) 1004 can include a single processor, which can have one or more cores, or multiple processors. In some embodiments, processing unit(s) 1004 can include a general-purpose primary processor as well as one or more special-purpose co-processors, such as graphics processors, digital signal processors, or the like. In some embodiments, some or all processing units 1004 can be implemented using customized circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself. In other embodiments, processing unit(s) 1004 can execute instructions stored in local storage 1006. Any type of processors in any combination can be included in processing unit(s) 1004.(0068] Local storage 1006 can include volatile storage media (e.g., DRAM, SRAM, SDRAM, or the like) and / or non-volatile storage media (e.g., magnetic or optical disk, flash memory, or the like). Storage media incorporated in local storage 1006 can be fixed, removed, or upgraded as desired. Local storage 1006 can be physically or logically divided into various subunits, such as a system memory, a read-only memory (ROM), and a permanent storage device. The system memory can be a read-and-write memory device or a volatile read-and-write memory, such as dynamic random-access memory. The system memory can store some or all of the instructions and data that processing unit(s) 1004 need at runtime. The ROM can store static data and instructions that are needed by processing unit(s) 1004. The permanent storage device can be a non-volatile read-and-write memory device that can store instructions and data even when module 1002 is powered down. The term “storage medium” as used herein includes anymedium in which data can be stored indefinitely (subject to overwriting, electrical disturbance, power loss, or the like) and does not include carrier waves and transitory electronic signals propagating wirelessly or over wired connections.

[0069] In some embodiments, local storage 1006 can store one or more software programs to be executed by processing unit(s) 1004, such as an operating system and / or programs implementing various server functions, such as functions of the system 400 of FIG. 4 or any other system described herein, or any other server(s) associated with system 400 or any other system described herein.

[0070] “Software” refers generally to sequences of instructions that, when executed by processing unit(s) 1004 cause server system 1000 (or portions thereof) to perform various operations, thus defining one or more specific machine embodiments that execute and perform the operations of the software programs. The instructions can be stored as firmware residing in read-only memory and / or program code stored in non-volatile storage media that can be read into volatile working memory for execution by processing unit(s) 1004. Software can be implemented as a single program or as a collection of separate programs or program modules that interact as desired. From local storage 1006 (or non-local storage described below), processing unit(s) 1004 can retrieve program instructions to execute and data to process in order to execute various operations described above.

[0071] In some server systems 1000, multiple modules 1002 can be interconnected via a bus or other interconnect 1008, forming a local area network that supports communication between modules 1002 and other components of server system 1000. Interconnect 1008 can be implemented using various technologies, including server racks, hubs, routers, etc.

[0072] A wide area network (WAN) interface 1010 can provide data communication capability between the local area network (interconnect 1008) and the network 1026, such as the Internet. Technologies can be used, including wired (e.g., Ethernet, IEEE 802.3 standards) and / or wireless technologies (e.g., Wi-Fi, IEEE 802.24 standards).

[0073] In some embodiments, local storage 1006 is intended to provide working memory for processing unit(s) 1004, providing fast access to programs and / or data to be processed while reducing traffic on interconnect 1008. Storage for larger quantities of data can be provided on the local area network by one or more mass storage subsystems 1012 that can be connected to interconnect 1008. Mass storage subsystem 1012 can be based on magnetic, optical, semiconductor, or other data storage media. Direct attached storage, storage area networks, network-attached storage, and the like can be used. Any data stores or other collections of data described herein as being produced, consumed, or maintained by a service or server can be stored in mass storage subsystem 1012. In some embodiments, additional data storage resources may be accessible via WAN interface 1010 (potentially with increased latency).

[0074] Server system 1000 can operate in response to requests received via WAN interface 1010. For example, one of modules 1002 can implement a supervisory function and assign discrete tasks to other modules 1002 in response to requests. Work allocation techniques can be used. As requests are processed, results can be returned to the requester via WAN interface 1010. Such operations can generally be automated. Further, in some embodiments, WAN interface 1010 can connect multiple server systems 1000 to each other, providing scalable systems capable of managing high volumes of activity. Other techniques for managing server systems and server farms (collections of server systems that cooperate) can be used, including dynamic resource allocation and reallocation.

[0075] Server system 1000 can interact with various user-owned or user-operated devices via a wide-area network such as the Internet. An example of a user-operated device is shown in FIG. 10 as client computing system 1014. Client computing system 1014 can be implemented, for example, as a consumer device such as a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, eyeglasses), desktop computer, laptop computer, and so on.

[0076] For example, client computing system 1014 can communicate via WAN interface 1010. Client computing system 1014 can include computer components such as processingunit(s) 1016, storage device 1018, network interface 1020, user input device 1022, and user output device 1024. Client computing system 1014 can be a computing device implemented in a variety of form factors, such as a desktop computer, laptop computer, tablet computer, smartphone, other mobile computing device, wearable computing device, or the like.|0077| Processing unit(s) 1016 and storage device 1018 can be similar to processing unit(s) 1004 and local storage 1006, as described above. Suitable devices can be selected based on the demands to be placed on client computing system 1014; for example, client computing system 1014 can be implemented as a “thin” client with limited processing capability or as a high-powered computing device. Client computing system 1014 can be provisioned with program code executable by processing unit(s) 1016 to enable various interactions with server system 1000.[0078| Network interface 1020 can provide a connection to the network 1026, such as a wide area network (e.g., the Internet), to which WAN interface 1010 of server system 1000 is also connected. In various embodiments, network interface 1020 can include a wired interface (e g., Ethernet) and / or a wireless interface implementing various RF data communication standards such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, 5G, LTE, etc.).

[0079] User input device 1022 can include any device (or devices) via which a user can provide signals to client computing system 1014; client computing system 1014 can interpret the signals as indicative of particular user requests or information. In various embodiments, user input device 1022 can include any or all of a keyboard, touch pad, touch screen, mouse or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, and so on.

[0080] User output device 1024 can include any device via which client computing system 1014 can provide information to a user. For example, user output device 1024 can include a display to present images generated by or delivered to client computing system 1014. The display can incorporate various image generation technologies, e.g., a liquid crystal display(LCD), a light-emitting diode (LED) including organic light-emitting diodes (OLED), a projection system, a cathode ray tube (CRT), or the like, together with supporting electronics (e.g., digital-to-analog or analog-to-digital converters, signal processors, or the like). Some embodiments can include a device, such as a touchscreen, that function as both input and output device. In some embodiments, other user output devices 1024 can be provided in addition to or instead of a display. Examples include indicator lights, speakers, tactile “display” devices, printers, and so on.

[0081] Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a computer-readable storage medium. Many of the features described in this specification can be implemented as processes that are specified as a set of program instructions encoded on a computer-readable storage medium. When these program instructions are executed by one or more processing units, they cause the processing unit(s) to perform various operation indicated in the program instructions. Examples of program instructions or computer code include machine code, such as that produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter. Through suitable programming, processing unit(s) 1004 and 1016 can provide various functionality for server system 1000 and client computing system 1014, including any of the functionality described herein as being performed by a server or client, or other functionality.

[0082] It will be appreciated that server system 1000 and client computing system 1014 are illustrative and that variations and modifications are possible. Computer systems used in connection with embodiments of the present disclosure can have other capabilities not specifically described here. Further, while server system 1000 and client computing system 1014 are described with reference to particular blocks, it is to be understood that these blocks are defined for convenience of description and are not intended to imply a particular physical arrangement of component parts. For instance, different blocks can be located in the same facility, in the same server rack, or on the same motherboard. Further, the blocks need not correspond to physically distinct components. Blocks can be configured to perform variousoperations, e.g., by programming a processor or providing appropriate control circuitry, and various blocks might or might not be reconfigurable depending on how the initial configuration is obtained. Embodiments of the present disclosure can be realized in a variety of apparatuses, including electronic devices implemented using any combination of circuitry and software.|0083| While the disclosure has been described with respect to specific embodiments, one skilled in the art will recognize that numerous modifications are possible. Embodiments of the disclosure can be realized using a variety of computer systems and communication technologies, including but not limited to the specific examples described herein. Embodiments of the present disclosure can be realized using any combination of dedicated components and / or programmable processors and / or other programmable devices. The various processes described herein can be implemented on the same processor or different processors in any combination. Where components are described as being configured to perform certain operations, such configuration can be accomplished, e.g., by designing electronic circuits to perform the operation, by programming programmable electronic circuits (such as microprocessors) to perform the operation, or by any combination thereof. Further, while the embodiments described above may make reference to specific hardware and software components, those skilled in the art will appreciate that different combinations of hardware and / or software components may also be used and that particular operations described as being implemented in hardware might also be implemented in software or vice versa.[0084| Computer programs incorporating various features of the present disclosure may be encoded and stored on various computer-readable storage media; suitable media include magnetic disk or tape, optical storage media such as compact disk (CD) or DVD (digital versatile disk), flash memory, and other non-transitory media. Computer-readable media encoded with the program code may be packaged with a compatible electronic device, or the program code may be provided separately from electronic devices (e.g., via Internet download or as a separately packaged computer-readable storage medium).

[0085] Thus, although the disclosure has been described with respect to specific embodiments, it will be appreciated that the disclosure is intended to cover all modifications and equivalents within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. A method of detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions, comprising: identifying, by one or more processors, for a first subject at risk of BCC in a first lesion on a region of an epidermis:(i) a first biomedical image of an outer layer of the region on the epidermis of the first subject acquired in accordance with dermoscopy, and(ii) a plurality of second biomedical images of at least one inner layer in the region on the epidermis of the first subject acquired in accordance with reflection confocal microscopy (RCM); applying, by the one or more processors, the first biomedical image and the plurality of second biomedical images to a model architecture, wherein the model architecture is established using a plurality of examples, each of the plurality of examples identifying: (i) a third biomedical image of an outer layer of a region with a second lesion on an epidermis of a respective second subject acquired in accordance with dermoscopy, (ii) a plurality of fourth biomedical images of at least one inner layer in the region with the second lesion on the epidermis of the respective second subject acquired in accordance with RCM, and (iii) a label identifying of one of a presence or absence of BCC in the second lesion in the region of the epidermis of the second subject; generating, by the one or more processors, based on applying the first biomedical image and the plurality of second biomedical images to the model architecture, a score indicating a likelihood of BCC in the first lesion; and storing, by the one or more processors, using one or more data structures, an association between the first subject and the score indicating the likelihood of BCC in the first lesion.

2. The method of claim 1, further comprising:generating, by the one or more processors, an output including information based on at least one of (i) the first biomedical image, (ii) the plurality of second biomedical images, or (iii) the association between the first subject and the score; and providing, by the one or more processors, the output including the information for presentation.

3. The method of claim 1, further comprising: selecting, by the one or more processors, from a plurality of candidate therapies, a therapy to administer to the first lesion based on the score indicating the likelihood of BCC in the first lesion; and providing, by the one or more processors, an output identifying the therapy selected for the first subject.

4. The method of claim 3, wherein the first lesion is administered with the therapy, wherein the plurality of candidate therapies comprises at least one of electrosurgery, Mohs surgery, excision surgery, radiotherapy, photodynamic therapy, cryosurgery, laser surgery, or a topical medication.

5. The method of claim 1, further comprising providing, by the one or more processors, an output indicating refraining of administration of therapy to the first lesion based on the score not satisfying a threshold.

6. The method of claim 1, further comprising determining, by the one or more processors, a classification indicating one of presence or absence of BCC in the first lesion based on a comparison between the score and a threshold, and wherein storing the association further comprises storing the association between the first subject and the classification.

7. The method of claim 1, wherein applying the first biomedical image to the model architecture further comprises applying the first biomedical image to a first machine learning (ML) model of the model architecture to determine a first likelihood of BCC in the first lesion, wherein applying the plurality of second biomedical images to the model architecture further comprises applying the plurality of second biomedical images to a second ML model of the model architecture to determine a second likelihood of BCC in the first lesion, and wherein generating the score further comprises generating the score as a function of the first likelihood and the second likelihood.

8. The method of claim 1, wherein applying the plurality of second biomedical images further comprises: determining, by applying each second biomedical image of the plurality of second biomedical images to a ML model of the model architecture, a respective likelihood of BCC in the first lesion; and generating, based on the respective likelihood for each second biomedical image, a composite likelihood of BCC in the first lesion over the plurality of second biomedical images.

9. The method of claim 1, wherein BCC comprises at least one of nodular BCC, superficial spreading BCC, sclerosing BCC, or pigmented BCC, and wherein the outer layer and inner layer correspond to at least one of stratum corneum, stratum granulosum, stratum basale, dermal-epidermal junction, or papillary dermis.

10. The method of claim 1, wherein dermoscopy comprises at least one of polarized light dermoscopy, contact dermoscopy, or non-contact dermoscopy, wherein the RCM comprises at least one of a handheld RCM (HH-RCM) or wide-probe RCM (WP-RCM), and wherein each of the plurality of second biomedical images correspond to a respective depth of a plurality of depths within the at least one inner layer in the region on the epidermis of the first subject.

11. A method of training model architectures for detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions, comprising: identifying, by one or more processors, a training dataset comprising a plurality of examples, at least one example of the plurality of examples comprising:(i) a first biomedical image of an outer layer of a region with a lesion on a epidermis of a subject acquired in accordance with dermoscopy,(ii) a plurality of second biomedical images of at least one inner layer in the region on the epidermis of the subject acquired in accordance with reflection confocal microscopy (RCM), and(ii) a label identifying of one of a presence or absence of BCC in the lesion in the region of the epidermis of the subject; applying, by the one or more processors, the first biomedical image and the plurality of second biomedical images to a model architecture comprising a plurality of weights, to generate a score indicating a likelihood of BCC in the lesion; determining, by the one or more processors, at least one metric loss based on the score and the label; and updating, by the one or more processors, at least one of the plurality of weights of the model architecture using the at least one loss metric.

12. The method of claim 11, further comprising determining, by the one or more processors, a classification indicating one of presence or absence of BCC in the lesion based on a comparison between the score and a threshold, and wherein determining the at least one metric further comprises determining the at least one metric based on a comparison of the label and the classification.

13. The method of claim 11, wherein applying the first biomedical image to the model architecture further comprises applying the first biomedical image to a first machine learning (ML) model of the model architecture to determine a first likelihood of BCC in the lesion,wherein applying the plurality of second biomedical images to the model architecture further comprises applying the plurality of second biomedical images to a second ML model of the model architecture to determine a second likelihood of BCC in the lesion, and wherein generating the score further comprises generating the score as a function of the first likelihood and the second likelihood.

14. The method of claim 11, wherein applying the plurality of second biomedical images further comprises: determining, by applying each second biomedical image of the plurality of second biomedical images to a ML model of the model architecture, a respective likelihood of BCC in the first lesion; and generating, based on the respective likelihood for each second biomedical image, a composite likelihood of BCC in the lesion over the plurality of second biomedical images.

15. The method of claim 11, wherein updating at least one of the plurality of weights further comprises updating at least one of the plurality of weights arranged across (i) a first ML model to determine a first likelihood of BCC using the first biomedical images and (ii) a second ML model to determine a second likelihood of BCC using the plurality of second biomedical images.

16. The method of claim 11, wherein identifying the training dataset further comprises identifying the training dataset comprising at least one example of the plurality of examples, wherein the label identifies one of: verified BCC, suspected BCC, benign, or normal.

17. The method of claim 11, wherein BCC comprises at least one of nodular BCC, superficial spreading BCC, sclerosing BCC, or pigmented BCC, and wherein the outer layer and inner layer correspond to at least one of stratum corneum, stratum granulosum, stratum basale, dermal-epidermal junction, or papillary dermis.

18. The method of claim 11, wherein dermoscopy comprises at least one of polarized light dermoscopy, contact dermoscopy, or non-contact dermoscopy, wherein the RCM comprises at least one of a handheld RCM (HH-RCM) or wide-probe RCM (WP-RCM), and wherein each of the plurality of second biomedical images correspond to a respective depth of a plurality of depths within the at least one inner layer in the region on the epidermis of the first subject.

19. A system for detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions, comprising: one or more processors coupled with memory, configured to: identify, for a first subject at risk of BCC in a first lesion on a region of an epidermis:(i) a first biomedical image of an outer layer of the region on the epidermis of the first subject acquired in accordance with dermoscopy, and(ii) a plurality of second biomedical images of at least one inner layer in the region on the epidermis of the first subject acquired in accordance with reflection confocal microscopy (RCM); apply the first biomedical image and the plurality of second biomedical images to a model architecture, wherein the model architecture is established using a plurality of examples, each of the plurality of examples identifying: (i) a third biomedical image of an outer layer of a region with a second lesion on an epidermis of a respective second subject acquired in accordance with dermoscopy, (ii) a plurality of fourth biomedical images of at least one inner layer in the region with the second lesion on the epidermis of the respective second subject acquired in accordance with RCM, and (iii) an identification of one of a presence or absence of BCC in the second lesion in the region of the epidermis of the second subject;generate, based on applying the first biomedical image and the plurality of second biomedical images to the model architecture, a score indicating a likelihood of BCC in the first lesion; and store, using one or more data structures, an association between the first subject and the score indicating the likelihood of BCC in the first lesion.

20. The system of claim 19, wherein the one or more processors are configured to: generate an output including information based on at least one of (i) the first biomedical image, (ii) the plurality of second biomedical images, or (iii) the association between the first subject and the score; and provide the output including the information for presentation.

21. The system of claim 19, wherein the one or more processors are configured to: select, from a plurality of candidate therapies, a therapy to administer to the first lesion based on the score indicating the likelihood of BCC in the first lesion; and provide an output identifying the therapy selected for the first subject.

22. The system of claim 21, wherein the first lesion is administered with the therapy, wherein the plurality of candidate therapies comprises at least one of electrosurgery, Mohs surgery, excision surgery, radiotherapy, photodynamic therapy, cryosurgery, laser surgery, or a topical medication.

23. The system of claim 19, wherein the one or more processors are configured to provide an output indicating refraining of administration of therapy to the first lesion based on the score not satisfying a threshold.

24. The system of claim 19, wherein the one or more processors are configured to: determine a classification indicating one of presence or absence of BCC in the first lesion based on a comparison between the score and a threshold, and store the association between the first subject and the classification.

25. The system of claim 19, wherein the one or more processors are configured to: apply the first biomedical image to a first machine learning (ML) model of the model architecture to determine a first likelihood of BCC in the first lesion; apply the plurality of second biomedical images to a second ML model of the model architecture to determine a second likelihood of BCC in the first lesion; generate the score as a function of the first likelihood and the second likelihood.

26. The system of claim 19, wherein the one or more processors are configured to: determine, by applying each second biomedical image of the plurality of second biomedical images to a ML model of the model architecture, a respective likelihood of BCC in the first lesion; and generate, based on the respective likelihood for each second biomedical image, a composite likelihood of BCC in the first lesion over the plurality of second biomedical images.

27. The system of claim 19, wherein BCC comprises at least one of nodular BCC, superficial spreading BCC, sclerosing BCC, or pigmented BCC, and wherein the outer layer and inner layer correspond to at least one of stratum corneum, stratum granulosum, stratum basale, dermal-epidermal junction, or papillary dermis.

28. The system of claim 19, wherein dermoscopy comprises at least one of polarized light dermoscopy, contact dermoscopy, or non-contact dermoscopy, wherein the RCM comprises at least one of a handheld RCM (HH-RCM) or wide-probe RCM (WP-RCM), and wherein each of the plurality of second biomedical images correspond to a respective depth of a plurality of depths within the at least one inner layer in the region on the epidermis of the first subject.

29. A system for training model architectures for detecting basal-cell carcinoma (BCC) in biomedical images of skin lesions, comprising: one or more processors coupled with memory, configured to: identify a training dataset comprising a plurality of examples, at least one example of the plurality of examples comprising:(i) a first biomedical image of an outer layer of a region with a lesion on a epidermis of a subject acquired in accordance with dermoscopy,(ii) a plurality of second biomedical images of at least one inner layer in the region on the epidermis of the subject acquired in accordance with reflection confocal microscopy (RCM), and(ii) a label identifying of one of a presence or absence of BCC in the lesion in the region of the epidermis of the subject; apply the first biomedical image and the plurality of second biomedical images to a model architecture comprising a plurality of weights, to generate a score indicating a likelihood of BCC in the lesion; determine at least one metric loss based on the score and the label; and update at least one of the plurality of weights of the model architecture using the at least one loss metric.

30. The system of claim 29, wherein the one or more processors are configured to: determine, a classification indicating one of presence or absence of BCC in the lesion based on a comparison between the score and a threshold, and determine the at least one metric based on a comparison of the label and the classification.

31. The system of claim 29, wherein the one or more processors are configured to: apply the first biomedical image to a first machine learning (ML) model of the model architecture to determine a first likelihood of BCC in the lesion,apply the plurality of second biomedical images to a second ML model of the model architecture to determine a second likelihood of BCC in the lesion, and generate the score as a function of the first likelihood and the second likelihood.

32. The system of claim 29, wherein the one or more processors are configured to: determine, by applying each second biomedical image of the plurality of second biomedical images to a ML model of the model architecture, a respective likelihood of BCC in the first lesion; and generate, based on the respective likelihood for each second biomedical image, a composite likelihood of BCC in the lesion over the plurality of second biomedical images.

33. The system of claim 29, wherein the one or more processors are configured to update at least one of the plurality of weights arranged across (i) a first ML model to determine a first likelihood of BCC using the first biomedical images and (ii) a second ML model to determine a second likelihood of BCC using the plurality of second biomedical images.

34. The system of claim 29, wherein identifying the training dataset further comprises identifying the training dataset comprising at least one example of the plurality of examples, wherein the label identifies one of: verified BCC, suspected BCC, benign, or normal.

35. The system of claim 29, wherein BCC comprises at least one of nodular BCC, superficial spreading BCC, sclerosing BCC, or pigmented BCC, and wherein the outer layer and inner layer correspond to at least one of stratum corneum, stratum granulosum, stratum basale, dermal-epidermal junction, or papillary dermis.

36. The system of claim 29, wherein dermoscopy comprises at least one of polarized light dermosocopy, contact dermoscopy, or non-contact dermoscopy, wherein the RCM comprises at least one of a handheld RCM (HH-RCM) or wide-probe RCM (WP-RCM), andwherein each of the plurality of second biomedical images correspond to a respective depth of a plurality of depths within the at least one inner layer in the region on the epidermis of the first subject.

Images