System and method for detection and grading of cervical abnormalities

Abstract

A cervical examination system and method is based upon the use of a dedicate image capture device used in combination with machine learning models as a diagnostic tool for point-of-exam use. By using

Description

[0001] SYSTEM AND METHOD FOR DETECTION AND GRADING OF CERVICAL ABNORMALITIES

[0002] Cross-Reference to Related. Applications

[0003] This application claims the benefit of U . S . Provisional Application No . 63 / 546, 601 , filed October 31 , 2023 and herein incorporated by reference .

[0004] Technical Field

[0005] This invention relates to an improvement in the conventional process of carrying out medical exams assessing cervical tissue abnormalities and, more particularly, to a system and method based on the utili zation of a dedicated image capture system and associated computer-assisted image analysis to quickly and accurately provide examination results .

[0006] Background

[0007] Screening for cervical precancer and cancer in many developing countries involves the application of acetic acid as well as Lugol ' s iodine in order to help visuali ze the extent of carcinoma or its precursors . In the field, this is called visual inspection with acetic acid (VIA) and visual inspection with Lugol ' s iodine (VILI ) . Similar to VIA, in VILI , the procedure involves performing a vaginal speculum exam during which Lugol ' s iodine solution is applied to the cervix . The cervix is then often viewed with the naked eye to identi fy specific color changes that occur due to the amount of glycogen present on the cervix . Ultimately, the result is presented as either positive or negative for potential precancerous lesions or cancer . Management involves either immediate treatment i f there is concern the patient may not return for a follow-up visit, or biopsy to confirm the precancer .

[0008] VILI is an attractive approach because it is simple and easy to learn, low-cost , with test results that are immediate, reducing the potential losses to follow-up . While VIA and the application of acetic acid is used in conj unction to assess the cervix, the results of VIA rely on the consistency of its performance . VIA relies on the assessment of the cervix at precisely 60 seconds after the application of acetic acid, after which the stain starts to fade and the abnormality visible at peak starts to subside . However, the application of acetic acid and its assessment often varies in practices around the world, where the presence of mucous , blood, etc . hampers the full exposure of the cervix to acetic acid, some practices wait less than or more than 60 seconds , and the precise timing is thus compromised . This results in a much larger potential for variability in the assessment of the cervix with the application of the acetic acid and makes assessment with Lugol ' s iodine much more attractive for its simplicity and lack of requirements on assessing the cervix a speci fic time after application .

[0009] Studies looking at the ef ficacy of VILI have reported the high sensitivity of the test at the expense of a moderate specificity that may result in over-treatment for single-visit sessions . The accuracy of the test is also subj ect to the experience of the health care provider performing the exam and his or her ability to evaluate the features of a lesion .

[0010] To find an obj ective technology with high accuracy for detecting cervical precancers / cancers , several researchers have trained Al-based image recognition models . Since these models were developed to be used in a screening setting ( rather than a triaging setting) , they were developed to provide a binary diagnosis of the presence or absence cervical abnormalities . Accuracy estimates in these models have found to be subj ect to verification bias ( i . e . , tending to be inflated) as no biopsies were obtained when cytology or colposcopy was normal . Moreover, no consistency or repeatability of examination conditions have been utilized, where in many cases a diverse quality of image capture has been found . For example , in many cases a hand-held phone is used to photograph the cervix during the examination, and the digital pictures taken by the phone used as the source of the digital images . Obviously, it is di f ficult to take good quality pictures of cervix with a hand-held phone consistently .

[0011] Thus , a need remains to improve the accuracy and specificity of visual inspection of the cervix, while also leveraging the capabilities of Al-based image recognition . In addition, improving the ability to make finer diagnostic decisions ( for example, to be able to distinguish between a low- grade lesion and a high-grade lesion) would increase the value of such a simple test, allowing it to be used more often and in settings where the level of care is not optimal

[0012] Summary of the Disclosure

[0013] The present invention is directed to an improvement in cervical examination procedures particularly useful in "screen- and-treat" and triage scenarios and, more particularly, to a system and method utili zing neural network models that are trained to find patterns of precancer / cancer among images of the cervix captured at defined points in time during the examination ( e . g . , after acetic acid application and / or after staining with Lugol ' s iodine ) .

[0014] A dedicated image capture system is used in accordance with the present invention to collect sets of images at specifically defined points in time during an examination and submit the images to trained machine learning models for evaluation and a preliminary grading in terms of the level of pre-caner / cancer found in the images by incorporating the use of trained machine-learning models that allows for a multi-level diagnostic grading to be performed, even at remote examination locations with minimal medical support .

[0015] It is an aspect of the present invention that a software- guided examination procedure may be used to direct the individual performing the examination, particularly to prompt the individual to capture one or more digital images of the cervix at defined points in time during the procedure . Inasmuch as each examination setting is presumed to utili ze the same / similar guided prompts , the digital images captured for each exam will consistently ( and repeatedly) be collected at the same point ( s ) in time during the cervical examination . As a result , a computer-assisted image analysis (machine learning model ) may be used to analyze the collected images and provide reliable diagnostic decisions of a finer grading than the simple "positive / negative" of standard visual image evaluations performed by the personnel at the examination facility .

[0016] In further accordance with the principles of the present invention, trained diagnostic models can be incorporated within computer processing components of the local image capture system being utili zed to perform cervical examinations , thus providing on-edge analysis without the need for wi- fi connection, making the inventive system useful for low resource settings (but not limited to these environments ) . Indeed, an advantage of the present invention is considered to be its portability and relative ease-of-use at locations with limited medical facilities , particularly in light of the confidence in properly grading the images collected during cervical exams .

[0017] More particularly, the teachings of the present invention are directed to developing, training, and validating machine- learning models that are initially built by using existing images of known histological diagnoses (including an absence of cancer) . The existing images are confirmed as collected at the same points in time during the examination process (e.g., a specific time interval subsequent to the application of acetic acid application, and / or after the application of Lugol's iodine, where the latter may include a further image confirmation that the complete cervix has been stained) and associated with a specific diagnosis, allowing for these images to be used to train and validate the selected machine learning models. For example, the software program may be configured to ensure that the images are collected at a defined time interval of within 60-120 seconds after the application of acetic acid, as well as ascertaining when the entire cervical area has been stained with Lugol's iodine.

[0018] Additionally, beyond the defined image collection points associated with the application of acetic acid and Lugol's iodine, it is contemplated that models may be trained to evaluate images captured at other points in the examination procedure; for example, after an initial saline wash step or before / after a biopsy is performed. Moreover, the images may be captured under different light settings (e.g., white light vs. green light) , with both sets of images submitted to specif ically- trained models for evaluation and grading.

[0019] The models may also be trained to recognize more (or fewer) levels of grading for diagnostic purposes. While an example three-level diagnostic grading is discussed in detail below, a four- or five-level grading may be used in training a model. For example, incorporating additional levels to further differentiate between "CIN 1" and "CIN 2" cases.

[0020] In general, the inventive techniques are broadly directed to training and using machine-learning models to provide diagnostic assessment of cervical images captured at any defined point in time during an examination, and perform the assessment in terms of a grading against a multi-level diagnostic scale of any pre-defined level of granularity beyond the two-level "positive" / "negative" of the prior art .

[0021] In an example embodiment , the collected images are first screened ( filtered) by the technician, or using computer capabilities such as algorithms , to eliminate any particular image that is of poor quality . The remaining images in the set may then be additionally processed ( e . g . , cropped, rotated, or the like ) to provide an optimum quality image set for review by the trained model .

[0022] Another aspect of the present invention is directed to the methodology followed in the training, validating and testing the created models in a manner that ensures reliable results in terms of attributing a selected diagnosis grade to current examination images . In one example , a k-fold cross-validation technique is used, with a plurality of m quality-ranked image sets associated with each fold .

[0023] An exemplary embodiment of the present invention may take the form of a system for detecting and grading cervical abnormalities , comprising : a dedicated image capture system configured to respond to a guided process for capturing images of the cervix at pre-defined points in an examination process , a computing device coupled to the dedicated image capture system and including a guided instruction set for prompting a user through a defined set of steps for performing a cervical examination, and an image evaluation and assessment module for receiving as an input the images captured by the dedicated image capture system, the image evaluation and assessment module including a processor component , a memory component , and one or more trained neural network models for evaluating the received images and providing as an output a report including a grading of the images based upon a set of categories used in the training of the neural network models .

[0024] Another embodiment of the present invention may take the form of a method of providing detection and grading cervical abnormalities during a cervical examination procedure . An example method includes the steps of : providing one or more trained neural network models for reviewing images of a cervix and providing a diagnostic assessment with respect to the presence of pre-cancer or cancer; installing the one or more trained models within a local computing system at an examination location; performing a guided cervical examination procedure , including image capture at predetermined time points during examination; submitting images captured by the cervical examination to an image assessment and evaluation module ; screening submitted images to delete defective images ; selecting a proper trained model from the one or more trained models ; submitting the screened images to the selected trained model ; performing an evaluation of the screened images with respect to the selected trained model ; and generating an output report including a diagnostic grade based upon the model ' s evaluation of the submitted images .

[0025] Some embodiments of the present invention may utili ze models that are specifically trained on images from HPV-positive patients so as to provide a reliable and accurate point-of-care diagnostic capability, crucial in providing di f ferentiation between, for example , low-grade pre-cancer lesions and highgrade pre-cancer lesions . More broadly, other embodiments may utilize images from both HPV-positive and HPV-negative patients ( as well as patients having no indicia of pre-cancer lesions ) .

[0026] Other and further aspects and embodiments of the present invention will become apparent during the course of the following discussion and by reference to the accompanying drawings .

[0027] Brief Description of the Drawings

[0028] Referring now to the drawings ,

[0029] FIG . 1 is a block diagram of an example cervical examination system including automated detection and assessment ( grading) of abnormalities found in digital images captured during the exam;

[0030] FIG . 2 is depiction of a set of steps that may be included in a software-guiding cervical examination procedure , useful in ensuring that digital images are captured at the proper points in time ;

[0031] FIG . 3 is a flow chart of an exemplary method of performing automated grading of cervical abnormalities in accordance with the principles of the present invention;

[0032] FIG . 4 is a flow chart of an exemplary procedure for creating trained machine-learning models for performing automated grading the method shown in FIG . 3 ; and

[0033] FIG . 5 contains a table of performance metrics utili zed to evaluate the trained models prior to distribution for use in local examination locations .

[0034] Detailed Description

[0035] A system and method for detecting and grading cervical cancers and pre-cancerous conditions is proposed that is based upon utilizing Al-based machine learning models to compare existing ( i . e . , model-trained) images of known histological diagnoses (hereinafter referred to at times as "ground truth images" ) with images of the cervix obtained during examination . It is an aspect of the present invention, as will be discussed in detail below, to control / guide the cervical examination procedure such that the images captured during a current procedure are collected at the same points in time as the ground truth images used in model training process . That is , by controlling the speci fic points in the examination procedure where images are collected ( e . g . , 60- 120 seconds after the application of acetic acid, after confirmation of cervix staining coverage during the application of Lugol ' s iodine , etc . ) , a trusted machine-learning model can be trained for use in evaluating newly-collected images with a high level of confidence . Indeed, the image capture process may be automated to ensure that they are collected at the proper time points after speci fic steps in the examination procedure .

[0036] FIG . 1 illustrates an exemplary cervical examination system 10 that utili zes machine-learning based methodologies to enable users to capture, store , and analyze multiple images of the cervix collected during an examination procedure . System 10 is shown as comprising a dedicated image capture device 12 that includes in this example a complementary metal-oxide semiconductor (CMOS ) sensor 14 and LED-driven optics 16. Dedicated image capture device 12 may also be formed to exhibit auto-focusing capabilities to generate image data of consistent guality . In this embodiment , dedicated image capture device 12 takes the shape of a small orb that is connected to a computing device 18 ( e . g . , a tablet computing device , smartphone device , or other computing device including a display component ) included within system 10 . Tablet computing device 18 is configured to store software developed specifically to guide users through a structured examination procedure , with the GUI 18-G of tablet 18 preferably used to provide a visual form of the software-guided examination procedure , preferably indicating a timeline of the procedure and the time-sensitive prompts for collecting images of the cervix .

[0037] Also shown in FIG . 1 is an image evaluation and assessment module 20 used to provide detection and assessment of cervical abnormalities based on captured images in accordance with the principles of the present invention . Module 20 includes a processor 22 , a memory 24 , and one or more trained neural network models 26. As will be described below in association with FIG . 3 , module 20 receives images captured during an examination and processes the images using the appropriate trained model ( s ) 26 to generate as an output a diagnostic report ("grading" ) of the pathology of the images . It is to be understood that while module 20 is shown as a separate component , it may be incorporated into computing device 18 in many situations . Also shown in FIG . 1 is a secure cloud infrastructure 30 that may be in communication with system 10 through a wi- fi enabled device syncing process . As discussed below, the trained neural network models may be developed within cloud infrastructure 30 and then downloaded to local systems 10 for installation once they have be verified as ready for use .

[0038] Processor 22 may comprise any device or system of devices that performs processing operations . A processor will generally include a chip, such as a single core or multi-core chip to be configured to provide a central processing unit (CPU) for performing di f ferent steps in the analysis of cervical images for detection and grading of abnormalities . Memory 24 may contain one or any combination of memory devices . A memory device is a physical device that stores data or instructions in a machine-readable format . Memory may include one or more sets of instructions ( i . e . , software ) which, when executed by one or more processors 22 can accomplish the various tasks required by the inventive system and method . Included may be a non- transitory memory device such as a solid state drive , flash drive, disk drive , hard drive , subscriber identity module ( SIM) card, secure digital card ( SD card) , micro SD card, or solid data drive ( SSD) , optical and magnetic media, others , or a combination thereof . Using the described components , image evaluation and assessment module 20 is operable to produce a report ( in this case in the form of a grading associated with the evaluated images ) , providing the report to the user via an input / output component 21 . As discussed below the generated report may also include a copy of a particular image used in arriving at the diagnosis and perhaps a heatmap .

[0039] The details of an exemplary software-guided examination procedure that may be utilized by tablet computing device 18 are described in detail in international patent application PCT / US2020 / 053368 , filed September 30 , 2020 , assigned to the same entity as this application, and incorporated herein by reference . The software generates a series of screens / prompts that assist the user in navigating between patients and procedures , and in particular, for a given patient, guides the user through a sequence of prescribed steps representative of a full cervical examination procedure .

[0040] FIG . 2 depicts an example software-guided examination procedure including pre-defined time prompts for instructing the user to capture the required images of the cervix . In this example , the procedure includes a first time prompt T PA at 60- 120 seconds after the application of acetic acid, and a second time prompt T PL after confirmation of suf ficient cervix staining coverage after the application of Lugol ' s iodine . The latter time prompt may incorporate an Al-based ( or algorithmic) evaluation of the staining, with the guided procedure either prompting the user to apply additional Lugol ' s iodine to the cervical region under examination, or confirmation suf ficient coverage and prompting the user to capture a set of images .

[0041] As mentioned above and discussed in further detail below, images may also be captured at other points in time during the examination process , such as after the saline wash or before / after performing a biopsy, as also shown in FIG . 2 . Additionally, in some situations , the images may be captured under di f ferent lighting conditions ( shown in FIG . 2 as "w" for white light and "g" for green light ) , with all of these images properly tagged with this information so that the appropriate trained model is used . Indeed, as will also be discussed below, the metadata will include a time prompt indicator ( TPA / TPL ) , lighting indicator (W / G) , and other relevant information .

[0042] Altogether, system 10 enables users to capture , store , and analyze images of the cervix that are collected at defined times in the procedure . The technician / viewer may also be able to digitally magnify the images for a more thorough assessment . The standardi zation of the image set further improves the ability of the models to properly evaluate and categorize the images .

[0043] FIG . 3 contains a flowchart of an example overall method of using cervical examination system 10 , in combination with the procedure of FIG . 2 , to perform cancer / pre-cancer detection and grading in accordance with the principles of the present invention . An initial set of steps in FIG . 3 are actually associated with the creation of the machine-learning (neural network) models and the installation of the trained models in the local examination equipment . These steps typically occur only infrequently ( excepting for updates in the modeling particulars , where for example the model may be updated after having been trained on more data) , with the following steps performed in each cervical examination procedure . In particular, a first preliminary step Pl is associated with the creation of proper neural network models for use in detecting and assessing pre-cancer / cancer diagnoses in accordance with the principles of the present invention . A following preliminary step P2 instructions for the trained models to be installed in the local cervical examination equipment ( such as system 10 shown in FIG . 1 ) .

[0044] With these " initiali zation" steps being completed, a standard software-guided cervical examination procedure may begin at step 100 with the collection of patient information, and linking a procedure ID to the patient record . At least a portion of the patient information will be used as metadata associated with the collected images ( e . g . , patient age, current HPV status , etc . ) .

[0045] A following step 110 is shown in FIG . 3 as associated with performing the actual cervical examination, as described above in association with FIG . 2 and discussed in detail in our copending application . One end-result of the examination process is the creation of sets of digital images of the cervix captured at the speci fic time prompts TP as described above . The ability to control / def ine the image collection time points via prompts in the software-guided procedure is an important feature in obtaining consistent images ( data) that may be analyzed using machine-learning models and provide a multi-level grading with a reasonable degree of certainty .

[0046] Another aspect of this step is that the user may be guided by image quality indicators in the process of capturing images in order to ensure that good quality images are being captured . As discussed above , the use of image quality evaluation may be particularly important in confirming a suf ficient staining of the cervix before capturing images at the Lugol ' s iodine application step of the process . The image quality evaluation may include the use of algorithms that measure the focus, brightness, amount of specular reflection, detection and magnification of the cervix, as well as the amount of blood detected in the image.

[0047] Continuing with the description of the process in FIG. 3, the collected images are next screened ("filtered") in step 120 to eliminate any individual images that are not usable (i.e., considered defective) . The images are also reviewed with respect to the presence of obstructing objects (like cotton swabs or a speculum) as well as certain image guality characteristics such as focus, brightness, and specular reflection. In the present embodiment, these quality characteristics are combined and compared to those of the images included in the training model. Here, the user may also be included in the image quality assessment, whereby the user may select or de-select certain captured images to retain for use only thus of sufficient quality.

[0048] In the following step 130, the filtered images are then processed, which may include performing a crop to a square size around the cervix to exclude any non-cervical artifacts. Color normalization may also be performed as part of this processing step. The image sets created at the completion of step 130 are then considered to be in a best form for evaluation by a selected trained model.

[0049] In the following step 140, an appropriate trained model 26 is selected, based on the type of images being reviewed. For example, a first model 26A may be selected to process image data associated with the acetic acid procedure step (collected at TPA) , or a second model 26L used to process image data associated with the Lugol's iodine procedure step (collected at TPL) . The images are then submitted to the selected trained model (step 150) , which then performs a grading of the images with respect to a set of pre-defined diagnostic categories . This may be presented as probabilities that the input image belongs to any of the defined diagnostic categories . In one example , a set of three categories is used in evaluation of the images , namely : ( 1 ) normal / LS IL, ( 2 ) HS IL, and ( 3 ) invasive cancer . Thus , rather than giving a binary diagnosis of negative or positive ( as is typical for many prior art procedures ) , this finer discrimination possible with the inventive technique allows for the healthcare provider to make a more informed management decision given local resources and settings . The grading of the images is provided as an output report from the diagnostic assessment process , step 160 in the flow chart of FIG . 3 .

[0050] Alongside the model prediction, the output report may include a copy of a speci fic input image so that the user is aware of what the model is using as its basis of prediction . In one aspect of this invention, the input image may also be overlayed with a heatmap showing the areas of the cervical image that the model found important in arriving at its prediction output . This is to aid the user in determining the reliability of the output report . The overlaid heatmap can also be used to assess what might be the "worst" or "diagnostically relevant" areas on the cervix that may warrant further inspection .

[0051] An aspect of the present invention is the process involved in training and testing known machine-learning models to properly assess the image data collected during the examination process described above . The following paragraphs , in association with the flowchart of FIG . 4 , describe in detail an exemplary procedure for model training, validation, and testing that may be used as part of the inventive procedure .

[0052] In particular and as mentioned above , the method of the present invention is based upon the development and training of neural network models to output the probability of a cervical image as belonging to one of a number of defined categories. A critical aspect of this model building process is the use of a large number of images (of known cervical histopathology) as its foundation (i.e., "ground truth" images) . In a preferred embodiment, a relatively large number of cervical images with known histological diagnoses that have been previously collected from patients (and have been properly de-identif led) are used to develop multiple machine learning algorithms that may be used. As described below, the created models are thereafter evaluated on a test set of images unseen by the models during their training and validation prior to being used in an actual examination procedure.

[0053] In terms of developing a suitable set of trained models for detecting and grading cervical pre-cancer / cancer, there are at least two initial parameters to consider: (1) at what points in time during the cervical examination will digital images be captured (i.e., the "prompts" as mentioned above) , which determines the number of individual models that need to be created (or in the case of multiple-image models, how many image inputs that model will have) ; and (2) how many grading levels / tiers should the model employ in providing a diagnostic output .

[0054] With respect to the first parameter, it is contemplated that in many cases of analyzing images from a cervical examination procedure, at least two models (or a model that takes at least two different image inputs) are beneficial; one trained to analyze acetic acid image data (TPA) , and another trained to analyze Lugol's iodine image data (TPL) . With respect to the second parameter, one exemplary training process may be trained to define three different categories of diagnosis - Normal / LSIL, HSIL, and invasive cancer. In another enhanced embodiment regarding this second parameter of grading levels, pl 6 stains of HSIL images may be used to further discriminate certain cases as being normal / low or truly HS IL . Specifically, in training the model , in addition to ground truth histology, pl 6 stains can be used to further discriminate "CIN 2" cases when labeling it for training the model so that the model is then better equipped during an examination to recogni ze an otherwise CIN 2 case as being either normal / low or truly HS IL .

[0055] In further accordance with the principles of the present invention, the collection of multiple images from a given procedure after the application of both acetic acid and Lugol ' s iodine is contemplated to allow for the development of the models to be trained on both sets of images . In practice, clinics may either use only the model trained on images collected after application of acetic acid or use this model in conj unction with the one that was trained after applying Lugol ' s iodine in assessing captured cervical images .

[0056] Referring to the flowchart of FIG . 4 , a first step 200 in the model making process is to collect existing images with known cervical histopathology . The existing images and their associated metadata are then confirmed ( step 210 ) as having been obtained at the specific points in time of the examination process . A relatively small-sized set of images from this confirmed collection is then set aside ( step 220 ) for later use in testing the developed model .

[0057] The large group of images not withheld is then used for training and validating a created model . These remaining images are first filtered ( step 230 ) to remove any images containing obstructing obj ects ( such as , for example , cotton swabs , speculum, etc . ) from further consideration . Preferably, an algorithm is used to perform this filtering to maintain a consistent filtering decision threshold . The remaining, filtered set of images is then separated into k equal folds for cross-validation processing (step 240) . For exemplary purposes only, it will be presumed that a 5-fold organization of images is created. The procedure ID from the metadata associated with the images may be used to ensure that images from the same procedure are assigned to the same fold.

[0058] A quality ranking of the images within each fold is then performed (step 250) . The quality of each image may be evaluated in terms of properties such as focus, brightness, level of specular reflection, and the like, with the presumption that devoted algorithms are used to measure these characteristics. Before moving onto the next step in the model building process, the lowest-ranked images in each fold are dropped out from further consideration (for example, dropping the lowest-ranked 20% of each fold) . Going forward from this step, quality thresholds will be associated with what percentage of the ranked images are retained for use.

[0059] The retained higher-quality images are then cropped (step 260) , with individual folds augmented such that any class imbalances are mitigated (step 270) . For example, the existing images may be "duplicated" and then subjected to additional processing (using, for example, random crops, flips, limited rotations, saturations, and brightness changes) such that images are augmented more frequently for the class or combination of classes (e.g., HSIL + cancer) that have fewer images. A model is then selected for use, where the model (which may be pretrained on a much larger dataset of images that can be unrelated to cervical images associated with present purposes) is finetuned using the pre-processed and augmented image set provided at steps 260 and 270. In accordance with known model training techniques, several iterations are then performed (step 280) to optimize training parameters and improve the model's performance on the validation set (i.e., one of the folds; in the 5-fold example , four folds are used for training and one for validation) .

[0060] Subsequent to this iterative training process , the final versions of the trained models are evaluated by using the image set withheld in step 210 . The test set images are first filtered for image quality before inference with the same thresholds used to filter image quality in the training and validation of the model . Since the test set can contain multiple images for a given procedure , the model performance presented, in terms of ROC AUG, accuracy, sensitivity, and specificity is based on the test set consisting of only one image per procedure , where the image used is the first one captured for which the model gave the most severe prediction .

[0061] To assess the discriminatory performance of a model with the above-defined three categories of grading, the probabilities given by the model in each of these categories for images in the test set may be used to calculate ROC curves . Because the ROC curve is used to measure the performance of binary classi fiers , in order to capture the performance of a model that gives three outputs , the ROC AUG is calculated as the average of the area under the curve of the ROC curves generated from all possible pairwise combinations of the three categories being either positive or negative . For each pair that can be generated from the three categories (normal / LS IL vs . HS IL, HS IL vs . invasive cancer, normal / LS IL vs . invasive cancer) , the mean ROC curve is determined when each category in the pair is considered as either positive or negative . From the three mean ROC curves that are generated for each pair (normal / LSIL vs . HS IL, HS IL vs . invasive cancer, normal / LSIL vs . invasive cancer ) , the average of these is taken and used to determine the overall ROC AUC .

[0062] Accuracy is calculated as the percentage of the total number of images that the model made inferences on that are correctly classified as being from women with disease (HSIL or invasive cancer) or as being disease-free (normal or LSIL) . Specificity of a model is calculated as the number of images correctly classified as normal / LSIL out of the total number of images inferred on with ground truth of histopathology proved normal or LSIL. is was calculated as the number of images correctly classified as HSIL or invasive cancer out of the total number of images inferred on with ground truth of HSIL or cancer on histopathology.

[0063] Once the models are trained (and tested as described in associated with FIG. 5, below) , they are loaded into the individual systems used to perform the examinations so that a real-time diagnosis may be obtained as part of the examination process as described in above in association with FIG. 3.

[0064] Regarding the testing of trained models, FIG. 5 includes a table of performance metrics (i.e., accuracy, sensitivity, and specificity) used to measure the performance of the trained models to detect HSIL+ lesions on a given test dataset. The testing is performed by using the set of test images that were set aside before model training FIG. 4, step 220) . Additionally presented is the use of both models in providing the final prediction, shown in the last two rows of the FIG. 5 table, where for combination-1, the final prediction is taken as the most severe prediction among the acetic acid model or the Lugol's iodine model wherever predictions from both models are available. Otherwise, if the quality of the image (s) was found to be insufficient for one model, the prediction given by the other model is used. In combination-2, only the procedures that the acetic acid model was able to make a diagnostic prediction are considered. Here, the final prediction is taken as the most severe among the acetic acid model or the Lugol's iodine model wherever predictions from both models are available. Otherwise, if the quality of the image (s) was found to be insufficient for the Lugol's iodine model, the prediction given by the acetic acid model is used. Because each model assesses an image quality before making a diagnostic assessment, each model gave predictions for a different number of procedures from the test set. The combined-1 model had a sensitivity of 80.5%, a specificity of 84.9% and an overall accuracy of 83.3%. Acetic acid only model had lower sensitivity (73.9%) compared to the combined-1 and combined-2 models, but higher specificity (87.3%) .

[0065] In addition, repeatability for a model may be assessed by taking into consideration cases where more than one image is captured at a particular step in the cervical examination procedure. This is possible since timepoints are included as metadata that is retained with the images. Specifically, the model's inferences on the first two or three images captured within 30 seconds of each other (at the same procedure step) may be compared.

[0066] Furthermore, explainability of a model may be assessed with a method that uses the gradients and feature maps of the last convolutional layer in the model to produce coarse localization heatmaps that highlight specific regions in an image that were used by the model to make its prediction. This importantly verifies that the model has picked up on, and learned, the important features or regions on the cervix that are relevant for diagnostic grading. With this training approach, it is largely seen that the heatmaps identify areas on the cervical transformation zone as the relevant region. Interestingly, when the hotspots appear elsewhere, or are disaggregated and patchy, the model's prediction probabilities tend to be lower.

[0067] Although implementation of a system and method for assessing cervical abnormalities has been described in language specific to structural features and / or methods , it is to be understood that the appended claims are not necessarily limited to these speci ficities . Rather, the speci fic features are presented as an example only of the contemplated system and method so as to aid the reader in understanding the presented attributes .

Claims

What is claimed is :1 . A system for detecting and grading cervical abnormalities , comprising : a dedicated image capture system configured to respond to a guided process for capturing images of the cervix at pre-defined points in an examination process ; a computing device coupled to the dedicated image capture system and including a guided instruction set for prompting a user through a defined set of steps for performing a cervical examination; and an image evaluation and assessment module for receiving as an input the images captured by the dedicated image capture system, the image evaluation and assessment module including a processor component , a memory component, and one or more trained neural network models for evaluating the received images and providing as an output a report including a grading of the images based upon a set of categories used in the training of the neural network models .2 . The system as defined in claim 1 , wherein the computing device includes a graphical user interface for displaying individual steps of the guided instruction set including prompts for initiating collection of images .

3. The system as defined in claim 1 , wherein the guided instruction set includes a step instructing the application of acetic acid and prompting image capture within a predetermined time period after the application .4 . The system as defined in claim 3 , wherein the prompting for image capture is set to occur within a time frame of 60-120 seconds after application of acetic acid .5 . The system as defined in claim 1 , wherein the guided instruction set includes a step instructing the application of Lugol ' s iodine and prompting image capture subsequent to the application of Lugol ' s iodine .

6. The system as defined in claim 5 , wherein the prompting for image capture occurs subsequent to a confirmation of suf ficient cervix staining with Lugol ' s iodine .7 . The system as defined in claim 6 wherein confirmation of suf ficient cervix staining is performed by automated image evaluation .8 . The system as defined in claim 1 wherein the image evaluation and assessment module comprises a first trained neural network model for evaluating acetic acid images and a second trained neural network model for evaluating images associated with the application of Lugol ' s iodine .

9. The system as defined in claim 8 wherein the image evaluation and assessment module further comprises additional trained neural network models for evaluating images captured in association with additional steps of the guided procedure , as defined by included prompts .10 . The system as defined in claim 1 , wherein the image evaluation and assessment module comprises one or more trainedneural network models created to evaluate images captured under di f ferent lighting conditions .11 . The system as defined in claim 1 , wherein the computing device is configured to provide prompts to a system user to manually perform image capture .12 . The system as defined in claim 1 , wherein the computing device configured to include automated prompts for image capture by the system itself .

13. A method of providing detection and grading cervical abnormalities during a cervical examination procedure , including providing one or more trained neural network models for reviewing images of a cervix and providing a diagnostic assessment with respect to the presence of pre-cancer or cancer ; installing the one or more trained models within a local computing system at an examination location; performing a guided cervical examination procedure, including image capture at predetermined time points during examination; submitting images captured by the cervical examination to an image assessment and evaluation module ; screening submitted images to delete defective , poor quality images ; selecting a proper trained model from the one or more trained models ; submitting the screened images to the selected trained model ; performing an evaluation of the screened images with respect to the selected trained model ; andgenerating an output report including a diagnostic grade based upon the model ' s evaluation of the submitted images .14 . The method of claim 13 , wherein the one or more trained neural network models are configured to perform a multilevel grading of at least three diagnostic levels , ranging from normal to IL to invasive cancer .15 . The method of claim 13 , wherein the images captured during the cervical examination are further subj ected to preprocessing subsequent to screening to create images of similar form in terms of one or image characteristics selecting from the group consisting of : focus , brightness , amount of blood, specular reflection, cropping image closer to cervix, and normali zing color space .

16. The method of claim 13 , wherein the images selected for submission to the selected model are manually selected by the user .17 . The method of claim 13 , wherein the generated output report further includes an overlay over the input image to show regions of the image used by the selected model in arriving at diagnostic grade .18 . The method of claim 13 , wherein the step of providing one or more trained neural network models includes the steps of collecting a number of existing cervical images of known histopathology; confirming each existing image as collected during a time period consistent with a prompting of the examination procedure , deleting any images failing the confirmation;reserving a group of confirmed images of known histopathology for later testing and defining remaining images as a group of input images ; selecting a neural network model ; submitting the group of input images to the selected neural network for training and validation in an iterative process to generate a final set of trained models ; testing the final set of trained models with the reserved group of confirmed images of histopathology; evaluating performance of each trained model during testing; and selecting for use as a final version of a trained model the one with a highest level of correct diagnostic grading with respect to the known histopathology .

19. The method of claim 18 , wherein prior to submitting the group of training images to the selected neural network model , the method further comprises the steps of : ranking the group of input images against each other in terms of image quality parameters selected from the group consisting of : focus , brightness , amount of blood, and specular reflection; determining image quality thresholds from the ranking to apply to input images for inference ; filtering the group of input images with respect to the determined thresholds ; pre-processing the input images using techniques such as cropping closer to the cervix, normalizing color space ; and augmenting the input images while mitigating class imbalances by applying one or more functions to the original input images selected from group consisting of : random crops , flips , limited rotations , saturations , and brightness changes .

Images