Efficient AI analysis of images by combining predictive models

Abstract

Systems, image analyzers, and methods are provided for diagnosing the presence of disease or illness in images of a subject, e.g., a veterinary subject, that include classifying images into body regions and obtaining classified, labeled, cropped, and oriented sub-images; sending the sub-images to an artificial intelligence processor to obtain evaluation results and comparing the evaluation results to a database library of evaluation results and to matched written templates or dataset clusters to obtain at least one cluster result; measuring the distance between the cluster results and the evaluation results to obtain at least one cluster diagnosis; and assembling the cluster diagnosis and matched written template to obtain a report for display to a radiologist. These systems, analyzers, and methods are accomplished in significantly reduced turnaround time, useful for cost and time savings.

Description

[Technical Field]

[0001] Related Applications This application claims the benefit of and priority to U.S. Provisional Patent Application No. 62 / 954,046, filed December 27, 2019, entitled "Efficient Artificial Intelligence Analysis of Radiographic Images," by inventors Seth Wallack, Ariel Ayaviri Omonte, and Ruben Venegas, and U.S. Provisional Patent Application No. 62 / 980,66, filed February 24, 2020, and U.S. Provisional Patent Application No. 63 / 083,422, filed September 25, 2020, entitled "Efficient Artificial Intelligence Analysis of Radiographic Images with Combined Predictive Modeling," by inventors Seth Wallack, Ariel Ayaviri Omonte, Ruben Venegas, Yuan-Ching Spencer Teng, and Paratheev Sabarantnam Streetharan, each of which is incorporated herein by reference in its entirety. [Background technology]

[0002] Artificial intelligence (AI) processors, e.g., trained neural networks, are useful for processing radiological images of animals to determine the likelihood that the imaged animal has a particular disease. Typically, different AI processors are used to evaluate different body regions (e.g., chest, abdomen, shoulders, forelimbs, hindlimbs, etc.) and / or specific orientations of each such body region (e.g., dorsoventral (VD) view, lateral view, etc.). A particular AI processor determines the likelihood that a particular disease is present for a particular body region of interest in each body region and / or orientation of each body region. Each such AI processor includes multiple trained models for evaluating a respective disease or organ within the imaged region. For example, for a lateral view of an animal's chest, the AI ​​processor uses different models to determine the likelihood that the animal has a particular lung-related disease, such as perihilar infiltrates, pneumonia, bronchitis, or pulmonary nodules.

[0003] The amount of processing performed by each individual AI processor and the amount of time required to complete that processing is significant. The tasks require either (1) manual identification and cropping of each image to define a specific body region and orientation before the image can be evaluated by a particular AI processor, or (2) presentation of the image to an AI processor for evaluation. Unlike human radiology, where radiological examinations are limited to specific areas, veterinary radiology routinely involves multiple unlabeled images with multiple body regions of unknown orientations in a single examination.

[0004] In a conventional workflow for processing animal radiology images, the system assumes that a user-identified body region is included in the image. The user-identified image is then sent to a specific AI processor that uses a machine learning model to evaluate, for example, the likelihood of a pathology being present for a particular body region. However, requiring the user to identify a body region makes the conventional workflow cumbersome and introduces errors when the identified body region is incorrect or when multiple regions are included in the image. Furthermore, if an image without a user-identified body region is sent to the system, the conventional workflow becomes inefficient (or even fails). When this occurs, the conventional workflow is inefficient because the unidentified image is sent to multiple AI processors that are not specialized for the imaged body region. Furthermore, the conventional workflow produces different results because inaccurate region identifications are sent to AI processors that evaluate different body regions.

[0005] A traditional workflow for using AI to analyze the diagnostic characteristics of radiological images and prepare a report based on the AI ​​model diagnosis results in an exponential number of possible output reports. The AI ​​model diagnosis results in either a normal or abnormal verdict for a particular disease. Some AI models also provide a severity rating for a particular disease, e.g., normal, minimal, mild, moderate, or severe. The collection of AI model diagnosis results determines which report should be selected from pre-built report templates. The process of creating and selecting a single report template from the collection of AI model diagnosis results scales exponentially with the number of AI models. Normal / abnormal diagnosis results from six different AI models would require 64 different report templates (2 to the power of 6). 10 models would require 1,024 templates, and 16 models would require 65,536 templates. An AI model that detects even worse severity scales, e.g., an AI model with 16 severity levels, each with five possible severity levels, would require over 150 billion templates. Therefore, manually generated reports for each combination of AI model diagnostic results are not adequately calibrated for the large number of AI models being interpreted together.

[0006] Therefore, what is needed is a novel system with various fully automated stages of image processing and image analysis, including determining whether an incoming image contains a particular body region in a particular orientation (such as a lateral view), appropriately cropping the image, creating one or more sub-images from the original image with one or more body regions or regions of interest, labeling the original image and any created sub-images, and evaluating the cropped image and sub-images against a targeted AI model, and further analyzing and providing a diagnostic radiologist report based on multiple examination results, including but not limited to the AI ​​model results. Summary of the Invention

[0007] One aspect of the invention described herein provides a method for analyzing diagnostic radiological images or images of a subject, the method comprising automatically processing the radiological images of the subject using a processor to classify the images into one or more body regions or orient and crop the classified images, thereby obtaining at least one oriented, cropped, and labeled sub-image for each automatically classified body region, directing the sub-images to at least one artificial intelligence processor, and evaluating the sub-images by the artificial intelligence processor to analyze the radiological images of the subject.

[0008] One embodiment of the method further includes using an artificial intelligence processor to assess the sub-images for body region and the presence of a pathology, such as the chest, abdomen, forelimbs, hind limbs, etc. One embodiment of the method further includes using an artificial intelligence processor to diagnose a pathology from the sub-images. One embodiment of the method further includes using an artificial intelligence processor to assess the sub-images for the position of the object. One embodiment of the method further includes correcting the positioning of the object to an appropriate positioning.

[0009] In one embodiment of the method, the processor automatically processes the radiographic image to obtain the sub-images rapidly. In one embodiment of the method, the processor processes the radiographic image to obtain the sub-images in less than about 1 minute, less than about 30 seconds, less than about 20 seconds, less than about 15 seconds, less than about 10 seconds, or less than about 5 seconds. In one embodiment of the method, the evaluating step further includes comparing the sub-images to a number of reference radiographic images in at least one of a number of libraries. In one embodiment of the method, each of the multiple libraries includes a respective one of the reference radiographic images.

[0010] In one embodiment of the method, the multiple libraries each include multiple reference radiographs, which may be species-specific or non-specific. One embodiment of the method further includes matching the sub-images to the reference radiographs to assess orientation and at least one body region. In one embodiment of the method, the reference radiographs are oriented within the Digital Imaging and Communication in Medicine (DICOM) standard hanging protocol.

[0011] In one embodiment of the method, cropping includes isolating specific body regions in the sub-images. In one embodiment of the method, the cropping further includes classifying the reference radiograph by veterinary radiography standard body region markers. In one embodiment of the method, the orienting further includes conforming the radiograph to a standard hanging protocol for veterinary radiographs. In one embodiment of the method, cropping further includes cropping the sub-images of the radiograph to a standard aspect ratio. In an alternative embodiment of the method, cropping does not further include cropping the sub-images of the radiograph to a standard aspect ratio. In one embodiment of the method, classifying further includes identifying and labeling body regions by veterinary standard body region markers. In one embodiment of the method, classifying further includes comparing the radiograph to a library of sample standard radiographs.

[0012] An embodiment of the method further comprises matching the radiological image to a sample image in the library to classify the radiological image into one or more body regions. In an embodiment of the method, cropping further comprises identifying boundaries of the radiological image that depict each classified body region. An embodiment of the method further comprises extracting a signature of the radiological image prior to classifying. In an embodiment of the method, the radiological image is obtained from a radiological examination selected from radiological images, i.e., X-ray, magnetic resonance imaging (MRI), magnetic resonance angiography (MRA), computed tomography (CT), fluoroscopy, mammography, nuclear medicine, positron emission tomography (PET), and ultrasound. In an embodiment of the method, the radiological image is a photograph.

[0013] In one embodiment of the method, the subject is selected from mammals, reptiles, fish, genus, amphibians, chordates, and birds. In one embodiment of the method, the mammal is selected from dogs, cats, rodents, horses, sheep, cattle, goats, camels, alpacas, buffalo, elephants, and humans. In one embodiment of the method, the subject is selected from pets, livestock, valuable zoo animals, wild animals, and research animals. One embodiment of the method further includes automatically generating at least one report involving evaluation of the sub-images by an artificial intelligence processor.

[0014] One aspect of the invention described herein provides a system for analyzing radiological images of a subject, the system including: a receiver that receives the radiological image of the subject; at least one processor that automatically performs image recognition and processes algorithms to identify, crop, orient, and label at least one body region in the image to obtain a sub-image; at least one artificial intelligence processor that evaluates the sub-image; and a device for displaying the sub-image and displaying the evaluated artificial intelligence results.

[0015] In one embodiment of the system, the processor rapidly and automatically processes the radiographic images to obtain sub-images. In one embodiment of the system, the processor processes the radiographic images to obtain tagged images in less than 1 minute, less than 30 seconds, less than 20 seconds, less than 15 seconds, less than 10 seconds, or less than 5 seconds. In one embodiment of the system, the standard radiographic images comply with veterinary standards for hanging protocols and body area tagging.

[0016] One aspect of the invention described herein provides a method for quickly and automatically preparing radiological images of a subject for display, the method comprising: processing raw radiological images of the subject using a processor to algorithmically classify the images into one or more distinct body region categories by automatically cropping and extracting signatures and comparing the cropped and oriented image signatures to a database of signatures of images of known orientations and body regions to obtain the orientation and body region label that best matches; and presenting the labeled image of each prepared body region for analysis on a display device.

[0017] One aspect of the invention described herein provides an improvement in a veterinary radiological diagnostic image analyzer, comprising executing a fast algorithm by a processor that pre-processes a radiological image of a subject to automatically identify one or more body regions in the image, the processor further operative to do at least one of automatically creating separate sub-images for each identified body region, cropping and optionally normalizing the aspect ratio of each created sub-image, automatically labeling each sub-image as a body region, and automatically orienting the body regions in the sub-images, the processor further operative to specifically at least one of evaluate the cropped, oriented, and labeled sub-images, and automatically direct the diagnostic sub-images to an artificial intelligence processor.

[0018] One aspect of the invention described herein provides a method for identifying and diagnosing the presence of a disease or disorder in at least one image of a subject, the improvement comprising: classifying, labeling, and orienting the image into one or more body regions to obtain classified, labeled, and oriented sub-images; directing the sub-images to at least one artificial intelligence (AI) processor to obtain an evaluation result; comparing the evaluation result to a database with matched handwritten templates or to obtain at least one cluster diagnosis; comparing the evaluation result to at least one data cluster and measuring the distance between the cluster result and the evaluation result to obtain at least one cluster diagnosis; assembling the cluster diagnosis and obtaining a report, thereby identifying and diagnosing the presence of a disease or disorder in the subject. Evaluation result is synonymous with AI result, and is used interchangeably with AI processor result and classification result.

[0019] An embodiment of the method further includes acquiring at least one radiological image or one target data point prior to classification. An embodiment of the method further includes compiling dataset clusters using a clustering tool selected from K-means clustering, mean-shift clustering, density-based spatial clustering, expectation-maximization (EM) clustering, and agglomerative hierarchical clustering prior to comparison. In an embodiment of the method, the compiling step further includes acquiring, processing, evaluating, and building a library with a large number of known diseases or disorders of a large number of identified diagnosed datasets and corresponding medical reports selected from radiology reports, laboratory reports, histology reports, physical examination reports, and microbiology reports.

[0020] In one embodiment of the method, the processing further includes classifying the identified and diagnosed multiple dataset images into body regions to obtain a number of classified diagnosed dataset images, and orienting and cropping the classified multiple dataset images to obtain an oriented, cropped, and labeled multiple dataset image. In one embodiment of the method, the evaluating further includes directing the multiple oriented, cropped, and labeled dataset sub-images and the corresponding medical report to at least one AI processor to obtain at least one diagnosed AI processor result. In one embodiment of the method, the directing further includes classifying the multiple oriented, cropped, and labeled dataset sub-images and the corresponding medical report with at least one variable of species, breed, weight, sex, and location.

[0021] In one embodiment of the method, building a library of multiple identified and diagnosed dataset images further includes creating at least one cluster of diagnostic AI processor results by having an AI processor obtain at least one exemplary result and compiling the dataset cluster. In some embodiments, the exemplary AI processor result is an exemplary case, exemplary result, exemplary point, or exemplary. These terms are synonymous and used interchangeably. One embodiment of the method further includes assigning at least one cluster diagnosis to the cluster of diagnosed AI processor results. In one embodiment of the method, assigning the cluster diagnosis further includes adding a report within the cluster and / or additional information written by the evaluator. In one embodiment of the method, measuring further includes determining a distance between the cluster result and at least one selected from the center of the evaluation result, the dataset cluster, and the cluster result.

[0022] One embodiment of the method further includes selecting from results of the case within the cluster with the closest match, results from another case in the cluster, and results from the focal case. In one embodiment of the method, selecting further includes adding result information of the cluster result by the rater to a report generated from the cluster. One embodiment of the method further includes editing the report by removing a portion of the report of a cluster diagnosis that has an occurrence below a threshold in multiple reports within the cluster. In one embodiment of the method, the report is generated from words deemed acceptable for use in report generation. Words in the report are taken from the closest matching exemplary result or the focal case. Acceptable words for report generation are excluded if they include at least one identifier selected from a subject name, date, reference to a previous test, or any other word that could generate a report that is closest to the exemplary result and is not universally usable for all new cases. This selection process is performed using natural language processing (NLP).

[0023] In one embodiment of the method, the prevalence threshold specified by the assessor is selected to be less than about 80%. In one embodiment of the method, the assessment results are rapidly processed by a diagnostic AI processor to obtain a report. In one embodiment of the method, the diagnostic AI processor processes the images in less than about 10 minutes, less than about 9 minutes, less than about 8 minutes, less than about 7 minutes, less than about 6 minutes, less than about 5 minutes, less than about 4 minutes, less than about 3 minutes, less than about 2 minutes, or less than about 1 minute to obtain the report. In one embodiment of the method, a library of identified, diagnosed, and dataset images with known diseases and disorders are classified into at least one of a number of animal species.

[0024] An embodiment of the method further comprises identifying the analyzed AI processor results with an identification tag. An embodiment of the method further comprises selecting the images and / or images of interest and adding the medical report to the dataset cluster.

[0025] One aspect of the invention described herein provides a system for diagnosing the presence of a disease or illness in an image of a subject and / or a medical result, the system including: a receiver for receiving the medical result of the image of the subject, or by cropping, orienting, and labeling, at least one processor for automatically executing an image identification and processing algorithm to identify at least one body region in the image and obtain a sub-image; at least one artificial intelligence processor for evaluating the at least one processor sub-image and / or medical report and obtaining an evaluation result; and at least one diagnostic artificial intelligence processor for automatically executing a clustering algorithm to obtain a cluster result, comparing the evaluation result and measuring distances between the cluster result and a specific dataset defined from one or more variables, pre-created cluster results from the evaluation result to obtain a cluster diagnosis, and further obtaining the results and assembling a report.

[0026] In one embodiment of the method, the diagnostic AI processor quickly processes the images and / or medical results to automatically generate a report. In one embodiment of the method, the diagnostic AI processor processes the images and / or medical results to obtain a report in less than about 10 minutes, less than about 9 minutes, less than about 8 minutes, less than about 7 minutes, less than about 6 minutes, less than about 5 minutes, less than about 4 minutes, less than about 3 minutes, less than about 2 minutes, or less than about 1 minute. One embodiment of the method further includes a device for displaying the generated report.

[0027] One aspect of the invention described herein provides a method for diagnosing the presence of a disease or condition in at least one image of a subject, comprising: classifying an image into at least one body region; labeling, cropping, and orienting the image to obtain at least one sub-image; orienting the sub-image and directing it to at least one artificial intelligence (AI) processor to process and obtain an evaluation result; and comparing the evaluation result with a database library having at least one dataset cluster to obtain a number of evaluation results and a number of matched handwritten templates, or to obtain at least one cluster result; measuring the distance between the cluster result and the evaluation result to obtain at least one cluster diagnosis; and obtaining a report, assembling the cluster diagnosis and the matched handwritten templates, and displaying the report to a radiologist, thereby identifying and diagnosing the presence of a disease or condition in the subject.

[0028] One embodiment of the method further includes analyzing the report after display to confirm the presence of a disease or disorder. An alternative embodiment of the method further includes editing the handwritten template. In one embodiment of the method, the processing time for processing the report is less than about 5 minutes, less than about 2 minutes, or less than 1 minute. In one embodiment of the method, the processing time for obtaining the report is less than about 10 minutes, less than about 7 minutes, or less than about 6 minutes.

[0029] In one embodiment of the method, processing the sub-images further includes training an AI processor to analyze the presence of a disease or disorder in the image of the subject. In one embodiment of the method, training the AI ​​processor includes communicating a library of training images to the AI ​​processor, selecting training images from the library of training images in which the disease or disorder is present, and comparing the training images to the library of training images, thereby training the AI ​​processor.

[0030] In one embodiment of the method, the library of training images includes positive control training images and negative control training images. In one embodiment of the method, the positive training images involve the disease or condition in the training images. In one embodiment of the method, the negative training images do not involve the disease or condition in the training images. In various embodiments of the method, the negative training images may involve a disease or condition other than the disease or condition in the training images. In one embodiment of the method, the library of training images further includes at least one of medical data, metadata, and auxiliary data. [Brief explanation of the drawings]

[0031] [Figure 1] 1 is a schematic diagram of a conventional workflow for processing radiological images (102) of an animal. As commonly presented in the veterinary field, the images (102) do not show any parts of the animal. The images (102) are processed by each of multiple AI processors (104a-104l) to determine whether a body region appears on the image and the likelihood that the animal depicted in the image (102) has a particular disease. Each of the AI ​​processors (104a-104l) evaluates the images (102) by comparing them to one or more machine learning models trained to determine the likelihood that an animal has a particular disease. [Figure 2]1 is a schematic diagram of one embodiment of a system or method described herein. A radiology image preprocessor (106) is configured to preprocess an image (102) to produce one or more images (108), each corresponding to a particular perspective of a particular body region. Three sub-images (108a-c) are created, where one sub-image (108a) is identified and cropped as a lateral view of the animal's thorax, a second sub-image (108b) is identified and cropped as a lateral view of the animal's abdomen, and a third sub-image (108c) is identified and cropped as a lateral view of the animal's pelvis. As shown, sub-image (108a) is processed only by the lateral thorax AI processor (104a), sub-image (108b) is processed only by the lateral abdomen AI processor, and sub-image (108c) is processed only by the lateral pelvis AI processor (104k). In some embodiments, the sub-images (108) are tagged to identify the body region and / or viewpoint that the sub-image (108) represents. [Figure 3] 1 illustrates a set of computational operations performed by one embodiment of the system and method of the present invention for the novel workflow described herein. An image (302) is processed using the radiology image preprocessor (106) and then processed by a subset of the AI ​​processor (104) corresponding to the identified body region / viewpoint. The system displays cropped images (304a) and (304b) for each identified body region / viewpoint. The total time it took the radiology image preprocessor to determine that image (302) represented both a "lateral chest" image and a "lateral abdomen" image was 24 seconds, as reflected by the timestamp for the log entry corresponding to bracket (306). [Figure 4] A set of findings, where findings (401-407) are a set of conventional single disease-based organ findings for the lung in radiological images, and below that are combinations of at least two single disease-based organ findings. The permutations and combinations of the seven single disease-based organ findings result in an exponential amount of report templates. [Figure 5A] Figures 5A-5F show a set of base organ findings for the lungs, classified based on severity as normal, minimal, mild, moderate, and severe, and displayed as separate AI model result templates. Boxes 501-557 represent one line item in a particular AI report template. One line item is selected based on each AI model result template that matches the finding listed in the "Code" section. [Figure 5B] Figures 5A-5F show a set of base organ findings for the lungs, classified based on severity as normal, minimal, mild, moderate, and severe, and displayed as separate AI model result templates. Boxes 501-557 represent one line item in a particular AI report template. One line item is selected based on each AI model result template that matches the finding listed in the "Code" section. [Figure 5C] Figures 5A-5F show a set of base organ findings for the lungs, classified based on severity as normal, minimal, mild, moderate, and severe, and displayed as separate AI model result templates. Boxes 501-557 represent one line item in a particular AI report template. One line item is selected based on each AI model result template that matches the finding listed in the "Code" section. [Figure 5D] Figures 5A-5F show a set of base organ findings for the lungs, classified based on severity as normal, minimal, mild, moderate, and severe, and displayed as separate AI model result templates. Boxes 501-557 represent one line item in a particular AI report template. One line item is selected based on each AI model result template that matches the finding listed in the "Code" section. [Figure 5E]Figures 5A-5F show a set of base organ findings for the lungs, classified based on severity as normal, minimal, mild, moderate, and severe, and displayed as separate AI model result templates. Boxes 501-557 represent one line item in a particular AI report template. One line item is selected based on each AI model result template that matches the finding listed in the "Code" section. [Figure 5F] Figures 5A-5F show a set of base organ findings for the lungs, classified based on severity as normal, minimal, mild, moderate, and severe, and displayed as separate AI model result templates. Boxes 501-557 represent one line item in a particular AI report template. One line item is selected based on each AI model result template that matches the finding listed in the "Code" section. [Figure 6] It is a collection of individual binaries (or a library of AI models) that radiological images are put to analysis to obtain a result of the radiological image's likelihood to be negative or positive for a disease or classification. [Figure 7] This is a radiographic image of the lateral chest of a dog, which has been preprocessed, cropped, labeled, and identified. The radiographic image is analyzed by a library of binary AI models, shown in Figure 6. [Figure 8] Screenshot of one binary AI model result obtained by analyzing a series of lateral radiological images similar to Image 7 through a specific binary AI model, for example, the Bronchitis AI model. [Figure 9A]9A-9E are a set of screenshots showing the AI ​​model results for each of the radiological images. A visual collection of the individual AI model results per image and the average AI model results for all images were evaluated for the particular case. An average evaluation result for each model was created by aggregating the individual image evaluation results and is displayed at the top of the screen in FIG. 9A. Each individual image and the AI ​​model results for that image are displayed in FIGS. 9B-9E, respectively. In FIG. 9A, the timestamp (901) indicates that the AI ​​analysis took less than three minutes. [Figure 9B] Individual AI model results are shown, including perihilar infiltrates, pneumonia, bronchitis, interstitium, diseased lung, aplastic trachea, cardiomegaly, pulmonary nodules, and pleural effusion. For each AI model, a label identifies the image as "normal" (902) or "abnormal" (903). Additionally, the probability that the image is "normal" or "abnormal" for each disease in the AI ​​model is provided (904). [Figure 9C] Four images are shown from the classification and harvesting of one radiological image. The timestamps (905-908) indicate that the AI ​​analysis took less than two minutes. [Figure 9D] Figures 9D and 9E show the results of each AI model for a radiographic image, including the label, likelihood, and aspect (909) of the radiographic image (e.g., lateral, dorsal, anterior-posterior, posteroanterior, ventrodorsal, dorsoventral, etc.). [Figure 9E] Figures 9D and 9E show the results of each AI model for a radiographic image, including the label, likelihood, and aspect (909) of the radiographic image (e.g., lateral, dorsal, anterior-posterior, posteroanterior, ventrodorsal, dorsoventral, etc.). [Figure 10] This is a screenshot of the AI ​​case results displayed in JavaScript Object Notation (JSON) format. The JSON format facilitates copying the average score results for all models in a case, which can be transferred to an AI evaluation tester for testing to evaluate the average score results by comparing them with the cluster results. [Figure 11]11 is a screenshot of a graphical user interface that allows a user to create a K-means cluster. The user assigns a name (1101) to the new cluster under "Code." The user selects various parameters for creating the cluster. The user selects a start case date (1102) and an end case date (1103) for selecting the cases. The user selects a start case ID (1104) and an end case ID (1105) for selecting the cases. The user selects the maximum number of cases (1106) that are to be included in the cluster. The user selects the species (1107), such as canine, feline, canine or feline, human, for the cases that are to be included in the cluster. The user selects specific diagnostic methods (1108), such as X-ray, CT, MRI, blood analysis, urinalysis, etc., that are to be included in creating the cluster. The user specifies that the assessment results should be divided into a specific number of clusters. The number of clusters ranges from a minimum of one cluster to a maximum number of clusters, limited only by the total number of cases that can be placed in the cluster. [Figure 12] This is a screenshot of AI cluster results listed as a table of values. The leftmost column (1201) is the case ID, the next nine columns are the respective binary mean assessment results (1202) for that particular case ID, the next column is the cluster indicator or cluster location (1203) that includes the particular case based on the collection of assessment results, the next four columns are the cluster coordinates and centroid coordinates, and the final number is the case ID of the centroid or center of that particular cluster (1204). A radiologist's report for the best-matching case ID is obtained. This radiologist's report is then used to generate a report for new AI cases. This process allows for infinite scalability in terms of the number of AI models incorporated compared to traditional semi-manual report generation processes. [Figure 13]This is an example of a clustering graph. The clustering graph is created by dividing the average evaluation results into a number of different clusters depending on user-defined parameters (1102-1108). In this example, the clustering graph is divided into 180 different clusters, each represented by a neighborhood of small dots of one color plotted on the graph. [Figure 14] 14 is a screenshot of a user interface showing an AI cluster model created based on user-defined parameters (1102-1108). The first column from the left shows the cluster ID (1401), the second column shows the name assigned to the cluster model (1402), the third column shows the number of different clusters (1403) the AI ​​data was divided into, and the fourth column shows the body region (1404) that was evaluated based on the cluster data results. [Figure 15] 15 is a screenshot of the user interface showing a screening evaluation configuration. The user interface allows you to assign a specific "cluster model" (1502) to a specific "screening evaluation configuration" name (1501). The status of the screening evaluation configuration (1503) provides additional data about the configuration, such as whether the configuration is in live, test, or draft mode. Live mode is for production, and test mode is for development. [Figure 16A]Figures 16A-16C are a set of screenshots of the user interface showing details for a particular cluster model. Figure 16A shows the user interface displaying data for the cluster model (1601) Breast (97). The AI-evaluated classifier types (1602) included in the cluster are listed. The specific species or collection of species (1603) for the cluster model is displayed. The maximum number of cases (1604) with evaluation results used to create the cluster is displayed. The user interface shows the start and end dates (1605) for the cases used to create the cluster. A link (1606) to the comma-separated values ​​(CSV) file of Figure 12 is displayed, showing the cluster in numeric table format. Some of the subclusters (1608) created from the parameters (1602-1605) are listed. The total number of subclusters (1609) created for that cluster group is displayed. For each subcluster, the centroid case ID (1610) is displayed. A link to the log (1607) for building the cluster is displayed. [Figure 16B] Figures 16A-16C are a pair of screenshots of the user interface showing details for a particular cluster model. Figure 16B is a screenshot of the log created for cluster model (1601) Chest (97). [Figure 16C] 16A-16C are a pair of screenshots of the user interface showing details for a particular cluster model. FIG. 16C is a screenshot of a portion of the AI ​​assessment model, including vertebral cardiac score, perihilar infiltration, pneumonia, bronchitis, interstitial, and diseased lung. [Figure 17A] 17A-17D are a set of screenshots of the user interface for the AI ​​Evaluation Tester. Figure 17A shows the user interface (AI Evaluation Tester) in which the average evaluation result values ​​for all models in the JSON format of Figure 10 have been imported (1701) using K-means from case clusters created from the AI ​​dataset to analyze the closest matching case / exemplar result matches in the clusters. [Figure 17B] 17A-17D are a set of screenshots of the user interface for the AI ​​evaluation tester. FIG. 17B shows the average evaluation result values ​​for all models in the JSON format of FIG. 10 imported into the AI ​​evaluation tester. [Figure 17C] Figures 17A-17D are a set of screenshots of the user interface for the AI ​​assessment tester, and Figures 17C and 17D show the imported assessment results for a particular case. [Figure 17D] Figures 17A-17D are a set of screenshots of the user interface for the AI ​​assessment tester. Figures 17C and 17D show the assessment results imported for a particular case. Figure 17D shows the exam assessment type (1702) and the cluster model (1703) associated with the exam assessment type selected by the user. By clicking Test (1704), the assessment results displayed in Figure 10 are analyzed and further assigned to the closest matching case / exemplar result match in the cluster. For the exemplar result match cluster, the closest matching radiologist report, top-ranking radiologist statement, and centroid radiologist report are collected and displayed. [Figure 18A]FIGS. 18A-18D are a set of screenshots of the user interface. FIGS. 18A and 18B are a set of screenshots of the user interface showing the results displayed after clicking a test (1704) on the AI ​​assessment tester. Based on the pre-generated cluster results, diagnostic and conclusory findings (1801) from the radiologist's report that most closely match the assessment results are displayed. Assessment findings (1802) are selected from the radiologist's report in the cluster of assessment results and filtered based on the prevalence of specific sentences in the findings section of the particular cluster. Recommendations (1803) from the radiologist's report in the cluster are selected based on the prevalence of each sentence or similar sentences in the recommendations section of this cluster. Interface (1804) shows the radiologist's report for the cluster, and interface (1805) shows the radiologist's report for the cluster's centroid. [Figure 18B] FIGS. 18A-18D are a set of screenshots of the user interface. FIGS. 18A and 18B are a set of screenshots of the user interface showing the results displayed after clicking a test (1704) on the AI ​​assessment tester. Based on the pre-generated cluster results, diagnostic and conclusory findings (1801) from the radiologist's report that most closely match the assessment results are displayed. Assessment findings (1802) are selected from the radiologist's report in the cluster of assessment results and filtered based on the prevalence of specific sentences in the findings section of the particular cluster. Recommendations (1803) from the radiologist's report in the cluster are selected based on the prevalence of each sentence or similar sentences in the recommendations section of this cluster. Interface (1804) shows the radiologist's report for the cluster, and interface (1805) shows the radiologist's report for the cluster's centroid. [Figure 18C]Figures 18A-18D are a set of screenshots of a user interface. Figure 18C is a screenshot of a user interface listing the ranking of statements in a radiology report based on a particular cluster result. The statements include a conclusion statement (1806), a finding statement (1807), and a recommendation statement (1808). [Figure 18D-1] Figures 18A-18D are a set of screenshots of a user interface that allows a user to edit a radiology report by adding or removing specific statements to edit the findings section (1809), conclusion section (1810), or recommendations section (1811). [Figure 18D-2] Figures 18A-18D are a set of screenshots of a user interface that allows a user to edit a radiology report by adding or removing specific statements to edit the findings section (1809), conclusion section (1810), or recommendations section (1811). [Figure 19] The radiologist report for the closest matching dataset case used to generate the radiology report for the new case. The AI ​​assessment tester displays the radiologist report that most closely matches the current AI assessment result based on the similarity of the assessment results between the new image AI assessment result and the AI ​​assessment results within the cluster, and the radiologist report from the centroid of the selected cluster. [Figure 20A] Figures 20A and 20B are a pair of radiological images: Figure 20A is a newly received radiological image under analysis; [Figure 20B] Figures 20A and 20B are a pair of radiological images. Figure 20B is a radiological image selected by the results of AI evaluation as the closest match based on the cluster model. The cluster match is based on the results of AI evaluation rather than the image match results. [Figure 21A]Figures 21A and 21B are schematic diagrams of a set of components in an AI radiology image processing device. Figure 21A is a schematic diagram showing a radiology image machine (2101) sending radiology images to a desktop application (2102), which sends the images to a web application (2103). A computer vision application (2104) and web application direct the images to an image web application, which directs the images to an image AI evaluation (2105). [Figure 21B] Figures 21A and 21B are schematic diagrams of a set of components in an AI radiology image processing device. Figure 21B is a schematic diagram of the components of image matching AI processing. Images uploaded from the local interface to the online network (LION) (2106) at the veterinary clinic are directed to the Vetconsole (2107), where the image is auto-rotated, auto-cropped, and sub-images are acquired. The sub-images are directed to three locations. The first location is directed to the VetAIconsole (2108) to classify the image. The second location is directed to the Image Matching Console (2109) to add the sub-image along with a report to the image matching database. The third location is directed to the Image Database (2110) to store the new image and corresponding case ID number. The Image Matching Console (2109) directs it to a sophisticated Image Matching Console (2111) or Vetimage Editor Console (2112) for further processing. [Figure 22A] Figures 22A and 22B are a pair of schematic diagrams of server architectures for image matching: Figure 22A is a schematic diagram of a server architecture currently used in AI radiology image analysis. [Figure 22B]Figures 22A and 22B are a pair of schematic diagrams of a server architecture for image matching. Figure 22B is a plan view of a server architecture for AI radiology image analysis, including pre-processing radiology images, analyzing the images using an AI diagnostic processor, and preparing a report based on the clustering results. Images from a PC (2201) are directed to an NGINX load balancing server, which directs the images to a V2 cloud platform (2203). The images are then directed to an image matching server (2204), a Vetimage server (2205), and a database Microsoft SQL server (2207). The Vetimage server directs the images to a VetAI server (2206), a database Microsoft SQL server (2207), and a data store server (2208). [Figure 23A]Figures 23A-23F are a series of schematic diagrams of the workflow for artificial intelligence automated cropping and evaluation of images acquired for a subject. The workflow is categorized into six columns based on the platform used to perform the task, including Clinic, V2 end-user application, Vetlmages application, VetConsole Python scripting application, VetAI machine learning application, ImageMatch orientation Python application, and ImageMatch validation Python application. Tasks are shaded with different shading levels based on the processor that performs the task, such as the sub-image processor, evaluation processor, and synthesis processor. The V2 application is an end-user application where users interact with the application and upload images for further analysis. The Vetlmages application processes images to generate AI results or reports, or evaluation results. VetConsole is a Python scripting app that improves images and processes images in bulk. VetAI is a machine learning application that creates AI models and further evaluates images entered into the system. ImageMatch orientation is a Python app that searches a database for accurately oriented images similar to the input image. ImageMatch Verification is a Python application that searches a database for correctly classified images similar to an input image. The Sub-Image Processor performs the tasks listed in Figures 23A-23C (2301-2332). The Evaluation Processor performs the tasks listed in Figures 23D and part of Figure 23E and 23F (2333-2346, 2356, 2357). The Synthesis Processor performs the tasks listed in Figures 23F and part of Figure 23B (2347-2355 and 2358-2363). [Figure 23B]Figures 23A-23F are a series of schematic diagrams of the workflow for artificial intelligence automated cropping and evaluation of images acquired for a subject. The workflow is categorized into six columns based on the platform used to perform the task, including Clinic, V2 end-user application, Vetlmages application, VetConsole Python scripting application, VetAI machine learning application, ImageMatch orientation Python application, and ImageMatch validation Python application. Tasks are shaded with different shading levels based on the processor that performs the task, such as the sub-image processor, evaluation processor, and synthesis processor. The V2 application is an end-user application where users interact with the application and upload images for further analysis. The Vetlmages application processes images to generate AI results or reports, or evaluation results. VetConsole is a Python scripting app that improves images and processes images in bulk. VetAI is a machine learning application that creates AI models and further evaluates images entered into the system. ImageMatch orientation is a Python app that searches a database for accurately oriented images similar to the input image. ImageMatch Verification is a Python application that searches a database for correctly classified images similar to an input image. The Sub-Image Processor performs the tasks listed in Figures 23A-23C (2301-2332). The Evaluation Processor performs the tasks listed in Figures 23D and part of Figure 23E and 23F (2333-2346, 2356, 2357). The Synthesis Processor performs the tasks listed in Figures 23F and part of Figure 23B (2347-2355 and 2358-2363). [Figure 23C]Figures 23A-23F are a series of schematic diagrams of the workflow for artificial intelligence automated cropping and evaluation of images acquired for a subject. The workflow is categorized into six columns based on the platform used to perform the task, including Clinic, V2 end-user application, Vetlmages application, VetConsole Python scripting application, VetAI machine learning application, ImageMatch orientation Python application, and ImageMatch validation Python application. Tasks are shaded with different shading levels based on the processor that performs the task, such as the sub-image processor, evaluation processor, and synthesis processor. The V2 application is an end-user application where users interact with the application and upload images for further analysis. The Vetlmages application processes images to generate AI results or reports, or evaluation results. VetConsole is a Python scripting app that improves images and processes images in bulk. VetAI is a machine learning application that creates AI models and further evaluates images entered into the system. ImageMatch orientation is a Python app that searches a database for accurately oriented images similar to the input image. ImageMatch Verification is a Python application that searches a database for correctly classified images similar to an input image. The Sub-Image Processor performs the tasks listed in Figures 23A-23C (2301-2332). The Evaluation Processor performs the tasks listed in Figures 23D and part of Figure 23E and 23F (2333-2346, 2356, 2357). The Synthesis Processor performs the tasks listed in Figures 23F and part of Figure 23B (2347-2355 and 2358-2363). [Figure 23D]Figures 23A-23F are a series of schematic diagrams of the workflow for artificial intelligence automated cropping and evaluation of images acquired for a subject. The workflow is categorized into six columns based on the platform used to perform the task, including Clinic, V2 end-user application, Vetlmages application, VetConsole Python scripting application, VetAI machine learning application, ImageMatch orientation Python application, and ImageMatch validation Python application. Tasks are shaded with different shading levels based on the processor that performs the task, such as the sub-image processor, evaluation processor, and synthesis processor. The V2 application is an end-user application where users interact with the application and upload images for further analysis. The Vetlmages application processes images to generate AI results or reports, or evaluation results. VetConsole is a Python scripting app that improves images and processes images in bulk. VetAI is a machine learning application that creates AI models and further evaluates images entered into the system. ImageMatch orientation is a Python app that searches a database for accurately oriented images similar to the input image. ImageMatch Verification is a Python application that searches a database for correctly classified images similar to an input image. The Sub-Image Processor performs the tasks listed in Figures 23A-23C (2301-2332). The Evaluation Processor performs the tasks listed in Figures 23D and part of Figure 23E and 23F (2333-2346, 2356, 2357). The Synthesis Processor performs the tasks listed in Figures 23F and part of Figure 23B (2347-2355 and 2358-2363). [Figure 23E]Figures 23A-23F are a series of schematic diagrams of the workflow for artificial intelligence automated cropping and evaluation of images acquired for a subject. The workflow is categorized into six columns based on the platform used to perform the task, including Clinic, V2 end-user application, Vetlmages application, VetConsole Python scripting application, VetAI machine learning application, ImageMatch orientation Python application, and ImageMatch validation Python application. Tasks are shaded with different shading levels based on the processor that performs the task, such as the sub-image processor, evaluation processor, and synthesis processor. The V2 application is an end-user application where users interact with the application and upload images for further analysis. The Vetlmages application processes images to generate AI results or reports, or evaluation results. VetConsole is a Python scripting app that improves images and processes images in bulk. VetAI is a machine learning application that creates AI models and further evaluates images entered into the system. ImageMatch orientation is a Python app that searches a database for accurately oriented images similar to the input image. ImageMatch Verification is a Python application that searches a database for correctly classified images similar to an input image. The Sub-Image Processor performs the tasks listed in Figures 23A-23C (2301-2332). The Evaluation Processor performs the tasks listed in Figures 23D and part of Figure 23E and 23F (2333-2346, 2356, 2357). The Synthesis Processor performs the tasks listed in Figures 23F and part of Figure 23B (2347-2355 and 2358-2363). [Figure 23F]Figures 23A-23F are a series of schematic diagrams of the workflow for artificial intelligence automated cropping and evaluation of images acquired for a subject. The workflow is categorized into six columns based on the platform used to perform the task, including Clinic, V2 end-user application, Vetlmages application, VetConsole Python scripting application, VetAI machine learning application, ImageMatch orientation Python application, and ImageMatch validation Python application. Tasks are shaded with different shading levels based on the processor that performs the task, such as the sub-image processor, evaluation processor, and synthesis processor. The V2 application is an end-user application where users interact with the application and upload images for further analysis. The Vetlmages application processes images to generate AI results or reports, or evaluation results. VetConsole is a Python scripting app that improves images and processes images in bulk. VetAI is a machine learning application that creates AI models and further evaluates images entered into the system. ImageMatch orientation is a Python app that searches a database for accurately oriented images similar to the input image. ImageMatch Verification is a Python application that searches a database for correctly classified images similar to an input image. The Sub-Image Processor performs the tasks listed in Figures 23A-23C (2301-2332). The Evaluation Processor performs the tasks listed in Figures 23D and part of Figure 23E and 23F (2333-2346, 2356, 2357). The Synthesis Processor performs the tasks listed in Figures 23F and part of Figure 23B (2347-2355 and 2358-2363). DETAILED DESCRIPTION OF THE INVENTION

[0032] Aspects of the invention herein describe a novel system with various stages of analysis, including determining whether an incoming image contains a specific body region in a particular orientation (e.g., a side view), cropping the image appropriately, and evaluating the cropped image by comparing the image to a targeted AI model. In various embodiments, newly received images are pre-processed to automatically identify and label one or more body regions and / or fields of view represented in the image without user input or intervention. In some embodiments, the image is automatically cropped to create one or more sub-images corresponding to each identified body region / field of view. In some embodiments, the image and / or sub-images are selectively processed to a targeted AI processor configured to evaluate the identified body region / field of view, to the exclusion of the remainder of the AI ​​processors in the system.

[0033] In some embodiments, the radiological image preprocessor (106) additionally or alternatively tags the entire image (102) and identifies identified body regions and / or fields of view within the image (102), then passes only those to the AI ​​processor (104) that correspond to the given tags. Thus, in such embodiments, the AI ​​processor (104) corresponds to cropping the image (102) to focus on relevant regions for further analysis using one or more trained machine learning models or otherwise. In some embodiments, in addition to tagging the image (102) to correspond to specific body regions / fields of view, the radiological image preprocessor (106) additionally crops the image (102) to primarily focus on regions of the image that actually represent parts of the animal and to remove as much of the black and green color as possible around those regions. In some embodiments, performing the cropping step facilitates further cropping and / or other processing by the AI ​​processor (104) that is subsequently deployed to evaluate the specific body regions / fields of view that correspond to the given tags.

[0034] The radiographic image preprocessor (106) may be implemented in any of several ways. In some embodiments, for example, the radiographic image preprocessor (106) uses one or more algorithms for automatically cropping the image (102) to identify one or more features indicative of particular body regions and to target regions containing such features and / or regions that actually represent animals. In some implementations, such algorithms are implemented using, for example, elements of the OpenCV-Python library. A description of the open source computer vision ("OpenCV") library and related documentation and tutorials, as well as documentation and tutorials, can be found using the uniform resource locator (URL) for OpenCV. The entire contents of the material accessible via the URL are incorporated herein by reference. In some embodiments, the radiological image preprocessor (106) additionally or alternatively uses image matching techniques to compare the image (102) and / or one or more cropped sub-images (108) to a repository of stored images known to represent particular fields of view of particular body regions, and determines the image (102) and / or sub-image (108) representing the body region / field of view that is deemed to most strongly correlate with the one or more stored images. In some embodiments, an AI processor trained to perform body region / field of view identification is additionally or alternatively used within the radiological image preprocessor (106).

[0035] In some embodiments, one or more AI processors described herein are implemented using the TensorFlow platform. Descriptions, documentation, and tutorials for the TensorFlow platform can be found using the TensorFlow website. The entire contents of the materials accessible via the website are incorporated herein by reference. The TensorFlow platform and methods for building AI processors are fully described in Hope, Tom, et al. 25 Learning TensorFlow: A Guide to Building Deep Learning Systems. O'Reilly., 2017, which is incorporated herein by reference in its entirety.

[0036] In the example shown in FIG. 3, the pre-processing performed by the radiology image preprocessor (106) included (1) an optional "general" auto-cropping step (reflected in the first five log entries depicted by bracket (306)), according to which the image (302) was first cropped to primarily focus on areas of the image that represent parts of the animal and to remove as much of the black-and-green color as possible around those areas; and (2) a "classified" auto-cropping step (reflected in log entries 6-9 within bracket (306)), according to which the image (302) was first cropped to focus primarily on areas of the image that represent parts of the animal and to remove as much of the black-and-green color as possible around those areas. The log included (3) a "classified" auto-cropping step, in which an initial attempt was made using, for example, elements of the OpenCV Python library, to crop the image (302) to identify and focus on a specific body region / field of view; and (4) an "image matching" step (reflected in the last three log entries depicted by brackets (306)), according to which the image (302) and / or one or more of its cropped sub-images (304a-b) were compared to a repository of stored images known to represent a specific field of view of a specific body region. As indicated by the corresponding timestamps, the general auto-cropping step was observed to complete in 2 seconds, the classified auto-cropping step was observed to complete in 3 seconds, and the image matching step was observed to complete in 19 seconds.

[0037] As shown by the log entry depicted by bracket (308a) in Figure 3, it took 4 seconds for the lateral thorax AI processor (104a) to determine whether the image (302) contained a lateral view of the animal's thorax. Similarly, as shown by the log entry depicted by bracket (308b), the lateral abdomen AI processor (104c) took 4 seconds to determine whether the image (302) contained a lateral view of the animal's abdomen.

[0038] If, instead, the system required processing a newly received image (302) using all possible AI processors (104a-1) rather than only the two corresponding to the body region / field of view identified by the radiology image preprocessor (106), the time taken by the AI ​​processors would have been significantly longer and / or the analysis would have consumed significantly more processing resources to complete. For example, in a system including 30 different AI processors (104), processing simply to identify an appropriate AI model for determining disease in the imaged animal would have required at least 120 seconds of processing time by the AI ​​processors (104) (i.e., 4 seconds per processor for 30 AI processors), and could have been much longer if multiple possible orientations of the image were considered by each of the AI ​​processors (104). By using the radiographic image preprocessor (106), on the other hand, it was observed that the identification of a suitable AI model took only 8 seconds of the AI ​​processor (104) processing time and 24 seconds of the radiographic image preprocessor (106) processing time for the preprocessing time by the radiographic image preprocessor (106).

[0039] It is useful to process radiological images of animals using artificial intelligence (AI) processors (e.g., trained neural networks) to determine the likelihood that the imaged animal has a particular medical condition. Typically, a separate AI processor is used to evaluate each body region (e.g., chest, abdomen, shoulders, forelimbs, hindlimbs, etc.) and / or a particular orientation of each such body region (e.g., ventrodorsal (VD) view, lateral view, etc.), with each such AI processor determining, for each body region and / or orientation, the likelihood that a particular disease is present with respect to the body region in question. Each such AI processor may include multiple trained models for evaluating each disease or organ within the imaged region. For example, with respect to a lateral view of an animal's chest, the AI ​​processor may use different models to determine the likelihood that the animal has a particular lung-related disease, such as perihilar infiltrates, pneumonia, bronchitis, pulmonary nodules, etc.

[0040] Detecting a single medical condition, such as the presence or absence of pneumonia or pneumothorax, is currently performed in radiology AI. In contrast to single-disease detection by current radiology AI, human radiologists analyze radiology images with a holistic approach by simultaneously assessing the presence or absence of multiple diseases. A limitation of current AI processing is the need to use separate AI detectors for each specific disease. However, a combination of diseases can result in a broad disease diagnosis. For example, in some cases, one or more diagnoses obtained from radiology images are caused by several broad diseases. Determining the broad diseases present in a subject's radiology images requires the use of supplementary diagnostic results in a process known as suggestive diagnosis. These supplementary diagnostic results are extracted from blood tests, patient history, biopsies, or other tests and procedures in addition to the radiology images. Current AI processing focuses on a single diagnostic result and is unable to identify broad diseases requiring suggestive diagnosis. Described herein is a novel AI process that can combine multiple diagnostic results to diagnose broad diseases.

[0041] AI processes currently use limited radiographic images that are directed at specific areas, as is typical in radiographic images of human subjects. In contrast, veterinary radiology typically includes multiple body regions within a single radiographic image. Described herein is a novel AI evaluation process that provides the broad assessment expected in veterinary radiology to evaluate all body regions included in a study.

[0042] The current conventional workflow for AI reporting of a single disease process is illustrated in Figure 4. The conventional single disease report shown in Figure 4 is insufficient for a suggestive diagnosis of radiological images. Furthermore, individualized rules for each combination of assessment results are inefficient for generating reports and cannot meet the reporting standards expected of veterinary radiologists. Even for a single disease process, the determination of a specific disease severity (e.g., normal, minimal, mild, moderate, severe among an exponential number of AI models) results in an exponential number of AI model result templates. The process of creating and selecting a single report template from a collection of AI model diagnosis results scales exponentially with the number of AI models. The number of AI models for a single disease process results in 57 different templates for five severity levels, as illustrated in Figures 5A-5F. Therefore, manually created reports for each combination of AI model diagnosis results do not scale well to a large number of AI models that are interpreted collectively.

[0043] Automated system for AI analysis Described herein is a novel system for analyzing images of a target animal, the system including a receiver for receiving an image of the target, at least one sub-image processor for identifying, cropping, orienting, and labeling at least one body region within the image to obtain a sub-image, at least one artificial intelligence evaluation processor for evaluating the sub-image for the presence of at least one disease, at least one synthesis processor for generating a comprehensive result report from the at least one sub-image evaluation and, optionally, non-image data, and a device for displaying the sub-images and the comprehensive synthetic diagnostic result report.

[0044] The system provides a significant advancement in veterinary diagnostic image analysis by (1) automating the extraction of sub-images, a task typically performed manually or with user assistance, through a sub-image processor, and (2) compositing a large collection of evaluation results and other non-image data points into a concise, cohesive overall report using a composition processor.

[0045] A case includes a collection of one or more images of a subject animal and may further include non-image data (such as, but not limited to, age, sex, location, medical history, and other medical test results). In one embodiment of the system, each image is sent to multiple sub-image processors that create multiple sub-images of different views of multiple body regions. Each sub-image is processed by multiple assessment processors that generate multiple assessment results for various diseases, findings, or other features across multiple body regions. A synthesis processor processes all or a subset of the assessment results and non-image data points to generate an overall synthetic diagnostic result report. In one embodiment of the system, the multiple synthesis processors generate multiple synthetic diagnostic result reports from different subsets of the assessment results and non-image data points. These diagnostic reports are then combined with ancillary data to create a final overall synthetic diagnostic result report.

[0046] In one embodiment of the system, each synthesis processor operates on a subset of sub-images and non-image data points corresponding to a body region, e.g., thorax or abdomen. As is typical in veterinary radiology, each synthesis diagnostic report includes the body region. The overall synthesis diagnostic results report includes descriptive data of the subject, e.g., name, age, address, breed, etc., and multiple sections corresponding to the output of each synthesis processor, e.g., a thorax results section and an abdominal results section.

[0047] In one embodiment of the system, the subject is selected from mammals, reptiles, fish, reptiles, amphibians, chordates, and birds. The mammals are dogs, cats, rodents, horses, sheep, cattle, goats, camels, alpacas, buffalo, elephants, and humans. The subjects are pets, livestock, zoo treasures, wild animals, and research animals.

[0048] The images received by the system are images from radiological examinations selected from X-ray (radiography), magnetic resonance imaging (MRI), magnetic resonance angiography (MRA), computed tomography (CT), fluoroscopy, mammography, nuclear medicine, positron emission tomography (PET), ultrasound, etc. In some embodiments, the images are photographs.

[0049] In some embodiments of the system, analysis of the subject images generates and displays a comprehensive composite results report in less than about 20 minutes, less than about 10 minutes, less than about 5 minutes, less than about 1 minute, less than about 30 seconds, less than about 20 seconds, less than about 15 seconds, less than about 10 seconds, or less than about 5 seconds.

[0050] Sub Image Processor The sub-image processor automatically acquires sub-images in a timely manner by orienting, cropping, and marking at least one body region in the image. The sub-image processor orients the image by rotating the image to a standard orientation that depends on the specific field of view. The orientation is determined by veterinary radiology standard hanging protocols. The sub-image processor crops the image by identifying boundaries in the image that depict one or more body regions and creating image data that encompasses the image within the identified boundaries.

[0051] In some embodiments, the boundaries are of a constant aspect ratio. In alternative embodiments, the boundaries are not of a constant aspect ratio. The sub-image processor labels the sub-images by reporting the boundaries and / or location of each body region encompassed within the sub-image. Body regions include, for example, thorax, abdomen, spine, forelimbs, head, neck, etc. In some embodiments, the sub-image processor labels the sub-images with veterinary radiology standard body region markers.

[0052] The sub-image processor orients, crops, and labels one or more sub-images by matching the image to a plurality of reference images in at least one of a plurality of libraries, each of which contains a respective plurality of reference images that may be species-specific or non-specific.

[0053] The sub-image processor extracts a signature of the image prior to orienting, cropping, and / or labeling the image, thereby enabling the image or sub-image to be quickly matched to a similar reference image. The sub-image processor processes the image to obtain a sub-image in less than about 20 minutes, less than about 10 minutes, less than about 5 minutes, less than about 1 minute, less than about 30 seconds, less than about 20 seconds, less than about 15 seconds, less than about 10 seconds, or less than about 5 seconds.

[0054] Evaluation Processor An artificial intelligence evaluation processor evaluates the sub-images for the presence or absence of a disease, finding, or other feature. The evaluation processor reports the likelihood of the presence of the disease, finding, or feature.

[0055] The evaluation processor diagnoses disease from the sub-images. The evaluation processor evaluates the sub-images for non-medical features, such as proper positioning of the subject. The evaluation processor generates instructions to correct the positioning of the subject.

[0056] Typically, the evaluation processor training includes negative control / normal and positive control / abnormal training sets for a disease, finding, or other feature. The positive control / abnormal training set typically includes cases assessed as having the disease, finding, or other feature present. The negative control / normal training set includes cases assessed as having the disease, finding, or other feature absent and / or cases considered completely normal. In some embodiments, the negative control / normal training set includes cases assessed as having other diseases, findings, or features present that are different from the one of interest. Thus, the evaluation processor is robust.

[0057] The evaluation processor processes the sub-images in less than about 20 minutes, less than about 10 minutes, less than about 5 minutes, less than about 1 minute, less than about 30 seconds, less than about 20 seconds, less than about 15 seconds, less than about 10 seconds, or less than about 5 seconds to report the presence of disease.

[0058] Synthesis Processor The synthesis processor receives at least one assessment from the assessment processor and generates a comprehensive results report. The synthesis processor may include non-image data points, such as species, breed, generation, weight, location, sex, medical test history including blood, urine, and fecal tests, radiology reports, lab reports, histology reports, physical exam reports, microbiology reports, or other medical and non-medical tests or results. The subject case exemplar results include at least one image, the results of the associated assessment processor, and zero or more collections of recent non-image data points.

[0059] In one embodiment of the method, a synthesis processor uses the case exemplar results to select a pre-written template to output as an overall results report, the template being automatically customized based on elements of the case exemplar results to provide a customized overall results report.

[0060] The synthesis processor assigns a subject's case exemplar results to a cluster group. The cluster group includes other similar case exemplar results from a reference library of case exemplar results from other subjects. In some cases, the cluster group includes partial case exemplar results, such as result reports. The reference library includes case exemplar results with known diseases and disorders from at least one of many animal species. To improve the performance of the synthesis processor over time, new case exemplar results are added to the reference library. The synthesis processor determines coordinates representing the location of each case exemplar result within the cluster group.

[0061] A single overall result report is assigned to the entire cluster group, and the overall result report is assigned to the subject by a synthesis processor. In some embodiments, several overall result reports are assigned to various case exemplar results within the cluster and / or various custom coordinates within the cluster, such as cluster centroids, without associated case exemplar results. The coordinates of the subject's case exemplar result are used to calculate the distance to the closest or non-closest case exemplar result or custom coordinate with an associated overall result report, and then assigned to the subject.

[0062] The comprehensive outcome report(s) are written by expert human reviewers. In an alternative embodiment, the comprehensive outcome report is generated from an existing radiology report. The existing radiology report is modified using natural language processing (NLP) to remove non-universally applicable content such as names, dates, references to previous studies, etc. to create a suitable comprehensive outcome report. Statements contained within the comprehensive outcome report are removed or edited if they do not meet a threshold of occurrence within a cluster.

[0063] The synthesis processor outputs an assigned comprehensive result report for the subject, thereby identifying and diagnosing the presence of one or more findings, diseases and / or disorders in the subject. Cluster groups are established from a reference library of case exemplar results using a clustering tool selected from K-means clustering, mean-shift clustering, density-based spatial clustering, expectation-maximization (EM) clustering, and agglomerative hierarchical clustering.

[0064] The synthesis processor processes the case exemplar results in less than about 20 minutes, less than about 10 minutes, less than about 9 minutes, less than about 8 minutes, less than about 7 minutes, less than about 6 minutes, less than about 5 minutes, less than about 4 minutes, less than about 3 minutes, less than about 2 minutes, less than about 1 minute, less than about 30 seconds, less than about 20 seconds, less than about 15 seconds, less than about 10 seconds, or less than about 5 seconds to generate a comprehensive results report.

[0065] Clustering is an AI technique for grouping unlabeled examples by similarities in the features of each example. A process for clustering patient studies based on AI processor diagnostic results, as well as non-radiographic and / or non-AI diagnostic results, is described herein. The clustering process groups reports that share similar diagnoses or output reports, thereby facilitating the detection of wholesome or broader diseases in a scalable manner.

[0066] Described herein are novel systems and methods with multiple stages of analysis that combine multiple methods of AI predictive image analysis of radiographic images with a report library database and newly received image quality assessments. In various embodiments, the novel systems described herein automatically detect the field, region, or area covered by each radiographic image.

[0067] In some embodiments, the system preprocesses the newly received radiographic image (102) prior to AI evaluation using a radiographic image preprocessor (106) to crop, rotate, flip, create sub-images, and / or standardize image exposure. If one or more body regions or fields of view are identified, the system further crops the image (102) corresponding to each identified region and field of view to generate one or more sub-images (108a, 108b, and 108c). In some embodiments, the system selectively processes and sends the image and / or sub-images to a desired AI processor configured to evaluate the identified region / field of view. Image (108a) is sent only to AI processor (104a), which is a lateral chest AI processor. Image (108b), which is a lateral abdomen AI processor, is sent only to AI processor (104c). Image (108c) is sent only to AI processor (104k), which is a lateral pelvis AI processor. The images are not sent to the unintended AI processors, nor to the remaining AI processors in the system. For example, the chest image (Figure 7) is sent to one or more AI processors for the diseases listed in Figure 6, such as heart failure, pneumonia, bronchitis, interstitial disease, diseased lungs, underdeveloped trachea, cardiomegaly, pulmonary nodules, pleural effusion, gastritis, esophagitis, bronchiectasis, lung hyperinflation, pulmonary vasodilation, and thoracic lymphadenopathy.

[0068] In some embodiments, the AI ​​model processor is a binary processor that provides a binary result: normal or abnormal. In various embodiments, the AI ​​model processor provides a diagnosis of normal or abnormal accompanied by a determination of the severity of a particular disease, for example, normal, minimal, mild, moderate, severe, etc.

[0069] In some embodiments, newly received AI model processor results are displayed in a user interface. See Figures 9A-9E. An average AI model processor result for each model is collected from the evaluation results of each individual image or sub-image and displayed. See Figure 9A. The user interface displays each individual image or sub-image and the AI ​​model processor results for that image. See Figures 9B-9E. The AI ​​analysis is completed in less than one, two, or three minutes.

[0070] In some embodiments, one or more clusters are constructed by the system using AI processor diagnostic results from a library of known radiographic images and corresponding radiology report databases to develop closest matching cases or AI processor "exemplar results" for one or more AI processor results. The exemplar results include at least one image, a collection of associated evaluation processor results, and zero or more collections of non-image data points, such as age, gender, location, breed, and medical test results. A synthesis processor determines coordinates representing the location of each case exemplar result within a cluster group. Thus, if two cases have similar exemplar results, the diagnosis is similar or largely identical, and a single overall result report applies to the two cases. In some embodiments, a single exemplar result is assigned to an entire cluster, and a subject case located in the cluster is assigned the exemplar result. In some embodiments, multiple exemplar results are assigned to a cluster associated with either specific coordinates (e.g., centroids) in the cluster or specific dataset cases. In some embodiments, exemplar results are generated manually or automatically from existing radiology reports associated with the cases.

[0071] In some embodiments, a user uses the user interface of FIG. 11 to specify various parameters for creating a cluster from a library of known radiographic images and corresponding radiology reports. The user assigns a name to the new cluster under "Code" (1101). The user selects various parameters for creating the cluster. The user selects a case start date (1102) and a case end date (1103) to select cases. The user selects a start case ID (1104) and an end case ID (1105) to select cases. The user selects the maximum number of cases (1106) to be included in the cluster. The user selects the species (1107) of cases to be included in the cluster, such as canine, feline, canine or feline, human, avian pet, livestock, etc. The user selects the specific diagnostic modalities (1108) to be included in the creation of the cluster, such as X-ray, CT, MRI, blood analysis, urinalysis, etc.

[0072] In various embodiments, the user specifies a specific number of clusters to divide the assessment results into. The number of clusters ranges from a minimum of one cluster to a maximum number of clusters limited only by the total number of cases included in the cluster. The system builds one or more clusters using non-radiographic and / or non-AI diagnostic results, such as blood tests, patient medical history, or other tests or processes, in addition to AI processor diagnostic results. The clusters are listed in numerical format in a comma-separated value (CSV) file format, as shown in FIG. 12. The CSV file lists the case IDs (1201) among the cases in the cluster. The average assessment result (1202) for each binary model for a particular case ID is listed in the CSV file. The cluster identifier or cluster location (1203) containing a particular case is listed in the CSV file based on the collection of assessment results. The CSV file lists the cluster coordinates. The case ID (1204) of the centroid or center of a particular cluster is listed in the CSV file.

[0073] In various embodiments, the clusters are represented by a clustering graph. See Figure 13. The clustering graph is created by dividing the average rating results into different clusters depending on user-defined parameters (1102-1108). The different clusters are represented by clusters of dots plotted on the graph. The clustering graph in Figure 13 shows 180 clusters of various sizes.

[0074] In some embodiments, the user interface shows an AI cluster model generated based on user-defined parameters (1102-1108). See FIG. 14. The user interface shows a screening evaluation configuration, where the user has assigned a specific "cluster model" (1502) to a specific "screening evaluation configuration" name (1501). The status of the screening evaluation configuration (1503) provides additional information about the configuration, such as whether the configuration is in live, test, or draft mode. Live mode is for production, and test or draft mode is for development.

[0075] In some embodiments, the user interface provides details about a specific cluster model, Breast 97 (1601). See FIG. 16A. In some embodiments, the user interface lists the AI-evaluated classifier types included in the cluster (1602). The user interface displays additional parameters used to build the cluster, such as the species specific to the cluster model (1603), the maximum number of cases with evaluation results (1604), or the start and end dates of the cases used to create the cluster (1605). The user interface provides a link (1606) to a comma-separated value (CSV) file that displays the cluster in a numeric table format. The user interface lists the subclusters (1608) created from the parameters (1602-1605). The user interface displays the total number of subclusters created for the cluster group (1609). The user interface provides the centroid case ID (1610) for each subcluster. A log of cluster building is provided in the user interface. See Figure 16B.

[0076] In various embodiments, the system utilizes one or more AI processors to evaluate newly received diagnostic-unconfirmed images and obtain newly received evaluation results, which the system compares with one or more clusters obtained from a library of known radiographic images and corresponding radiology database reports.

[0077] The user imports newly received AI processor results into the AI ​​Eval Tester (see Figure 17A). The user specifies the screening evaluation type (1702) and the corresponding cluster model (1703).

[0078] The system compares the newly received evaluation results, as well as non-radiographic and / or non-AI diagnostic results, with one or more clusters obtained from a library of known radiographic images and corresponding radiology report databases, among other available ones. The system evaluates the distance between the location of the newly received AI processor results and the cluster results and utilizes one or more cluster results to generate a radiologist report. In some embodiments, the system selects to utilize the entire radiologist report or a portion of the radiologist report from a known cluster result, depending on the location of the newly received AI processor results relative to the known cluster result. In various embodiments, the system selects to utilize the entire radiologist report or a portion of the radiologist report from other results in the same cluster. In some embodiments, the system selects to utilize the entire radiologist report or a portion of the radiologist report from the centroid of the cluster result.

[0079] The user interface displays the results of the AI ​​Eval tester. See FIG. 18A. In various embodiments, the diagnosis and final findings (1801) from the radiologist report that most closely matches the evaluation result based on the pre-created cluster results are displayed. In some embodiments, evaluation findings (1802) are selected from the radiologist reports in the clusters of evaluation results and filtered based on the occurrence of specific sentences in the findings section of a particular cluster. In some embodiments, recommendations (1803) from the radiologist reports in the clusters are selected based on the occurrence of each sentence or similar sentences in the recommendations section of the cluster. The user interface displays the radiologist reports of the clusters (1804) and the radiologist reports of the cluster centroids (1805). The user interface allows the user to edit the report by adding or removing specific sentences to the findings section (1809), conclusions section (1810), or recommendations section (1811). See FIG. 18D. The radiologist reports for the closest matching database cases are used to generate a radiology report for the newly received x-ray images. The statements in the radiology report based on a particular cluster result are ranked and listed by their rank and frequency of occurrence. See Figures 18A and 18B.

[0080] In various embodiments, the system utilizes one or more AI processors to evaluate newly received diagnostic-unconfirmed images and obtain newly received evaluation results, which the system compares with one or more clusters obtained from a library of known radiographic images and corresponding radiology database reports.

[0081] The system compares the newly received evaluation results, as well as non-radiographic and / or non-AI diagnostic results, with one or more clusters obtained from a library of known radiographic images and corresponding radiology report databases, among other available ones. The system evaluates the distance between the location of the newly received AI processor results and the cluster results and uses one or more cluster results to generate a radiologist report. In some embodiments, the system selects to use the entire radiologist report or a portion of the radiologist report from a known cluster result, depending on the location of the newly received AI processor results relative to the known cluster result. In various embodiments, the system selects to use the entire radiologist report or a portion of the radiologist report from other results in the same cluster. In some embodiments, the system selects to use the entire radiologist report or a portion of the radiologist report from the centroid of the cluster result.

[0082] In some embodiments, one or more of the AI ​​processors described herein are implemented using the TensorFlow platform. A description of the TensorFlow platform, as well as materials and tutorials, can be found on the TensorFlow website. The entire contents of the resources available at the TensorFlow website are incorporated herein by reference in their entirety.

[0083] In some embodiments, one or more of the clustering models described herein are implemented using the Plotly platform. A description of the Plotly platform, as well as materials and tutorials, can be found on the scikit-learn website. The entire contents of the resources available on the scikit-learn website are incorporated herein by reference in their entirety. The development of AI processors and clustering models using the TensorFlow platform and Scikit-learn is fully described in the following references: Geron Aurelien. Hands-on Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. O'Reilly, 2019; Hope, Tom, et al. Learning TensorFlow: A Guide to Building Deep Learning Systems. O'Reilly, 2017; and Sievert, Carson. Interactive Web-Based Data Visualization with R, Plotly, and Shiny. CRC Press, 2020. These references are incorporated herein by reference in their entirety.

[0084] If a radiologist had evaluated each model and attempted to create rules based on the separate AI processor results found together, rule and report creation would have been time-intensive. Additionally, as the number of AI processor models already incorporated increases, adding a single additional AI processor model into this scenario becomes exponentially more difficult. By employing a novel workflow of AI processor result clustering, or "exemplar result" comparison between new images and known datasets, to create a radiologist report, the problem of manual report construction when multiple AI processor results are found is solved. Manual report construction and rule creation via separate AI processor results, which previously took months, now takes minimal time with the new workflow.

[0085] In some embodiments of the system, the components used for AI evaluation are as set forth in Figure 21 A. In various embodiments of the system, the components used for image match AI processing are as set forth in Figure 21 B.

[0086] In various embodiments of the system, a server architecture for AI radiograph analysis includes pre-processing the radiograph images, analyzing the images using an AI diagnostic processor, and generating a report based on the clustering results. See Figure 22B. In some embodiments, various servers are used, including an NGINX load balancing server (2202), a V2 cloud platform (2203), a database, a Microsoft SQL server (2207), and a data store server (2208).

[0087] In various embodiments of the system, a user flags cases to train the AI ​​system. In some embodiments of the system, a user flags a case if the radiology report needs editing because it is inaccurate, or if the report is inappropriate, or if the case has a new diagnosis and therefore the radiology report needs new wording for the diagnosis.

[0088] A series of schematic diagrams of the AI ​​auto-cropping and evaluation workflow are illustrated in Figures 23A-23F. A user accesses the V2 end-user application (2301) to upload images (in formats such as DICOM, JPEG, JPG, PNG, etc.) to be analyzed by the system. In some embodiments, the images are uploaded directly to the VetImages application (2305). V2 processes the images (2302), stores them in a data store, and requests VetImages to further process the images (2303). VetImages receives a request from V2 and begins asynchronous processing (2304). VetImages accesses the images from the data store (2307) and requests VetConsole to preprocess the images (2308). VetConsole uses OpenCV (2309) to improve image quality and auto-crop the images (2310). Tasks subsequent to accessing the images from the data store are performed by the sub-image processor.

[0089] VetConsole sends the image-enhanced, auto-cropped image to VetImages, which analyzes the image and requests VetConsole to classify (2311) the chest, abdomen, and pelvis in the image. VetConsole classifies (2312) the chest, abdomen, and pelvis in the image and sends the coordinates to VetImages. VetImages sends the image and coordinates to ImageMatch validation (2313). ImageMatch validation matches the image and coordinates to correctly classified images in its database and sends the distance and path of the matched image to VetImages (2314). The VetImages application receives data about the matched image and uses database information to verify the body region (2315). The next task is to determine the image orientation. The image is rotated and flipped (2317). After each rotation and flip, the image is sent to the ImageMatch orientation application (2318) for comparison with the matched image and to measure the distance and image path between the matched image and the newly received image. The ImageMatch orientation application sends a result (2319) containing the distance and image path between the newly received image and the matched image. The orientation of the newly received image with the smallest distance from the matched image is selected by the VetImages application (2320). The process of checking each orientation and each flip is repeated until the image has been rotated 360 degrees and flipped at the appropriate angle. In some embodiments, the image with the selected orientation is sent to VetAI to detect the chest (2321) and abdomen (2323) and to obtain coordinates for cropping the sub-image with the chest (2322) and abdomen (2324). The process of obtaining coordinates is coordinated in TensorFlow.

[0090] The VetImages application obtains coordinates from ImageMatch validation and crops the image according to the coordinates to obtain the subimage (2325). The subimage is sent to the ImageMatch validation application for matching (2326). The database image is matched to the subimage (2327), and the distance and image path between the matched database image and subimage are sent to the VetImages application. The VetImages application receives the distance and image path data (2328) and uses the data received from the matched image to verify the body region. The VetImages application analyzes each subimage to check if it is valid. If the subimage is not valid (2331), the general cropped image from the VetConsole application is saved to a database or data store (2332). If the subimage is valid (2330), the subimage is saved to a database or data store (2332). The image saved to the database or data store is the cropped image used for further processing or analysis. The VetImages application stores 2332 the data obtained from the VetConsole for the sub-images or general cropped images in a database.

[0091] The following tasks are performed by the evaluation processor: The VetImages application sends the cropped image to the VetAI application for positioning evaluation (2333). Data received by the VetImages application from the VetAI application (2334) is stored in a database (2335) and signals the V2 application to send an email (2336) to the clinic. The VetImages application accesses live AI models from the database (2337). The cropped image is sent to the appropriate AI model in the VetAI application based on the body region of the cropped image (2339). An appropriate AI model is predefined for each body region. The VetAI application sends the AI ​​evaluation indicators and machine learning (ML) AI evaluation results to the VetImages application (2340), and the VetImages application stores these data in a database for the cropped image (2341). The VetImages application calculates the image indicator and likelihood based on the AI ​​evaluation results for the cropped image (2342). The process of sending the cropped image to obtain the AI ​​evaluation results (2339) is repeated (2342) until the given AI model is processed (2338).

[0092] The VetImages application analyzes whether each image from the case has been processed to obtain an AI assessment result (2344). If all images from the case have not been processed, VetImages returns the next image in the case to the process. If all images from the case have been processed, VetImages calculates the case's label and likelihood as a whole based on the label and likelihood of each cropped image (2345). The VetImages application then changes the status from the database to Live and Screening Assessment Type (2346). When changing the VetImages application's status to Live, tasks are performed by the synthesis processor. The VetImages application assesses whether all screening assessments have been completed (2347). If all screening assessments have not been completed, the VetImages application assesses whether screening assessments must be completed by clustering (2348). If the screening evaluation must be completed by clustering, the AI ​​evaluation results for the processed images are sent to the VetAI application (2349), and the best-match cluster results are sent to the VetImages application (2350), where the VetImages application generates screening results based on the best-match clustering results and stores them in a database (2351). If the VetImages application determines that screening evaluation by clustering is not to be performed, the search rules are accessed, and the AI ​​evaluation results are processed based on the search rules to obtain and store the screening results in a database (2353). The process of obtaining screening results and storing them in a database is repeated until the screening evaluation of all images of the case is completed and a complete result report is obtained (2354).

[0093] The VetImages application assesses (2355) whether the species of interest has been identified and stored in the database. If the species has not been identified, the VetAI application evaluates (2357) the species of interest and sends the species evaluation results to the VetImages application. The tasks for evaluating the species (2356-2357) are performed by an evaluation processor. In some embodiments, the VetImages application assesses (2358) whether the species is canine. If the species is positively identified as canine, the case is flagged (2359) and the evaluation is attached to the results report. The VetImages application notifies V2 that the case evaluation is complete (2360). The V2 application assesses (2361) whether the case is flagged. If the report is flagged, the results report is saved in the case document (2362) and the results report is emailed to the client (2363). If the report is not flagged, the results report is not saved in the case document but is emailed to the client (2363).

[0094] Clustering Clustering is a type of unsupervised learning method in which references are derived from a dataset consisting of input data without labeled responses. Generally, clustering is used as a process to find inherent, meaningful structures, explanatory underlying processes, generative features, and groupings within a set of instances.

[0095] Clustering is the task of dividing a population or data points into groups such that data points in the same group are similar to other data points in the same group and dissimilar to data points in other groups. Thus, clustering is a method of grouping objects based on the similarities and dissimilarities between them.

[0096] Clustering is an important process for determining intrinsic groupings among unlabeled data. There are no criteria for good clustering, as clustering depends on the user choosing criteria that are useful to meet the user's goals. For example, clusters can be based on finding representatives of homogeneous groups (data aggregation), finding "natural clusters" and describing their unknown properties ("natural" data types), finding useful and relevant groupings ("useful" data classes), or finding anomalous data objects (outlier detection). The algorithm generates hypotheses that account for point similarities, and each hypothesis generates different but equally valid clusters.

[0097] Clustering method: There are various methods for clustering:

[0098] Density-based methods: These methods consider clusters as dense regions that have a certain degree of similarity and are distinct from less dense regions of space. These methods have good accuracy and the ability to merge two clusters. Examples of density-based methods include Density-Based Spatial Clustering for Applications with Noise (DBSCAN) and Point Ordering for Identifying Cluster Structure (OPTICS).

[0099] Hierarchy-based methods: Clusters formed by this method form a hierarchy-based tree-type structure. New clusters are formed using pre-formed clusters. Hierarchy-based methods are classified into two categories: agglomerative (bottom-up approach) and divisive (top-down approach). Examples of hierarchy-based methods are: Clustering Using Representatives (CURE), Balanced Iterative Reducing Clustering, Hierarchies (BIRCH), etc.

[0100] Partitioning methods: Partitioning methods divide objects into k clusters, and each partition forms one cluster. This method is used to optimize an objective criteria similarity function. Examples of partitioning methods are K-means, Clustering Large Applications based upon Randomized Search (CLARANS), etc.

[0101] Grid-based methods: In grid-based methods, the data space is formulated into a finite number of cells that form a grid-like structure. All clustering operations performed on these grids are fast and independent of the number of data objects. Examples of grid-based methods are Statistical Information Grid (STING), wave cluster, and Clustering In Quest (CLIQUE).

[0102] K-means clustering K-means clustering is an unsupervised machine learning algorithm. Typically, unsupervised algorithms make inferences from a dataset using only input vectors, without reference to known or labeled outcomes. The goal of K-means is to simply group similar data points together and discover underlying patterns. To achieve this goal, K-means searches for a constant number (k) of clusters in the dataset.

[0103] A cluster refers to a set of data points that are agglomerated together for a certain similarity. The target number k refers to the number of centroids the user desires in the dataset. A centroid is an imaginary or real location that represents the center of a cluster. Every data point is assigned to each cluster by reducing the sum of squares within the cluster. The K-means algorithm identifies k number of centroids and then assigns every data point to the nearest cluster while keeping the centroids as small as possible.

[0104] The "mean" in K-means refers to the averaging of data to find centroids. To process the training data, the K-means algorithm in data mining starts with an initial group of randomly selected centroids that are used as starting points for all clusters, and then performs repeated (iterative) calculations to optimize the centroid locations. The algorithm stops creating and optimizing clusters when the centroids stabilize and their values ​​no longer change due to successful clustering, or when a defined number of iterations has been achieved.

[0105] The method for k-means clustering follows a simple approach: classifying a given dataset into a certain number of clusters (assuming k clusters) determined a priori. k centers or centroids are defined, one for each cluster. The next step is to take each point belonging to the given dataset and associate it with the nearest center. When there are no more outstanding points, the first step is completed and an initial group generation is performed. The next step is to recalculate k new centroids as the centroids of the clusters resulting from the previous step. Once the k new centroids are calculated, new bindings are performed between the same dataset points and the nearest new centers, thereby generating a loop. This loop results in the k centers changing location in stages until they no longer change location. The k-means clustering algorithm aims to minimize an objective function known as the squared error function, calculated as follows:

[0106]

number

[0107] Algorithmic Steps for K-Means Clustering The algorithm for k-means clustering is as follows:

[0108] In K-means clustering, c cluster centers are randomly selected, the distance between each data point and the cluster center is calculated, the data point is assigned to the closest cluster center, and new cluster centers are recalculated using the following formula:

[0109]

number

[0110] The distance between each data point and the newly obtained cluster center is measured, and if the data point is reassigned, the process continues until no data points are reassigned.

[0111] AI application process flow In some embodiments, Digital Imaging and Communications in Medicine (DICOM) images are submitted via LION and sent to the V2 platform over Hypertext Transfer Protocol Secure (HTTPS) by the DICOM ToolKit library (Offis.de DCMTK library). The DICOM images are temporarily stored on the V2 platform. In some embodiments, a DICOM record is created with limited information and the status is set to 0. In various embodiments, once the DICOM images are available in temporary storage, the V2 PHP / Laravel application begins processing the DICOM images through a Cron job.

[0112] In some embodiments, Cron job (1) monitors V2 for new DICOM images, retrieves the DICOM images from temporary storage, extracts tags, extracts frames (single or multiple subimages), saves the images and tags in a data store, and sets a processing status in the database. In some embodiments, Cron job (1) converts and compresses the DICOM images to lossless JPG format using the Offis.de library DCMTK and sets the processing status to 1. In some embodiments, Cron job (1) runs automatically, such as every 5 minutes, 4 minutes, 3 minutes, 2 minutes, or every minute. In some embodiments, Cron job (1) saves the DICOM image metadata in a table called DICOM in Microsoft SQL Server, extracts the images / frames, and stores the images / frames in a directory for the image manager. In various embodiments, a record is created during processing that stores additional information about the image and the case ID associated with the image. The record stores additional data such as physical examination findings for the subject, multi-visit studies for the subject, the series of images obtained during each examination, and the hierarchy of images included in the case.

[0113] In various embodiments, the DICOM images and metadata are processed by a Vet Images application written in the PHP Laravel framework. V2 makes a REST service request to VetImages to process each image asynchronously. In some embodiments, VetImages responds immediately to V2 to confirm that the request was accepted and the image cropping and evaluation process continues in the background. Because images are processed in parallel, the entire process runs quickly.

[0114] In various embodiments, VetImages passes or forwards images to a module called VetConsole, which is written in Python and uses OpenCV computer vision technology to preprocess the images. VetConsole identifies body regions such as the chest, abdomen, and pelvis as a fallback in case the AI ​​Cropping server cannot classify the body region in the image. VetImages rotates and flips the image until the correct orientation is obtained. In some embodiments, VetImages uses an image matching server to check various angles and projections of the image. In various embodiments, the image matching server is written in Python and Elastic Search to identify image matches. In some embodiments, the image database for the image matching server is carefully selected so that it returns results only if the image is found to be in the correct orientation and projection.

[0115] In various embodiments, once the image orientation is determined, VetImages sends a REST API request to a Keras / TensorFlow server for classification and to determine the body region of interest in the image. The VetImages REST API request is validated using an image matching server to verify that the returned image regions are classified as body regions, such as chest, abdomen, pelvis, stifle, etc. In some embodiments, if the cropping assessment result is disabled, the VetConsole cropping result is enabled and utilized.

[0116] In various embodiments, if VetImages determines that the image contains a classified and validated body region, an AI Evaluation process is initiated to generate an AI report. In an alternative embodiment, if VetImages determines that the image does not contain a classified and validated body region, the cropping image process ends without results and without report generation.

[0117] In various embodiments, VetImages sends a REST service call to Keras / TensorFlow with the cropped images classified by an AI assessment model for disease hosted on a TensorFlow application server written in Python / Django. VetImages stores the results of the AI ​​assessment model for final assessment and report generation.

[0118] VetImages also sends the chest-cropped images to a TensorFlow server to determine if the images were properly positioned relative to the parameters set by the user. VetImages sends the results of the AI ​​evaluation model to the V2 platform to inform clinics of the results of the positioning evaluation for each image.

[0119] In some embodiments, VetImages waits until all images for a case have been cropped and evaluated by the TensorFlow server while the images are processed in parallel. In some embodiments, once all images for a case have been evaluated, VetImages processes all results for the case using expert-defined rules to determine the content of the report in a more human-readable manner. In alternative embodiments, VetImages uses a clustering model created by the user to determine the content of the AI ​​report. In some embodiments, the AI ​​report is assembled using previous radiologist reports, which are used to build the cluster model in VetImages. In some embodiments, clustering is used to classify cases / images using predicted results from other diagnostic models using scikit-learn.

[0120] In some embodiments, once expert rules or cluster models are used to determine the content of the AI ​​report, VetImages checks the species of the case, and in some embodiments, the report is generated and sent to the V2 clinic only if the species of the case is determined by VetImages to be canine.

[0121] In some embodiments, VetImages sends a request to the V2 platform to notify the clinic that a new AI report has been sent to the clinic. In some embodiments, the V2 platform validates the clinic admin user license or user license. In various embodiments, if the clinic has a valid license, V2 attaches a copy of the report to or within the Case Document(s) so that the report is available from the V1 platform. In some embodiments, V2 sends an email notification to the clinic email containing the link(s) so that the email receiver can conveniently and immediately open the generated report.

[0122] Methods for machine learning, clustering, and programming are fully described in the following references: Shaw, Zed. Learn Python the Hard Way: A Very Simple Introduction to the Terrifyingly Beautiful World of Computers and Code. Addison-Wesley, 2017; Ramalho, Luciano. Fluent Python. O'Reilly, 2016; Atienza, Rowel. Advanced Deep Learning with TensorFlow 2 and Keras: Apply DL, GANs, VAEs, Deep RL, Unsupervised Learning, Object Detection and Segmentation, and More. Packt, 2020; Vincent, William S. Django for Professionals: Production Websites with Python & Django. Still River Press, 2020; Bradski, Gary R., and Adrian Kaehler. Learning OpenCV: O'Reilly, 2011; Battiti, Roberto, and Mauro Brunato. The LION Way: Machine Learning plus Intelligent Optimization, Version 2.0, (April 2015). LIONlab Trento University, 2015; Pianykh, Oleg S. Digital Imaging and Communications in Medicine (DICOM) a Practical Introduction and Survival Guide. Springer Berlin Heidelberg, 2012; Busuioc, Alexandru.The PHP Workshop: a New, Interactive Approach to Learning PHP. Packt Publishing, Limited, 2019; Stauffer, Matt. Laravel - Up and Running: a Framework for Building Modern PHP Apps. O'Reilly Media, Incorporated, 2019; Kassambara, Alboukadel. Practical Guide to Cluster Analysis in R: Unsupervised Machine Learning. STHDA, 2017; and Wu, Junjie. Advances in K-Means Clustering: A Data Mining Thinking. Springer, 2012. These references are incorporated herein by reference in their entirety.

[0123] It will be understood that any feature described with respect to any one of the embodiments may be used alone or in combination with the other features described, and also with one or more features of any other embodiment or one or more features of any combination of any other embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention as defined in the claims.

[0124] The present invention has now been fully described, and it is further illustrated by the following claims. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, numerous equivalents to the specific methods described herein. Such equivalents are within the scope of the present invention and the claims. The contents of all references cited in this application, including non-patent literature, issued patents, and published patent applications, are hereby incorporated by reference in their entirety.

Claims

1. 1. A method of analyzing a diagnostic image of a subject, the method comprising: automatically processing the images of the subject using a sub-image processor to classify the images into one or more body regions and orienting and cropping the classified images to obtain at least one classified image; orienting and cropping the classified images according to Digital Imaging and Communication in Medicine (DICOM) standard hanging protocols to obtain at least one oriented, cropped, and labeled sub-image for each automatically classified body region; transmitting said sub-images to at least one artificial intelligence processor based on said body region; evaluating the sub-images with the artificial intelligence processor to obtain an evaluation result; and comparing the assessment results to a database having a plurality of dataset clusters calculated based on previous assessment results and matched written templates corresponding to the classified images to obtain at least one cluster result indicative of the presence of a disease or disorder of interest.

2. The method of claim 1 further comprising using the artificial intelligence processor to assess the sub-images for body region and the presence of a pathology.

3. The method of claim 2 further comprising using the artificial intelligence processor to diagnose a medical condition from the sub-images.

4. The method of claim 1 , further comprising using the artificial intelligence processor to assess the sub-images with respect to the location of the body region of the subject.

5. The method of claim 4 , further comprising the step of correcting the position of the body region of the subject to an appropriate position.

6. The method of claim 1 , wherein the sub-image processor automatically processes the image to obtain the sub-image.

7. 7. The method of claim 6, wherein the sub-image processor processes the image to acquire the sub-image in less than about 1 minute, less than about 30 seconds, less than about 20 seconds, less than about 15 seconds, less than about 10 seconds, or less than about 5 seconds.

8. The method of claim 1 , wherein the evaluating step further comprises comparing the sub-image to a plurality of reference images in at least one of a plurality of libraries.

9. The method of claim 8 , wherein the plurality of libraries each include a plurality of reference images.

10. 10. The method of claim 9, wherein the plurality of libraries comprises a respective plurality of reference images, each of which is species-specific or non-specific.

11. The method of claim 1 , further comprising matching the sub-image with a reference image to assess orientation and at least one body region.

12. 9. The method of claim 8, wherein the reference image is oriented within the Digital Imaging and Communication in Medicine (DICOM) standard hanging protocol.

13. The method of claim 1 , wherein cropping further comprises isolating a particular body region in the sub-image.

14. 10. The method of claim 9, further comprising classifying the reference image by veterinary standard body area labels.

15. The method of claim 1 , wherein orienting further comprises adapting the image to a veterinary standard hanging protocol.

16. The method of claim 1 , wherein cropping further comprises cropping the sub-image to a standard aspect ratio.

17. 10. The method of claim 1, wherein classifying further comprises identifying and labeling the body region with veterinary standard body region labels.

18. The method of claim 1 , wherein classifying further comprises comparing the image to a library of sample standard images.

19. 20. The method of claim 18, further comprising classifying the image into one or more body regions by matching the image to sample standard images in the library.

20. The method of claim 1 , wherein cropping further comprises identifying a boundary in the image that depicts each classified body region.

21. The method of claim 1 further comprising extracting a signature of the image prior to classifying.

22. 10. The method of claim 1, wherein the image is obtained from a radiological examination selected from radiology, magnetic resonance imaging (MRI), magnetic resonance angiography (MRA), computed tomography (CT), fluoroscopy, mammography, nuclear medicine, positron emission tomography (PET), and ultrasound.

23. The method of claim 1 , wherein the image is a photograph.

24. 10. The method of claim 1, wherein the subject is selected from mammals, reptiles, fish, amphibians, chordates, and birds.

25. 25. The method of claim 24, wherein the mammal is selected from a dog, a cat, a rodent, a horse, a sheep, a cow, a goat, a camel, an alpaca, a buffalo, an elephant, and a human.

26. 10. The method of claim 1, wherein the subject is selected from pets, livestock, precious zoo animals, wild animals, and research animals.

27. 10. The method of claim 1, further comprising automatically generating, by said artificial intelligence processor, at least one report containing an evaluation of said sub-images.

28. 1. A system for analyzing an image of an object, the system comprising: a receiver for receiving an image of the object; at least one processor that automatically executes image identification and processing algorithms to identify, crop, orient, and label at least one body region in the image and acquire sub-images; at least one artificial intelligence processor that evaluates the sub-images based on the body region and compares the evaluation results with a database having a plurality of data set clusters calculated based on previous evaluation results and matched written templates corresponding to classifications of the images to obtain at least one cluster result indicative of the presence of a disease or disorder in the subject; a device for displaying results of an artificial intelligence processor evaluated based on the sub-images and the cluster results.

29. 30. The system of claim 28, wherein the processor automatically processes the image to obtain the sub-image.

30. 30. The system of claim 29, wherein the processor processes the image in less than 1 minute, less than 30 seconds, less than 20 seconds, less than 15 seconds, less than 10 seconds, or less than 5 seconds to obtain a labeled image.

31. 30. The system of claim 28, further comprising a library of standard images.

32. 32. The system of claim 31, wherein the standard images comply with veterinary standards regarding hanging protocols and body area markings.

33. 1. A method for automatically preparing an image of an object for display, the method comprising: using a processor to process raw images of the subject, automatically cropping and extracting signatures, and algorithmically classifying the images into one or more distinct body region categories by comparing the cropped and oriented image signatures to a database of image signatures of known orientations and body regions to obtain the best matching orientation and body region indicators; sending the image to at least one artificial intelligence (AI) evaluation processor based on the body region markings and orientation to obtain an evaluation result, and comparing the evaluation result to a database having a plurality of data set clusters calculated based on previous evaluation results and matched writing templates corresponding to the marked image to obtain at least one cluster result; and displaying, on a display device, each labeled image of the body region prepared for analysis based on the cluster results.

34. An improvement in a veterinary radiological diagnostic image analyzer, the improvement comprising: using a processor to pre-process a radiological image of a subject and running a fast algorithm to automatically identify one or more body regions in the image; the processor further operative to do at least one of automatically creating separate sub-images for each identified body region; cropping and optionally standardizing the aspect ratio of the created sub-images; automatically labeling each sub-image as a body region; and automatically orienting the body regions in the sub-images; the processor further operative to automatically send the diagnostic sub-images to at least one artificial intelligence processor specific to evaluating the cropped, oriented, and labeled sub-images to generate an evaluation result; and comparing the evaluation result to previous evaluation results and a database having a plurality of dataset clusters calculated based on matched written templates corresponding to the labeled diagnostic sub-images to obtain at least one cluster result indicative of the presence of a disease or disorder in the subject.

Images