Image data bias detection with explainability in machine learning

Abstract

Embodiments of this disclosure relate to the detection of interpretable image data bias in machine learning. Bias in machine learning (ML) refers to the tendency of ML algorithms to learn relevant and important patterns incompletely or incorrectly from the dataset. This inaccuracy may cause the algorithm to miss important relationships between patterns and features in the data, resulting in inaccurate predictions. This document describes a system and method for detecting potential ML bias in an input image dataset. After receiving a target image, a subset of images related to the target image is extracted. The target image and the subset of images are analyzed under imbalanced evaluation and data bias evaluation to determine whether any potential data bias exists in the ML training pipeline. If any data bias is identified, one or more messages summarizing the evaluation and including explanations for achieving more accurate predictions in the image evaluation are sent to the user.

Description

Background Technology

[0001] Artificial intelligence possesses immense power and potential to positively impact life and nature. Its applications have already permeated all industries, including healthcare, manufacturing, finance, retail, and life sciences. Applying the principles and power of AI in an unbiased, non-discriminatory manner, with ethical standards and credibility, is of paramount importance.

[0002] Bias in machine learning (ML) refers to the tendency of ML algorithms to learn relevant and important patterns from a dataset incompletely, or to learn patterns incorrectly from the data. This inaccuracy may cause the algorithm to miss important relationships between patterns and features in the data, or to assert that there are actually no relationships between patterns and features in the data, thus leading to inaccurate predictions.

[0003] When ML bias is present in an artificial intelligence (AI) environment, biases can occur at all stages of ML development. ML bias can lead to social bias through discriminatory data that mixes in personal attributes such as socioeconomic status, age, race, gender, disability, income, religion, and demographics. This social bias can further lead to statistical bias, where the ML model does not accurately represent the data. Statistical bias in the data can be caused by attribute imbalance, insufficient data, and inadequate data collection. Statistical bias in the data can lead to biased predictions, which can have serious consequences in AI environments in specific industries such as healthcare and finance.

[0004] Currently, image datasets are inherently unstructured and heterogeneous. Data bias can exist in image datasets due to the variety of imaging devices and the image quality provided for each type, image transformation, the influence of attributes due to inaccurate representation of the image dataset (i.e., skin color in facial recognition or gender in healthcare), and domain-specific data. Therefore, there is a need for ML models trained with images possessing diverse features to help reduce and prevent inaccurate predictions of real-world data displayed on images. This paper discloses a solution for detecting potential ML biases in input image datasets by evaluating feature differences within the input image dataset and providing the user with results analysis. Attached Figure Description

[0005] This disclosure is described in detail below with reference to one or more different embodiments and the accompanying drawings. These drawings are provided for illustrative purposes only and depict only typical or exemplary embodiments. References to these illustrative examples are not intended to limit or define this disclosure, but rather to provide examples to aid in understanding its contents. Additional examples are discussed in the detailed description, and further description is provided therein.

[0006] Figure 1This is an example illustration of a computational system for detecting potential ML data biases in an input image dataset according to an example embodiment described in this disclosure.

[0007] Figure 2 This is an example flowchart illustrating how to detect potential ML biases in an input image dataset according to various embodiments of the present disclosure.

[0008] Figure 3 This is an example illustration of a process for detecting potential ML data bias in an input image dataset according to various embodiments of the present disclosure.

[0009] Figure 4 This is an example illustration of a process for performing mean-shift clustering on an image according to various embodiments of the present disclosure.

[0010] Figure 5 This is an example computing component comprising one or more hardware processors and a machine-readable storage medium storing a machine-readable / machine-executable instruction set, which, when executed, causes one or more hardware processors to perform illustrative methods for detecting potential ML biases in an input image dataset according to various embodiments of this disclosure.

[0011] Figure 6 A block diagram of an example computer system in which various embodiments of the present disclosure may be implemented is shown.

[0012] These accompanying drawings are not exhaustive and do not limit this disclosure to the exact form disclosed. Detailed Implementation

[0013] Image datasets can be used to provide accurate predictions about the features displayed in a particular image. Machine learning (ML) models trained with images having different features, attributes, and labels help reduce and prevent inaccurate predictions of real-world data displayed on images. To help improve the probability of obtaining accurate predictions, it is crucial to have accurate image classification labels, attributes associated with each image, and features displayed in each image to aid the learning of the ML model. The accuracy of image, attribute, and feature classification labels can contribute to all forms of detection, such as early disease diagnosis, facial recognition, signature forgery detection, and foreign object detection. ML models can also help with domain adaptation of images, allowing image features to be scanned, extracted, and classified by an ML model with similar reference images, regardless of the domain source used to capture the image.

[0014] To meet the need for accurate predictions of features displayed in images, one solution is to incorporate potential data bias detection into the ML training pipeline. The ML training pipeline can include an evaluation of feature differences within an image dataset. This evaluation can detect and determine whether the images in the dataset contain data biases that may lead to inaccurate predictions. The ML training pipeline can further compute an analysis of the evaluation. This analysis can provide the user with an explanation of the cause of any identified data biases, allowing the user to implement corrective actions, such as image augmentation or ML model hyperparameter tuning, to understand underrepresented features. The system can adaptively receive input image datasets while performing the data bias detection evaluation with or without external intervention.

[0015] This solution also incorporates concurrent pipelined processing of mean-shift clustering computations across multiple CPU resources to accelerate the computational process of the ML training pipeline model. This further improves overall image processing throughput and the speed of data bias detection evaluation when images are input into the bias detection pipeline. Enhanced clustering techniques across multiple CPU resources, combined with accelerated evaluation and computation of the ML pipeline model, are highlighted components to improve the timely detection of potential data biases and enable accurate predictions in image evaluation. These highlighted components also reduce manual user inspection, decrease trial-and-error in image segmentation, increase the accuracy of feature detection, and enhance the computational performance of the solution.

[0016] This document describes a solution to the aforementioned problems. In some embodiments, the computational system may be an ML training pipeline and a model. In other embodiments, the computational system may provide a systematic approach to detecting and analyzing potential data biases in an input image dataset within an ML pipeline. In various embodiments, the computational system may receive an input target image. The input target image may include one or more attributes and features. A user may specify specific attributes of the target image to be evaluated against an image training dataset to determine potential data biases. Upon receiving the input target image, the computational system may identify, separate, and extract subsets of images from an image training database based on the attributes and features of the input target image. The image training database may store multiple image training datasets that can be used for any type of input target image with any combination of attributes and features. The computational system may then perform a class imbalance evaluation on the subset of images. The class imbalance evaluation may indicate whether the subset of images extracted based on the attributes of the input target image contains any potential data biases. If the class imbalance evaluation determines that there is an imbalance in the subset of images compared to other groups of images, it may discover potential biases.

[0017] The computational system can also perform mean-shift clustering on a subset of images and the input target image to determine clusters of data distribution for each image, where each cluster represents a feature of the corresponding image. The computational system can then compare the clusters in the input target image with the clusters in each image of the subset and analyze the distributional differences of the clusters between the input target image and each individual image in the subset. Once the clusters in the input target image have been compared and analyzed with the clusters of all images in the subset, the computational system can perform a data bias assessment on the clusters in the input target image. The data bias assessment can detect and determine any potential data bias in the input target image. Data bias exists if the overall distributional difference between the clusters of the input target image and the clusters of all images in the subset is greater than a threshold. Any data bias determined based on the class imbalance assessment and the cluster distributional differences can be summarized. The computational system can then send a message summarizing the data bias to the user. These and other features of the examples of this disclosure are discussed herein.

[0018] Figure 1 An example of a computing system 100 that may be within or otherwise associated with device 150 is shown. In some embodiments, computing system 100 may be an ML pipeline and model. In some examples, device 150 may be a computing device such as a desktop computer, laptop computer, mobile phone, tablet device, Internet of Things (IoT) device, etc. Device 150 may output and display an image 160 of a dataset on its screen. Image 160 may be a two-dimensional (2D) graphical representation of a dataset showing various outputs predicted by an ML model based on various X and Y variables. Computational component 110 may perform one or more available evaluations on the input image dataset to detect any potential data bias. Image 160 may display any potential biases on the input image dataset based on one or more of the performed evaluations. The computing component 110 may include one or more hardware processors and logic 130, which implement instructions to perform functions of the computing component 110, such as receiving an input target image, identifying and extracting a subset of images from an image database based on one or more attributes of the input target image, analyzing the subset of images to determine class imbalance assessment, performing mean-shift clustering on the input target image and each image in the subset, determining one or more clusters in the input target image and each image in the subset based on mean-shift clustering, performing data bias assessment on the clusters of the input target image, and sending one or more data bias messages based on the class imbalance assessment and the data bias assessment. The computing component 110 may store details in a database 120 about scenarios or conditions in which certain algorithms, image datasets, and assessments are performed to determine potential data bias in the input image dataset. Some scenarios or conditions will be illustrated in the following figures.

[0019] Figure 2 An example scenario is illustrated, in which process 200 can selectively perform one or more types of evaluations on the input image dataset, for example, to detect any potential data biases in the ML pipeline. In some embodiments, process 200 may be, for example, by... Figure 1 The computational component 110 performs the operation. In other embodiments, process 200 can be implemented as follows: Figure 1 The computing component 110. The computing component 110 may be, for example... Figure 3 Process 300 Figure 4 Process 400 and Figure 5 The computing component 500. The computing component 110 may include a server.

[0020] In box 210, computing component 110 receives a target image. The target image may be input by a user into device 150 for receiving by computing component 110. The target image may include one or more features, attributes, and / or tags. Features may include prominent parts of an image of people, objects, structures, items, and anything else that may be displayed in the image. Attributes may include age, gender, race, ethnicity, religion, income, demographics, material, and any other factors or features that may be directly associated with one or more features in a particular image. Tags may be a classification or description of the entire image. In some embodiments, attributes associated with the target image may be assigned by the user before the target image is input into device 150. In other embodiments, attributes associated with the target image may be determined using one or more algorithms for detecting image attributes after computing component 110 receives the target image. The target image may be tagged with a classification or description that describes its image type or content displayed in the image. In some embodiments, tags may be assigned to the target image before it is input into device 150.

[0021] In one example, the target image could be an X-ray image showing human features (such as lungs). Before the user inputs the target image into device 150, the user can assign attributes to the person whose lungs and heart are shown in the X-ray image. As an example, the user can assign the attribute $50K to income and the country name to ethnicity. The user can assign the attribute female to gender and the country name to ethnicity. After device 150 receives the image, computing component 110 can use one or more algorithms to determine any additional attributes associated with the person whose lungs and heart are shown in the X-ray target image, such as assigning 56 to age, male to gender, and white / Caucasian to race. The target image can also be labeled with a classification "lung" before being input into device 150.

[0022] In box 212, computation component 110 extracts a subset of target images from an image database. After receiving the target images and determining one or more attributes associated with them, the computation component may first identify groups or subsets of reference images from a reference image database, wherein the subsets of reference images are selected based on the classification labels of the target images and / or the attributes associated with them. Reference images can be considered as a source image dataset. The reference image database may contain a large number of images, where each image belongs to one or more subsets based on any combination of classification labels and attributes associated with each image. The image database may be stored in database 120. After the subsets of reference images have been identified based on the target images' classification labels(s) and / or attributes(s), computation component 110 can extract the subsets of reference images from the reference image database.

[0023] In box 214, computation component 110 can analyze a subset of images to determine imbalance assessment. After extracting a subset of images from an image database based on the target image's classification label(s) and / or attributes(s), computation component 110 can analyze the subset of images by performing a class imbalance assessment on the subset of images. The class imbalance assessment can indicate whether the subset of images extracted based on the target image's classification label(s) and / or attributes(s) leads to any potential data bias.

[0024] In some embodiments, histogram analysis can be used to perform class imbalance assessment. To perform class imbalance assessment, the computing device may first identify one or more reference image groups from a reference image database, wherein the one or more reference image groups contain multiple classification labels and / or multiple attributes similar to the target image. After identifying one or more reference image groups, the computing component 110 may determine the number of images in each reference image group. The computing component 110 may then analyze the different number of images in each reference image group and compare it to the total number of images in all reference image groups. This analysis and comparison may determine the average number or percentage of images in a particular reference image group across all reference image groups. In some embodiments, the average number or percentage of images may represent an imbalance threshold or minimum number of reference images in each reference image group necessary to obtain accurate predictions in the analysis of the input target image and dataset. In other embodiments, the imbalance threshold may be a predetermined percentage of the total number of reference images across all reference image groups. A subset of reference images that does not contain a number or percentage of reference images equal to or greater than the imbalance threshold may be determined to be imbalanced. An imbalanced subset of reference images may contain potential data bias and provide inaccurate predictions in the ML model.

[0025] In one example, for a target image with the classification labels “lung” and “pneumonia” and an age attribute of 56, a sex of male, and an ethnicity of Caucasian / Caucasian, a subset of images containing 55 reference images was identified as having the same labels and attributes and was extracted. The computational component 110 also identified six reference image groups from a reference image database. Each of these six reference image groups has the label “lung” and the same attributes: age 56, sex of male, and ethnicity of Caucasian / Caucasian, but includes a second label that is not “pneumonia,” such as “asthma,” “pneumothorax,” “smoking,” “cancer,” “bronchitis,” and “emphysema.” These six reference image groups have numbers of 88, 106, 94, 98, 65, and 77 reference images, respectively. The average number of reference images in these six reference image groups is 88 ((88+106+94+98+65+77) / 6). If the imbalance threshold is 88, then a subset of the images is imbalanced because its number is 55, and the number of reference images is below the imbalance threshold of 88. If the imbalance threshold is a predetermined percentage of 10%, then a subset of the images is balanced because the imbalance threshold will be 52.8 (10% * (88 + 106 + 94 + 98 + 65 + 77)) and 55 is greater than 52.8.

[0026] The computation component 110 can then determine the outcome of the class imbalance assessment for a subset of images. The computation component 110 can compare the number of reference images in the subset of images to an imbalance threshold for the reference images. If the subset of images extracted based on features and / or attributes of the target image contains a number of reference images below the imbalance threshold, the subset of images can be determined to be imbalanced and contain potential data bias. If the subset of images contains a number of reference images equal to or greater than the imbalance threshold, the subset of images can be determined to be non-imbalanced and contain no potential data bias.

[0027] In box 216, computation component 110 may send a message to the user about data biases found in a subset of images. Computation component 110 may summarize the imbalance assessment performed on the subset of images. This summary may include all labels, attributes, and reference image groups used in the imbalance assessment. The summary may also include a detailed explanation of the causes of the imbalanced results. Computation component 110 may include the summary of the imbalance assessment along with the imbalanced results in a message. Computation component 110 may then send this message to the user. The user can gain a full understanding of the causes of the imbalance in the subset of images and be able to adjust the data using techniques such as data augmentation, adjusting the ML model, or other factors of the computational system in the ML pipeline to obtain balanced results from the imbalance assessment.

[0028] In box 218, computation component 110 can perform mean-shift clustering on each reference and target image within a subset of the images. The mean-shift clustering performed on each image can determine one or more clusters of the data distribution for each image. Each cluster in the image can represent a feature of the corresponding image. Each feature represented by a cluster can be a significant or prominent feature in the corresponding image.

[0029] Mean-shift clustering can be a nonparametric algorithm that uses kernel density estimation and kernel bandwidth to establish the underlying data distribution of a specific image. Kernel density estimation and kernel bandwidth can be used to iteratively assign data points to clusters by drifting points towards the highest density data points. Kernel bandwidth can be manually specified by the user as part of a trial-and-error process for prior visualization of the image. In one example, a high and fine kernel bandwidth might be used, which could result in a large cluster count with low density for each cluster. In another example, a short and coarse kernel bandwidth might be used, which could result in a small cluster count with high density for each cluster. Using manually specified kernel bandwidth can be a drawback, as it is a tedious and error-prone process when evaluating large, non-uniform image sets. This can be overcome by applying an algorithm that automatically calculates the kernel bandwidth for mean-shift clustering.

[0030] The mean-shift clustering algorithm may include four steps. In the first step, computational component 110 may compute the Hopkins statistic (“H”) for a specific image. In the second step, computational component 110 may use the Hopkins statistic (H) to derive the quantile values ​​(“Q”) for the specific image using a mirror sigmoid function derived from H. In the third step, computational component 110 may use the quantile values ​​(Q) to estimate the kernel bandwidth of the specific image. In the fourth step, computational component 110 may perform mean-shift clustering on the specific image. This mean-shift clustering algorithm may be performed for each reference image within a subset of the image and for the target image.

[0031] In some embodiments, mean-shift clustering of each image can be performed in series. In other embodiments, mean-shift clustering of each image can be performed in parallel across multiple processors in computing system 200. The ability to perform mean-shift clustering of multiple images in parallel across multiple processors can accelerate the computational performance of clustering images. This acceleration of the computational performance of clustering images may further lead to an acceleration of the computational performance of performing data bias assessment on the target images.

[0032] In box 220, computation component 110 can determine clusters of data points in each reference image within the target image and subsets of the image. Computation component 110 can group each individual data point in the image into clusters by placing boundaries around each individual cluster to clearly show the data points in the image within each cluster.

[0033] In box 222, computation component 110 can perform a data bias assessment on the clusters of the target image. Data bias assessment can be performed by evaluating the distributional differences between the clusters of the reference images and the clusters of the target image. In some embodiments, an ML algorithm can be used to perform the data bias assessment, wherein the ML algorithm can create an ML model using the data from the assessment of the distributional differences of clusters across reference images. The ML model can then be trained to identify patterns from the assessed data to determine the presence of data bias. Computation component 110 can compare the clusters in the target image with the clusters in each reference image within a subset of the images and analyze the distributional differences of the clusters between the target image and each individual reference image within the subset of images. Once the clusters in the target image have been compared and analyzed with the clusters of all reference images within the subset of images, a data bias can be determined. Data bias may exist if the overall distributional difference between clusters is greater than a consistency threshold. The overall distributional difference between clusters can be a computationally balanced measure of each difference between the clusters of the target image and each reference image within the subset of images. In some embodiments, the consistency threshold can be a predetermined amount of difference. In other embodiments, the consistency threshold can be a predetermined percentage of the total difference between the clusters of the target image and all reference images within a subset of the image. When the data bias assessment results in an overall distributional difference between clusters exceeding the consistency threshold, the calculation component 110 can determine an inconsistency. The inconsistency can indicate the presence of data bias in the target image.

[0034] In box 224, computation component 110 may send a message to the user regarding data biases found in the target image. Computation component 110 may summarize the data bias assessment performed on the target image. This summary may include all labels, attributes, and subsets of images used in the data bias assessment. The summary may also include values ​​of variables determined by the performance of mean-shift clustering on the target image and on each reference image within the subset of images. The summary may also include a detailed explanation of the reasons for the inconsistent results in the data bias assessment. Computation component 110 may include the summary of the data bias assessment along with the inconsistent results in a message. Computation component 110 may then send this message to the user. The user can gain a full understanding of the reasons for the inconsistencies in the target image and can adjust the data using techniques such as data augmentation, adjusting the ML model, or other factors of the computational system in the ML pipeline to obtain consistent results from the data bias assessment.

[0035] For the sake of brevity, process 200 is described as being performed for a single received target image. It should be understood that, in a typical embodiment, computing component 110 may manage multiple target images in a short-term sequential manner. For example, in some embodiments, computing component 110 may perform many (if not all) of the steps in process 200 on multiple target images when a target image is received.

[0036] As explained, process 200 can provide solutions for improving the detection of potential data biases in the ML training pipeline and achieving more accurate predictions in image evaluation. Process 200 can also improve the computational process of the ML training pipeline model by incorporating concurrent pipelined processing across multiple CPU resources. These improvements can also reduce manual user inspection, reduce trial and error in image segmentation, improve the accuracy of feature detection, and accelerate the computational performance of the solution.

[0037] Figure 3 An example scenario is illustrated, in which process 300 can selectively perform various types of evaluations on the input image dataset, for example, to detect any potential data biases in the ML pipeline. In some embodiments, process 300 may be, for example, by Figure 1 The computational component 110 performs the operation. In other embodiments, process 300 can be implemented as follows: Figure 1 The computing component 110. The computing component 110 may be, for example... Figure 2 Process 200 Figure 4 Process 400 and Figure 5 The computing component 500. The computing component 110 may include a server.

[0038] Step 310 of process 300 is similar to box 210 of process 200. In step 310, computing component 110 receives a target image. The target image may be input by a user into device 150 for receiving by computing component 110. The target image may include one or more features, attributes, and / or tags. Features may include prominent parts of an image of people, objects, structures, items, and anything else that may be displayed in the image. Attributes may include age, gender, race, ethnicity, religion, income, demographics, material, and any other factors or features that may be directly associated with one or more features in a particular image. Tags may be a classification or description of the entire image. In some embodiments, attributes associated with the target image may be assigned by the user before the target image is input into device 150. In other embodiments, attributes associated with the target image may be determined using one or more algorithms for detecting image attributes after computing component 110 receives the target image. The target image may be tagged with a classification or description that describes its image type or content displayed in the image. In some embodiments, tags may be assigned to the target image before it is input into device 150.

[0039] In step 312, the computing component 110 determines the attributes associated with the target image. Upon receiving the target image, the computing component 110 can determine the attributes associated with it. The target image may have already been assigned attributes. The computing component 110 can scan the target image and identify any additional attributes associated with it that have not yet been assigned. Any identified additional attributes can be assigned to the target image by the computing component 110.

[0040] In step 314, the computing component 110 determines the tags associated with the target image. After receiving the target image, the computing component 110 can determine the tags associated with the target image. The target image may have already been assigned a tag. The computing component 110 can scan the target image and identify any additional tags associated with the target image that have not yet been assigned. Any identified additional tags can be assigned to the target image by the computing component 110.

[0041] Step 316 of process 300 is similar to box 212 of process 200. In step 316, computation component 110 extracts a subset of images from the image data source. After receiving the target image and determining one or more attributes and labels associated with the target image, the computation component may first identify a group or subset of reference images from image data source 340, wherein the subset of reference images is selected based on the classification label of the target image and / or the attributes associated with the target image. The reference images may be considered as a source image dataset. Image data source 340 may be an image database and may contain a large number of images, wherein each image belongs to one or more subsets based on any combination of classification labels and attributes associated with each image. Image data source 340 may be stored in database 120. After the subset of reference images has been identified according to the classification(s) and / or attributes(s) of the target image, computation component 110 can extract the subset of reference images from image data source 340.

[0042] Step 318 of process 300 is similar to box 214 of process 200. In step 318, computation component 110 may analyze a subset of images to determine imbalance assessment. After extracting a subset of images from image data source 340 based on the target image's classification label(s) and / or attributes(s), computation component 110 may analyze the subset of images by performing a class imbalance assessment on the subset of images. The class imbalance assessment may indicate whether the subset of images extracted based on the target image's classification label(s) and / or attributes(s) contains any potential data bias.

[0043] In some embodiments, histogram analysis can be used to perform class imbalance assessment. To perform class imbalance assessment, the computing device may first identify one or more reference image groups from a reference image database, wherein the one or more reference image groups contain multiple classification labels and / or multiple attributes similar to the target image. After identifying one or more reference image groups, the computing component 110 may determine the number of images in each reference image group. The computing component 110 may then analyze the different number of images in each reference image group and compare it to the total number of images in all reference image groups. This analysis and comparison may determine the average number or percentage of images in a particular reference image group across all reference image groups. In some embodiments, the average number or percentage of images may represent an imbalance threshold or minimum number of reference images in each reference image group necessary to obtain accurate predictions in the analysis of the input target image and dataset. In other embodiments, the imbalance threshold may be a predetermined percentage of the total number of reference images across all reference image groups. A subset of reference images that does not contain a number or percentage of reference images equal to or greater than the imbalance threshold may be determined as imbalanced. An imbalanced subset of reference images may introduce potential data bias and provide inaccurate predictions in the ML model.

[0044] In one example, for a target image with the classification labels “arm” and “fracture”, age as attribute 12, gender as female, and ethnicity as Hispanic, a subset of images containing 72 reference images is identified as having the same labels and attributes and is extracted. The computation component 110 also identifies four groups of reference images from image data source 340. Each of these four groups of reference images has the label “arm” and the same attributes: age 12, gender as female, and ethnicity as Hispanic, but includes a second label that is not “fracture”, such as “burn,” “bruise,” “dislocation,” and “breakage.” These four groups of reference images have numbers of 34, 156, 27, and 43, respectively. The average number of reference images in these four groups is 65 ((34+156+27+43) / 4). If the imbalance threshold is 65, the subset of images is balanced because its number of 72 reference images is greater than the imbalance threshold of 65. If the imbalance threshold is a predetermined percentage of 30%, then a subset of the image will be imbalanced because the imbalance threshold will be 78 (30% * (34 + 156 + 27 + 43)) and 72 is less than 78.

[0045] Step 320 of process 300 is similar to box 218 of process 200. In step 320, computation component 110 may perform mean-shift clustering for each reference image and target image in a subset of images. The mean-shift clustering performed on each image may determine one or more clusters of the data distribution for each image. Each cluster in the image may represent a feature of the corresponding image. Each feature represented by the cluster may be a feature that is important or prominent in the corresponding image.

[0046] Mean-shift clustering can be a nonparametric algorithm that uses kernel density estimation and kernel bandwidth to establish the underlying data distribution of a specific image. Kernel density estimation and kernel bandwidth can be used to iteratively assign data points to clusters by drifting points towards the highest density data points. Kernel bandwidth can be manually specified by the user as part of a trial-and-error process for prior visualization of the image. In one example, a high and fine kernel bandwidth might be used, which could result in a large cluster count with low density for each cluster. In another example, a short and coarse kernel bandwidth might be used, which could result in a small cluster count with high density for each cluster. Using manually specified kernel bandwidth can be a drawback, as it is a tedious and error-prone process when evaluating large, non-uniform image sets. This can be overcome by applying an algorithm that automatically calculates the kernel bandwidth for mean-shift clustering.

[0047] The mean-shift clustering algorithm may include four steps. In the first step, computational component 110 may compute the Hopkins statistic (“H”) for a specific image. In the second step, computational component 110 may use the Hopkins statistic (H) to derive the quantile values ​​(“Q”) for the specific image using a mirror sigmoid function derived from H. In the third step, computational component 110 may use the quantile values ​​(Q) to estimate the kernel bandwidth of the specific image. In the fourth step, computational component 110 may perform mean-shift clustering on the specific image. This mean-shift clustering algorithm may be performed for each reference image within a subset of the image and for the target image.

[0048] Mean-shift clustering performed on each image can determine one or more clusters of the data distribution in each corresponding image. Each cluster in each corresponding image can represent a feature of the corresponding image. Each feature represented by a cluster can be a feature that is important or prominent in the corresponding image. Mean-shift clustering can be performed on each reference image in a subset of images and on the target image. In some embodiments, mean-shift clustering of each image can be performed in series. In other embodiments, mean-shift clustering of each image can be performed in parallel across multiple processors 342 in computing system 300. The ability to perform mean-shift clustering of multiple images in parallel across multiple processors 342 can accelerate the computational performance process of clustering images. The acceleration of the computational performance process of clustering images may further lead to an acceleration of the computational performance process of performing data bias assessment on the target image.

[0049] The computation component 110 can then determine the clusters of data points in each reference image within the target image and subsets of the image. The computation component 110 can group each individual data point in the image into clusters by placing boundaries around each individual cluster to clearly show the data points in the image within each cluster.

[0050] Step 322 of process 300 is similar to block 222 of process 200. In step 322, computation component 110 may perform a data bias assessment on the clusters of the target image. The data bias assessment may be performed by evaluating the distributional differences between the clusters of the reference images and the clusters of the target image. Computation component 110 may compare the clusters in the target image with the clusters in each reference image within a subset of the image and analyze the distributional differences of the clusters between the target image and each individual reference image within the subset of the image. Once the clusters in the target image have been compared and analyzed with the clusters of all reference images within the subset of the image, a data bias can be determined. Data bias may exist if the overall distributional difference between clusters is greater than a consistency threshold. The overall distributional difference between clusters may be a calculated balance of each difference between the clusters of the target image and each reference image within the subset of the image. In some embodiments, the consistency threshold may be a predetermined amount of difference. In other embodiments, the consistency threshold may be a predetermined percentage of the total difference between the clusters of the target image and all reference images within the subset of the image. When data bias assessment results in an overall distribution difference between clusters exceeding a consistency threshold, computation component 110 can determine an inconsistency. An inconsistency indicates the presence of data bias in the target image.

[0051] Step 324 of process 300 is similar to blocks 216 and 224 of process 200. In step 324, computation component 110 may send a message to the user regarding the data bias found in the subset of images and the target image. Computation component 110 may summarize the imbalance assessment performed on the subset of images and the data bias assessment performed on the target image. The summary may include all labels, attributes, and reference image groups used in the imbalance assessment and data bias assessment. The summary may also include the values ​​of variables determined by the performance of mean-shift clustering on the target image and on each reference image in the subset of images. The summary may also include a detailed explanation of the causes of the imbalanced and inconsistent results. Computation component 110 may include the summary of the imbalance assessment and data bias assessment, along with the imbalanced and inconsistent results, in a message. Computation component 110 may then send this message to the user. The user can gain a full understanding of the causes of the imbalance in the subset of images and the inconsistencies in the target image, and can adjust techniques such as the counting of reference data and other factors of the computational system of the ML model or ML pipeline to eliminate any data bias.

[0052] For the sake of brevity, process 300 is described as being performed for a single received target image. It should be understood that, in a typical embodiment, computing component 110 may manage multiple target images in a short-term sequential manner. For example, in some embodiments, computing component 110 may perform many (if not all) of the steps in process 300 on multiple target images when a target image is received.

[0053] As explained, Process 300 offers solutions for improving the detection of potential data biases in the ML training pipeline and achieving more accurate predictions in image evaluation. Process 300 also improves the computational process of the ML training pipeline model by incorporating concurrent pipelined processing across multiple CPU resources. These improvements also reduce manual user inspection, decrease trial-and-error in image segmentation, improve the accuracy of feature detection, and accelerate the computational performance of the solution.

[0054] Figure 4 An example scenario is illustrated, in which process 400 may perform mean-shift clustering on an input image dataset, for example, to determine one or more clusters in each image of the input image dataset. In some embodiments, process 400 may be, for example, by... Figure 1 The computational component 110 performs the operation. In other embodiments, process 400 can be implemented as follows: Figure 1 The computing component 110. The computing component 110 may be, for example... Figure 2 Process 200 Figure 3 Process 300 and Figure 5 The computing component 500. The computing component 110 may include a server.

[0055] Steps 410 and 412 of process 400 are similar to boxes 210 and 212 of process 200. In steps 410 and 412, computation component 110 receives the target image and extracts a subset of reference images from the image database. Computation component 110 may receive the target image from the user before mean-shift clustering can be performed. The target image may include one or more features, attributes, and / or labels. Features may include people, objects, structures, items, and any other things that can be displayed in the image. Attributes may include age, gender, race, ethnicity, religion, income, demographics, material, and any other factors or features that may be directly associated with one or more features in a particular image. Labels may be classifications or descriptions of the entire image. In some embodiments, attributes associated with the target image may be assigned by the user before the target image is input to device 150. In other embodiments, attributes associated with the target image may be determined using one or more algorithms for detecting image attributes after the computation component 110 receives the target image. The target image may be labeled with a classification or description that describes its image type or content displayed in the image. In some embodiments, a tag for the target image may be assigned before the target image is input to the device 150. In other embodiments, the tag for the target image may be determined after the computing component 110 receives the target image by scanning the target image and determining one or more tags to be assigned to the target image based on features displayed in the image.

[0056] In one example, the target image could be an X-ray image showing human features (e.g., lungs). Before the user inputs the target image into device 150, the user can assign attributes to the person whose lungs and heart are shown in the X-ray image. The user could assign the attribute $50K for income and French for ethnicity. After device 150 receives the image, computing component 110 can use one or more algorithms to determine any additional attributes associated with the person whose lungs and heart are shown in the X-ray target image, such as assigning 56 for age, male for sex, and white / Caucasian for race. The target image can also be tagged with a classification "lung" before being input into device 150.

[0057] The computing component 110 can then extract a subset of images from the image database. After receiving the target image and determining one or more attributes associated with it, the computing component can first identify groups or subsets of reference images from a reference image database, wherein the subsets of reference images are selected based on the classification labels of the target image and / or the attributes associated with it. The reference images can be considered as a source image dataset. The reference image database can contain a large number of images, where each image belongs to one or more subsets based on any combination of classification labels and attributes associated with each image. The image database can be stored in database 120. After the subsets of reference images have been identified according to the target image's classification labels(multiple) and / or attributes(multiple), the computing component 110 can extract the subsets of reference images from the reference image database.

[0058] In step 414, computation component 110 calculates the Hopkins statistic for each reference image and target image. After the target image has been received and a subset of the reference images has been identified and extracted, computation component 110 can continue to perform mean-shift clustering on the target image and each reference image in the subset. Mean-shift clustering can include four steps. In the first step, computation component 110 can determine the Hopkins statistic (“H”) for a particular image. The Hopkins statistic can be used to assess the clustering tendency of a dataset in a particular image. Assessing the clustering tendency of a data distribution in an image can test the spatial randomness of the data by measuring the probability that a given dataset is generated from a uniform data distribution. The Hopkins statistic (H) can be a value between 0 and 1. If H is closer to the value 0, the data is uniformly distributed, and there are no meaningful data clusters in the corresponding image. If H is closer to the value 1, the data is not uniformly distributed, and the image contains meaningful data clusters.

[0059] The Hopkins statistic (H) can be expressed by the symbols "n" and "x". i "and "y i The variable composition is "n". The symbol "n" can represent the set of data points from a real dataset in a specific image. The symbol "x" is composed of variables. i "Can represent the distance from each real point to each corresponding nearest real data point neighbor. In one example, if point "p i " is the first point in the real dataset, and its nearest neighbor is point "p". j ", then "x" i "is "p i "and "p j The distance between them. The symbol "y" i " can represent the distance from each artificial data point to its nearest neighbor, where each artificial data point is represented by the symbol "q". iThe artificial data points can be determined from a generated simulated dataset derived from a random uniform distribution of "n" data points. The Hopkins statistic (H) can be represented by the average nearest neighbor distance in the random dataset divided by the sum of the average nearest neighbor distances across the real and simulated datasets. The Hopkins statistic (H) can be composed of the following formula:

[0060]

[0061] In step 416, the computation component 110 derives the quantile values ​​for each reference and target image. After calculating the Hopkins statistic (H), the computation component 110 can proceed to the second step of mean-shift clustering, which determines the quantile values ​​(“Q”) for a specific image. The quantile values ​​(Q) can be automatically determined using a mirrored sigmoid function derived from the Hopkins statistic (H). The quantile values ​​(Q) can consist of variables with the symbols “H”, “a”, and “b”. The symbol “H” can represent the Hopkins statistic. The symbol “a” can be 0.5, representing the normalized value of “H” from 0 to 1. The symbol “b” can be a constant, with a default optimal value of 10 for the sigmoid function, and is tunable to adjust the slope. The quantile value (Q) can then be formulated as: Q = 1 / (1 + exp(b*(Ha))).

[0062] In step 418, the computing component 110 estimates the kernel bandwidth for each reference and target image. After the quantile values ​​(Q) have been determined, the computing component 110 can then proceed to the third step of determining the estimated kernel bandwidth for a particular image. To determine the estimated kernel bandwidth, the computing component 110 can first determine the number of neighbors ("k-NN") by multiplying the number of data points ("Y") in the particular image by the quantile value (Q). After determining the number of neighbors (k-NN), the computing component 110 can use it to determine the estimated kernel bandwidth. The kernel bandwidth can be the average pairwise distance calculated by k-NN between data samples.

[0063] In step 420, computation component 110 performs mean-shift clustering for each reference image and target image. After the estimated kernel bandwidth has been calculated, computation component 110 can proceed to the fourth step of performing mean-shift clustering on a specific image. Mean-shift clustering performed on a specific image can determine one or more clusters of data distribution in the specific image. Each cluster in the specific image can represent a feature of the specific image. Each feature represented by a cluster can be a feature that is important or prominent in the specific image. Mean-shift clustering can be performed on each reference image in a subset of images and on the target image. In some embodiments, mean-shift clustering for each image can be performed in series. In other embodiments, mean-shift clustering for each image can be performed in parallel across multiple processors 342 in computation system 300. The ability to perform mean-shift clustering of multiple images in parallel across multiple processors can accelerate the computational performance process of clustering images. The acceleration of the computational performance process of clustering images may further lead to an acceleration of the computational performance process of performing data bias assessment on the target image.

[0064] Figure 5 A computing component 500 is illustrated, comprising one or more hardware processors 502 and a machine-readable storage medium 504 storing a set of machine-readable / machine-executable instructions that, when executed, cause the hardware processor(s) 502 to perform illustrative methods that reduce computing costs while maintaining network services and performance. It should be understood that, unless otherwise stated, additional, fewer, or alternative steps may be performed in a similar or alternative order or in parallel within the scope of the various examples discussed herein. The computing component 500 may be implemented as... Figure 1 Computing component 110 Figure 2 Process 200 Figure 3 Process 300 and Figure 4 The process is 400. Figure 5 This summarizes and further elaborates on some of the aspects described above.

[0065] In step 506, the hardware processor(s) 502 may execute machine-readable / machine-executable instructions stored in the machine-readable storage medium 504 to receive a target image from the user. The target image may include one or more features, attributes, and / or tags. In some examples, the target image is assigned one or more features, attributes, and / or tags before it is received from the user. In other examples, after the target image has been received from the user, one or more features, attributes, and / or tags identifying the target image are assigned to it.

[0066] In step 508, the (multiple) hardware processors 502 may execute machine-readable / machine-executable instructions stored in the machine-readable storage medium 504 to extract a subset of images from the image database. After receiving the target image and determining one or more attributes and / or tags associated with the target image, a group or subset of reference images may be identified and extracted from the reference image database. The subset of reference images is selected based on the tags and / or attributes associated with the target image.

[0067] In step 510, the (multiple) hardware processors 502 may execute machine-readable / machine-executable instructions stored in machine-readable storage medium 504 to analyze a subset of images to determine an imbalance assessment. After extracting a subset of images from the image database based on the labels and / or attributes of the target image, the subset of images can be analyzed by performing a class imbalance assessment on the subset of images. The class imbalance assessment can indicate whether the subset of images extracted based on the classification labels and / or attributes of the target image contains any potential data bias.

[0068] To perform class imbalance assessment, one or more reference image groups containing classification labels and / or attributes similar to the target image can be identified from a reference image database. After identifying one or more reference image groups, the number of images in each reference image group can be determined. The number of distinct images in each reference image group can then be analyzed and compared to the total number of images in all reference image groups. This analysis and comparison can determine the average number or percentage of images in a particular reference image group across all reference image groups. In some embodiments, the average number or percentage of images can represent an imbalance threshold or minimum number of reference images in each reference image group necessary to obtain accurate predictions in the analysis of the input target image and dataset. In other embodiments, the imbalance threshold can be a predetermined percentage of the total number of reference images across all reference image groups. A subset of reference images that does not contain a number or percentage of reference images equal to or greater than the imbalance threshold can be identified as imbalanced. An imbalanced subset of reference images may introduce potential data bias and provide inaccurate predictions in the ML model.

[0069] In step 512, the (multiple) hardware processors 502 may execute machine-readable / machine-executable instructions stored in the machine-readable storage medium 504 to perform mean-shift clustering on a subset of the images and on each of the target images. The mean-shift clustering performed on each image may determine one or more clusters of the data distribution for each image. Each cluster in the images may represent a feature of the corresponding image. Each feature represented by a cluster may be a significant or prominent feature in the corresponding image.

[0070] The mean-shift clustering algorithm may include four steps. The first step may be to compute the Hopkins statistic (“H”) for a specific image. The second step may be to use the Hopkins statistic (H) to derive the quantile values ​​(“Q”) for the specific image using a mirror sigmoid function derived from H. The third step may be to use the quantile values ​​(Q) to estimate the kernel bandwidth of the specific image. The fourth step may be to perform mean-shift clustering on the specific image. This mean-shift clustering algorithm can be performed for each reference image within a subset of the image, as well as for the target image.

[0071] In step 514, the (multiple) hardware processors 502 may execute machine-readable / machine-executable instructions stored in the machine-readable storage medium 504 to determine a subset of the images and one or more clusters in each of the target images. The computing component 110 may group each individual data point in the image into clusters by placing boundaries around each individual cluster to clearly show the data points in the image within each cluster.

[0072] In step 516, the hardware processor(s) 502 may execute machine-readable / machine-executable instructions stored in the machine-readable storage medium 504 to perform a data bias assessment on one or more clusters of the target image. The data bias assessment can be performed by evaluating the distributional differences between clusters in the reference images and clusters in the target image. Clusters in the target image can be compared to clusters in each reference image within a subset of the image. The comparison of clusters can be analyzed to determine the distributional differences of clusters between the target image and each individual reference image within the subset of the image. Once the clusters in the target image have been compared and analyzed with the clusters in all reference images within the subset of the image, a data bias can be determined. Data bias may exist if the overall distributional difference between clusters is greater than a consistency threshold. When the data bias assessment results in an overall distributional difference between clusters greater than a consistency threshold, the computation component 110 can determine an inconsistency result. An inconsistency result may indicate the presence of data bias in the target image.

[0073] In step 518, the (multiple) hardware processors 502 may execute machine-readable / machine-executable instructions stored in machine-readable storage medium 504 to send a first message about data bias based on the determination of imbalance in an imbalance assessment. Imbalance can be determined when the result of the imbalance assessment is that a subset of reference images does not contain a number or percentage of reference images equal to or greater than an imbalance threshold. A summary of the imbalance assessment can be created and placed in a message. This summary may include detailed information about the images, labels, attributes, and features used in the imbalance assessment. The summary may also include a detailed analysis of the causes of the imbalance results. The message may include a summary and recommendations on how to eliminate data bias and obtain a balanced result in the imbalance assessment.

[0074] In step 520, the (multiple) hardware processors 502 may execute machine-readable / machine-executable instructions stored in machine-readable storage medium 504 to send a second message regarding data bias based on the determination of inconsistency in a data bias assessment. Inconsistency can be determined when the result of the data bias assessment is that the overall distribution difference between clusters is greater than a consistency threshold. A summary of the data bias assessment may be created and placed in a message. The summary may include detailed information about the mean-drift clusters used in the data bias assessment, including images, labels, attributes, features, data clusters, and variables. The summary may also include a detailed analysis of the causes of the inconsistent results. The message may include a summary and recommendations on how to eliminate data bias and obtain consistent results in the data bias assessment.

[0075] Subsequently, (multiple) hardware processors 502 can receive subsequent target images from the user, and repeat the above steps for each received subsequent image until no more target images are received from the user.

[0076] Figure 6 A block diagram of an example computer system in which various examples of the present disclosure may be implemented is shown. Computer system 600 may include a bus 602 or other communication mechanism for transmitting information, and one or more hardware processors 604 coupled to the bus 602 for processing information. The hardware processors 604 may be, for example, one or more general-purpose microprocessors. Computer system 600 may be an example of client-server communication or similar devices.

[0077] Computer system 600 may also include main memory 606, such as random access memory (RAM), cache, and / or other dynamic storage devices, coupled to bus 602, for storing information and instructions to be executed by hardware processor(s) 604. Main memory 606 may also be used to store temporary variables or other intermediate information during the execution of instructions by hardware processor(s) 604. When stored in a storage medium accessible to hardware processor(s) 604, these instructions present computer system 600 as a dedicated machine that can be customized to perform the operations specified in the instructions.

[0078] The computer system 600 may also include a read-only memory (ROM) 608 or other static storage device coupled to the bus 602 for storing static information and instructions of the hardware processor(s) 604. A storage device 610, such as a disk, optical disk, or USB thumb drive (flash drive), may be provided and coupled to the bus 602 for storing information and instructions.

[0079] The computer system 600 may also include at least one network interface 612 coupled to the bus 602, such as a network interface controller module (NIC), a network adapter, or a combination thereof, for connecting the computer system 600 to at least one network.

[0080] Generally, the terms "component," "module," "engine," "system," and "database" used in this document can refer to logic contained in hardware or firmware, or to a collection of software instructions that may have entry and exit points written in programming languages ​​such as Java, C, or C++. Software components or modules can be compiled and linked into executable programs installed in dynamic link libraries, or can be written in interpreted programming languages ​​such as BASIC, Perl, or Python. It should be understood that software components can be invoked from other components or themselves, and / or can be invoked in response to detected events or interrupts. Software components configured to execute on a computing device, such as computing system 600, can be provided on a computer-readable medium such as optical discs, digital video discs, flash drives, magnetic disks, or any other tangible medium, or as digital downloads (and can initially be stored in a compressed or installable format that needs to be installed, decompressed, or decrypted before execution). Such software code can be stored, in part or in whole, on a storage device executing the computing device for execution by the computing device. Software instructions can be embedded in firmware such as EPROM. It should also be understood that hardware components may include connected logic units, such as gates and flip-flops, and / or may include programmable units, such as programmable gate arrays or processors.

[0081] Computer system 600 can implement the techniques or sciences described herein using custom hard-wired logic, one or more ASICs or FPGAs, firmware, and / or program logic, which, in combination with computer system 600, make computer system 600 a special-purpose machine or program it as such. According to one or more examples, the techniques described herein are executed by computer system 600 in response to hardware processor(s) 604 executing one or more sequences of one or more instructions contained in main memory 606. Such instructions may be read into main memory 606 from another storage medium, such as storage device 610. Execution of the instruction sequence contained in main memory 606 causes hardware processor(s) 604 to perform the process steps described herein. In alternative examples, hard-wired circuitry may be used in place of or in combination with software instructions.

[0082] As used herein, the term "non-transient medium" and similar terms refer to any medium that stores data and / or instructions that enable a machine to operate in a particular manner. Such non-transient medium can include non-volatile media and / or volatile media. Non-volatile media can include, for example, optical discs or magnetic disks, such as storage device 610. Volatile media can include dynamic memory, such as main memory 606. Common forms of non-transient media include, for example, floppy disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs and EPROMs, FLASH-EPROMs, NVRAMs, any other memory chips or cassette tapes, and their network versions.

[0083] Non-transient media differ from transmission media, but can be used in conjunction with them. Transmission media can participate in information transmission between non-transient media. For example, transmission media can include coaxial cables, copper wires, and optical fibers, including the cables that make up bus 602. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.

[0084] As used herein, the term “or” may be interpreted as inclusive or exclusive. Furthermore, descriptions of resources, operations, or structures in the singular form should not be construed as excluding the plural form. Conditional language, such as, among others, “may,” “can,” “able,” or “may,” unless specifically stated or otherwise understood in the context in which they are used, is generally intended to convey that some examples include certain features, elements, and / or steps, while other examples do not.

[0085] Unless otherwise expressly stated, the terms and phrases used in this document, and their variations thereof, should be interpreted as open-ended, not restrictive. Adjectives and terms with similar meanings such as “regular,” “traditional,” “normal,” “standard,” “known,” etc., should not be interpreted as limiting the described item to a given time period or items available at a given time, but should be understood to include regular, traditional, normal, or standard techniques that may be available or known at any time now or in the future. In some cases, the appearance of expansive words and phrases such as “one or more,” “at least,” “but not limited to,” or other similar phrases should not be interpreted as an intention or necessity to narrow the scope where such expansive phrases could be used.

Claims

1. A computer-implemented method for a computing system including a server and a database, the method comprising: Receive a target image, wherein the target image includes one or more attributes; Extract a subset of images from the image database based on one or more of the aforementioned attributes; Analyze the subset to determine imbalanced evaluation; Perform mean-shift clustering on each image in the subset and the target image, wherein the mean-shift clustering includes: Calculate the Hopkins statistic for the data points of the first image; The quantile values ​​are determined based on the calculated Hopkins statistic; The kernel bandwidth is determined based on the second number of data points and the determined quantile value; and The mean-shift clustering is performed based on the determined kernel bandwidth; Based on the mean-shift clustering performed, one or more clusters are determined for each image in the subset and for the target image; Data bias evaluation is performed on the one or more clusters of the target image, wherein the data bias evaluation is performed according to a machine learning (ML) algorithm; A first message regarding data bias is sent based on the determination of the imbalance according to the imbalance assessment; and A second message regarding the data deviation is sent based on the determination of inconsistency in the data deviation assessment performed.

2. The computer-implemented method of claim 1, wherein the one or more attributes are assigned to the target image before the target image is received.

3. The computer-implemented method according to claim 1, wherein after receiving the target image, the one or more attributes are determined based on an attribute detection algorithm.

4. The computer-implemented method of claim 1, wherein the determined imbalance assessment comprises: Determine the number of one or more images in one or more image groups in the image database; Determine whether the first quantity of the subset is below an imbalance threshold based on the number of one or more images in the one or more image groups; and If the first quantity of the subset is lower than the imbalance threshold, then the imbalance in the subset is determined.

5. The computer-implemented method of claim 1, wherein the data deviation assessment performed comprises: The one or more clusters in the target image are compared with the one or more clusters in each image of the subset; Determine a first difference between the one or more clusters in the target image and the one or more clusters in the first image of the subset; Determine the total difference, wherein the total difference is a calculated balance for each difference in each image of the images in the subset; as well as If the total difference between the one or more clusters is greater than a consistency threshold, then the inconsistency between the target image and the subset is determined.

6. A computing system, comprising: One or more processors; as well as The memory stores instructions that, when executed by the one or more processors, cause the one or more processors to: Receive a target image, wherein the target image includes one or more attributes; Extract a subset of images from the image database based on one or more of the aforementioned attributes; Analyze the subset to determine imbalanced evaluation; Perform mean-shift clustering on each image in the subset and the target image, wherein the mean-shift clustering includes: Calculate the Hopkins statistic for the data points of the first image; The quantile values ​​are determined based on the calculated Hopkins statistic; The kernel bandwidth is determined based on the second number of data points and the determined quantile value; and The mean-shift clustering is performed based on the determined kernel bandwidth; Based on the mean-shift clustering performed, one or more clusters are determined for each image in the subset and for the target image; Data bias evaluation is performed on the one or more clusters of the target image, wherein the data bias evaluation is performed according to a machine learning (ML) algorithm; A first message regarding data bias is sent based on the determination of the imbalance according to the imbalance assessment; and A second message regarding the data deviation is sent based on the determination of inconsistency in the data deviation assessment performed.

7. The computing system of claim 6, wherein the one or more attributes are assigned to the target image before the target image is received.

8. The computing system of claim 6, wherein after receiving the target image, the one or more attributes are determined based on an attribute detection algorithm.

9. The computing system of claim 6, wherein the determined imbalance assessment comprises: Determine the number of one or more images in one or more image groups in the image database; Determine whether the first quantity of the subset is below an imbalance threshold based on the number of one or more images in the one or more image groups; as well as If the first quantity of the subset is lower than the imbalance threshold, then the imbalance in the subset is determined.

10. The computing system of claim 6, wherein the data bias assessment performed comprises: The one or more clusters in the target image are compared with the one or more clusters in each image of the subset; Determine a first difference between the one or more clusters in the target image and the one or more clusters in the first image of the subset; Determine the total difference, wherein the total difference is a calculated balance for each difference in each image of the images in the subset; as well as If the total difference between the one or more clusters is greater than a consistency threshold, then the inconsistency between the target image and the subset is determined.

11. A non-transient storage medium storing instructions, the instructions, when executed by at least one processor of a computing system, causing the computing system to perform a method, the method comprising: Receive a target image, wherein the target image includes one or more attributes; Extract a subset of images from the image database based on one or more of the aforementioned attributes; Analyze the subset to determine imbalanced evaluation; Perform mean-shift clustering on each image in the subset and the target image, wherein the mean-shift clustering includes: Calculate the Hopkins statistic for the data points of the first image; The quantile values ​​are determined based on the calculated Hopkins statistic; The kernel bandwidth is determined based on the second number of data points and the determined quantile value; and The mean-shift clustering is performed based on the determined kernel bandwidth; Based on the mean-shift clustering performed, one or more clusters are determined for each image in the subset and for the target image; Data bias evaluation is performed on the one or more clusters of the target image, wherein the data bias evaluation is performed according to a machine learning (ML) algorithm; A first message regarding data bias is sent based on the determination of the imbalance according to the imbalance assessment; and A second message regarding the data deviation is sent based on the determination of inconsistency in the data deviation assessment performed.

12. The non-transient storage medium of claim 11, wherein the one or more attributes are assigned to the target image before the target image is received.

13. The non-transient storage medium according to claim 11, wherein, After receiving the target image, one or more attributes are determined based on an attribute detection algorithm.

14. The non-transient storage medium of claim 11, wherein the determined imbalance assessment comprises: Determine the number of one or more images in one or more image groups in the image database; Determine whether the first quantity of the subset is below an imbalance threshold based on the number of one or more images in the one or more image groups; as well as If the first quantity of the subset is lower than the imbalance threshold, then the imbalance in the subset is determined.

15. The non-transient storage medium of claim 11, wherein the data deviation assessment performed includes: The one or more clusters in the target image are compared with the one or more clusters in each image of the subset; Determine a first difference between the one or more clusters in the target image and the one or more clusters in the first image of the subset; Determine the total difference, wherein the total difference is a calculated balance for each difference in each image of the images in the subset; as well as If the total difference between the one or more clusters is greater than a consistency threshold, then the inconsistency between the target image and the subset is determined.

Images