Data classification methods for classifying inlier and outlier data

Abstract

To provide a data classification method for classifying unlabeled images into inlier and outlier data.SOLUTION: A data classification method for classifying inlier and outlier data includes the steps of: obtaining a plurality of unlabeled images; selecting a designated inlier image among the unlabeled images; computing a similarity matrix including first similarity scores of the unlabeled images relative to the designated inlier image; and classifying each of the unlabeled images into an inlier data set or an outlier data set based on the similarity matrix, so as to generate a plurality of pieces of inlier-outlier predictive data of the unlabeled images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to classification methods, and more particularly to classification methods for classifying unmarked images into inlier or outlier data. [Background technology]

[0002] Outlier detection in machine learning techniques is the process of identifying data instances in a dataset that deviate significantly from a normal distribution. Outlier detection is of great importance in a variety of applications, such as medical prediction, fraud detection, network security, quality control, and anomaly detection in healthcare or industrial processes. Summary of the Invention

[0003] an aggregation step of aggregating the intermediate inlier-outlier prediction data of the unmarked images generated in each of the Q prediction rounds to select a plurality of aggregate predicted inlier images from the unmarked images; an aggregation step of a second similarity matrix including a plurality of second similarity scores for the aggregate predicted inlier images of the unmarked images; and an aggregation step of classifying each of the unmarked images into an inlier dataset or an outlier dataset based on the second similarity matrix to generate a plurality of inlier-outlier prediction data of the unmarked images.

[0004] Another embodiment of the present disclosure provides a data classification method, including the steps of obtaining a plurality of unmarked images; selecting designated inlier images from the unmarked images; calculating a similarity matrix including a plurality of first similarity scores of the unmarked images to the designated inlier images; and classifying each of the unmarked images into an inlier dataset or an outlier dataset based on the similarity matrix, to generate a plurality of inlier-outlier predicted data for the unmarked images.

[0005] Another embodiment of the present disclosure includes the steps of: obtaining a plurality of unmarked images; performing Q prediction rounds on the unmarked images, where Q is a positive integer, each of the Q prediction rounds including: randomly selecting a plurality of default inlier images from among the unmarked images; calculating a first similarity matrix including a plurality of first similarity scores for the unmarked images to the default inlier images; and generating a plurality of intermediate inlier-outlier prediction data for the unmarked images in one of the prediction rounds based on the first similarity matrix; aggregating the intermediate inlier-outlier prediction data for the unmarked images generated in each of the Q prediction rounds to select a plurality of aggregate predicted inlier images from among the unmarked images; and calculating a plurality of second class predictions for the aggregate predicted inlier images of the unmarked images. The present invention provides a data classification method, comprising: calculating a second similarity matrix including similarity scores; classifying each of the unmarked images into an inlier dataset or an outlier dataset based on the second similarity matrix to generate a plurality of first inlier-outlier predicted data for the unmarked images; displaying a portion of the first inlier-outlier predicted data for the unmarked images; obtaining adjusted input data modified based on the first inlier-outlier predicted data; calculating a third similarity matrix including a plurality of third similarity scores for the adjusted input data for the unmarked images; and classifying each of the unmarked images into the inlier dataset or the outlier dataset based on the third similarity matrix to generate a plurality of second inlier-outlier predicted data for the unmarked images.

[0006] It is to be understood that the foregoing general description and the following specific description are exemplary only and are intended to provide further interpretation of the present disclosure. [Brief explanation of the drawings]

[0007] A more complete understanding of the present disclosure can be obtained by reading the following detailed description of the embodiments and by referring to the drawings, in which: [Figure 1] 1 is a flowchart illustrating a method for classifying data according to some embodiments of the present disclosure. [Figure 2] 2 is a schematic diagram illustrating an electronic device for implementing the data classification method of FIG. 1 according to some embodiments of the present disclosure. [Figure 3] FIG. 1 is a schematic diagram illustrating an illustrative example of an unmarked image. [Figure 4] FIG. 10 is a schematic diagram showing manual input marks in a portion of an image. [Figure 5A] FIG. 10 is a schematic diagram illustrating a similarity matrix in an illustrative example. [Figure 5B] FIG. 1 is a schematic diagram showing how a similarity matrix in an illustrative example is actually created to generate inlier-outlier prediction data. [Figure 5C] FIG. 1 is a schematic diagram showing how a similarity matrix in an illustrative example is actually created to generate inlier-outlier prediction data. [Figure 6] FIG. 1 is a schematic diagram illustrating the display of inlier-outlier prediction data according to some embodiments. [Figure 7] 1 is a flowchart illustrating another method for classifying data according to some embodiments of the present disclosure. [Figure 8] FIG. 1 is a schematic diagram illustrating aggregate prediction data for all prediction rounds according to some embodiments of the present disclosure. [Figure 9] 1 is a flowchart illustrating a method for classifying data according to some embodiments of the present disclosure. [Figure 10] FIG. 10 is a schematic diagram illustrating forecast data and adjustment input data of a portion of first inlier-outlier forecast data displayed on a display according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0008] Reference will now be made in detail to embodiments of the present disclosure, examples of which are illustrated in the drawings. Where possible, the same numbers are used in the drawings and the description to refer to the same or like parts.

[0009] Please refer to Figure 1, which shows a flowchart of a data classification method 100 according to some embodiments of the present disclosure. The data classification method 100 classifies unmarked images into an inlier dataset or an outlier dataset, and generates inlier-outlier prediction data for unmarked images in the dataset.

[0010] In the best case scenario, a machine learning model is trained based on a training dataset that is free of outliers. The inclusion of outliers in a training dataset can have various effects on the performance and behavior of a machine learning model. For example, outliers in a training dataset can cause problems such as model deviation, increased model complexity, overfitting, reduced stability, and difficulty in anomaly detection.

[0011] In the medical and healthcare application field, the unmarked images may include different types of medical examination images, such as chest X-ray images, brain MRI images, and abdominal ultrasound images. Different medical examination images have different applications in different diagnoses.

[0012] For example, chest X-ray images are advantageous for training a machine learning model to detect pneumonia, but brain MRI images and abdominal ultrasound images are not suitable for pneumonia detection. The training dataset includes brain MRI images and abdominal ultrasound images, which may adversely affect the training of the pneumonia detection model. In this case, chest X-ray images should be considered inliers, and brain MRI images and abdominal ultrasound images should be considered outliers.

[0013] Manually marking inliers and outliers in a dataset can be a time-consuming and costly process, especially when dealing with large datasets. The method requires human expert review to identify inliers and outliers, which may be impractical for large datasets.

[0014] In some embodiments, the data classification method 100 generates inlier-outlier prediction data for unmarked images in a dataset in a simpler manner. Please further refer to FIG. 2, which shows a schematic diagram of an electronic device 200 for implementing the data classification method 100 of FIG. 1 according to some embodiments of the present disclosure. As shown in FIG. 2, the electronic device 200 includes an input interface 220, a processing unit 240, a storage unit 260, and a display 280. In some embodiments, the electronic device 200 may be a computer, a smartphone, a tablet computer, an image processing server, a data server, or any equivalent image processing device.

[0015] The input interface 220 is used to receive the unmarked image ULIMG and other manual instructions. In some embodiments, the electronic device 200 can classify the unmarked image ULIMG and then display the classification result (i.e., inlier-outlier prediction data PRED for the unmarked image ULIMG) on the display 280. The input interface 220 may include a data transmission interface, a wireless communication circuit, a keyboard, a mouse, a microphone, or any equivalent input device. The processing unit 240 is connected to the input interface 220, the storage unit 260, and the display 280. The storage unit 260 is used to store program code. The program code stored in the storage unit 260 is used to instruct the processing unit 240 to perform the data classification method 100 shown in FIG. 1. In some embodiments, the processing unit 240 may be a processor, a graphics processor, an application-specific integrated circuit, or any equivalent processing circuit.

[0016] Please also refer to Figure 3, which shows a schematic diagram of an illustrative example of an unmarked image ULIMG.

[0017] 1 and 2, processing unit 240 performs step S110 to obtain unmarked images ULIMG from an external source (e.g., a hospital's medical image server) via input interface 220. In other embodiments, unmarked images ULIMG may be stored in storage unit 260, and processing unit 240 can obtain unmarked images ULIMG from storage unit 260. As shown in FIG. 3, unmarked images ULIMG include six images IMG1-IMG6, which in this illustrative example are a combination of four chest X-ray images IMG1-IMG4 and two brain MRI images IMG5-IMG6.

[0018] It should be noted that the six images IMG1-IMG6 of the unmarked image ULIMG shown in Figure 3 are for ease of illustration, but the present disclosure is not limited thereto. In actual applications, the unmarked image ULIMG may include hundreds, thousands, or even more images.

[0019] 1, 2, and 3, the processing unit 240 performs step S112 to preprocess the unmarked image ULIMG. The images IMG1-IMG6 in the unmarked image ULIMG may be acquired by various inspection devices and stored in various file formats. In the preprocessing process of step S112, the processing unit 240 performs pixel value windowing, intensity scaling, size scaling, and / or normalization on the unmarked image ULIMG to convert the multiple unmarked images ULIMG into the same format, the same size ratio, and / or the same color ratio.

[0020] 1, 2, and 3, the processing unit 240 performs step S114 after preprocessing to extract corresponding feature vectors from the preprocessed unmarked images ULIMG. In some embodiments, the processing unit 240 may operate a neural network model (e.g., a contrastive language-image pre-trained neural network or a residual network) to extract the feature vectors. One feature vector is extracted from each of the images IMG1-IMG6 in the unmarked images ULIMG.

[0021] Meanwhile, step S116 is performed to collect several manually input marks MLB via the input interface 220. In some embodiments, a user may manually assign manually input marks MLB corresponding to images IMG1-IMG6 in the unmarked image ULIMG. Further, refer to FIG. 4, which shows a schematic diagram of manually input marks MLB in some images IMG1-IMG6 in some embodiments. In the illustrative example shown in FIG. 4, it is assumed that a user inputs manually input marks MLB to designate one image IMG1 as an "inlier" and another image IMG5 as an "outlier." In response to the manually input marks MLB collected by the user, the processing unit 240 performs step S120 to select image IMG1 as a designated inlier image INL based on the manually input marks MLB in the unmarked image ULIMG, and select image IMG5 as a designated outlier image OUTL based on the manually input marks MLB in the unmarked image ULIMG.

[0022] In this case, the user provides manual input marks MLB for some images in the unmarked images ULIMG (e.g., two images IMG1 and IMG5). The remaining four images IMG2, IMG3, IMG4, and IMG6 are not yet marked. The data classification method 100 shown in FIG. 1 is used to generate inlier-outlier prediction data for the unmarked images IMG2, IMG3, IMG4, and IMG6 based on a small number of manually marked images (e.g., designated inlier images INL and designated outlier images OUTL). The present disclosure is not limited to the number of unmarked images and manually input marks herein. In a practical application, for example, the unmarked images ULIMG may include 5,000 images, and the user may manually mark five images, and the data classification method 100 is used to generate inlier-outlier prediction data PRED for the other 4,995 images.

[0023] In the above embodiment, the manual input mark MLB includes an inlier mark and an outlier mark to select the designated inlier image INL and the designated outlier image OUTL, however, the present disclosure is not limited thereto.

[0024] In another embodiment, the manually input marks MLB include one, two, three or more inlier marks to select at least one designated inlier image INL. The manually input marks MLB include zero, one, two, three or more outlier marks. In other words, a designated outlier image OUTL is not necessarily required to generate the inlier-outlier prediction data PRED.

[0025] 1, 2, and 4, processing unit 240 executes step S130 to calculate a similarity matrix between the unmarked image ULIMG and the manually marked images (e.g., designated inlier images INL and designated outlier images OUTL). See also FIG. 5A, which shows a schematic diagram of a similarity matrix SMX1 in an illustrative example.

[0026] 5A, the similarity matrix SMX1 includes a first similarity score SSc1 for an unmarked image (e.g., IMG1-IMG6) with respect to a designated inlier image INL, as shown in the first column of the similarity matrix SMX1. Also shown in FIG. 5A, the similarity matrix SMX1 includes a second similarity score SSc2 for an unmarked image (e.g., IMG1-IMG6) with respect to a designated outlier image OUTL.

[0027] In some embodiments, the processing unit 240 is used to calculate a first similarity score SSc1 by performing a similarity algorithm between the feature vectors extracted from the unmarked images (e.g., IMG1-IMG6) and the designated inlier image INL. The similarity algorithm may be selected from a cosine similarity algorithm, a Euclidean distance similarity algorithm, a Manhattan distance algorithm, or a Hamming distance algorithm.

[0028] The processing unit 240 may execute a cosine similarity algorithm to calculate the first similarity score SSc1 in the cosine similarity equation.

number

[0029] In the above equation (1), A and B are the feature vectors of the two images to be compared.

[0030] For example, the processing unit 240 may execute a cosine similarity algorithm to calculate a similarity score SS21 between the image IMG2 and the designated inlier image INL (i.e., image IMG1) in terms of a cosine similarity equation.

number

[0031] In the above equation (2), V1 is the feature vector of image IMG1, and V2 is the feature vector of image IMG2. Similarly, other similarity scores in the first similarity score SSc1 can be calculated based on the similarity algorithm.

[0032] If the feature vectors of the two images IMG2 and IMG1 are similar to each other, the similarity score SS21 approaches 1. In this case, since images IMG2 and IMG1 are similar to each other, the similarity score SS21 is 0.92, which is close to 1. On the other hand, if the feature vectors of the two images are not similar to each other, the similarity score approaches 0.

[0033] Similarly, processing unit 240 may execute a cosine similarity algorithm to calculate a second similarity score SSc2 between images IMG1-IMG6 and the designated outlier image OUTL (i.e., image IMG5).

[0034] For example, processing unit 240 may execute a cosine similarity algorithm to calculate a similarity score SS15 between image IMGl and the designated outlier image OUTL (i.e., image IMG5) in a cosine similarity equation.

number

[0035] In the above equation (3), V1 is the feature vector of image IMG1, and V5 is the feature vector of image IMG5. In this case, images IMG1 and IMG5 are not similar to each other, so the similarity score SS15 is 0.53, which is not close to 1.

[0036] As shown in Figures 1, 2, and 5A, the processing unit 240 performs step S140 to classify each unmarked image ULIMG into an inlier data set or an outlier data set based on the similarity matrix SMX1, thereby generating inlier-outlier predicted data PRED for the unmarked image ULIMG. Please also refer to Figures 5B and 5B, which are schematic diagrams showing how the similarity matrix SMX1 in an illustrative example is actually generated to generate inlier-outlier predicted data. In some embodiments, step S140 further includes three specific steps S141-S143.

[0037] 5B, processing unit 240 performs step S141 to select the maximum value along each row of similarity matrix SMX1 to generate another matrix SMX1a. Next, processing unit 240 performs step S142 to multiply matrix SMX1a by weighting matrix WMX to generate another matrix SMX1b. The weighting values ​​in weighting matrix WMX are determined by determining whether the maximum value selected in step S141 corresponds to the designated inlier image INL or the designated outlier image OUTL. As shown in matrix SMX1b, the similarity scores corresponding to images IMG5 and IMG6 are converted to negative values ​​in step S142.

[0038] 1, 2, and 5C, processing unit 240 performs step S143 to predict inlier-outlier prediction data for images IMG1-IMG6 by comparing the similarity scores in matrix SMX1b (corresponding to images IMG1-IMG6) with a threshold similarity value, where the default value of the threshold similarity value is assumed to be 0.93.

[0039] As shown in FIG. 5C, image IMG3 has a similarity score of "0.94" that exceeds the threshold similarity value of "0.93" and is therefore predicted as an inlier in the inlier-outlier prediction data PRED.

[0040] As shown in FIG. 5C, images IMG2, IMG4, IMG5, and IMG6 have corresponding similarity scores of “0.92,” “0.91,” “−1.0,” and “−0.86,” all of which are lower than the threshold similarity value of “0.93,” and therefore images IMG2, IMG4, IMG5, and IMG6 are predicted as outliers in the inlier-outlier prediction data PRED.

[0041] In the example shown in Figures 5A-5C above, the first similarity score SSc1 and the second similarity score SSc2 in the similarity matrix SMX1 are processed to form a matrix SMX1b, which is used to generate inlier-outlier prediction data PRED.

[0042] In some other embodiments, the designated outlier image OUTL is not necessarily required to generate the inlier-outlier prediction data PRED. When only the designated inlier image INL but not the designated outlier image OUTL is selected in the manual input mark, the processing unit 240 may perform step S140, in which the processing unit 240 classifies each unmarked image ULIMG by comparing the first similarity score SSc1 of the unmarked images (e.g., IMG1-IMG6) in FIG. 5A with the threshold similarity value “0.93,” and generates the inlier-outlier prediction data PRED based on the first similarity score SSc1 without considering the second similarity score SSc2.

[0043] Based on the above embodiment, the data classification method 100 shown in Figure 1 includes inlier-outlier prediction data PRED for all unmarked images generated based on a small number of manually marked images (i.e., designated inlier images INL and designated outlier images OUTL). In other words, the data classification method 100 can save a large amount of time and labor costs in marking inliers and outliers in a large dataset.

[0044] As shown in Figures 1 and 2, step S160 is used to display the inlier-outlier prediction data PRED on the display 280. Please refer to Figure 6, which shows a schematic diagram of the inlier-outlier prediction data PRED displayed on the display 280 according to some embodiments. In this case, the user can check the inlier-outlier prediction data PRED of the images IMG1-IMG6. If the inlier-outlier prediction data PRED is not ideal for the user, the user can provide feedback to adjust the classification parameters.

[0045] In some embodiments, if a user checks the inlier-outlier prediction data PRED and finds that the current threshold similarity value is not ideal, the data classification method 100 can perform step S154 to adjust the threshold similarity value based on a feedback instruction input via the input interface 220. For example, the threshold similarity value may be adjusted lower to a threshold similarity value of "0.90." As shown in FIGS. 1, 2, and 5A, the processing unit 240 again performs step S140 to reclassify each of the unmarked images ULIMG with reference to the adjusted similarity threshold of "0.90." In this case, images IMG1-IMG4 are classified as "inliers" based on the adjusted threshold similarity value of "0.90." In other words, the similarity threshold can be adjusted higher or lower according to a feedback instruction.

[0046] In some other embodiments, if a user checks the inlier-outlier prediction data PRED and finds that the manually entered marks are not ideal, the data classification method 100 can perform step S156 to collect adjusted manually entered marks based on feedback instructions entered via the input interface 220. For example, the user may manually designate images IMG1 and IMG4 as "inliers" and remove the "outlier" marks from image IMG5. As shown in FIGS. 1, 2, and 5A, the processing unit 240 again performs step S120 to reselect images IMG1 and IMG4 as designated inlier images INL. Step S130 is again performed to recalculate a similarity matrix (not shown) based on the designated inlier images INL, this time including images IMG1 and IMG4. The similarity matrix includes two columns of similarity scores for unmarked images IMG1-IMG6 to the designated inlier images IMG1 and IMG4. One column of similarity scores relates unmarked images IMG1-IMG6 to designated inlier image IMG1, and the other column of similarity scores relates unmarked images IMG1-IMG6 to designated inlier image IMG4. Step S140 is performed again to reclassify the unmarked images IMG1-IMG6 based on the newly calculated similarity matrix. In this case, if the initial set of manually entered marks is not ideal, the user may modify the manually entered marks in step S156 and generate new inlier-outlier prediction data PRED accordingly.

[0047] In the data classification method 100 shown in FIG. 1 in the above embodiment, classification is performed using a small amount of manually input marks. However, the present disclosure is not limited thereto. Please further refer to FIG. 7, which shows a flowchart of another data classification method 300 according to some embodiments of the present disclosure. The data classification method 300 in FIG. 7 may be performed by the electronic device 200 shown in FIG. 2. Compared with the data classification method 100 in FIG. 1, the data classification method 300 shown in FIG. 7 does not require collecting any manually input marks from the user.

[0048] As in some embodiments shown in FIGS. 2 and 7 , the processing unit 240 performs step S310 to obtain an unmarked image ULIMG (see the embodiment in FIG. 3 ). The specific content of step S310 is similar to step S110 in the aforementioned embodiments, and therefore will not be described again. The processing unit 240 performs step S312 to preprocess the unmarked image. In the preprocessing process of step S312, the processing unit 240 performs pixel value windowing, intensity scaling, size scaling, and / or normalization on the unmarked image ULIMG to convert multiple unmarked images ULIMG into the same format, the same size ratio, and / or the same color ratio. The specific content of step S312 is similar to step S112 in the aforementioned embodiments, and therefore will not be described again. The processing unit 240 performs step S314 to extract a feature vector from the unmarked image ULIMG. In some embodiments, the processing unit 240 may operate a neural network model (e.g., a contrastive language-image pre-training (CLIP) neural network or a residual neural network (ResNet)) to extract the feature vector. The specific content of step S314 is similar to step S114 in the above-described embodiments, and therefore will not be described here.

[0049] After step S314, the data classification method 300 is used to perform Q prediction rounds R1, R2...RQ on the unmarked image ULIMG to generate intermediate inlier-outlier prediction data in each of the Q prediction rounds R1, R2...RQ, where Q is a positive integer.

[0050] During the prediction round R1, the processing unit 240 executes step S320 to randomly select multiple default inlier images from the unmarked images ULIMG. These default inlier images are images randomly sampled from the unmarked images ULIMG and are considered "inliers" in the current prediction round R1. In practical applications, the unmarked images ULIMG in a dataset typically contain relatively more inlier data and relatively less outlier data (e.g., the ratio of inliers to outliers may be 5:1 or 10:1). Therefore, among these default inlier images randomly sampled from the unmarked images ULIMG, actual inlier data is more likely to be selected and actual outlier data is less likely to be selected.

[0051] Then, in prediction round R1, processing unit 240 executes step S330 to calculate a first similarity matrix including a plurality of first similarity scores for these default inlier images of the unmarked image ULIMG. The details of step S330 are similar to step S130 in the previous embodiment, and therefore will not be described again. The difference between step S330 in FIG. 7 and step S130 in FIG. 1 is that the default inlier images used in calculating the first similarity matrix are based on those randomly selected in step S320, rather than on manually input marks.

[0052] Then, in prediction round R1, processing unit 240 performs step S340 to classify each of the unmarked images ULIMG based on the first similarity matrix, and generates intermediate inlier-outlier prediction data for the unmarked images in prediction round R1. The details of step S340 are similar to step S140 in the above-described embodiment, and therefore will not be described again here.

[0053] 7, step S342 is used to check whether Q prediction rounds R1-RQ have been completed. In this case, since they have not been completed, the data classification method 300 returns to step S320 and proceeds to another round of prediction R2. During prediction round R2, the processing unit 240 performs steps S320-S340 to select another set of default inlier images and generate intermediate inlier-outlier prediction data accordingly. Similarly, steps S320-S340 are repeatedly performed in subsequent prediction rounds until prediction round RQ has been completed.

[0054] After the Q prediction rounds R1-RQ are completed, the data classification method 300 proceeds to step S350, where the processing unit 240 executes step S350 to aggregate the intermediate outlier prediction data of the Q prediction rounds R1-RQ to select inlier images for the aggregate prediction. See also FIG. 8 , which is the aggregate prediction data PRED of all prediction rounds according to some embodiments of the present disclosure. ALL As shown in FIG. 8, the set prediction data PRED ALL is the intermediate inlier-outlier prediction data PRED from prediction round R1 R1 , the intermediate inlier-outlier prediction data PRED from prediction round R2 R2 , ..., and the intermediate inlier-outlier prediction data PRED from the prediction round RQ RQ Includes.

[0055] 8, each of the images IMG1-IMG6 is classified as an "inlier" or an "outlier" in each of the prediction rounds R1-RQ. In some embodiments, in step S350, only unmarked images that are classified as inliers in all Q prediction rounds are selected as aggregate predicted inlier images.

[0056] As shown in Figure 8, images IMG1 and IMG3 are all intermediate inlier-outlier predicted data PRED R1 -PREDRQ Since the images IMG1 and IMG3 are classified as "inliers" in , the images IMG1 and IMG3 are selected as the collective predicted inlier images INLa.

[0057] On the other hand, image IMG2 is the intermediate inlier-outlier prediction data PRED R1 Similarly, image IMG4 is classified as an "outlier" in the intermediate inlier-outlier prediction data PRED R2 Image IMG2 is disqualified because it is classified as an "outlier" in

[0058] In some embodiments, Q is a positive integer between about 10 and 20. If Q is less than 10, the ensemble predicted inlier images may not be accurate enough (e.g., an actual outlier image may be accidentally combined with the ensemble predicted inlier image). If Q is greater than 20, it may be too strict and difficult to select the ensemble predicted inlier images.

[0059] 2, 7, and 8, the processing unit 240 executes step S360 to calculate a second similarity matrix including first similarity scores of the unmarked image ULIMG to the aggregate predicted inlier image INLa. The details of step S360 are similar to step S130 in the previous embodiment, and therefore will not be described again. The difference between step S360 in FIG. 7 and step S130 in FIG. 1 is that the aggregate predicted inlier image INLa used in calculating the second similarity matrix is ​​based on the content of step S350, rather than on manually input marks.

[0060] 2 and 7, the processing unit 240 performs step S370 to classify each of the unmarked images ULIMG based on the second similarity matrix, and generate inlier-outlier prediction data for the unmarked images ULIMG. The details of step S370 are the same as step S140 in the previous embodiment, and therefore will not be described again.

[0061] In some embodiments, step S380 is performed to display the inlier-outlier prediction data (see FIG. 6) generated in step S370 to a user.

[0062] In the embodiment shown in FIG. 7 , the data classification method 300 does not need to collect any manually input marks from a user. The data classification method 300 provides an alternative method, generating inlier-outlier prediction data for the unmarked image ULIMG based on random sampling in prediction rounds R1-RQ, without requiring any manually input marks. The data classification method 300 can be executed automatically without waiting for manually input marks. In some cases, the data classification method 300 may be inaccurate because it may select actual outlier data as default inliers after random sampling.

[0063] In some other embodiments, the present disclosure provides a method that combines the data classification method 100 shown in FIG. 1 and the data classification method 300 shown in FIG. 7. Please refer further to FIG. 9, which shows a flowchart of a data classification method 500 according to some embodiments of the present disclosure. Steps S510, S512, S514, S520, S530, S540, S542, S550, S560, and S570 of the data classification method 500 in FIG. 9 are similar to steps S310, S312, S314, S320, S330, S340, S342, S350, S360, and S370 of the data classification method 300 in FIG. 7, and therefore will not be described again.

[0064] As described in the previous embodiments, step S570 is used to generate first inlier-outlier prediction data for the unmarked image ULIMG. Step S580 is used to display a portion of the first inlier-outlier prediction data for the unmarked image ULIMG. In practical applications, first outlier prediction data may be generated from thousands of unmarked images ULIMG. Step S580 is used to display a relatively small amount of first inlier-outlier prediction data on display 280. Please also refer to FIG. 10, which shows a schematic diagram of prediction data PREDp1 and adjustment input data ADJ for a portion of the first inlier-outlier prediction data displayed on display 280 according to some embodiments.

[0065] In some embodiments, the first outlier prediction data is generated automatically without requiring manual input. The first outlier prediction data may include some erroneous prediction data. A user may review the first outlier prediction data and provide adjustment input data corresponding to the first outlier prediction data. As shown in the embodiment of FIG. 10 , the adjustment input data indicates that some of the prediction data for images IMG4 and IMG6 are incorrect, and some of the prediction data for images IMG3 and IMG5 are correct.

[0066] 2 and 10, step S590 is used to collect adjustment input data ADJ via the input interface 220. Adjusted manual input marks LBadj corresponding to images IMG3-IMG6 are formed based on the adjustment input data ADJ, as shown in FIG.

[0067] In response to the adjusted input data ADJ (and the adjusted manual input mark LBadj), processing unit 240 executes step S591 to select images IMG3 and IMG4 from among the unmarked images ULIMG as designated inlier images INL based on the adjusted input data ADJ, and select images IMG5 and IMG6 from among the unmarked images ULIMG as designated outlier images OUTL based on the adjusted input data ADJ.

[0068] 2 and 9, the processing unit 240 executes step S592 to calculate a third similarity matrix including third similarity scores of the unmarked image ULIMG with respect to the designated inlier image INL and the designated outlier image OUTL based on the adjusted input data ADJ. The details of step S592 in Fig. 9 are the same as step S130 in Fig. 1 described in the above embodiment, and therefore will not be described again.

[0069] 2 and 9, the processing unit 240 performs step S593 to classify each unmarked image ULIMG into an inlier dataset or an outlier dataset based on the third similarity matrix, thereby generating second inlier-outlier predicted data for the unmarked image ULIMG. For details of step S593 in FIG. 6, see step S593 in FIG. 3. Details of step S593 in FIG. 9 are similar to step S140 in FIG. 1 described in the above embodiment, and therefore will not be described again.

[0070] As shown in FIGS. 2 and 9, step S594 is used to display the second inlier-outlier prediction data on display 280.

[0071] In some embodiments, the user can check the second inlier-outlier prediction data on the display 280. If the second inlier-outlier prediction data is incorrect, the user can again provide another adjustment input data, and the data classification method 500 can again repeat steps S590-S594.

[0072] The data classification method 500 in Fig. 9 is based on a combination method of the data classification method 100 shown in Fig. 1 and the data classification method 300 shown in Fig. 7. The data classification method 500 can realize the advantages of time efficiency and accuracy when classifying the unmarked image ULIMG into an inlier dataset or an outlier dataset.

[0073] In some embodiments, the inlier dataset is used as training data for training the machine learning model. The outlier dataset is filtered and not used as training data. In this case, the outlier dataset does not affect the training process of the machine learning model. In this way, problems that occur in the training process of the machine learning model, such as model deviation, increased model complexity, overfitting, reduced stability, and difficulty in anomaly detection, can be avoided.

[0074] Although the present disclosure has been described in considerable detail with reference to exemplary embodiments, other embodiments may be practiced, and therefore, the spirit and scope of the claims of the present disclosure should not be limited to the description of the exemplary embodiments contained herein.

[0075] It will be apparent to those skilled in the art that various modifications and equivalent changes can be made to the structure of the present disclosure without departing from the scope or spirit of the disclosure. Accordingly, all modifications and equivalent changes made to the present disclosure that come within the scope of the following claims are within the scope of the present disclosure. [Explanation of symbols]

[0076] 100, 300, 500 data classification methods S110, S112, S114, S116, S120, S130, S140, S141, S142, S150, S152, S154, S310, S312, S314, S320, S330, S340, S342, S350, S360, S370, S380, S510, S512, S514, S520, S530, S540, S542, S550, S560, S570, S580, S590, S591, S592, S593, S594 steps 200 Electronic equipment 220 Input Interface 240 processing units 260 Memory Unit 280 display ULIMG Unmarked Image IMG1, IMG2, IMG3, IMG4, IMG5, IMG6 images MLB Manual Entry Mark INL Designated Inlier Image OUTL Designated Outlier Image PRED forecast data SMX1, SMX1a, SMX1b similarity matrix SS15, SS21 similarity scores SSc1 First similarity score SSc2 Second similarity score WMX Weighting Matrix R1, R2…RQ prediction rounds INLa Ensemble predicted inlier images PRED R1 , PRED R2 , PRED RQ Intermediate Inlier-Outlier Forecast Data PRED ALL Collective forecast data PREDp1 Part of the prediction data ADJ Adjustment input data Manual input mark after LBadj adjustment

Claims

1. acquiring a plurality of unmarked images; performing Q prediction rounds on the unmarked image, Q being a positive integer, each of the Q prediction rounds comprising: randomly selecting a plurality of default inlier images from among the unmarked image; calculating a first similarity matrix including a plurality of first similarity scores of the unmarked image to the default inlier images; and generating a plurality of intermediate inlier-outlier prediction data for the unmarked image in one of the prediction rounds based on the first similarity matrix; aggregating the intermediate inlier-outlier prediction data for the unmarked images generated in each of the Q prediction rounds and selecting a plurality of aggregate predicted inlier images from among the unmarked images; calculating a second similarity matrix comprising a plurality of second similarity scores of the unmarked image to the collective predicted inlier image; classifying each of the unmarked images into an inlier data set or an outlier data set based on the second similarity matrix to generate a plurality of inlier-outlier predictions for the unmarked images; Data classification methods, including:

2. The intermediate inlier-outlier prediction data for the unmarked image is 2. The method of claim 1, wherein in all the Q prediction rounds, a portion of the unmarked images classified as inliers is aggregated by selecting them as the aggregate predicted inlier images.

3. The method of claim 1 , wherein the first similarity score is calculated by running a similarity algorithm between a plurality of feature vectors obtained from the unmarked image and the default inlier image.

4. 4. The data classification method according to claim 3, wherein the similarity algorithm is a cosine similarity algorithm, a Euclidean distance similarity algorithm, a Manhattan distance algorithm, or a Hamming distance algorithm.

5. The intermediate inlier-outlier prediction data is comparing the first similarity score of the unmarked image with a threshold similarity value; predicting one of the unmarked images as an inlier in the intermediate inlier-outlier prediction data in response to the first similarity score in one of the unmarked images being greater than the threshold similarity value; predicting another of the unmarked images as an outlier in the intermediate inlier-outlier prediction data in response to the first similarity score in the other of the unmarked images being less than the threshold similarity value; The data classification method of claim 1 , wherein the data is generated by:

6. The method of claim 1 , wherein the inlier dataset is used as training data for training a machine learning model, and the outlier dataset is not used as the training data.

7. 2. The data classification method according to claim 1, wherein Q is in the range of 10 to 20.

8. acquiring a plurality of unmarked images; selecting a designated inlier image from among the unmarked images; selecting a designated outlier image from among the unmarked images; calculating a first similarity matrix including a plurality of first similarity scores of the unmarked image to the designated inlier images and a plurality of second similarity scores of the unmarked image to the designated outlier images; selecting a maximum value along each row of the first similarity matrix to generate a second similarity matrix, the second similarity matrix having a third similarity score based on the first similarity score and the second similarity score; multiplying the second similarity matrix by a weighting matrix to generate a third similarity matrix having fourth similarity scores, wherein a weighting value in the weighting matrix is ​​determined depending on whether a maximum value corresponds to the designated inlier image or the designated outlier image; classifying each of the unmarked images into an inlier data set or an outlier data set based on the fourth similarity scores in the third similarity matrix to generate a plurality of inlier-outlier predictions for the unmarked images; Data classification methods, including:

9. acquiring a plurality of unmarked images; performing Q prediction rounds on the unmarked image, Q being a positive integer, each of the Q prediction rounds comprising: randomly selecting a plurality of default inlier images from among the unmarked image; calculating a first similarity matrix including first similarity scores of the unmarked image to the default inlier images; and generating a plurality of intermediate inlier-outlier prediction data for the unmarked image in one of the prediction rounds based on the first similarity matrix; aggregating the intermediate inlier-outlier prediction data for the unmarked images generated in each of the Q prediction rounds and selecting a plurality of aggregate predicted inlier images from among the unmarked images; calculating a second similarity matrix comprising a plurality of second similarity scores of the unmarked image to the collective predicted inlier image; classifying each of the unmarked images into an inlier data set or an outlier data set based on the second similarity matrix to generate a first plurality of inlier-outlier predictions for the unmarked images; displaying a portion of the first inlier-outlier prediction data for the unmarked image; obtaining modified adjustment input data based on the first inlier-outlier prediction data; calculating a third similarity matrix comprising a plurality of third similarity scores for the unmarked image to the conditioned input data; classifying each of the unmarked images into the inlier data set or the outlier data set based on the third similarity matrix to generate a plurality of second inlier-outlier predictions for the unmarked images; Data classification methods, including:

Images