Data annotation system using active learning model and the method thereof

Abstract

A data annotation method using an active learning model according to an embodiment of the present invention is characterized by comprising: a step of manually labeling a first sample data from a dataset without data annotations; a step of initializing the labeled first sample data as network parameters of an active learning model and learning; a step of selecting and learning instances of second sample data having a confidence value higher than a confidence threshold using an active learning model from the dataset without data annotations; a step of repeating the selection and learning process of the second sample data; and a step of fine-tuning the parameters of the active learning model of the repeated selection and learning process of the second sample data.

Description

Technology Field

[0001] The present invention relates to a data annotation system and method using an active learning model, and more specifically, to a data annotation system and method using an active learning model that can improve efficiency by increasing classification accuracy by selecting and learning from highly reliable data instances in an unlabeled dataset using an active learning model, and by replacing manual data annotation work to reduce work time and cost. Background Technology

[0003] Generally, data labeling involves assigning specific values ​​to training data prior to modeling tasks in machine learning (ML) or deep learning (DL). While data labeling is fundamental to the Artificial Intelligence (AI) industry, it is also recognized by companies as a core element for the advancement of the AI ​​sector. Traditional data labeling had the disadvantage of being costly and time-consuming because it was performed manually by humans. Furthermore, even when automated data labeling is performed, classifying images requires at least thousands or tens of thousands of images. Such large-scale data labeling presents problems: it incurs significant costs, including time, and as the number of images required for labeling increases, it becomes difficult to ensure consistent and accurate labeling. The problem to be solved

[0005] To address the aforementioned problems, the present invention provides a data annotation system and method using an active learning model that can improve efficiency by increasing classification accuracy through the selection and training of highly confident data instances from an unlabeled dataset using an active learning model, and by replacing manual data annotation work to reduce work time and cost. means of solving the problem

[0007] A data annotation method using an active learning model according to an embodiment of the present invention is characterized by comprising: a step of manually labeling a first sample data from a dataset without data annotations; a step of initializing the labeled first sample data as network parameters of an active learning model and learning; a step of selecting and learning instances of second sample data having a confidence value higher than a confidence threshold using an active learning model from the dataset without data annotations; a step of repeating the selection and learning process of the second sample data; and a step of fine-tuning the parameters of the active learning model of the repeated selection and learning process of the second sample data.

[0008] Here, in particular, the fine-tuning step is characterized by further including the adjustment of the instance confidence threshold of the repeated second sample data.

[0009] Here, the step of selecting and learning instances of the second sample data is characterized by selecting uncertain instances from the second sample data and generating and assigning pseudo-labels to the selected instances.

[0010] Here, in particular, the selection of the uncertain instance is characterized in that, in the dataset without data annotations, the active learning model selects a second sample data instance by measuring uncertainty combining minimum confidence, margin sampling, and entropy.

[0011] Here, in particular, the minimum confidence level is characterized by the fact that in the active learning model, all data sample values ​​in the dataset without data annotations are sorted in ascending order, and the class having the lowest confidence value among the prediction probabilities in a given sample data class is defined as having uncertainty.

[0012] Here, in particular, the margin sampling is characterized by the fact that in the active learning model, the margin sampling values ​​of all data samples in the dataset without data annotations are sorted in ascending order, and the smaller the probability difference between the class with the highest prediction probability and the class with the second highest prediction probability for a given sample data class, the greater the uncertainty of the corresponding sample data.

[0013] Here, in particular, the above entropy is characterized in that in the active learning model, the higher the entropy of a given sample data class in the dataset without data annotations, the greater the uncertainty.

[0014] Here, the characteristic feature is that a sample data class with high confidence is selected, particularly when the entropy is below a threshold value.

[0015] Here, the feature is that a pseudo-label is generated and assigned, particularly to the second sample data with high entropy.

[0016] Here, in particular, the characteristic feature is that in the step of repeating the selection and learning process of the second sample data, the second sample data is added to the first sample data to perform repeated learning.

[0017] Here, in particular, the feature is that the damping rate of fine-tuning is calculated using a backpropagation loss function in the step of fine-tuning the parameters of the active learning model.

[0018] In addition, a data annotation system using an active learning model according to one embodiment of the present invention is characterized by comprising: a database storing a data set without annotations; a selection unit selecting an instance of second sample data having a confidence value higher than a confidence threshold using an active learning model from a data set without annotations stored in the database; a learning unit initializing and learning a first sample data from a data set without annotations stored in the database as network parameters of an active learning model, and repeatedly learning the instance of second sample data selected by the selection unit through an active learning model; and an adjustment unit fine-tuning the parameters of the active learning model of the selection and learning process of the second sample data repeated in the learning unit and adjusting the confidence threshold of the instance of the selected second sample data.

[0019] Here, in particular, the selection unit is characterized by selecting an uncertain instance from the second sample data and generating and assigning a pseudo-label of the selected instance.

[0020] Here, in particular, the selection of the uncertain instance is characterized in that, in the dataset without data annotations, the active learning model selects a second sample data instance by measuring uncertainty combining minimum confidence, margin sampling, and entropy.

[0021] Here, in particular, the minimum confidence level is characterized by the fact that in the active learning model, all data sample values ​​in the dataset without data annotations are sorted in ascending order, and the class having the lowest confidence value among the prediction probabilities in a given sample data class is defined as having uncertainty.

[0022] Here, in particular, the margin sampling is characterized by the fact that in the active learning model, the margin sampling values ​​of all data samples in the dataset without data annotations are sorted in ascending order, and the smaller the probability difference between the class with the highest prediction probability and the class with the second highest prediction probability for a given sample data class, the greater the uncertainty of the corresponding sample data.

[0023] Here, in particular, the above entropy is characterized in that in the active learning model, the higher the entropy of a given sample data class in the dataset without data annotations, the greater the uncertainty.

[0024] Here, the characteristic feature is that a sample data class with high confidence is selected, particularly when the entropy is below a threshold value.

[0025] Here, the feature is that a pseudo-label is generated and assigned, particularly to the second sample data with high entropy. Effects of the invention

[0027] According to one embodiment of the present invention described above, by using an active learning model to select and learn highly reliable data instances from an unlabeled dataset, classification accuracy can be increased, and efficiency can be improved by replacing manual data annotation work and reducing work time and cost. Brief explanation of the drawing

[0029] FIG. 1 is a schematic diagram illustrating the configuration of a data annotation system using an active learning model according to an embodiment of the present invention. FIG. 2 is a diagram schematically illustrating the sequence of a data annotation method using an active learning model of the present invention. FIG. 3 is a diagram schematically illustrating a framework for a model update process by adding a dataset of the active learning model of the present invention. FIG. 4 is a diagram illustrating the architecture of the active learning model of the present invention. FIG. 5 is a schematic diagram illustrating the structure of a layer of the active learning model of the present invention. Specific details for implementing the invention

[0030] The present invention is susceptible to various modifications and may have various embodiments, and specific embodiments will be described in detail with reference to the drawings. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each drawing.

[0031] Terms such as first, second, A, B, etc., may be used to describe various components, but said components shall not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of multiple related description items or any of the multiple related description items.

[0032] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0033] Throughout the specification and claims, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0035] FIG. 1 is a schematic diagram illustrating the configuration of a data annotation system using an active learning model according to one embodiment of the present invention.

[0036] As illustrated in FIG. 1, a data annotation system (100) using an active learning model according to one embodiment of the present invention may be configured to include a database (110), a selection unit (120), a learning unit (130), and an adjustment unit (140). Here, each component is intended to be functionally distinguished in a processor that performs a Deep Learning Based Active Learning (DLBAL) model, but is not limited thereto.

[0037] Meanwhile, the configuration of the data annotation system (100) using the active learning model of the present invention is described briefly, and will be described in detail in the data annotation method using the active learning model of the present invention to be described later.

[0038] The above database (110) stores unannotated datasets and stores labeled datasets by selecting highly reliable instances from many unlabeled datasets through a deep learning active learning model.

[0039] The selection unit (120) selects an instance of second sample data having a confidence value higher than a confidence threshold using an active learning model from a dataset without data annotations stored in the database (110).

[0040] More specifically, the selection unit (120) selects an uncertain instance from the second sample data and generates and assigns a pseudo-label to the selected instance. At this time, the selection of the uncertain instance can be made by the active learning model in the data set without data annotations to select the second sample data instance by measuring uncertainty combining least confidence, margin sampling, and entropy.

[0041] The above Least Confidence is defined as the class with the lowest confidence value among the prediction probabilities in a given sample data class by sorting all data sample values ​​in ascending order in the above active learning model without data annotations.

[0042] The above margin sampling is defined in the active learning model by sorting the margin sampling values ​​of all data samples in the unannotated dataset in ascending order, such that for a given sample data class, the smaller the probability difference between the class with the highest prediction probability and the class with the second highest prediction probability, the greater the uncertainty of the corresponding sample data.

[0043] The above entropy is defined in the active learning model such that the higher the entropy of a given sample data class in a dataset without data annotations, the greater the uncertainty. Here, if the entropy is below a threshold, a sample data class with high confidence can be selected.

[0044] Then, a pseudo-label is generated and assigned to the second sample data with high entropy.

[0046] The learning unit (130) learns by initializing the first sample data from the unannotated data set stored in the database (110) as network parameters of the active learning model, and iteratively learns instances of the second sample data selected by the selection unit (120) through the active learning model.

[0047] More specifically, the learning unit (130) repeatedly performs the process of selecting samples from a pool of unlabeled datasets of a Deep Learning Based Active Learning (DLBAL) network, adding annotations, and then adding them to the training data.

[0049] The adjustment unit (140) fine-tunes the parameters of the active learning model of the selection and learning process of the second sample data repeated in the learning unit (130) and adjusts the instance confidence threshold of the selected second sample data.

[0051] FIG. 2 is a diagram schematically illustrating the sequence of a data annotation method using the active learning model of the present invention, FIG. 3 is a diagram schematically illustrating a framework for a model update process by adding a dataset of the active learning model of the present invention, FIG. 4 is a diagram illustrating the architecture of the active learning model of the present invention, and FIG. 5 is a diagram schematically illustrating the structure of a layer of the active learning model of the present invention.

[0053] As illustrated in FIG. 2, a data annotation method using an active learning model according to one embodiment of the present invention first performs the step of manually assigning a label by selecting a first sample data from a dataset without data annotations (S210).

[0054] More specifically, as illustrated in FIG. 3, the Deep Learning Based Active Learning (DLBAL) model of the present invention performs the process of selecting samples from an unlabeled data pool, adding annotations, and then adding them to the training data.

[0055] For example, a Deep Learning Based Active Learning (DLBAL) model learns labeled training data by having a worker manually select sample data from an unannotated dataset and add data annotations. Here, the sample data from the unannotated dataset that has been annotated by a worker can be the first sample data.

[0056] At this time, the Deep Learning Based Active Learning (DLBAL) model can apply EfficientNet-B0 as a classification algorithm model, and accurately perform large-scale image recognition.

[0057] As shown in Figure 4, the EfficientNet-B0 standard model architecture of the active learning model is displayed.

[0058] In other words, the deep learning active learning model expands the network width, depth, and resolution scaling factors, where the number of connected layers is equal to the network depth, and the width of the convolutional layers is proportional to the number of filters. Resolution is determined by the height and width of the input image.

[0059] In other words, as shown in Fig. 5, the EfficientNet-B0 network may include three dense layers, dropout, batch normalization, global mean pooling, and a softmax classifier.

[0060] Additionally, a global mean pooling layer is used to convert the feature map of the 6th MBConv layer into a one-dimensional array. Three dense layers, batch normalization, a ReLu activation function, and a dropout layer are added subsequently. The final feature vector is used as input to the first and second dense layers, which generate vectors of sizes 1024 and 512, respectively. 256 channels for 256 classes are generated in the last dense layer, and a Softmax layer is used to classify the 256 classes in the final dense layer.

[0061] In addition, each hidden layer uses ReLU as the activation function. This is because ReLU accelerates learning and mitigates the risk of the vanishing gradient problem, thereby increasing computational efficiency. To address the overfitting problem, batch normalization and dropout layers following hidden layers of 1024 and 512 neurons can also be utilized. Softmax is used as the final classifier layer to estimate the probability scores of each class.

[0063] Then, a step of initializing the first sample data with the above-mentioned label as network parameters of an active learning model and training is performed (S220).

[0064] More specifically, the deep learning active learning model (DLBAL) strategy of the present invention proposed for performing deep image classification and data annotation tasks progressively selects relevant samples for model updates.

[0065] for example, A class represented as m and samples n Assuming there is a dataset, an initially labeled data sample Mark as and uncommented data samples It is indicated as.

[0066] The label for is And, Is j Indicates that it belongs to the nth class.

[0067] The first issue with image classification is that almost all data samples lack annotations, and of is not defined and must be determined during the learning process.

[0068] The second can be gradually added to this system. This indicates that the amount of data can be continuously expanded. The DLBAL method of the present invention can comprehensively process continuously expanding unlabeled data.

[0069] DLBAL for deep image classification can be represented as [Equation 1] as follows.

[0071] [Mathematical Formula 1]

[0072]

[0074] Here It indicates the indicator function as 1 for true and 0 for false.

[0075] also is CNN network parameters and CNN's j Reflects the softmax score for the i-th class. Sample this j It belongs to the nth class. In particular, data samples with pseudo-annotations and network parameters We will build a technology that updates alternately.

[0076] Also, annotated samples before starting learning It is empty. For each class A limited number of training data samples are taken from and manually labeled to be used as inputs for initializing the CNN network parameters.

[0078] Next, a step is performed to select and train instances of second sample data having a confidence value higher than a confidence threshold using an active learning model from the above data set without data annotations (S230).

[0079] More specifically, to perform deep image classification and data annotation tasks, the deep learning active learning model gradually selects relevant data samples for updates.

[0080] The process of selecting an instance of the second sample data involves selecting an uncertain instance from the second sample data and generating and assigning a pseudo-label of the selected instance.

[0081] Here, the selection of the uncertain instance is such that in the dataset without data annotations, the active learning model selects a second sample data instance by measuring the uncertainty combined with minimum confidence, margin sampling, and entropy.

[0082] The above least confidence is all data samples in the dataset without data annotations in the above active learning model The values ​​are expressed in ascending order as shown in the following mathematical formula 2.

[0083] [Mathematical Formula 2]

[0084]

[0085] thus, According to the definition, a classifier defines a data sample as having uncertainty if the confidence of the most likely class for a given data instance is low.

[0086] The above margin sampling In the above active learning model, unannotated data samples are sorted in ascending order based on their values.

[0087] The value is calculated for each sample by taking the difference between the highest prediction probability and the second highest prediction probability for the sample. This is expressed as shown in the following mathematical formula 3.

[0088] [Mathematical Formula 3]

[0089]

[0090] Therefore, the first and next possible class label models estimated by classification are denoted as 1 and 2, respectively. The smaller the margin value, the more uncertain the model is regarding the correct label for that sample. Therefore, sorting unannotated data samples in ascending order based on this allows selecting the most uncertain samples to include in the training set or add annotations to improve the model's learning efficiency.

[0091] In other words, for a given sample data class, the smaller the probability difference between the class with the highest prediction probability and the class with the second highest prediction probability, the higher the uncertainty of that sample data is defined.

[0093] The above entropy In the above active learning model, the higher the entropy of a given sample data class in a dataset without data annotations, the more uncertainty is defined.

[0094] In other words, it is generally used as a measure of the uncertainty of a model's prediction for a given sample. The concept is that if a sample has high entropy, it provides more information to the model because it is more uncertain, which can potentially help reduce the overall uncertainty of the model. Prediction probabilities across various classes for a specific sample It is used to calculate. It is expressed as shown in the following mathematical formula 4.

[0095] [Mathematical Formula 4]

[0096]

[0097] Here is the expected probability of class. For the sample, the model parameters are given and the sum takes all classes. The value indicates the degree of uncertainty or unpredictability of the model prediction for the sample. Then, all unannotated data samples Sort in descending order based on value. Largest value Samples with [value] are selected for labeling or inclusion in the training set because they are considered to have the largest value. Informative samples. By selecting samples with high entropy for labeling or inclusion in the training set, active learning model methods can improve model performance with a smaller number of labeled samples compared to passive learning, where all samples are labeled. Additionally, entropy [is a threshold value] High reliability below A sample is selected. Then, an accurately identified pseudo-label is assigned. The definition of a pseudo-label is given by the following mathematical formula 5.

[0098] [Mathematical Formula 5]

[0099]

[0100] Here silver indicates that it is a highly reliable sample. The selected instance is It is designated as. Classification probability Compared to, Entropy used for categories It comprehensively integrates the classification probabilities of the remaining classes, and the data sample must be correctly classified with high confidence. is set to a high value to provide a high level of reliability while assigning pseudo-labels.

[0101] In conclusion, when the above entropy is below the threshold, a sample data class with high confidence is selected, and a pseudo-label is generated and assigned to the second sample data with high entropy.

[0103] Next, a step of repeating the selection and learning process of the second sample data is performed (S240). Here, the step of repeating the selection and learning process of the second sample data involves adding the second sample data to the first sample data to perform repeated learning.

[0104] Finally, a step of fine-tuning the parameters of the active learning model of the selection and learning process of the repeated second sample data is performed (S250).

[0105] More specifically, the fine-tuning step is a pseudo-labeled sample and manually labeled samples When this is fixed, the above mathematical formula 1 can be simplified as follows.

[0107] [Mathematical Formula 6]

[0108]

[0109] Here silver It represents the quantity of data samples. Backpropagation techniques can be used to update it. In particular, if this represents the loss function of Equation 5, the partial derivative of can be represented according to the following Equation 7.

[0110] In other words, the fine-tuning of the parameters of the aforementioned active learning model calculates the damping rate of fine-tuning using a loss function of the backpropagation method.

[0112] [Mathematical Formula 7]

[0113]

[0114] Here, represents the activation of the i-th data instance, and the definition of the softmax layer of the last CNN layer before being passed to the softmax layer is as follows in Equation 8.

[0115] [Mathematical Formula 8]

[0116]

[0118] In addition, the instance confidence threshold of the aforementioned iterated second sample data is simultaneously adjusted. That is, through the iterative learning process, the classification performance of the classifier improves, and since more high-confidence samples are selected, incorrect annotations can be reduced. To ensure the reliability of selecting high-confidence samples, the threshold for more confident instances is set at the completion of each iteration. It is adjusted as shown in the following mathematical formula 9.

[0120] [Mathematical Formula 9]

[0121]

[0122] Here, the threshold attenuation rate is controlled by and is the starting threshold.

[0124] Accordingly, according to the present invention described above, by using an active learning model to select and learn highly reliable data instances from an unlabeled dataset, classification accuracy can be increased, and efficiency can be improved by replacing manual data annotation work and reducing work time and cost.

[0126] One embodiment of the present invention may also be implemented in the form of a recording medium comprising computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include both computer storage media and communication media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data. A communication medium typically includes computer-readable instructions, data structures, program modules, or other data of a modulated data signal such as a carrier wave, or other transmission mechanisms, and includes any information transmission medium.

[0127] Although the method and system of the present invention have been described in relation to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

[0128] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0129] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0131] 110: Database 120: Selection 130: Learning Department 140: Adjustment Department

Claims

Claim 1 A step of manually labeling a first sample data from a dataset without data annotations in a selection unit; a step of initializing the labeled first sample data as network parameters of an active learning model and training in a training unit; a step of selecting and training instances of second sample data having a confidence value higher than a confidence threshold using an active learning model from the dataset without data annotations in a training unit; a step of repeating the selection of the second sample data in the selection unit and the training process in the training unit; A data annotation method using an active learning model, comprising: a step of fine-tuning parameters of an active learning model in the selection and learning process of the repeated second sample data in the adjustment section; wherein the step of repeating the selection and learning process of the second sample data calculates a change in the reliability distribution and selection stability of the second sample data selected between consecutive repetition steps, and the fine-tuning step corrects the value or scale of some network parameters of the active learning model so as to be reflected in the calculation of the reliability value when selecting the second sample data in the next repetition step, using the change in the reliability distribution and selection stability calculated in the selection and learning process of the repeated second sample data as input, wherein the correction calculates the attenuation rate of the fine-tuning using a loss function of the backpropagation method. Claim 2 A data annotation method using an active learning model according to claim 1, characterized in that, in the fine-tuning step, the method further includes adjusting the instance confidence threshold of the repeated second sample data. Claim 3 A data annotation method using an active learning model according to claim 1, wherein the step of selecting and learning instances of the second sample data is characterized by selecting uncertain instances from the second sample data and generating and assigning pseudo-labels of the selected instances. Claim 4 A data annotation method using an active learning model according to claim 3, wherein the selection of the uncertain instance is characterized in that the active learning model selects a second sample data instance by measuring uncertainty combining minimum confidence, margin sampling, and entropy in the data set without data annotation. Claim 5 A data annotation method using an active learning model according to claim 4, wherein the minimum confidence is defined as the class having the lowest confidence value among the prediction probabilities in a given sample data class by sorting all data sample values ​​in ascending order in the active learning model without data annotation. Claim 6 A data annotation method using an active learning model according to claim 4, wherein the margin sampling is defined as having uncertainty in the corresponding sample data as the smaller the difference in probability between the class with the highest prediction probability and the class with the second highest prediction probability for a given sample data class, by sorting the margin sampling values ​​of all data samples in an active learning model in ascending order in a dataset without data annotations. Claim 7 A data annotation method using an active learning model according to claim 4, characterized in that the entropy is defined as having uncertainty as the entropy of a given sample data class in a dataset without data annotations in the active learning model is higher. Claim 8 A data annotation method using an active learning model according to claim 7, characterized by selecting a sample data class with high confidence when the entropy is less than a threshold value. Claim 9 A data annotation method using an active learning model according to claim 7, characterized by generating and assigning pseudo-labels to the second sample data with high entropy. Claim 10 A data annotation method using an active learning model according to claim 1, characterized in that, in the step of repeating the selection and learning process of the second sample data, the second sample data is added to the first sample data to perform iterative learning. Claim 11 delete Claim 12 A database storing an unannotated dataset; a selection unit that selects an instance of second sample data having a confidence value higher than a confidence threshold using an active learning model from the unannotated dataset stored in the database; and a learning unit that initializes and learns a first sample data from the unannotated dataset stored in the database as network parameters of an active learning model, and iteratively learns the instance of second sample data selected by the selection unit through the active learning model. A data annotation system using an active learning model, comprising: an adjustment unit that fine-tunes parameters of an active learning model in the selection and learning process of a second sample data repeated in the learning unit and adjusts the instance confidence threshold of the selected second sample data; wherein, in the selection and learning process of second sample data repeated through the learning unit, the reliability distribution change and selection stability of the second sample data selected between consecutive repeated operations are calculated, and the adjustment unit corrects the values ​​or scales of some network parameters of the active learning model so as to be reflected in the calculation of a confidence value when selecting second sample data in the next repeated operation using the calculated reliability distribution change and selection stability as input, wherein the correction is performed by reflecting a damping coefficient set using the gradient of a loss function calculated by backpropagation. Claim 13 A data annotation system using an active learning model according to claim 12, wherein the selection unit selects uncertain instances from the second sample data and generates and assigns pseudo-labels of the selected instances. Claim 14 A data annotation system using an active learning model according to claim 13, wherein the selection of the uncertain instance is characterized in that the active learning model selects a second sample data instance by measuring uncertainty combining minimum confidence, margin sampling, and entropy in the data set without data annotation. Claim 15 A data annotation system using an active learning model according to claim 14, wherein the minimum confidence is defined as the class having the lowest confidence value among the prediction probabilities in a given sample data class by sorting all data sample values ​​in ascending order in the active learning model without data annotations. Claim 16 A data annotation system using an active learning model according to claim 14, wherein the margin sampling is defined as having uncertainty in the corresponding sample data as the smaller the difference in probability between the class with the highest prediction probability and the class with the second highest prediction probability for a given sample data class, by sorting the margin sampling values ​​of all data samples in the data set without data annotations in ascending order in the active learning model. Claim 17 A data annotation system using an active learning model according to claim 14, characterized in that the entropy is defined as having higher uncertainty as the entropy of a given sample data class in a dataset without data annotations in the active learning model. Claim 18 A data annotation system using an active learning model according to claim 17, characterized by selecting a sample data class with high confidence when the entropy is below a threshold value. Claim 19 A data annotation system using an active learning model according to claim 17, characterized by generating and assigning pseudo-labels to the second sample data with high entropy. Claim 20 A computer-readable recording medium storing a program for performing on a computer the method described in any one of claims 1 to 10.

Images