METHOD FOR SELECTING TRAINING DATA TO TRAIN A DEEP LEARNING MODEL AND TRAINING DATA SELECTING DEVICE USING THE SAME

Abstract

A method for selecting training data for training a deep learning model is provided. [Solution] The method includes a step of a training data selection device acquiring at least one individual type corresponding to each of the multiple training data included in all training data stored in a data pool, and generating a bipartite graph matching each of the multiple training data included in all training data with the individual type; and a step of selecting n pieces of training data for training a deep learning model by referring to the bipartite graph to select n pieces of training data matching the individual types from all training data, such that the number of each of the individual types matching the n pieces of training data is within a predetermined threshold deviation.

Description

[Technical field]

[0001] The present invention relates to a method for uniformly selecting learning data for training a deep learning model from all learning data stored in a data pool without data bias or variation, and a learning data selection device using the same. [Background technology]

[0002] In general, deep learning models recognize complex patterns in images, text, sound, and other data to generate accurate insights and predictions, and are applied in a variety of fields, including computer vision, speech recognition, autonomous vehicles, robotics, natural language processing, and medical image analysis.

[0003] In order for such deep learning models to perform a desired task accurately, they must be trained using large amounts of training data.

[0004] Conventional methods for selecting training data for training a deep learning model from a collected data pool include random sampling, which selects a target number of pieces of training data from all the training data stored in the data pool, and vector quantization, which clusters and groups vectors representing all the training data generated by embedding extraction, and selects representative values ​​for each group of grouped vectors.

[0005] For example, Patent Document 1 discloses a clustering device and method based on similarity using a deep learning technique, and Patent Document 2 discloses a system, method, and program for extracting high-contribution items for improving the performance of a multilayer neural network (deep learning). Patent Document 3 discloses a device and method for training a deep learning classification model, and Patent Document 4 discloses a system and method for training a machine learning model using active learning.

[0006] However, when the learning data is selected by the conventional method, there is a problem in that bias and variation occurs according to type.

[0007] As an example, if a data pool stores 1 million training images, of which 70% are training images related to sunny weather, 20% are training images related to cloudy weather, 5% are training images related to foggy weather, and 5% are training images related to snowy weather and / or rainy weather, when randomly sampling 10,000 training images, only about 500 training images will be probabilistically selected from the total of 50,000 training images related to snowy weather and / or rainy weather, resulting in bias and variation in the training images selected for each weather type.

[0008] In addition, when vector quantization is used to select training images, it is possible to somewhat alleviate bias and variation in the types of training images selected by embedding extraction and clustering, but it is not possible to fundamentally prevent problems related to bias and variation in the data.

[0009] Therefore, the present applicant proposes a method for uniformly selecting learning data for training a deep learning model from all learning data stored in a data pool by type without bias or variation. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] Korean Patent Publication No. 10-2023-0068941 [Patent Document 2] Patent No. 6458072 [Patent Document 3] Patent No. 7225614 [Patent Document 4] U.S. Patent No. 11663409 Summary of the Invention [Problem to be solved by the invention]

[0011] An object of the present invention is to solve all of the above problems.

[0012] Another object of the present invention is to enable the selection of learning data for training a deep learning model from a data pool evenly by type, without bias toward a particular type.

[0013] It is yet another object of the present invention to enable uniform selection of learning data suitable for various scenarios according to the type of learning data from a data pool. [Means for solving the problem]

[0014] According to one embodiment of the present invention, there is provided a method for selecting training data for training a deep learning model, the method including: (a) a step in which a training data selection device acquires at least one individual type (attribute) corresponding to each of a large number of training data included in all training data stored in a data pool, and generates a bipartite graph matching each of the large number of training data included in the all training data with the individual type; and (b) a step in which the training data selection device selects n pieces of training data (n is a target number of training data for training a deep learning model and is an integer representing a plural number) matching the individual type from among the all training data by referring to the bipartite graph, and selects the n pieces of training data for training the deep learning model such that the number of each of the individual types matching the n pieces of training data is within a predetermined threshold deviation.

[0015] In one example, in the step (b), the individual types include 1_1 individual types through 1_x individual types (where x is an integer equal to or greater than 1) corresponding to a first type possessed by each of the multiple learning data, and 2_1 individual types through 2_y individual types (where y is an integer equal to or greater than 1) corresponding to a second type possessed by each of the multiple learning data, and the learning data selection device selects the n learning data such that the number of the 1_1 individual types through 1_x individual types corresponding to the first type and the number of the 2_1 individual types through 2_y individual types corresponding to the second type, which match the n learning data, are within the threshold deviation, the number of the 1_1 individual types through the 1_x individual types are within a first threshold deviation, and the number of the 2_1 individual types through the 2_y individual types are within a second threshold deviation.

[0016] In one example, in the step (b), the training data selection device (i) refers to the bipartite graph to confirm, among the individual types, the number of corresponding individual types that match each of the all of the training data, and selects specific training data having the largest number of corresponding individual types; (ii) among the remaining individual types excluding the corresponding individual types, the number of corresponding individual types that match each of the all of the training data, and selects other specific training data having the largest number of corresponding individual types. By repeating these processes, a cycle is performed to select a portion of training data that matches all of the individual types, and the process of performing the cycle from the remaining training data excluding the portion of training data selected from the all of the training data is repeated until the n training data are selected.

[0017] In one example, in step (b), the training data selection device refers to the bipartite graph and selects a specific subset having the smallest number of training data from among subsets consisting of a predetermined number of training data including all of the individual types through an optimization algorithm, calculates remaining training data excluding a specific number of training data included in the specific subset, and repeats the process of selecting at least one other specific subset consisting of a predetermined number of training data including all of the individual types from the remaining training data through the optimization algorithm, so that the number of selected training data becomes equal to or greater than n.

[0018] In one example, in the step (b), the learning data selection device uses linear programming a P-dimensional vector (the P-dimensional vector represents a sum of the fitnesses of the Q learning data belonging to each of the P individual types) generated by calculating a matrix product of a PxQ binary matrix corresponding to P individual types and Q learning data in the bipartite graph and a Q-dimensional vector representing each selection fitness variable of the Q learning data for each of the P individual types using linear programming, the sum of the fitness being 1 or more, and the selection fitness variable in the Q-dimensional vector being between 0 and 1, the specific subset including the specific learning data corresponding to the specific selection fitness variable having a minimum sum of the selection fitness variables is selected from among the selection fitness variables of the Q-dimensional vector, the specific learning data included in the specific subset is calculated from the Q learning data, the process of calculating the remaining learning data by excluding the specific learning data included in the specific subset from the Q learning data, and repeating the process of selecting the at least one other specific subset using the linear programming for the remaining learning data until the number of selected learning data is equal to or greater than n.

[0019] In one example, in step (b), the training data selection device selects the specific subset using a dual linear programming method that applies at least one of merging, separating, and sign changing of constraints in linear programming.

[0020] In one example, in the step (b), the training data selection device uses integer programming to calculate a matrix product of a PxQ binary matrix corresponding to P individual types and Q training data in the bipartite graph and a Q-dimensional vector representing each selection variable of the Q training data for each of the P individual types, to generate a P-dimensional vector (the P-dimensional vector represents a selection quantity of training data belonging to each of the P individual types), the selection quantity in the P-dimensional vector being equal to or greater than 1, and from among the selection variables of the Q-dimensional vector whose selection variables in the Q-dimensional vector satisfy 0 or 1, selects the specific subset including specific training data corresponding to a specific selection variable having a minimum sum of the selection variables, calculates the remaining training data by excluding the specific training data included in the specific subset from the Q training data, and repeats the process of selecting another specific subset using the integer programming for the remaining training data until the number of selected training data becomes equal to or greater than n.

[0021] In one example, in step (b), the training data selection device selects the specific subset using a dual integer programming method that applies at least one of merging, separating, and sign changing constraints in the integer programming method.

[0022] In one example, in the step (a), the training data selection device transmits the plurality of training data to a labeler terminal, and generates at least one individual type corresponding to each of the plurality of training data using a labeler corresponding to the labeler terminal.

[0023] In one example, all of the training data are training images, and in step (a), the training data selection device performs a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a k-th embedding operation (where k is an integer greater than or equal to 1) on each of the training images to generate a k-th scene vector corresponding to each of the training images and clustering the k-th scene vector to generate a k-th scene cluster, and generates the individual type corresponding to the training image by referring to the first scene cluster to the k scene cluster.

[0024] In one example, all of the training data are training images, and in step (a), the training data selection device performs object detection on each of the training images to detect at least one object from each of the training images, generates cropped images by cropping areas corresponding to bounding boxes of each of the detected objects in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, clusters the object vectors to generate object clusters, and generates the individual types corresponding to the training images by referring to the object clusters.

[0025] In one example, all of the training data are training images, and in step (a), the training data selection device refers to ground truth information included in each of the training images, generates cropped images by cropping areas corresponding to bounding boxes of each object from each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, clusters the object vectors to generate object clusters, and generates the individual types corresponding to the training images by referring to the object clusters.

[0026] In one example, all of the training data are training images, and in the step (a), the training data selection device (i) performs a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a kth embedding operation (where k is an integer greater than or equal to 1) on each of the training images to generate a kth scene vector corresponding to each of the training images and clustering the kth scene vector to generate a kth scene cluster, (ii) performs object detection on each of the training images to detect at least one object from each of the training images, generates cropped images by cropping areas corresponding to bounding boxes of each of the detected objects in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and clusters the object vectors to generate object clusters, and (iii) generates the individual type corresponding to the training image by referring to the first scene cluster through the kth scene cluster and the object cluster.

[0027] In one example, all of the training data are training images, and in the step (a), the training data selection device (i) performs a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a k-th embedding operation (where k is an integer greater than or equal to 1) on each of the training images to generate a k-th scene vector corresponding to each of the training images and clustering the k-th scene vector to generate a k-th scene cluster, (ii) generates cropped images by cropping areas corresponding to bounding boxes of each object from each of the training images with reference to each of the ground truth information included in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and clusters the object vectors to generate object clusters, and (iii) generates the individual type corresponding to the training image with reference to the first scene cluster through the k-th scene cluster and the object cluster.

[0028] According to one embodiment of the present invention, there is provided a training data selection device for selecting training data for training a deep learning model, the device including: a memory storing instructions for selecting training data for training a deep learning model; and a processor executing an operation for selecting training data for training the deep learning model in accordance with the instructions stored in the memory; wherein the processor executes a process of (I) acquiring at least one individual type (attribute) corresponding to each of a plurality of training data included in all training data stored in a data pool, and generating a bipartite graph matching each of the plurality of training data included in all training data with the individual type; and (II) selecting n pieces of training data for training the deep learning model (n is a target number of training data for training the deep learning model and is an integer representing a plurality of numbers) matching the individual type from among all training data by referring to the bipartite graph, such that the number of each of the individual types matching the n pieces of training data is within a predetermined threshold deviation.

[0029] In one example, the individual types include 1_1 individual types through 1_x individual types (where x is an integer equal to or greater than 1) corresponding to a first type possessed by each of the multiple learning data, and 2_1 individual types through 2_y individual types (where y is an integer equal to or greater than 1) corresponding to a second type possessed by each of the multiple learning data, and in the (II) process, the processor selects the n learning data such that the number of the 1_1 individual types through 1_x individual types corresponding to the first type and the number of the 2_1 individual types through 2_y individual types corresponding to the second type, which match the n learning data, are within the threshold deviation, the number of the 1_1 individual types through the 1_x individual types are within a first threshold deviation, and the number of the 2_1 individual types through the 2_y individual types are within a second threshold deviation.

[0030] In one example, in the process (II), the processor repeats the process of (i) referring to the bipartite graph, checking the number of corresponding individual types among the individual types that match each of the all of the learning data, and selecting specific learning data having the largest number of corresponding individual types, and (ii) checking the number of corresponding individual types among the remaining individual types excluding the corresponding individual types that match each of the all of the learning data, and selecting other specific learning data having the largest number of corresponding individual types, thereby performing a cycle of selecting a portion of learning data that matches all of the individual types, and repeating the process of performing the cycle from the remaining learning data excluding the portion of learning data selected from the all of the learning data, until the n number of learning data are selected.

[0031] In one example, in the process (II), the processor refers to the bipartite graph and selects a specific subset having the smallest number of learning data from among subsets consisting of a predetermined number of learning data including all of the individual types through an optimization algorithm, calculates remaining learning data excluding a specific number of learning data included in the specific subset, and repeats the process of selecting at least one other specific subset consisting of a predetermined number of learning data including all of the individual types through the optimization algorithm from the remaining learning data, so that the number of selected learning data becomes equal to or greater than n.

[0032] In one example, the processor performs linear programming in the process (II). a P-dimensional vector (the P-dimensional vector represents a sum of the fitnesses of the Q learning data belonging to each of the P individual types) generated by calculating a matrix product of a PxQ binary matrix corresponding to P individual types and Q learning data in the bipartite graph and a Q-dimensional vector representing each selection fitness variable of the Q learning data for each of the P individual types using Linear Programming, the sum of fitness in the P-dimensional vector is 1 or more, and the selection fitness variable in the Q-dimensional vector satisfies 0 to 1, and the specific subset including specific learning data corresponding to a specific selection fitness variable having a minimum sum of the selection fitness variables is selected from among the selection fitness variables of the Q-dimensional vector; the specific learning data included in the specific subset is calculated from the Q learning data, and the process of selecting at least one other specific subset using the linear programming is repeated for the remaining learning data until the number of selected learning data is equal to or greater than n.

[0033] In one example, in the (II) process, the processor selects the specific subset using a dual linear programming method that applies at least one of merging, separating, and changing the sign of constraints in linear programming.

[0034] In one example, in the process (II), the processor uses integer programming to calculate a matrix product of a PxQ binary matrix corresponding to P individual types and Q training data in the bipartite graph and a Q-dimensional vector representing each selection variable of the Q training data for each of the P individual types to generate a P-dimensional vector (the P-dimensional vector represents a selection quantity of training data belonging to each of the P individual types), the selection quantity in the P-dimensional vector being equal to or greater than 1, and the selection variable in the Q-dimensional vector being equal to 0 or 1, to select the specific subset including specific training data corresponding to a specific selection variable having a minimum sum of the selection variables, calculate the remaining training data by excluding the specific training data included in the specific subset from the Q training data, and repeat the process of selecting another specific subset using the integer programming for the remaining training data until the number of selected training data becomes equal to or greater than n.

[0035] In one example, in the (II) process, the processor selects the specific subset using a dual integer programming method that applies at least one of merging, separating, and changing the sign of constraints in the integer programming method.

[0036] In one example, in the process (I), the processor transmits the plurality of training data to a labeler terminal, and uses a labeler corresponding to the labeler terminal to generate at least one individual type corresponding to each of the plurality of training data.

[0037] In one example, all of the training data are training images, and in process (I), the processor performs a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a k-th embedding operation (where k is an integer greater than or equal to 1) on each of the training images to generate a k-th scene vector corresponding to each of the training images and clustering the k-th scene vector to generate a k-th scene cluster, and generates the individual type corresponding to the training image by referring to the first scene cluster to the k scene cluster.

[0038] In one example, all of the training data are training images, and in process (I), the processor performs object detection on each of the training images to detect at least one object from each of the training images, generates cropped images by cropping areas corresponding to bounding boxes of each of the detected objects in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, clusters the object vectors to generate object clusters, and generates the individual types corresponding to the training images by referring to the object clusters.

[0039] In one example, all of the training data are training images, and in the process (I), the processor refers to ground truth information included in each of the training images, generates cropped images by cropping areas corresponding to bounding boxes of each object from each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, clusters the object vectors to generate object clusters, and generates the individual types corresponding to the training images by referring to the object clusters.

[0040] In one example, all of the training data are training images, and in the process (I), the processor (i) performs a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a kth embedding operation (where k is an integer greater than or equal to 1) on each of the training images to generate a kth scene vector corresponding to each of the training images and clustering the kth scene vector to generate a kth scene cluster, (ii) performs object detection on each of the training images to detect at least one object from each of the training images, generates cropped images by cropping areas corresponding to bounding boxes of each of the detected objects in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and clusters the object vectors to generate object clusters, and (iii) generates the individual type corresponding to the training image by referring to the first scene cluster through the kth scene cluster and the object cluster.

[0041] In one example, all of the training data are training images, and in the process (I), the processor (i) performs a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a kth embedding operation (where k is an integer greater than or equal to 1) on each of the training images to generate a kth scene vector corresponding to each of the training images and clustering the kth scene vector to generate a kth scene cluster, (ii) generates cropped images by cropping areas corresponding to bounding boxes of each object from each of the training images with reference to respective ground truth information included in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and clusters the object vectors to generate object clusters, and (iii) generates the individual type corresponding to the training image with reference to the first scene cluster through the kth scene cluster and the object cluster. Effect of the Invention

[0042] The present invention makes it possible to select learning data for training a deep learning model from a data pool evenly by type, without bias toward a particular type.

[0043] The present invention makes it possible to uniformly select learning data suitable for various scenarios according to the type of learning data from a data pool. [Brief description of the drawings]

[0044] The following drawings, attached for use in explaining the embodiments of the present invention, are only a part of the embodiments of the present invention, and a person having ordinary knowledge in the technical field to which the present invention pertains (hereinafter, "ordinary engineers") can derive other drawings based on these drawings without performing inventive work.

[0045] [Figure 1] FIG. 1 is a diagram illustrating a schematic diagram of a training data selection device that selects training data for training a deep learning model according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating a method for selecting training data for training a deep learning model according to a first embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram illustrating an example of generating an individual type of learning data in the first embodiment of the present invention. [Figure 4] FIG. 4 is a diagram illustrating another example of generating an individual type of learning data in the first embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram illustrating an example of a bipartite graph in which each of the learning data is matched with an individual type in the first embodiment of the present invention. [Figure 6a] FIG. 6a is a diagram illustrating a process of selecting training data by referring to a bipartite graph in the first embodiment of the present invention. [Figure 6b] FIG. 6b is a diagram illustrating a process of selecting training data by referring to a bipartite graph in the first embodiment of the present invention. [Figure 6c] FIG. 6c is a diagram illustrating a process of selecting training data by referring to a bipartite graph in the first embodiment of the present invention. [Figure 6d] FIG. 6d is a diagram illustrating a process of selecting training data by referring to a bipartite graph in the first embodiment of the present invention. [Figure 7] FIG. 7 is a diagram illustrating a method for selecting training data for training a deep learning model according to a second embodiment of the present invention. [Figure 8] FIG. 8 is a diagram illustrating a method for selecting training data for training a deep learning model according to a third embodiment of the present invention. [Figure 9] FIG. 9 is a diagram illustrating a method for selecting training data for training a deep learning model according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] The following detailed description of the invention refers to the accompanying drawings, which show by way of example specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable one of ordinary skill in the art to practice the invention. It should be understood that the various embodiments of the invention are different from one another, but are not necessarily mutually exclusive. For example, a particular shape, structure, and characteristic described in this specification in relation to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of individual components in each disclosed embodiment may be modified without departing from the spirit and scope of the invention. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, with the full scope of equivalents to which such claims are entitled, if properly described. In the drawings, like reference numerals refer to the same or similar features across the various aspects.

[0047] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those skilled in the art to easily carry out the present invention.

[0048] FIG. 1 is a schematic diagram of a training data selection device for selecting training data for training a deep learning model according to one embodiment of the present invention. The training data selection device 100 includes a memory 110 storing instructions for selecting training data for training a deep learning model, and a processor 120 that performs operations for selecting training data for training the deep learning model in accordance with the instructions stored in the memory 110.

[0049] Specifically, the training data selection device 100 may typically utilize, but is not limited to, a combination of computing devices (e.g., devices that may include a computer processor, memory, storage, input devices and output devices, and other conventional computing device components; electronic communication devices such as routers, switches, etc.; electronic information storage systems such as network attached storage (NAS) and storage area networks (SAN)) and computer software (i.e., instructions that cause a computing device to function in a particular manner) to achieve the desired system performance.

[0050] Furthermore, the processor 120 of the learning data selection device 100 may include hardware components such as an MPU (Micro Processing Unit) or a CPU (Central Processing Unit), a cache memory, a data bus, etc. Furthermore, the network controller 100 may further include a software component such as an operating system and an application that executes a specific purpose.

[0051] However, this does not exclude the case where the training data selection device 100 includes an integrated processor in which a medium, a processor, and a memory for implementing the present invention are integrated.

[0052] Meanwhile, the processor 120 of the training data selection device 100 may execute a process of generating at least one individual type (attribute) corresponding to each of a large number of training data included in all the training data stored in the data pool according to the instructions stored in the memory 110, and generating a bipartite graph in which each of the large number of training data included in all the training data is matched with the individual type. The processor 120 of the training data selection device 100 may execute a process of selecting n pieces of training data for training a deep learning model such that the number of each of the individual types matching the n pieces of training data is within a predetermined threshold deviation when selecting n pieces of training data matching the individual types from all the training data by referring to the bipartite graph according to the instructions stored in the memory 110. The n is a target number of training data for training a deep learning model, and may be an integer representing a multiple number.

[0053] With reference to the thus configured learning data selection device according to an embodiment of the present invention, a method for selecting learning data for learning a deep learning model will be described below.

[0054] [First Example] FIG. 2 is a diagram illustrating a method for selecting training data for training a deep learning model according to a first embodiment of the present invention. The first embodiment of the present invention will be described below with reference to FIG. 2.

[0055] First, the training data selection device 100 may acquire at least one individual attribute corresponding to each of a plurality of training data included in all the training data stored in the data pool (S110).

[0056] In this case, the individual types may include 1_1 individual types through 1_x individual types corresponding to the first type possessed by each of the plurality of learning data, and 2_1 individual types through 2_y individual types corresponding to the second type possessed by each of the plurality of learning data. Each of the x and y may be an integer of 1 or more. However, the present invention is not limited thereto, and three or more types may be included according to the characteristics of the learning data.

[0057] For example, if the learning data is text data, it can be configured into various individual types corresponding to various type types, such as type types related to genres corresponding to individual types such as essays, novels, reports, travelogues, letters, etc., type types related to publication locations corresponding to individual types such as blogs, newspapers, SNS posts, etc., type types related to languages ​​corresponding to individual types such as Korean, English, German, etc., type types related to fields corresponding to individual types such as economic terms, social terms, mathematics terms, etc.

[0058] As another example, if the training data is audio data, it can be configured into various individual types corresponding to various type types, such as type types related to time periods corresponding to individual types such as day, night, dawn, and evening, type types related to places corresponding to individual types such as city, countryside, mountain, and sea, and type types related to sound subjects corresponding to individual types such as car sounds, human sounds, bird sounds, and wind sounds.

[0059] As yet another example, if the learning data is image data, it can be configured into various individual types corresponding to various type types, such as type types related to time periods corresponding to individual types such as day, night, evening, and dawn, type types related to weather corresponding to individual types such as sunny, cloudy, rainy, and fog, and type types related to objects corresponding to individual types such as automobiles, people, and animals.

[0060] In addition, at least one individual type (attribute) corresponding to each of the plurality of training data can be automatically extracted by labeling each of the plurality of training data with a predefined type by a labeler, or by utilizing various techniques such as AI (Artificial Intelligence) and clustering.

[0061] As an example, the training data selection device 100 may transmit a plurality of training data to a labeler terminal, and generate at least one individual type corresponding to each of the plurality of training data using a labeler corresponding to the labeler terminal.

[0062] As another example, the training data selection device 100 can automatically generate various individual patterns corresponding to various pattern types for the entire image that each of the training images, which is the entire training data, has.

[0063] That is, referring to Fig. 3, the training data selection device 100 may perform a process of performing a first embedding operation 10_1 on each of the training images to generate a first scene vector corresponding to each of the training images and clustering 20 the first scene vector to generate a first scene cluster, or a process of performing a k-th embedding operation 10_k on each of the training images to generate a k-th scene vector corresponding to each of the training images and clustering 20 the k-th scene vector to generate a k-th scene cluster. The k may be an integer equal to or greater than 1. The training data selection device 100 may generate an individual type corresponding to the training images by referring to the first scene cluster to the k-th scene cluster.

[0064] In this case, the training data selection device 100 may use visual foundation models such as CLIP (Contrastive Language-Image pre-training), DINO (self-DIstillation with NO labels), and DINOv2 for the first to k-th embedding operations, and the first to k-th embedding models for the first to k-th embedding operations may be different embedding models that generate scene vectors corresponding to the first to k-th types, which are different types. The training data selection device 100 may use various clustering algorithms such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise), K-Means, K-Medians, and Mean-Shift to cluster the first to k-th scene vectors.

[0065] In addition, in the above, in order to generate at least one individual type for each of the training images, each of the training images is subjected to an embedding operation and clustering is performed, but differently, an individual type can be generated by using meta information included in each of the training images. For example, when k is 1, that is, when only an individual type for one type corresponding to each of the training images is generated by one embedding operation, an individual type for a type different from the one type generated by the embedding operation can be further generated by using meta information of each of the training images.

[0066] That is, by checking the meta information included in each of the learning images and further referring to the shooting time included in each of the meta information, an individual type corresponding to each of the learning images can be generated.

[0067] For example, in order to generate individual types according to time period types such as day, night, evening, dawn, etc., the training data selection device 100 can group the training images according to preset time periods with reference to the shooting times of each of the training images, and set individual types related to the time periods for each group of grouped training images.

[0068] In addition, when generating individual types corresponding to each of the training images by further referring to the shooting time of each of the training images, the training data selection device 100 can also use only a specific embedding model, which is any one of the first embedding model to the kth embedding model, to generate an individual type corresponding to one of the types using the specific embedding model and an individual type corresponding to another type using meta information.

[0069] That is, the training data selection device 100 can perform a specific embedding operation on each of the training images to generate a specific scene vector corresponding to each of the training images, and can cluster the specific scene vector to generate a specific scene cluster. The training data selection device 100 can then check the shooting time of each of the training images by referring to the respective meta information included in each of the training images. Thereafter, the training data selection device 100 can also generate an individual type corresponding to each of the training images by referring to the specific scene cluster and the shooting time.

[0070] As yet another example, the training data selection device 100 may generate individual types corresponding to objects contained in an image, instead of generating various individual types corresponding to various type types for the entire image.

[0071] 4, the training data selection device 100 performs object detection 1 on each of the training images, which are all the training data, to detect at least one object from each of the training images, generates cropped images by cropping 2 regions corresponding to bounding boxes of each of the detected objects in each of the training images, performs embedding operation 10 on each of the cropped images to generate object vectors corresponding to each of the cropped images, clusters 20 the object vectors to generate object clusters, and generates individual types corresponding to the training images by referring to the object clusters. In this case, to perform object detection on each of the training images, the training data selection device 100 may use an open-world object detection model such as Grounding DINO, OWL-ViT (short for Vision Transformer for Open-World Localization), or YOLO (You Only Look Once)-World.

[0072] At this time, the training data selection device 100 can generate an individual type corresponding to the training image by further referring to the scene cluster generated based on the above description in addition to the object cluster.

[0073] That is, the training data selection device 100 can perform a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a kth embedding operation on each of the training images to generate a kth scene vector corresponding to each of the training images and clustering the kth scene vector to generate a kth scene cluster. The training data selection device 100 can perform object detection on each of the training images to detect at least one object from each of the training images, generate cropped images by cropping areas corresponding to bounding boxes of each of the detected objects in each of the training images, perform an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and cluster the object vectors to generate object clusters. Thereafter, the training data selection device 100 can generate individual types corresponding to the training images by referring to the first scene cluster to the kth scene cluster and the object cluster.

[0074] On the other hand, in the above, a cropped image is generated after performing object detection on each of the training images. However, in a case where each of the training images is labeled with its own ground truth, a cropped image can also be generated by referring to each of the ground truths of the training images.

[0075] In other words, the training data selection device 100 may refer to each of the ground truth information included in each of the training images, generate cropped images by cropping areas corresponding to the bounding boxes of each of the objects from each of the training images, perform an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, cluster the object vectors to generate object clusters, and generate individual types corresponding to the training images by referring to the object clusters.

[0076] At this time, the training data selection device 100 can generate an individual type corresponding to the training image by further referring to the scene cluster generated based on the above description in addition to the object cluster.

[0077] That is, the training data selection device 100 may perform a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a kth embedding operation on each of the training images to generate a kth scene vector corresponding to each of the training images and clustering the kth scene vector to generate a kth scene cluster. The training data selection device 100 may generate cropped images by cropping regions corresponding to bounding boxes of each object from each of the training images with reference to the respective ground truth information included in each of the training images, perform an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and cluster the object vectors to generate object clusters. Thereafter, the training data selection device 100 may generate individual types corresponding to the training images with reference to the first scene cluster to the kth scene cluster and the object cluster.

[0078] Next, referring back to FIG. 2, the training data selecting device 100 can generate a bipartite graph that matches each of a large number of training data included in the entire training data with an individual type (S120).

[0079] 5 as an example, it can be understood that the bipartite graph is a matching of the relationship between the training data group and the individual type group, and that training data 1 has individual type 1 and individual type 6, training data 2 has individual type 1, individual type 5, and individual type 6, training data 3 has individual type 2 and individual type 7, training data 4 has individual type 1, training data 5 has individual type 3, individual type 6, and individual type 8, training data 6 has individual type 4, and training data 7 has individual type 2, individual type 5, individual type 7, and individual type 8. Meanwhile, FIG. 5 exemplarily illustrates some training data and some individual types in the bipartite graph.

[0080] 2 again, the training data selection device 100 may select n pieces of training data for training a deep learning model such that the number of each of the individual types matching the n pieces of training data is within a predetermined threshold deviation when selecting n pieces of training data matching individual types from among all training data by referring to the bipartite graph (S130). The n is a target number of training data for training a deep learning model, and may be an integer representing a multiple number.

[0081] In this case, when the individual types include 1_1 individual types through 1_x individual types corresponding to the first type possessed by each of the multiple learning data, and 2_1 individual types through 2_y individual types corresponding to the second type possessed by each of the multiple learning data, the learning data selection device 100 can make the number of 1_1 individual types through 1_x individual types corresponding to the first type and the number of 2_1 individual types through 2_y individual types corresponding to the second type, which match the n learning data, within a threshold deviation, and in addition, can select the n learning data such that the number of 1_1 individual types through 1_x individual types is within a first threshold deviation and the number of 2_1 individual types through 2_y individual types is within a second threshold deviation. In other words, the learning data selection device 100 can select learning data such that the number of learning data for each individual type is as equal as possible for all individual types by type.

[0082] Meanwhile, the learning data selection device 100 can use at least a part of a heuristic algorithm, an optimization algorithm, etc., to select learning data for each individual type so that the number of learning data is as equal as possible.

[0083] First, the process in which the training data selection device 100 selects training data using a heuristic algorithm will be described below.

[0084] The training data selection device 100 can refer to the bipartite graph to check the number of corresponding individual types that match each of all the training data among the individual types, and select a specific training data with the largest number of corresponding individual types.

[0085] The learning data selection device 100 may then check the number of remaining corresponding individual types that match each of the entire learning data among the remaining individual types excluding the corresponding individual type, and select another specific learning data having the largest number of remaining corresponding individual types.

[0086] In this manner, the process of selecting the learning data having the largest number of remaining individual types from among the remaining individual types excluding the selected individual types is repeated until all individual types are selected, so that the learning data selection device 100 can perform a cycle of selecting a portion of learning data that matches all individual types.

[0087] Thereafter, the training data selection device 100 may repeat the process of performing the above cycle from the remaining training data, excluding a selected portion of the training data that matches all of the individual types, until n pieces of training data are selected.

[0088] As an example, referring to Figures 6a to 6d, in Figure 6a, since learning data 7 matches the four most common individual types, the learning data selection device 100 can select learning data 7 from learning data 1 to learning data 7.

[0089] Then, in the bipartite graph in Figure 6a, if we remove individual types 2, 5, 7, and 8 that match training data 7, we are left with individual types 1, 3, 4, and 6, as shown in Figure 6b.

[0090] 6b, since the learning data 1, the learning data 2, and the learning data 5 match the two most numerous individual types, the learning data selection device 100 can select the learning data 1, which is one of the learning data 1, the learning data 2, and the learning data 5. In this case, when there are a plurality of learning data with the most numerous corresponding individual types, such as the learning data 1, the learning data 2, and the learning data 5, the learning data selection device 100 can select the specific learning data, the learning data 1, by selecting one of the plurality of learning data, the learning data 1, the learning data 2, and the learning data 5, according to a first criterion. The first criterion can be set in various ways, such as selecting one of the plurality of learning data by a random selection algorithm, or selecting one of the plurality of learning data based on the sequential ranking of the plurality of learning data.

[0091] Then, in the bipartite graph of FIG. 6b, if we remove individual types 1 and 6 that match training data 1, individual types 3 and 4 remain, as shown in FIG. 6c.

[0092] Then, in FIG. 6c, since the learning data 5 and the learning data 6 are matched to the single individual type with the largest number, the learning data selection device 100 can select the learning data 5, which is one of the learning data 5 and the learning data 6. In this case, when there are a plurality of remaining learning data with the largest number of remaining corresponding individual types, such as the learning data 5 and the learning data 6, the learning data selection device 100 can select the learning data 5, which is the specific learning data, by selecting one of the plurality of remaining learning data, the learning data 5 and the learning data 6, according to the second criterion. The second criterion can be set in various ways, such as selecting one of the plurality of learning data by a random selection algorithm or selecting one of the plurality of learning data based on the sequential ranking of the plurality of learning data, and the second criterion can be set in the same manner as the first criterion described above, or can be set differently.

[0093] Then, in the bipartite graph of Figure 6c, if we remove individual type 3 that matches training data 5, individual type 4 remains, as shown in Figure 6d.

[0094] 6d, since learning data 6 matches the most common individual type, the learning data selection device 100 selects learning data 6, thereby making it possible to perform one cycle of selecting a portion of the selected learning data that matches all of the individual types. Thereafter, the learning data selection device 100 repeats the above cycle from the remaining learning data excluding learning data 1, learning data 5, and learning data 7, which are the portions of the learning data selected from all of the learning data, and can select n pieces of learning data, which is the desired number.

[0095] Meanwhile, in the above, the selection of the part of the learning data that matches all the individual types has been described, but the learning data can be selected by further referring to the type type to which each of the individual types belongs. That is, the part of the learning data that matches all the individual types that belong to all the type types can be selected. For example, when selecting the part of the learning data from the type types of the daytime and nighttime and the type types of the automobile and human objects, it is also possible to select the part of the learning data that matches evenly when matching the daytime individual types and the nighttime individual types, matches evenly when matching the automobile individual types and the human individual types in the daytime individual types, and matches evenly when matching both the automobile individual types and the human individual types in the nighttime individual types.

[0096] Next, the process in which the training data selection device 100 selects training data using an optimization algorithm will be described below.

[0097] The training data selection device 100 may repeat the process of selecting a specific subset having the smallest number of training data from among the subsets consisting of a predetermined number of training data including all individual types through an optimization algorithm with reference to the bipartite graph, calculating remaining training data excluding a specific number of training data included in the specific subset, and selecting at least one other specific subset consisting of a predetermined number of training data including all individual types through an optimization algorithm from the remaining training data, so that the number of selected training data is n or more. In this case, making the number of selected training data a desired number, n or more, does not mean that the number of selected training data is unconditionally n or more, but may mean that the total number of selected training data is not less than n depending on the number of the specific subsets finally selected. In other words, when the number of training data selected so far is less than n, and the total number of training data, calculated by adding up the number of specific subsets currently selected, is equal to or greater than n, the process of selecting training data is terminated, and the current total number of training data can be generated as a training dataset for training a deep learning model.

[0098] As an example, the training data selection device 100 may use linear programming as an optimization algorithm to calculate the product of a PxQ binary matrix corresponding to P individual types and Q training data in a bipartite graph and a Q-dimensional vector representing each selected fitness variable of the Q training data for each of the P individual types, and generate a P-dimensional vector having a fitness sum of 1 or more, and may select a specific subset including specific training data corresponding to a specific selection fitness variable having a minimum sum of the selected fitness variables from among the selected fitness variables of the Q-dimensional vectors whose selected fitness variables are 0 to 1, and may calculate remaining training data excluding the specific training data included in the specific subset from the Q training data. The P-dimensional vector may represent the fitness sum of the Q training data belonging to each of the P individual types. Then, the training data selection device 100 can repeat the process of selecting at least one other specific subset from the remaining training data using linear programming, until the number of selected training data becomes n or more.

[0099] In this case, the linear programming can be expressed as follows: Purpose: min 1 T x Condition: Ax ≧ 1, 0≦x≦1

[0100] In the above, A is a PxQ binary matrix, and A[j,i] represents whether training data i matches individual type j. x is a Q-dimensional vector, and x[i] represents the selection fitness variable of training data i. Ax is the matrix product of A and x, and Ax[j] can represent the sum of the fitness scores of the learning data belonging to individual type j.

[0101] Furthermore, the training data selection device 100 can select a specific subset using a dual linear programming method that applies at least one of merging, separating, and changing the sign of constraints in the linear programming method.

[0102] The dual linear programming method modified from the linear programming method is expressed as follows, but the present invention is not limited thereto, and the dual linear programming method can be applied by various methods of modifying the constraints in the linear programming method.

[0103] Purpose: min 1 T x Condition: Bx ≧ b, x ≧ 0

[0104] Then B=[A;-I], a (P+Q)xQ binary matrix, B[j,i]=A[j,i], B[P+i,i]= -1 for all i ∈ [1,2,···,Q], B[P+i,j]= 0 for all j≠i b=[1 P ;-1 P ], a (P+Q)-dimensional vector, b[j]= 1 for all j ∈ [1,2,···,P], b[P+i]= -1 for all i ∈ [1,2,···,Q] It can be expressed as follows.

[0105] As another example, the training data selection device 100 may use integer programming as an optimization algorithm to calculate a matrix product of a PxQ binary matrix corresponding to P individual types and Q training data in a bipartite graph and a Q-dimensional vector representing each selected variable of the Q training data for each of the P individual types, and the selected quantity in the P-dimensional vector is 1 or more, and among the selected variables of the Q-dimensional vector whose selected variables satisfy 0 or 1, the device may select a specific subset including specific training data corresponding to a specific selected variable having a minimum sum of the selected variables, and calculate remaining training data by excluding the specific training data included in the specific subset from the Q training data. The P-dimensional vector may represent the selected quantity of the training data belonging to each of the P individual types. The device 100 may repeat the process of selecting at least one other specific subset using integer programming for the remaining training data until the number of selected training data is n or more.

[0106] In this case, the integer programming can be expressed as follows: Purpose: min 1 T x Condition: Ax ≧ 1, x[i] ∈ {0,1} for all i

[0107] In the above, A is a PxQ binary matrix, and A[j,i] represents whether training data i matches individual type j. x is a Q-dimensional vector, and x[i] can represent the selection variable (1 or 0) of training data i. Ax is the matrix product of A and x, and Ax[j] can represent the selection quantity of learning data belonging to individual type j.

[0108] Furthermore, the training data selection device 100 can select a specific subset using a dual linear programming method that applies at least one of merging, separating, and changing the sign of constraints in the integer programming method.

[0109] According to the first embodiment of the present invention, it is possible to select training data for each type as evenly as possible for all individual types. In addition, if the ground truth is not labeled for all training data stored in the data pool, it is only necessary to label the ground truth for the training data selected as evenly as possible for each type, thereby reducing the cost of generating a training data set.

[0110] [Second Example] Fig. 7 is a diagram illustrating a method for selecting learning data for learning a deep learning model according to a second embodiment of the present invention, and the second embodiment of the present invention will be described below with reference to Fig. 7. In the following description, detailed description of parts that can be easily understood from the description of the first embodiment will be omitted.

[0111] First, the training data selection device 100 can acquire at least one individual type corresponding to each of a large number of training images stored in a data pool (S210).

[0112] In this case, the individual types may include 1_1 individual types through 1_x individual types corresponding to the first type of each of the multiple learning images, and 2_1 individual types through 2_y individual types corresponding to the second type of each of the multiple learning images. Each of the x and y may be an integer of 1 or more. However, the present invention is not limited thereto, and three or more types may be included according to the characteristics of the learning images.

[0113] For example, the information may be configured into various individual types corresponding to various type types, such as type types related to time periods corresponding to individual types such as day, night, evening, and dawn, type types related to weather corresponding to individual types such as sunny, cloudy, rainy, and fog, and type types related to locations corresponding to individual types such as city, countryside, and highway.

[0114] In addition, at least one individual type (attribute) corresponding to each of the multiple training images can be extracted automatically by labeling each of the multiple training images with a predefined type by a labeler, or by utilizing various techniques such as AI (Artificial Intelligence) and clustering.

[0115] As an example, the training data selection device 100 may transmit a number of training images to a labeler terminal, and generate at least one individual type corresponding to each of the number of training images using a labeler corresponding to the labeler terminal.

[0116] As another example, the training data selecting device 100 can automatically generate various individual patterns corresponding to various pattern types for the entire image that each of the multiple training images has.

[0117] That is, referring again to FIG. 3, the training data selection device 100 may perform a process of performing a first embedding operation 10_1 on each of the training images to generate a first scene vector corresponding to each of the training images and clustering 20 the first scene vector to generate a first scene cluster, or a process of performing a k-th embedding operation 10_k on each of the training images to generate a k-th scene vector corresponding to each of the training images and clustering 20 the k-th scene vector to generate a k-th scene cluster. The k may be an integer equal to or greater than 1. The training data selection device 100 may generate individual types corresponding to the training images by referring to the first scene cluster to the k-th scene cluster.

[0118] In this case, the training data selection device 100 may use visual foundation models such as CLIP (Contrastive Language-Image pre-training), DINO (self-DIstillation with NO labels), and DINOv2 for the first to k-th embedding operations, and the first to k-th embedding models for the first to k-th embedding operations may be different embedding models that generate scene vectors corresponding to the first to k-th types, which are different types. The training data selection device 100 may use various clustering algorithms such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise), K-Means, K-Medians, and Mean-Shift to cluster the first to k-th scene vectors.

[0119] In addition, in the above, in order to generate at least one individual type for each of the training images, each of the training images is subjected to an embedding operation and clustering is performed, but differently, an individual type can be generated by using meta information included in each of the training images. For example, when k is 1, that is, when only an individual type for one type corresponding to each of the training images is generated by one embedding operation, an individual type for a type different from the one type generated by the embedding operation can be further generated by using meta information of each of the training images.

[0120] That is, by checking the meta information included in each of the learning images and further referring to the shooting time included in each of the meta information, an individual type corresponding to each of the learning images can be generated.

[0121] For example, in order to generate individual types according to time period types such as day, night, evening, dawn, etc., the training data selection device 100 can group the training images according to preset time periods with reference to the shooting times of each of the training images, and set individual types related to the time periods for each group of grouped training images.

[0122] In addition, when generating individual types corresponding to each of the training images by further referring to the shooting time of each of the training images, the training data selection device 100 can also use only a specific embedding model, which is any one of the first embedding model to the kth embedding model, to generate an individual type corresponding to one of the types using the specific embedding model and an individual type corresponding to another type using meta information.

[0123] That is, the training data selection device 100 can perform a specific embedding operation on each of the training images to generate a specific scene vector corresponding to each of the training images, and can cluster the specific scene vector to generate a specific scene cluster. The training data selection device 100 can then check the shooting time of each of the training images by referring to the respective meta information included in each of the training images. Thereafter, the training data selection device 100 can also generate an individual type corresponding to each of the training images by referring to the specific scene cluster and the shooting time.

[0124] Next, referring back to FIG. 7, the training data selection device 100 can generate a bipartite graph that matches each of the multiple training images with an individual type (S220).

[0125] Next, the training data selection device 100 may execute a cycle of selecting a portion of training images that match all individual types from among a large number of training images by referring to the bipartite graph (S230).

[0126] In other words, the training data selection device 100 can refer to the bipartite graph to check the number of corresponding individual types that match each of the multiple training images among the individual types, and select a specific training image that has the largest number of corresponding individual types.

[0127] The learning data selection device 100 may then check the number of remaining corresponding individual types that match each of the multiple learning images among the remaining individual types excluding the corresponding individual types, and select another specific learning image having the largest number of remaining corresponding individual types.

[0128] In this manner, the process of selecting the learning image with the largest number of remaining individual types from among the remaining individual types excluding the selected individual types is repeated until all individual types are selected, so that the learning data selection device 100 can perform a cycle of selecting a portion of learning images that match all individual types.

[0129] In this case, when there are a plurality of training images having the largest number of corresponding individual types, the training data selection device 100 may select one of the plurality of training images according to a first criterion to select a specific training image, and when there are a plurality of remaining training images having the largest number of remaining corresponding individual types, the training data selection device 100 may select one of the plurality of remaining training images according to a second criterion to select another specific training image.

[0130] Thereafter, the training data selection device 100 may repeat the process of performing the above cycle from the remaining training images, excluding a portion of the training images selected that match all of the individual types, until n training images are selected (S240).

[0131] In this case, when the individual types include 1_1 individual types through 1_x individual types corresponding to the first type possessed by each of the multiple learning images, and 2_1 individual types through 2_y individual types corresponding to the second type possessed by each of the multiple learning images, the learning data selection device 100 can make the number of 1_1 individual types through 1_x individual types corresponding to the first type and the number of 2_1 individual types through 2_y individual types corresponding to the second type, which match the n learning images, within a threshold deviation, and in addition, can select the n learning images such that the number of 1_1 individual types through 1_x individual types is within a first threshold deviation and the number of 2_1 individual types through 2_y individual types is within a second threshold deviation. In other words, the learning data selection device 100 can select learning images such that the number of learning images for each individual type is as equal as possible for all individual types by type. For example, when selecting some learning images from daytime and nighttime time zone types and automobile and human object type types, it is possible to select some learning images that match evenly when matching daytime individual types and nighttime individual types, that match evenly when matching automobile individual types and human individual types in the daytime individual types, and that match evenly when matching both automobile individual types and human individual types in the nighttime individual types.

[0132] [Third Example] Fig. 8 is a diagram illustrating a method for selecting learning data for learning a deep learning model according to a third embodiment of the present invention, and the third embodiment of the present invention will be described below with reference to Fig. 8. In the following description, detailed description of parts that can be easily understood from the description of the first embodiment will be omitted.

[0133] First, the training data selection device 100 can acquire at least one individual type corresponding to each of a number of training images stored in a data pool (S310).

[0134] In this case, the individual types may include 1_1 individual types through 1_x individual types corresponding to the first type of each of the multiple learning images, and 2_1 individual types through 2_y individual types corresponding to the second type of each of the multiple learning images. Each of the x and y may be an integer of 1 or more. However, the present invention is not limited thereto, and three or more types may be included according to the characteristics of the learning images.

[0135] For example, the information may be configured into various individual types corresponding to various type types, such as type types related to time periods corresponding to individual types such as day, night, evening, and dawn, type types related to weather corresponding to individual types such as sunny, cloudy, rainy, and fog, and type types related to locations corresponding to individual types such as city, countryside, and highway.

[0136] In addition, at least one individual type (attribute) corresponding to each of the multiple training images can be extracted automatically by labeling each of the multiple training images with a predefined type by a labeler, or by utilizing various techniques such as AI (Artificial Intelligence) and clustering.

[0137] As an example, the training data selection device 100 may transmit a number of training images to a labeler terminal, and generate at least one individual type corresponding to each of the number of training images using a labeler corresponding to the labeler terminal.

[0138] As another example, the training data selecting device 100 can automatically generate various individual patterns corresponding to various pattern types for the entire image that each of the multiple training images has.

[0139] That is, referring again to FIG. 3, the training data selection device 100 may perform a process of performing a first embedding operation 10_1 on each of the training images to generate a first scene vector corresponding to each of the training images and clustering 20 the first scene vector to generate a first scene cluster, or a process of performing a k-th embedding operation 10_k on each of the training images to generate a k-th scene vector corresponding to each of the training images and clustering 20 the k-th scene vector to generate a k-th scene cluster. The k may be an integer equal to or greater than 1. The training data selection device 100 may generate individual types corresponding to the training images by referring to the first scene cluster to the k-th scene cluster.

[0140] In this case, the training data selection device 100 may use visual foundation models such as CLIP (Contrastive Language-Image pre-training), DINO (self-DIstillation with NO labels), and DINOv2 for the first to k-th embedding operations, and the first to k-th embedding models for the first to k-th embedding operations may be different embedding models that generate scene vectors corresponding to the first to k-th types, which are different types. The training data selection device 100 may use various clustering algorithms such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise), K-Means, K-Medians, and Mean-Shift to cluster the first to k-th scene vectors.

[0141] In addition, in the above, in order to generate at least one individual type for each of the training images, each of the training images is subjected to an embedding operation and clustering is performed, but differently, an individual type can be generated by using meta information included in each of the training images. For example, when k is 1, that is, when only an individual type for one type corresponding to each of the training images is generated by one embedding operation, an individual type for a type different from the one type generated by the embedding operation can be further generated by using meta information of each of the training images.

[0142] That is, by checking the meta information included in each of the learning images and further referring to the shooting time included in each of the meta information, an individual type corresponding to each of the learning images can be generated.

[0143] For example, in order to generate individual types according to time period types such as day, night, evening, dawn, etc., the training data selection device 100 can group the training images according to preset time periods with reference to the shooting times of each of the training images, and set individual types related to the time periods for each group of grouped training images.

[0144] In addition, when generating individual types corresponding to each of the training images by further referring to the shooting time of each of the training images, the training data selection device 100 can also use only a specific embedding model, which is any one of the first embedding model to the kth embedding model, to generate an individual type corresponding to one of the types using the specific embedding model and an individual type corresponding to another type using meta information.

[0145] That is, the training data selection device 100 can perform a specific embedding operation on each of the training images to generate a specific scene vector corresponding to each of the training images, and can cluster the specific scene vector to generate a specific scene cluster. The training data selection device 100 can then check the shooting time of each of the training images by referring to the respective meta information included in each of the training images. Thereafter, the training data selection device 100 can also generate an individual type corresponding to each of the training images by referring to the specific scene cluster and the shooting time.

[0146] Next, referring back to FIG. 8, the training data selecting device 100 can generate a bipartite graph that matches each of the multiple training images with an individual type (S320).

[0147] Next, the training data selection device 100 can refer to the bipartite graph and use an optimization algorithm to select a specific subset having the smallest number of training images from among the subsets consisting of a predetermined number of training images including all individual types, and calculate the remaining training images excluding a specific number of training images included in the specific subset (S330).

[0148] As an example, the training data selection device 100 may use linear programming as an optimization algorithm to calculate the product of a PxQ binary matrix corresponding to P individual types and Q training images in a bipartite graph and a Q-dimensional vector representing the selection fitness variables of the Q training images in each of the P individual types, and generate a P-dimensional vector having a total fitness of 1 or more, and may select a specific subset including a specific training image corresponding to a specific selection fitness variable having a minimum sum of the selection fitness variables from among the selection fitness variables of the Q-dimensional vectors whose selection fitness variables are 0 to 1, and may calculate the remaining training images excluding the specific training images included in the specific subset from the Q training images. The P-dimensional vector may represent the total fitness of the Q training images belonging to each of the P individual types.

[0149] In this case, the linear programming can be expressed as follows: Purpose: min 1 T x Condition: Ax ≧ 1, 0≦x≦1

[0150] In the above, A is a PxQ binary matrix, and A[j,i] represents whether training image i matches individual type j; x is a Q-dimensional vector, and x[i] can represent the selection fitness variable of training image i. Ax is the matrix product of A and x, and Ax[j] can represent the sum of the fitness scores of training images belonging to individual type j.

[0151] Furthermore, the training data selection device 100 can select a specific subset using a dual linear programming method that applies at least one of merging, separating, and changing the sign of constraints in the linear programming method.

[0152] The dual linear programming method modified from the linear programming method is expressed as follows, but the present invention is not limited thereto, and the dual linear programming method can be applied by various methods of modifying the constraints in the linear programming method.

[0153] Purpose: min 1 T x Condition: Bx ≧ b, x ≧ 0

[0154] Then B=[A;-I], a (P+Q)xQ binary matrix, B[j,i]=A[j,i], B[P+i,i]= -1 for all i ∈ [1,2,···,Q], B[P+i,j]= 0 for all j≠i b=[1 P ;-1 P ], a (P+Q)-dimensional vector, b[j]= 1 for all j ∈ [1,2,···,P], b[P+i]= -1 for all i ∈ [1,2,···,Q] It can be expressed as follows.

[0155] As another example, the training data selecting device 100 may use integer programming as an optimization algorithm to calculate a matrix product of a PxQ binary matrix corresponding to P individual types and Q training images in a bipartite graph and a Q-dimensional vector representing each selection variable of the Q training images in each of the P individual types, and generate a P-dimensional vector having a selection quantity of 1 or more, and may select a specific subset including a specific training image corresponding to a specific selection variable having a minimum sum of the selection variables from among the selection variables in the Q-dimensional vector whose selection variables satisfy 0 or 1, and may calculate remaining training images excluding the specific training images included in the specific subset from the Q training images. The P-dimensional vector may represent the selection quantity of training images belonging to each of the P individual types.

[0156] In this case, the integer programming can be expressed as follows: Purpose: min 1 T x Condition: Ax ≧ 1, x[i] ∈ {0,1} for all i

[0157] In the above, A is a PxQ binary matrix, and A[j,i] represents whether training image i matches individual type j; x is a Q-dimensional vector, and x[i] can represent the selection variable (1 or 0) of training image i. Ax is the matrix product of A and x, and Ax[j] can represent the selection quantity of training images belonging to individual type j.

[0158] Furthermore, the training data selection device 100 can select a specific subset using a dual linear programming method that applies at least one of merging, separating, and changing the sign of constraints in the integer programming method.

[0159] Thereafter, the training data selection device 100 may repeat the process of selecting at least one other specific subset consisting of a predetermined number of training images including all individual types through an optimization algorithm for the remaining training images until n training images for training the deep learning model are selected (S340).

[0160] In this case, when the individual types include 1_1 individual types through 1_x individual types corresponding to the first type possessed by each of the multiple learning images, and 2_1 individual types through 2_y individual types corresponding to the second type possessed by each of the multiple learning images, the learning data selection device 100 can make the number of 1_1 individual types through 1_x individual types corresponding to the first type and the number of 2_1 individual types through 2_y individual types corresponding to the second type, which match the n learning images, within a threshold deviation, and in addition, can select the n learning images such that the number of 1_1 individual types through 1_x individual types is within a first threshold deviation and the number of 2_1 individual types through 2_y individual types is within a second threshold deviation. In other words, the learning data selection device 100 can select learning images such that the number of learning images for each individual type is as equal as possible for all individual types by type. For example, when selecting some learning images from daytime and nighttime time zone types and automobile and human object type types, it is possible to select some learning images that match evenly when matching daytime individual types and nighttime individual types, that match evenly when matching automobile individual types and human individual types in the daytime individual types, and that match evenly when matching both automobile individual types and human individual types in the nighttime individual types.

[0161] [Fourth Example] Fig. 9 is a diagram illustrating a method for selecting learning data for learning a deep learning model according to a fourth embodiment of the present invention, and the fourth embodiment of the present invention will be described below with reference to Fig. 9. In the following description, detailed description of parts that can be easily understood from the description of the first embodiment will be omitted.

[0162] First, the training data selection device 100 can acquire at least one individual type (attribute) corresponding to at least one object included in each of a number of training images stored in a data pool (S410).

[0163] In this case, at least one individual type (attribute) corresponding to each object can be extracted by labeling each of the multiple training images with a predefined type by a labeler, or by automatically extracting the individual type present in each of the multiple training images using various techniques such as AI (Artificial Intelligence) and clustering.

[0164] As an example, the training data selection device 100 may transmit a number of training images to a labeler terminal, and generate at least one individual type corresponding to each of the objects included in each of the multiple training images using a labeler corresponding to the labeler terminal.

[0165] As another example, the training data selection device 100 can automatically generate at least one individual type corresponding to each of the objects included in each of the multiple training images.

[0166] 4 again, the training data selection device 100 may perform object detection 1 on each of the training images to detect at least one object from each of the training images, generate cropped images by cropping 2 regions corresponding to bounding boxes of each of the detected objects in each of the training images, perform embedding operation 10 on each of the cropped images to generate object vectors corresponding to each of the cropped images, cluster 20 the object vectors to generate object clusters, and generate individual types corresponding to the training images by referring to the object clusters. In this case, to perform object detection on each of the training images, the training data selection device 100 may use an open-world object detection model such as Grounding DINO, OWL-ViT (short for Vision Transformer for Open-World Localization), or YOLO (You Only Look Once)-World.

[0167] In this case, the training data selection device 100 can check each meta information included in each of the training images in addition to the object cluster, and further refer to each shooting time included in each meta information to generate an individual type corresponding to each of the training images.

[0168] On the other hand, in the above, a cropped image is generated after performing object detection on each of the training images. However, in a case where each of the training images is labeled with its own ground truth, a cropped image can also be generated by referring to each of the ground truths of the training images.

[0169] In other words, the training data selection device 100 may refer to each of the ground truth information included in each of the training images, generate cropped images by cropping areas corresponding to the bounding boxes of each of the objects from each of the training images, perform an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, cluster the object vectors to generate object clusters, and generate individual types corresponding to the training images by referring to the object clusters.

[0170] Next, referring back to FIG. 9, the training data selection device 100 can generate a bipartite graph that matches each of the multiple training images with an individual type (S420).

[0171] Next, the training data selection device 100 can refer to the bipartite graph and use an optimization algorithm to select a specific subset having the smallest number of training images from among the subsets consisting of a predetermined number of training images including all individual types, and calculate the remaining training images excluding a specific number of training images included in the specific subset (S430).

[0172] As an example, the training data selection device 100 may use linear programming as an optimization algorithm to calculate the product of a PxQ binary matrix corresponding to P individual types and Q training images in a bipartite graph and a Q-dimensional vector representing the selection fitness variables of the Q training images in each of the P individual types, and generate a P-dimensional vector having a total fitness of 1 or more, and may select a specific subset including a specific training image corresponding to a specific selection fitness variable having a minimum sum of the selection fitness variables from among the selection fitness variables of the Q-dimensional vectors whose selection fitness variables are 0 to 1, and may calculate the remaining training images excluding the specific training images included in the specific subset from the Q training images. The P-dimensional vector may represent the total fitness of the Q training images belonging to each of the P individual types.

[0173] In this case, the linear programming can be expressed as follows: Purpose: min 1 T x Condition: Ax ≧ 1, 0≦x≦1

[0174] In the above, A is a PxQ binary matrix, and A[j,i] represents whether training image i matches individual type j; x is a Q-dimensional vector, and x[i] can represent the selection fitness variable of training image i. Ax is the matrix product of A and x, and Ax[j] can represent the sum of the fitness scores of training images belonging to individual type j.

[0175] Furthermore, the training data selection device 100 can select a specific subset using a dual linear programming method that applies at least one of merging, separating, and changing the sign of constraints in the linear programming method.

[0176] The dual linear programming method modified from the linear programming method is expressed as follows, but the present invention is not limited thereto, and the dual linear programming method can be applied by various methods of modifying the constraints in the linear programming method.

[0177] Purpose: min 1 T x Condition: Bx ≧ b, x ≧ 0

[0178] Then B=[A;-I], a (P+Q)xQ binary matrix, B[j,i]=A[j,i], B[P+i,i]= -1 for all i ∈ [1,2,···,Q], B[P+i,j]= 0 for all j≠i b=[1 P ;-1 P ], a (P+Q)-dimensional vector, b[j]= 1 for all j ∈ [1,2,···,P], b[P+i]= -1 for all i ∈ [1,2,···,Q] It can be expressed as follows.

[0179] As another example, the training data selecting device 100 may use integer programming as an optimization algorithm to calculate a matrix product of a PxQ binary matrix corresponding to P individual types and Q training images in a bipartite graph and a Q-dimensional vector representing each selection variable of the Q training images in each of the P individual types, and generate a P-dimensional vector having a selection quantity of 1 or more, and may select a specific subset including a specific training image corresponding to a specific selection variable having a minimum sum of selection variables from among the selection variables of the Q-dimensional vector whose selection variables satisfy 0 or 1, and may calculate remaining training images excluding the specific training images included in the specific subset from the Q training images. The P-dimensional vector may represent the selection quantity of training images belonging to each of the P individual types.

[0180] In this case, the integer programming can be expressed as follows: Purpose: min 1 T x Condition: Ax ≧ 1, x[i] ∈ {0,1} for all i

[0181] In the above, A is a PxQ binary matrix, and A[j,i] represents whether training image i matches individual type j; x is a Q-dimensional vector, and x[i] can represent the selection variable (1 or 0) of training image i. Ax is the matrix product of A and x, and Ax[j] can represent the selection quantity of training images belonging to individual type j.

[0182] Furthermore, the training data selection device 100 can select a specific subset using a dual linear programming method that applies at least one of merging, separating, and changing the sign of constraints in the integer programming method.

[0183] Thereafter, the training data selection device 100 may repeat the process of selecting at least one other specific subset consisting of a predetermined number of training images including all individual types through an optimization algorithm for the remaining training images until n training images for training the deep learning model are selected (S440).

[0184] The above-described embodiments of the present invention may be realized in the form of program instructions that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and constructed for the present invention, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, and flash memories. Examples of program instructions include not only machine code, such as produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform the processes of the present invention, and vice versa.

[0185] Although the present invention has been described above by using specific details such as concrete components and limited examples and drawings, this is merely provided to facilitate a more comprehensive understanding of the present invention, and the present invention is not limited to the above examples. Those skilled in the art will appreciate that various modifications and variations can be made from such descriptions.

[0186] Therefore, the spirit of the present invention should not be limited to the embodiments described above, and all modifications equivalent to or equivalent to the scope of the claims below, as well as the scope of the spirit of the present invention, are intended to fall within the scope of the spirit of the present invention. [Explanation of symbols]

[0187] 100: Learning data selection device 110:Memory 120: Processor

Claims

1. A method for selecting training data for training a deep learning model, comprising: (a) a training data selection device acquires at least one individual type (attribute) corresponding to each of a large number of training data included in all training data stored in a data pool, and generates a bipartite graph by matching each of the large number of training data included in the all training data with the individual type; and (b) when the training data selection device selects n pieces of training data (where n is a target number of pieces of training data for training a deep learning model and is an integer representing a multiple number) matching the individual types from among all the training data with reference to the bipartite graph, selecting the n pieces of training data for training the deep learning model such that the number of each of the individual types matching the n pieces of training data falls within a predetermined threshold deviation; Including, In the step (b), the individual types include 1_1 individual types through 1_x individual types (where x is an integer of 1 or more) corresponding to a first type possessed by each of the plurality of learning data, and 2_1 individual types through 2_y individual types (where y is an integer of 1 or more) corresponding to a second type possessed by each of the plurality of learning data, The learning data selection device selects the n learning data such that the number of the 1_1 individual type through the 1_x individual type corresponding to the first type and the number of the 2_1 individual type through the 2_y individual type corresponding to the second type, which are matched with the n learning data, are within the threshold deviation, the number of the 1_1 individual type through the 1_x individual type are within a first threshold deviation, and the number of the 2_1 individual type through the 2_y individual type are within a second threshold deviation. method.

2. A method for selecting training data for training a deep learning model, comprising: (a) a training data selection device acquires at least one individual type (attribute) corresponding to each of a large number of training data included in all training data stored in a data pool, and generates a bipartite graph by matching each of the large number of training data included in the all training data with the individual type; and (b) when the training data selection device selects n pieces of training data (where n is a target number of pieces of training data for training a deep learning model and is an integer representing a multiple number) matching the individual types from among all the training data with reference to the bipartite graph, selecting the n pieces of training data for training the deep learning model such that the number of each of the individual types matching the n pieces of training data falls within a predetermined threshold deviation; Including, In the step (b), the training data selection device repeats the steps of (i) checking, with reference to the bipartite graph, the number of corresponding individual types among the individual types that match each of the all training data, and selecting specific training data having the largest number of corresponding individual types; (ii) checking, with reference to the bipartite graph, the number of corresponding individual types among the remaining individual types excluding the corresponding individual types, that match each of the all training data, and selecting another specific training data having the largest number of corresponding individual types, thereby performing a cycle of selecting a portion of training data that matches all of the individual types; and repeating the process of performing the cycle from the remaining training data excluding the portion of training data selected from the all training data, until the n training data are selected. method.

3. A method for selecting training data for training a deep learning model, comprising: (a) a training data selection device acquires at least one individual type (attribute) corresponding to each of a large number of training data included in all training data stored in a data pool, and generates a bipartite graph by matching each of the large number of training data included in the all training data with the individual type; and (b) when the training data selection device selects n pieces of training data (where n is a target number of pieces of training data for training a deep learning model and is an integer representing a multiple number) matching the individual types from among all the training data with reference to the bipartite graph, selecting the n pieces of training data for training the deep learning model such that the number of each of the individual types matching the n pieces of training data falls within a predetermined threshold deviation; Including, In the step (b), The learning data selection device comprises: selecting a specific subset having the smallest number of pieces of learning data from among the subsets including a predetermined number of pieces of learning data including all of the individual types through an optimization algorithm by referring to the bipartite graph, and calculating remaining learning data excluding a specific number of pieces of learning data included in the specific subset; repeating the process of selecting at least one other specific subset consisting of a predetermined number of learning data including all of the individual types through the optimization algorithm for the remaining learning data, so that the number of selected learning data becomes equal to or greater than n; method.

4. In the step (b), The learning data selection device performs linear programming. the method of claim 3, further comprising: calculating a matrix product of a P×Q binary matrix corresponding to P individual types and Q learning data in the bipartite graph using a linear programming method and a Q-dimensional vector representing a selection fitness variable of each of the Q learning data for each of the P individual types, the P-dimensional vector representing a total fitness of the Q learning data belonging to each of the P individual types, the total fitness being 1 or more, and the selection fitness variable in the Q-dimensional vector being between 0 and 1, selecting the specific subset including the specific learning data corresponding to a specific selection fitness variable having a minimum sum of the selection fitness variables from among the selection fitness variables of the Q-dimensional vector, calculating the remaining learning data by excluding the specific learning data included in the specific subset from the Q learning data, and repeating the process of selecting the at least one other specific subset using the linear programming method for the remaining learning data until the number of selected learning data is equal to or greater than n.

5. In the step (b), The learning data selection device uses linear programming. The method of claim 4, further comprising: selecting the particular subset using a dual linear programming method with at least one of merging, separating, and changing sign of constraints in the approximation.

6. In the step (b), The learning data selection device uses integer programming. a P-dimensional vector (the P-dimensional vector represents a selection quantity of learning data belonging to each of the P individual types) generated by calculating a matrix product of a P×Q binary matrix corresponding to P individual types and Q learning data in the bipartite graph and a Q-dimensional vector representing each selection variable of the Q learning data in each of the P individual types using the above formula, the selection quantity in the P-dimensional vector being 1 or more, and the selection variable in the Q-dimensional vector being 0 or 1, the specific subset including specific learning data corresponding to a specific selection variable having a minimum sum of the selection variables is selected from among the selection variables of the Q-dimensional vector, and the remaining learning data is calculated by excluding the specific learning data included in the specific subset from the Q learning data; The method of claim 3 , further comprising repeating the process of selecting another specific subset from the remaining training data using the integer programming method until the number of selected training data becomes equal to or greater than n.

7. In the step (b), The learning data selection device is a The method of claim 6 , further comprising: selecting the particular subset using a dual integer programming method that applies at least one of merging, separating, and changing the sign of the constraints in g).

8. In the step (a), 2. The method of claim 1, wherein the training data selection device transmits the plurality of training data to a labeler terminal, and generates the at least one individual type corresponding to each of the plurality of training data by a labeler corresponding to the labeler terminal.

9. A method for selecting training data for training a deep learning model, comprising: (a) a training data selection device acquires at least one individual type (attribute) corresponding to each of a large number of training data included in all training data stored in a data pool, and generates a bipartite graph by matching each of the large number of training data included in the all training data with the individual type; and (b) when the training data selection device selects n pieces of training data (where n is a target number of pieces of training data for training a deep learning model and is an integer representing a multiple number) matching the individual types from among all the training data with reference to the bipartite graph, selecting the n pieces of training data for training the deep learning model such that the number of each of the individual types matching the n pieces of training data falls within a predetermined threshold deviation; Including, The entire training data is training images, In the step (a), The training data selection device performs a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images, and clustering the first scene vector to generate a first scene cluster, or a process of performing a k-th embedding operation (where k is an integer equal to or greater than 1) on each of the training images to generate a k-th scene vector corresponding to each of the training images, and clustering the k-th scene vector to generate a k-th scene cluster, and generates the individual type corresponding to the training image by referring to the first scene cluster to the k-th scene cluster. method.

10. The entire training data is training images, In the step (a), 2. The method of claim 1 , wherein the training data selection device performs object detection on each of the training images to detect at least one object from each of the training images, generates cropped images by cropping areas corresponding to bounding boxes of each of the detected objects in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, clusters the object vectors to generate object clusters, and generates the individual types corresponding to the training images by referring to the object clusters.

11. The entire training data is training images, In the step (a), 2. The method of claim 1 , wherein the training data selection device generates cropped images by cropping regions corresponding to bounding boxes of respective objects from each of the training images with reference to respective ground truth information included in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, clusters the object vectors to generate object clusters, and generates the individual types corresponding to the training images with reference to the object clusters.

12. A method for selecting training data for training a deep learning model, comprising: (a) a training data selection device acquires at least one individual type (attribute) corresponding to each of a large number of training data included in all training data stored in a data pool, and generates a bipartite graph by matching each of the large number of training data included in the all training data with the individual type; and (b) when the training data selection device selects n pieces of training data (where n is a target number of pieces of training data for training a deep learning model and is an integer representing a multiple number) matching the individual types from among all the training data with reference to the bipartite graph, selecting the n pieces of training data for training the deep learning model such that the number of each of the individual types matching the n pieces of training data falls within a predetermined threshold deviation; Including, The entire training data is training images, In the step (a), The training data selection device (i) performs a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a kth embedding operation (where k is an integer equal to or greater than 1) on each of the training images to generate a kth scene vector corresponding to each of the training images and clustering the kth scene vector to generate a kth scene cluster; (ii) performs object detection on each of the training images to detect at least one object from each of the training images, generates cropped images by cropping regions corresponding to bounding boxes of each of the detected objects in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and clusters the object vectors to generate object clusters; and (iii) generates the individual type corresponding to the training image by referring to the first scene cluster through the kth scene cluster and the object cluster. method.

13. A method for selecting training data for training a deep learning model, comprising: (a) a training data selection device acquires at least one individual type (attribute) corresponding to each of a large number of training data included in all training data stored in a data pool, and generates a bipartite graph by matching each of the large number of training data included in the all training data with the individual type; and (b) when the training data selection device selects n pieces of training data (where n is a target number of pieces of training data for training a deep learning model and is an integer representing a multiple number) matching the individual types from among all the training data with reference to the bipartite graph, selecting the n pieces of training data for training the deep learning model such that the number of each of the individual types matching the n pieces of training data falls within a predetermined threshold deviation; Including, The entire training data is training images, In the step (a), The training data selection device (i) performs a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a kth embedding operation (where k is an integer equal to or greater than 1) on each of the training images to generate a kth scene vector corresponding to each of the training images and clustering the kth scene vector to generate a kth scene cluster; (ii) generates cropped images by cropping regions corresponding to bounding boxes of respective objects from each of the training images with reference to respective ground truth information included in each of the training images, performs an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and clusters the object vectors to generate object clusters; and (iii) generates the individual type corresponding to the training image with reference to the first scene cluster through the kth scene cluster and the object cluster. method.

14. A learning data selection device that selects learning data for learning a deep learning model, A memory having instructions for selecting training data for training a deep learning model; and a processor that performs operations in accordance with the instructions stored in the memory to select training data for training the deep learning model; Including, The processor executes a process of (I) acquiring at least one individual type (attribute) corresponding to each of a large number of learning data included in all learning data stored in a data pool, and generating a bipartite graph in which each of the large number of learning data included in all learning data is matched with the individual type, and (II) selecting n pieces of learning data (n is a target number of learning data for training the deep learning model and is an integer representing a multiple number) matching the individual type from all learning data by referring to the bipartite graph, such that the number of each of the individual types matching the n pieces of learning data is within a predetermined threshold deviation, for training the deep learning model; the individual types include 1_1 individual types through 1_x individual types (where x is an integer of 1 or more) corresponding to a first type possessed by each of the plurality of learning data, and 2_1 individual types through 2_y individual types (where y is an integer of 1 or more) corresponding to a second type possessed by each of the plurality of learning data, The processor further comprises: In the process (II), the n learning data are selected such that the number of the 1_1 individual type through the 1_x individual type corresponding to the first type and the number of the 2_1 individual type through the 2_y individual type corresponding to the second type, which are matched with the n learning data, are within the threshold deviation, the number of the 1_1 individual type through the 1_x individual type are within a first threshold deviation, and the number of the 2_1 individual type through the 2_y individual type are within a second threshold deviation. Training data selection device.

15. A learning data selection device that selects learning data for training a deep learning model, comprising: A memory having instructions for selecting training data for training a deep learning model; and a processor that performs operations in accordance with the instructions stored in the memory to select training data for training the deep learning model; Including, The processor executes a process of (I) acquiring at least one individual type (attribute) corresponding to each of a large number of learning data included in all learning data stored in a data pool, and generating a bipartite graph in which each of the large number of learning data included in all learning data is matched with the individual type, and (II) selecting n pieces of learning data (n is a target number of learning data for training the deep learning model and is an integer representing a multiple number) matching the individual type from all learning data by referring to the bipartite graph, such that the number of each of the individual types matching the n pieces of learning data is within a predetermined threshold deviation, for training the deep learning model; The processor further comprises: In the process (II), (i) referring to the bipartite graph, among the individual types, the number of corresponding individual types that match with each of the all learning data is calculated. (ii) confirming the number of the remaining corresponding individual types that match each of the entire learning data among the remaining individual types excluding the corresponding individual types, and selecting another specific learning data having the largest number of the remaining corresponding individual types, a cycle is executed to select a portion of the learning data that matches all of the individual types, and the process of executing the cycle from the remaining learning data excluding the portion of the learning data selected from the entire learning data is repeated until the n number of learning data is selected. Training data selection device.

16. A learning data selection device that selects learning data for training a deep learning model, comprising: A memory having instructions for selecting training data for training a deep learning model; and a processor that performs operations in accordance with the instructions stored in the memory to select training data for training the deep learning model; Including, The processor executes a process of (I) acquiring at least one individual type (attribute) corresponding to each of a large number of learning data included in all learning data stored in a data pool, and generating a bipartite graph in which each of the large number of learning data included in all learning data is matched with the individual type, and (II) selecting n pieces of learning data (n is a target number of learning data for training the deep learning model and is an integer representing a multiple number) matching the individual type from all learning data by referring to the bipartite graph, such that the number of each of the individual types matching the n pieces of learning data is within a predetermined threshold deviation, for training the deep learning model; The processor further comprises: In the process (II), by referring to the bipartite graph, a specific subset having the smallest number of pieces of learning data is selected from among the subsets consisting of a predetermined number of pieces of learning data including all of the individual types through an optimization algorithm, and a remaining amount of learning data is calculated excluding a specific number of pieces of learning data included in the specific subset; repeating the process of selecting at least one other specific subset consisting of a predetermined number of learning data including all of the individual types through the optimization algorithm for the remaining learning data, so that the number of selected learning data becomes equal to or greater than n; Training data selection device.

17. The processor, In the process (II), linear programming ), a P-dimensional vector (the P-dimensional vector represents a sum of the fitnesses of the Q learning data belonging to each of the P individual types) is generated by calculating a matrix product of a P×Q binary matrix corresponding to the P individual types and the Q learning data in the bipartite graph and a Q-dimensional vector representing a selection fitness variable of each of the Q learning data in each of the P individual types using the above-mentioned method, the sum of the fitnesses being 1 or more, and the selection fitness variable in the Q-dimensional vector being 0 or more and 1 or less, among the selection fitness variables of the Q-dimensional vector, the specific subset including the specific learning data corresponding to the specific selection fitness variable having a minimum sum of the selection fitness variables is selected, and the remaining learning data is calculated by excluding the specific learning data included in the specific subset from the Q learning data; The training data selection device according to claim 16, further comprising: repeating the process of selecting the at least one other specific subset using the linear programming method for the remaining training data, so that the number of selected training data becomes equal to or greater than n.

18. The processor, In the process (II), linear programming 18. The apparatus of claim 17, wherein the particular subset is selected using a dual linear programming method that applies at least one of merging, separating, and sign changing constraints in (a).

19. The processor, In the process (II), integer programming is used. g), a P×Q binary matrix corresponding to the P individual types and the Q learning data in the bipartite graph is multiplied by a Q-dimensional vector representing a selection variable of each of the Q learning data in each of the P individual types to generate a P-dimensional vector (the P-dimensional vector represents a selection quantity of learning data belonging to each of the P individual types), and the selection quantity in the P-dimensional vector is 1 or more, and the selection variable in the Q-dimensional vector satisfies 0 or 1. selecting the specific subset including specific learning data corresponding to a specific selection variable having a minimum sum of the selection variables from among the selection variables of the Q-dimensional vector, and calculating the remaining learning data by excluding the specific learning data included in the specific subset from the Q pieces of learning data; The training data selection device according to claim 16, further comprising: repeating the process of selecting another specific subset from the remaining training data using the integer programming until the number of selected training data becomes equal to or greater than n.

20. The processor, In the process (II), the integer programming method 20. The apparatus of claim 19, wherein the particular subset is selected using a dual integer programming method that applies at least one of merging, separating, and sign changing constraints in (a) and (b).

21. The processor, 15. The training data selection device according to claim 14, wherein in the process (I), the plurality of training data are transmitted to a labeler terminal, and a labeler corresponding to the labeler terminal is used to generate the at least one individual type corresponding to each of the plurality of training data.

22. A learning data selection device that selects learning data for training a deep learning model, comprising: A memory having instructions for selecting training data for training a deep learning model; and a processor that performs operations in accordance with the instructions stored in the memory to select training data for training the deep learning model; Including, The processor executes a process of (I) acquiring at least one individual type (attribute) corresponding to each of a large number of learning data included in all learning data stored in a data pool, and generating a bipartite graph in which each of the large number of learning data included in all learning data is matched with the individual type, and (II) selecting n pieces of learning data (n is a target number of learning data for training the deep learning model and is an integer representing a multiple number) matching the individual type from all learning data by referring to the bipartite graph, such that the number of each of the individual types matching the n pieces of learning data is within a predetermined threshold deviation, for training the deep learning model; The entire training data is training images, The processor, In the process (I), a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster, or a process of performing a k-th embedding operation (where k is an integer equal to or greater than 1) on each of the training images to generate a k-th scene vector corresponding to each of the training images and clustering the k-th scene vector to generate a k-th scene cluster, and generating the individual type corresponding to the training image by referring to the first scene cluster to the k-th scene cluster. Training data selection device.

23. The entire training data is training images, The processor, 15. The training data selection device of claim 14, wherein in the process (I), object detection is performed on each of the training images to detect at least one object from each of the training images, cropping areas corresponding to bounding boxes of each of the detected objects in each of the training images to generate cropped images, performing an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, clustering the object vectors to generate object clusters, and generating the individual types corresponding to the training images by referring to the object clusters.

24. The entire training data is training images, The processor, In the process (I), by referring to the ground truth information included in each of the training images, a cropped image is generated by cropping an area corresponding to a bounding box of each object from each of the training images, an embedding operation is performed on each of the cropped images to generate an object vector corresponding to each of the cropped images, the object vectors are clustered to generate an object cluster, and a front-end image corresponding to the training image is generated by referring to the object cluster. The training data selection device according to claim 14, further comprising: a processor for generating the individual patterns.

25. A learning data selection device that selects learning data for training a deep learning model, comprising: A memory having instructions for selecting training data for training a deep learning model; and a processor that performs operations in accordance with the instructions stored in the memory to select training data for training the deep learning model; Including, The processor executes a process of (I) acquiring at least one individual type (attribute) corresponding to each of a large number of learning data included in all learning data stored in a data pool, and generating a bipartite graph in which each of the large number of learning data included in all learning data is matched with the individual type, and (II) selecting n pieces of learning data (n is a target number of learning data for training the deep learning model and is an integer representing a multiple number) matching the individual type from all learning data by referring to the bipartite graph, such that the number of each of the individual types matching the n pieces of learning data is within a predetermined threshold deviation, for training the deep learning model; The entire training data is training images, The processor further comprises: In the process (I), (i) a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster or a process of performing a kth embedding operation (where k is an integer equal to or greater than 1) on each of the training images to generate a kth scene vector corresponding to each of the training images and clustering the kth scene vector to generate a kth scene cluster; (ii) performing object detection on each of the training images to detect at least one object from each of the training images, generating cropped images by cropping regions corresponding to bounding boxes of each of the detected objects in each of the training images, performing an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and clustering the object vectors to generate object clusters; and (iii) generating the individual type corresponding to the training image by referring to the first scene cluster through the kth scene cluster and the object cluster. Training data selection device.

26. A learning data selection device that selects learning data for training a deep learning model, comprising: A memory having instructions for selecting training data for training a deep learning model; and a processor that performs operations in accordance with the instructions stored in the memory to select training data for training the deep learning model; Including, The processor executes a process of (I) acquiring at least one individual type (attribute) corresponding to each of a large number of learning data included in all learning data stored in a data pool, and generating a bipartite graph in which each of the large number of learning data included in all learning data is matched with the individual type, and (II) selecting n pieces of learning data (n is a target number of learning data for training the deep learning model and is an integer representing a multiple number) matching the individual type from all learning data by referring to the bipartite graph, such that the number of each of the individual types matching the n pieces of learning data is within a predetermined threshold deviation, for training the deep learning model; The entire training data is training images, The processor further comprises: In the process (I), (i) a process of performing a first embedding operation on each of the training images to generate a first scene vector corresponding to each of the training images and clustering the first scene vector to generate a first scene cluster or a process of performing a kth embedding operation (where k is an integer equal to or greater than 1) on each of the training images to generate a kth scene vector corresponding to each of the training images and clustering the kth scene vector to generate a kth scene cluster; (ii) generating cropped images by cropping regions corresponding to bounding boxes of respective objects from each of the training images with reference to respective ground truth information included in each of the training images, performing an embedding operation on each of the cropped images to generate object vectors corresponding to each of the cropped images, and clustering the object vectors to generate object clusters; and (iii) generating the individual type corresponding to the training image with reference to the first scene cluster through the kth scene cluster and the object cluster. Training data selection device.

Images