Method for generating training data to be used for training artificial intelligence and training data generating device using the same

Abstract

The present invention relates to a method for generating a training data set for artificial intelligence learning, comprising: (a) when a pseudo-annotation corresponding to at least one training image having at least one initial ground truth annotation included in a public data set—said that the pseudo-annotation is an image caption that captions the training image—is obtained, a training data set generating device parses noun phrases from the pseudo-annotation and extracts nouns corresponding to physical objects from the parsed noun phrases; (b) the training data generating device detects bounding boxes of objects corresponding to the nouns in the training image, generates additional ground truth annotations by referencing each of the detected bounding boxes and each of the nouns corresponding thereto, and adds the additional ground truth annotations to the training image; and (c) the training data generating device generates final ground truth annotations corresponding to the training image by referencing the initial ground truth annotations and the additional ground truth annotations. It concerns a method including.

Description

Technology Field

[0001] The present invention relates to generating a training data set for artificial intelligence learning, and more specifically, to a method for generating a training data set for artificial intelligence learning that utilizes a collected public data set to effectively expand the number of ground truth annotations and to set category names more accurately, and to a training data generation device using the same. Background Technology

[0002] Artificial intelligence (AI) models process and analyze vast amounts of data to make better decisions and generate accurate predictions, and with technological advancements, AI models are being used in various fields.

[0003] Furthermore, to accurately train an AI model, a training dataset covering all possible scenarios is required.

[0004] To generate such training data sets, public data sets are conventionally utilized, such as data sets collected through web crawling and data sets built in advance as Intellectual Property (IP).

[0005] However, while web-crawled datasets cover general domains, they have the disadvantage that annotations for some objects are missing or not detailed.

[0006] Furthermore, while datasets constructed in advance as IPs cover detailed domains, they have the disadvantage that annotated category names are abstract. For example, drivers, pedestrians, and police officers might be annotated simply as "people."

[0007] In addition, datasets constructed in advance as IPs have the disadvantage of containing typos in some annotated category names.

[0008] Accordingly, when training an AI model using a training data set generated by conventional methods, the loss computed during the AI ​​model's training process is inaccurate, and consequently, a problem arises where the AI ​​model's training is not performed accurately.

[0009] For example, in training a Vision Language Model (VLM), a loss for visual grounding must be calculated by providing category names as text prompts, and a contrastive loss for alignment between multiple modals in the embedding space must be calculated to support multimodal prompts such as text, vision, and audio. However, if there is an error in the category names included in the ground truth annotation, a problem arises in which the trained VLM not only fails to accurately detect the object corresponding to the input text but also fails to accurately recognize the correspondence relationship between multimodal inputs.

[0010] Accordingly, the applicant intends to propose a method that utilizes collected public data sets to effectively expand the number of ground truth annotations and enable more accurate setting of category names. The problem to be solved

[0011] The present invention aims to solve all the problems of the aforementioned prior art.

[0012] In addition, the present invention has another objective of enabling the effective expansion of the number of ground truth annotations by utilizing public data sets.

[0013] In addition, another objective of the present invention is to enable the accurate generation of annotations for objects that are not included in the ground truth annotations of a public data set.

[0014] In addition, the present invention has another objective of making the category names of ground truth annotations added to a public dataset correspond to physical objects.

[0015] In addition, another objective of the present invention is to prevent errors in the category names of ground truth annotations added to public datasets. means of solving the problem

[0016] A representative configuration of the present invention for achieving the above objective is as follows.

[0017] According to one embodiment of the present invention, a method for generating a training data set for artificial intelligence learning comprises: (a) when a pseudo-annotation corresponding to at least one training image having at least one initial ground truth annotation included in a public data set—said that the pseudo-annotation is an image caption that captions the training image—is obtained, a training data set generating device parses noun phrases from the pseudo-annotation and extracts nouns corresponding to physical objects from the parsed noun phrases; (b) the training data generating device detects bounding boxes of objects corresponding to the nouns in the training image, generates additional ground truth annotations by referencing each of the detected bounding boxes and each of the nouns corresponding thereto, and adds the additional ground truth annotations to the training image; and (c) the step of the training data generating device generating final ground truth annotations corresponding to the training image by referring to the initial ground truth annotations and the additional ground truth annotations; a method is provided comprising:

[0018] In the above embodiment, in step (a), the learning data generating device can extract candidate nouns by performing at least one of the following sub-processes: (i) changing the part of speech of the noun phrases through a spaCy model, a sub-process removing quantifiers and ordinal numbers from the noun phrases, and a sub-process lemmatizing the nouns in the noun phrases; and (ii) extracting the nouns corresponding to the physical object among the candidate nouns by querying the identifier of WordNet or Wikidata for each of the candidate nouns to check whether they correspond to the physical object noun.

[0019] In the above embodiment, in step (a), the training data generating device can determine whether the specific candidate noun corresponds to the physical object by checking whether the physical object noun is included in the hypernym corresponding to the specific candidate noun in the WordNet for any one of the candidate nouns, or by traversing the instance and subclass corresponding to the identifier of the specific candidate noun in the Wikidata by a preset depth and checking whether at least one preset noun identifier corresponding to the physical object noun appears.

[0020] In the above embodiment, in step (a), the training data generating device may input the specific candidate noun into a Large Language Model (LM) to cause the LLM to check whether the physical object noun is included in the hypernym corresponding to the specific candidate noun in the WordNet, or may traverse instances and subclasses corresponding to the identifier of the specific candidate noun in the Wikidata by a preset depth to check whether at least one preset noun identifier corresponding to the physical object noun appears.

[0021] In the above embodiment, in step (c), the training data generating device checks whether each of the initial category names included in the initial ground truth annotations and the additional category names included in the additional ground truth annotations is a word existing in a preset dictionary, and if a specific category name, which is one of the initial category names and the additional category names, is a word existing in the preset dictionary, the specific category name is maintained as is, and if the specific category name is not a word existing in the preset dictionary, the specific category name can be changed using a general word within a preset edit distance from the specific category name through a SymSpell algorithm.

[0022] In the above embodiment, in step (c), the training data generating device may perform at least one of the following sub-processes in determining whether each of the initial category names and the additional category names is a word existing in the dictionary data: a sub-process determining whether the specific category name is found in a word set generated by referencing the dictionary data; a sub-process determining whether the sentence set corresponding to the specific category name in WordNet is empty; and a sub-process determining whether the specific category name is found in WordFreg.

[0023] In the above embodiment, in step (b), the training data generating device may generate the additional ground truth annotation through at least one of the following: a sub-process for filtering bounding boxes among the detected bounding boxes that have a size smaller than a preset threshold size by referring to each of the detected bounding boxes and each of the corresponding nouns; a sub-process for filtering bounding boxes among the detected bounding boxes that have a prediction confidence value lower than a preset threshold prediction confidence value; a sub-process for filtering bounding boxes among the detected bounding boxes that have a text-region alignment score lower than a preset threshold text-region alignment score; a sub-process for filtering bounding boxes that overlap among the detected bounding boxes through NMS (Non-Maximum Suppression); and a sub-process for removing nouns among the nouns that correspond to objects not present in the training image.

[0024] In the above embodiment, in step (a), the training data generating device can obtain the remaining candidate images as training images by filtering out images having a resolution lower than a preset threshold resolution, images tagged with NSFW (No Safe For Work), and low-content web images from among the candidate images included in the public data set.

[0025] In the above embodiment, in step (b), the learning data generating device may generate object masks from each of the detected bounding boxes and generate additional ground truth annotations by referencing each of the object masks and each of the nouns.

[0026] In addition, according to one embodiment of the present invention, a learning data generation device for generating a learning data set for artificial intelligence learning comprises: a memory in which instructions for generating a learning data set for artificial intelligence learning are stored; and a processor for generating the learning data set for artificial intelligence learning according to the instructions stored in the memory. A training data generation device is provided that includes, wherein the processor performs the following steps: (I) when a pseudo-annotation corresponding to at least one training image having at least one initial ground truth annotation included in a public data set—the pseudo-annotation being an image caption that captions the training image—is obtained, parses noun phrases from the pseudo-annotation and extracts nouns corresponding to physical objects from the parsed noun phrases; (II) detect bounding boxes of objects corresponding to the nouns in the training image, generate additional ground truth annotations by referencing each of the detected bounding boxes and each of the nouns corresponding thereto, and add the additional ground truth annotations to the training image; and (III) generate final ground truth annotations corresponding to the training image by referencing the initial ground truth annotations and the additional ground truth annotations.

[0027] In the above embodiment, the processor can extract candidate nouns by performing at least one of the following sub-processes in the (I) process: (i) changing the part of speech of the noun phrases through a spaCy model, a sub-process removing quantifiers and ordinal numbers from the noun phrases, and a sub-process lemmatizing the nouns in the noun phrases; and (ii) extracting the nouns corresponding to the physical object from among the candidate nouns by querying the synset of WordNet or the identifier of Wikidata for each of the candidate nouns to determine whether they correspond to the physical object noun.

[0028] In the above embodiment, the processor can determine whether the specific candidate noun corresponds to the physical object by, in the process (I), checking whether the physical object noun is included in the hypernym corresponding to the specific candidate noun in the WordNet for any one of the candidate nouns, or by traversing the instance and subclass corresponding to the identifier of the specific candidate noun in the Wikidata by a preset depth and checking whether at least one preset noun identifier corresponding to the physical object noun appears.

[0029] In the above embodiment, the processor may input the specific candidate noun into the Large Language Model (LLM) in the process (I) to have the LLM check whether the physical object noun is included in the hypernym corresponding to the specific candidate noun in the WordNet, or traverse instances and subclasses corresponding to the identifier of the specific candidate noun in the Wikidata by a preset depth to check whether at least one preset noun identifier corresponding to the physical object noun appears.

[0030] In the above embodiment, the processor checks whether each of the initial category names included in the initial ground truth annotations and the additional category names included in the additional ground truth annotations is a word that exists in a preset dictionary in the process (III); if a specific category name, which is one of the initial category names and the additional category names, is a word that exists in the preset dictionary, the specific category name is maintained as is, and if the specific category name is not a word that exists in the preset dictionary, the specific category name can be changed using a general word within a preset edit distance from the specific category name through the SymSpell algorithm.

[0031] In the above embodiment, the processor may perform at least one of the following subprocesses in the process (III) to determine whether each of the initial category names and the additional category names is a word existing in the dictionary data: a subprocess to determine whether the specific category name is found in a word set generated by referencing the dictionary data; a subprocess to determine whether the sentence set corresponding to the specific category name in WordNet is empty; and a subprocess to determine whether the specific category name is found in WordFreg.

[0032] In the above embodiment, the processor may generate the additional ground truth annotation through at least one of the following sub-processes: a sub-process that filters bounding boxes among the detected bounding boxes having a size smaller than a preset threshold size by referring to each of the detected bounding boxes and each of the corresponding nouns in the process (II); a sub-process that filters bounding boxes among the detected bounding boxes having a prediction confidence value lower than a preset threshold prediction confidence value; a sub-process that filters bounding boxes among the detected bounding boxes having a text-region alignment score lower than a preset threshold text-region alignment score; a sub-process that filters bounding boxes that overlap through Non-Maximum Suppression (NMS) among the detected bounding boxes; and a sub-process that removes nouns among the nouns that correspond to objects not present in the training image.

[0033] In the above embodiment, the processor can obtain the remaining candidate images as the training images by filtering out images having a resolution lower than a preset threshold resolution, images tagged with NSFW (No Safe For Work), and low-content web images from among the candidate images included in the public dataset in the (I) process.

[0034] In the above embodiment, the processor may, in the process (II), generate object masks from each of the detected bounding boxes and generate additional ground truth annotations by referencing each of the object masks and each of the nouns.

[0035] In addition, a computer-readable recording medium for recording a computer program for executing the method of the present invention is further provided. Effects of the invention

[0036] According to the present invention, the number of ground truth annotations can be effectively expanded by utilizing public data sets.

[0037] According to the present invention, it is possible to accurately generate annotations for objects that are not included in the ground truth annotations of a public data set.

[0038] According to the present invention, the category name of a ground truth annotation added to a public dataset can be made to correspond to a physical object.

[0039] According to the present invention, errors in the category names of ground truth annotations added to a public dataset can be prevented. Brief explanation of the drawing

[0040] The drawings attached below for use in describing embodiments of the present invention are merely some of the embodiments of the present invention, and other drawings can be obtained based on these drawings without inventive work by a person skilled in the art to which the present invention pertains (hereinafter "person skilled in the art"). FIG. 1 schematically illustrates a training data generation device for generating a training data set for artificial intelligence learning according to an embodiment of the present invention, and FIG. 2 schematically illustrates a method for generating a training data set for artificial intelligence learning according to an embodiment of the present invention, and FIG. 3 exemplarily illustrates the state of parsing noun phrases from pseudo-annotations of training images in a method for generating a training data set for artificial intelligence learning according to an embodiment of the present invention, and FIG. 4 schematically illustrates the process of extracting nouns corresponding to physical objects from parsed noun phrases in a method for generating a learning data set for artificial intelligence learning according to an embodiment of the present invention. Specific details for implementing the invention

[0041] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be modified from one embodiment to another without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of individual components within each embodiment may be modified without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limited in meaning, and the scope of the invention should be understood to encompass the scope claimed by the claims and all equivalents thereof. Similar reference numerals in the drawings indicate identical or similar components across various aspects.

[0042] Hereinafter, in order to enable a person skilled in the art to easily practice the present invention, various preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0043] FIG. 1 schematically illustrates a learning data generation device for generating a learning data set for artificial intelligence learning according to an embodiment of the present invention. Referring to FIG. 1, the learning data generation device (1000) may include a memory (1100) in which instructions for generating a learning data set for artificial intelligence learning are stored, and a processor (1200) that generates the learning data set for artificial intelligence learning according to the instructions stored in the memory (1100).

[0044] Specifically, the learning data generating device (1000) may achieve desired system performance by utilizing a combination of computer hardware (e.g., a device that may include components of a computer processor, memory, storage, input device and output device, and other conventional computing devices; electronic communication devices such as routers and switches; and electronic information storage systems such as Network Attached Storage (NAS) and Storage Area Network (SAN)) and computer software (i.e., instructions that cause the computer hardware to function in a specific way), but is not limited thereto.

[0045] Additionally, the processor (1200) of the training data generation device (1000) may include hardware configurations such as an MPU (Micro Processing Unit) or CPU (Central Processing Unit), cache memory, and data bus. Furthermore, the training data generation device (1000) may further include software configurations such as an operating system and an application for a specific purpose.

[0046] However, this does not exclude the case where the learning data generation device (1000) includes an integrated processor in which a medium, a processor, and a memory are integrated for implementing the present invention.

[0047] Meanwhile, the processor (1200) of the learning data generation device (1000) can perform a process of parsing noun phrases from the pseudo-annotation and extracting nouns corresponding to physical objects from the parsed noun phrases when a pseudo-annotation corresponding to at least one learning image (10) having at least one initial ground truth annotation included in the public data set is obtained according to instructions stored in memory (1100). And, the processor (1200) of the training data generation device (1000) can detect bounding boxes of objects corresponding to nouns in the training image (10) according to instructions stored in memory (1100), generate additional ground truth annotations (21) by referencing each of the detected bounding boxes and each of the nouns corresponding thereto, and perform a process of adding the additional ground truth annotations (21) to the training image (10). At this time, the number of additional ground truth annotations (21) generated may vary depending on the training image, and in the case of a specific training image, there may be no additional ground truth annotations or only one may be generated. Subsequently, the processor (1200) of the training data generation device (1000) can perform a process of generating final ground truth annotations corresponding to the training image (10) by referring to the initial ground truth annotations and additional ground truth annotations (21) according to the instructions stored in the memory (1100).

[0048] A method for generating a training data set for artificial intelligence learning using a training data generating device (1000) configured in this manner is described as follows with reference to FIGS. 1 and FIG. 2.

[0049] First, the training data generation device (1000) can acquire (S100) a public data set including at least one training image (10) having initial ground truth annotations.

[0050] At this time, the public data set may be a data set collected through web crawling or a data set established in advance as IP, but the present invention is not limited thereto, and may be a data set collected through various channels to generate a training data set for training an AI model.

[0051] Additionally, the training data generation device (1000) may obtain the remaining candidate images as training images after filtering out images having a resolution lower than a preset threshold resolution, images tagged with NSFW (No Safe For Work), and low-content web images from among the candidate images included in the acquired public data set. In this case, NSFW refers to content that is inappropriate to view in a workplace or public place.

[0052] Next, the training data generation device (1000) can parse noun phrases from pseudo-annotations for a training image (10) and extract nouns corresponding to physical objects from the parsed noun phrases (S200).

[0053] At this time, the pseudo-annotation for the training image (10) may be an image caption that captures the training image (10). In addition, the pseudo-annotation may be one that is collected simultaneously with the collection of the public data set, or one that is generated by the training data generation device (1000) by capturing the training image (10) through an image captioning model.

[0054] In addition, referring to FIG. 3 as an example, noun phrases (211) can be parsed through a spaCy model in a pseudo-annotation such as FIG. 3 (b), which is an image caption that captures the training image in FIG. 3 (a). However, the present invention is not limited thereto, and various language models for parsing noun phrases in pseudo-annotations may be used in addition to the spaCy model. For reference, the noun phrases parsed in FIG. 3 (b) may be words within the gray boxes, and the reference numeral “211” indicating a noun phrase is shown for only one of the parsed noun phrases.

[0055] Meanwhile, referring to Fig. 4, the process of extracting nouns corresponding to physical objects from parsed noun phrases is explained in more detail as follows.

[0056] The learning data generation device (1000) can perform a sub-process of changing the part of speech of noun phrases (S210) through a space model.

[0057] For example, possessives such as “his,” “her,” “their,” and “its” contained in noun phrases can be changed to the definite article “the.”

[0058] And, the learning data generation device (1000) can perform a sub-process of removing quantifiers and ordinal numbers from noun phrases (S220) through a space model.

[0059] For example, quantifiers such as “other,” “another,” “some,” “any,” “most,” “several,” “few,” and “many” can be removed, and ordinal numbers such as “first,” “second,” and “third” can be removed.

[0060] Additionally, the learning data generation device (1000) can perform a sub-process of lemmatizing (S230) nouns in noun phrases through a space model.

[0061] For example, nouns in noun phrases can be changed to their base forms, such as changing “hands” to “hand,” “wolves” to “wolf,” “mice” to “mouse,” and “teeth” to “tooth.”

[0062] Through such sub-processes, the learning data generation device (1000) can extract candidate nouns to be added as category names.

[0063] However, the present invention is not limited thereto, and various language models other than the space model can be used to extract candidate nouns from noun phrases in various ways. Additionally, FIG. 4 illustrates a sub-process for changing the part of speech of noun phrases (S210), a sub-process for removing quantifiers and ordinal numbers from noun phrases (S220), and a sub-process for rematizing nouns in noun phrases (230) in sequential order, but the present invention is not limited thereto, and the order in which sub-processes for extracting candidate nouns from nouns are performed may be arbitrarily determined or may be performed simultaneously.

[0064] Afterwards, the learning data generation device (1000) can check (S240) whether candidate nouns correspond to physical objects.

[0065] That is, the learning data generation device (1000) can extract nouns corresponding to physical objects from among the candidate nouns by querying the synset of WordNet or the identifier of Wikidata for each of the candidate nouns to check whether they correspond to physical object nouns.

[0066] For example, a learning data generation device (1000) can determine whether a specific candidate noun corresponds to a physical object by checking whether a physical object noun is included in the hypernym corresponding to the specific candidate noun in WordNet for any of the candidate nouns, or by traversing instances and subclasses corresponding to the identifier of the specific candidate noun in Wikidata by a preset depth and checking whether at least one preset noun identifier corresponding to the physical object noun appears. At this time, the noun identifier corresponding to the physical object noun may include an identifier corresponding to a “physical object” or an identifier corresponding to an “object,” but the present invention is not limited thereto and may be identifiers corresponding to various nouns corresponding to physical objects.

[0067] As another example, the training data generating device (1000) may input a specific candidate noun, which is one of the candidate nouns, into a Large Language Model (LM) to have the LLM check whether a physical object noun is included in the hypernym corresponding to the specific candidate noun in WordNet, or may traverse instances and subclasses corresponding to the identifier of the specific candidate noun in Wikidata by a preset depth to check whether at least one preset noun identifier corresponding to the physical object noun appears.

[0068] More specifically, to illustrate with an example using Wikidata, it can be confirmed that “Baekdusan” is an instance of “mountain,” that “mountain” is a subclass of “natural geographic object,” that “natural geographic object” is a subclass of “natural physical object,” and that “natural physical object” is a subclass of “physical object,” thereby confirming that “Baekdusan” is a noun corresponding to a physical object.

[0069] Referring again to FIGS. 1 and FIGS. 2, the learning data generating device (1000) can detect (S300) bounding boxes of objects corresponding to nouns in the learning image (10).

[0070] At this time, the training data generation device (1000) can detect bounding boxes of objects corresponding to nouns in the training image (10) through a text prompt-based object detector such as GroundingDINO, TAP, etc. or a vision prompt-based object detector such as T-rex2. However, the present invention is not limited thereto, and various types of models that detect objects corresponding to nouns in the training image (10), such as a multimodal prompt-based object detector that supports both text and vision, may be used.

[0071] Next, the learning data generation device (1000) generates additional ground truth annotations (21) by referencing each of the detected bounding boxes and each of the corresponding nouns, and can add the additional ground truth annotations (21) to the learning image (10) (S400).

[0072] At this time, the learning data generation device (1000) can filter out and remove some bounding boxes or some nouns among the detected bounding boxes or nouns that are judged to have low accuracy or reliability by referring to each of the detected bounding boxes and each of the nouns corresponding thereto.

[0073] For example, a learning data generation device (1000) can filter out and remove bounding boxes among the detected bounding boxes that have a size smaller than a preset threshold size.

[0074] As another example, the learning data generation device (1000) can filter out and remove bounding boxes among the detected bounding boxes that have a prediction confidence value lower than a preset threshold prediction confidence value. At this time, the prediction confidence value may be generated by an object detector that detects objects corresponding to nouns.

[0075] As another example, the training data generation device (1000) checks text-region alignment scores, which are alignment scores between image regions of nouns and their corresponding bounding boxes, through a CLIP (Contrastive Language-Image Pre-training) model, and can filter out and remove bounding boxes among the detected bounding boxes that have a text-region alignment score lower than a preset threshold text-region alignment score.

[0076] As another example, the learning data generation device (1000) can filter out and remove overlapping bounding boxes among the detected bounding boxes through Non-Maximum Suppression (NMS).

[0077] As another example, the training data generation device (1000) can remove nouns corresponding to objects that do not exist in the training image. That is, nouns corresponding to objects that do not actually exist in the training image (10) may be included in the pseudo-annotation due to the hallucination of the artificial intelligence model that generates pseudo-annotations in the training image (10), and can be removed by verifying this. At this time, in order to check whether the nouns are hallucinations, the pseudo-annotation can be cross-validated using models such as InternVL3 and Qwen3-VL.

[0078] Additionally, the learning data generation device (1000) may add a mask annotation to generate a more accurate ground truth annotation.

[0079] That is, the learning data generation device (1000) may generate object masks from each of the detected bounding boxes and generate additional ground truth annotations by referencing each of the object masks and each of the nouns.

[0080] For example, a training data generation device (1000) can input a training image (10), bounding box information, and a noun into a segmentation model such as SAM3, so that the segmentation model can generate an object mask corresponding to an object within the bounding box.

[0081] Next, the training data generation device (1000) can generate final ground truth annotations (S500) corresponding to the training image (10) by referring to initial ground truth annotations and additional ground truth annotations.

[0082] For example, a learning data generation device (1000) can check whether each of the initial category names included in the initial ground truth annotations and the additional category names included in the additional ground truth annotations is a word that exists in a pre-configured dictionary. For reference, the dictionary data may mean dictionary data rather than previous data.

[0083] At this time, the learning data generation device (1000) may perform at least one of the following sub-processes to determine whether a specific category name is a word existing in the dictionary data for each of the initial category names and additional category names: a sub-process to determine whether a specific category name is found in a word set generated by referencing the dictionary data; a sub-process to determine whether a sentence set corresponding to a specific category name in WordNet is empty; and a sub-process to determine whether a specific category name is found in WordFreg.

[0084] In addition, the learning data generation device (1000) maintains the specific category name as is if the specific category name, which is one of the initial category names and additional category names, is a word that exists in the dictionary data, and if the specific category name is not a word that exists in the dictionary data, it can change the specific category name using a general word within a preset edit distance from the specific category name through the SymSpell algorithm.

[0085] For example, if a specific category name is “pizzza,” it can be changed to the generic word “pizza,” and if it is “hankerchief,” it can be changed to “handkerchief.”

[0086] However, the present invention is not limited thereto, and various language models that perform typo correction and spelling correction in category names, in addition to SymSpell, may be used.

[0087] According to one embodiment of the present invention, by using training images having final ground truth annotations, a training data set having extended ground truth annotations corresponding to physical objects and more accurately set category names can be generated, and accordingly, an artificial intelligence model can be trained to perform a task more accurately.

[0088] The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0089] Although the present invention has been described above with specific details such as specific components, limited embodiments, and drawings, this is provided only to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above embodiments, and a person skilled in the art to which the invention belongs can make various modifications and variations from this description.

[0090] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all modifications equivalent to or equivalent to the claims set forth below, as well as the claims described below, shall be considered to fall within the scope of the concept of the present invention. Explanation of the symbols

[0091] 1000: Training data generation device, 1100: Memory, 1200: Processor

Claims

Claim 1 A method for generating a training data set for artificial intelligence learning comprises: (a) when a pseudo-annotation corresponding to at least one training image having at least one initial ground truth annotation included in a public data set—said that the pseudo-annotation is an image caption that captions the training image—is obtained, a training data generating device parses noun phrases from the pseudo-annotation and extracts nouns corresponding to physical objects from the parsed noun phrases—said that the physical objects correspond to object categories to be detected through an object detector; (b) the training data generating device detects bounding boxes of objects corresponding to the nouns in the training image through the object detector, generates additional ground truth annotations by referencing each of the detected bounding boxes and each of the nouns corresponding thereto, and adds the additional ground truth annotations to the training image; and (c) a step in which the training data generating device generates final ground truth annotations corresponding to the training image by referring to the initial ground truth annotations and the additional ground truth annotations; a method comprising. Claim 2 In claim 1, in step (a), the training data generating device extracts candidate nouns by performing at least one of a sub-process of changing the part of speech of the noun phrases through a spaCy model, a sub-process of removing quantifiers and ordinal numbers from the noun phrases, and a sub-process of lemmatizing the nouns in the noun phrases, and (ii) extracts the nouns corresponding to the physical object among the candidate nouns by querying the identifier of WordNet or Wikidata for each of the candidate nouns to check whether they correspond to the physical object noun. Claim 3 In paragraph 2, in step (a), the training data generating device checks whether the specific candidate noun corresponds to the physical object by checking, for any one of the candidate nouns, whether the physical object noun is included in the hypernym corresponding to the specific candidate noun in the WordNet, or by traversing the instance and subclass corresponding to the identifier of the specific candidate noun in the Wikidata by a preset depth and checking whether at least one preset noun identifier corresponding to the physical object noun appears. Claim 4 In paragraph 3, in step (a), the training data generating device inputs the specific candidate noun into a Large Language Model (LM) to cause the LLM to check whether the physical object noun is included in the hypernym corresponding to the specific candidate noun in the WordNet, or to traverse instances and subclasses corresponding to the identifier of the specific candidate noun in Wikidata by a preset depth to check whether at least one preset noun identifier corresponding to the physical object noun appears. Claim 5 In claim 1, in step (c), the training data generating device checks whether each of the initial category names included in the initial ground truth annotations and the additional category names included in the additional ground truth annotations is a word existing in a preset dictionary, and if a specific category name, which is either the initial category name or the additional category name, is a word existing in the preset dictionary, the specific category name is maintained as is, and if the specific category name is not a word existing in the preset dictionary, the specific category name is changed using a general word within a preset edit distance from the specific category name through a SymSpell algorithm. Claim 6 In claim 5, in step (c), the training data generating device performs at least one of the following sub-processes in determining whether each of the initial category names and the additional category names is a word existing in the dictionary data: a sub-process of determining whether the specific category name is found in a word set generated by referencing the dictionary data; a sub-process of determining whether the sentence set corresponding to the specific category name in WordNet is empty; and a sub-process of determining whether the specific category name is found in WordFreg. Claim 7 In claim 1, in step (b), the training data generating device generates the additional ground truth annotation through at least one of the following: a sub-process for filtering bounding boxes among the detected bounding boxes that have a size smaller than a preset threshold size by referring to each of the detected bounding boxes and each of the corresponding nouns; a sub-process for filtering bounding boxes among the detected bounding boxes that have a prediction confidence value lower than a preset threshold prediction confidence value; a sub-process for filtering bounding boxes among the detected bounding boxes that have a text-region alignment score lower than a preset threshold text-region alignment score; a sub-process for filtering bounding boxes among the detected bounding boxes that overlap through NMS (Non-Maximum Suppression); and a sub-process for removing nouns among the nouns that correspond to objects not present in the training image. Claim 8 In claim 1, in step (a), the training data generating device filters out images having a resolution lower than a preset threshold resolution, images tagged with NSFW (No Safe For Work), and low-content web images from among the candidate images included in the public data set, and obtains the remaining candidate images as the training images. Claim 9 A method according to claim 1, wherein in step (b), the learning data generating device generates object masks from each of the detected bounding boxes and generates additional ground truth annotations by referencing each of the object masks and each of the nouns. Claim 10 A training data generation device for generating a training data set for artificial intelligence learning, comprising: a memory in which instructions for generating a training data set for artificial intelligence learning are stored; and a processor that generates a training data set for artificial intelligence learning according to instructions stored in the memory; wherein the processor comprises: (I) a process of, when a pseudo-annotation corresponding to at least one training image having at least one initial ground truth annotation included in a public data set—the pseudo-annotation being an image caption that captions the training image—is obtained, parsing noun phrases from the pseudo-annotation and extracting nouns corresponding to physical objects—the physical objects corresponding to object categories to be detected through an object detector—from the parsed noun phrases; (II) a process of detecting bounding boxes of objects corresponding to the nouns in the training image through the object detector, generating additional ground truth annotations by referencing each of the detected bounding boxes and each of the nouns corresponding thereto, and adding the additional ground truth annotations to the training image; and (III) the initial ground truth annotations and the additional ground A training data generation device that performs a process of generating final ground truth annotations corresponding to the training image by referring to truth annotations. Claim 11 In claim 10, the processor is a learning data generation device that, in the process (I), extracts candidate nouns by performing at least one of the following: (i) a sub-process of changing the part of speech of the noun phrases through a spaCy model, a sub-process of removing quantifiers and ordinal numbers from the noun phrases, and a sub-process of lemmatizing the nouns in the noun phrases; and (ii) extracts the nouns corresponding to the physical object among the candidate nouns by querying the identifier of WordNet or Wikidata for each of the candidate nouns to determine whether they correspond to the physical object noun. Claim 12 In claim 11, the processor is a learning data generation device that, in the process (I), checks whether the physical object noun is included in the hypernym corresponding to the specific candidate noun in the WordNet for any one of the candidate nouns, or checks whether at least one noun identifier corresponding to the physical object noun appears by traversing instances and subclasses corresponding to the identifier of the specific candidate noun in the Wikidata by a preset depth. Claim 13 In claim 12, the processor is a learning data generation device that, in process (I), inputs the specific candidate noun into a Large Language Model (LM) to cause the LLM to check whether the physical object noun is included in a hypernym corresponding to the specific candidate noun in the WordNet, or traverses instances and subclasses corresponding to the identifier of the specific candidate noun in Wikidata by a preset depth to check whether at least one preset noun identifier corresponding to the physical object noun appears. Claim 14 In claim 10, the processor, in the process (III), checks whether each of the initial category names included in the initial ground truth annotations and the additional category names included in the additional ground truth annotations is a word existing in a preset dictionary, and if a specific category name, which is either the initial category name or the additional category name, is a word existing in the preset dictionary, the specific category name is maintained as is, and if the specific category name is not a word existing in the preset dictionary, the specific category name is changed using a general word within a preset edit distance from the specific category name through a SymSpell algorithm. Claim 15 In claim 14, the processor is a learning data generation device that, in the process (III), checks whether each of the initial category names and the additional category names is a word existing in the dictionary data, and performs at least one of the following sub-processes: checking whether the specific category name is found in a word set generated by referencing the dictionary data; checking whether the sentence set corresponding to the specific category name in WordNet is empty; and checking whether the specific category name is found in WordFreg. Claim 16 In claim 10, the processor is a training data generation device that generates the additional ground truth annotation through at least one of the following sub-processes: a sub-process that filters bounding boxes having a size smaller than a preset threshold size among the detected bounding boxes by referring to each of the detected bounding boxes and each of the corresponding nouns in the process (II); a sub-process that filters bounding boxes having a prediction confidence value lower than a preset threshold prediction confidence value among the detected bounding boxes; a sub-process that filters bounding boxes having a text-region alignment score lower than a preset threshold text-region alignment score among the detected bounding boxes; a sub-process that filters overlapping bounding boxes among the detected bounding boxes through NMS (Non-Maximum Suppression); and a sub-process that removes nouns corresponding to objects not present in the training image among the nouns. Claim 17 In claim 10, the processor is a training data generation device that, in the process (I), filters out from among the candidate images included in the public data set images having a resolution lower than a preset threshold resolution, images tagged with NSFW (No Safe For Work), and low-content web images, and obtains the remaining candidate images as the training images. Claim 18 In claim 10, the processor is a learning data generating device that, in the process (II), generates object masks from each of the detected bounding boxes and generates additional ground truth annotations by referencing each of the object masks and each of the nouns.

Images