Method for generating training datasets for artificial intelligence learning and a training data generation apparatus utilizing the same

Abstract

This invention provides a method for generating training datasets for artificial intelligence learning and a training data generation apparatus. [Solution] The method by the learning data generation device includes the steps of: parsing noun phrases with pseudo annotations corresponding to a learning image having an initial ground truth annotation, extracting nouns corresponding to physical objects from the parsed noun phrases (S100); detecting the bounding boxes of objects corresponding to the nouns from the learning image (S200); generating additional ground truth annotations by referencing each of the nouns (S400); and adding the additional ground truth annotations to the learning image (S500); and generating a final ground truth annotation corresponding to the learning image by referencing the initial ground truth annotation and the additional ground truth annotation.

Description

Technical Field

[0001] The present invention relates to generation of a training data set for artificial intelligence learning, and more particularly, to a method for generating a training data set for artificial intelligence learning and a training data generation apparatus using the same, which can effectively expand the number of ground truth annotations and set category names more accurately by utilizing collected public data sets.

Background Art

[0002] Artificial intelligence, that is, AI (Artificial Intelligence) models process and analyze huge amounts of data to make better decisions and generate accurate predictions, and with technological development, AI models are being used in various fields.

[0003] In order to accurately train an AI model, a training data set that covers all possible scenarios is required.

[0004] In order to generate such a training data set, conventionally, public data sets are used, such as data sets collected through web crawling and data sets built as IP (Intellectual Property) in dictionaries.

[0005] However, although web-crawled data sets cover general domains, they have the disadvantage that there are no annotations for some objects, or the annotations are not detailed.

[0006] Further, although data sets built as IP in dictionaries cover detailed domains, they have the disadvantage that annotated category names are abstract. For example, drivers, pedestrians, police officers, and the like can simply be annotated as "human".

[0007] Furthermore, datasets constructed as IPs in dictionaries have the disadvantage of containing typographical errors in some of the annotated category names.

[0008] As a result, when training an AI model using training datasets generated by conventional methods, the loss calculated during the AI ​​model's training process is inaccurate, leading to a problem where the AI ​​model is not trained accurately.

[0009] For example, in training a Vision Language Model (VLM), category names must be provided as text prompts to calculate the loss for visual grounding, and contrast loss must be calculated in the embedding space for alignment between multiple modals to support multimodal prompts such as text, vision, and audio. However, if there are errors in the category names included in the ground truth annotations, the trained VLM will not only be unable to accurately detect the object corresponding to the input text, but will also be unable to accurately recognize the correspondences for multimodal inputs.

[0010] Therefore, the applicant seeks to propose a method that leverages collected public datasets to effectively determine the number of ground truth annotations and enable more accurate category naming. [Overview of the project] [Problems that the invention aims to solve]

[0011] The purpose of this invention is to solve all of the problems of the conventional technology described above.

[0012] Another objective of the present invention is to enable the effective determination of the number of ground truth annotations by utilizing public datasets.

[0013] Another objective of the present invention is to enable the accurate generation of annotations for objects that are not included in the ground truth annotations of public datasets.

[0014] Furthermore, the present invention also aims to ensure that the category names of ground truth annotations added to public datasets correspond to physical objects.

[0015] Another objective of the present invention is to prevent errors in category names of ground truth annotations added to public datasets. [Means for solving the problem]

[0016] According to one embodiment of the present invention, a method for generating a training dataset for artificial intelligence learning includes: (a) once pseudo annotations (the pseudo annotations are image captions that caption the training images) corresponding to at least one training image having at least one initial ground truth annotation included in a public dataset are obtained, a training dataset generator parses noun phrases using the pseudo annotations and extracts nouns corresponding to physical objects from the parsed noun phrases; (b) the training data generator detects bounding boxes of objects corresponding to the nouns from the training images, generates additional ground truth annotations by referencing each of the detected bounding boxes and each of the corresponding nouns, and adds the additional ground truth annotations to the training images; and (c) the training data generator generates final ground truth annotations corresponding to the training images by referencing the initial ground truth annotations and the additional ground truth annotations.

[0017] In one example, in step (a) above, the learning data generation device extracts candidate nouns by (i) performing at least one of the following subprocesses through the spaCy model: a subprocess to change the part of speech of the noun phrase, a subprocess to remove quantifiers and ordinal numbers from the noun phrase, and a subprocess to lemmatize the nouns in the noun phrase; and (ii) extracting the nouns that correspond to physical objects from among the candidate nouns by querying the WordNet synset or Wikidata identifier for each of the candidate nouns to see if they correspond to physical object nouns.

[0018] In one example, in step (a) above, the learning data generation device checks whether the physical object noun is included in the hypernym corresponding to any one of the candidate nouns in the WordNet, or traverses the instances and subclasses corresponding to the identifier of the specific candidate noun in the Wikidata to a pre-set depth and checks whether at least one pre-set noun identifier corresponding to the physical object noun appears, thereby confirming whether the specific candidate noun corresponds to the physical object.

[0019] In one example, in step (a) above, the learning data generation device inputs the specific candidate noun into an LLM (Large Language Model) and uses the LLM to check whether the physical object noun is included in the hypernyms corresponding to the specific candidate noun in the WordNet, or traverses the instances and subclasses corresponding to the identifiers of the specific candidate noun in the wikidata to a pre-set depth and checks whether at least one pre-set noun identifier appears corresponding to the physical object noun.

[0020] In one example, in step (c) above, the learning data generation device checks whether each of the initial category names included in the initial ground truth annotation and the additional category names included in the additional ground truth annotation are words that already exist in the dictionary data. If a specific category name, which is either one of the initial category names or the additional category names, is a word that exists in the dictionary data, the device maintains that specific category name as is. If the specific category name is not a word that exists in the dictionary data, the device modifies the specific category name using the SymSpell algorithm, utilizing the specific category name and a general-purpose word within the already set edit distance.

[0021] In one example, in step (c) above, the learning data generation device checks whether the initial category name and the additional category name are words that exist in the dictionary data by performing at least one of the following subprocesses: checking whether the specific category name is queried in a word set (words-set) generated by referring to the dictionary data; checking whether there is an available synoset in WordNet corresponding to the specific category name; and checking whether the specific category name is queried in WordFreg.

[0022] In one example, in step (b) above, the training data generation device generates the additional ground truth annotation by referring to each of the detected bounding boxes and each of the corresponding nouns, through at least one of the following subprocesses: a subprocess for filtering out bounding boxes smaller than a pre-set critical size among the detected bounding boxes; a subprocess for filtering out bounding boxes with a predicted confidence value lower than a pre-set critical predicted confidence value among the detected bounding boxes; a subprocess for filtering out bounding boxes with a text-to-region alignment score lower than a pre-set critical text-to-region alignment score among the detected bounding boxes; a subprocess for filtering out bounding boxes that overlap through NMS (Non-Maximum Suppression) among the detected bounding boxes; and a subprocess for removing nouns corresponding to objects that do not exist in the training image from the nouns.

[0023] In one example, in step (a) above, the training data generation device filters out candidate images from the public dataset that have a resolution lower than the previously set critical resolution, images tagged as NSFW (No Safe For Work), and low-content web images, and acquires the remaining candidate images as the training images.

[0024] In one example, in step (b), the training data generation device generates an object mask from each of the detected bounding boxes and generates the additional ground truth annotation by referencing each of the object masks and each of the nouns.

[0025] According to one embodiment of the present invention, a learning data generation device for generating a learning dataset for artificial intelligence learning includes: a memory storing instructions for generating a learning dataset for artificial intelligence learning; and a processor that generates the learning dataset for artificial intelligence learning using the instructions stored in the memory, wherein the processor (i) when a pseudo annotation (the pseudo annotation is an image caption that captions the learning image) corresponding to at least one learning image having at least one initial ground truth annotation included in a public dataset is obtained, the pseudo annotation A learning data generation device that performs the following processes: (II) parsing noun phrases and extracting nouns corresponding to physical objects from the parsed noun phrases; (II) detecting bounding boxes of objects corresponding to the nouns from the learning image, generating additional ground truth annotations by referencing each of the detected bounding boxes and each of the corresponding nouns, and adding the additional ground truth annotations to the learning image; and (III) generating final ground truth annotations corresponding to the learning image by referencing the initial ground truth annotations and the additional ground truth annotations.

[0026] In one example, the processor extracts candidate nouns by (i) performing at least one of the following subprocesses in the (I) process: (i) changing the part of speech of the noun phrase through a spaCy model, removing quantifiers and ordinal numbers from the noun phrase, and lemmatizing the nouns in the noun phrase; and (ii) querying each of the candidate nouns against a WordNet synset or Wikidata identifier to determine if it corresponds to a physical object noun, thereby extracting the nouns corresponding to the physical object from among the candidate nouns.

[0027] In one example, in the step (I), for a specific candidate noun that is any one of the candidate nouns, the processor checks whether the hypernym corresponding to the specific candidate noun in WordNet includes the physical object noun, or traverses, to a preset depth, instances and subclasses corresponding to the identifier of the specific candidate noun in Wikidata, and checks whether at least one preset noun identifier corresponding to the physical object noun appears, thereby confirming whether the specific candidate noun corresponds to the physical object.

[0028] In one example, in the step (I), the processor inputs the specific candidate noun into an LLM (Large Language Model), and causes the LLM to check whether the hypernym corresponding to the specific candidate noun in WordNet includes the physical object noun, or traverses, to a preset depth, instances and subclasses corresponding to the identifier of the specific candidate noun in Wikidata, and check whether at least one preset noun identifier corresponding to the physical object noun appears.

[0029] In one example, in the step (III), for each of the initial category name included in the initial ground truth annotation and the additional category name included in the additional ground truth annotation, the processor checks whether the name exists as a word in preset dictionary data; when a specific category name that is any one of the initial category name and the additional category name is a word existing in the dictionary data, the processor maintains the specific category name as it is, and when the specific category name is not a word existing in the dictionary data, the processor modifies the specific category name through the SymSpell algorithm by using the specific category name and a general word within a preset edit distance.

[0030] In one example, the processor, in process (III), checks whether the initial category name and the additional category name are words that exist in the dictionary data by performing at least one of the following subprocesses: a subprocess that checks whether the specific category name is queried in a word set (words-set) generated by referring to the dictionary data; a subprocess that checks whether there is an available synoset in WordNet corresponding to the specific category name; and a subprocess that checks whether the specific category name is queried in WordFreg.

[0031] In one example, the processor generates the additional ground truth annotation in the (II) process by referring to each of the detected bounding boxes and each of the corresponding nouns, through at least one of the following subprocesses: a subprocess that filters out bounding boxes among the detected bounding boxes that are smaller than a pre-set critical size; a subprocess that filters out bounding boxes among the detected bounding boxes that have a predicted confidence value lower than a pre-set critical predicted confidence value; a subprocess that filters out bounding boxes among the detected bounding boxes that have a text-to-region alignment score lower than a pre-set critical text-to-region alignment score; a subprocess that filters out bounding boxes among the detected bounding boxes that overlap through NMS (Non-Maximum Suppression); and a subprocess that removes nouns from the nouns that correspond to objects not present in the training image.

[0032] In one example, the processor, in process (I), filters out candidate images from the public dataset that have a resolution lower than a previously set critical resolution, images tagged as NSFW (No Safe For Work), and low-content web images, and acquires the remaining candidate images as training images.

[0033] In one example, the processor generates an object mask from each of the detected bounding boxes in the (II) process, and generates the additional ground truth annotation by referencing each of the object masks and each of the nouns. [Effects of the Invention]

[0034] According to the present invention, it becomes possible to effectively determine the number of ground truth annotations by utilizing public datasets.

[0035] According to the present invention, it becomes possible to accurately generate annotations for objects that are not included in the ground truth annotations of public datasets.

[0036] According to the present invention, the category names of ground truth annotations added to a public dataset can be made to correspond to physical objects.

[0037] According to the present invention, it becomes possible to prevent errors in the category names of ground truth annotations added to public datasets. [Brief explanation of the drawing]

[0038] The following drawings, attached for use in describing embodiments of the present invention, represent only a portion of embodiments of the present invention, and a person with ordinary skill in the art to which the present invention pertains (hereinafter referred to as "ordinary art") can obtain the other drawings from these drawings without performing any inventive work.

[0039] [Figure 1] This is a simplified diagram of a learning data generation device that generates a learning dataset for artificial intelligence learning according to one embodiment of the present invention. [Figure 2] This briefly illustrates a method for generating a training dataset for artificial intelligence learning according to one embodiment of the present invention. [Figure 3] This is an illustrative example of a method for generating a training dataset for artificial intelligence learning according to one embodiment of the present invention, showing the state in which noun phrases have been parsed from pseudoannotations of training images. [Figure 4] This diagram shows a simplified process for extracting nouns corresponding to physical objects from parsed noun phrases in a method for generating a training dataset for artificial intelligence learning according to one embodiment of the present invention. [Modes for carrying out the invention]

[0040] The detailed description of the present invention, as described below, refers to the accompanying drawings illustrating specific embodiments in which the present invention may be carried out. These embodiments are described in sufficient detail so that those skilled in the art can carry out the present invention. It should be understood that the various embodiments of the present invention are different from one another but do not need to be mutually exclusive. For example, certain shapes, structures and characteristics described herein can be embodied by modifying one embodiment to another without departing from the spirit and scope of the present invention. It should also be understood that the position or arrangement of individual components within each embodiment can be modified without departing from the spirit and scope of the present invention. Therefore, the detailed description below should not be taken as restrictive, and the scope of the present invention should be accepted as encompassing the scope claimed in the claims and all equivalent scopes thereto. Similar reference numerals in the drawings refer to parts that are identical or have similar functions in various aspects.

[0041] In the following, several preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that a person with ordinary skill in the art to which the present invention pertains can easily implement the present invention.

[0042] Figure 1 shows a simplified representation of a learning data generation device that generates a learning dataset for artificial intelligence learning according to one embodiment of the present invention. Referring to Figure 1, the learning data generation device 1000 may include a memory 1100 that stores instructions for generating a learning dataset for artificial intelligence learning, and a processor 1200 that generates the learning dataset for artificial intelligence learning using the instructions stored in the memory 1100.

[0043] Specifically, the learning data generation device 1000 may, but is not limited to, achieve desired system performance by utilizing a combination of computer hardware (e.g., a device that may include computer processors, memory, storage, input and output devices, and other components of existing computing devices; electronic communication devices such as routers and switches; and electronic information storage systems such as networked storage (NAS) and storage area networks (SANs)) and computer software (i.e., instructions for making the computer hardware function in a particular way).

[0044] Furthermore, the processor 1200 of the learning data generation device 1000 may include hardware configurations such as an MPU (Micro Processing Unit) or CPU (Central Processing Unit), cache memory, and data bus. The learning data generation device 1000 may also further include an operational structure and software configurations for applications that perform specific objectives.

[0045] However, this does not exclude the case in which the learning data generation device 1000 includes an integrated processor in which a medium, processor, and memory are integrated for carrying out the present invention.

[0046] On the other hand, the processor 1200 of the training data generation device 1000 can, based on instructions stored in memory 1100, perform the process of parsing noun phrases using pseudo annotations and extracting nouns corresponding to physical objects from the parsed noun phrases, once pseudo annotations corresponding to at least one training image 10 with at least one initial ground truth annotation included in the public dataset, i.e., image captions that caption the training image 10, are obtained. Then, based on instructions stored in memory 1100, the processor 1200 of the training data generation device 1000 can detect bounding boxes of objects corresponding to nouns in the training image 10, generate additional ground truth annotations 21 by referencing each of the detected bounding boxes and their corresponding nouns, and add the additional ground truth annotations 21 to the training image 10. At this time, the number of additional ground truth annotations 21 generated may vary depending on the training image, and in the case of a particular training image, there may be no additional ground truth annotations or only one may be generated. Thereafter, the processor 1200 of the training data generation device 1000 can perform the process of generating the final ground truth annotation corresponding to the training image 10 by referring to the initial ground truth annotation and the additional ground truth annotations 21, based on the instructions stored in the memory 1100.

[0047] The method for generating a training dataset for artificial intelligence learning using the training data generation device 1000 configured in this way will be explained below with reference to Figures 1 and 2.

[0048] First, the training data generation device 1000 can acquire a public dataset (S100) that includes at least one training image 10 with initial ground truth annotation.

[0049] In this case, the public dataset may be a dataset collected through web crawling or a dataset constructed as IP in a dictionary, but the present invention is not limited thereto, and may also be a dataset collected by the user through various means to generate a training dataset for training an AI model.

[0050] Furthermore, the training data generator 1000 can also filter out candidate images from the acquired public dataset that have a resolution lower than a previously set critical resolution, images tagged with NSFW (No Safe For Work), and low-content web images, and acquire the remaining candidate images as training images. In this case, NSFW refers to content that is inappropriate for viewing in the workplace or public places.

[0051] Next, the learning data generation device 1000 can parse noun phrases using pseudo annotations on the learning image 10 and extract nouns corresponding to physical objects from the parsed noun phrases (S200).

[0052] In this case, pseudoannotation for the training image 10 may be an image caption that captions the training image 10. Furthermore, pseudoannotation may be collected simultaneously with the collection of the public dataset, or it may be generated by the training data generation device 1000 by captioning the training image 10 through an image captioning model.

[0053] Furthermore, referring to Figure 3 as an example, the noun phrase 211 can be parsed through the spaCy model using pseudo-annotation like that shown in Figure 3(b), which is an image caption that captions the learning image in Figure 3(a). However, the present invention is not limited to this, and various language models that parse noun phrases using pseudo-annotation other than the spaCy model can also be used. Incidentally, in Figure 3(b), the parsed noun phrase may be a word within a gray box, and the drawing reference numeral "211" indicating the noun phrase is displayed for only one noun phrase among the parsed noun phrases.

[0054] On the other hand, the process of extracting nouns corresponding to physical objects from parsed noun phrases, with reference to Figure 4, can be explained in more detail as follows:

[0055] The learning data generation device 1000 can perform a subprocess of changing the part of speech of noun phrases through a Spacy model (S210).

[0056] For example, possessive pronouns such as "his," "her," "their," and "its" within a noun phrase can be changed to the definite article "the."

[0057] Furthermore, the learning data generation device 1000 can perform a subprocess (S220) of removing quantifiers and ordinal numbers from noun phrases through a Spacy model.

[0058] For example, quantifiers such as "other," "another," "some," "any," "most," "several," "few," and "many" can be removed, as can ordinal numbers such as "first," "second," and "third."

[0059] Furthermore, the learning data generation device 1000 can perform a subprocess of rematizing nouns in noun phrases (S230) through a Spacy model.

[0060] For example, you can change nouns in noun phrases to their base form, such as changing "hands" to "hand," "wolves" to "wolf," "mice" to "mouse," and "teeth" to "tooth."

[0061] Through such subprocesses, the learning data generation device 1000 can extract candidate nouns to be added as category names.

[0062] However, the present invention is not limited to this, and candidate nouns can be extracted from noun phrases in various ways using a variety of language models other than the Spacey model. Also, although Figure 4 shows the subprocess of changing the part of speech of the noun phrase (S210), the subprocess of removing quantifiers and ordinal numbers from the noun phrase (S220), and the subprocess of rematizing the noun in the noun phrase (S230) proceeding in sequence, the present invention is not limited to this, and the order in which the subprocesses for extracting candidate nouns from nouns are performed can be arbitrarily determined, or they can be performed simultaneously.

[0063] Subsequently, the learning data generation device 1000 can verify whether the candidate noun corresponds to a physical object (S240).

[0064] In other words, the learning data generator 1000 can extract nouns that correspond to physical objects from among the candidate nouns by querying the synset of WordNet or the identifier of Wikidata for each candidate noun to confirm whether it corresponds to a physical object noun.

[0065] For example, the learning data generator 1000 can determine whether a particular candidate noun corresponds to a physical object by checking whether the hypernym corresponding to that particular candidate noun in WordNet includes a physical object noun, or by traversing the instances and subclasses corresponding to the identifier of the particular candidate noun in Wikidata to a pre-set depth and checking whether at least one pre-set noun identifier corresponding to a physical object noun appears. In this case, the noun identifier corresponding to a physical object noun may include an identifier corresponding to "physical object" or an identifier corresponding to "object," but the present invention is not limited to this, and may also include identifiers corresponding to various nouns corresponding to physical objects.

[0066] As another example, the learning data generator 1000 can input a specific candidate noun (one of the candidate nouns) into an LLM (Large Language Model) and use the LLM to check whether a physical object noun is included in the hypernomial words corresponding to the specific candidate noun in WordNet, or it can traverse instances and subclasses corresponding to the identifier characters of the specific candidate noun in Wikidata to a pre-set depth and check whether at least one pre-set noun identifier character corresponding to a physical object noun appears.

[0067] To explain more specifically using Wikidata as an example, we can confirm that "Baekdusan" is an instance of "mountain," "mountain" is a subclass of "natural geographic object," "natural geographic object" is a subclass of "natural physical object," and "natural physical object" is a subclass of "physical object." Through this, we can confirm that "Baekdusan" is a noun corresponding to a physical object.

[0068] Referring again to Figures 1 and 2, the learning data generation device 1000 can detect the bounding boxes of objects corresponding to nouns from the learning image 10 (S300).

[0069] At this time, the learning data generation device 1000 can detect the bounding boxes of objects corresponding to nouns from the learning image 10 through an object detector based on text prompts such as GroundingDINO or TAP, or an object detector based on vision prompts such as T-rex2. However, the present invention is not limited thereto, and various forms of models can be used to detect objects corresponding to nouns from the learning image 10, such as an object detector based on multimodal prompts that support both text and vision.

[0070] Next, the learning data generation device 1000 can generate additional ground truth annotations 21 by referring to each of the detected bounding boxes and their corresponding nouns, and add the additional ground truth annotations 21 to the learning image 10 (S400).

[0071] At this time, the learning data generator 1000 can filter and remove some bounding boxes or nouns that are deemed to have low accuracy or reliability among the detected bounding boxes or nouns by referring to each of the detected bounding boxes and each of the corresponding nouns.

[0072] For example, the training data generator 1000 can filter out and remove bounding boxes that are smaller than a pre-set critical size from among the detected bounding boxes.

[0073] As another example, the training data generator 1000 can filter and remove bounding boxes that have a predicted confidence value lower than a previously set critical predicted confidence value among the detected bounding boxes. In this case, the predicted confidence value may be generated by an object detector that detects objects corresponding to nouns.

[0074] As another example, the training data generator 1000 can check the text-region alignment score, which is the alignment score between a noun and the corresponding bounding box image region, through the CLIP (Contrastive Language-Image Pre-training) model, and filter out bounding boxes with a text-region alignment score lower than a pre-set critical text-region alignment score among the detected bounding boxes.

[0075] As another example, the training data generation device 1000 can filter and remove overlapping bounding boxes within the detected bounding boxes through NMS (Non-Maximum Suppression).

[0076] As another example, the training data generator 1000 can remove nouns corresponding to objects that do not exist in the training image. That is, nouns corresponding to objects that do not actually exist in the training image 10 may be included in the pseudo-annotations due to hallucination by the artificial intelligence model that generates pseudo-annotations in the training image 10, and these can be identified and removed. At this time, the pseudo-annotations can be cross-validated using models such as InternVL3 and Qwen3-VL to confirm whether or not the noun is hallucination.

[0077] Furthermore, the training data generation device 1000 can also add mask annotations to generate more accurate ground truth annotations.

[0078] In other words, the training data generator 1000 can also generate an object mask from each of the detected bounding boxes, and generate additional ground truth annotations by referencing each of the object masks and each of the nouns.

[0079] For example, the learning data generation device 1000 can input the learning image 10, bounding box information, and nouns into a segmentation model such as SAM3, and use the segmentation model to generate object masks corresponding to objects within the bounding box.

[0080] Next, the training data generation device 1000 can generate a final ground truth annotation corresponding to the training image 10 by referring to the initial ground truth annotation and the additional ground truth annotation (S500).

[0081] For example, the learning data generator 1000 can check whether the initial category names included in the initial ground truth annotations and the additional category names included in the additional ground truth annotations are words that already exist in the dictionary data. Incidentally, the dictionary data does not necessarily mean previous data, but could mean dictionary data.

[0082] At this time, the learning data generation device 1000 can also perform at least one of the following subprocesses when checking whether the initial category name and the additional category name are words that exist in the dictionary data: a subprocess that checks whether a specific category name is queried in a word set (words-set) generated by referring to the dictionary data; a subprocess that checks whether a synoset corresponding to a specific category name in WordNet is available; and a subprocess that checks whether a specific category name is queried in WordFreg.

[0083] The learning data generator 1000 can maintain a specific category name if it is a word that exists in the dictionary data, either among the initial category names or additional category names. If the specific category name is not a word that exists in the dictionary data, it can change the specific category name using the SymSpell algorithm with a general-purpose word that is within a pre-set edit distance of the specific category name.

[0084] For example, if a specific category name is "pizzza," it can be changed to the general word "pizza," and if it's "hankerchief," it can be changed to "handkerchief."

[0085] However, the present invention is not limited to this, and various language models other than SymSpell can be used to perform tasks such as correcting misspellings and proofreading spellings in category names.

[0086] According to this embodiment of the present invention, it is possible to generate a training dataset with more accurately defined category names while having extended ground truth annotations corresponding to physical objects by utilizing a training image with final ground truth annotations, thereby enabling the artificial intelligence model to be trained to perform its target task more accurately.

[0087] The embodiments of the present invention described above are embodied in the form of program instructions that can be executed through a variety of computer components and can be stored on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., individually or in combination. The program instructions recorded on the computer-readable recording medium may be specifically designed and configured for the present invention, or may be publicly known and usable by those skilled in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code, such as that produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices are configured to operate as one or more software modules to perform the processing according to the present invention, and vice versa.

[0088] Although the present invention has been described above with reference to specific components and other details, as well as limited embodiments and drawings, these are provided only to aid in a more general understanding of the invention. The present invention is not limited to the above embodiments, and a person with ordinary skill in the art to which the invention belongs can make various modifications and variations from this description.

[0089] Therefore, the concept of the present invention should not be limited to the embodiments described above, and it can be said that not only the claims described later, but also all modifications that are equivalent or equivalent to the claims of this invention, fall within the scope of the concept of the present invention. [Explanation of Symbols]

[0090] 1000 Training Data Generator 1100 memory 1200 Processor

Claims

1. In a method for generating training datasets for artificial intelligence learning, (a) Once pseudo annotations (the pseudo annotations are image captions that caption the training images) corresponding to at least one training image having at least one initial ground truth annotation included in the public dataset are obtained, the training data generator parses noun phrases using the pseudo annotations and extracts nouns corresponding to physical objects from the parsed noun phrases, (b) The learning data generation device detects bounding boxes of objects corresponding to nouns from the learning image, generates additional ground truth annotations by referencing each of the detected bounding boxes and each of the corresponding nouns, and adds the additional ground truth annotations to the learning image. (c) The learning data generation device generates a final ground truth annotation corresponding to the learning image by referring to the initial ground truth annotation and the additional ground truth annotation, Includes, In step (a) above, The learning data generation device extracts candidate nouns by performing at least one of the following subprocesses through a SpaCy model: a subprocess to change the part of speech of the noun phrase, a subprocess to remove quantifiers and ordinal numbers from the noun phrase, and a subprocess to rematize the nouns in the noun phrase; and (ii) extracts the nouns corresponding to physical objects from the candidate nouns by querying each of the candidate nouns against a WordNet synset or Wikidata identifier to confirm whether they correspond to physical object nouns.

2. In step (a) above, The method according to claim 1, wherein the learning data generating device confirms whether the specific candidate noun corresponds to the physical object by checking in WordNet whether the hypernonym corresponding to the specific candidate noun is included in the hypernonym of the specific candidate noun, or by traversing the instances and subclasses corresponding to the identifier of the specific candidate noun in Wikidata to a pre-set depth and checking whether at least one pre-set noun identifier corresponding to the physical object noun appears.

3. In step (a) above, The method according to claim 2, wherein the learning data generation device inputs the specific candidate noun into an LLM (Large Language Model), uses the LLM to check in WordNet whether the physical object noun is included in the hypernyms corresponding to the specific candidate noun, or traverses the instances and subclasses corresponding to the identifiers of the specific candidate noun in the wikidata to a pre-set depth, and checks whether at least one pre-set noun identifier appears corresponding to the physical object noun.

4. A method for generating a training dataset for artificial intelligence learning, (a) Once pseudo annotations (the pseudo annotations are image captions that caption the training images) corresponding to at least one training image having at least one initial ground truth annotation included in the public dataset are obtained, the training data generator parses noun phrases using the pseudo annotations and extracts nouns corresponding to physical objects from the parsed noun phrases, (b) The learning data generation device detects bounding boxes of objects corresponding to nouns from the learning image, generates additional ground truth annotations by referencing each of the detected bounding boxes and each of the corresponding nouns, and adds the additional ground truth annotations to the learning image. (c) The learning data generation device generates a final ground truth annotation corresponding to the learning image by referring to the initial ground truth annotation and the additional ground truth annotation, Includes, In step (c) above, The learning data generation device checks whether each of the initial category names included in the initial ground truth annotation and the additional category names included in the additional ground truth annotation are words that already exist in the dictionary data. If a specific category name, which is either one of the initial category names or the additional category names, is a word that exists in the dictionary data, the device maintains the specific category name as is. If the specific category name is not a word that exists in the dictionary data, the device modifies the specific category name using the SymSpell algorithm and a general-purpose word within a pre-set edit distance from the specific category name.

5. In step (c) above, The method according to claim 4, wherein the learning data generation device performs at least one of the following subprocesses when checking whether the initial category name and the additional category name are words that exist in the dictionary data: a subprocess when checking whether the specific category name is queryable in a word set (words-set) generated by referring to the dictionary data; a subprocess when checking whether a synoset corresponding to the specific category name in WordNet is available; and a subprocess when checking whether the specific category name is queryable in WordFreg.

6. In step (b) above, The method according to claim 1 or 4, wherein the training data generation device generates the additional ground truth annotation by referring to each of the detected bounding boxes and each of the corresponding nouns, through at least one of the following subprocesses: filtering out bounding boxes among the detected bounding boxes that are smaller than a pre-set critical size; filtering out bounding boxes among the detected bounding boxes that have a predicted confidence value lower than a pre-set critical predicted confidence value; filtering out bounding boxes among the detected bounding boxes that have a text-to-region alignment score lower than a pre-set critical text-to-region alignment score; filtering out bounding boxes among the detected bounding boxes that overlap through NMS (Non-Maximum Suppression); and removing nouns from the nouns that correspond to objects not present in the training image.

7. In step (a) above, The method according to claim 1 or 4, wherein the training data generation device filters out candidate images included in the public dataset from images with a resolution lower than a previously set critical resolution, images tagged with NSFW (No Safe For Work), and low-content web images, and acquires the remaining candidate images as training images.

8. In step (b) above, The method according to claim 1 or 4, wherein the training data generation device generates an object mask from each of the detected bounding boxes, and generates the additional ground truth annotation by referencing each of the object masks and each of the nouns.

9. In a training data generation device that generates training datasets for artificial intelligence learning, Memory containing instructions for generating training datasets for artificial intelligence learning, A processor that generates the training dataset for artificial intelligence learning based on the instructions stored in the memory, Includes, The processor performs the following processes: (I) when pseudo annotations (the pseudo annotations are image captions that caption the training images) corresponding to at least one training image having at least one initial ground truth annotation included in the public dataset are obtained, the process of parsing noun phrases with the pseudo annotations and extracting nouns corresponding to physical objects from the parsed noun phrases; (II) when bounding boxes of objects corresponding to the nouns are detected from the training images, when additional ground truth annotations are generated by referencing each of the detected bounding boxes and each of the corresponding nouns, and when additional ground truth annotations are added to the training images; and (III) when final ground truth annotations corresponding to the training images are generated by referencing the initial ground truth annotations and the additional ground truth annotations. The processor is a learning data generating device that, in the process of (I), (i) extracts candidate nouns by performing at least one of the following subprocesses through a SpaCy model: a subprocess to change the part of speech of the noun phrase, a subprocess to remove quantifiers and ordinal numbers from the noun phrase, and a subprocess to rematize the nouns in the noun phrase; and (ii) extracts the nouns corresponding to physical objects from the candidate nouns by querying the WordNet synset or Wikidata identifier for each of the candidate nouns to see if they correspond to physical object nouns.

10. The learning data generation device according to claim 9, wherein the processor, in process (I), checks whether the physical object noun is included in the hypernym corresponding to the specific candidate noun in WordNet for any one of the candidate nouns, or traverses the instances and subclasses corresponding to the identifier of the specific candidate noun in Wikidata to a pre-set depth and checks whether at least one pre-set noun identifier corresponding to the physical object noun appears.

11. The learning data generation device according to claim 10, wherein the processor, in process (I), inputs the specific candidate noun into LLM (Large Language Model), and uses the LLM to check in WordNet whether the physical object noun is included in the hypernym corresponding to the specific candidate noun, or traverses the instances and subclasses corresponding to the identifier characters of the specific candidate noun in the wikidata to a pre-set depth, and checks whether at least one pre-set noun identifier character corresponding to the physical object noun appears.

12. A learning data generation device for generating a learning dataset for artificial intelligence learning, Memory containing instructions for generating training datasets for artificial intelligence learning, A processor that generates the training dataset for artificial intelligence learning based on the instructions stored in the memory, Includes, The processor performs the following processes: (I) when pseudo annotations (the pseudo annotations are image captions that caption the training images) corresponding to at least one training image having at least one initial ground truth annotation included in the public dataset are obtained, the process of parsing noun phrases with the pseudo annotations and extracting nouns corresponding to physical objects from the parsed noun phrases; (II) when bounding boxes of objects corresponding to the nouns are detected from the training images, when additional ground truth annotations are generated by referencing each of the detected bounding boxes and each of the corresponding nouns, and when additional ground truth annotations are added to the training images; and (III) when final ground truth annotations corresponding to the training images are generated by referencing the initial ground truth annotations and the additional ground truth annotations. The processor, in the (III) process, checks whether each of the initial category names included in the initial ground truth annotation and the additional category names included in the additional ground truth annotation are words that already exist in the dictionary data, and if a specific category name, which is either one of the initial category names or the additional category names, is a word that exists in the dictionary data, the processor maintains the specific category name as is, and if the specific category name is not a word that exists in the dictionary data, the processor modifies the specific category name using the SymSpell algorithm and a general-purpose word within the already set edit distance of the specific category name, thereby creating a learning data generation device.

13. The learning data generation apparatus according to claim 12, wherein the processor, in the (III) process, checks whether the initial category name and the additional category name are words that exist in the dictionary data, by performing at least one of the following subprocesses: checking whether the specific category name is queried in a word set (words-set) generated by referring to the dictionary data; checking whether a synoset corresponding to the specific category name in WordNet is available; and checking whether the specific category name is queried in WordFreg.

14. The learning data generation apparatus according to claim 9 or 12, wherein the processor generates the additional ground truth annotation in the (II) process by referring to each of the detected bounding boxes and each of the corresponding nouns, through at least one of the following subprocesses: filtering out bounding boxes among the detected bounding boxes that are smaller than a pre-set critical size; filtering out bounding boxes among the detected bounding boxes that have a predicted confidence value lower than a pre-set critical predicted confidence value; filtering out bounding boxes among the detected bounding boxes that have a text-to-region alignment score lower than a pre-set critical text-to-region alignment score; filtering out bounding boxes among the detected bounding boxes that overlap through NMS (Non-Maximum Suppression); and removing from the nouns nouns nouns corresponding to objects that do not exist in the learning image.

15. The training data generation apparatus according to claim 9 or 12, wherein the processor, in process (I), filters out candidate images included in the public dataset from images with a resolution lower than a previously set critical resolution, images tagged with NSFW (No Safe For Work), and low-content web images, and acquires the remaining candidate images as training images.

16. The training data generation apparatus according to claim 9 or 12, wherein the processor generates an object mask from each of the detected bounding boxes in the (II) process, and generates the additional ground truth annotation by referencing each of the object masks and each of the nouns.

Images