Method and apparatus for generating learning models using multiple label sets

Abstract

The present disclosure relates to a method and system for optimizing a learning model for a target device. The method according to an embodiment of the present disclosure may include a method for generating a learning model using multiple label sets, the method including the steps of obtaining an image set including a plurality of images, and generating a plurality of different types of label sets for the image set.

Description

[Technical Field]

[0001] The technical idea of ​​this disclosure relates to a method and apparatus for generating a learning model using multiple label sets. [Background technology]

[0002] Machine learning is a field of artificial intelligence (AI) that develops algorithms and technologies that enable computers to learn based on data. It is said to be a core technology in a wide range of fields, including image processing, video recognition, voice recognition, and internet search, and has shown excellent results in prediction, object detection, object classification, object segmentation, and anomaly detection.

[0003] To perform machine learning, it is necessary to generate training data first, and to generate such training data, labeling must be performed to mark the correct answer (label) to be detected in the training data. For example, when trying to detect a specific object from an image, labels containing information such as the type, position, and boundary of the object are generated by associating them with the training data, and a machine learning algorithm learns the training data based on these labels, thereby generating a learning model that detects the object.

[0004] However, conventional labeling methods limit the ability to derive results based on a wide variety of labels from the same training data and to utilize them as an ensemble, since each training data set contains only a single type of label. In addition, the method of generating training data separately for each individual label has the disadvantage of requiring extra resources to generate, store, and manage the training data.

[0005] On the other hand, when constructing training data, collecting some types of labeled data may be an easy task, but for other types, labeling itself may be relatively difficult. For example, while it may be easy to collect data that has been classified into classes, collecting data that has been segmented into objects corresponding to each class may be difficult. As such, there is the inconvenience that it takes a huge amount of time and cost to secure training data that has been labeled with a high level of difficulty. Summary of the Invention [Problem to be solved by the invention]

[0006] The technical idea of ​​this disclosure aims to provide a method and apparatus for generating a learning model using multiple label sets to solve the above problems.

[0007] The technical problems that the method for generating a learning model using multiple label sets and the apparatus therefor according to the technical idea of ​​this disclosure aim to solve are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0008] According to one aspect of the technical idea of ​​this disclosure, a method for generating a learning model using multiple label sets may include the steps of obtaining an image set including multiple images, and generating multiple label sets of different types for the image set.

[0009] According to an exemplary embodiment, the method may further include inputting the image set into a network function to generate a plurality of learning models corresponding to at least two or more of the plurality of label sets.

[0010] The method for generating a learning model using multiple label sets may further include inputting the image set into a network function to generate a plurality of learning models corresponding to at least two or more of the plurality of label sets.

[0011] According to this exemplary embodiment, the step of generating the plurality of label sets may include the steps of receiving, from a plurality of users, label information for at least one of a plurality of images belonging to the image set, and generating the plurality of label sets corresponding to each of the label information by labeling the plurality of images included in the image set based on the label information.

[0012] According to an exemplary embodiment, the method for generating a learning model using a multiple label set further includes a step of comparing performance with at least two or more of the plurality of learning models, and the performance may be calculated based on at least one of recall, precision, accuracy, and combinations thereof.

[0013] According to an exemplary embodiment, each of the plurality of learning models may perform any one of object detection, object classification, object segmentation, character recognition, prediction, and anomaly detection.

[0014] According to an exemplary embodiment, the method for generating a learning model using the multiple label set may further include a step of combining at least two of the learning models to construct an ensemble model.

[0015] According to an exemplary embodiment, the plurality of label sets may comprise a first label set and a second label set of different types having different labeling difficulty levels, and generating the plurality of label sets may include generating the second label set based on the first label set having a lower labeling difficulty level.

[0016] According to an exemplary embodiment, the step of generating the plurality of label sets may include the steps of: receiving first label information for at least one of the plurality of images belonging to the image set from at least one user; labeling the plurality of images included in the image set based on the first label information, thereby generating a first label set including a first label for each of the plurality of images; inputting the image set and the first label set into a first network function to perform learning, and generating a second label set including a second label for each of the plurality of images based on an output result.

[0017] According to an exemplary embodiment, the output result of the first network function may include information regarding whether the first network function output a result corresponding to the first label based on at least one portion for each of the plurality of images.

[0018] According to an exemplary embodiment, the output result of the first network function may include, for each of the plurality of images, information in the form of a heat map as to whether or not the first network function has output a result corresponding to the first label based on at least one portion of the plurality of images.

[0019] According to an exemplary embodiment, the output results of the first network function may include a Class Activation Map (CAM) for each of the plurality of images for a result corresponding to the first label.

[0020] According to an exemplary embodiment, in the step of generating the second label set, The second labels may be generated by extracting weakly labeled regions from each of the class activation maps corresponding to each of the plurality of images by setting a threshold value to a predetermined value or range.

[0021] According to an exemplary embodiment, the method for generating a learning model using multiple label sets may further include generating a learning model by inputting the image set and the second label set into a second network function and performing training.

[0022] According to an exemplary embodiment, the first label may be an object classification label, and the second label may be an object recognition label and / or an object classification label.

[0023] According to an exemplary embodiment, the first label may be an object recognition label and the second label may be an object segmentation label.

[0024] According to one aspect of the technical idea of ​​this disclosure, an apparatus for generating a learning model using multiple label sets includes at least one processor and a memory that stores a program that can be run by the processor, and the processor may run the program to obtain an image set including multiple images and generate multiple label sets of different types for the image set. [Effects of the Invention]

[0025] According to an embodiment of the technical idea of ​​this disclosure, different types of label sets are generated for a single image set, and based on this, multiple learning models corresponding to the respective label sets are generated, and performance comparison is performed, thereby making it possible to select a label set and / or learning model that is optimized for the task being performed.

[0026] According to an embodiment of the technical idea of ​​this disclosure, a more accurate final result can be obtained by combining multiple learning models generated based on different label sets to construct an ensemble model suitable for the task being performed.

[0027] According to an embodiment of the technical idea of ​​this disclosure, after learning a model based on labels of a type that is easy to label, labels of a relatively low level of difficulty can be automatically generated to generate a learning model.

[0028] The effects obtained by the method and apparatus according to the technical idea of ​​this disclosure are not limited to the effects described above, and other effects not mentioned will be clearly understood by a person having ordinary skill in the technical field to which this disclosure pertains from the following description.

[0029] To more fully understand the drawings referred to in this disclosure, a brief description of each drawing is provided. [Brief explanation of the drawings]

[0030] [Figure 1] 1 is a flowchart illustrating a method for generating a learning model using multiple label sets according to an embodiment of the present disclosure. [Figure 2] 2 is a flowchart for explaining step S120 of FIG. 1 according to a first embodiment. [Figure 3] 1 is a flowchart illustrating a method for generating a learning model using multiple label sets according to an embodiment of the present disclosure. [Figure 4] 1 is a flowchart illustrating a method for generating a learning model using multiple label sets according to an embodiment of the present disclosure. [Figure 5] FIG. 1 is a conceptual diagram illustrating the structure of training data and network functions according to an embodiment of the present disclosure. [Figure 6] 10 is a flowchart for explaining a second embodiment regarding step S120 in FIG. [Figure 7] 7 is a flowchart for explaining an embodiment relating to step S123 in FIG. 6. [Figure 8] FIG. 1 is a conceptual diagram illustrating the structure of training data and network functions according to an embodiment of the present disclosure. [Figure 9] FIG. 10 is a diagram for exemplifying a process of generating a second label from a first label in a method for generating a learning model using a multiple label set according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram for exemplifying a process of generating a second label from a first label in a method for generating a learning model using a multiple label set according to an embodiment of the present disclosure. [Figure 11] FIG. 10 is a diagram for exemplifying a process of generating a second label from a first label in a method for generating a learning model using a multiple label set according to an embodiment of the present disclosure. [Figure 12] FIG. 10 is a diagram for exemplifying a process of generating a second label from a first label in a method for generating a learning model using a multiple label set according to an embodiment of the present disclosure. [Figure 13] FIG. 1 is a block diagram illustrating a simplified configuration of a device for generating a learning model using multiple label sets according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0031] Since the technical idea of ​​this disclosure can be variously modified and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail. However, this is not intended to limit the technical idea of ​​this disclosure to the specific embodiments, and it is understood that all modifications, equivalents, and alternatives falling within the scope of the technical idea of ​​this disclosure are included.

[0032] In explaining the technical ideas of this disclosure, if it is recognized that a specific description of known technologies related to the present invention may obscure the gist of this disclosure, the detailed description will be omitted. Note that numbers used in the explanation of this disclosure (e.g., "first," "second," etc.) are merely identification codes for distinguishing certain components from other components.

[0033] Furthermore, in this disclosure, when a component is referred to as being "coupled" or "connected" to another component, the component may be directly coupled or connected to the other component, but unless otherwise specified in this specification or clearly contradicted by the context, there may also be other components between them and the components may be coupled or connected via the other components.

[0034] Furthermore, the terms "unit," "device," "subsystem," "module," and the like used in this disclosure refer to a unit that processes at least one function or operation, which may be realized by hardware, software, or a combination of hardware and software, such as a processor, microprocessor, microcontroller, central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), digital signal processor (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), etc.

[0035] It should be made clear that the distinctions between components in this disclosure are merely made according to the main function that each component is responsible for. That is, two or more components described below may be combined into one component, or one component may be further divided into two or more components for each of its functions. It goes without saying that each component described below may perform some or all of the functions that are performed by other components in addition to its own main function, or that some of the main functions that each component is responsible for may be exclusively performed by other components.

[0036] The method according to the embodiment of the present disclosure may be performed in a computing device such as a personal computer, workstation, or server with computing power, or may be performed in a separate device for this purpose.

[0037] The method may also be performed in one or more computing devices. For example, at least one step of the method according to the embodiments of the present disclosure may be performed in a client device, and other steps may be performed in a server device. In such a case, the client device and the server device may be connected via a network to transmit and receive the computation results. Alternatively, the method may be performed using a distributed computing technique.

[0038] Furthermore, throughout this specification, the terms network function, computational model, neural network, and neural network may be used interchangeably. A neural network may generally be composed of a collection of interconnected computational units that may be referred to as nodes. Such nodes may also be referred to as neurons. A neural network is composed of at least one or more nodes. The nodes (or neurons) that make up a neural network may be connected to each other by one or more links.

[0039] Within a neural network, one or more nodes connected via links may form a relative input node / output node relationship. The concepts of input node and output node are relative, and any node that is an output node relative to one node may also be an input node relative to another node, and vice versa. As noted above, the input node / output node relationship may be generated around links. One or more output nodes may be connected to an input node via links, and vice versa.

[0040] In a relationship between an input node and an output node connected via a link, the value of data at the output node may be determined based on data input to the input node. Here, the link connecting the input node and the output node may have a weighting value. The weighting value may be variable and may be variable by a user or an algorithm so that the neural network performs a desired function. For example, when one or more input nodes are connected to one output node via respective links, the output node may determine the output node value based on the value input to the input node connected to the output node and the weighting value set for the link corresponding to each input node.

[0041] A subset of nodes constituting a neural network may constitute a layer. Some of the nodes constituting a neural network may constitute a layer based on their distance from the first input node. For example, a set of nodes whose distance from the first input node is n may constitute n layers. The distance from the first input node may be defined by the minimum number of links that must be traversed to reach that node from the first input node. However, this definition of a layer is merely arbitrary for illustrative purposes, and the order of layers within a neural network may be defined in a manner different from that described above. For example, the layer of a node may be defined by its distance from the final output node.

[0042] The neural network may include a deep neural network (DNN) that includes multiple hidden layers in addition to an input layer and an output layer. The deep neural network can be used to understand the latent structures of data. The deep neural network may include a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder, a generative adversarial network (GAN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a Q-network, a U-network, a Siamese network, a generative adversarial network (GAN), or the like. The above descriptions of deep neural networks are merely examples, and this disclosure is not intended to be limited thereto.

[0043] A neural network may be trained by at least one of supervised learning, unsupervised learning, semisupervised learning, or reinforcement learning. Training a neural network may be a process of applying knowledge to the neural network to cause the neural network to perform a particular operation.

[0044] The embodiments of this disclosure will be described in detail below.

[0045] FIG. 1 is a flowchart illustrating a method for generating a learning model using multiple label sets according to an embodiment of the present disclosure, and FIG. 2 is a flowchart illustrating an embodiment of step S120 in FIG.

[0046] In step S110, the device may acquire an image set including a plurality of images. Here, the plurality of images constituting the image set may be received from at least one server and / or terminal, or may be acquired by at least one imaging device connected via wired or wireless communication. For example, the plurality of images may be process images acquired in a process line such as product production, manufacturing, or processing, or may be medical images or videos of a patient's body parts captured using medical imaging equipment, but are not limited thereto.

[0047] In step S120, the device may generate multiple label sets for the image set.

[0048] In an embodiment, each of the plurality of label sets may include different types of labels, and the types of the labels or label sets including the labels may be distinguished according to the results to be obtained from the plurality of images through the network function.

[0049] For example, if an image set is composed of multiple process images, each image may contain labels suitable for performing various tasks, such as object classification, which allows you to determine whether a product is good or bad, object segmentation, which allows you to accurately determine the location of defects, object detection, which allows you to determine the location of each component that makes up the product, and character recognition, which allows you to determine serial numbers.Using this method, multiple label sets of different types (or purposes) may be generated corresponding to one image set.

[0050] However, this is merely an example and is not intended to be limiting, and label sets may be generated to suit a wide variety of functions that can be performed by network functions, such as object classification, object segmentation, object detection, character recognition, as well as prediction, anomaly detection, etc.

[0051] Depending on the embodiment, the device may be configured to include multiple label sets of the same type in the image set. For example, the multiple label sets may be generated based on label information generated by different users to detect the same object. In this case, the optimal label set and / or learning model may be selected by evaluating the performance of the learning models generated based on each label set.

[0052] In an embodiment, step S120 may be performed based on label information received from multiple users. For example, step S120 may include step S121 and step S122 as shown in FIG.

[0053] In step S121, the device may receive label information for at least one of a plurality of images belonging to an image set from a plurality of users (i.e., user terminals). That is, the device may provide an image set to the user terminal and, in response, receive label information for at least one of the plurality of images from the user terminal. In this case, the label information may be for performing object classification, object segmentation, object detection, character recognition, prediction, anomaly detection, etc.

[0054] In step S122, the device may generate a plurality of label sets corresponding to the respective label information by labeling each of the images constituting the image set based on the label information received from the user. As described above, the generated plurality of label sets may be of different types and may be intended to obtain different outputs.

[0055] In an embodiment, at least some of the plurality of label sets may be generated based on the results of training a network function based on other label sets.

[0056] The label sets may include a first label set and a second label set, and the second label set may be generated based on a training result generated by inputting the first label set into a predetermined network function. For example, if the first label set is composed of labels for object classification, the second label set for object segmentation may be generated based on a class activation map (CAM) generated in the training result of the network function.

[0057] In an embodiment, at least some of the label sets may be generated by modifying other label sets. For example, a second label set for object detection may be generated by modifying the width / height of labels included in a first label set for object segmentation.

[0058] In step S130, the device may input the image set to a network function to generate at least one learning model. For example, multiple learning models corresponding to at least two or more of the multiple label sets may be generated, and preferably, a learning model corresponding to each label set may be generated.

[0059] In this case, the learning model may perform one of object detection, object classification, object segmentation, character recognition, prediction, and anomaly detection, but is not limited thereto.

[0060] As described above with reference to Figures 3 and 4, a user may select from multiple learning models the model that is most suitable for the task, or may combine two or more learning models to create an ensemble model to derive more accurate results.

[0061] FIG. 3 is a flowchart illustrating a method for generating a learning model using multiple label sets according to an embodiment of the present disclosure.

[0062] Here, steps S310 to S330 of the method 300 are the same as steps S110 to S130 described above with reference to FIG. 1, and therefore a duplicated description will be omitted.

[0063] In step S340, the device may compare the performance of at least two or more of the multiple learning models generated based on the multiple label sets.

[0064] In an embodiment, the device may calculate the performance of the learning model based on at least one of recall, precision, accuracy, and a combination thereof. A variety of methods, such as cross-validation, can be used to calculate the performance of the learning model.

[0065] By performing such a performance comparison, it is possible to determine what type of label set and learning model is most useful for the task at hand. For example, when creating multiple label sets to identify the same defect using object segmentation and object detection, it is possible to compare which model performs better analysis depending on the type of defect.

[0066] FIG. 4 is a flowchart illustrating a method for generating a learning model using multiple label sets according to an embodiment of the present disclosure.

[0067] Here, steps S410 to S430 of the method 400 are the same as steps S110 to S130 described above with reference to FIG. 1, and therefore a duplicated description will be omitted.

[0068] In step S440, the device may combine at least two or more of the generated learning models to construct an ensemble model.

[0069] In other words, depending on the characteristics of the task to be performed, two or more learning models may be combined in a predetermined manner and the output values ​​from each learning model may be combined to arrive at a more accurate final result.

[0070] For example, if there is a learning model that classifies areas of a printed circuit board (PCB) using object segmentation and another learning model that can find broken lines (i.e., disconnections) using object segmentation or object detection, even if they both find broken lines, they may judge the pass / fail status differently depending on the classified PCB area. Therefore, the two models may be combined to build an ensemble model, and the output values ​​of these models may be combined to derive the final result.

[0071] Furthermore, for example, in relation to object segmentation, if class A is detected well in the dark and class B is detected well in the bright, an ensemble model can be constructed by combining an object segmentation model for identifying classes A and B with an object classification model for sorting dark and bright images, and the output values ​​of both learning models can be combined to derive a more accurate final result.

[0072] In an embodiment, the ensemble model may be generated based on a voting, bagging, boosting method, or the like, but is not limited thereto.

[0073] FIG. 5 is a conceptual diagram illustrating the structure of training data and network functions according to an embodiment of the present disclosure.

[0074] As shown in the figure, one image set 1310 includes multiple individual images, and each individual image includes a different type of label, thereby generating multiple label sets corresponding to the image set 1310.

[0075] The image set 1310 is then input to a network function 1320 to train the network function 1320, which allows multiple learned models 1330 to be generated that correspond to at least some of the multiple label sets.

[0076] In this case, the network function 1320 may be the same or different depending on the type of label set.

[0077] FIG. 6 is a flowchart for explaining a second embodiment regarding step S120 in FIG.

[0078] Step S120 may include steps S121 to S123 as shown in FIG.

[0079] In step S121, the device may receive first label information for at least one of a plurality of images belonging to an image set from a user (i.e., a user terminal). That is, the device may provide an image set to the user terminal and, in response, receive first label information for at least one of the plurality of images from the user terminal. In this case, the first label information may be for generating a first label corresponding to any one of object classification, object segmentation, object detection, character recognition, prediction, and anomaly detection.

[0080] In step S122, the device may generate a first label set including first labels for each of the multiple images by labeling each of the multiple images that make up the image set based on the first label information received from the user.

[0081] In step S123, the device may input the image set and the first label set into a first network function to perform learning, and generate a second label set including different types of second labels for each of the multiple images based on the output results.

[0082] In an embodiment, the first label and the second label may be types of labels with different labeling difficulty levels, that is, for example, in step S123, the device may generate a second label set with a high labeling difficulty level based on a first label set with a relatively low labeling difficulty level.

[0083] In an embodiment, the first label may be an object classification label, and the second label may be at least one of an object detection label and an object segmentation label. Also, in an embodiment, the first label may be an object detection label, and the second label may be an object segmentation label.

[0084] However, this is merely an example, and the present invention is not limited thereto, and depending on the embodiment, further labels for an image set may be generated based on different types of labels in a variety of ways, such as automatically generating character recognition labels based on object recognition labels or object segmentation labels.

[0085] In this case, the output result of the first network function may include information regarding whether the first network function output a result corresponding to the first label for each of the multiple images based on at least a portion of any of the images.

[0086] For example, the output result may be a display of at least a portion of the image that influenced the output of the result corresponding to the first label. The output result may include probability information associated with the result for each position on the image. The output result may include different representations depending on the degree of influence on the output of the result corresponding to the first label. For example, the output result may represent saturation, brightness, hue, texture, etc. differently depending on the degree of influence on the output of the result corresponding to the first label. For example, a portion that had a large influence may be displayed with high saturation, and a portion that had a small influence may be displayed with relatively low saturation. Alternatively, the output result may include a heat map displaying the degree of influence on the output of the result corresponding to the first label. For example, the device may obtain the output result using gradient-weighted class activation mapping (Grad-CAM), deep seeded region growing (DSRG), or the like.

[0087] In an embodiment, the device may generate the output result based on the results of operations in at least one or more layers or filters rather than the final output layer of the first network function.

[0088] In an embodiment, when the first label is an object classification label, the first network function may include one or more network functions for classifying objects. For example, the first network function may be trained using training data including classification results of objects included in images as first labels to classify objects included in a plurality of images constituting an image set. In this case, the output result of the first network function may be a representation, in a predetermined format, of the region and / or position of the image that influenced the output of the classification result.

[0089] In an embodiment, the output result of the first network may be a class activation map that contains information about at least a portion of the image that influenced the output result. Class activation maps are specifically discussed in the paper Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization (published October 7, 2017, by Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra).

[0090] In this case, step S123 may include step S123-1 and step S123-2 as shown in FIG.

[0091] In step S123-1, the device may input the image set and the first label set to a first network function to perform learning, and output a class activation map for each of the multiple images.

[0092] For example, the first label set may consist of object classification labels for a number of images that make up the image set.

[0093] Then, in step S123-2, the device may automatically generate second labels by setting a threshold value or range for each of the class activation maps to extract weakly labeled regions, where a weak label may refer to an automatically generated label that is not as accurate as a label manually generated by a user but has accuracy above a certain threshold.

[0094] For example, a second label may be automatically generated by setting a certain range for probability, color tone, saturation, brightness, etc. for the class activation map and extracting a predetermined area corresponding to this.

[0095] In this case, the second label may be at least one of an object recognition label and an object segmentation label.

[0096] Meanwhile, although not shown, the apparatus may generate a learning model by inputting the image set and the generated second label set to a second network function and performing learning in step S130 of FIG. 1. For example, the second labels may be object segmentation labels, and the second network function may include one or more network functions for performing segmentation on the objects. The second network function may include a network function (encoder) that reduces the dimension of the image and a network function (decoder) that expands the dimension of the image in order to perform segmentation on the objects included in the input image.

[0097] FIG. 8 is a conceptual diagram illustrating the structure of training data and network functions according to an embodiment of the present disclosure.

[0098] As shown in the figure, an image set 810 may include multiple individual images, each of which may include a first label, thereby providing a first label set corresponding to the image set 1310. For example, the first labels may be object classification labels.

[0099] The image set 810 may then be input along with the first label set to a first network function 820 for training to generate a first learned model, for example, an object classification model.

[0100] At this time, a class activation map or the like containing information about at least a portion of the images that influenced the output result may be obtained as an output of the first network function 820.

[0101] Based on the output of the first network function, second labels, which are weak labels for each individual image, are generated, allowing the image set 810 to include a second label set that is different in type from the first label set. For example, the second labels may be object segmentation labels.

[0102] The image set 810 may then be input along with a second label set to a second network function 840 for training to generate a second learned model, for example, an object segmentation model.

[0103] 9 to 11 are diagrams for exemplifying the process of generating a second label from a first label in the method for generating a learning model using a multiple label set according to an embodiment of the present disclosure.

[0104] First, referring to FIGS. 9 and 10, an object recognition label set or an object segmentation label set with a relatively high labeling difficulty may be generated based on an object classification label set with a low labeling difficulty.

[0105] First, as shown in Figure 9, an image (Figure 9(a)) containing object classification labels for the presence or absence of a lesion may be input into a network function, and the output result may be a class activation map (Figure 9(b)) that represents the probability of at least one region or position in the image that influenced the classification as a lesion in a predetermined manner (such as a heat map).

[0106] 10, a threshold value may be set to a predetermined value or range to extract weak label regions, and an object recognition label or an object segmentation label may be generated based on the weak label regions. In this case, the object recognition label or the object segmentation label may be a weak label.

[0107] Next, referring to FIG. 11, the generated object recognition labels or object segmentation labels may be input together with the image into an appropriate network function, which may then perform learning and output object recognition (FIG. 11(a)) or object segmentation results (FIG. 11(b)).

[0108] FIG. 12 is a diagram for exemplifying a process of generating a second label from a first label in a method for generating a learning model using a multiple label set according to an embodiment of the present disclosure.

[0109] Referring to FIG. 12, an object segmentation label set with a relatively high level of labeling difficulty may be generated based on an object recognition label set with a low level of labeling difficulty.

[0110] For example, a network function may be trained using an image (FIG. 12(a)) containing object recognition labels for detecting product defects or abnormalities, and as a result, a result image (FIG. 12(b)) containing object recognition results may be obtained.

[0111] Then, weakly labeled regions may be extracted from the resulting image by setting ranges of brightness, saturation, hue, etc., and object segmentation labels may be generated based on these.

[0112] FIG. 13 is a block diagram showing a simplified configuration of a device for generating a learning model using multiple label sets according to an embodiment of the present disclosure.

[0113] The communication unit 1310 may receive data (e.g., image sets, label information, etc.) from an external device. The communication unit 1310 may include a wired / wireless communication unit. If the communication unit 1310 includes a wired communication unit, the communication unit 1310 may include one or more components for communicating via a local area network (LAN), a wide area network (WAN), a value-added network (VAN), a mobile radio communication network, a satellite communication network, or a combination thereof. If the communication unit 1310 includes a wireless communication unit, the communication unit 1310 may transmit and receive data or signals wirelessly using cellular communication, wireless LAN (e.g., Wi-Fi), etc. In an embodiment, the communication unit may transmit and receive data or signals to and from an external device or an external server under the control of the processor 540.

[0114] The input unit 1320 may receive various user commands through external operations. To this end, the input unit 1320 may include or be connected to one or more input devices. For example, the input unit 1320 may be connected to various input interfaces, such as a keypad or a mouse, to receive user commands. To this end, the input unit 1320 may include not only a USB port but also an interface such as Thunderbolt. Furthermore, the input unit 1320 may include various input devices, such as a touch screen or a button, or may be coupled to these to receive external user commands.

[0115] The memory 1330 may store programs and / or program commands for operation of the processor 1340, and may temporarily or permanently store input / output data. The memory 1330 may include at least one type of storage medium selected from the group consisting of a flash memory type, a hard disk type, a multimedia card micro type, a card-type memory (e.g., SD or XD memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0116] Memory 1330 may also store various network functions and algorithms, and may store a wide variety of data, programs (one or more instructions), applications, software, commands, code, etc. for driving and controlling device 700.

[0117] The processor 1340 may control the overall operation of the device 1300. The processor 1340 may execute one or more programs stored in the memory 1330. The processor 1340 may refer to a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), or a dedicated processor on which the methods according to the concepts of the present disclosure are performed.

[0118] According to an embodiment of the present disclosure, the processor 1340 may perform computations for training a neural network. The processor 1340 may perform computations for training a neural network, such as processing input data for training in deep learning (DN), extracting features from the input data, calculating errors, and updating weights of the neural network using back propagation. At least one of a CPU, a GPU, a TPU, and a dedicated processor of the processor 1340 may process the training of the network function.

[0119] In an embodiment, the processor 1340 may obtain an image set including a plurality of images, generate a plurality of different types of label sets for the image set, and input the image set to a network function to generate at least one learning model corresponding to at least two or more of the plurality of label sets.

[0120] In this embodiment, the processor 1340 may receive label information from multiple users regarding at least one of the multiple images belonging to the image set, and generate the multiple label sets corresponding to each of the label information by labeling the multiple images included in the image set based on the label information.

[0121] In an embodiment, the plurality of label sets may include a first label set and a second label set, and the processor 1340 may generate the second label set based on an output result generated by inputting the first label set into the network function.

[0122] In an embodiment, the processor 1340 may compare the performance of at least two or more of the plurality of learning models based on at least one of recall, precision, accuracy, and combinations thereof.

[0123] In an embodiment, the processor 1340 may combine at least two of the learning models to construct an ensemble model.

[0124] In an embodiment, the processor 1340 may obtain an image set including a plurality of images and generate a plurality of label sets of different types for the image set, where the plurality of label sets may include a first label set and a second label set of different types having different labeling difficulties, and the processor 1340 may generate the second label set based on the first label set having a lower labeling difficulty.

[0125] In an embodiment, processor 1340 may receive first label information regarding at least one of the images belonging to the image set from at least one user, generate a first label set including a first label for each of the images by labeling the images included in the image set based on the first label information, input the image set and the first label set to a first network function to perform learning, and generate a second label set including a second label for each of the images based on the output result.

[0126] In an embodiment, processor 1340 may generate the second label by setting a threshold value to a predetermined value or range for each class activation map corresponding to each of the plurality of images and extracting weak labeling regions.

[0127] In an embodiment, the processor 1340 may generate a learning model by inputting the image set and the second label set into a second network function and performing learning.

[0128] Methods according to embodiments of the present disclosure may be embodied in computer-readable media and recorded as program commands that can be executed by a variety of computer means. The computer-readable media may include, alone or in combination, program commands, data files, data structures, and the like. The program commands recorded on the media may be those specially designed and constructed for this disclosure, or they may be those well known and available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as compact disc read-only memories (CD-ROMs) and digital versatile discs (DVDs); magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program commands, such as read-only memories (ROMs), random access memories (RAMs), and flash memories. Examples of program commands include not only machine language code, such as produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

[0129] Additionally, methods according to the disclosed embodiments may be provided in a computer program product, which may be traded as a commodity between sellers and buyers.

[0130] The computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, the computer program product may include a product in the form of a software program (e.g., a downloadable application) that is electronically distributed by an electronic device manufacturer or via an online marketplace (e.g., Google Play Store, application store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or may be temporarily generated. In this case, the storage medium may be a storage medium of a manufacturer's server, an online marketplace server, or a relay server that temporarily stores the software program.

[0131] In a system including a server and a client device, the computer program product may comprise a storage medium of the server or a storage medium of the client device. Alternatively, if a third device (e.g., a smartphone) is present and connected to the server or the client device via communication, the computer program product may comprise a storage medium of the third device. Alternatively, the computer program product may include the S / W program itself, which is transmitted from the server to the client device or the third device, or transmitted from the third device to the client device.

[0132] In this case, one of the server, the client device, and the third device may run the computer program product to perform the method according to the disclosed embodiments, or two or more of the server, the client device, and the third device may run the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0133] For example, a server (e.g., a cloud server or an artificial intelligence server) may launch a computer program product stored on the server to control client devices communicatively connected to the server to perform methods according to the disclosed embodiments.

[0134] Although the embodiments have been described in detail above, the scope of the present disclosure is not limited thereto in any way, and various modifications and improvements made by those skilled in the art using the basic concepts of the present disclosure as defined in the appended claims also fall within the scope of the present disclosure.

Claims

1. In a method for generating a learning model using multiple label sets, obtaining an image set including a plurality of images; generating a plurality of different types of label sets for the image set; and generating a plurality of learning models corresponding to at least two or more label sets among the plurality of label sets by inputting the image set including label information regarding at least two or more label sets among the plurality of label sets into a network function and training the network function; the plurality of label sets include a first label set and a second label set of different types; The step of generating a plurality of label sets includes: generating the second label set based on the first label set; The first label set is an object classification label set, and the second label set is at least one of an object recognition label set and an object segmentation label set. A method characterized by:

2. The step of generating a plurality of label sets includes: receiving the label information for at least one of a plurality of images belonging to the image set from a plurality of users, respectively; and generating the plurality of label sets corresponding to each of the label information by labeling the plurality of images included in the image set based on the label information. The method of claim 1.

3. further comprising comparing performance with at least two or more of the plurality of learning models; The performance is calculated based on at least one of recall, precision, accuracy, and combinations thereof. The method of claim 1.

4. Each of the plurality of learning models performs one of object detection, object classification, object segmentation, character recognition, prediction, and anomaly detection. The method of claim 1.

5. The method further includes combining at least two of the learning models to construct an ensemble model. The method of claim 1.

6. The step of generating a plurality of label sets includes: receiving first label information from at least one user for at least one of the images in the image set; generating a first label set including first labels for each of the plurality of images by labeling the plurality of images included in the image set based on the first label information; and a step of inputting the image set and the first label set into a first network function to perform learning, and generating a second label set including second labels for each of the plurality of images based on an output result. The method of claim 1.

7. The output result of the first network function is The information includes information regarding whether the first network function has output a result corresponding to the first label based on at least one portion of each of the plurality of images. The method of claim 6.

8. The output result of the first network function is The information includes, for each of the plurality of images, information in the form of a heat map as to whether a result corresponding to the first label is output based on at least one portion of the plurality of images. The method of claim 7.

9. The output result of the first network function is a class activation map (CAM) for each of the plurality of images for the results corresponding to the first label; The method of claim 8.

10. In the step of generating the second label set, The second labels are generated by extracting weak labeling regions by setting a threshold value to a predetermined value or range for each of the class activation maps corresponding to each of the plurality of images.

10. The method of claim 9.

11. The method further includes a step of generating a learning model by inputting the image set and the second label set into a second network function and performing learning. The method of claim 7.

12. In a device for generating a learning model using multiple label sets, at least one processor; a memory for storing a program executable by the processor; Equipped with The processor executes the program to acquire an image set including a plurality of images, and generate a plurality of different types of label sets for the image set; inputting the image set including label information relating to at least two or more of the plurality of label sets to a network function and training the network function to generate a plurality of learning models corresponding to at least two or more of the plurality of label sets; the plurality of label sets include a first label set and a second label set of different types; The processor: generating the second label set based on the first label set; The first label set is an object classification label set, and the second label set is at least one of an object recognition label set and an object segmentation label set. An apparatus characterized in that

Images