Method for training vision inspection model and system therefor

Abstract

A method for training a vision inspect model, according to an embodiment of the present invention, is a method by which a computing system comprising a memory and a processor trains a vision inspection model and comprises the steps of: performing a first process of detecting noisy label data within a training data set; acquiring a noisy data set and a clean data set according to the noisy label data detected on the basis of the first process; and performing a second process of training a vision inspection model on the basis of the acquired noisy data set and clean data set.

Description

Vision inspection model learning method and system thereof

[0001] The present invention relates to a vision inspection model learning method and system thereof. More specifically, the present invention relates to a two-stage vision inspection model learning method and system thereof, which separately trains a first-stage process for detecting noisy label data and a second-stage process for learning a classification model using learning policies optimized for each purpose, in order to efficiently learn a data set containing noisy labels.

[0002] In visual inspection, machine learning methods such as classification and detection are used to automate the recognition of potential defects occurring on the product's exterior, and ultimately the presence or absence of defects (OK / NG) is classified to prevent defective products from being leaked to the outside.

[0003] These machine learning methods require a training data set built through a labeling process in which a person manually labels the collected image data by determining the presence or absence of defects and assigning the correct answer to the data.

[0004] In order to secure the diversity of image patterns that include various environmental factors, a large amount of training data is required. However, since humans must label a large amount of data, errors by workers occur due to various factors such as decreased concentration of workers, differences in defect identification skills, and ambiguity of identification criteria and data. As a result, the possibility of incorrectly labeled data being included in the training data set is very high in the real world.

[0005] This noisy label issue occurs even in public datasets widely used in the field of computer vision, such as MNIST, CIFAR-10, and / or ImageNet.

[0006] Although many studies have proposed approaches that can learn robustly even on datasets containing noisy labels through technical approaches such as sample selection and / or loss regularization, most of these studies aim to improve classification accuracy evaluation metrics on general public datasets such as CIFAR-100.

[0007] Therefore, existing studies can obtain improved evaluation index results if the classification accuracy for good products is high even if the accuracy for defects is low.

[0008] However, in the field of vision inspection, the goal should be to improve the precision of good product data while maintaining the maximum recall of defective data, unlike existing studies, because the leakage of defective products to the outside must be prevented as much as possible.

[0009] That is, in the field of vision inspection, model learning must be performed to achieve high-performance classification accuracy, but technological development for this is insufficient.

[0010] Meanwhile, existing proposed technologies approach the problem by simultaneously detecting noisy label data and training a classification model.

[0011] Detecting noise label data is advantageous in that it selects noise labels through the distribution of correct labels within clusters with similar feature patterns in a feature pattern space widely distributed around image features (Texture). However, the learning policy for detecting such noise label data acts as a factor that lowers the performance of classification model learning that learns feature patterns centered around the ground-truth labels to better distinguish classes.

[0012] That is, in the case of the conventional method that approaches the detection of noisy label data and simultaneously trains a classification model, such as in the preceding literature 1 (“SSR: An Efficient and Robust Framework for Learning with Unknown Label Noise,”), there is a problem of deterioration in classification performance due to conflicting learning policies between feature learning for detecting noisy label data and classification learning focused on class differentiation. In addition, there is an issue of deterioration in performance for vision inspection data with a non-complex pattern shape due to feature learning that only considers feature transformations for one’s own sample without considering other samples based on negative cosine similarity.

[0013] On the other hand, in the case of conventional methods such as the preceding literature 2 (“Identifying Mislabeled Data using the Area Under the Margin Ranking,”), learning convergence is achieved for easy samples that are accurately labeled in the early learning stage, and learning convergence is achieved for noisy label samples in the later learning stage. This method proposes to detect noisy label samples based on the loss difference between generations (Epochs).

[0014] However, this conventional method has the problem that it is difficult to determine whether a sample is a noise label based on a generation at a certain point in learning, and there is a large fluctuation in the generation standard depending on the data.

[0015] On the other hand, contrastive learning and its derived variants in supervised learning have brought about significant advances in the field of image classification.

[0016] Here, contrastive learning refers to a learning method that reduces the distance between similar image samples in the feature space and increases the distance between dissimilar image samples.

[0017] In particular, in contrastive learning based on supervised learning, class information is utilized to make feature vectors of the same class closer in the feature space while learning to make feature vectors of different classes farther apart from each other, thereby improving the discriminability between classes.

[0018] This type of contrastive learning is a learning method that is strong in tasks that require high accuracy, such as industrial defect detection, because it can not only improve the classification accuracy of the model, but also induce the model to learn robustly even in the face of various variations in the input.

[0019] However, as described above, noisy label data not only significantly reduces the effectiveness of the above-mentioned contrastive learning, but also acts as a factor that interferes with learning during classification learning, thereby lowering the overall model performance.

[0020] Therefore, there is a need to develop and introduce new technologies that can solve the problems described above.

[0021] One embodiment of the present invention aims to provide a two-stage vision inspection model learning method and system thereof, which separately learns a first stage process of detecting noisy label data and a second stage process of learning a classification model using learning policies optimized for each purpose in order to efficiently learn a data set including noisy labels.

[0022] However, the technical problems to be solved by the present invention and embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

[0023] A method for learning a vision inspection model according to an embodiment of the present invention is a method for a computing system including a memory and a processor to learn a vision inspection model, the method comprising: performing a first process (Process 1) for detecting noisy label data in a training data set (Training Data Set); acquiring a noisy data set (Noisy Data Set) and a clean data set (Clean Data Set) based on the detected noisy label data based on the first process; and performing a second process (Process 2) for learning a vision inspection model based on the acquired noisy data set and clean data set.

[0024] In another aspect, the vision inspection model includes at least one encoder, an adaptation layer, a projection head, and a classifier.

[0025] In another aspect, the step of performing the first process includes the steps of initializing the vision inspection model, performing first warm-up training (Warm-up Training 1) for training the initialized vision inspection model for a predetermined number of epochs based on the training data set, detecting noise label data in the training data set based on the first warm-up trained vision inspection model, obtaining a random clean data set according to the detected noise label data, performing noise label detection training for training the first warm-up trained vision inspection model for a predetermined number of epochs based on the obtained random clean data set and the training data set, detecting noise label data in the training data set based on the noise label detection trained vision inspection model, and obtaining a noise data set and a clean data set according to the detected noise label data.

[0026] In another aspect, the step of performing the first warm-up learning includes a step of performing learning based on cross entropy loss.

[0027] In another aspect, the first warm-up learning step further includes the steps of extracting a predetermined data pair from the learning data set, performing data augmentation based on the extracted data pair, and obtaining weak augmentation data, which is data in which a relatively small deformation has occurred, and strong augmentation data, which is data in which a relatively large deformation has occurred, based on the performed data augmentation.

[0028] In another aspect, the first warm-up learning step further includes a step of obtaining first mix-up data, which is data obtained by mixing up the first weakly modified augmented data and the second weakly modified augmented data, and second mix-up data, which is data obtained by mixing up the first weakly modified augmented data and the first strong modified augmented data.

[0029] In another aspect, the first warm-up learning step further includes a step of performing learning according to the cross entropy loss based on the first mix-up data, the second mix-up data, and the first strong deformation augmentation data.

[0030] In another aspect, the step of detecting the noise label data includes the steps of calculating cosine similarity between other learning data based on the first learning data in the learning data set, the step of extracting a predetermined number (k) of neighboring learning data that are other learning data having a high calculated cosine similarity, the step of calculating a voting score for each of the extracted neighboring learning data, and the step of detecting the noise label data based on the calculated voting score.

[0031] In another aspect, the step of performing the noise label detection learning includes the step of training the projection head based on the first clean data set and contrastive loss, and the step of training the encoder, the adaptive layer, and the classifier based on the training data set and the cross entropy loss.

[0032] In another aspect, the step of performing the noise label detection learning includes the step of detecting an original data set corresponding to the first clean data set, the step of performing data augmentation based on the detected original data set, and the step of performing the noise label detection learning based on the augmented data.

[0033] In another aspect, the step of performing the second process includes a step of newly initializing the vision inspection model, a step of performing second warm-up training (Warm-up Training 2) for training the newly initialized vision inspection model for a predetermined number of generations based on the clean data set, and a step of performing high-performance classification model training for training the second warm-up trained vision inspection model for a predetermined number of generations based on the clean data set and the noise data set.

[0034] In another aspect, the step of performing the high-performance classification model learning includes a step of performing learning by using cross-entropy loss and supervised contrastive loss in parallel based on the clean data set, and a step of performing learning by using the contrastive loss based on the noise data set.

[0035] In another aspect, the vision inspection model learning method according to an embodiment of the present invention further includes a step of providing the vision inspection model learned based on the second process through a predetermined application service.

[0036] Meanwhile, a vision inspection model learning system according to an embodiment of the present invention includes at least one memory; and at least one processor for learning a vision inspection model by reading at least one application stored in the memory, wherein instructions of the processor include instructions for performing a first process (Process 1) for detecting noisy label data in a training data set (Training Data Set), a step for acquiring a noisy data set (Noisy Data Set) and a clean data set (Clean Data Set) according to the detected noisy label data based on the first process, and a second process (Process 2) for learning a vision inspection model based on the acquired noisy data set and the clean data set.

[0037] On the other hand, a computing device according to an embodiment of the present invention includes at least one encoder; at least one adaptation layer; at least one projection head; at least one classifier; and at least one processor for controlling the encoder, the adaptation layer, the projection head, and the classifier, wherein the processor performs a first process (Process 1) for detecting noisy label data in a training data set, acquires a noisy data set and a clean data set according to the detected noisy label data based on the first process, and performs a second process (Process 2) for training a vision inspection model based on the acquired noisy data set and the clean data set.

[0038] A vision inspection model learning method and system according to one embodiment of the present invention provides a two-stage vision inspection model learning method and system that separately learns a first stage process of detecting noise label data and a second stage process of learning a classification model using learning policies optimized for each purpose, thereby preventing a decrease in model performance due to learning data with incorrect labels, while maintaining the maximum recall for defective data and improving the precision for good data, thereby enabling the construction of a high-performance vision inspection model.

[0039] At this time, the vision inspection model learning method and system according to one embodiment of the present invention can detect noise label data while stably learning the model through the use of a warm-up learning strategy, a mix-up technique, and an adaptive layer in the first stage process, and can perform learning to improve the classification performance of the model through contrastive loss and cross-entropy loss based on supervised learning using the data refined as described above in the second stage process.

[0040] That is, the vision inspection model learning method and system according to one embodiment of the present invention can provide a high-performance vision inspection model learned by minimizing the influence of noise label data and maximizing the strength of contrastive learning by implementing an optimized learning policy for each step as described above.

[0041] However, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood from the description below.

[0042] FIG. 1 illustrates an example of a block diagram of a computing system that implements a two-stage vision inspection model learning service according to one embodiment of the present invention.

[0043] FIG. 2 illustrates an example block diagram of a computing device that implements a two-stage vision inspection model learning service according to one embodiment of the present invention.

[0044] FIG. 3 illustrates an example block diagram from another aspect of a computing device implementing a two-stage vision inspection model learning service according to one embodiment of the present invention.

[0045] Figure 4 illustrates a flowchart for explaining a first stage learning process according to one embodiment of the present invention.

[0046] Figure 6 illustrates an example showing changes in logit values ​​according to learning generations according to one embodiment of the present invention.

[0047] FIG. 7 illustrates a block flow diagram for explaining a warm-up learning framework of a first stage learning process according to one embodiment of the present invention.

[0048] FIG. 8 illustrates a block flow diagram for explaining a model learning framework after performing warm-up learning according to one embodiment of the present invention.

[0049] FIG. 9 illustrates a conceptual diagram for explaining a method for extracting a random clean set according to one embodiment of the present invention.

[0050] FIG. 10 illustrates a flowchart for explaining a method for extracting a random clean set according to one embodiment of the present invention.

[0051] FIG. 11 illustrates an example for explaining a method for selecting a clean set sample according to one embodiment of the present invention.

[0052] FIG. 12 illustrates a flowchart for explaining a second stage learning process according to one embodiment of the present invention.

[0053] FIG. 13 illustrates a block flow diagram for explaining a warm-up learning framework of a second-stage learning process according to one embodiment of the present invention.

[0054] The present invention is capable of various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, as well as the methods for achieving them, will become clear with reference to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms. In the following embodiments, the terms "first," "second," etc. are not used in a limiting sense but are used for the purpose of distinguishing one component from another. Furthermore, the singular expression includes the plural expression unless the context clearly indicates otherwise. Furthermore, terms such as "include" or "have" indicate the presence of a feature or component described in the specification, and do not preemptively exclude the possibility that one or more other features or components may be added. Furthermore, in the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to what is shown.

[0055] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same drawing reference numerals, and redundant descriptions thereof will be omitted.

[0056]

[0057] [Example system providing a two-stage vision inspection model learning service]

[0058] Hereinafter, an exemplary system for implementing a two-stage vision inspection model learning service that separately learns a first-stage process of detecting noisy label data and a second-stage process of learning a classification model according to a learning policy optimized for each purpose in order to efficiently learn a data set containing noisy labels is described in detail with reference to the attached drawings.

[0059] FIG. 1 illustrates an example of a block diagram of a computing system that implements a two-stage vision inspection model learning service according to one embodiment of the present invention.

[0060] Referring to FIG. 1, a computing system (1000) implementing a two-stage vision inspection model learning service of the present invention includes a user computing device (110), a server computing system (130), and a training computing system (150), and the devices are capable of communicating via a network (170).

[0061] The vision inspection model learning method according to one embodiment of the present invention may be implemented and provided locally by a user computing device (110), implemented and provided in the form of a web service by a server computing system (130) communicating with the user computing device (110), or implemented and provided by a user computing device (110) and a server computing system (130) in connection with each other.

[0062] At this time, in the embodiment, the user computing device (110) and / or the server computing system (130) can train the machine learning model (120 and / or 140) through interaction with a training computing system (150) communicatively connected via a network (170). The training computing system (150) may be separate from the server computing system (130) or may be part of the server computing system (130).

[0063] And at this time, the artificial intelligence model can be 1) trained directly locally by the user computing device (110), 2) trained by the server computing system (130) and the user computing device (110) interacting with each other through a network (170), and 3) trained by a separate training computing system (150) using various training techniques and learning techniques. And the artificial intelligence model trained by the training computing system (150) can be implemented in a manner of being provided / updated by transmitting the model to the user computing device (110) and / or the server computing system (130) through a network (170).

[0064] In some embodiments, the training computing system (150) may be part of a server computing system (130) or part of a user computing device (110).

[0065] The user computing device (110) may include any type of computing device, such as a smart phone, a mobile phone, a digital broadcasting device, a personal digital assistant (PDA), a portable multimedia player (PMP), a desktop, a wearable device, an embedded computing device, and / or a tablet PC.

[0066] Such a user computing device (110) includes at least one processor (111) and memory (112). Here, the processor (111) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0067] The memory (112) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof, and may include web storage of a server that performs a memory storage function on the Internet. The memory (112) may store data (113) and commands (114) necessary for the at least one processor (111) to perform functional operations such as training an artificial intelligence model or executing a vision inspection using an artificial intelligence model.

[0068] In one embodiment, the user computing device (110) may store at least one machine learning model (120).

[0069] In detail, the machine learning model (120) may be various machine learning models such as multiple neural networks (e.g., deep neural networks) or other types of machine learning models including nonlinear models and / or linear models, and may be composed of a combination thereof.

[0070] At this time, the neural network may include at least one of feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, and / or other types of neural networks.

[0071] In one embodiment, the user computing device (110) may receive at least one machine learning model (120) from a server computing system (130) via a network (170), store the model in a memory (112), and then execute the stored machine learning model (120) by a processor (111) to perform vision inspection, etc.

[0072] In another embodiment, the server computing system (130) may include at least one machine learning model (140), perform an operation through the machine learning model (140), and communicate data related thereto with the user computing device (110) in conjunction with the user computing device (110) to provide a two-stage vision inspection model learning service to the user.

[0073] For example, a user computing device (110) can perform a two-stage vision inspection model learning service in a manner that a server computing system (130) provides output for a user's input using a machine learning model (140) via the web.

[0074] Additionally, the artificial intelligence model may be implemented in such a way that at least some of the machine learning models (120 and / or 140) are executed on a user computing device (110) and the rest are executed on a server computing system (130).

[0075] In addition, the user computing device (110) may include at least one input component (121) that detects a user's input. For example, the user input component (121) may include a touch sensor (e.g., a touch screen and / or a touch pad, etc.) that detects a touch of a user's input medium (e.g., a finger or a stylus), an image sensor that detects a user's motion input, a microphone, a button, a mouse, and / or a keyboard that detects a user's voice input, etc. In addition, the user input component (121) may include an interface and an external controller when receiving an input to an external controller (e.g., a mouse and / or a keyboard, etc.) through an interface.

[0076] The server computing system (130) includes at least one processor (131) and memory (132). Here, the processor (131) may be composed of at least one or a plurality of processors electrically connected among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0077] And the memory (132) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. This memory (132) may store data (133) and commands (134) necessary for the processor (131) to perform functional operations such as training an artificial intelligence model or executing vision inspection using an artificial intelligence model.

[0078] In one embodiment, the server computing system (130) may be implemented to include at least one computing device. For example, the server computing system (130) may be implemented to operate multiple computing devices according to a sequential computing architecture, a parallel computing architecture, or a combination thereof. Additionally, the server computing system (130) may include multiple computing devices connected via a network (170).

[0079] Additionally, the server computing system (130) may store at least one machine learning model (140). For example, the server computing system (130) may include a neural network and / or other multi-layer nonlinear model as the machine learning model (140). Exemplary neural networks may include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0080] The training computing system (150) includes at least one processor (151) and memory (152). Here, the processor (151) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0081] And the memory (152) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. This memory (152) may store data (153) and instructions (154) necessary for the processor (151) to perform learning of an artificial intelligence model, etc.

[0082] For example, the training computing system (150) may include a model trainer (160) that trains a machine learning model (120 and / or 140) stored in the user computing device (110) and / or server computing system (130) using various training or learning techniques, such as backpropagation of errors (according to the framework illustrated in FIG. 3).

[0083] For example, such a model trainer (160) may perform updates to one or more parameters of a machine learning model (120 and / or 140) in a backpropagation manner based on a defined loss function.

[0084] In some implementations, performing backward propagation of errors may include performing truncated backpropagation through time. The model trainer (160) may perform a number of generalization techniques (e.g., weight reduction, dropout, and / or knowledge distillation) to improve the generalization ability of the trained machine learning model (120 and / or 140).

[0085] In particular, the model trainer (160) can train the machine learning model (120 and / or 140) based on a series of training data (161). Here, the training data (161) can include data in different formats, such as, for example, images, audio samples, and / or text. Examples of image types that can be used can include video frames, LiDAR point clouds, X-ray images, computed tomography scans, hyperspectral images, and / or various other forms of images.

[0086] Such training data (161) may be provided by a user computing device (110) and / or a server computing system (130). When the training computing device trains a machine learning model (120 and / or 140) on specific data of the user computing device (110), the machine learning model (120 and / or 140) may be characterized as a personalized model.

[0087] And the model trainer (160) includes computer logic utilized to provide the desired function.

[0088] Additionally, the model trainer (160) may be implemented as hardware, firmware, and / or software that controls a general-purpose processor. In one implementation, the model trainer (160) includes a program file stored in a storage device, which may be loaded into a memory (152) and executed by one or more processors (151). In another implementation, the model trainer (160) includes one or more sets of computer-executable data (153) and instructions (154) stored in a tangible computer-readable storage medium, such as a RAM hard disk or an optical or magnetic medium.

[0089] The network (170) includes, but is not limited to, a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), a Wireless Local Area Network (Wireless LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, and / or a Digital Multimedia Broadcasting (DMB) network.

[0090] In general, communication over a network (170) may be performed using any type of wired and / or wireless connection, using various communication protocols (e.g., TCP / IP, HTTP, SMTP, and / or FTP), encodings or formats (e.g., HTML and / or XML), and / or protection schemes (e.g., VPN, Secure HTTP, and / or SSL).

[0091] FIG. 2 illustrates an example block diagram of a computing device that implements a two-stage vision inspection model learning service according to one embodiment of the present invention.

[0092] As shown in FIG. 2, the computing device (100) included in the user computing device (110), the server computing system (130), and the training computing system (150) includes a plurality of applications (e.g., Application 1 to Application N). Each application may include a machine learning library and one or more machine learning models. For example, the applications may include an image processing (e.g., Detection, Classification, and / or Segmentation, etc.) application, a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and / or a chat-bot application.

[0093] In an embodiment, the computing device (100) may include a model trainer (160) for training an artificial intelligence model, and may store and operate the trained artificial intelligence model to provide output data according to predetermined input data (e.g., predetermined images, etc.).

[0094] Each application of the computing device (100) may communicate with a number of other components of the computing device (100), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In one embodiment, each application may communicate with each device component using an API (e.g., a public API). In one embodiment, the API used by each application may be specific to that application.

[0095] FIG. 3 illustrates an example block diagram from another aspect of a computing device (100) implementing a two-stage vision inspection model learning service according to one embodiment of the present invention.

[0096] Referring to FIG. 3, a computing device (300) includes a plurality of applications (e.g., Application 1 to Application N). Each application may communicate with a central intelligence layer. For example, the applications may include an image processing application, a text messaging application, an email application, a dictation application, a virtual keyboard application, and / or a browser application. In one embodiment, each application may communicate with the central intelligence layer (and models stored therein) using an API (e.g., a common API across all applications).

[0097] The central intelligence layer may include multiple machine learning models. For example, as illustrated in FIG. 3, at least some of the machine learning models may be provided to each application and managed by the central intelligence layer. In other implementations, two or more applications may share a single machine learning model. For example, in some implementations, the central intelligence layer may provide a single model to all applications. In some implementations, the central intelligence layer may be included within the operating system of the computing device (300) or implemented differently.

[0098] The central intelligence layer may communicate with a central device data layer. The central device data layer may be a centralized data repository for the computing device (300). As illustrated in FIG. 3, the central device data layer may communicate with a number of other components of the computing device (300), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer may communicate with each device component using an API (e.g., a private API).

[0099] The techniques described herein may refer to servers, databases, software applications, and other computer-based systems, as well as actions taken and information transmitted to or from such systems. It will be appreciated that the inherent flexibility of computer-based systems allows for a wide range of possible configurations, combinations, and division of labor and functionality between and among components. For example, the processes described herein may be implemented using a single device or component, or multiple devices or components operating in combination. Databases and applications may be implemented on a single system or in a distributed system across multiple systems. Distributed components may operate sequentially or in parallel.

[0100]

[0101] [2-Stage Vision Inspection Model Learning Service Provision Method]

[0102] Hereinafter, a method for implementing a two-stage vision inspection model learning service in which a computing system (1000) according to one embodiment of the present invention separately learns a first stage process of detecting noisy label data and a second stage process of learning a classification model according to a learning policy optimized for each purpose in order to efficiently learn a data set including noisy labels is described in detail.

[0103] Here, the Vision Inspection Model (VIM) according to an embodiment of the present invention may mean an image deep learning model that performs anomaly detection based on a predetermined input image and classifies and / or recognizes the image based on the anomaly detection.

[0104] For reference, anomaly detection can refer to the process of identifying abnormal patterns, outliers, and / or exceptions from specific data.

[0105] That is, anomaly detection can be a process of detecting components that deviate from the properties of normal data.

[0106] As an example, anomaly detection can be implemented based on a method such as grouping certain data into clusters and considering points that deviate from the clusters as outliers.

[0107] Therefore, in the embodiment, the vision inspection model (VIM) can determine whether a given input image contains a specific abnormal property, and classify and / or recognize the image based on the determination result.

[0108] Returning again, the two-stage vision inspection model (VIM) learning method of the computing system (1000) according to one embodiment of the present invention can learn the first stage process of detecting noise label data from the data set used when learning the vision inspection model (VIM) as described above and the second stage process of learning the vision inspection model (VIM) by separating them into learning policies optimized for each purpose.

[0109] That is, the two-stage vision inspection model (VIM) learning method of the computing system (1000) according to one embodiment of the present invention can use different learning policies for each stage through a two-stage approach that separates the noise label data detection stage and the high-performance classification model (in the embodiment, the vision inspection model (VIM)) learning stage.

[0110] For reference, the detection of noise label data, which is an incorrectly labeled correct answer (ground-truth), is advantageous in that it selects noise labels through the distribution of correct labels within clusters with similar feature patterns in a feature pattern space widely distributed around the image feature (texture). However, the learning policy for detecting such noise label data acts as a factor that lowers the performance of the classification model learning feature patterns centered around the correct answer (ground-truth) labels to better distinguish classes.

[0111] A two-stage vision inspection model (VIM) learning method of a computing system (1000) according to one embodiment of the present invention can solve the above issues of a conventional general learning method that performs noise label detection and classification learning in an integrated manner by performing learning optimized for the purpose of each stage.

[0112] In addition, a two-stage vision inspection model (VIM) learning method of a computing system (1000) according to one embodiment of the present invention utilizes the principle of contrastive learning within a noise label detection process, thereby implementing learning that takes into account even feature differences from other samples, thereby securing diversity of feature patterns extracted from the process and detecting cluster samples having similar features more accurately.

[0113] For reference, contrastive learning is a learning method that pulls similar image samples closer together in the feature space and pushes dissimilar image samples farther apart.

[0114] In contrastive learning using supervised learning, class information is used to learn how to make feature vectors of the same class closer together in the feature space while making feature vectors of different classes farther apart, thereby improving the discriminability between classes.

[0115] This is a learning method that is strong in tasks such as industrial defect detection that require high precision, as it not only improves classification accuracy but also enables robust learning in response to various variations in the input.

[0116] When contrastive learning and classification learning are performed in parallel, improved judgment accuracy results can be obtained compared to only performing classification learning based on the existing cross entropy loss.

[0117] However, noisy label data not only significantly reduces the effectiveness of this contrastive learning, but also acts as a factor that interferes with classification learning, lowering the overall classification performance.

[0118] Therefore, the 2-Stage Vision Inspection Model (VIM) learning method of the computing system (1000) according to one embodiment of the present invention can implement Vision Inspection Model (VIM) learning that reduces the influence of noise label data and takes advantage of the strengths of the contrastive learning described above by providing the 2-Stage Vision Inspection Model (VIM) learning service as described above.

[0119] In addition, a two-stage vision inspection model (VIM) learning method of a computing system (1000) according to one embodiment of the present invention can separate a data set that does not include a noise label (hereinafter, a clean set) and a data set that includes a noise label (hereinafter, a noisy set) within the vision inspection model (VIM) learning process and use them according to different learning policies.

[0120] At this time, the two-stage vision inspection model (VIM) learning method of the computing system (1000) according to one embodiment of the present invention can improve the precision of good product data while maintaining the maximum recall rate for defective data by utilizing the principle of contrastive learning even in the vision inspection model (VIM) learning process.

[0121] Hereinafter, a method for performing a first step process of detecting noise label data and a second step process of learning a classification model by a computing system (1000) according to one embodiment of the present invention will be separately described in detail.

[0122]

[0123] <Step 1: Noise Label Data Detection Process>

[0124] In an embodiment of the present invention, the computing system (1000) can perform warm-up training using cross entropy loss for a predetermined number of epochs based on the entire training data set.

[0125] In addition, in an embodiment of the present invention, the computing system (1000) can perform learning using cross entropy loss and contrastive loss after performing warm-up learning, thereby inducing similar samples to be concentrated in one cluster and different samples to be separated from each other.

[0126] Additionally, in an embodiment of the present invention, the computing system (1000) can evaluate the presence or absence of noise labels for the entire learning data at the end of each generation (Epoch) and update the clean set to be used for learning in the next generation (Epoch).

[0127] At this time, in the embodiment of the present invention, the computing system (1000) can train the vision inspection model (VIM) more robustly by utilizing the mixup technique.

[0128] For reference, Mixup is a data augmentation technique that helps calibrate the network during the learning phase by mixing two samples and assigning an intermediate adjusted label, and can reduce the influence of noisy label samples or ambiguous samples.

[0129] In an embodiment of the present invention, the computing system (1000) can filter out noise label data from the entire learning data set through the first-stage learning process described above, and can implement a high-performance classification model (in an embodiment, a vision inspection model (VIM)) that excludes the influence of noise label data by performing the second-stage learning process described below based on the clean set and the noise set evaluated in the final generation (Epoch).

[0130] In the following description, the classification model is described based on the vision inspection model (VIM) according to an embodiment of the present invention, but is not limited thereto.

[0131] Figure 4 illustrates a flowchart for explaining a first stage learning process according to one embodiment of the present invention.

[0132] In detail, referring to FIG. 4, a computing system (1000) according to an embodiment of the present invention may include a step of initializing a classification model (S101), a step of performing warm-up learning based on a predetermined generation (Epoch) (S103), a step of extracting a random clean set based on the entire learning data (S105), a step of performing noise label detection learning based on the random clean set (S107), and a step of extracting a decision clean set and a noise set based on the entire learning data (S109).

[0133] In more detail, in an embodiment, the computing system (1000) can initialize a classification model. (S101)

[0134] That is, in the embodiment, the computing system (1000) can perform initialization of a vision inspection model (VIM) according to the embodiment of the present invention.

[0135] FIG. 5 illustrates an example of an internal block diagram of a vision inspection model (VIM) according to one embodiment of the present invention.

[0136] Here, referring to FIG. 5, a vision inspection model (VIM) for classifying a given defect may include at least one encoder (E: Encoder), an adaptation layer (M: Adaption Layer), a projection head (PH: Projection Head), and a classifier (C: Classifier).

[0137] At this time, the encoder (E) for extracting features of a given input image in the embodiment may be initialized to an arbitrary value and trained with industrial vision inspection data from the beginning, or an encoder (E) of a backbone network pre-trained with a large data set such as ImageNet may be used to utilize the rich image feature expression ability from the beginning of training.

[0138] That is, in one embodiment, the vision inspection model (VIM) can be implemented using an encoder (E) of a pre-trained backbone network as in FIG. 5, wherein the backbone network can be based on various architectural structures including ResNet and / or ConvNeXt.

[0139] Meanwhile, the adaptation layer (M), projection head (PH), and classifier (C) described in the embodiment are composed of a fully connected layer and can be implemented in the form of a single linear layer or a multi-layer perceptron.

[0140] At this time, the projection head (PH) according to the embodiment can calculate contrast loss by projecting the output of the adaptation layer (M) into a low dimensional space.

[0141] Additionally, in the embodiment, the classifier (C) can perform learning for the purpose of distinguishing classes of vision inspection data.

[0142] In the embodiment, the projection head (PH) as above is used only in the learning stage, and during inference, a class of a given vision inspection data can be classified through a network composed of a learned encoder (E), an adaptation layer (M), and a classifier (C) excluding the projection head (PH).

[0143] Additionally, in the embodiment, the computing system (1000) can perform warm-up training based on a predetermined generation (Epoch). (S103)

[0144] In detail, in an embodiment, the computing system (1000) can perform warm-up learning to learn a vision inspection model (VIM) using cross entropy loss based on the entire learning data set.

[0145] At this time, in the embodiment, the computing system (1000) can execute the above-described warm-up learning for a preset number of epochs (hereinafter, the warm-up learning epoch threshold) for warm-up learning.

[0146] Figure 6 illustrates an example showing changes in logit values ​​according to learning generations (Epochs) according to one embodiment of the present invention.

[0147] Referring to Figure 6, clear data samples that are generally easy to classify tend to converge early, while data samples that are difficult to classify, such as data containing noise labels, tend to converge late.

[0148] Therefore, even if training is performed with the entire training data without classifying noise labels in the initial learning stage, the influence of noisy label data can be reduced within a small number of generations (Epochs) because learning convergence occurs for relatively accurately labeled easy samples.

[0149] In an embodiment, through such warm-up learning, the computing system (1000) can build an initial model for classifying noise labels.

[0150] FIG. 7 illustrates a block flow diagram for explaining a warm-up learning framework of a first stage learning process according to one embodiment of the present invention.

[0151] Referring to FIG. 7 disclosed in the present invention, data processing, input / output of each network, and loss function processing processes in warm-up learning performed in the first stage learning process can be confirmed.

[0152] In more detail, according to FIG. 7, in the embodiment, the computing system (1000) can extract a predetermined data pair from the entire learning data or the learning data in batch units.

[0153] Additionally, in the embodiment, the computing system (1000) can perform data augmentation to reflect various transformations that may occur in a real environment based on the extracted data pairs.

[0154] In an embodiment, the computing system (1000) may perform strong augmentation that relatively significantly changes the image, such as random cropping, rotation, color jitter, and / or adding noise.

[0155] Additionally, the computing system (1000) as an embodiment can perform weak augmentation, which relatively small changes to the image, such as horizontal flipping, small rotation, and / or translation.

[0156] Additionally, in an embodiment, the computing system (1000) can perform the warm-up learning described above based on the augmented learning data.

[0157] At this time, in the embodiment, the computing system (1000) can perform warm-up learning according to the augmented learning data using the mixup technique.

[0158] Here, again, mixup is a technique that mixes two samples to assign an intermediate adjusted label, which can be used to alleviate the influence of noisy label samples and support more stable learning performance.

[0159] As an example, the computing system (1000) may perform a mixup according to the following [Mathematical Formula 1] based on a predetermined first weakly transformed learning data sample - a second weakly transformed learning data sample and a predetermined first weakly transformed learning data sample - a first strongly transformed learning data sample.

[0160] [Mathematical Formula 1]

[0161]

[0162]

[0163] At this time, 'x' in [Mathematical Formula 1] a ' and 'x b ' represents an image, and 'y a ,' and 'y b ' can represent the label of the corresponding image.

[0164] Additionally, in the embodiment, the computing system (1000) processes a strong deformation augmentation processed image (x as ) and two images that have been mixed up ( xas_b_mix , x aw_b_mix ) as input to the network consisting of the encoder (E), adaptive layer (M) and classifier (C) described above, and the output value (p) as , p as_b_mix , p aw_b_mix ) can be obtained.

[0165] Additionally, in the embodiment, the computing system (1000) obtains the acquired output value (p as , p as_b_mix , p aw_b_mix ) and the label of the corresponding image (y as , y as_b_mix , y aw_b_mix ) can be used to perform network learning (i.e., warm-up learning) by calculating the cross entropy loss according to [Mathematical Formula 2].

[0166] [Equation 2]

[0167]

[0168]

[0169]

[0170] Here, 'E' in [Mathematical Formula 2] represents the encoder (E), 'M' represents the adaptation layer (M), and 'C' represents the classifier (C) network, and 'z i ' is image 'x i ''s feature vector, 'p i ' can mean the predicted confidence value.

[0171] At this time, in the embodiment, the computing system (1000) can perform learning by gradually increasing the learning rate when performing warm-up learning from a minimum value to a preset predefined value.

[0172] Accordingly, the computing system (1000) can implement a stable learning environment that can adapt to a new data set while avoiding rapid changes that may lead to poor learning convergence or overfitting.

[0173] Through the above warm-up learning, the computing system (1000) in the embodiment can obtain an initial vision inspection model (VIM) (i.e., an initial classification model) for classifying noise labels.

[0174] At this time, in the embodiment, the computing system (1000) can continue to perform the above-described warm-up learning if the number of generations (Epochs) of the performed warm-up learning is less than the preset warm-up learning generation (Epoch) threshold (t1).

[0175] On the other hand, in the embodiment, the computing system (1000) can complete the warm-up learning if the number of generations (Epochs) of the performed warm-up learning is greater than or equal to a preset warm-up learning generation (Epoch) threshold (t1).

[0176] FIG. 8 illustrates a block flow diagram for explaining a model learning framework after performing warm-up learning according to one embodiment of the present invention.

[0177] Thus, in the embodiment, the computing system (1000) can obtain an initial vision inspection model (VIM) (i.e., an initial classification model) having a learning framework such as FIG. 8.

[0178] Additionally, in the embodiment, the computing system (1000) can extract a random clean set based on the entire learning data. (S105)

[0179] Here, the random clean set according to the embodiment may mean a clean set (i.e., a data set that does not include noise labels) for learning the vision inspection model (VIM) in the first-stage learning process.

[0180] In detail, in an embodiment, the computing system (1000) can extract the random clean set described above by determining whether the entire training data contains a noise label when the warm-up training is completed.

[0181] FIG. 9 illustrates a conceptual diagram for explaining a method for extracting a random clean set according to one embodiment of the present invention, and FIG. 10 illustrates a flowchart for explaining a method for extracting a random clean set according to one embodiment of the present invention.

[0182] In more detail, referring to FIGS. 9 and 10, in an embodiment, the computing system (1000) can extract a random clean set by predicting noise label data based on the k-Nearest Neighbor (k-NN) algorithm.

[0183] For reference, k-NN (k-Nearest Neighbor) may be an algorithm that detects the k nearest neighbor data around a given data point and predicts the label of the data point based on the information of the detected neighbor data.

[0184] Specifically, in the embodiment, the computing system (1000) detects k neighboring samples having the most similar features extracted based on the entire learning data based on a learned vision inspection model (VIM), votes for a ground truth label defined as true for supervised learning for the detected k neighboring samples, and calculates a score based on the vote to predict whether the entire learning data is a noise label.

[0185] That is, in the embodiment, the computing system (1000) can determine whether the label is a noise label as described above based on the characteristic that samples with similar characteristics are likely to have the same label, and in other cases, the label is likely to be defined as an incorrect label.

[0186] At this time, in the embodiment, the computing system (1000) may initially extract a random clean set based on the k-Nearest Neighbor (k-NN) algorithm using the vision inspection model (VIM) learned in the warm-up learning step, and thereafter, may extract a random clean set based on the k-Nearest Neighbor (k-NN) algorithm at the end of each generation (Epoch) using the vision inspection model (VIM) learned through the S107 step described below.

[0187] Referring further to FIG. 10, as an embodiment, the computing system (1000) may include a step (S201) of calculating cosine similarity based on a specific data sample, a step (S203) of extracting k neighboring samples having high calculated cosine similarity, a step (S205) of measuring a voting score considering data imbalance, and a step (S207, S209) of including the specific data sample in a clean set (i.e., a random clean set) or a noise set based on the measured voting score.

[0188] In detail, in an embodiment, the computing system (1000) can calculate cosine similarity according to [Mathematical Formula 3] below to measure feature similarity between specific data samples.

[0189] [Equation 3]

[0190]

[0191] Here, 'z' in [Equation 3] i ' and 'z j ' can mean the feature vector of the image as the output of the encoder (E) and the adaptation layer (M), as can be confirmed in [Mathematical Formula 2].

[0192] That is, in the embodiment, the computing system (1000) may be configured to compute a specific image sample 'x i 'N' is the set of k samples with the highest cosine similarity. i ' can be used to predict whether data is noisy labeled by measuring the voting score based on the labels defined as true.

[0193] However, in many cases, the number of data samples for each class within the learning data set is not uniform, and if the number of samples for each class is not considered, the sample set 'N' for a class containing a relatively large number of samples i' may increase the probability of being included in the clean set. That is, classes containing relatively many samples may be more likely to be incorrectly predicted as the clean set, and conversely, classes containing relatively few samples may be more likely to be incorrectly predicted as the noise set.

[0194] In order to alleviate the impact of this data imbalance problem, in the embodiment, the computing system (1000) can use a weight that takes into account the number of data samples per class, as in [Mathematical Formula 4] below.

[0195] [Equation 4]

[0196]

[0197] Here, 'c' in [Equation 4] l ' can mean the number of training data samples for the 'l'th class for the 'L' classes to be classified in the training data set.

[0198] At this time, in the embodiment, the computing system (1000) considers the number of data samples per class and sets k neighboring samples 'N i The label voting value in ' can be calculated according to [Mathematical Formula 5].

[0199] [Equation 5]

[0200]

[0201] Here, ' in [Equation 5] ' can mean Hadamard Product.

[0202] Additionally, in the embodiment, the computing system (1000) may be configured to: i The voting score for ' can be calculated according to [Mathematical Formula 6].

[0203] [Equation 6]

[0204]

[0205] Here, each element value of q in [Mathematical Formula 6] can represent the weighted sum for the number of neighboring samples of the 'l'th class, and 'q max ' can mean the maximum value among the elements of 'q'.

[0206] Also, 'q' in [Equation 6] t ' means the value of the 't'th element among the elements of 'q', where 't' is a specific image sample 'x' i ' can be a class (Ground truth label).

[0207] That is, in the embodiment, the computing system (1000) determines a specific image sample 'x' within a set of neighboring samples. i The more samples there are with the same label as 'v i 'increases, and when the number of neighboring samples with the same label as a specific image sample is maximum, 'v i ' can be set to '1'.

[0208] In this way, in the embodiment, the computing system (1000) can measure each voting score for all samples of the learning data set after one epoch of learning is completed.

[0209] At this time, a higher voting score means that the same label was assigned to image samples with similar characteristics, which may indicate a high level of consistency in the labeling task.

[0210] That is, in the embodiment, the computing system (1000) may determine that a sample with a higher voting score is assigned a consistent label and thus is a clean sample (i.e., a sample included in the clean set) rather than a noise label sample.

[0211] Accordingly, in the embodiment, the computing system (1000) sets the measured voting score to a preset threshold (th clean : Below, it can be compared with the voting score threshold.

[0212] At this time, in the embodiment, the computing system (1000) determines whether the measured voting score is greater than a preset voting score threshold (th clean ) exceeds, the training data can be included in the clean set (i.e., a random clean set).

[0213] FIG. 11 illustrates an example for explaining a method for selecting a clean set sample according to one embodiment of the present invention.

[0214] FIG. 11 disclosed in the present invention is a neighboring sample set in the case of 'k = 3' and can confirm an example of performing sample selection from a learning data set that distinguishes two classes.

[0215] For example, referring to Fig. 11, the weights for alleviating the data imbalance problem are ' ' is '[1 / 2, 1 / 5]', and for a specific sample 'X', 'q' is '[0.17, 0.13]', and since a specific sample 'X' belongs to 'class 1', 'q t ' is '0.13', and 'q max ' becomes '0.17'. Therefore, the voting score of a specific sample 'X' becomes '0.8', and the threshold 'th clean If 'is defined as '0.7', a specific sample 'X' can be determined as a clean set.

[0216] At this time, in the embodiment, the computing system (1000) uses the classifier (C) to classify 'p i ' If the predicted result is above a certain threshold, it is judged as a highly reliable result by the classification model (i.e., vision inspection model (VIM)), so the label of the corresponding learning data can be changed to the predicted result of the classifier (C) according to [Mathematical Formula 7] below.

[0217] [Equation 7]

[0218]

[0219] At this time, in the example, a specific sample image 'x iOne-hot label vector 'y' according to [Mathematical Formula 5] i 'is, 'x i 's class label 'l i ' can match.

[0220] Returning again, the computing system (1000) in the embodiment can extract a random clean set based on the entire training data according to the process described above.

[0221] Additionally, in the embodiment, the computing system (1000) can perform random clean set-based noisy label detection learning. (S107)

[0222] In detail, in an embodiment, the computing system (1000) can perform noise label detection learning based on a random clean set in succession to the model learned in step S103 described above.

[0223] That is, the computing system (1000) can perform noise label detection learning by learning a vision inspection model (VIM) on which warm-up learning is performed based on a random clean set.

[0224] At this time, in the embodiment, the computing system (1000) can perform noise label detection learning based on cross entropy loss and contrastive loss by dividing the clean set (i.e., random clean set) selected in the aforementioned step S105 and the entire learning data set.

[0225] In more detail, in an embodiment, the computing system (1000) can detect an original data set corresponding to a random clean set extracted as above.

[0226] That is, the computing system (1000) can detect original data before mixup processing is performed on each learning data included in a random clean set.

[0227] Additionally, in the embodiment, the computing system (1000) can perform data augmentation based on the detected original data set.

[0228] In other words, the computing system (1000) can perform strong augmentation and / or weak augmentation using the original data set.

[0229] For a detailed description of this, the description disclosed in step S103 described above applies.

[0230] Thus, in the embodiment, the computing system (1000) can perform the noise label detection learning described above based on the augmented learning data.

[0231] At this time, in the embodiment, the computing system (1000) can proceed with learning for the encoder (E), the adaptation layer (M), and the classifier (C) in the same manner as the results learned in the aforementioned step S103.

[0232] Additionally, in the embodiment, the computing system (1000) can perform learning on the projection head (PH) through contrastive loss as in [Mathematical Formula 8] below.

[0233] [Equation 8]

[0234]

[0235] Here, 's' in [Mathematical Formula 8] represents the cosine similarity between the output vectors of the projection head (PH), and 'τ' may be a Temperature Scaling parameter for model calibration.

[0236] Accordingly, in the embodiment, the computing system (1000) can perform learning to induce a deformation derived from one sample image to have a feature vector of a similar shape and to induce a deformation to have a feature vector of a different shape from other sample images in the batch.

[0237] Since noise label data detection, such as in the aforementioned S105 step, involves selecting noise labels through the distribution of correct labels within a cluster with similar feature patterns centered on image features, it is important to minimize feature changes for variations of the same image while being well distinguished from other image samples.

[0238] In an embodiment of the present invention, the computing system (1000) can easily achieve the above purpose by performing learning through contrastive loss as in [Mathematical Formula 8].

[0239] At this time, in the embodiment, the computing system (1000) can perform network learning through a loss function such as [Mathematical Formula 9] below.

[0240] [Equation 9]

[0241]

[0242] At this time, ' in [Mathematical Formula 9] 'and ' ' can be a hyper-parameter that determines the weight of each loss function.

[0243] In this way, in the embodiment, the computing system (1000) applies the principle of contrastive learning to the first-stage learning process for noise label detection, thereby implementing learning that takes into account even the feature differences from other samples, unlike the conventional technology that uses the negative cosine similarity loss function to maintain consistency in features and induces only the feature distribution caused by the transformation of one sample to be similar without considering the feature differences from other samples, thereby securing the diversity of the extracted feature patterns and extracting cluster samples with similar features more accurately.

[0244] At this time, in the embodiment, the computing system (1000) may continue to perform the above-described random clean set extraction and noise label detection learning if the number of generations (Epochs) of the performed noise label detection learning is less than the preset noise label detection learning generation (Epoch) threshold (t2).

[0245] On the other hand, in the embodiment, the computing system (1000) can complete clean set extraction and noise label detection learning if the number of generations (Epochs) of the performed noise label detection learning is greater than or equal to a preset noise label detection learning generation (Epoch) threshold (t2).

[0246] Additionally, in the embodiment, the computing system (1000) can extract a decision clean set and a noise set based on the entire learning data. (S109)

[0247] Here, the decision clean set according to the embodiment may mean a clean set (i.e., a data set that does not include noise labels) for learning the vision inspection model (VIM) in the second-stage learning process.

[0248] That is, in the embodiment, the decision clean set may be a clean set that is finally classified and output from the vision inspection model (VIM) learned through the first-stage learning process.

[0249] Additionally, the decision noise set according to the embodiment may mean a noise set (i.e., a data set including noise labels) for learning a vision inspection model (VIM) in the second-stage learning process.

[0250] That is, in the embodiment, the decision noise set may be a noise set that is finally classified and output from the vision inspection model (VIM) learned through the first-stage learning process.

[0251] In detail, in the embodiment, when noise label detection learning is completed, the computing system (1000) can extract the above-described decision clean set and decision noise set by determining whether or not there is a noise label for the entire learning data.

[0252] In other words, the computing system (1000) can extract a decision clean set and a decision noise set according to the entire learning data using a vision inspection model (VIM) for which noise label detection learning has been completed.

[0253] At this time, in the embodiment, the computing system (1000) distinguishes and extracts the clean set and the noise set in the same manner as in the step S105 described above, and a detailed description thereof is applied in accordance with the description disclosed in the step S105.

[0254] Thus, in the embodiment, the computing system (1000) can obtain a decision clean set according to the extracted clean set and a decision noise set according to the extracted noise set.

[0255] In this way, in the embodiment, the computing system (1000) performs warm-up learning using cross entropy loss for a predetermined number of epochs based on the entire learning data set, and after performing the warm-up learning, performs noise label detection learning using cross entropy loss and contrastive loss, thereby inducing similar samples to be concentrated in one cluster and different samples to be spaced apart from each other, thereby evaluating the presence or absence of noise labels for the entire learning data at the end of each epoch, and thereby updating the clean set and / or noise set to be used in learning the next epoch.

[0256] Thus, the computing system (1000) can obtain a clean set and a noise set filtered with high accuracy from the entire learning data set, and based on this, perform a second-stage learning process as follows to build a high-performance classification model (in the embodiment, a vision inspection model (VIM)) that significantly excludes the influence of noise label data.

[0257]

[0258] Step 2: High-performance classification model learning process

[0259] Continuing, in an embodiment of the present invention, the computing system (1000) can perform a learning process (i.e., a second stage process) to enhance the performance of a classification model (i.e., a vision inspection model (VIM)) based on the decision clean set and decision noise set defined through the first stage process.

[0260] In detail, in the embodiment, the computing system (1000) can perform learning according to cross entropy loss, contrastive loss, and supervised contrastive loss based on the decision clean set and decision noise set acquired as described above.

[0261] At this time, in the embodiment, the computing system (1000) may use the decision noise set only for the purpose of learning a feature pattern to utilize the raw information included in the corresponding image sample.

[0262] In an embodiment of the present invention, if the first stage process has the purpose of detecting noise label samples and distinguishing between a clean set and a noise set, the second stage process may have the purpose of achieving a classification model performance optimized for a vision inspection data set.

[0263] That is, through the second stage process, the computing system (1000) may aim to implement a vision inspection model (VIM) that improves the precision of good product data while maintaining the maximum recall rate for defective data, taking into account the characteristics of vision inspection, in which prevention of defect leakage is a very important factor.

[0264] FIG. 12 illustrates a flowchart for explaining a second stage learning process according to one embodiment of the present invention.

[0265] In detail, referring to FIG. 12, a computing system (1000) according to one embodiment of the present invention may include a step (S301) of initializing a classification model, a step (S303) of performing warm-up learning based on a decision clean set, and a step (S305) of performing high-performance classification model learning based on a decision clean set and a decision noise set.

[0266] In more detail, in an embodiment, the computing system (1000) can initialize a classification model. (S301)

[0267] Specifically, in an embodiment, the computing system (1000) can perform classification model initialization using a novel vision inspection model (VIM).

[0268] In other words, the computing system (1000) can proceed with the second stage process using a new vision inspection model (VIM) without using the model learned in the first stage process.

[0269] At this time, in the embodiment, the computing system (1000) can perform the initialization of the classification model (i.e., the vision inspection model (VIM)) described above in the same manner as the step S101 described above.

[0270] Additionally, in the embodiment, the computing system (1000) can perform warm-up training based on a decision clean set. (S303)

[0271] In detail, in an embodiment, the computing system (1000) can perform warm-up learning to learn a vision inspection model (VIM) using cross entropy loss based on a decision clean set defined through the first stage process.

[0272] At this time, in the embodiment of the present invention, the specific method by which the computing system (1000) performs warm-up learning applies the description of step S103 described above, and the differences from step S103 are mainly described below.

[0273] As mentioned above, data samples that are generally clear and easily classified tend to converge early on, so in the first stage process, the influence of noisy label data is mitigated by learning for a small number of pre-set generations (Epochs). However, since warm-up learning is performed with the entire training data set that includes noisy label samples, the influence of noisy labels may still be partially included.

[0274] Therefore, in the second stage process according to the embodiment of the present invention, warm-up learning is performed using the clean set (i.e., decision clean set) selected in the first stage process to block the influence of noise label samples.

[0275] FIG. 13 illustrates a block flow diagram for explaining a warm-up learning framework of a second-stage learning process according to one embodiment of the present invention.

[0276] Referring to FIG. 13 disclosed in the present invention, data processing, input / output of each network, and loss function processing processes in warm-up learning performed in the second stage learning process can be confirmed.

[0277] According to FIG. 13, in an embodiment, the computing system (1000) can perform second-stage warm-up learning for a vision inspection model (VIM) using strong augmentation, weak augmentation, and mixup techniques in the same manner as the warm-up learning process of the first-stage learning process.

[0278] Here, again, mixup can significantly alleviate the data imbalance problem when used together with oversampling for classes containing a small number of samples, because it supports effective network calibration during the learning phase by mixing two samples and assigning an intermediate adjusted label.

[0279] At this time, in the embodiment, the computing system (1000) can perform decision clean set-based warm-up learning using the cross entropy loss as in the aforementioned [Mathematical Formula 2].

[0280] Additionally, in the embodiment, the computing system (1000) can continue to perform the above-described warm-up learning if the number of generations (Epochs) of the performed warm-up learning is less than a preset warm-up learning generation (Epoch) threshold (t3).

[0281] On the other hand, in the embodiment, the computing system (1000) can complete the warm-up learning if the number of generations (Epochs) of the performed warm-up learning is greater than or equal to a preset warm-up learning generation (Epoch) threshold (t3).

[0282] In this way, in the embodiment, the computing system (1000) can prevent the influence of noise labels that may be included in the entire learning data set by performing warm-up learning in the second stage process using the clean set (i.e., decision clean set) defined through the first stage process.

[0283] In addition, in the embodiment, the computing system (1000) can perform high-performance classification model training based on a decision clean set and a noise set. (S305)

[0284] In detail, in the embodiment, the computing system (1000) can perform high-performance classification model learning based on the decision clean set and decision noise set in succession to the model learned in the aforementioned step S303.

[0285] That is, the computing system (1000) can perform high-performance classification model learning by learning a vision inspection model (VIM) for which warm-up learning is performed based on a decision clean set and a decision noise set.

[0286] At this time, in the embodiment, the computing system (1000) can divide (distinguish) the decision clean set and the decision noise set as in FIG. 8 and perform learning according to cross entropy loss, contrastive loss, and supervised contrastive loss.

[0287] In more detail, in the embodiment, the computing system (1000) can utilize feature information included in noise label data for learning a high-performance classification model by using contrastive loss such as [Mathematical Formula 8] described above based on a decision noise set.

[0288] That is, in the embodiment, the computing system (1000) performs feature information learning using contrastive loss on a noise label sample that is determined to have incorrect label information assigned but for which raw information included in the corresponding image sample is desired to be utilized, thereby excluding the noise label information from learning and utilizing only the feature information for learning.

[0289] Meanwhile, in the embodiment, the computing system (1000) can perform high-performance classification model learning by using the cross entropy loss according to the aforementioned [Mathematical Formula 2] and the supervised contrastive loss according to the following [Mathematical Formula 10] in parallel based on the decision clean set.

[0290] [Equation 10]

[0291]

[0292] Here, ' in [Equation 11] ' can be a hyper-parameter that determines the weight of the loss function.

[0293] Also, in [Equation 10], ' ' represents a set of samples (Positives) of the same class among the samples included in the batch, and ' ' can represent the entire batch sample set.

[0294] Additionally, 'v' in [Equation 10] i ' is the output vector of the projection head (PH), and 'τ' may be a Temperature Scaling parameter for model calibration.

[0295] In this way, in the embodiment, the computing system (1000) can further perform supervised contrastive loss learning based on a decision clean set, thereby making feature vectors between the same classes close together and feature vectors between different classes distant from each other in the feature space, thereby improving the distinguishing power between each class.

[0296] Thus, the computing system (1000) can provide a classification model optimized for tasks such as defect detection that require high precision by constructing a vision inspection model (VIM) that not only improves classification accuracy but also operates robustly to various changes in input.

[0297] At this time, in the embodiment, the computing system (1000) can perform network learning through a loss function such as [Mathematical Formula 10].

[0298] At this time, in the embodiment, the computing system (1000) can continue to perform high-performance classification model learning as described above if the number of generations (Epochs) of the performed high-performance classification model learning is less than the preset high-performance classification model learning generation (Epoch) threshold (t4).

[0299] On the other hand, in the embodiment, the computing system (1000) can complete high-performance classification model learning if the number of generations (Epochs) of the performed high-performance classification model learning is greater than or equal to a preset high-performance classification model learning generation (Epoch) threshold (t4).

[0300] Thus, the computing system (1000) according to an embodiment of the present invention can provide a classification model (i.e., a vision inspection model (VIM)) efficiently learned based on a data set including noise labels.

[0301] In an embodiment, the computing system (1000) may provide a vision inspection model (VIM) learned according to an embodiment based on a predetermined application service (e.g., an outlier detection service, etc.).

[0302] As described above, in the embodiment of the present invention, the computing system (1000) can detect noise label data while stably learning a model through the use of a warm-up learning strategy, a mixup technique, and an adaptive layer (M) in the first stage process, and can improve the classification performance of the model through supervised contrastive loss and cross entropy loss using the data refined as described above in the second stage process.

[0303] That is, in the embodiment, the computing system (1000) can prevent model performance degradation due to incorrectly labeled learning data by using different customized learning policies at each stage through a two-stage approach that separates the noise label data detection stage and the high-performance classification model learning stage.

[0304] In particular, in the embodiment, the computing system (1000) performs learning based on contrastive loss in parallel, which, unlike existing technologies, does not use incorrect label information of a noise label data set while using feature information included in the data, thereby building and providing a vision inspection model (VIM) that maintains the maximum recall for defective data while improving precision for good data.

[0305] Meanwhile, the embodiments of the present invention described above may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. Hardware devices may be changed into one or more software modules to perform processing according to the present invention, and vice versa.

[0306] The specific implementations described in the present invention are exemplary embodiments and do not limit the scope of the present invention in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the lines connecting or connecting members between components illustrated in the drawings are merely representative of functional connections and / or physical or circuit connections, and may be replaced or represented as various additional functional connections, physical connections, or circuit connections in an actual device. In addition, unless specifically mentioned as “essential,” “important,” etc., a component may not be absolutely necessary for the application of the present invention.

[0307] Although the detailed description of the present invention has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art or having ordinary knowledge in the art that various modifications and changes can be made to the present invention without departing from the spirit and technical scope of the present invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

[0308]

[0309] The present invention relates to a vision inspection model learning method and system thereof, and is applicable to the artificial intelligence industry, and thus has industrial applicability.

Claims

1. A method for training a vision inspection model using a computing system including memory and a processor, A step of performing a first process (Process 1) for detecting noisy label data in a training data set; A step of obtaining a noisy data set and a clean data set based on the detected noise label data based on the first process; and A step of performing a second process (Process 2) for training a vision inspection model based on the acquired noise data set and clean data set; including; How to train a vision inspection model.

2. In paragraph 1, The above vision inspection model is, Contains at least one encoder, an adaptation layer, a projection head, and a classifier. How to train a vision inspection model.

3. In paragraph 2, The step of performing the above first process is: A step of initializing the above vision inspection model, A step of performing a first warm-up training (Warm-up Training 1) to train the initialized vision inspection model based on the above learning data set for a predetermined number of generations (Epochs), A step of detecting noise label data within the training data set based on the first warm-up learned vision inspection model; A step of obtaining a random clean data set according to the above-detected noise label data, A step of performing noise label detection learning to train the first warm-up learned vision inspection model for a predetermined number of generations based on the acquired random clean data set and the learning data set, A step of detecting noise label data within the learning data set based on the vision inspection model learned for detecting the noise label, A step of obtaining a noise data set and a clean data set according to the detected noise label data. How to train a vision inspection model.

4. In paragraph 3, The step of performing the above first warm-up learning is: Includes a step of performing learning based on Cross Entropy Loss. How to train a vision inspection model.

5. In paragraph 4, The first warm-up learning step is: A step of extracting a predetermined data pair from the above learning data set, A step of performing data augmentation based on the above extracted data pairs, It further includes a step of obtaining weak augmentation data, which is data in which a relatively small deformation has occurred, and strong augmentation data, which is data in which a relatively large deformation has occurred, based on the data augmentation performed above. How to train a vision inspection model.

6. In paragraph 5, The first warm-up learning step is: The first mixup data is data that is a mixup of the first weakly modified augmented data and the second weakly modified augmented data, It further includes a step of obtaining second mix-up data which is data obtained by mixing up the first weak deformation augmentation data and the first strong deformation augmentation data. How to train a vision inspection model.

7. In paragraph 6, The first warm-up learning step is: Further comprising a step of performing learning according to the cross entropy loss based on the first mix-up data, the second mix-up data, and the first strong deformation augmentation data. How to train a vision inspection model.

8. In paragraph 3, The step of detecting the above noise label data is: A step of calculating cosine similarity between other learning data based on the first learning data in the above learning data set, A step of extracting a predetermined number (k) of neighboring learning data, which are other learning data with high cosine similarity, as calculated above, and A step of calculating a voting score for each extracted neighbor learning data, A step of detecting the noise label data based on the above-described calculated voting score. How to train a vision inspection model.

9. In paragraph 3, The step of performing the above noise label detection learning is: A step of training the projection head based on the first clean data set and contrastive loss, A step of training the encoder, the adaptive layer, and the classifier based on the training data set and the cross entropy loss. How to train a vision inspection model.

10. In paragraph 3, The step of performing the above noise label detection learning is: A step of detecting an original data set corresponding to the first clean data set, A step of performing data augmentation based on the above-detected original data set, A step of performing noise label detection learning based on the augmented data is included. How to train a vision inspection model.

11. In paragraph 3, The step of performing the above second process is: A step of newly initializing the above vision inspection model, A step of performing a second warm-up training (Warm-up Training 2) for training the newly initialized vision inspection model for a predetermined number of generations based on the above clean data set, A step of performing high-performance classification model learning by learning the second warm-up learned vision inspection model based on the above clean data set and the above noise data set for a predetermined number of generations. How to train a vision inspection model.

12. In paragraph 11, The steps of performing the above high-performance classification model learning are: A step of performing learning by using cross entropy loss and supervised contrastive loss in parallel based on the above clean data set, A step of performing learning according to contrast loss based on the above noise data set is included. How to train a vision inspection model.

13. In paragraph 1, It further includes a step of providing a vision inspection model learned based on the above second process through a predetermined application service. How to train a vision inspection model.

14. At least one memory; and At least one processor for training a vision inspection model by reading out at least one application stored in the memory; The instructions of the above processor are: A step of performing a first process (Process 1) for detecting noisy label data in a training data set, A step of obtaining a noisy data set and a clean data set based on the detected noise label data based on the first process, Includes a command for performing a step of performing a second process (Process 2) for training a vision inspection model based on the acquired noise data set and clean data set. Vision inspection model learning system.

15. At least one encoder; At least one Adaptation Layer; At least one projection head; At least one classifier; and At least one processor controlling the encoder, the adaptation layer, the projection head, and the classifier; The above processor, The first process (Process 1) is performed to detect noisy label data in the training data set, Obtain a noisy data set and a clean data set based on the detected noise label data based on the first process above, A second process (Process 2) is performed to train a vision inspection model based on the acquired noise data set and clean data set. Computing device.

Images