Visual inspection model learning method and system

Abstract

A method for learning a visual inspection model according to an embodiment of the present invention is a method for learning a visual inspection model by a computing system including a memory and a processor, the method comprising: a step of performing a first process in which noise label data (Noise Label Data) within a learning data set (Training Data Set) is detected, and a step of performing a second process in which noise label data (Noise Label Data) within the learning data set (Training Data Set) is detected; a step for acquiring a noise data set (Noisy Data Set) and a clean data set (Clean Data Set) from noise tag data detected on the basis of the first process; and a step of performing a second process (Process 2) in which a visual inspection model is learned on the basis of the acquired noise data set and clean data set, and a step of performing a second process in which the learning of the visual inspection model is performed on the basis of the acquired noise data set and clean data set.

Description

Technical Field

[0001] This invention relates to a visual detection model learning method and system. More specifically, it discloses a two-stage visual detection model learning method and system that separately learns a first-stage process and a second-stage process according to a learning strategy optimized for each objective. In the first-stage process, noisy label data is detected in order to efficiently learn a dataset including noisy labels. In the second-stage process, a classification model is learned. Background Technology

[0002] In visual inspection, machine learning methods such as classification or detection are used to automate the identification of potential defects in the appearance of products. Ultimately, defects are classified as either OK or NG to prevent defective products from reaching the outside world.

[0003] This machine learning method requires a dataset built through a labeling process. During the labeling process, the collected image data is manually reviewed to determine if there are any defects, and the data is manually labeled with true values.

[0004] To ensure the diversity of image patterns containing various environmental factors, a large amount of training data is required. However, since manual annotation of massive amounts of data is necessary, errors can easily occur due to various factors such as decreased staff attention, differences in defect judgment proficiency, identification standards, and data ambiguity. As a result, the possibility of incorrectly labeled data in real-world training datasets is very high.

[0005] This problem of noisy labels can occur even in public datasets widely used in computer vision fields such as MNIST, CIFAR-10, and / or ImageNet.

[0006] Although numerous studies have proposed robust learning solutions even in datasets with noisy labels through techniques such as sample selection and / or loss regularization, most of these studies aim to improve classification accuracy metrics on general public datasets such as CIFAR-100.

[0007] Therefore, in existing research, even if the accuracy for defects decreases, higher evaluation results can still be obtained as long as the accuracy for classifying qualified products is high.

[0008] However, in the field of visual inspection, it is necessary to prevent defective products from reaching the outside world to the greatest extent possible. Therefore, unlike existing research, the goal should be to improve the precision of qualified product data while maintaining the maximum recall rate of defective data.

[0009] In other words, in the field of visual inspection, learning models should be developed to achieve high classification accuracy, but the technology development for this purpose is still incomplete.

[0010] On the other hand, the proposed technologies all adopt the approach of learning a classification model while detecting noisy label data.

[0011] Noise label detection is advantageous when selecting noise labels from clusters of ground-truth labels within a feature pattern space that is widely distributed around image features. However, the learning strategy used for detecting such noise label data can degrade the learning performance of classification models, which learn feature patterns centered on ground-truth labels in order to better distinguish between classes.

[0012] That is, the existing approach, which solves the problem by learning a classification model while detecting noisy label data, as disclosed in prior literature 1 ("SSR: An Efficient and Robust Framework for Learning with Unknown Label Noise"), suffers from a degradation in classification performance due to the conflicting learning strategies between feature learning for detecting noisy label data and classification learning focused on class distinction. In addition, feature learning based on negative cosine similarity, which only considers the feature changes of its own sample without considering other samples, suffers from a degradation in performance for visual detection data with simple pattern patterns.

[0013] On the other hand, existing methods, such as prior literature 2 (“Identifying Mislabeled Data using the Area Under the Margin Ranking”), disclose the following: by utilizing the feature that the learning convergence is achieved on accurately labeled and easily accessible samples in the early learning steps and on noisy labeled samples in the later learning steps, the noisy labeled samples are detected based on the loss difference between epochs.

[0014] However, this existing approach has the problem of difficulty in determining which point in the learning process should be used to judge whether a sample is a noisy label sample, and the period standard fluctuates greatly depending on the data.

[0015] In addition, contrastive learning and its derivative supervised learning methods have brought significant developments to the field of image classification.

[0016] Contrastive learning refers to a learning method that shortens the distance between similar image samples and lengthens the distance between dissimilar image samples in the feature space.

[0017] In particular, in contrastive learning based on supervised learning, class information is used to learn so that the feature vectors of the same class are close to each other in the feature space, and the feature vectors of different classes are far apart, thus improving the discriminability between classes.

[0018] This contrastive learning not only improves the model's classification accuracy but also guides the model to robustly learn various variations of the input, making it an advantageous learning method for tasks requiring high accuracy, such as industrial defect detection.

[0019] However, the noisy labeled data mentioned above not only significantly reduces the effectiveness of contrastive learning as described above, but also becomes an element that hinders learning when performing classification learning, thus reducing the overall performance of the model.

[0020] Therefore, it is necessary to develop and introduce new technologies that can solve the problems mentioned above. Summary of the Invention

[0021] Technical problems to be solved

[0022] An embodiment of the present invention aims to provide a two-stage visual detection model learning method and system, wherein a first-stage process and a second-stage process are learned separately according to a learning strategy optimized for each objective. In the first-stage process, noisy label data is detected in order to efficiently learn a dataset including noisy labels. In the second-stage process, a classification model is learned.

[0023] It should be noted that the technical problems to be solved by the present invention and its embodiments may include other technical problems, and are not limited to the technical problems described above.

[0024] means of solving technical problems

[0025] According to an embodiment of the present invention, a visual inspection model learning method is used by a computing system including a memory and a processor to learn a visual inspection model. The visual inspection model learning method includes: a first process (Process 1), wherein, in the first process, noise label data is detected in a training dataset; a noise dataset and a clean dataset are obtained from the noise label data detected based on the first process; and a second process (Process 2), wherein, in the second process, a visual inspection model is learned based on the obtained noise dataset and clean dataset.

[0026] On the other hand, the aforementioned visual detection model includes at least one encoder, an adaptation layer, a projection head, and a classifier.

[0027] On the other hand, the steps of performing the first process described above include: initializing the visual detection model; performing a first warm-up training (Warm-up Training 1), wherein, in the first warm-up training, the initialized visual detection model is trained for a predetermined number of epochs based on the learning dataset; detecting noisy label data within the learning dataset based on the visual detection model trained in the first warm-up training; obtaining an arbitrary clean dataset based on the detected noisy label data; learning noise label detection for a predetermined number of epochs based on the obtained arbitrary clean dataset and the learning dataset; detecting noisy label data within the learning dataset based on the visual detection model trained in the noise label detection; and obtaining a noise dataset and a clean dataset obtained from the detected noise label data.

[0028] On the other hand, the steps for the first warm-up training mentioned above include: learning based on cross-entropy loss.

[0029] On the other hand, the first warm-up training step also includes: extracting predetermined data pairs from the learning dataset; performing data augmentation based on the extracted data pairs; and obtaining weak augmentation data and strong augmentation data based on the data augmentation, wherein the weak augmentation data is data with relatively small deformation and the strong augmentation data is data with relatively large deformation.

[0030] On the other hand, the first warm-up training step also includes the step of obtaining a first mixed data and a second mixed data, wherein the first mixed data mixes a first weakly augmented data and a second weakly augmented data, and the second mixed data mixes the first weakly augmented data and the first strongly augmented data.

[0031] On the other hand, the first warm-up training step also includes: a learning step based on the first mixed data, the second mixed data, and the first strongly enhanced data, based on the cross-entropy loss.

[0032] On the other hand, the steps for detecting the aforementioned noisy label data include: calculating the cosine similarity between the first learning data in the aforementioned learning dataset and other learning data; extracting a predetermined number of k neighboring learning data, wherein the neighboring learning data are other learning data with high calculated cosine similarity; calculating the voting score according to the extracted neighboring learning data; and detecting the aforementioned noisy label data based on the calculated voting score.

[0033] On the other hand, the steps for learning the above-mentioned noise label detection include: learning the above-mentioned projection head based on the above-mentioned first clean dataset and contrastive loss; and learning the above-mentioned encoder, the above-mentioned adaptation layer and the above-mentioned classifier based on the above-mentioned learning dataset and the above-mentioned cross-entropy loss.

[0034] On the other hand, the steps for performing the above-mentioned noise label detection learning include: a step of detecting the original dataset corresponding to the first clean dataset; a step of performing data augmentation based on the detected original dataset; and a step of performing the above-mentioned noise label detection learning based on the augmented data.

[0035] On the other hand, the steps of performing the second process described above include: a step of re-initializing the visual detection model; a step of performing a second warm-up training 2, in which the newly initialized visual detection model learns a predetermined number of periods based on the clean dataset; and a step of training a high-performance classification model, in which the visual detection model after the second warm-up training learns a predetermined number of periods based on the clean dataset and the noisy dataset.

[0036] On the other hand, the steps for training the aforementioned high-performance classification model include: learning using both cross-entropy loss and supervised contrastive loss based on the aforementioned clean dataset; and learning based on contrastive loss based on the aforementioned noisy dataset.

[0037] On the other hand, the visual detection model learning method according to an embodiment of the present invention further includes the step of providing a visual detection model learned based on the second process described above through a predetermined application service.

[0038] On the other hand, a visual inspection model learning system according to an embodiment of the present invention includes: at least one memory; and at least one processor, which reads at least one application stored in the memory and performs visual inspection model learning, wherein the instructions of the processor include instructions to perform the following steps: performing a first process (Process 1), wherein in the first process, noise label data is detected in a training dataset; obtaining a noise dataset (Noisy Data Set) and a clean dataset (Clean Data Set) obtained from the noise label data detected based on the first process; and performing a second process (Process 2), wherein in the second process, a visual inspection model is learned based on the obtained noise dataset and clean dataset.

[0039] On the other hand, a computing device according to an embodiment of the present invention includes: at least one encoder; at least one adaptation layer; at least one projection head; at least one classifier; and at least one processor, which controls the encoder, the adaptation layer, the projection head, and the classifier, wherein the processor performs a first process (Process 1) to detect noisy label data in a training dataset, obtains a noisy dataset and a clean dataset obtained from the noisy label data detected based on the first process, and performs a second process (Process 2) to learn a vision inspection model based on the obtained noisy dataset and clean dataset.

[0040] Invention Effects

[0041] According to an embodiment of the present invention, a visual inspection model learning method and system are provided, which separately learn a first step of learning noisy label data and a second step of learning a classification model according to a learning strategy optimized for each objective. This results in a high-performance visual inspection model that can prevent the degradation of model performance caused by learning data with incorrect labels, while maintaining the maximum recall rate for defective data and improving the precision rate for qualified product data.

[0042] At this point, according to an embodiment of the visual detection model learning method and system of the present invention, in the first step, the model is stably learned while detecting noisy label data through warm-up training strategies, hybrid techniques and the use of adaptation layers, etc. In the second step, the model can be improved by learning contrastive loss and cross-entropy loss obtained by supervised learning using the data refined as described above.

[0043] That is, according to an embodiment of the visual detection model learning method and system of the present invention, as described above, by implementing a learning strategy optimized for each step, it has the effect of providing a high-performance visual detection model, wherein the high-performance visual detection model is learned while minimizing the influence of noisy label data and maximizing the advantages of contrastive learning.

[0044] It should be noted that, as can be clearly understood from the following description, other effects that can be obtained through the present invention are not limited to those described above. Attached Figure Description

[0045] Figure 1 A block diagram example of a computational system for implementing a 2-stage visual inspection model learning service according to an embodiment of the present invention is shown.

[0046] Figure 2 A block diagram example of a computing device for implementing a 2-stage visual inspection model learning service according to an embodiment of the present invention is shown.

[0047] Figure 3 A block diagram example of another aspect of a computing device for implementing a 2-Stage visual inspection model learning service according to an embodiment of the present invention is shown.

[0048] Figure 4 A flowchart illustrating the first step of the learning process according to an embodiment of the present invention is shown.

[0049] Figure 6 An example of the variation of logistic regression (Logit) values ​​during the training period is shown according to an embodiment of the present invention.

[0050] Figure 7 A flowchart illustrating a warm-up training framework for explaining the first step of the learning process according to an embodiment of the present invention is shown.

[0051] Figure 8 A flowchart illustrating a model learning framework after preheating training according to an embodiment of the present invention is shown.

[0052] Figure 9 A conceptual diagram illustrating a method for extracting a candidate clean set according to an embodiment of the present invention is shown.

[0053] Figure 10 A flowchart illustrating a method for extracting a candidate clean set according to an embodiment of the present invention is shown.

[0054] Figure 11 An example is shown illustrating a method for selecting a clean set of samples according to an embodiment of the present invention.

[0055] Figure 12 A flowchart illustrating the second step of the learning process according to an embodiment of the present invention is shown.

[0056] Figure 13 A flowchart illustrating a warm-up training framework for a second-step learning process according to an embodiment of the present invention is shown. Detailed Implementation

[0057] This invention can have many variations and embodiments. Specific embodiments are shown in the accompanying drawings, and the details are described below. The effects and features of the invention, as well as methods for achieving them, can be clearly understood by referring to the accompanying drawings and the various embodiments described below. However, the invention can be implemented in many ways and is not limited to the various embodiments disclosed below. In the embodiments below, terms such as "first," "second," etc., are used to distinguish one constituent element from other constituent elements and do not have a limiting meaning. Furthermore, the singular expression includes the plural unless otherwise indicated in the context. The terms "including" or "having" indicate the presence of a feature or constituent element described in the specification, but do not preclude the possibility of adding more than one other feature or constituent element. In the accompanying drawings, the dimensions of the constituent elements may be enlarged or reduced for ease of explanation. For example, the dimensions and thicknesses of the various components shown in the drawings are arbitrarily shown for ease of explanation, so the invention is not necessarily limited to what is shown in the drawings.

[0058] Hereinafter, several embodiments of the present invention will be described in detail with reference to the accompanying drawings. When describing the invention with reference to the accompanying drawings, the same or corresponding constituent elements will be labeled with the same reference numerals, and repeated descriptions of them will be omitted.

[0059] [Exemplary System for Providing a 2-Stage Visual Inspection Model Learning Service]

[0060] The following describes in detail, with reference to the accompanying drawings, an exemplary system for implementing a 2-stage visual detection model learning service, wherein, in the above-mentioned 2-stage visual detection model learning, a first step process for efficiently learning a dataset including noisy labels and a second step process for learning a classification model are separated according to a learning strategy optimized for each objective.

[0061] Figure 1 A block diagram example of a computational system for implementing a 2-stage visual inspection model learning service according to an embodiment of the present invention is shown.

[0062] Reference Figure 1 The computing system 1000 that implements the 2-stage visual detection model learning service of the present invention includes a user computing device 110, a server computing system 130 and a training computing system 150, which can communicate through a network 170.

[0063] According to an embodiment of the visual detection model learning method of the present invention, 1) it can be implemented and provided locally by a user computing device 110, 2) it can also be implemented and provided by a server computing system 130 that communicates with the user computing device 110 in the form of a web page service, and 3) it can also be implemented and provided by the user computing device 110 and the server computing system 130 in a coordinated manner.

[0064] In this embodiment, the user computing device 110 and / or the server computing system 130 can learn machine learning models 120 and / or 140 through interaction with the training computing system 150, which is communicatively connected via network 170. The training computing system 150 may be configured separately from the server computing system 130 or may be part of the server computing system 130.

[0065] Furthermore, for the artificial intelligence model, 1) it can be learned directly by the user computing device 110 locally; 2) it can be learned by the server computing system 130 and the user computing device 110 interacting with each other via the network 170; and 3) it can be learned by other training computing systems 150 using various training and learning techniques. Alternatively, it can be implemented by transmitting the artificial intelligence model learned by the training computing system 150 to the user computing device 110 and / or the server computing system 130 via the network 170 for provision / update.

[0066] In some embodiments, the training computing system 150 may be part of the server computing system 130 or part of the user computing device 110.

[0067] User computing device 110 may include smartphones, mobile phones, digital broadcasting devices, PDAs (personal digital assistants), PMPs (portable multimedia players), desktop computers, wearable devices, embedded computing devices, and / or tablet PCs, as well as all other types of computing devices.

[0068] This user computing device 110 includes at least one processor 111 and a memory 112. The processor 111 may consist of at least one or more processors electrically connected to other processors, including a central processing unit (CPU), a graphics processing unit (GPU), ASICs (application-specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or electrical units for performing other functions.

[0069] The memory 112 may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, and disks, as well as combinations thereof. It may include web storage devices of servers that perform storage functions on the Internet. This memory 112 may store data 113 and instructions 114 required by at least one of the processors 111 to perform functional actions, such as learning an artificial intelligence model or performing visual inspection through an artificial intelligence model.

[0070] In one embodiment, the user computing device 110 may store at least one or more machine learning models 120.

[0071] In detail, the machine learning model 120 can be a variety of machine learning models, such as multiple neural networks (e.g., deep neural network) or other types of machine learning models, including nonlinear models and / or linear models, and can be composed of combinations thereof.

[0072] In this case, the neural network may include at least one of multiple feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, and / or other types of neural networks.

[0073] In one embodiment, the user computing device 110 can receive at least one machine learning model 120 from the server computing system 130 via the network 170 and store it in the memory 112. The stored machine learning model 120 is then executed by the processor 111, thereby enabling visual detection and the like.

[0074] In other embodiments, the server computing system 130 includes at least one machine learning model 140 and performs actions through the machine learning model 140, and is linked with the user computing device 110 by interacting with the user computing device 110 on the relevant data, thereby enabling it to provide the user with a 2-stage visual detection model learning service.

[0075] For example, user computing device 110 can perform a two-stage visual inspection model learning service, whereby server computing system 130 uses a webpage to output the results of machine learning model 140 in response to user input.

[0076] Furthermore, artificial intelligence models can also be implemented by executing at least a portion of machine learning models 120 and / or 140 on the user computing device 110, with the remainder executed on the server computing system 130.

[0077] Furthermore, the user computing device 110 may include at least one input component 121 for sensing user input. For example, the user input component 121 may include a touch sensor (e.g., a touchscreen and / or touchpad, etc.) that senses the touch of a user's input medium (e.g., a finger or stylus), an image sensor that senses the user's gesture input, a microphone that senses the user's voice input, buttons, a mouse, and / or a keyboard, etc. Moreover, when receiving input from an external controller (e.g., a mouse and / or keyboard, etc.) through an interface, the user input component 121 may include both the interface and the external controller.

[0078] The server computing system 130 includes at least one processor 131 and a memory 132. The processor 131 may consist of at least one or more processors electrically connected to other processors, including a central processing unit (CPU), a graphics processing unit (GPU), ASICs (application-specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or electrical units for performing other functions.

[0079] Additionally, the memory 132 may include one or more non-transitory / transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, and disks, as well as combinations thereof. This memory 132 may store data 133 and instructions 134 required by the processor 131 to perform functional actions, including learning an artificial intelligence model or performing visual inspection using an artificial intelligence model.

[0080] In one embodiment, the server computing system 130 may include at least one or more computing devices. For example, the server computing system 130 may be configured to allow multiple computing devices to operate according to a sequential computing architecture, a parallel computing architecture, or a combination thereof. Furthermore, the server computing system 130 may include multiple computing devices connected via a network 170.

[0081] Furthermore, the server computing system 130 can store at least one machine learning model 140. For example, the server computing system 130, as a machine learning model 140, may include neural networks and / or other multi-layer nonlinear models. Exemplary neural networks may include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0082] The training computing system 150 includes at least one processor 151 and a memory 152. The processor 151 may consist of at least one or more processors electrically connected to each other, including a central processing unit (CPU), a graphics processing unit (GPU), ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or electrical units for performing other functions.

[0083] Additionally, the memory 152 may include one or more non-transitory / transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, and disks, as well as combinations thereof. This memory 152 may store data 153 and instructions 154 required by the processor 151 for tasks such as learning artificial intelligence models.

[0084] For example, the training computation system 150 may include a model trainer 160. This model trainer 160 uses various training or learning techniques (based on...) Figure 3 (The framework shown) trains the machine learning model stored in the user computing device 110 and / or the server computing system 130.

[0085] For example, this model trainer 160 can update one or more parameters of the machine learning model 120 and / or 140 through backpropagation based on a defined loss function.

[0086] In some implementation examples, backpropagation of errors may include truncated backpropagation through time. To improve the generalization ability of the trained machine learning models 120 and / or 140, the model trainer 160 may perform several generalization techniques (e.g., weight decay, dropout, and / or knowledge distillation).

[0087] In particular, the model trainer 160 can train machine learning models 120 and / or 140 based on a series of training data 161. The training data 161 may include data of different styles, such as images, audio samples, and / or text. Examples of image types that can be used include video frames, LiDAR point clouds, X-ray images, computed tomography scans, hyperspectral images, and / or images of various other types.

[0088] This training data 161 can be provided by the user computing device 110 and / or the server computing system 130. When the machine learning model 120 and / or 140 is trained by the training computing device on the specific data of the user computing device 110, the machine learning model 120 and / or 140 can be designated as a personalized model.

[0089] In addition, the model trainer 160 includes computer logic for providing the desired functionality.

[0090] Furthermore, the model trainer 160 can be implemented via hardware, firmware, and / or software that controls a general-purpose processor. In one implementation, the model trainer 160 includes a program file stored on a storage device and is loaded into memory 152 and processed by one or more processors 151. In other implementations, the model trainer 160 includes one or more sets of computer-readable data 152 and instructions 154 stored in a tangible computer-readable storage medium such as RAM, hard disk, optical media, or magnetic media.

[0091] Network 170 includes, but is not limited to, 3GPP (3rd Generation Partnership Project) networks, LTE (Long Term Evolution) networks, WIMAX (World Interoperability for Microwave Access) networks, the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth networks, satellite broadcasting networks, analog broadcasting networks, and / or DMB (Digital Multimedia Broadcasting) networks.

[0092] Typically, communication over network 170 can be conducted using any type of wired and / or wireless connection, through various communication protocols (e.g., TCP / IP, HTTP, SMTP, and / or FTP), encodings or formats (e.g., HTML and / or XML), and / or security schemes (e.g., VPN, Secure HTTP, and / or SSL).

[0093] Figure 2 A block diagram example of a computing device for implementing a 2-stage visual inspection model learning service according to an embodiment of the present invention is shown.

[0094] Reference Figure 2 The computing device 100, included in the user computing device 110, the server computing system 130, and the training computing system 150, comprises multiple applications (e.g., application 1 to application N). Each application may include a machine learning library and more than one machine learning model. For example, applications may include image processing (e.g., detection, classification, and / or segmentation, etc.) applications, SMS applications, email applications, dictation applications, virtual keyboard applications, browser applications, and / or chatbot applications, etc.

[0095] In an embodiment, the computing device 100 may include a model trainer 160 for learning an artificial intelligence model, and may store and run the learned artificial intelligence model to provide output data based on predetermined input data (in an embodiment, predetermined images, etc.).

[0096] Applications of computing device 100 can communicate with multiple different components of the computing device, such as at least one sensor, context manager, device state component, and / or additional components. In one embodiment, applications can communicate with device components using APIs (e.g., public APIs). In another embodiment, the APIs used by applications can be application-specific APIs.

[0097] Figure 3 A block diagram example of another aspect of a computing device 100 implementing a 2-Stage visual inspection model learning service according to an embodiment of the present invention is shown.

[0098] Reference Figure 3The computing device 300 includes multiple applications (e.g., application 1 to application N). Each application can communicate with a central intelligence layer. For example, applications may include image processing applications, SMS applications, email applications, dictation applications, virtual keyboard applications, and / or browser applications, etc. In one embodiment, each application can communicate with the central intelligence layer (and the models stored therein) using an API (a common API for all applications).

[0099] The central intelligence layer can include multiple machine learning models. For example, such as Figure 3 As shown, at least a portion of each machine learning model can be provided to various applications and managed by a central intelligent layer. In other implementations, two or more applications can share a single machine learning model. For example, in a partial implementation, the central intelligent layer can provide a single model to all applications. In a partial implementation, the central intelligent layer can be contained within or differently configured within the operating system of the computing device 300.

[0100] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized data storage location for computing device 300. For example... Figure 3 As shown, the central device data layer can communicate with multiple different components of the computing device 300, such as more than one sensor, a context manager, a device state component, and / or additional components. In some implementation examples, the central device data layer can communicate with the individual device components using APIs (e.g., private APIs).

[0101] The techniques described in this specification can be applied not only to servers, databases, multiple software applications, and other computer-based systems, but also to multiple actions taken and information transmitted to or received from such systems. It should be understood that the inherent flexibility of computer-based systems allows for a wide range of feasible configurations, combinations, and business segmentation, as well as the relationships and functionality between components. For example, the multiple processes described in this specification can be implemented using a single device or component, or multiple devices or components operating in combination. Databases and multiple applications can be implemented in a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0102] [Method for Providing a 2-Stage Visual Inspection Model Learning Service]

[0103] The following describes in detail a method for implementing a two-stage visual detection model learning service using a computing system 1000 according to an embodiment of the present invention. In the two-stage visual detection model learning service, a first-stage process and a second-stage process for learning a classification model are separately learned according to a learning strategy optimized for each objective. In the first-stage process, in order to efficiently learn a dataset including noise labels, noise label data is detected.

[0104] The Vision Inspection Model (VIM) according to an embodiment of the present invention can represent an image deep learning model that performs anomaly detection based on a predetermined input image and classifies and / or recognizes the corresponding image accordingly.

[0105] For reference, anomaly detection can refer to the process of identifying abnormal patterns, outliers, and / or exceptions from specific data.

[0106] In other words, anomaly detection can be the process of detecting constituent elements that deviate from normal data attributes.

[0107] As an example, anomaly detection can be implemented by grouping predetermined data into clusters and treating points that deviate from the clusters as outliers.

[0108] Therefore, in the embodiment, the visual inspection model VIM determines whether the predetermined input image includes specific abnormal attributes, and based on the determination result, it can classify and / or identify the corresponding image.

[0109] The following describes a two-stage visual detection model (VIM) learning method for a computing system 1000 according to an embodiment of the present invention. Based on a learning strategy optimized for each objective, the first step process and the second step process can be learned separately. In the first step process, noisy label data is detected from the dataset used when learning the visual detection model VIM as described above. In the second step process, the corresponding visual detection model VIM is learned.

[0110] That is, the two-stage visual inspection model (VIM) learning method of the computing system 1000 according to an embodiment of the present invention, by separating the noise label data detection step and the high-performance classification model (visual inspection model VIM in the embodiment) learning step into a two-stage solution, different learning strategies can be adopted for each step.

[0111] For reference, the detection of noisy label data as ground truth is beneficial for selecting noisy labels in the feature pattern space that is widely distributed around image features, by means of the distribution of ground truth labels within clusters with similar feature patterns. However, the learning strategy used for the detection of such noisy label data can become a factor that reduces the learning performance of the classification model, in which the classification model learns feature patterns centered on ground truth labels in order to better distinguish classes.

[0112] The two-stage visual inspection model (VIM) learning method of the computing system 1000 according to an embodiment of the present invention can solve the above-mentioned problems in the existing general learning method that combines noise label detection and classification learning as described above by performing learning to optimize the purpose of each step.

[0113] Furthermore, the two-stage visual inspection model VIM learning method of the computing system 1000 according to an embodiment of the present invention utilizes the principle of contrastive learning in the noise label detection process, realizing learning that takes into account the feature differences between it and other samples, thereby ensuring the diversity of feature patterns extracted from the corresponding process and accurately detecting cluster samples with similar features.

[0114] For reference, contrastive learning refers to a learning method that shortens the distance between similar image samples and lengthens the distance between dissimilar image samples in the feature space.

[0115] In contrastive learning, a supervised learning approach, class information is used to learn so that feature vectors of the same class are close together in the feature space, while feature vectors of different classes are far apart, thus improving the discriminability between classes.

[0116] This learning method not only improves classification accuracy but also robustly learns various variations of the input through guidance, making it an advantageous learning method for tasks requiring high precision, such as industrial defect detection.

[0117] When performing both contrastive and classification learning simultaneously, higher decision accuracy can be achieved than simply performing existing classification learning based on cross-entropy loss.

[0118] However, noisy labeled data not only significantly reduces the effectiveness of this contrastive learning, but also becomes an obstacle to learning during classification, thereby reducing overall classification performance.

[0119] Therefore, the two-stage visual inspection model (VIM) learning method of the computing system 1000 according to an embodiment of the present invention provides the two-stage visual inspection model (VIM) learning service as described above, thereby enabling the visual inspection model (VIM) learning to reduce the influence of noisy label data and leverage the advantages of the aforementioned contrastive learning.

[0120] Furthermore, the two-stage visual inspection model (VIM) learning method of the computing system 1000 according to an embodiment of the present invention separates the dataset without noise labels (hereinafter referred to as the clean set) and the dataset with noise labels (hereinafter referred to as the noisy set) within the visual inspection model (VIM) learning process, and can be used according to different learning strategies.

[0121] At this point, the two-stage visual inspection model (VIM) learning method of the computing system 1000 according to an embodiment of the present invention also utilizes the principle of contrastive learning in the visual inspection model (VIM) learning process, thereby improving the precision of qualified product data while maintaining the maximum recall rate for defective data.

[0122] Below, we will separately and in detail describe the method for the first step of detecting noise label data and the method for the second step of learning a classification model using a computing system 1000 according to an embodiment of the present invention.

[0123] <Step 1: The Process of Detecting Noise Tag Data>

[0124] In this embodiment of the invention, the computing system 1000 can perform warm-up training using cross-entropy loss within a predetermined epoch, based on the overall training dataset.

[0125] Furthermore, in this embodiment of the invention, after preheating training, the computing system 1000 learns using cross-entropy loss and contrastive loss, thereby guiding multiple similar samples to be grouped into a cluster and guiding different samples to be separated from each other.

[0126] Furthermore, in this embodiment of the invention, the computing system 1000 can evaluate whether there are noisy labels on the overall learning data at the end of each epoch, thereby updating the clean set used when learning the next epoch.

[0127] In this embodiment of the invention, the computing system 1000 can utilize a mixup technique to more robustly learn the visual inspection model (VIM).

[0128] For reference, mixing is a data augmentation technique that combines two samples to assign a calibration label in an intermediate stage, thereby helping to calibrate the network during the learning process and reducing the influence of noisy or blurred samples.

[0129] In this embodiment of the invention, the computing system 1000 can filter out noisy label data from the overall learning dataset through the first learning process described above, and perform the second learning process described later based on the clean set and the noisy set evaluated at the last epoch, thereby enabling a high-performance classification model (in this embodiment, the visual detection model VIM) that excludes the influence of noisy label data.

[0130] In the following description, the classification model is described using the visual inspection model VIM according to an embodiment of the present invention, but it is not limited thereto.

[0131] Figure 4 A flowchart illustrating the first step of the learning process according to an embodiment of the present invention is shown.

[0132] For details, refer to Figure 4 According to an embodiment of the present invention, the computing system 1000 may include the steps of initializing a classification model (S101), performing warm-up training based on a predetermined period (Epoch) (S103), extracting a candidate clean set based on the overall learning data (S105), performing noise label detection learning based on the candidate clean set (S107), and extracting a determined clean set and a noise set based on the overall learning data (S109).

[0133] More specifically, in this embodiment, the computing system 1000 can initialize the classification model. (S101)

[0134] That is, in the embodiment, the computing system 1000 can initialize the visual inspection model VIM according to the embodiment of the present invention.

[0135] Figure 5An example internal block diagram of a visual inspection model (VIM) according to an embodiment of the present invention is shown.

[0136] Among them, reference Figure 5 A visual inspection model (VIM) used to classify predetermined defects may include at least one encoder (E), an adaptation layer (M), a projection head (PH), and a classifier (C).

[0137] In this embodiment, the encoder E used to extract features from the predetermined input image can be initialized to any value so that it can learn from industrial visual inspection data from the beginning. Alternatively, a backbone network encoder E can be used, wherein, in order to use rich image feature representation capabilities from the early stage of learning, the backbone network encoder E is pre-trained using a large dataset such as ImageNet.

[0138] That is, in one embodiment, the visual inspection model VIM can adopt the following... Figure 5 The encoder E shown is a pre-trained backbone network implementation. In this case, the backbone network can be based on various architectures, including ResNet and / or ConvNeXt.

[0139] On the other hand, in the embodiments, the aforementioned adapter layer M, projection head PH, and classifier C are composed of a fully connected layer, which can be implemented as a single linear layer or a multi-layer perceptron.

[0140] At this point, the projection head PH, according to the embodiment, can project the output of the adapter layer M onto the low-dimensional space, thereby calculating the contrast loss.

[0141] Furthermore, in the embodiments, the classifier C can perform learning for the purpose of distinguishing categories of visual detection data.

[0142] In this embodiment, the projection head PH, as described above, is only used for the learning step. During inference, the predetermined category of visual detection data is classified through a network consisting of the encoder E (which has completed the learning process), the adapter layer M, and the classifier C, in addition to the projection head PH.

[0143] Furthermore, in this embodiment, the computing system 1000 can perform warm-up training based on a predetermined epoch. (S103)

[0144] In detail, in the embodiment, the computing system 1000 can perform preheating training, that is, learn the visual detection model VIM based on the overall learning dataset and using cross-entropy loss.

[0145] In this embodiment, for warm-up training, the computing system 1000 can perform the above-mentioned warm-up training for a predetermined number of epochs (hereinafter referred to as the warm-up training epoch threshold).

[0146] Figure 6 An example of the variation of logit values ​​according to a training epoch is shown according to an embodiment of the present invention.

[0147] Reference Figure 6 Generally speaking, data samples that are easy to classify and have clear definitions achieve learning convergence first in the initial stage, while data samples that are difficult to classify, such as data with noisy labels, achieve learning convergence later.

[0148] Therefore, in the initial learning steps, even without classifying noisy labels, learning with the whole learning data can achieve learning convergence on relatively correctly labeled easy samples, thus being less affected by noisy labeled data within a small number of epochs.

[0149] In this embodiment, through this warm-up training, the computing system 1000 can build an initial model for classifying noise labels.

[0150] Figure 7 A flowchart illustrating a warm-up training framework for explaining the first step of the learning process according to an embodiment of the present invention is shown.

[0151] Referring to the disclosure in this invention Figure 7 This allows us to understand the data processing, input / output of each network, and loss function processing flow during the warm-up training in the first learning step.

[0152] More in detail, according to Figure 7 In one embodiment, the computing system 1000 can extract a predetermined data pair from the overall learning data or from batch units of learning data.

[0153] Furthermore, in the embodiment, the computing system 1000 can perform data augmentation based on the extracted data pairs in order to reflect the various deformations that may occur in the real environment.

[0154] As an example, the computing system 1000 can perform strong augmentation, which can cause relatively large changes to the image through random cropping, rotation, color jitter, and / or adding noise.

[0155] Furthermore, as an example, the computing system 1000 can perform weak augmentation, which causes relatively small changes to the image through horizontal flipping, small rotation and / or translation.

[0156] Furthermore, in this embodiment, the computing system 1000 can perform the aforementioned warm-up training based on the enhanced learning data.

[0157] In this embodiment, the computing system 1000 can perform preheating training based on learning data enhanced using a mixup technique.

[0158] In other words, mixing is a technique that combines two samples to assign calibration labels in an intermediate stage. It is a technique that mitigates the influence of noisy labeled samples and provides more stable learning performance.

[0159] As an example, the computing system 1000 can perform a mixup based on the following [Mathematical Formula 1], based on a predetermined first weak learning data sample - a second weak learning data sample and a predetermined first weak learning data sample - a first strong learning data sample.

[0160] [Mathematical Expression 1]

[0161]

[0162]

[0163] At this point, 'xa' and 'xb' in [Mathematical Formula 1] can represent images, and 'ya' and 'yb' can represent the labels of the corresponding images.

[0164] Furthermore, in the embodiment, the computing system 1000 can obtain the output values ​​(pas, pas_b_mix, paw_b_mix) after inputting the strongly enhanced image (xas) and the two images (xas_b_mix, xaw_b_mix) after mixing into the network composed of the encoder E, the adaptation layer M and the classifier C described above.

[0165] Furthermore, in the embodiment, the computing system 1000 can calculate the cross-entropy loss according to the following [Mathematical Formula 2] based on the output values ​​(pas, pas_b_mix, paw_b_mix) and the corresponding image labels (yas, yas_b_mix, yaw_b_mix) obtained by the system and perform network learning (i.e., warm-up training).

[0166] [Mathematical Expression 2]

[0167]

[0168]

[0169]

[0170] In [Mathematical Formula 2], 'E' can represent encoder E, 'M' can represent adaptation layer M, 'C' can represent classifier network C, 'zi' can represent feature vector of image 'xi', and 'pi' can represent prediction confidence value.

[0171] In this embodiment, the computing system 1000 can gradually increase the learning rate from the minimum value to a predetermined predefined value during the warm-up training when learning.

[0172] Therefore, the computing system 1000 avoids drastic changes that could lead to poor learning convergence or overfitting, and can achieve a stable learning environment that can adapt to new datasets.

[0173] Through the warm-up training described above, in this embodiment, the computing system 1000 can obtain an initial visual detection model VIM (i.e., an initial classification model) for classifying noise labels.

[0174] In this embodiment, when the number of preheating training periods (Epochs) is less than a predetermined preheating training period (Epoch) threshold t1, the computing system 1000 can continue the above-mentioned preheating training.

[0175] Conversely, in an embodiment, the computing system 1000 may terminate the warm-up training when the number of warm-up training epochs exceeds a predetermined warm-up training epoch threshold t1.

[0176] Figure 8 A flowchart illustrating a model learning framework after preheating training according to an embodiment of the present invention is shown.

[0177] Therefore, in this embodiment, the computing system 1000 can obtain information with the following characteristics: Figure 8 The initial visual detection model VIM (i.e., the initial classification model) of the learning framework shown.

[0178] Furthermore, in this embodiment, the computing system 1000 can extract a candidate clean set based on the overall learning data. (S105)

[0179] In this embodiment, the candidate clean set can represent the clean set (i.e., the dataset that does not contain noise labels) used for learning the visual detection model VIM in the first step learning process.

[0180] In detail, in the embodiment, after the warm-up training ends, the computing system 1000 determines whether the overall learning data is a noise label, thereby extracting the aforementioned candidate clean set.

[0181] Figure 9 A conceptual diagram illustrating a method for extracting a candidate clean set according to an embodiment of the present invention is shown. Figure 10 A flowchart illustrating a method for extracting a candidate clean set according to an embodiment of the present invention is shown.

[0182] For more details, see [link / reference] Figure 9 as well as Figure 10 In an embodiment, the computing system 1000 can predict noisy label data based on the k-NN (k-Nearest Neighbor) algorithm, thereby extracting a candidate clean set.

[0183] For reference, k-NN (k-Nearest Neighbor) can be an algorithm that detects the k nearest neighbors around a given data point and predicts the label of the corresponding data point based on the information of the detected neighbors.

[0184] Specifically, in an embodiment, the computing system 1000 can detect the k-nearest neighbors based on the visual detection model VIM that is being learned, and use the detected k-nearest neighbors as objects to perform voting on ground truth labels defined as true for supervised learning, and calculate the score based on the voting to predict whether the overall learning data is a noise label.

[0185] That is, in the embodiment, the computing system 1000 can make the above-mentioned judgment on whether it is a noise label based on the characteristic that multiple samples with similar features are more likely to have the same label, while other cases are defined as having a higher probability of being wrong labels.

[0186] At this point, in the embodiment, initially, the computing system 1000 can use the visual detection model VIM, which is learned in the preheating training step, to extract a candidate clean set based on the k-NN (k-Nearest Neighbor) algorithm. Then, using the visual detection model VIM, which is learned through the S107 step described later, at the end of each epoch, a candidate clean set based on the k-NN (k-Nearest Neighbor) algorithm is extracted.

[0187] Further reference Figure 10 As an example, the computing system 1000 may include the steps of calculating cosine similarity based on a specific data sample (S201), extracting the k neighboring samples with high calculated cosine similarity (S203), measuring the voting score considering data imbalance (S205), and including the specific data sample in a clean set (i.e., candidate clean set) or a noisy set based on the measured voting score (S207, S209).

[0188] In detail, in an embodiment, in order to measure the feature similarity between specific data samples, the computing system 1000 can calculate the cosine similarity according to the following [Mathematical Formula 3].

[0189] [Mathematical Expression 3]

[0190]

[0191] As can be confirmed in [Mathematical Formula 2], 'zi' and 'zj' in [Mathematical Formula 3] are the outputs of encoder E and adapter layer M, which can represent the feature vectors of the image.

[0192] That is, in the embodiment, the computing system 1000 can predict whether the data is noise label in the following way: for a specific image sample 'xi', in the set of k samples 'Ni' with the highest cosine similarity, the voting score is measured based on the label defined as true.

[0193] It should be noted that, in most cases, the number of data samples in each class within the training dataset is unbalanced. If we disregard the number of samples in each class, the probability that the class with a relatively large number of samples is included in the sample set 'Ni' increases. That is, the class with a relatively large number of samples is more likely to be mispredicted as a clean set, and conversely, the class with a relatively small number of samples is more likely to be mispredicted as a noisy set.

[0194] To mitigate the impact of this data imbalance problem, in an embodiment, the computing system 1000 may utilize the following [Mathematical Formula 4] to take into account the weighted value of the number of data samples for each category.

[0195] [Mathematical Expression 4]

[0196]

[0197] In [Mathematical Formula 4], 'cl' can represent the number of learning data samples in the 'l'th category out of the 'L' categories that need to be classified in the learning dataset.

[0198] In this embodiment, the computing system 1000 can calculate the label voting value in the set of k neighboring samples 'Ni' that takes into account the number of data samples for each category according to the following [Mathematical Formula 5].

[0199] [Mathematical Expression 5]

[0200]

[0201] Among them, [Mathematical Formula 5]' 'It can represent the Hadamard Product.'

[0202] Furthermore, in an embodiment, the calculation system 1000 can calculate the voting score for a specific image sample 'xi' according to the following [Mathematical Formula 6].

[0203] [Mathematical Expression 6]

[0204]

[0205] In [Mathematical Formula 6], each element of q can represent the weighted sum of the number of adjacent samples in the 'l' category, and 'qmax' can represent the maximum value among the elements of 'q'.

[0206] Furthermore, 'qt' in [Mathematical Formula 6] can represent the value of the 't'th element in the 'q' elements, where 't' is the category (Ground truth label) of a specific image sample 'xi'.

[0207] That is, in the embodiment, the more samples in the adjacent sample set that have the same label as a specific image sample 'xi', the more the calculation system 1000 increases 'vi'. When the number of adjacent samples with the same label as a specific image sample reaches the maximum, 'vi' can be set to '1'.

[0208] Thus, in this embodiment, after completing a learning epoch, the computing system 1000 can measure the voting scores of each sample in the learning dataset.

[0209] At this point, an increase in the voting score indicates that image samples with similar features have been assigned the same label, which suggests that the labeling operation is highly consistent.

[0210] That is, in the embodiment, the higher the voting score, the more the calculation system 1000 regards it as having been assigned a consistent label, and can determine it as a clean sample (i.e., a sample contained in a clean set) rather than a noisy labeled sample.

[0211] Therefore, in this embodiment, the calculation system 1000 can compare the measured voting score with a predetermined threshold (thclean, hereinafter referred to as the voting score threshold).

[0212] In this embodiment, when the measured voting score exceeds a predetermined voting score threshold (thclean), the computing system 1000 can include the corresponding learning data in the clean set (i.e., the candidate clean set).

[0213] Figure 11 An example is shown illustrating a method for selecting clean set samples according to an embodiment of the present invention.

[0214] The invention disclosed herein Figure 11 It is the set of neighboring samples when 'k=3', which shows an example of sample selection in a learning dataset that distinguishes two categories.

[0215] For example, refer to Figure 11 As a weighting value used to alleviate the problem of data imbalance Given that 'X' is '[1 / 2, 1 / 5]', the 'q' of a specific sample 'X' is '[0.17, 0.13]', and the specific sample 'X' belongs to 'Category 1', so 'qt' is '0.13' and 'qmax' is '0.17'. Therefore, the voting score of the specific sample 'X' is '0.8'. When the threshold 'thclean' is defined as '0.7', the specific sample 'X' can be determined as a clean set.

[0216] In this embodiment, when the 'pi' prediction result made by classifier C is above a certain threshold, it is equivalent to the classification model (i.e., the visual detection model VIM) determining a very high reliability result. Therefore, the computing system 1000 can change the label of the corresponding learning data according to the prediction result of classifier C based on the following [Mathematical Formula 7].

[0217] [Mathematical Expression 7]

[0218]

[0219] In this embodiment, the one-hot encoded label vector 'yi' of a specific sample image 'xi' according to [Mathematical Formula 5] can be consistent with the category label 'li' of 'xi'.

[0220] The return method description shows that, in this embodiment, the computing system 1000 can extract a candidate clean set based on the overall learning data according to the above process.

[0221] Furthermore, in this embodiment, the computing system 1000 can perform noise label detection learning based on a clean candidate set. (S107)

[0222] In detail, in the embodiment, after the model learned in step S103 above, the computing system 1000 can continue to learn noise label detection based on the candidate clean set.

[0223] That is, the computing system 1000 can perform noise label detection learning, in which the visual detection model VIM, which has been pre-trained based on a clean candidate set, is learned during the noise label detection learning.

[0224] At this point, in the embodiment, the computing system 1000 can segment the clean set (i.e., the candidate clean set) identified in step S105 above and the overall learning dataset to perform noise label detection learning based on cross-entropy loss and contrastive loss.

[0225] More specifically, in an embodiment, the computing system 1000 can extract the original dataset corresponding to the candidate clean set extracted as described above.

[0226] That is, for each piece of learning data contained in the candidate clean set, the computing system 1000 can detect the original data before performing the mixup process.

[0227] Furthermore, in this embodiment, the computing system 1000 can perform data augmentation based on the detected raw dataset.

[0228] In other words, the computing system 1000 can use the original dataset for strong augmentation and / or weak augmentation.

[0229] For a detailed explanation of this, please refer to the description disclosed in step S103 above.

[0230] Therefore, in this embodiment, the computing system 1000 can perform the above-mentioned noise label detection learning based on the enhanced learning data.

[0231] In this embodiment, based on the learning results in step S103 above, the computing system 1000 can learn the encoder E, the adaptation layer M, and the classifier C in the same manner.

[0232] Furthermore, in an embodiment, the computing system 1000 can learn the projection head PH using the contrastive loss described in [Mathematical Formula 8] below.

[0233] [Mathematical Expression 8]

[0234]

[0235] In [Mathematical Equation 8], 's' represents the cosine similarity between the output vectors of the projection head PH, and 'τ' can be the temperature scaling used for model calibration.

[0236] Thus, in an embodiment, the computing system 1000 can guide the learning of feature vectors derived from a sample image that have a similar manner to those of other sample images in the batch.

[0237] Noise label data detection, such as step S105 above, is centered on image features and selects noise labels by means of the distribution of ground truth labels within clusters with similar feature patterns. Therefore, it is particularly important to minimize feature changes in the same image while distinguishing it well from other image samples.

[0238] In this embodiment of the invention, the computing system 1000 learns through contrastive loss as shown in [Mathematical Formula 8], thereby easily achieving the above-mentioned objective.

[0239] In this embodiment, the computing system 1000 can perform network learning using the loss function of the following [Mathematical Formula 9].

[0240] [Mathematical Expression 9]

[0241]

[0242] At this point, [Mathematical Equation 9]' 'and' ' is a hyperparameter that determines the weighting of each loss function.

[0243] Thus, in this embodiment, the computing system 1000 applies the principle of contrastive learning in the first step of learning for noise label detection. This differs from the prior art, which uses the negative cosine similarity loss function to maintain feature consistency and guides the feature distribution caused by the deformation of a sample to become similar without considering the feature differences between it and other samples. Instead, it achieves learning that takes into account the feature differences between it and other samples, thereby ensuring the diversity of extracted feature patterns and enabling more accurate extraction of cluster samples with similar features.

[0244] In this embodiment, when the number of epochs for noise label detection learning is less than the predetermined threshold t2 for noise label detection training epochs, the computing system 1000 can continue to perform the above-mentioned candidate clean set extraction and noise label detection learning.

[0245] Conversely, in an embodiment, when the number of epochs of noise label detection learning exceeds a predetermined noise label detection training epoch threshold t2, the computing system 1000 may terminate clean set extraction and noise label detection learning.

[0246] Furthermore, in this embodiment, the computing system 1000 can extract a defined clean set and a noisy set based on the overall learning data. (S109)

[0247] In this embodiment, the clean set can be defined as the clean set used for learning the Visual Detection Model (VIM) in the second learning process (i.e., the dataset that does not contain noise labels).

[0248] That is, in the embodiment, the clean set can be the clean set that is finally classified and output from the visual detection model VIM learned through the first step learning process.

[0249] Furthermore, the noise set determined according to the embodiment can represent the noise set used for learning the visual detection model VIM in the second step learning process (i.e., the dataset containing noise labels).

[0250] That is, in the embodiment, the noise set can be the noise set that is finally classified and output from the visual detection model VIM learned through the first step learning process.

[0251] In detail, in the embodiment, when the noise label detection learning ends, the computing system 1000 determines whether the overall learning data is a noise label, thereby extracting the aforementioned determined clean set and determined noise set.

[0252] In other words, the computing system 1000 can use the Visual Detection Model (VIM) learned from the end-noise label detection learning to extract a clean set and a noise set based on the overall learning data.

[0253] In this embodiment, the computing system 1000 distinguishes and extracts the clean set and the noise set in the same manner as step S105 above. For a detailed description of this, please refer to the description disclosed in step S105.

[0254] Thus, in this embodiment, the computing system 1000 can obtain a determined clean set based on the extracted clean set and a determined noise set based on the extracted noise set.

[0255] Thus, in this embodiment, the computing system 1000 performs a predetermined number of warm-up training sessions using cross-entropy loss based on the overall learning dataset. After the warm-up training, it performs noise label detection learning using both cross-entropy loss and contrastive loss, thereby guiding similar samples to cluster together and different samples to separate from each other. At the end of each epoch, it evaluates whether there are noise labels on the overall learning data, thereby updating the clean set and / or noise set used when learning in the next epoch.

[0256] Thus, the computing system 1000 can obtain a clean set and a noisy set filtered with high accuracy from the overall learning dataset, and based on this, perform the following second-step learning process, thereby being able to construct a high-performance classification model (in this embodiment, the visual detection model VIM) that significantly excludes the influence of noisy labeled data.

[0257] <Step 2: High-performance classification model learning process>

[0258] Next, in this embodiment of the invention, the computing system 1000 can perform a learning process (i.e., the second step process) to improve the performance of the classification model (i.e., the visual inspection model VIM) based on the determined clean set and determined noise set defined by the first step process.

[0259] In detail, in the embodiments, the computing system 1000 can learn based on the determined clean set and determined noise set obtained as described above, according to cross-entropy loss, contrastive loss and supervised contrastive loss.

[0260] In this embodiment, the computing system 1000 may limit the determination of the noise set to use for the purpose of learning feature patterns by utilizing the raw information contained in the corresponding image samples.

[0261] In this embodiment of the invention, if the purpose of the first step is to detect noisy labeled samples and distinguish between the clean set and the noisy set, then the purpose of the second step can be to achieve the performance of the classification model optimized for the visual detection dataset.

[0262] That is, through the second step process, the computing system 1000 can achieve the following visual inspection model VIM: given the characteristics of visual inspection, which has become a very important element in preventing the outflow of defects, the accuracy of qualified product data is improved while maintaining the maximum recall rate for defect data.

[0263] Figure 12 A flowchart illustrating the second step of the learning process according to an embodiment of the present invention is shown.

[0264] For details, refer to Figure 12 According to an embodiment of the present invention, the computing system 1000 may include the steps of initializing a classification model (S301), performing warm-up training based on a determined clean set (S303), and training a high-performance classification model based on a determined clean set and a determined noise set (S305).

[0265] More specifically, in this embodiment, the computing system 1000 can initialize the classification model. (S301)

[0266] Specifically, in an embodiment, the computing system 1000 may initialize the classification model using a new visual inspection model, VIM.

[0267] In other words, the computing system 1000 can use the new visual inspection model VIM for the second step process, instead of using the model learned in the first step process.

[0268] At this point, in the embodiment, the computing system 1000 can initialize the classification model (i.e., the visual inspection model VIM) in the same manner as step S101 described above.

[0269] Furthermore, in this embodiment, the computing system 1000 can perform warm-up training based on a deterministic clean set. (S303)

[0270] In detail, in an embodiment, the computing system 1000 may perform preheating training for learning the visual detection model VIM based on a determined clean set defined through the first step process, using cross-entropy loss.

[0271] In this embodiment of the invention, the specific method for the computing system 1000 to perform preheating training is based on the description of step S103 above. The following mainly describes the differences from step S103.

[0272] As mentioned above, data samples that are easy to classify typically achieve learning convergence first in the initial stage. Therefore, in the first step, a predetermined number of epochs of learning are performed to mitigate the influence of noisy labeled data. However, warm-up training is conducted using the overall learning dataset containing noisy labeled samples, which may include some of the influence of noisy labels.

[0273] Therefore, in the second step of the present invention, the clean set identified in the first step (i.e., the clean set is determined) is used for preheating training to block the influence of noisy labeled samples.

[0274] Figure 13 A flowchart illustrating a warm-up training framework for the second step of the learning process according to an embodiment of the present invention is shown.

[0275] Referring to the disclosure of this invention Figure 13 This allows us to understand the data processing, input / output of each network, and loss function processing flow during the warm-up training in the second learning step.

[0276] according to Figure 13In an embodiment, the computing system 1000 may perform a second-step warm-up training of the visual inspection model VIM in the same manner as the warm-up training process in the first-step learning process, using strong augmentation, weak augmentation, and mixup techniques.

[0277] In other words, mixing involves combining two samples to assign calibrated labels to intermediate steps, thereby providing effective network calibration during the learning process. Therefore, if used in conjunction with oversampling of classes containing a small number of samples, it can significantly alleviate the problem of data imbalance.

[0278] In this embodiment, the computing system 1000 may use the CrossEntropy Loss as described in [Mathematical Formula 2] above for warm-up training based on a deterministic clean set.

[0279] Furthermore, in the embodiment, when the number of preheating training periods (Epochs) is less than a predetermined preheating training period (Epoch) threshold t3, the computing system 1000 can continue to perform the aforementioned preheating training.

[0280] Conversely, in an embodiment, the computing system 1000 may terminate the warm-up training when the number of warm-up training epochs exceeds a predetermined warm-up training epoch threshold t3.

[0281] Thus, in this embodiment, the computing system 1000 uses a clean set defined by the first step process (i.e., determining the clean set) for warm-up training in the second step process, thereby preventing the influence of noisy labels that may be included in the overall learning dataset.

[0282] Furthermore, in this embodiment, the computing system 1000 can perform high-performance classification model training based on a determined clean set and a noisy set. (S305)

[0283] In detail, in the embodiment, after the model learned in step S303 above, the computing system 1000 can continue to train a high-performance classification model based on a determined clean set and a determined noisy set.

[0284] That is, the computing system 1000 can perform high-performance classification model training to learn the visual detection model VIM, wherein the visual detection model VIM is pre-trained based on a determined clean set and a determined noisy set.

[0285] In this embodiment, the computing system 1000 can, as follows: Figure 8 The diagram illustrates the segmentation (distinction) process to determine the clean set and the noisy set, followed by learning based on Cross Entropy Loss, Contrastive Loss, and Supervised Contrastive Loss.

[0286] More specifically, in the embodiments, the computing system 1000, based on a determined noise set, utilizes the contrastive loss as described above [Mathematical Equation 8] to use the feature information contained in the noisy label data for training a high-performance classification model.

[0287] That is, in the embodiment, the computing system 1000 performs feature learning using contrastive loss for noise-labeled samples that are determined to have incorrectly assigned label information or noise-labeled samples that need to utilize the raw information contained in the corresponding image samples, so that noise label information can be excluded during learning and only feature information can be used.

[0288] On the other hand, in an embodiment, the computing system 1000 can train a high-performance classification model based on a determined clean set, using the cross-entropy loss according to [Mathematical Formula 2] above and the supervised contrastive loss according to [Mathematical Formula 10] below.

[0289] [Mathematical Expression 10]

[0290]

[0291] Among them, [Mathematical Formula 11]' 'It can be a hyperparameter that determines the weighting of the loss function.'

[0292] Furthermore, [Mathematical Formula 10]' 'Represents the set of positives belonging to the same category within a batch'. 'It can represent the entire batch of sample sets.'

[0293] Furthermore, in [Mathematical Equation 10], 'vi' is the output vector of the projection head PH, and 'τ' can be a temperature scaling parameter used for model calibration.

[0294] Thus, in the embodiment, the computing system 1000 further performs supervised contrastive loss learning based on the determined clean set, which makes the feature vectors of the same category close together in the feature space, while the feature vectors of different categories are separated, thereby improving the discriminative power between categories.

[0295] Therefore, the computing system 1000 constructed a visual inspection model, VIM, which not only enhanced classification accuracy but also worked robustly to various input variations, providing a classification model optimized for tasks such as defect detection that require high precision.

[0296] In this embodiment, the computing system 1000 can perform network learning using the loss function described in [Mathematical Formula 10] above.

[0297] In this embodiment, when the number of epochs for training the high-performance classification model is less than the predetermined threshold t4 for training the high-performance classification model, the computing system 1000 can continue to perform the high-performance classification model training as described above.

[0298] Conversely, in an embodiment, when the number of epochs of high-performance classification model training exceeds a predetermined high-performance classification model training epoch threshold t4, the computing system 1000 may terminate the high-performance classification model training.

[0299] Therefore, the computing system 1000 according to an embodiment of the present invention is able to provide a classification model (i.e., a visual detection model VIM) that is efficiently learned based on a dataset including noise labels.

[0300] As an example, the computing system 1000 can provide a visual detection model (VIM) learned according to the example based on a predetermined application service (e.g., outlier detection service, etc.).

[0301] As described above, in this embodiment of the invention, the computing system 1000 stably learns the model in the first step by using a warm-up training strategy, a mixup technique, and the use of an adaptation layer M, while detecting noisy labeled data. In the second step, the classification performance of the model can be improved by using supervised learning-based contrastive loss and cross-entropy loss based on the refined data as described above.

[0302] That is, in the embodiment, the computing system 1000 uses a two-step solution that separates the noise label data detection step and the high-performance classification model training step, and uses different customized learning strategies in each step, thereby blocking the degradation of model performance caused by learning data with incorrect labels.

[0303] In particular, in this embodiment, unlike the prior art, the computing system 1000 simultaneously performs learning based on contrastive loss without using the error label information of the noisy labeled dataset but using the feature information contained in the corresponding data, thereby enabling the construction and provision of a visual inspection model (VIM) that maintains maximum recall for defective data while improving precision for qualified product data.

[0304] Furthermore, the embodiments of the present invention described above can be implemented as program instructions that can be executed by various computer components and can be recorded in a computer-readable storage medium. The aforementioned computer-readable storage medium may contain program instructions, data files, data structures, etc., individually or in combination. The program instructions recorded in the aforementioned computer-readable storage medium may be instructions specifically designed and constructed for the present invention, or instructions known and usable by those skilled in the art of computer software. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical storage media such as CD-ROMs and DVDs; magneto-optical media such as floppy disks; and hardware devices specifically used for storing and executing program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine language code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc. To perform the processing according to the present invention, the hardware device can be changed to one or more software modules, and vice versa.

[0305] The specific implementation described in this invention is merely one example and does not limit the scope of protection of this invention in any way. For the sake of brevity, descriptions of existing electronic structures, control systems, software, and other functional aspects of the aforementioned systems may be omitted. Furthermore, the line connections or connecting parts between the constituent elements shown in the drawings are exemplary illustrations of functional and / or physical or circuit connections. In actual devices, these connections may be substituted, or they may be represented by various other functional, physical, or circuit connections. Moreover, unless specifically stated as "essential" or "important," the corresponding constituent element may not be essential for the application of this invention.

[0306] Furthermore, while the detailed description of the present invention has been presented with reference to preferred embodiments, it should be understood that those skilled in the art or with conventional knowledge of the art can make various modifications and alterations to the present invention without departing from the spirit and scope of the invention as set forth in the claims. Therefore, the scope of the present invention should be defined based on the claims and should not be limited to the content described in the detailed description.

[0307] Industrial availability

[0308] This invention relates to a visual inspection model learning method and system, which can be applied to the artificial intelligence industry and therefore has industrial applicability.

Claims

1. A vision inspection model learning method of learning a vision inspection model by a computing system including a memory and a processor, the vision inspection model learning method comprising: a step of performing a first process (Process 1) in which noisy label data in a training data set is detected; a step of obtaining a noisy data set and a clean data set based on the detected noisy label data; and a step of performing a second process (Process 2) in which a vision inspection model is learned based on the obtained noisy data set and clean data set.

2. The vision inspection model learning method according to claim 1, wherein the vision inspection model includes at least an encoder, an adaption layer, a projection head, and a classifier.

3. The vision inspection model learning method according to claim 2, wherein the step of performing the first process includes: a step of initializing the vision inspection model; a step of performing a first warm-up training (Warm-up Training 1) in which the initialized vision inspection model is learned for a predetermined number of epochs based on the training data set; a step of detecting noisy label data in the training data set based on the vision inspection model after the first warm-up training; a step of obtaining an arbitrary clean data set from the detected noisy label data; a step of performing noisy label detection learning in which the vision inspection model after the first warm-up training is learned for a predetermined number of epochs based on the obtained arbitrary clean data set and the training data set; a step of detecting noisy label data in the training data set based on the vision inspection model after the noisy label detection learning; and a step of obtaining a noisy data set and a clean data set based on the detected noisy label data.

4. The vision inspection model learning method according to claim 3, wherein the step of performing the first warm-up training includes: a step of learning based on a cross entropy loss.

5. The vision inspection model learning method according to claim 4, wherein the step of performing the first warm-up training further includes: a step of extracting a predetermined data pair from the training data set; a step of performing data augmentation based on the extracted data pair; and a step of performing data augmentation based on the extracted data pair. Based on the data augmentation performed above, a step of obtaining weak augmentation data and strong augmentation data, the weak augmentation data being data with relatively small deformation, and the strong augmentation data being data with relatively large deformation. 6.The visual inspection model learning method according to claim 5, wherein The step of performing the first warm-up training further includes: A step of obtaining first mixed data and second mixed data, wherein the first mixed data is mixed with the first weak augmentation data and the second weak augmentation data, and the second mixed data is mixed with the first weak augmentation data and the first strong augmentation data. 7.The visual inspection model learning method according to claim 6, wherein The step of performing the first warm-up training further includes: Based on the first mixed data, the second mixed data, and the first strong augmentation data, a step of performing learning based on the cross-entropy loss. 8.The visual inspection model learning method according to claim 3, wherein The step of detecting the noisy label data includes: A step of calculating a Cosine similarity between the first learning data and other learning data in the learning data set; A step of extracting k pieces of adjacent learning data, the adjacent learning data being other learning data with a high Cosine similarity calculated above; A step of calculating a voting score based on the extracted adjacent learning data; and A step of detecting the noisy label data based on the calculated voting score. 9.The visual inspection model learning method according to claim 3, wherein The step of performing the noisy label detection learning includes: A step of learning the projection head based on the first clean data set and a Contrastive Loss; and A step of learning the encoder, the adaptation layer, and the classifier based on the learning data set and the cross-entropy loss. 10.The visual inspection model learning method according to claim 3, wherein The step of performing the noisy label detection learning includes: A step of detecting an original data set corresponding to the first clean data set; A step of performing data augmentation based on the detected original data set; and The step of performing the noisy label detection learning based on the augmented data. 11.The visual inspection model learning method according to claim 3, wherein The step of performing the second process includes: A step of newly initializing the visual inspection model; A step of performing second warm-up training, wherein in the second warm-up training, the newly initialized visual inspection model is learned based on the clean data set for a predetermined number of epochs; and performing a high-performance classification model training, wherein, in the high-performance classification model training, a predetermined number of epochs of learning of the second pre-trained vision inspection model is performed based on the clean data set and the noisy data set. 12.The vision inspection model learning method of claim 11, wherein, the performing of the high-performance classification model training comprises: performing learning using both a cross-entropy loss and a supervised contrastive loss based on the clean data set; and performing learning based on a contrastive loss based on the noisy data set. 13.The vision inspection model learning method of claim 1, further comprising: providing the vision inspection model learned based on the second process through a predetermined application service. 14.A vision inspection model learning system comprising: at least one memory; and at least one processor that reads at least one application program stored in the memory, learns a vision inspection model, wherein the instructions of the processor include instructions to: perform a first process in which noisy label data in a training data set is detected, obtain a noisy data set and a clean data set based on the noisy label data detected based on the first process, and perform a second process in which a vision inspection model is learned based on the obtained noisy data set and clean data set. 15.A computing device comprising: at least one encoder, at least one adaption layer, at least one projection head, at least one classifier, and at least one processor that controls the encoder, the adaption layer, the projection head, and the classifier, wherein the processor performs a first process in which noisy label data in a training data set is detected, the processor obtains a noisy data set and a clean data set based on the noisy label data detected based on the first process, the processor performs a second process in which a vision inspection model is learned based on the obtained noisy data set and clean data set.

Images