Method, computer-readable medium and device for learning system for new visual inspection tasks using few-shot meta-learning

Abstract

To provide systems and methods that create a model to perform visual inspection across multiple domains of input data.SOLUTION: A method includes the steps of: processing, for first input of a plurality of labeled images of a new domain task, a first plurality of labeled images through a plurality of backbone snapshots, each of the backbone snapshots representing a model trained across a plurality of other domain tasks, each of the plurality of backbone snapshots being configured to output a first plurality of features responsive to the input; processing second input of a second plurality of unlabeled images through the plurality of backbone snapshots to output a second plurality of features responsive to the second input; and generating a representative model for the new domain task from clustering and transformation of the first plurality of features so as to be associated from the clustered and transformed second plurality of features.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure is directed generally to visual inspection systems, and more particularly to methods, computer-readable media, and apparatus in meta-learning systems for visual inspection tasks. [Background technology]

[0002] Many companies, including those in manufacturing and logistics, are leveraging computer vision techniques to automate visual inspection processes. For example, in manufacturing plants, cameras can be installed on production lines to continuously capture images of products as they are being made. These images are fed into computer vision algorithms to detect defects in the manufactured products. If a defect is detected, the system can alert an operator and intervene in the process to make any necessary corrections.

[0003] Most computer vision algorithms in related technologies are deep learning (DL)-based algorithms. These algorithms do not require the specification of handcrafted features or domain knowledge; instead, they take large amounts of labeled data, such as images of defective and non-defective products and their corresponding labels, as input and automatically train a model to detect whether a particular image is defective. However, traditional methods often require training a new model for each new domain (e.g., a factory production line), and training a DL model requires a considerable amount of data.

[0004] Cross-domain meta-learning is a technique for training generic DL models across multiple domains, allowing them to be fine-tuned and reused in new domains without requiring significant time-consuming retraining of the model or large amounts of data from the new domain.

[0005] In the above context, cross-domain is defined as the process of applying a model across different domains of data. For example, a model originally used to classify defects in transistors (domain A) can be applied to classify defects in carpets (domain B) and defects in metal nuts (domain C).

[0006] In the context of meta-learning, prominent examples of meta-learning algorithms include algorithms such as Prototypical Networks, Model Agnostic Meta Learning, and MetaDelta. These related technology methods are all variants of meta-learning algorithms that attempt to train models that quickly adapt to new data domains. However, most of these related technology methods have been primarily used for general image classification rather than visual inspection. With regard to visual inspection, common deep learning methods include transfer learning and domain adaptation. These methods also attempt to use models trained in one domain to be deployed in another domain. Nevertheless, most of these methods do not generalize well to multiple domains with limited data. Summary of the Invention [Problem to be solved by the invention]

[0007] Many companies rely on manual visual inspection, which is inefficient and time-consuming. While using deep learning-based computer vision models for automated visual inspection can help, they often do not generalize across multiple domains, often requiring new models to be developed for each new visual inspection domain. Training a model for each new domain is time- and resource-consuming and involves collecting new datasets, which is impractical for many industrial applications because defect data is typically rare. Related art techniques mentioned above, such as transfer learning and domain adaptation, often require a significant amount of labeled data, often exceeding the amount of defect data typically available in the visual inspection domain. Furthermore, related art methods typically transfer only across similar, specific domains (e.g., classifying glass cracks to wood cracks) and do not generalize well to diverse domains, such as classifying glass cracks to defects in electronic components or hazelnuts.

[0008] Furthermore, meta-learning has not been successfully applied to visual inspection, and other techniques such as transfer learning and domain adaptation do not work well in cross-domain applications where data is limited.

[0009] The embodiments described herein are directed to systems and methods for creating models that perform visual inspection across multiple domains of input data. An exemplary implementation uses minimal labeled examples to adapt to new visual inspection domains.

[0010] Furthermore, the exemplary implementation is computationally efficient (e.g., on the order of a few seconds) in adapting to new visual inspection domains, and can be directly applied to new visual inspection tasks without costly modifications to the model architecture.

[0011] The embodiments described herein can further include cross-domain visual inspection adaptability through the use of a squeeze excitation module and anti-aliasing filters within the backbone architecture.

[0012] In the examples described herein, we use a combination of classification and contrastive losses in the meta-learning step to improve cross-domain adaptation for visual testing.

[0013] This implementation also leverages a snapshot ensembling scheme to obtain multiple backbone versions, resulting in better generalization across the visual inspection domain.

[0014] The examples described herein can be generalized to directly adapt to any n-way k-shot visual inspection task, where N-way denotes the number of classes in the new visual inspection task (n>=2), and K-shot denotes the number of labeled examples per class provided to the model for adaptation.

[0015] This embodiment also results in an ensemble of trained models that is computationally efficient to deploy with limited computational resources. [Means for solving the problem]

[0016] Accordingly, in a first aspect, an embodiment of the present invention provides a method in a system for learning a new visual inspection task using a few-shot meta-learning method, comprising: for an input of a first plurality of labeled images of a new domain task, processing the first plurality of labeled images through a plurality of backbone snapshots, each backbone snapshot representing a model trained across a plurality of other domain tasks, each of the plurality of backbone snapshots outputting a first plurality of features in response to the input; processing a second input of a second plurality of unlabeled images through the plurality of backbone snapshots to output a second plurality of features in response to the second input; performing a feature transformation process configured to transform the first plurality of features and the second plurality of features into similarity values ​​associating the first plurality of features with the second plurality of features based on similarity; performing a clustering process to cluster the first plurality of features with the second plurality of features based on the similarity values; and generating a representative model of the new domain task from the clustering and transformation of the first plurality of features and from the clustered and transformed second plurality of features to be associated.

[0017] Also, in a second aspect, an embodiment of the present invention provides a non-transitory computer-readable medium having stored thereon instructions for performing a process, the instructions including: for an input of a first plurality of labeled images of a new domain task, processing the first plurality of labeled images through a plurality of backbone snapshots, each backbone snapshot representing a model trained across a plurality of other domain tasks, each of the plurality of backbone snapshots outputting a first plurality of features in response to the input; and for processing a second plurality of unlabeled images through the plurality of backbone snapshots. 12. A computer-readable medium having stored thereon instructions: processing a second input to output a second plurality of features responsive to the second input; performing a feature transformation process configured to transform the first plurality of features and the second plurality of features into similarity values ​​that associate the first plurality of features with the second plurality of features based on similarity; performing a clustering process to cluster the first plurality of features with the second plurality of features based on the similarity values; and generating a representative model of a new domain task from the clustering and transformation of the first plurality of features and as associated from the clustered and transformed second plurality of features.

[0018] Also, in a third aspect, an embodiment of the present invention relates to a computer device having a processor, wherein the processor, for an input of a first plurality of labeled images of a new domain task, processes the first plurality of labeled images through a plurality of backbone snapshots, each backbone snapshot representing a model trained across a plurality of other domain tasks, each of the plurality of backbone snapshots outputting a first plurality of features responsive to the input; processes a second input of a second plurality of unlabeled images through the plurality of backbone snapshots to output a second plurality of features responsive to the second input; performs a feature transformation process configured to transform the first plurality of features and the second plurality of features into similarity values ​​associating the first plurality of features with the second plurality of features based on similarity; performs a clustering process to cluster the first plurality of features with the second plurality of features based on the similarity values; and generates a representative model of the new domain task from the clustering and transformation of the first plurality of features and from the clustered and transformed second plurality of features to be associated.

[0019] Also, in a fourth aspect, embodiments of the present disclosure may include an apparatus that may include one or more processors configured to: for a first input of a plurality of labeled images of a new domain task, process the first plurality of labeled images through a plurality of backbone snapshots, each backbone snapshot representing models trained across a plurality of other domain tasks, each of the plurality of backbone snapshots representing models trained across a plurality of other domain tasks, outputting a first plurality of features responsive to the input; processing a second input of a second plurality of unlabeled images through the plurality of backbone snapshots to output a second plurality of features responsive to a second input; performing a feature transformation process configured to transform the first plurality of features and the second plurality of features into similarity values ​​that associate the first plurality of features and the second plurality of features together based on similarity; performing a clustering process to cluster the first plurality of features and the second plurality of features based on the similarity values; and generating a representative model of the new domain task from the clustering and transformation of the first plurality of features and from the clustered and transformed second plurality of features to be associated. [Effects of the Invention]

[0020] According to the present invention, a system and method can be provided for creating models that perform visual inspection across multiple domains of input data. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 shows an example of the overall framework of the proposed cross-domain meta-learning technique according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a meta-learning phase according to an embodiment. [Figure 3] FIG. 3 illustrates a meta-testing process for processing a support set, according to an embodiment. [Figure 4] FIG. 4 illustrates a meta-test process for processing a query set, according to an embodiment. [Figure 5] FIG. 5 illustrates an example of a meta-testing stage that includes a self-optimal transport (SOT) feature transformation as an instantiation of feature post-processing, according to an embodiment. [Figure 6] FIG. 6 shows an example of classification by an iterative soft k-means decoder, according to an embodiment. [Figure 7] FIG. 7 illustrates an example of a flow diagram for generating a representative model, according to an embodiment. [Figure 8] FIG. 8 illustrates an exemplary computing environment including an exemplary computer device suitable for use in some exemplary implementations. DETAILED DESCRIPTION OF THE INVENTION

[0022] The following detailed description provides details of the figures and examples of the present application. Reference numerals and redundant element descriptions between figures have been omitted for clarity. Terms used throughout this specification are provided by way of example and are not intended to be limiting. For example, the use of the term "automatic" can include fully automatic or semi-automatic implementations with user or administrator control over certain aspects of the implementation, depending on the desired implementation of one skilled in the art practicing embodiments of the present invention. Selections can be performed by a user through a user interface or other input means, or can be performed through a desired algorithm. The exemplary implementations described herein can be utilized either singularly or in combination, and the functions of the exemplary implementations can be implemented through any means according to the desired implementation.

[0023] FIG. 1 illustrates an example of the overall framework of a learning system, which is a proposed cross-domain meta-learning technique, according to an embodiment. The meta-learning method illustrated in FIG. 1 includes a meta-learning stage 101 for generating a meta-learned backbone 102 and a meta-testing stage 103. During meta-learning 101, a classification loss 110 and a contrastive loss 111 are used together to train the backbone. In this embodiment, the meta-learning method can be referred to as a few-shot meta-learning method due to its characteristics. A detailed description of the framework is provided in FIGS. 2 to 6.

[0024] 2 illustrates an example meta-learning phase, according to an embodiment. During meta-learning, a backbone model is trained on multiple domains of visual inspection data (dataset A, dataset B, dataset C). In the exemplary implementation described herein, the backbone is a large-scale deep learning model, such as a residual network. Such a backbone can have a large number (e.g., millions) of parameters and is trained on multiple different image domains using multiple different datasets. Such a backbone can also be obtained from publicly available sources, such as datasets of metal objects and circuit boards.

[0025] Specifically, a multilayer perceptron (MLP) is used on a backbone to train the model using a combination of classification and contrastive loss. Because the MLP functions as a classifier, all target datasets are passed through the backbone, generating features that the MLP uses to train the backbone to become familiar with multiple target domains. In the example of FIG. 2, three domains are used, but more domains may be used according to the desired implementation, and the present disclosure is not limited thereto. Training is based on a classification loss (e.g., as defined by the classification objective) and a contrastive loss that enforces differences between classes. As a result of training, a backbone is obtained that generates features that are far from each other for different classes and close to each other for same classes. Such training can improve the quality of the features generated by the backbone. As shown in Figure 2, the backbone generates features that are fed into the MLP, which learns end-to-end based on classification and contrastive losses. Once such meta-learning across multiple domains has been performed, the MLP classifier can be removed and the backbone is kept to generate features.

[0026] Figure 3 illustrates a meta-testing process for processing a support set, according to an embodiment. During meta-testing, a support set containing a small number of labeled examples from new visual inspection domain data (dataset D) is fed to the backbone model trained in the meta-learning phase. This generates a feature set associated with each class label, called a prototype. This process is repeated using multiple backbones obtained in the meta-learning phase to obtain an ensemble of feature sets.

[0027] To make learning more efficient, the support set is run through multiple backbones generated by snapshot ensemble. During training, copies of the backbone are kept separate for ensemble processing. The support set is then run through the copies of the backbone to generate multiple feature vectors, which are aggregated prototypes across classes.

[0028] Figure 4 shows a meta-test process for processing a query set, according to an embodiment. After processing the support set, a query set containing unlabeled data from a new visual inspection domain is also processed by the ensemble of backbone models. This step also generates an ensemble feature set associated with each query input image.

[0029] In the example of Figure 4, the prototypes of Figure 3 are collected, and then a query is processed as shown in Figure 4. That is, the system processes unlabeled images related to the task or object to be learned. The unlabeled images are sent through an ensemble of backbones to generate a query feature vector. Once the query feature vector is generated, it is matched with the prototype feature vector as described in Figure 5.

[0030] Figure 5 shows an example meta-testing stage that includes self-optimal transport (SOT) feature transformation as an instantiation of feature post-processing, according to an embodiment. The ensemble of prototype and query features (from the processing of Figures 3 and 4) is then further transformed using a post-processing technique that maps the features to another space. An example of such a post-processing technique is the self-optimal transport (SOT) feature transformation algorithm.

[0031] In the example of Figure 5, the SOT feature transformation algorithm improves the quality of the features. In the process, the SOT feature transformation algorithm spreads the transformed prototype and the transformed query features so that different features are pushed farther apart and similar features are brought closer together. Figure 6 illustrates an example of classification using an iterative soft k-means decoder, according to an embodiment. Labels for query features are assigned based on the closest distance between the query feature and each prototype. In the example of Figure 6, prototype features are matched with query features (unlabeled features) so that labels can be assigned using a clustering algorithm such as soft k-means classification. Through the examples shown in Figures 2-6, it is possible to adapt the backbone model to new tasks, albeit with limited examples.

[0032] An example embodiment can be considered as follows.

[0033] During meta-learning, an exemplary implementation can utilize one or more industrial visual inspection datasets as pre-training data for the meta-learning phase, as shown in Figure 2. To instantiate the backbone model, the open-source ImageNet pre-trained SEResNext101 with an anti-aliasing filter can be used and trained on the visual inspection pre-training data. The backbone model can be trained using a traditional supervised learning procedure. For example, the model training procedure can be implemented as follows: The model is trained for approximately four hours using the Adam optimizer with a learning rate of 8e-5 and a batch size of 60. Standard image augmentation techniques, such as color normalization, horizontal flipping, blurring, and perspective transformation, are performed on the fly during training. Furthermore, the model is trained using a classification loss—specifically, a combination of a supervised cross-entropy loss and a contrastive loss (e.g., triplet margin loss), where the contrastive loss has a weighting factor of 10 and the supervised loss has a weighting factor of 1. The top two best-performing models are saved based on their validation scores during the training process for use as an ensemble to extract a more diverse feature set in the meta-testing phase.

[0034] In the meta-testing stage, given a new domain of visual inspection data, the ensemble of backbone models trained in the previous meta-learning stage is loaded. Next, a small number of labeled examples are collected from the new domain of visual inspection data to construct a support set, and the remaining unlabeled data is set aside as a query set. After obtaining the support set and query set, the query set and support set features can be generated by passing the images through the ensemble of backbone models and concatenating the features, as shown in Figures 3 and 4. Based on Figure 5, the features in the support set and query set are transformed using a self-optimal transport algorithm (SOT), and the SOT-transformed features are sent to an iterative soft k-means decoder to assign labels to the query set based on the closest distance between the query feature and each prototype.

[0035] Through the examples described herein, there is a novel solution for automated visual inspection that uses minimal computational resources and labeled data to yield a trained model that clients can easily deploy, regardless of the domain. This method can also be rapidly deployed based on client preferences, making it an ideal solution for time- and resource-constrained companies. The automated visual inspection provided by the examples described herein allows clients to significantly reduce expenditures for manual inspection while benefiting from highly accurate and efficient processing. Furthermore, the examples described herein reduce the development time and cost for creating new automated visual inspection solutions for different domains, thereby increasing the ability to innovate and providing clients with cost-effective solutions.

[0036] 7 is an exemplary flow diagram for generating a representation model, according to an embodiment. To begin the flow, an input of a first plurality of labeled images of a new domain task is provided, as shown in FIG. 3 (Dataset D). At 701, the flow processes a first plurality of labeled images through a plurality of backbone snapshots, each representing a model trained across a plurality of other domain tasks (e.g., as illustrated in FIG. 2 ). Each of the plurality of backbone snapshots is configured to output a first plurality of features responsive to an input, as illustrated in FIG. At 702, the flow processes a second input of a second plurality of unlabeled images through a plurality of backbone snapshots to output a second plurality of features responsive to the second input, as illustrated in FIG. At 703, the flow performs a feature transformation process configured to transform the first plurality of features and the second plurality of features into a similarity value that associates the first feature and the second plurality of features together based on similarity, as illustrated in FIG. 5 . At 704, the flow performs a clustering process to cluster the first plurality of features and the second plurality of features based on similarity values, as illustrated in FIG. At 705, a representative model of the new domain task is generated as related from the clustering and transformation of the first plurality of features and from the clustered and transformed second plurality of features.

[0037] 8 illustrates an exemplary computing environment having an exemplary computing device suitable for use in some exemplary implementations. The computing device 805 of the computing environment 800 can include one or more processing units, cores, or processors 810, memory 815 (e.g., RAM, ROM, and / or the like), internal storage 820 (e.g., magnetic, optical, solid-state storage, and / or organic), and / or IO interface 825, any of which can be coupled to or embedded in the computing device 805 over a communication mechanism or bus 830 for communicating information. The IO interface 825 can also be configured to receive images from a camera or provide images to a projector or display, depending on the desired implementation.

[0038] Computing device 805 can be communicatively coupled to input / user interface 835 and output device / interface 840. Either or both of input / user interface 835 and output device / interface 840 can be wired or wireless interfaces and can be detachable. Input / user interface 835 can include any device, component, sensor, or interface, physical or virtual, that can be used to provide input (e.g., buttons, touchscreen interface, keyboard, pointing / cursor control, microphone, camera, Braille, motion sensor, accelerometer, optical reader, and / or the like). Output device / interface 840 can include a display, television, monitor, printer, speaker, Braille, etc. In some exemplary implementations, input / user interface 835 and output device / interface 840 can be embedded in or physically coupled to computing device 805. In other exemplary implementations, other computing devices can function as or provide the functionality of input / user interface 835 and output device / interface 840 of computing device 805.

[0039] Examples of computing devices 805 may include, but are not limited to, highly mobile devices (e.g., smartphones, devices mounted on vehicles and other machines, devices carried by humans and animals, etc.), mobile devices (e.g., tablets, notebooks, laptops, personal computers, portable televisions, radios, etc.), and devices not designed for mobility (e.g., desktop computers, other computers, information kiosks, televisions with one or more processors embedded therein and / or coupled thereto, radios, etc.).

[0040] Computing device 805 may be communicatively coupled (e.g., via IO interface 825) to external storage 845 and network 850 for communicating with any number of networked components, devices, and systems, including one or more computing devices of the same or different configurations. Computing device 805 or any connected computing device may function, provide a service, or be referred to as a server, a client, a thin server, a general machine, a special purpose machine, or another label.

[0041] IO interface 825 may include, but is not limited to, wired and / or wireless interfaces using any communication or IO protocol or standard (e.g., Ethernet, 802.11x, Universal System Bus, WiMAX, modem, cellular network protocol, etc.) for communicating information to and from at least all connected components, devices, and networks in computing environment 800. Network 850 may be any network or combination of networks (e.g., the Internet, a local area network, a wide area network, a telephone network, a cellular network, a satellite network, etc.).

[0042] The computing device 805 can use and / or communicate computer-usable or computer-readable media, including transient and non-transitory media. Transitory media include transmission media (e.g., metallic cables, optical fibers), signals, carrier waves, etc. Non-transitory media include magnetic media (disks, tapes, etc.), optical media (CD ROM, digital video disks, Blu-ray disks, etc.), solid-state media (RAM, ROM, flash memory, solid-state storage, etc.), and other non-volatile storage or memory.

[0043] The computing device 805 can be used to implement techniques, methods, applications, processes, or computer-executable instructions in some exemplary computing environments. The computer-executable instructions can be obtained from a transitory medium, stored on a non-transitory medium, or obtained from a non-transitory medium. The executable instructions can be from one or more of any programming, scripting, and machine language (e.g., C, C++, C#, Java, Visual Basic, Python, Perl, JavaScript, etc.).

[0044] The processor(s) 810 can run under any operating system (OS) (not shown) in a native or virtual environment. One or more applications can be deployed, including a logic unit 860, an application programming interface (API) unit 865, an input unit 870, an output unit 875, and an inter-unit communication mechanism 895 through which different units communicate with each other, with the OS, and with other applications (not shown). The described units and elements may vary in design, function, configuration, or implementation and are not limited to the provided description. The processor(s) 810 can be in the form of a hardware processor, such as a central processing unit (CPU), or a combination of hardware and software units.

[0045] In some exemplary implementations, when information or instructions for execution are received by API unit 865, it may be communicated to one or more other units (e.g., logic unit 860, input unit 870, output unit 875). In some embodiments, logic unit 860 may be configured to control the flow of information between units and direct the services provided by API unit 865, input unit 870, and output unit 875, as in some exemplary embodiments described above. For example, one or more process or implementation flows may be controlled by logic unit 860 alone or in cooperation with API unit 865. Input unit 870 may be configured to obtain inputs for the calculations described in the exemplary embodiments, and output unit 875 may be configured to provide outputs based on the calculations described in the exemplary embodiments.

[0046] The processor(s) 810 may be configured to execute a method or instructions including: for an input of a first plurality of labeled images of a new domain task, processing the first plurality of labeled images through a plurality of backbone snapshots, each of the backbone snapshots representing models trained across a plurality of other domain tasks, each of the plurality of backbone snapshots configured to output a first plurality of features responsive to the input, as described with respect to 701 of FIG. 7 . 7 includes processing a second input of a second plurality of unlabeled images through a plurality of backbone snapshots to output a second plurality of features responsive to the second input, as described with reference to 702 of FIG. 7; performing a feature transformation process configured to transform the first plurality of features and the second plurality of features into similarity values ​​that associate the first plurality of features and the second plurality of features together based on similarity, as described with reference to 703 of FIG. 7; performing a clustering process to cluster the first plurality of features and the second plurality of features based on the similarity values, as described with reference to 704 of FIG. 7; and generating a representative model of the new domain task from the clustering and transformation of the first plurality of features and from the clustered and transformed second plurality of features to be associated, as described with reference to 705 of FIG. 7 .

[0047] The processor(s) 810 may be configured to execute methods or instructions described herein, where multiple backbone snapshots are meta-learned in combination with a multi-layer perceptron (MLP) layer, e.g., an industrial task, as described with respect to FIG. 2.

[0048] As described herein, multiple backbone snapshots can be meta-learned based on classification loss and contrastive loss, as shown in FIG. 2.

[0049] As described herein, the new domain task can be directed to any target object or task according to a desired implementation. For example, if the new domain task is directed to visual inspection, the labeled images can include different defect types (as classes) of the visual inspection object or task, and the representative model can be configured to output the defects detected in the visual inspection. Other domain tasks can also be used, and the disclosure is not limited thereby.

[0050] As described herein, the multiple backbone snapshots can be representative of industrial tasks or objects. For example, the backbone can be trained across industrial objects (e.g., solder, screws, nails, etc.) with their respective potential defects, or tasks (e.g., visual inspection tasks such as visually inspecting solder application, visually inspecting screw driving, etc.) with their respective potential defects, according to a desired implementation.

[0051] As described herein, each of the multiple backbone snapshots can be configured with a squeeze layer, an excitation layer, and an anti-aliasing filter to facilitate cross-domain visual inspection. The squeezed excitation layer enables the backbone snapshot to extract more significant features across all channels of the input, and the anti-aliasing filter enables the backbone snapshot to mitigate sampling distortions when extracting features from an image. Combining these two techniques enables the backbone snapshot to extract more general and less noisy features, thereby improving overall performance across different domains of visual inspection tasks.

[0052] As described herein in Figures 4 and 5, a new domain task can be associated with a first plurality of classes for the task, and the plurality of labeled images includes examples for each of the plurality of classes. The classes can have different defects, different subtasks, etc., depending on the desired implementation.

[0053] As described above, the present embodiment can provide a system and method for creating models that perform visual inspection across multiple domains of input data.

[0054] Additionally, the present embodiment is computationally efficient (e.g., on the order of a few seconds) in adapting to new visual inspection domains, and the exemplary implementation can be directly applied to new visual inspection tasks without costly modifications to the model architecture.

[0055] According to this embodiment, cross-domain visual inspection adaptability can be further included by using a squeeze excitation module and anti-aliasing filters within the backbone architecture.

[0056] According to this embodiment, we use a combination of classification and contrastive losses in the meta-learning step to improve cross-domain adaptation in visual testing.

[0057] This implementation also leverages a snapshot ensembling scheme to obtain multiple backbone versions, resulting in better generalization across the visual inspection domain.

[0058] This example can be generalized to directly adapt to any n-way k-shot visual inspection task, where N-way denotes the number of classes in the new visual inspection task (n>=2), and K-shot denotes the number of labeled examples per class provided to the model for adaptation.

[0059] This embodiment also results in an ensemble of trained models that is computationally efficient to deploy with limited computational resources.

[0060] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing arts to convey the essence of their innovations to others skilled in the art. An algorithm is a sequence of defined steps leading to a desired end state or result. In the examples, the steps performed require measurable quantities of physical manipulations to achieve a measurable result.

[0061] Unless otherwise specifically stated, as will be apparent from the discussion, it will be understood that throughout this specification discussions using terms such as "processing," "computing," "calculating," "determining," "displaying," and the like may include the operations and processing of a computer system or other information processing device that manipulates and transforms data represented as physical (electronic) quantities in the registers and memory of the computer system into other data similarly represented as physical quantities in the memory or registers of the computer system or other information storage, transmission, or display device.

[0062] Exemplary embodiments also relate to apparatuses for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored on computer-readable media, such as computer-readable storage media or computer-readable signal media. Computer-readable storage media include tangible media, such as, but not limited to, optical disks, magnetic disks, read-only memory, random-access memory, solid-state devices and drives, as well as other types of tangible or non-transitory media suitable for storing electronic information. Computer-readable signal media may include media such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. A computer program may include a pure software implementation containing instructions for performing the operations of a desired implementation.

[0063] Various general-purpose systems may be used with the programs and modules according to the embodiments herein, or it may prove convenient to construct more specialized apparatus to perform the desired method steps. Moreover, the embodiments are not described with reference to a particular programming language. It will be understood that a variety of programming languages ​​may be used to implement the techniques of the embodiments described herein. Instructions in the programming language may be executed by one or more processing units, such as a central processing unit (CPU), processor, or controller.

[0064] As is known in the art, the operations described above can be performed by hardware, software, or some combination of software and hardware. Various aspects of the exemplary implementations may be implemented using circuits and logic devices (hardware), while other aspects may be implemented using instructions stored on a machine-readable medium (software) that, when executed by a processor, cause the processor to perform methods that implement the present invention. Furthermore, some exemplary implementations of the present application may be performed solely in hardware, while other exemplary implementations may be performed solely in software. Furthermore, the various functions described may be performed in a single unit or may be spread across multiple components in any number of ways. When performed by software, the methods may be executed by a processor, such as a general-purpose computer, based on instructions stored on a computer-readable medium. If desired, the instructions may be stored on the medium in a compressed and / or encrypted format. Additionally, other embodiments of the present invention will be apparent to those skilled in the art from consideration of this specification and practice of the techniques of the present invention. Various aspects and / or components of the described exemplary embodiments may be used alone or in any combination. It is intended that the specification and exemplary embodiments be considered exemplary only, with the true scope and spirit of the present application being indicated by the following claims. [Explanation of symbols]

[0065] 101: Meta-learning, 102: Backbone, 103: Meta-testing phase, 110: Classification loss, 111: Contrast loss 800: Computing Environment, 805: Computer equipment, 810: processor, 815: Memory, 820: Internal storage, 825:IO interface, 830: Bus, 835: Input / User Interface, 840: Output device / interface, 845: External storage, 850: Network, 860: logical unit, 865: Application Programming Interface (API) Unit 870: Input unit, 875: Output unit, 895: Inter-unit communication mechanism.

Claims

1. 1. A method for a learning system for a new visual inspection task using a few-shot meta-learning method, comprising: For an input of a first plurality of labeled images of a new domain task, process the first plurality of labeled images through a plurality of backbone snapshots, each of the backbone snapshots representing a model trained across a plurality of other domain tasks, each of the plurality of backbone snapshots outputting a first plurality of features in response to the input; processing a second input of a second plurality of unlabeled images via the plurality of backbone snapshots to output a second plurality of features responsive to the second input; performing a feature transformation process configured to transform the first plurality of features and the second plurality of features into a similarity value relating the first plurality of features to the second plurality of features based on similarity; performing a clustering process for clustering the first plurality of features and the second plurality of features based on the similarity value; Generate a representative model of the new domain task from the clustering and transformation of the first plurality of features and from the clustered and transformed second plurality of features. Methods in learning systems.

2. A method in a learning system according to claim 1, comprising: The backbone snapshots are combined with a multi-layer perceptron (MLP) for meta-learning. Methods in learning systems.

3. A method in a learning system according to claim 2, comprising: The backbone snapshots are meta-learned based on classification loss and contrastive loss. Methods in learning systems.

4. A method in a learning system according to claim 1, comprising: The new domain task is visual inspection, the labeled images include defect types of the visual inspection object or task, and the representative model is configured to output defects detected in the visual inspection. Methods in learning systems.

5. A method in a learning system according to claim 1, comprising: The features generated from the backbone snapshots are representative of an industrial task or object. Methods in learning systems.

6. A method in a learning system according to claim 1, comprising: each of the plurality of backbone snapshots is constructed with squeeze and excite layers and anti-aliasing filters to facilitate cross-domain visual inspection; Methods in learning systems.

7. A method in a learning system according to claim 1, comprising: the new domain task is associated with a first plurality of classes for the domain task, and the plurality of labeled images is composed of examples for each of the plurality of classes. Methods in learning systems.

8. A non-transitory computer readable medium having stored thereon instructions for performing a process, the instructions for computer execution including: For an input of a first plurality of labeled images of a new domain task, process the first plurality of labeled images through a plurality of backbone snapshots, each of the backbone snapshots representing a model trained across a plurality of other domain tasks, each of the plurality of backbone snapshots outputting a first plurality of features in response to the input; processing a second input of a second plurality of unlabeled images via the plurality of backbone snapshots to output a second plurality of features responsive to the second input; performing a feature transformation process configured to transform the first plurality of features and the second plurality of features into a similarity value relating the first plurality of features to the second plurality of features based on similarity; performing a clustering process for clustering the first plurality of features and the second plurality of features based on the similarity value; Generate a representative model of the new domain task from the clustering and transformation of the first plurality of features and from the clustered and transformed second plurality of features. A computer-readable medium that stores instructions for execution by a computer.

9. 9. The computer-readable medium of claim 8, The instructions executed by the computer include: The backbone snapshots are combined with a multi-layer perceptron (MLP) for meta-learning. A computer-readable medium that stores instructions for execution by a computer.

10. 10. The computer readable medium of claim 9, The instructions executed by the computer include: meta-learning the backbone snapshots based on classification loss and contrastive loss; A computer-readable medium that stores instructions for execution by a computer.

11. 9. The computer-readable medium of claim 8, the new domain task is visual inspection, the labeled images include defect types of the visual inspection object or task, and the computer-executable instructions are configured to cause the representative model to output defects detected during the visual inspection. A computer-readable medium that stores instructions for execution by a computer.

12. 9. The computer-readable medium of claim 8, The features generated from the backbone snapshots are representative of an industrial task or object. A computer-readable medium that stores instructions for execution by a computer.

13. 9. The computer-readable medium of claim 8, The computer-implemented instructions configure each of the plurality of backbone snapshots with squeeze and excitation layers and anti-aliasing filters to facilitate cross-domain visual inspection. A computer-readable medium that stores instructions for execution by a computer.

14. 9. The computer-readable medium of claim 8, A computer-readable medium having computer-executable instructions stored thereon, the computer-executable instructions associating the new domain task with a first plurality of classes for the task and causing the plurality of labeled images to be constructed from examples for each of the plurality of classes.

15. 1. A computing device having a processor, comprising: The processor, For an input of a first plurality of labeled images of a new domain task, process the first plurality of labeled images through a plurality of backbone snapshots, each of the backbone snapshots representing a model trained across a plurality of other domain tasks, each of the plurality of backbone snapshots outputting a first plurality of features responsive to the input; processing a second input of a second plurality of unlabeled images via the plurality of backbone snapshots to output a second plurality of features responsive to the second input; performing a feature transformation process configured to transform the first plurality of features and the second plurality of features into a similarity value relating the first plurality of features to the second plurality of features based on similarity; performing a clustering process for clustering the first plurality of features and the second plurality of features based on the similarity value; Generate a representative model of the new domain task from the clustering and transformation of the first plurality of features and from the clustered and transformed second plurality of features. Computer equipment.

Images