Systems and methods for training a model using a small number of classification processes

Abstract

A method of providing a model for classification of data elements is provided, including: feeding a plurality of labeled auxiliary data elements and at least one labeled related data element for each of a plurality of related classification categories into a synthesizer component for output of at least one synthesized labeled related data element for each of the plurality of related classification categories, feeding the synthesized labeled related data elements and a plurality of unlabeled training related data elements to a domain adaptation module for output of a respective related classification category for each of the plurality of unlabeled training related data elements, iteratively training the synthesis component and the domain adaptation component end-to-end, and providing the trained domain adaptation component for output of a related classification category in response to an input of a query unlabeled data element.

Description

TECHNICAL FIELD

[0001] The present invention, in some embodiments thereof, relates to machine learning and, more particularly, but not exclusively, to systems and methods for training a model using a few-shot classification process. BACKGROUND

[0002] A few-shot classification process trains a model using a few labeled training data elements to classify previously unseen data elements (e.g., images). SUMMARY

[0003] According to a first aspect, a method of computing a model for classification of data elements, comprises: feeding a plurality of labeled auxiliary data elements and at least one labeled related data element for each of a plurality of related classification categories into a synthesizer component for outputting at least one synthesized labeled related data element for each of the plurality of related classification categories, feeding the synthesized labeled related data elements and a plurality of unlabeled training related data elements into a domain adaptation module for outputting a respective related classification category for each of the plurality of unlabeled training related data elements, iteratively training the synthesizer component and the domain adaptation component end-to-end, and providing the trained domain adaptation component for outputting a related classification category in response to an input of a query unlabeled data element.

[0004] According to a second aspect, a method of classifying a query data element into a classification category by a model, comprises: feeding the query data element into a domain adaptation component trained by: feeding a plurality of labeled auxiliary data elements and at least one labeled related data element for each of a plurality of related classification categories into a synthesizer component for outputting at least one synthesized labeled related data element for each of the plurality of related classification categories, feeding the synthesized labeled related data elements and a plurality of unlabeled training related data elements into a domain adaptation module for outputting a respective related classification category for each of the plurality of unlabeled training related data elements, and iteratively training the synthesizer component and the domain adaptation component end-to-end, and outputting by the trained domain adaptation component a related classification category for the query data element.

[0005] According to a third aspect, a system for computing a neural network model for data element classification comprises at least one hardware processor executing code for: feeding a plurality of labeled auxiliary data elements and at least one labeled related data element for each of a plurality of related classification categories into a synthesizer component for outputting at least one synthesized labeled related data element for each of the plurality of related classification categories, feeding the synthesized labeled related data elements and a plurality of unlabeled training related data elements into a domain adaptation component for outputting a respective related classification category for each of the plurality of unlabeled training related data elements, iteratively training the synthesizer component and the domain adaptation component end-to-end, and providing the trained domain adaptation component for outputting a related classification category in response to an input of a query unlabeled data element.

[0006] In another implementation of the first, second, and third aspects, the number of at least one labeled related data element is 1-5 per related classification category.

[0007] In another implementation of the first, second, and third aspects, further comprising creating an initial model, wherein the model for data element classification is trained using the initial model, the initial model trained by: receiving the plurality of labeled auxiliary data elements representing each of a plurality of auxiliary classification categories, feeding the plurality of labeled auxiliary data elements into the synthesizer component of the model for outputting a plurality of synthesized labeled auxiliary data elements for each of the plurality of auxiliary classification categories, feeding the synthesized labeled related data elements into a source input of a domain adaptation component of the model, and feeding a further plurality of unlabeled auxiliary data elements into a target input of the domain adaptation component of the model for outputting a certain auxiliary classification category selected from the plurality of auxiliary classification categories for each of the plurality of unlabeled auxiliary data elements, and iteratively training the synthesizer component and the domain adaptation component end-to-end to create the initial model.

[0008] In another implementation of the first, second, and third aspects, the synthesizer component is implemented as an extension of an incremental encoder component of the model, the incremental encoder component trained to output a vector representation of the at least one synthesized labeled related data element for each of the plurality of related classification categories in response to an input of the plurality of labeled auxiliary data elements and the at least one labeled related data element.

[0009] In another embodiment of the first, second, and third aspects, further comprising training, during the creation of the initial model, an incremental-impersonator component of the incremental encoder and an incremental-discriminator component of the incremental encoder responsive to input of the at least one labeled relevant data element without the plurality of labeled ancillary data elements, simulating the incremental encoder components to output pseudo vector representations, wherein the model is trained from the initial model by feeding the at least one labeled relevant data element into the incremental-impersonator and incremental-discriminator components of the incremental encoder without feeding the plurality of labeled ancillary data elements.

[0010] In another implementation of the first, second, and third aspects, each of the plurality of labeled ancillary data elements is labeled with a respective ancillary classification category selected from a plurality of ancillary classification categories, wherein the plurality of ancillary classification categories are different from the plurality of relevant classification categories.

[0011] In another implementation of the first, second, and third aspects, the data elements are images depicting visual objects, wherein the classification categories are visual classification categories.

[0012] In another implementation of the first, second, and third aspects, the domain adaptation component computes an adjustment to a distribution of the synthetic labeled relevant data elements to correlate with a distribution of the plurality of unlabeled training relevant data elements, wherein a certain relevant classification category selected for each of the plurality of unlabeled training relevant data elements is computed based on the adjusted synthetic labeled relevant data elements.

[0013] In another implementation of the first, second, and third aspects, the synthesizer component receives a certain labeled ancillary data element and a corresponding labeled relevant data element for output of a synthesized labeled relevant data element.

[0014] In another implementation of the first, second, and third aspects, a number of labeled ancillary data elements for each of the plurality of ancillary classification categories is sufficient to train a standard classifier for accurately classifying a query data element into one of the plurality of ancillary classification categories.

[0015] In another implementation of the first, second, and third aspects, the synthesizer component and the domain adaptation component are iteratively trained end-to-end to minimize an objective function of the domain adaptation component for input of the synthetic labeled relevant data elements output by the synthesizer component and the plurality of unlabeled training relevant data elements.

[0016] In a further implementation of the first, second and third aspects, further comprising mapping each of the labeled auxiliary data elements and the at least one labeled related data element into a respective vector in a multi-dimensional space, wherein the feeding comprises feeding the mapped respective vectors into the synthesizer component, wherein the distribution of the at least one synthesized labeled related data element for each of the plurality of related classification categories output by the synthesizer component is related to a distribution of the mapped vectors of the labeled auxiliary data elements in the multi-dimensional space centered at the vector of the at least one labeled related data element.

[0017] In a further implementation of the first, second and third aspects, the respective vectors are obtained from non-classification layers of a neural network trained for classifying labeled auxiliary data elements into corresponding labels selected from a plurality of auxiliary classification categories.

[0018] In a further implementation of the first, second and third aspects, each of the synthesizer component and the domain adaptation component are implemented as neural networks.

[0019] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application pertains. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of embodiments of the application, exemplary methods and / or materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and are not intended to be necessarily limiting. BRIEF DESCRIPTION OF DRAWINGS

[0020] Some embodiments of the application are herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in

[0021] In the drawings:

[0022] FIG. 1A is a high level flowchart of a method of computing a model for classification of data elements using a small amount of labeled training related data elements according to some embodiments of the application;

[0023] FIG. 1B is a flowchart of an exemplary method of computing an initial state of a model according to some embodiments of the application;

[0024] FIG. 1Cis a flowchart of an exemplary method for computing a model for classifying query data elements using an initial model according to some embodiments of the present application;

[0025] FIG. 2 is a block diagram of components of a system for training a model using a small amount of labeled training related data elements and / or for classifying data elements using the model according to some embodiments of the present application, the model trained using the small amount of labeled training related data elements;

[0026] FIG. 3 is a flowchart of a method for classifying data elements using a model trained using a small amount of labeled training related data elements according to some embodiments of the present application;

[0027] FIG. 4 is a dataflow diagram of an exemplary process and / or architecture implementation of an incremental encoder according to some embodiments of the present application; and

[0028] FIG. 5 is a dataflow diagram depicting an exemplary process for training a model using an initial model according to some embodiments of the present application. DETAILED DESCRIPTION

[0029] The present application, in some embodiments thereof, relates to machine learning and, more specifically, but not exclusively, to systems and methods for training a model using a small amount of classification process.

[0030] Aspects of some embodiments of the present application relate to systems, methods, apparatuses and / or code instructions for computing a model for classification of data elements using a small amount of classification process. A plurality of labeled helper data elements representing a plurality of helper classification categories and one or more labeled related data elements for each related classification category are fed into a synthesizer component of the model. As described herein, the small amount of classification process is based on the number of labeled related data elements (per related classification category) being a small amount (e.g., 1-5, 1-10) or being otherwise considered a small amount (e.g., as described herein, insufficient to train a standard classifier using standard methods for accurate classification). The synthesizer component outputs a synthesized labeled related data element for each related classification category. The synthesized labeled related data element and a plurality of unlabeled training related data elements are fed into a domain adaptation component of the model for outputting a respective related classification category for each unlabeled training related data element. The synthesizer component and the domain adaptation component are trained iteratively end-to-end. The trained domain adaptation component is provided for outputting a related classification category in response to an input of a query unlabeled data element.

[0031] Optionally, an initial version of the computational model is computed. The initial model is used to train a model for classification of data elements. The initial model is trained by feeding the labeled auxiliary data elements into a synthesizer component of the model for output of synthesized labeled auxiliary data elements for each auxiliary classification category. The synthesized labeled related data elements are fed into a source input of a domain adaptation component. Other unlabeled auxiliary data elements are fed into a target input of the domain adaptation component for output of a certain auxiliary classification category selected from the auxiliary classification categories for each unlabeled auxiliary data element. The synthesizer component and the domain adaptation component are trained end-to-end to create the initial model.

[0032] Aspects of some embodiments of the present invention relate to systems, methods, apparatuses, and / or code instructions for classifying a query data element into a classification category using a model trained using a small amount of classification process. The query data element is fed into a trained domain adaptation component of the trained model. As described herein, the model is trained using labeled auxiliary data elements (many of them) for each auxiliary classification category and one or more (i.e., a small amount) of labeled related data elements for each related classification category. The trained domain adaptation component outputs the related classification category of the query data element.

[0033] At least some implementations of the systems, methods, apparatuses, and / or code instructions described herein address the technical problem of improving the process of training a new data element classification model using a semi-supervised few-shot classification process, e.g., improving the accuracy of the classification output.

[0034] At least some implementations of the systems, methods, apparatuses, and / or code instructions described herein improve the technical field of machine learning (particularly neural network models) by improving the process of training a neural network-based model using a semi-supervised few-shot classification process, e.g., improving the accuracy of the classification output.

[0035] Standard fully supervised processes for model training to accurately classify data elements (e.g., neural network-based classifiers) into one or more visual categories require a large set of labeled training data elements that provide multiple samples of each possible visual category. Labeling is done manually by humans. Technical problems arise when such labeled data elements are difficult to obtain, e.g., limited human resources available to label data elements and / or high cost of labeling data elements. For example, obtaining medical images with manual labeling is difficult because labeling is done by expert radiologists. Models trained using standard methods with a limited size set of labeled images have poor performance. At least some implementations of the systems, methods, apparatuses, and / or code instructions described herein use a limited set of labeled data elements (which can be relatively easy to generate because of the small size of the set) to generate high performance classifiers in conjunction with an easily and / or widely available set of unlabeled data elements obtained, e.g., from publicly available sources such as a database of data elements.

[0036] The domain adaptation components described herein are trained differently even when based on standard practice architectures. Using standard training methods, a domain adaptation component receives labeled source data and unlabeled target data assumed to belong to the same set of classes as the source data. The training involves learning a classifier of the target samples that assigns labels (from the set of source labels) to these samples. In standard operating mode, the domain adaptation component is trained in isolation, provided with a fixed set of source labeled samples and a set of target samples. Specifically, the domain adaptation component is trained (and evaluated) on exactly one set of classes. In contrast, as described herein, the training of the domain adaptation component is performed differently. Specifically, first, the input source samples are not fixed elements of a given dataset. The domain adaptation component is trained as part of a larger network that includes a sample synthesizer described herein. The output of the synthesizer (which produces synthetic examples) is fed into the source input of the domain adaptation component and facilitates joint training of the synthesizer and the domain adaptation component through the objective function of the domain component for the input configuration ({synthetic sample, label}, {target sample}) described herein. Second, there can be two training phases corresponding to (1) pre-training (bootstrap) the model on auxiliary classes in order to implement a few-shot learning mechanism and (2) the need to train the model using few novel classes. As a result, the domain adaptation component functions in a larger model where both source and target data come from different sets of classes in the two training phases.

[0037] The synthetic processes described herein, in which a synthesizer component computes synthetic data elements, are different from other standard methods for generating additional samples of a class based on some sample of data elements. Standard methods are described, for example, by Swami Sankaranarayanan, Yogesh Balaji, Arpit Jain, Ser Nam Lim, Rama Chellappa, Learning from Synthetic Data: Addressing Domain Transfer for Semantic Segmentation, CVPR 2018; and Atapour-Abarghuei, A. and Breckon, T. P. (2018) ‘Real-time monocular depth estimation using synthetic data with domain adaptation.’, CVPR 2018. The synthesizer components described herein can be generated as out-of-class sample augmentation data for a relevant class, rather than manually designed graphics as done by standard methods. The synthesizer components described herein do not use a generative model (i.e., an explicit prescription of how to produce examples), which is in contrast to standard methods that use a defined model to produce graphically generated data elements of samples (e.g., rendered images of objects). Furthermore, the synthesizer components are trained end-to-end in conjunction with the domain adaptation components described herein, enabling effective utilization of unlabelled data, which is not present in standard methods. In another example, Yu-Xiong Wang, Ross Girshick, Martial Hebert, Bharath Hariharan, Low-shot Learning From Image Data, arXiv: 1801.05401 does not use unlabelled data elements, in contrast to the synthesizer components described herein that use unlabelled data elements. In yet another example, Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Rogerio Feris, Abhishek Kumar, Raja Giryes, Alex M. Bronstein, ‘Incremental Encoder: An Efficient Sample Synthesis Method for Few-Shot Object Recognition’, NIPS 2018 involves an architecture that requires external sampling of deformations from pairs of training examples, which is a different approach than the systems, methods, and / or apparatus described herein that use a generative model that relies on anchor inputs to sample deformations. Reference is made below to FIG. 4 Additional features of the incremental encoder implementation of the synthesizer component compared to standard implementations of the incremental encoder are described.

[0038] At least some of the systems, methods, apparatuses, and / or code instructions described herein provide additional improvements over standard methods including, for example, joint end-to-end training of a synthesizer component and a domain adaptation component, and / or use of a domain adaptation component with synthetic data elements.

[0039] Before one or more embodiments of the application are explained in detail, it is to be understood that the application is not limited in its application to the details of construction and the arrangement of the components and / or methods set forth in the following description and / or illustrated in the drawings and / or the Examples. The application is capable of other embodiments or of being practiced or carried out in various ways.

[0040] The application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0041] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non- exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0042] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0043] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0044] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0045] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0046] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0047] The flow and block diagrams in the drawings show the architectural, functional and operational concepts of possible implementations of systems, methods and computer program products according to different embodiments of the present application. In this regard, each block in the flow or block diagrams can represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0048] Reference will now be made to FIGS. 1A-1C , FIGS. 1A-1C is a flowchart of a method of training a model using a small amount of labeled training-related data elements according to some embodiments of the present application. Reference is also made to FIG. 2 , FIG. 2 is a block diagram of components of a system 200 for training a model using a small amount of labeled training-related data elements and / or for classifying data elements using a model trained with a small amount of labeled training-related data elements according to some embodiments of the present application. Reference is also made to FIG. 3 which is a flowchart of a method of classifying data elements using a model trained with a small amount of labeled training-related data elements according to some embodiments of the present application. The system 200 can be implemented by a processor 202 of a computing device 204 executing code instructions (e.g., code 206A and / or) stored in a memory 206 (also referred to as program storage) of the computing device 204, with reference to FIGS. 1A-1C and / or FIG. 3 the actions of the method described.

[0049] The computing device 204 can be implemented as, for example, one or a combination of the following: a set of connected devices, a client terminal, a server, a virtual server, a computing cloud, a virtual machine, a smart television, a desktop computer, a thin client, a network node, a network server, and / or a mobile device (e.g., a smart phone, a tablet computer, a laptop computer, a wearable computer, a glasses computer, and a watch computer).

[0050] Different architectures of the system 200 can be implemented, for example:

[0051] The computing device 204 can be implemented as one or more servers (e.g., web servers, network servers, computing clouds, virtual servers, network nodes, video storage servers, video editing stations) that provide services to a plurality of client terminals 210 over a network 212, e.g., receiving a small amount of labeled training-related data elements 256 and creating a trained model 208A. The labeled auxiliary training data elements 250 and / or the unlabeled training-related data elements 252 can be obtained, e.g., from the client terminals 210, from the servers 216, and / or from local storage associated with the computing device 204. Each respective trained model 208A can be provided back to each respective client terminal 210. Alternatively or in addition, the trained model 208A can be hosted by the computing device 204 for providing remote data element classification services to the client terminal(s) 210.

[0052] Communications between the client terminals 210 and / or the servers 216 and / or the computing device 204 over the network 212 can be implemented, e.g., via an application programming interface (API), a software development kit (SDK), an add-on to an existing application executing on the client terminal(s), a library and / or an additional component that adds functionality to an existing application executing on the client terminal(s), an application for download and execution on the client terminal 210 in communication with the computing device 204, a function and / or interface call to code executed by the computing device 204, a remote access session executing on a website hosted by the computing device 204 accessed via a web browser executing on the client terminal(s) 210.

[0053] The computing device 204 can be implemented as a standalone device (e.g., a client terminal, a smartphone, a smart television, a computing cloud, a virtual machine, a kiosk) that includes implementing the references FIGS. 1A-1C and / or FIG. 3 The local storage of one or more actions described. For example, the computing device 204 trains its own model (i.e., generates a trained model 208A) using its own acquired small amount of labeled training-related data elements 256 and / or labeled auxiliary training data elements 250 and / or unlabeled training-related data elements 252 (which, as described herein, can be obtained from different sources) for its own data element classification requirements.

[0054] The computing device 204 can train a model (i.e., generate a trained model 208A) using a small amount of labeled training related data elements 256 that it acquired itself and / or labeled auxiliary training data elements 250 and / or unlabeled training related data elements 252 (which can be obtained from different sources as described herein). A data element classification service using the trained model 208A can be provided to the client terminal(s) 210. For example, the client terminal(s) 210 provide query data elements 254 to the computing device 204 over the network 212. The computing device 204 feeds the query data elements 254 into the trained model 208A to obtain a classification category. The classification category is provided back to the respective query client terminal 210.

[0055] The hardware processor 202 of the computing device 204 can be implemented, for example, as a central processing unit (CPU), a graphics processing unit (GPU), a field- programmable gate array (FPGA), a digital signal processor (DSP), and an application- specific integrated circuit (ASIC). The processor 202 can include a single processor or multiple processors (homogeneous or heterogeneous) arranged for parallel processing, as a cluster, and / or as one or more multi-core processing devices.

[0056] The memory 206 stores code instructions executable by the hardware processor 202, for example, random access memory (RAM), read-only memory (ROM), and / or storage devices, for example, non-volatile memory, magnetic media, semiconductor memory devices, hard disk drives, removable storage, and optical media (e.g., DVD, CD-ROM). The memory 206 stores code 206A that, when executed by the hardware processor 202, implements one or more features and / or actions of the described methods. For example, the memory 206 can store the trained model 208A when feeding the query data elements 254 into the trained model 208A to obtain a classification category. As described herein, the model 208A can include a synthesizer component 208A-1 and / or a domain adaptation component 208A-2. Note that the trained model 208A can include the domain adaptation component 208A-2 without the synthesizer component 208A-1, for example, when the synthesizer component 208A-1 is used during training for end-to-end training with the domain adaptation component 208A, and the classification of query data elements can be performed by the trained domain adaptation component 208A without the synthesizer component 208A-1, as described herein. Note that the synthesizer component 208A-1 and / or the domain adaptation component 208A-2 can be included in the model training code 206A. FIGS. 1A-1C

[0057] ​The computing device 204 can include data storage device(s) 208 for storing data (e.g., storing the trained model 208A, the few labeled training-related data elements 256, and / or the labeled auxiliary training data elements 250 and / or the unlabeled training-related data elements 252). The data storage device 208 can be implemented, for example, as a memory, a local hard drive, a virtual storage, a removable storage unit, an optical disc, a storage device, and / or a remote server and / or a computing cloud (e.g., accessed using a network connection).

[0058] The network 212 can be implemented, for example, as the Internet, a local area network, a virtual network, a wireless network, a cellular network, a local bus, a point-to-point link (e.g., wired), and / or a combination of the above.

[0059] The computing device 204 can include a network interface 218 for connecting to the network 212, for example, a network interface card, an antenna, a wireless interface for connecting to a wireless network, a physical interface for connecting to a cable for network connection, a software-implemented virtual interface, network communication software providing a higher layer of network connection, and / or one or more of the other implementations.

[0060] The computing device 204 and / or the client terminal(s) 210 include and / or are in communication with one or more physical user interfaces 214, which include mechanisms for user interaction, for example, for inputting data (e.g., selecting query data elements, assigning labels to create labeled data elements, selecting training data elements) and / or viewing data (e.g., viewing the classification category of a query data element output by a model).

[0061] The exemplary physical user interfaces 214 include, for example, one or more of a touchscreen, a display, a gesture-activated device, a keyboard, a mouse, and voice-activated software using a speaker and a microphone.

[0062] The client terminal(s) 210 and / or the server(s) 216 can be implemented, for example, as a desktop computer, a server, a smart television, a virtual server, a web server, a data element storage server, a cloud storage, a web server, a virtual machine, a thin client, and a mobile device.

[0063] Reference is now made to FIG. 1A , FIG. 1A is a high-level flowchart of a method of computing a model for classification of data elements using few labeled training-related data elements in accordance with some embodiments of the application.

[0064] At 102, a dataset of data elements for training a model is provided and / or specified and / or accessed. The data elements are each associated with a classification category, e.g., labeled and / or annotated with a respective classification category. The classification categories can be semantic categories.

[0065] Optionally, the data elements are images depicting visual objects. Other examples of data elements include documents, text, code, audio (e.g., music) files, and numbers. The classification categories are visual classification categories, optionally labels of the depicted visual objects. For example, the visual classification categories are indications of which animal (e.g., cat, dog, horse, and lion) is depicted in the images.

[0066] The dataset for training the model includes relevant data elements and auxiliary data elements. The relevant data elements and the auxiliary data elements are within the same classification domain, where the relevant data elements and the auxiliary data elements have different (i.e., non-overlapping) respective classification categories. For example, the relevant data elements and the auxiliary data elements (e.g., images) are within a classification domain of animals. The relevant data elements include a classification category of lions, and the auxiliary data elements include classification categories of dogs, cats, and horses.

[0067] Due to, e.g., difficulty in obtaining relevant data elements and / or labels, the number of relevant data elements with associated labels can be small. For example, it can be difficult to obtain images of lions in the wild, while it can be easy to obtain images of dogs, cats, and horses, as these animals are domesticated. In another example, it can be difficult to obtain certain medical images with associated medical findings, e.g., for rare medical conditions and / or when radiologists have difficulty accessing to assign labels to the images.

[0068] A standard classifier (e.g., a standard neural network architecture) trained using only the labeled relevant data elements is statistically inaccurate for classifying a query data element into one of the relevant classification categories. The small number of samples of the labeled relevant data elements is insufficient to train the standard classifier to obtain sufficient classification accuracy. The number of relevant data elements with associated labels is small, e.g., 1-5, or about 1-10, or about 1-3, or about 3-5, or about 3-10, or other ranges and / or numbers. Note that the number of relevant data elements considered small can depend on the number of data elements needed to obtain statistical precision of the trained standard classifier. The number represented by the small number is lower than the number needed to obtain the desired precision by the standard classifier.

[0069] For example, the number of labeled auxiliary data elements can be large due to, e.g., general ease of access and / or availability. For example, the labeled auxiliary data elements can be obtained from a large publicly accessible database (e.g., a database of labeled images used to train a standard neural network) and / or can be readily created. For example, in the example discussed above, images of dogs, cats, and horses can be readily obtained.

[0070] The number (also referred to herein as a quantity) of labeled auxiliary data elements per auxiliary classification category can be sufficient to train a standard classifier for accurately classifying a query data element into one of the auxiliary classification categories. For example, the number of labeled auxiliary data elements is, e.g., more than 100, or more than 1000, or more than 10,000, or more than 100,000, or 100-10,000, or 1000-100,000, or other ranges and / or values per auxiliary classification category. Large quantities of images can be obtained, e.g., as Imagenet categories (e.g., many animals, plants, and human objects).

[0071] Each of the auxiliary data elements is labeled with a respective auxiliary classification category selected from the plurality of auxiliary classification categories. The auxiliary classification categories are different from (i.e., unique, non-overlapping with) the relevant classification categories.

[0072] In example implementations, the labeled relevant data elements are provided by a user (e.g., via a client terminal of the user), e.g., the user takes a picture and enters a label. The auxiliary data elements can be obtained from a public source, such as a publicly accessible database. The public source can be automatically determined by the user and / or manually specified.

[0073] At 104, an initial state of the model can be computed. The initial state of the model is obtained by training with only the labeled auxiliary data elements, without the relevant data elements.

[0074] Alternatively, in some embodiments, the initial state of the model is not computed, and the process proceeds to 106.

[0075] Reference is now made to FIG. 1B , FIG. 1B is a flowchart of an example method of computing an initial state of a model (referred to herein as an initial model) in accordance with some embodiments of the invention.

[0076] At 150, labeled auxiliary data elements representing each of one or more (e.g., a plurality of) auxiliary classification categories are received and / or specified and / or accessed, as described herein.

[0077] At 152, the labeled auxiliary data elements are fed into a synthesizer component of the model.

[0078] Optionally, the synthesizer component is implemented as a neural network, optionally a deep neural network.

[0079] Optionally, the synthesizer component is implemented as an extension of the incremental encoder component of the model. The data encoder is trained to output, responsive to input of the labeled auxiliary data elements and the labeled related data elements, a vector representation of the synthesized labeled related data elements for each of the related classification categories.

[0080] Optionally, during creation of the initial model, the incremental forger component of the incremental encoder is trained together with the incremental discriminator component of the incremental encoder to mimic the incremental encoder component to output a fake vector representation responsive to input of the labeled related data elements without the labeled auxiliary data elements.

[0081] Reference is now made to FIG. 4 , FIG. 4 is a dataflow graph 402 of an exemplary process and / or architecture implementation of an incremental encoder according to some embodiments of the application.

[0082] The incremental encoder for the synthesizer component can be an extension of additional functionality added to a standard incremental encoder, reference Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Rogerio Feris, Abhishek Kumar, Raja Giryes, and Alex M. Bronstein, ‘Incremental encoder: An efficient sample synthesis method for few-shot object recognition’, NeurIPS 2018, Montreal, at least one of the inventors of which is the same as the present application, and which is incorporated herein in its entirety by reference. Using the standard incremental encoder process, the auxiliary data elements are used both for training the initial model when the related categories are not used, and for the phase of training the model from the initial model, as described herein, for example with reference to FIG. 1C To train the model from the initial model (for example, as described with reference to FIG. 1C ), the input to the incremental encoder is a pair of data elements represented as (X, Y), where X represents some auxiliary category and Y represents some related category. The implementation of the incremental encoder described herein (for example, as depicted in FIG. 4 ) includes additional features designed to remove the dependency on auxiliary data defined by the standard implementation of the incremental encoder and allow the use of only related data elements during training of the model from the initial model.

[0083] In the extended architecture of the model 402, the incremental forger subnetwork 404 is trained together with the incremental-discriminator subnetwork 406 during training of the initial model (for example, as described with reference toFIG. 1C The described). While the delta-encoder subnetwork 408 learns to produce a real intermediate representation vector z 410, the delta-impostor subnetwork 404 learns to mimic its behavior and produce a fake vector z 412, not from (X, Y) but from Y alone. Then, at the model training phase (from the initial model, as referenced FIG. 1C The necessity of providing permanent samples X is removed.

[0084] Note that a standard implementation of the delta-encoder process synthesizes data elements from input data elements. The implementation of the delta-encoder described herein can be considered an extension of the standard delta-encoder implementation with the additional feature of synthesizing intermediate representation vectors z 410, which facilitates the synthesis of final output synthesized data elements.

[0085] Now returning reference to FIG. 1B At 154, the synthesizer component outputs a plurality of synthesized labeled auxiliary data elements for each auxiliary classification category.

[0086] At 156, the relevant data elements that are synthesized labeled are fed into the source input of the domain adaptation component of the model. Other unlabeled auxiliary data elements are fed into the target input of the domain adaptation component of the model.

[0087] Optionally, the domain adaptation component is implemented as a neural network, optionally a deep neural network.

[0088] The domain adaptation component can be implemented, for example, based on VADA and / or DIRT-T, as described in Rui Shu, Hung H. Bui, Hirokazu Narui, Stefano Ermon, DIRT-T: A Unsupervised Domain Adaptation Method, arXiv: 1802.08735, which is incorporated by reference herein in its entirety.

[0089] At 158, the domain adaptation component outputs a certain auxiliary classification category selected from a plurality of auxiliary classification categories for each unlabeled auxiliary data element.

[0090] At 160, the synthesizer component and the domain adaptation component are iteratively trained end-to-end to create an initial model.

[0091] At 162, the initial model is provided, for example, stored in a memory and / or data storage device and / or provided to another computing device and / or another process for further training (e.g., where the initial model is trained by one computing device and / or process and the model is trained using the initial model by another computing device and / or another process, or alternatively, the same computing device and / or the same process computes an implementation of the model from the initial model).

[0092] Now returning reference toFIG. 1A At 106, the model is trained.

[0093] Optionally, the model is trained using an initial model. The initial model can be used as a starting point. The initial model can be updated with additional training to create the model.

[0094] Reference is now made to FIG. 1C , FIG. 1C is a flowchart of an exemplary method of calculating a model for classifying query data elements using optionally an initial model, according to some embodiments of the application.

[0095] At 170, unlabeled training-related data elements, labeled auxiliary data elements, and one or more (i.e., several) labeled related data elements for each related classification category are received and / or specified and / or accessed, as described herein.

[0096] At 172, the labeled auxiliary data elements and the one or more (i.e., several) labeled related data elements for each related classification category are fed into a synthesizer component of the model.

[0097] The synthesizer component is configured to receive a certain labeled auxiliary data element and a corresponding labeled related data element, and output a synthesized labeled related data element.

[0098] Optionally, each of the labeled auxiliary data elements and the labeled related data elements are mapped into respective vectors in a multi-dimensional space (optionally, an embedding space). The mapped vectors are fed into the synthesizer component.

[0099] The respective vectors can be computed as embeddings of a neural network, optionally a convolutional neural network (CNN), e.g., obtained from a non-classification layer of a neural network trained for classifying labeled auxiliary data elements into corresponding labels selected from auxiliary classification categories. The non-classification layer can be the last layer before the classification layer. The last classification layer of the neural network can be removed, such that the neural network outputs vector representations.

[0100] At 174, the synthesizer component outputs one or more synthesized labeled related data elements for each related classification category.

[0101] Optionally, a distribution of the synthesized labeled related data elements for each related classification category output by the synthesizer component is related to a distribution of the mapped vectors of the labeled auxiliary data elements in the multi-dimensional space centered around the vector of the labeled related data elements.

[0102] When the distribution is low-dimensional, i.e., the distribution lies on a low-dimensional smooth manifold in the space, and has only a few modes (i.e., is well-composed), one can say that the set of vectors produced by the non-classification layer from data elements of a particular class are distributed in the embedding space in a "good" way.

[0103] During training, the synthesizer component learns intra-class variation from data elements of the same class, essentially learning the distribution of the auxiliary class.

[0104] During synthesis, a pair of data elements (one from the auxiliary class and one from the relevant class) are provided as input to the synthesizer component. A new sample from the relevant class is synthesized and output by the synthesizer component. When the synthesis is invoked with multiple samples of the same auxiliary class and one or several samples of the relevant class, the resulting distribution of synthesized data elements resembles (e.g., is statistically significantly correlated with) the distribution of the auxiliary class centered around the data elements of the relevant class.

[0105] At 176, the synthesized labeled relevant data elements and the unlabeled training relevant data elements are fed into a domain adaptation component of the model.

[0106] Optionally, the domain adaptation component computes an adjustment to the distribution of the synthesized labeled relevant data elements to correlate with the distribution of the unlabeled training relevant data elements. The certain relevant classification class selected for each unlabeled relevant data element is computed based on the adjusted synthesized labeled relevant data elements.

[0107] At 178, the domain adaptation component outputs the respective relevant classification class for each unlabeled training relevant data element.

[0108] At 180, the synthesizer component and the domain adaptation component are trained end-to-end iteratively. The end-to-end iterative training can be performed to minimize an objective function of the domain adaptation component for input of the synthesized labeled relevant data elements (i.e., output by the synthesizer component) and the unlabeled training relevant data elements. The objective function can be, for example, the DIRT-T method for unsupervised domain adaptation as described by Rui Shu, Hung H. Bui, Hirokazu Narui, Stefano Ermon, arXiv: 1802.08735, which is incorporated by reference herein in its entirety.

[0109] Reference is now made to FIG. 5 which is a dataflow diagram depicting an exemplary process for training a model using an initial model, in accordance with some embodiments of the present application. Reference is made to FIG. 5 The described dataflow can be based on and / or integrated with and / or replace reference FIGS. 1A-1C One or more features of the described method and / or by reference FIG. 2Components of the system 200 are described.

[0110] A few labeled related data elements 502 are fed into an updated delta encoder 504 as described with reference to FIG. 1B The updated delta encoder 504 includes delta forger and delta discriminator components as described herein. The updated delta encoder 504 outputs a number of synthetic labeled data element examples 506 without being fed labeled auxiliary data element examples 508. The number of synthetic labeled data element examples 506 and labeled auxiliary data element examples 508 are fed into a domain adaptation component 510 that computes a classifier 512 for classifying new unlabeled query data elements. The classifier 512 is a domain adaptation component 510 after end-to-end training with an iterated synthesizer component as described herein.

[0111] The domain adaptation component 510 can be implemented, for example, using a VADA 510A, a DIR-T 510B, and / or a CO-DA 510C.

[0112] Now referring back to FIG. 1A At 108, the trained domain adaptation component is provided. The trained domain adaptation component is used to output a relevant classification category in response to an input of a query unlabeled data element. The trained domain adaptation component can be stored, for example, on a memory and / or data storage device of a computing device, provided to another computing device (e.g., over a network to a server), and / or provided to another process (e.g., for integration within another data stream).

[0113] Note that for classification of an unlabeled query data element, the synthesizer component trained end-to-end with the domain adaptation component is not necessarily required. Classification of the query data element is performed by the trained domain adaptation component without the trained synthesizer component.

[0114] Now referring to FIG. 3 FIG. 3 .

[0115] At 302, a query data element is received for classification into a classification category by a trained model. The query data element is unlabeled. For example, the query data element is an image for classification.

[0116] At 304, the query data element is fed into a trained domain adaptation component of the model.

[0117] At 306, a relevant classification category for the query data element is output by the trained domain adaptation component.

[0118] At 308, the relevant classification category is provided, e.g., stored locally in memory and / or a data storage device, provided to the client terminal from which the query data element originated, and / or provided to another process for further processing.

[0119] The description of various embodiments of the application has been presented for purposes of illustration but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technology found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0120] It is contemplated that many related models and data elements will be developed during the life of a patent maturing from this application, and that the scope of the terms models and data elements are intended to include all such new technologies a priori.

[0121] As used herein, the term "about" means ±10%.

[0122] The terms "comprising," "including," "containing," "having," and "including" and their cognate forms are used interchangeably and mean "including but not limited to." The term encompasses the terms "consisting of" and "consisting essentially of."

[0123] The phrase "consisting essentially of" means that the composition or method can include additional ingredients and / or steps, but only if the additional ingredients and / or steps do not materially alter the basic and novel characteristics of the claimed composition or method.

[0124] As used herein, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. For example, the term "a compound" or "at least one compound" can include a plurality of compounds, including mixtures thereof.

[0125] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0126] The word "optionally" is used herein to mean "may or can not be present." Any particular embodiment of the application can include one or more "optional" features, unless such feature conflicts with a claimed feature.

[0127] Throughout this application, various embodiments of the application can be presented in a range format. It should be understood that the description in range format is merely for convenience and brevity and is to be interpreted -in the context of the specification as a whole. Therefore, the description of a range it should be considered to have specifically disclosed the range limits (including individual numbers within that range), unless the context clearly dictates otherwise. For example, a description of a range such as 1 to 6 should be considered to have specifically disclosed the sub-ranges, such as 1 to 3, 1 to 4, 1 to 5, 2 to 4, 2 to 6, 3 to 6, etc., as well as individual numbers within that range, for example, 1, 2, 3, 4, 5, and 6. This applies regardless of the breadth of the range.

[0128] Whenever a numerical range is indicated in this document, it is meant to include any cited number (fractional or integral) within the indicated range. The phrases "range between" and "range from" a first indicated number and a second indicated number and are used interchangeably, and are meant to include the first and second indicated numbers and all the fractions and integers there between.

[0129] It should be understood that certain features of the application described in the context of separate embodiments can also be provided in combination in a single embodiment. Conversely, various features of the application described in the context of a single embodiment can also be provided separately or in any appropriate

[0130] Although the application has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations as fall within the spirit and broad scope of the appended claims.

[0131] All publications, patents and patent applications mentioned in this specification are herein incorporated in their entirety by reference for the purpose of describing and disclosing, for example, the constructs, methodologies and / or procedures described in such publications, which might serve as background to the present application. In addition, as to each term used it should be understood that the term is provided herein to describe a specific embodiment, which should not be considered as limiting to the present application. Moreover, unless otherwise required by context, it will be understood that when a term is used herein to begin a phrase of the like, that the complete phrase is to be construed as applying to that term. Additionally, any priority document referenced herein is hereby incorporated by reference in its entirety to the extent that it is not inconsistent with this application.

Claims

1. A method of computing a model for classification of data elements, comprising: feeding a plurality of labeled auxiliary data elements and at least one labeled related data element for each of a plurality of related classification categories into a synthesizer component for output of at least one synthesized labeled related data element for each of the plurality of related classification categories; feeding the synthesized labeled related data elements and a plurality of unlabeled training related data elements into a domain adaptation component for output of a respective related classification category for each of the plurality of unlabeled training related data elements; iteratively training the synthesizer component and the domain adaptation component end-to-end; and providing the trained domain adaptation component for output of a related classification category in response to input of a query unlabeled data element, wherein the data element is an image depicting a visual object, wherein the classification category is a visual classification category.

2. The method of claim 1, wherein, a number of the at least one labeled related data element is 1-5 per related classification category.

3. The method of claim 1, further comprising creating an initial model, wherein, training the model for classification of data elements using the initial model, the initial model being trained by: receiving the plurality of labeled auxiliary data elements, the plurality of labeled auxiliary data elements representing each of a plurality of auxiliary classification categories; feeding the plurality of labeled auxiliary data elements into the synthesizer component of the model for output of a plurality of synthesized labeled auxiliary data elements for each of the plurality of auxiliary classification categories; feeding the synthesized labeled related data elements into a source input of the domain adaptation component of the model and feeding a further plurality of unlabeled auxiliary data elements into a target input of the domain adaptation component of the model for output of a certain auxiliary classification category selected from the plurality of auxiliary classification categories for each of the plurality of unlabeled auxiliary data elements; and iteratively training the synthesizer component and the domain adaptation component end-to-end to create the initial model. the synthesizer component is implemented as an extension of an incremental encoder component of the model, the incremental encoder component being trained to output a vector representation of the at least one synthesized labeled related data element for each of the plurality of related classification categories in response to input of the plurality of labeled auxiliary data elements and the at least one labeled related data element.

4. The method of claim 3, wherein, 5. The method of claim 4, further comprising: during the creation of the initial model, training an incremental-forgery component of the incremental-encoder along with an incremental-discriminator component of the incremental-encoder to mimic the incremental encoder component to output a pseudo vector representation in response to input of the at least one labeled related data element without the plurality of labeled auxiliary data elements, wherein the model is trained from the initial model by feeding the at least one labeled related data element into the incremental-forgery and incremental-discriminator components of the incremental-encoder without feeding the plurality of labeled auxiliary data elements. ​ 6. The method of claim 1, wherein each of the plurality of labeled auxiliary data elements is labeled with a respective auxiliary classification category selected from a plurality of auxiliary classification categories, wherein the plurality of auxiliary classification categories is different from the plurality of relevant classification categories.

7. The method of claim 1, wherein, The domain adaptation component computes an adjustment to a distribution of the synthesized labeled relevant data elements to correlate with a distribution of the plurality of unlabeled training relevant data elements, wherein a certain relevant classification category selected for each of the plurality of unlabeled training relevant data elements is computed based on the adjusted synthesized labeled relevant data elements.

8. The method of claim 1, wherein, The synthesizer component receives a certain labeled auxiliary data element and a respective labeled relevant data element for output of a synthesized labeled relevant data element.

9. The method of claim 1, wherein, A number of labeled auxiliary data elements for each of a plurality of auxiliary classification categories is sufficient to train a standard classifier for accurately classifying a query data element into one of the plurality of auxiliary classification categories.

10. The method of claim 1, wherein, The synthesizer component and the domain adaptation component are iteratively trained end-to-end to minimize an objective function of the domain adaptation component for the input of the plurality of unlabeled training relevant data elements and the synthesized labeled relevant data elements output by the synthesizer component.

11. The method of claim 1, further comprising: mapping each of the labeled auxiliary data elements and the at least one labeled relevant data element into a respective vector in a multi-dimensional space, wherein the feeding comprises feeding the mapped respective vectors into the synthesizer component, wherein a distribution of at least one synthesized labeled relevant data element for each of the plurality of relevant classification categories output by the synthesizer component correlates with a distribution in the multi-dimensional space of the mapped vectors of the labeled auxiliary data elements centered around the vector of the at least one labeled relevant data element.

12. The method of claim 11, wherein, The respective vectors are obtained from a non-classification layer of a neural network trained for classifying the labeled auxiliary data elements into a corresponding label selected from a plurality of auxiliary classification categories.

13. The method of claim 1, wherein, Each of the synthesizer component and the domain adaptation component is implemented as a neural network.

14. A method of classifying a query data element into a classification category by a model, comprising: feeding a query data element into a domain adaptation component trained by: feeding a plurality of labeled auxiliary data elements and at least one labeled relevant data element for each of a plurality of relevant classification categories into a synthesizer component for output of at least one synthesized labeled relevant data element for each of the plurality of relevant classification categories, feeding the synthesized labeled relevant data elements and a plurality of unlabeled training relevant data elements into a domain adaptation component for output of a respective relevant classification category for each of the plurality of unlabeled training relevant data elements, and iteratively training the synthesizer component and the domain adaptation component end-to-end; and outputting, by the trained domain adaptation component, a relevant classification category for the query data element, wherein the data element is an image depicting a visual object, wherein the classification category is a visual classification category.

15. A system for computing a neural network model for classifying data elements, comprising: at least one hardware processor that executes code for: feeding a plurality of labeled auxiliary data elements and at least one labeled relevant data element for each of a plurality of relevant classification categories into a synthesizer component for outputting at least one synthesized labeled relevant data element for each of the plurality of relevant classification categories; feeding the synthesized labeled relevant data elements and a plurality of unlabeled training relevant data elements into a domain adaptation component for outputting a respective relevant classification category for each of the plurality of unlabeled training relevant data elements; iteratively training the synthesizer component and the domain adaptation component end-to-end; and providing the trained domain adaptation component for outputting a relevant classification category in response to an input of a query unlabeled data element, wherein the data element is an image depicting a visual object, wherein the classification category is a visual classification category.

Images