Machine learning based visual device selection system

Abstract

The present disclosure relates to determining a visual device for a patient or user. In embodiments, a machine learning-based approach considers the user's face in the context of a database of labeled faces and visual devices, each of which reflects the aesthetic value of a proposed visual device for the patient's or user's face.

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates to eyewear, and more particularly to matching visual instruments with a patient's face. [Background technology]

[0002] During the selection of new visual equipment or eyewear, patients are often left to self-reflection in judging the aesthetics of the new eyewear on the patient's face. At the same time, the patient may be dealing with their own opinion of the new eyewear on their face and the hypothetical opinions of third parties (e.g., friends, family, experts, etc.) about the fit of the new eyewear to their face. The eyewear selection task, which considers aesthetic appeal along with the appropriate visual eyewear needs, can be cumbersome, and there is no efficient way to confidently purchase new eyewear that the user, the user's doctor, and the user's friends will surely like. The present invention provides a solution to this problem.

[0003] The above "Background" discussion is intended to generally present the context of the present disclosure. Aspects of the inventors' efforts to the extent described in this Background section and the discussion that may not otherwise qualify as prior art at the time of filing are not admitted, either explicitly or implicitly, as prior art against the present invention. Summary of the Invention [Means for solving the problem]

[0004] The present disclosure relates to an apparatus, method, and computer-readable storage medium for determining a matched vision instrument.

[0005] According to an embodiment, the present disclosure further relates to an apparatus for determining a matched visual instrument, the apparatus comprising: a processing circuit configured to receive at least one input, the at least one input including an image of a human face; apply a neural network to the at least one input, the neural network generating at least one match measure for the at least one input; and determine a matched visual instrument based on the at least one match measure generated by the neural network, the at least one match measure corresponding to relative synchronization between the human face and the visual instrument.

[0006] According to an embodiment, the present disclosure further relates to a method for determining a matched visual device, the method including: receiving at least one input by a processing circuit, the at least one input including an image of a human face; applying by the processing circuit a neural network to the at least one input, the neural network generating at least one match measure for the at least one input; and determining by the processing circuit a matched visual device based on the at least one match measure generated by the neural network, the neural network including an implicit input, and the at least one match measure corresponding to correlative synchronization between the human face and the visual device.

[0007] The above paragraphs have been provided as a general introduction and are not intended to limit the scope of the following claims. A better understanding of the various features and advantages described herein may be obtained by reference to the following detailed description, which describes exemplary embodiments and the accompanying drawings.

[0008] A more complete understanding of the present disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description, when considered in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0009] [Figure 1] 1 is a flow diagram of an implementation of a machine learning based visual equipment selection tool according to an exemplary embodiment of the present disclosure. [Figure 2A] 1 is a schematic diagram of aspects of an image input process according to an exemplary embodiment of the present disclosure. [Figure 2B] 1 is a schematic diagram of aspects of an image preparation process according to an exemplary embodiment of the present disclosure. [Diagram 3] 1 is a schematic diagram of aspects of an image preparation process according to an exemplary embodiment of the present disclosure. [Figure 4] 1 is a flow diagram aspect of a process for training a neural network of a visual equipment selection tool according to an exemplary embodiment of the present disclosure. [Figure 5A] FIG. 1 is a schematic diagram of an input of a labeling process according to an exemplary embodiment of the present disclosure. [Figure 5B] FIG. 1 is a schematic diagram of a labeling process according to an exemplary embodiment of the present disclosure. [Figure 6A] 1 is a graphic of inputs to a training process for machine learning based visual instrument selection according to an exemplary embodiment of the present disclosure. [Figure 6B] 1 is a graphic of inputs to a training process for machine learning based visual instrument selection according to an exemplary embodiment of the present disclosure. [Figure 6C] 1 is a graphic of inputs to a machine learning visual instrument selection training process according to an exemplary embodiment of the present disclosure. [Figure 6D] 1 is a graphic of inputs to a machine learning visual instrument selection training process according to an exemplary embodiment of the present disclosure. [Figure 6E] 1 is a graphic of inputs to a machine learning visual instrument selection training process according to an exemplary embodiment of the present disclosure. [Figure 7A] FIG. 2 is a schematic diagram illustrating labeling of an input according to an exemplary embodiment of the present disclosure. [Figure 7B] FIG. 2 is a schematic diagram illustrating labeling of an input according to an exemplary embodiment of the present disclosure. [Figure 7C]FIG. 2 is a schematic diagram illustrating labeling of an input according to an exemplary embodiment of the present disclosure. [Figure 7D] FIG. 2 is a schematic diagram illustrating labeling of an input according to an exemplary embodiment of the present disclosure. [Figure 8A] FIG. 1 is a schematic diagram illustrating multiple labeling groups according to an exemplary embodiment of the present disclosure. [Figure 8B] FIG. 1 is a schematic diagram illustrating labeling of an input with multiple labeling groups according to an exemplary embodiment of the present disclosure. [Figure 8C] FIG. 1 is a schematic diagram illustrating labeling of an input with multiple labeling groups according to an exemplary embodiment of the present disclosure. [Figure 8D] FIG. 1 is a schematic diagram illustrating labeling of an input with multiple labeling groups according to an exemplary embodiment of the present disclosure. [Figure 8E] FIG. 1 is a schematic diagram illustrating labeling of an input with multiple labeling groups according to an exemplary embodiment of the present disclosure. [Figure 9A] FIG. 1 is a schematic diagram illustrating a labeling group according to an exemplary embodiment of the present disclosure. [Figure 9B] 1 is a graphic of inputs to the training process of a machine learning based visual equipment selection tool according to an exemplary embodiment of the present disclosure. [Figure 9C] 1 is a graphic of inputs to the training process of a machine learning based visual equipment selection tool according to an exemplary embodiment of the present disclosure. [Figure 9D] 1 is a graphic of inputs to the training process of a machine learning based visual equipment selection tool according to an exemplary embodiment of the present disclosure. [Figure 9E] 1 is a graphic of inputs to the training process of a machine learning based visual equipment selection tool according to an exemplary embodiment of the present disclosure. [Figure 9F] 1 is a graphic of inputs to the training process of a machine learning based visual equipment selection tool according to an exemplary embodiment of the present disclosure. [Figure 10A]FIG. 1 is a schematic diagram illustrating labeling of an input by a labeling group according to an exemplary embodiment of the present disclosure. [Figure 10B] FIG. 1 is a schematic diagram illustrating labeling of an input by a labeling group according to an exemplary embodiment of the present disclosure. [Figure 10C] FIG. 1 is a schematic diagram illustrating labeling of an input by a labeling group according to an exemplary embodiment of the present disclosure. [Figure 10D] FIG. 1 is a schematic diagram illustrating labeling of an input by a labeling group according to an exemplary embodiment of the present disclosure. [Figure 11A] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual equipment selection tool, where the inputs are morphological features and visual equipment attributes, according to an exemplary embodiment of the present disclosure. [Figure 11B] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual equipment selection tool, where the inputs are morphological features and visual equipment attributes, according to an exemplary embodiment of the present disclosure. [Figure 11C] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual device selection tool, where the inputs are face images and visual device attributes, according to an exemplary embodiment of the present disclosure. [Figure 11D] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual equipment selection tool, where the inputs are morphological features and visual equipment images, according to an exemplary embodiment of the present disclosure. [Figure 11E] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual device selection tool, where the inputs are a face image and a visual device image, according to an exemplary embodiment of the present disclosure. [Figure 11F] FIG. 1 is a schematic diagram of a machine learning based neural network of visual instrument attributes, where the inputs are morphological features, visual measurements, and images of the visual instrument, according to an exemplary embodiment of the present disclosure. [Figure 11G] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual instrument selection tool, where the inputs are morphological features, visual measurements, and visual instrument attributes, according to an exemplary embodiment of the present disclosure. [Figure 11H]FIG. 1 is a schematic diagram of a neural network of a machine learning based visual device selection tool, where the inputs are face images, visual measurements, and visual device attributes, according to an exemplary embodiment of the present disclosure. [Figure 11I] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual instrument selection tool, where the inputs are morphological features, visual measurements, and images of visual instruments, according to an exemplary embodiment of the present disclosure. [Figure 11J] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual instrument selection tool, where the inputs are a face image, visual measurements, and an image of a visual instrument, according to an exemplary embodiment of the present disclosure. [Figure 11K] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual equipment selection tool where the input is morphological features, according to an exemplary embodiment of the present disclosure. [Figure 11L] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual equipment selection tool where the input is morphological features, according to an exemplary embodiment of the present disclosure. [Figure 11M] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual equipment selection tool where the input is morphological features, according to an exemplary embodiment of the present disclosure. [Figure 11N] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual equipment selection tool where the input is morphological features, according to an exemplary embodiment of the present disclosure. [Figure 11O] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual instrument selection tool, where the inputs are morphological features and visual measurements, according to an exemplary embodiment of the present disclosure. [Figure 11P] FIG. 1 is a schematic diagram of a neural network of a machine learning based visual instrument selection tool, where the inputs are morphological features and visual measurements, according to an exemplary embodiment of the present disclosure. [Figure 11Q] FIG. 1 is a schematic diagram of a neural network for a machine learning based visual instrument selection tool, where the inputs are face images and visual measurements, according to an exemplary embodiment of the present disclosure. [Figure 11R]FIG. 1 is a schematic diagram of a neural network for a machine learning based visual instrument selection tool, where the inputs are face images and visual measurements, according to an exemplary embodiment of the present disclosure. [Figure 12A] FIG. 2 is a schematic diagram illustrating preparation of a training database for a neural network training process according to an exemplary embodiment of the present disclosure. [Figure 12B] FIG. 1 is a schematic diagram of a neural network for the training process of a machine learning based visual device selection tool, where the input is metric facial landmark coordinates. [Figure 13A] 1 is a generalized flow diagram of a neural network configured to handle heterogeneous input data, according to an exemplary embodiment of the present disclosure. [Figure 13B] 1 is a generalized flow diagram aspect of a neural network configured to handle heterogeneous input data, according to an exemplary embodiment of the present disclosure. [Figure 13C] 1 is a generalized flow diagram aspect of a neural network configured to handle heterogeneous input data, according to an exemplary embodiment of the present disclosure. [Figure 13D] 1 is a generalized flow diagram aspect of a neural network configured to handle heterogeneous input data, according to an exemplary embodiment of the present disclosure. [Figure 13E] 1 is a generalized flow diagram aspect of a neural network configured to handle heterogeneous input data, according to an exemplary embodiment of the present disclosure. [Figure 13F] 1 is a generalized flow diagram aspect of a neural network configured to handle heterogeneous input data, according to an exemplary embodiment of the present disclosure. [Figure 14] 1 is a flow diagram of training a neural network of a machine learning based visual equipment selection tool according to an exemplary embodiment of the present disclosure. [Figure 15A] 1 is a generalized flow diagram of an embodiment of an artificial neural network. [Figure 15B] 1 is a flowchart of an implementation of a convolutional neural network according to an exemplary embodiment of the present disclosure. [Figure 16] This is an example of a feedforward artificial neural network. [Figure 17] 1 is a hardware configuration of a machine learning based visual equipment selection tool according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The terms "a" or "an," as used herein, are defined as one or more than one. The term "plurality," as used herein, is defined as two or more than two. The term "another," as used herein, is defined as at least two or more. The terms "including" and / or "having," as used herein, are defined as comprising (i.e., open terms). The terms "visual device," "eyeglass," "eyeglasses," and "visual devices" may be used interchangeably to refer to devices having both a frame and lenses. The term "visual device" may be used to refer to one visual device, while the term "visual devices" may be used to refer to two or more visual devices. References throughout this document to "one embodiment," "particular embodiment," "embodiment," "implementation," "example," or similar terms mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the present disclosure. Similarly, the terms "facial image" and "human facial image" are corresponding terms that may be used interchangeably. Thus, the appearances of such phrases in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments without limitation.

[0011] Today, patients or other users looking for visual devices or eyeglasses are often given little guidance on what is ophthalmologically appropriate and aesthetically pleasing. For some, cultural trends drive the decision. For others, the opinions of friends and family are paramount. Yet for those who prioritize ergonomic fit and visual acuity, the opinion of a trained vision professional is needed. Currently, users have access to approaches that provide some, but not all, of the above features. For example, in one approach, a decision tree can be implemented to match the frame of a visual device with morphological features detected from an individual's facial landmarks, and this match determines the ideal visual device. In another approach, the user may be asked to answer questions specifically about their style, lifestyle, and personality to identify the user's tastes and habits. Such features can then be used to suggest the ideal visual device based on implementing a decision tree or content-based filtering. Yet another approach, which is an extension of the above, utilizes a user preference model that performs an analysis of the user's browsing history to identify visual device features such as frames that appear relevant to the user (e.g., content-based filtering) or the consumer profile to which the user is closest (e.g., collaboration-based filtering).

[0012] While the above approaches partially address user needs, they do not provide users with robust, end-to-end input when making visual device selections. To this end, the present disclosure describes a machine learning based visual device selection tool that presents users with tailored visual device selections based on morphological features, structural features, ophthalmic needs, and aesthetic appeal.

[0013] Referring now to the figures, FIG. 1 is a generalized flow diagram of a machine learning based visual equipment selection tool (ML-VEST) 100 according to an exemplary embodiment of the present disclosure. The ML-VEST 100 can include an input preparation process 110, a machine learning application process 115, and a visual equipment selection process 125. First, a user provides an input 105 to the ML-VEST 1000. Based on the neural network training, the input 105 can be directly applied 114 to the machine learning application process 115 or can be provided to the input preparation process 110, where the input 105 is prepared according to the specific implementation specifications of the machine learning application process 115. In an embodiment, the input 105 requires preparation and can therefore be an image of the user's face provided to the input preparation process 110. The prepared input or the prepared image can then be passed to the machine learning application process 115. A fit measure 120 may be generated from the machine learning application process 115, where prepared images corresponding to the user's face are scored based on the "fit" of the visual device or glasses, where fit is a relative synchronization that provides a measure that quantifies the fit between the user's face and the visual device. Based on the magnitude of the scored fit measure 120 or fit measures 120 of the visual device, the visual device selection process 125 may select the ideal glasses for the face of the user who provided the input. In an embodiment, the user may provide an image of the user's face and an image of the visual device of interest. In processing the image, the ML-VEST 100 may generate a fit measure 120 that, when compared to a predefined threshold of the fit measure 120, indicates that the visual device should be selected as the ideal visual device for the user. In an embodiment, the fit measure 120 may be associated with a confidence that, when compared to a confidence threshold, indicates whether the fit measure 120 is accurate or not. In another embodiment, the user may provide only an image of the face as input, and the ideal or matching visual device may be a visual device selected from a database of glasses, or may be selected from a subset of glasses pre-selected by or provided to the user, for example.For this purpose, as described above, a fit measure 120 for each selected visual instrument can be generated, and a comparison of the generated fit measures 120 can indicate which visual instrument should be selected as an ideal visual instrument, which can then be recommended to the user, the ideal visual instrument reflecting the user's unique morphological characteristics in conjunction with the user's preferences regarding aesthetic appearance and visual acuity.

[0014] 2A illustrates at least one input 205 that may be provided by a user to the ML-VEST. As described with reference to FIG. 1, the at least one input 205 may be provided to an input preparation process or directly to a machine learning application process. The at least one input 205 provided by the user may include, among others, an image of the user's face 206, an image of the user's face and a separately provided image of the visual device 207, an image of the user's face wearing the visual device 208, and a visual measurement 209 corresponding to the user. The visual measurement may be a standard ophthalmic measurement of visual acuity.

[0015] As shown in Figure 2B, each of the at least one inputs described above may be prepared before being provided to the machine learning application process when provided to the input preparation process. Accordingly, Figure 2B describes an input preparation process 210 that may be performed on at least one received input. In embodiments, the input preparation process 210 described herein may be performed on inputs provided by a user during application of ML-VEST and on inputs provided during training of the neural network of ML-VEST.

[0016] From a high level, as applied during application of ML-VEST or during training of ML-VEST, an input preparation process 210 executes at least one input preparation function 211 and generates at least one input preparation output 213. As will be appreciated by those skilled in the art, the at least one input preparation function 211 and the at least one input preparation output 213 may be selected such that similar processes are performed during application of ML-VEST and during training of the neural network of ML-VEST.

[0017] From a low level, the at least one input preparation function 211 may include, for example, image classification, image segmentation, and convolution 212, among others. Image segmentation may be performed to detect relevant characteristics of the at least one input, both during training of the neural network of ML-VEST and during application of ML-VEST. These relevant characteristics, referred to as the at least one input preparation output 213, may be, for example, morphological features such as "face width" and "nose size", or visual instrument attributes such as "frame shape" and "frame color". Additional morphological features include face shape, skin color, eye color, hair color, etc. Such morphological features may be calculated via image processing (i.e., image segmentation / classification) as described above, or may be manually identified or measured in the input image, where manual measurement requires a calibration object to accurately calculate the dimensions of the features. Additional visual instrument attributes may include lens width, lens height, bridge distance, temple length, etc. Such visual device attributes may be calculated via image processing (i.e., image segmentation / classification) as described above, or may be manually identified or measured in the input image, where manual measurement requires a calibration object to accurately calculate the attributes. In some cases, the visual device attributes may be accessed from a database that contains visual device attributes.

[0018] In an embodiment, in addition to the image segmentation and classification described above, convolution 212 may be performed on at least one input. Convolution 212 may include the use of convolution filters and may facilitate feature extraction. As described below, convolution 212 may be performed by a neural network in ML-VEST, thereby eliminating the input preparation process 210.

[0019] In an embodiment, the at least one input may be provided to the input preparation process 210 or directly to the machine learning application process. For example, the at least one input may be a corresponding user's visual measurement. The at least one input provided by the user may include SPH and ADD and may be provided to the neural network of ML-VEST.

[0020] 3, following the input preparation process, if necessary, at least one prepared input may be sent to the ML-VEST machine learning application process 315. In general, the input to the machine learning application process may include at least one input provided directly to the machine learning application process (e.g., face image and vision device image, face image, face image wearing the vision device) and at least one prepared input (e.g., morphological features from the face image, vision device attributes from the vision device image). Considered together, several use cases of inputs of the neural network of ML-VEST can be considered: (1) morphological features and visual device attributes obtained from face images and visual device images or from face images wearing visual devices, (2) face images and visual device attributes, (3) morphological features and visual device attributes, (4) face images and visual device images, (5) morphological features, visual device attributes, and visual measurements, (6) face images, visual device attributes, and visual measurements, (7) morphological features, visual device images, and visual measurements, (8) face images, visual device images, and visual measurements, (9) morphological features, (10) face images, (11) morphological features and visual measurements, (12) face images and visual measurements.

[0021] Returning to Figure 3, based on the selected use case (discussed above), machine learning can be applied to the prepared at least one input, and a fitness measure 320 can be generated as an output of the machine learning application process 315. A detailed description of the neural network of ML-VEST for each use case is provided with reference to subsequent figures. An evaluation of the magnitude of a fitness measure or a comparison of the magnitudes of multiple fitness measures can then be used to select an ideal vision device 325 for the user.

[0022] The type of fitness measure 320 and the resulting ideal vision device 325 selected may be based on the training of the neural network of ML-VEST. Accordingly, FIG. 4A provides a flow diagram of the training of the neural network used during the machine learning application process of ML-VEST.

[0023] In general, training involves providing ideal inputs to ML-VEST and providing labeler group scores to a group of labelers during the labeling process 440 to generate training fitness measure data or "ground truth" data. To train the neural network 430, the fitness measures 420 generated by the ML-VEST neural network 430 can be compared to the training fitness measures scored by the labeler group during the labeling process 440. An error value 438 generated therebetween can be evaluated and parameters of the ML-VEST neural network 430 can be adjusted 439 accordingly so that future fitness measures generated by the ML-VEST neural network 430 are increasingly accurate with respect to the fitness measures scored during the labeling process 440.

[0024] In particular, the training first includes at least one input 405 received from a training database 436. As suggested in FIG. 2A, the training database 436 can be comprised of a plurality of inputs including face images, face images aligned with images of visual devices, face images wearing visual devices, and visual measurements corresponding to the visual acuity of the user's eyes. The plurality of inputs stored in the training database 436 is intended to be from a homogenous population of people and a wide variety of visual devices, allowing ML-VEST to robustly select an ideal visual device for a random user. However, it can be understood that the plurality of inputs stored in the training database 436 can be any variety of inputs and can be tailored to a particular application. For example, the plurality of inputs stored in the training database 436 can include, among others, face images (or morphological features thereof) of people from a population, visual measurements corresponding to the face images, and visual device images (or visual device attributes thereof).

[0025] At least one input 405 from the plurality of inputs may be provided to an input preparation process 410, if desired, or directly to the neural network 430. Additionally, at least one input 405 may be provided to a labeling process 440. In an embodiment, the at least one input 405 simultaneously provided to the input preparation process 410 and the labeling process 440 may be a subset of the plurality of inputs stored in a training database, as shown in FIG. 5A. In one case, the subset may include images of faces and images of visual devices. Thus, the trained neural network 430 is capable of generating a fitness measure 420 for any visual device. In another case, the subset may include only images of faces. Rather than providing an image of a face for each image of a visual device, the image of a face is provided along with a predefined list of visual devices, and the predefined list of visual devices is an implicit input to the neural network 430. In training the neural network 430, the facial image can be scored (i.e., labeled) against each visual device of the predefined list of visual devices, and the output of the neural network 430 can then be a list of match scores for each visual device of the predefined list of visual devices with the facial image. In other words, the implicit input can be a selected subset or a predefined list of visual devices. The selected subset of visual devices can be used to evaluate the facial image as described in the use case, and each visual device of the selected subset of visual devices is given a match score for the facial image. It can be understood that the selection of the inputs, including the implicit inputs, of the multiple inputs stored in the training database 436 can be based on the particular implementation of the neural network 430 of ML-VEST.

[0026] According to an embodiment, as introduced above, the labeling process 440 can provide "ground truth" or training data on which the neural network 430 can train and thereby learn how to accurately classify or predict fitness measures. In the context of the present disclosure, the labeling process 440 can include, among other things, the scoring and commenting of each of a plurality of images provided as at least one input.

[0027] In an embodiment, the labeler may, for example, view the person's face from the input image aligned with the image of the visual device and provide a score and comments thereon. Referring to FIG. 5A, the input to the labeling process includes the image of the face aligned with the image of the visual device 508, the image of the face wearing the visual device 509, and the visual measurements corresponding to the image of the face 549, as described above. In addition, the input to the labeling process may include the image of the face wearing the virtual visual device 552. The image of the face wearing the virtual visual device 552, referred to as a "virtual try-on motion", provides a way for the labeler to visualize the visual device worn on the face when an authentic image of the face wearing the particular visual device is not available. The virtual try-on motion generated within the labeling process may be created by first dewarping the image of the visual device so that only the pattern of the visual device remains, and then copying the pattern of the visual device onto the image of the face, thereby emulating an authentic image of the face wearing the visual device. The virtual try-on motion may then be evaluated and labeled by the labeler as normal.

[0028] As introduced above, the labeler provides a label, a set of labels, or a score that defines the characteristics of the output layer of the neural network of ML-VEST. For example, referring to FIG. 5B, the labeling process 540 can include determining whether the visual device matches the face, referred to as binary visual device match 541. The labeling process 540 can further include determining a match score between the visual device and the face, referred to as overall visual device match score 542. Furthermore, the labeling process 540 can include determining a match score between the visual device and the face for each criterion of a predefined criteria list, referred to as visual device match by criteria 543. This predefined criteria list can include, for example, a match score of face width to frame width, a match score of face shape to frame shape, a match score of face bottom shape to frame height, and a match score of skin width to frame color / decoration, among others. The above labels of the labeling process 540 can be represented as corresponding values. For example, a binary visual instrument fit may be represented as 0 or 1, an overall visual instrument match score 542 may be represented as a score from 0 to N (e.g., 2 stars out of 5), and a visual instrument match score by criteria may be represented as a score from 0 to N for each criterion (e.g., 4 stars out of 5). In addition to providing a label for each image according to a particular process of the labeling process 540, the labeler may provide comments 544 as to why a particular label was assigned, including, for example, phrases such as "this visual instrument is too large for this face width" or "this visual instrument is too dark for this skin color," among others. In an embodiment, the labeling process 540 may include visual measurements corresponding to the facial image in addition to the facial image and the visual instrument image, allowing a person with visual expertise to take visual acuity into account when labeling.

[0029] It can be appreciated that the labeling process can be repeated until all possible combinations of faces and visual devices are labeled, e.g., a level can be provided for each combination of one image of a face with multiple images of a visual device.

[0030] Returning now to FIG. 4A, with the understanding that the labels of the labeling process 440 are the output of the neural network 430, the neural network 430 can be trained. A more complete, but still general, description of training a neural network is described with respect to FIGS. 14-16. As shown in FIG. 4A, the at least one processed input can be provided to a first hidden or input layer of the neural network 430. In one example, the neural network 430 can be a fully connected neural network, where each fully connected layer of the neural network can learn from all combinations of features or outputs of the previous layer. As discussed with respect to the input preparation process 410, the input layer of the neural network 430 can vary according to the use case. After passing the at least one processed input through the N hidden layers of the neural network 430, a fitness measure 420 can be generated from the output layer. The generated fitness measure 420 necessarily matches the labels of the labeling process 440 or the training fitness measure. Thus, the value of the fit measure 420 may be compared to the labels or training data of the labeling process 440 in an error determination 438 to determine the accuracy of the output of the neural network 430. Based on the error determination 438, the training process 435 may proceed or return to the first hidden layer of the neural network 430, and the coefficients / weights of each hidden layer may be updated based on the error of the error determination 438. As shown, the training process 435 of the ML-VEST and the neural network 430 may continue, among other things, until the error determination 438 meets a criterion. The criterion may be one of a wide variety of criteria, including an error value or a number of iterations. Once the error between the fit measure and the training data meets the criterion of the error determination 438, the neural network 430 is ready to be implemented within the ML-VEST.

[0031] During implementation within ML-VEST, the match measure 420 can be one of a plurality of match measures 420 describing at least one input, including an image of a human face and each visual device of a plurality of visual devices in a visual device database, and the match measure 420 can be further output to a visual device selection process 425 shown in FIG. 4B. The visual device selection process 425 can take each of the plurality of match measures 420 and select a matched visual device. In one case, the matched visual device maximizes a binary visual device match, a global visual device match score, or a criterion visual device match score, together with the accompanying comments, as determined by training 435 of a neural network 430. In another case, the matched visual device can be determined by comparing each of the binary visual device match, the global visual device match score, or the criterion visual device match score, together with the accompanying comments, to a predefined threshold.

[0032] 6A-6E show exemplary inputs of ML-VEST. For example, FIG. 6A describes a case where at least one input of the training of ML-VEST includes input images obtained from a training database, where the input images include images of people's faces and separately images of visual devices 608. FIG. 6B shows an input of the training of ML-VEST where at least one input includes images of people's faces 607 and visual device attributes 651, where the visual device attributes 651 are similarly obtained from the training database. In view of FIG. 6A, FIG. 6C shows a case where at least one input of the training of ML-VEST includes images of people's faces 609 wearing visual devices. FIG. 6D and FIG. 6E include morphological features of the facial images as at least one input, where the morphological features are stored in the training database. Referring to FIG. 6D, at least one input of ML-VEST can be morphological features 653 of the facial images and visual device attributes 651 of multiple visual devices in the training database. As shown in FIG. 6E, the at least one input obtained from the training database may include morphological features 653 of an image of a face and an image 605 of a vision instrument.

[0033] FIG. 7A reflects the labeling process described in view of FIG. 5, where at least one input 705 to the labeling process includes an image of a human face and separately an image of a vision device. The at least one input 705 of FIG. 7A, or in another embodiment the at least one input that has been processed, can be labeled by a labeler. In an embodiment, the labeler is one of a group of labelers 745. With respect to FIG. 7A, the group of labelers 745 can be a mixed group of people. The labels of each of the mixed group of people for the image of the human face with the vision device can include a binary vision device fit 741, an overall vision device match score 742, and a benchmark vision device match score 743, as described in FIG. 5. These labels, as determined by the group of labelers 745, can be deployed as "ground truth" or as training data during training of the neural network of ML-VEST, and can define the output layer of the neural network. The above labeling process can be repeated for each combination of images of people's faces and visual instruments from a plurality of visual instruments in the training database.

[0034] FIG. 7B reflects the labeling process described in view of FIG. 5, where at least one input 705 includes an image of a face of a person wearing a vision device. The at least one input 705 of FIG. 7B, or in another embodiment the at least one input that has been processed, can be labeled by a labeler. In an embodiment, the labeler is one of a group of labelers 745. With respect to FIG. 7B, the group of labelers 745 can be a mixed group of people. The labels of each of the mixed group of people for the image of the face of the people wearing the vision device can include a binary vision device fit 741, an overall vision device match score 742, and a vision device match score by norm 743, as described in FIG. 5. These labels, as determined by the group of labelers 745, can be deployed as "ground truth" or as training data during training of the neural network of ML-VEST, and can define the output layer of the neural network. The above labeling process can be repeated for each one of the images of the face of the people wearing the vision device of the multiple images of the training database.

[0035] FIG. 7C reflects the labeling process described in view of FIG. 5, where the at least one input 705 includes an image of a person's face and an image of a visual device. The at least one input 705 of FIG. 7B can be labeled by a labeler. In an embodiment, the at least one input 705 can be a processed at least one input to facilitate the labeler's task. The processed at least one input can be a virtual try-on action 752 or a 2D VTO, where the image of the face and the image of the visual device are manipulated so that the face can appear to be wearing the visual device. The group of labelers 745 providing a match score for the processed at least one input can be a group of people in a unified group. As described in FIG. 5, the labels of each of the unified group of people for the virtual try-on action 752 of the image of the person's face "wearing" the visual device include a binary visual device match 741, an overall visual device match score 742, and a normative visual device match score 743. These labels, as determined by a group of labelers 745, can be deployed as "ground truth" or as training data during training of the ML-VEST neural network, and can define the output layer of the neural network. The labeling process described above can be repeated for each one of the virtual try-on actions 752 of the facial images "wearing" the visual device from the multiple images in the training database.

[0036] FIG. 7D reflects the labeling process described in conjunction with FIG. 5, where at least one input 705 includes an image of a human face. At least one input 705 in FIG. 7B can be labeled by a labeler in conjunction with a virtual try-on action 752 of a visual device among a subset 732 of images of visual devices selected from a training database, for example. The group of labelers 745 providing a match score for the virtual try-on action 752 can be a group of people in a harmonious group. As described in FIG. 5, each label of the group of people in a harmonious group for the virtual try-on action 752 of the images of human faces “wearing” the visual device of the subset includes a binary visual device match 741, an overall visual device match score 742, and a visual device match score by a standard 743. These labels, as determined by the group of labelers 745, can be deployed as “ground truth” or as training data during training of the neural network of ML-VEST, and can define the output layer of the neural network. In one example, the above labeling process is repeated for a virtual try-on operation 752 for each one of the facial images of people "wearing" a visual device from a subset of images of visual devices in the training database from visual device 1 or VE1 through VE2 to VE N.

[0037] According to an embodiment, the group of labelers in the labeling process includes subgroups with related characteristics, as described in FIG. 8A. For example, the group of labelers 845 may include multiple categories of labelers 850, each category of the multiple categories of labelers 850 being linked by a common denominator. In one category of labelers, each labeler is generally defined as a consumer group, such as, for example, gender, age, socio-professional category, region, and style, among others. These labelers can be further defined as a combination of consumer groups, such as, for example, "frequent flyer male in his 50s," "workaholic female in her 40s with children," and the like. Thus, by applying each of the multiple categories of labelers 850 to the training process of FIG. 4A, a neural network can be trained such that the output of the neural network reflects the opinions of the group of people defined as described above. For example, a neural network trained with opinions of a category of labelers defined as "employed single female in her 20s" generates a fitness measure indicative of the opinions of such category of labelers accordingly. During implementation of ML-VEST, the desired categories 850 of labelers may be preselected by the user to provide specific opinions of interest.

[0038] It can be understood that by changing the labeler category of the labeler group, ML-VEST can be adjusted according to the user's desires. For example, a user may want to know which visual device would look best on his face according to the opinion of a local man. In another case, a user may want to know which visual device would look best on his face according to the opinion of a celebrity. In either case, ML-VEST and the group of labelers inside can make the necessary changes and adjustments to achieve a result that reflects the desired opinion.

[0039] To this end, FIG. 8B reflects the labeling process described in view of FIG. 5, where the group of labelers can be one of multiple categories of labelers and the at least one input 805 can include a facial image displayed side-by-side with a visual device image. The at least one input 805 of FIG. 8B, or in another embodiment the processed at least one input, can be labeled by the group of labelers. The group of labelers can be a first category of labelers 846, and can be as many as N categories of labelers 847, each category reflecting a particular group of people defined, for example, by traditional consumer segmentation criteria (described in FIG. 8A). Each label from the first category of labelers 846 to the Nth category of labelers 847 for the side-by-side display of the facial image and the visual device image includes a binary visual device fit 841, an overall visual device match score 842, and a benchmark visual device match score 843, as described in FIG. 5. These labels, as determined by the labeler's category, can be deployed as "ground truth" or as training data during training of the ML-VEST neural network, and can define the output layer of the neural network. In one example, the labeling process is repeated for each combination of face image and vision instrument of the multiple vision instruments in the training database.

[0040] FIG. 8C reflects the labeling process described in view of FIG. 5, where the group of labelers can be one of multiple categories of labelers and the at least one input 805 can include an image of a face wearing a visual device. The at least one input 805 of FIG. 8C, or in another embodiment the at least one processed input, can be labeled by the group of labelers. The group of labelers can be a first category of labelers 846, and can be as many as N categories of labelers 847, each category reflecting a particular group of people defined, for example, by traditional consumer segmentation criteria (described in FIG. 8A). Each label from the first category of labelers 846 to the Nth category of labelers 847 for the image of a face wearing a visual device includes a binary visual device fit 841, an overall visual device match score 842, and a visual device match score by criteria 843, as described in FIG. 5. These labels, as determined by the labeler's category, can be deployed as "ground truth" or as training data during training of the ML-VEST neural network, and can define the output layer of the neural network. In one example, the labeling process is repeated for each combination of face image and vision instrument of the multiple vision instruments in the training database.

[0041] In view of FIG. 7C, FIG. 8D reflects labeling such that the group of labelers can be one of multiple categories of labelers and the at least one input 805 can include a face image aligned with an image of a visual device. The at least one input 805 of FIG. 8D or in another embodiment the at least one processed input can be labeled by the group of labelers. In an embodiment, the at least one input 805 can be at least one processed input to facilitate the labeler's task. The at least one processed input can be a virtual try-on action 852, where the face image and the image of the visual device are manipulated to make the face appear to be wearing the visual device. The group of labelers can be a first category of labelers 846 and can be as many as N categories of labelers 847, where each category reflects a particular group of people defined, for example, by traditional consumer segmentation criteria (described in FIG. 8A). Each label from the first labeler category 846 to the Nth labeler category 847 for the virtual try-on actions 852 of the face images "wearing" the visual device of the subset can include a binary visual device fit 841, an overall visual device match score 842, and a visual device match score by reference 843, as described in Fig. 5. These labels, as determined by the labeler category, can be deployed as "ground truth" or as training data during training of the neural network of ML-VEST, and can define the output layer of the neural network. In one example, the above labeling process is repeated for the virtual try-on actions 852 of each combination of face images "wearing" the visual device of the multiple visual devices in the training database.

[0042] In view of Fig. 7D, Fig. 8E reflects labeling where the group of labelers can be one of multiple categories of labelers and where at least one input 805 can include an image of a person's face. In an embodiment, to facilitate the labeler's task, at least one input 805 can be a virtual try-on action 852, where the image of the face and the image of the visual device of the subset of images of the visual device selected from the training database 832 are manipulated and combined to make the face appear to be "wearing" the visual device. The group of labelers can be a first category of labelers 846 and can be as many as N categories of labelers 847, where each category reflects a particular group of people defined, for example, by traditional consumer segmentation criteria (described in Fig. 8A). Each label from the labeler's first category 846 to the labeler's Nth category 847 for the virtual try-on actions 852 of the facial images "wearing" the visual device of the subset may include a binary visual device fit 841, an overall visual device match score 842, and a visual device match score by reference 843, as described in FIG. 5. These labels, as determined by the labeler's category, may be deployed as "ground truth" or as training data during training of the neural network of ML-VEST, and may define the output layer of the neural network. The above labeling process may be repeated for each one of the virtual try-on actions 852 of the facial images of people "wearing" the visual device of the subset of the visual device images of the training database from visual device 1 or VE1 to VE2 to VE N.

[0043] According to an embodiment, referring to FIG. 9A, the group of labelers 945 may include multiple categories of labelers, one of which is the category of expert labelers 948. The above category of expert labelers 948 may be heterogeneous or may be divided into subcategories of expert labelers. For example, the category of expert labelers 948 may include eye care specialists, beauticians, and physiognomists, among others. In another case, the categories of eye care specialists, beauticians, and physiognomists, etc. may include subcategories and provide occupation-specific labels for combinations of faces and visual instruments.

[0044] To this end, as shown in the exemplary at least one input and at least one processed input of Figs. 9B-9F, the inclusion of an expert labeler category of the labeler defines additional characteristics of the at least one input to be taken into account during labeling. For example, Fig. 9B describes a case where at least one input 905 of training of ML-VEST, obtained from a training database, includes a face image and, separately, an image 908 of a visual device. Furthermore, due to the case of an expert labeler who is an eye care professional, the face image of the at least one input can be further associated with visual measurements 949, such as, for example, prescription and interpupillary distance, among others. Similarly, Fig. 9C illustrates at least one input 905 of training of ML-VEST including a face image and visual device attributes 951, which are identified from an input preparation process substantially similar to the process described for Fig. 3. Furthermore, similar to Fig. 9B, the faces of people in the at least one input 905 can be associated with visual measurements 949, such as those described above. In view of FIG. 9B, FIG. 9D illustrates an example in which at least one input 905 of the training of ML-VEST includes an image 909 of a face wearing a visual device. Furthermore, the at least one input 905 of FIG. 9D can be further associated with a visual measurement 949 as described above. Referring to FIG. 9E, the at least one input 905 of ML-VEST can include morphological features 953 of the face image and visual device attributes 951 of a plurality of visual devices of the training database. Furthermore, the morphological features 953 of the face image can be associated with a visual measurement 949 as described above. Referring to FIG. 9F, the at least one input 905 of ML-VEST can include morphological features 953 of the face image and visual device images of the training database. Furthermore, the morphological features 953 of the face image can be associated with a visual measurement 949 as described above.

[0045] FIG. 10A reflects the labeling process described in view of FIG. 5, where at least one input 1005 includes a facial image and, separately, an image of a visual instrument. Additionally, FIG. 10A utilizes a group of labelers 1045, which are expert labelers 1048 and may be eye care professionals in one example, so that at least one input 1005 may include visual measurements 1049 associated with the facial image. As suggested, at least one input 1005 of FIG. 10A, or in another embodiment, at least one processed input, may be labeled by a category of expert labelers 1048 of the group of labelers 1045. Each label of the category of expert labelers 1048 may include a binary visual instrument fit 1041, an overall visual instrument match score 1042, and a visual instrument match score by criteria 1043, as described in FIG. 5. Additionally, these labels may include comments about the fit of the visual instrument and, in the case of eye care professionals, comments in the context of the visual measurements 1049 directed to the visual measurements 1049. These labels, as determined by expert labeler categories 1048, can be deployed as "ground truth" or as training data during training of ML-VEST's neural network and can define the output layer of the neural network. In one example, the labeling process described above is repeated for each combination of face image and vision device of the multiple vision devices in the training database.

[0046] FIG. 10B reflects the labeling process described in view of FIG. 5, where at least one input 1005 includes an image of a face wearing a visual device. Additionally, FIG. 10B utilizes a group of labelers 1049, which are expert labelers 1048 and may be eye care professionals in one example, so that at least one input 1005 may include visual measurements 1049 associated with the image of the face wearing the visual device. As suggested, at least one input 1005 of FIG. 10B, or in another embodiment, at least one processed input, may be labeled by an expert labeler category 1048 of a group of labelers 1045. Each label of the expert labeler category 1048 may include a binary visual device fit 1041, an overall visual device match score 1042, and a visual device match score by criteria 1043, as described in FIG. 5. Additionally, these labels may include comments about the fit of the visual device and, in the case of eye care professionals, comments in the context of the visual measurements 1049 directed to the visual measurements 1049. These labels, as determined by expert labeler categories 1048, can be deployed as "ground truth" or as training data during training of ML-VEST's neural network and can define the output layer of the neural network. The above labeling process can be repeated for each one of the images of the vision-equipped faces of the multiple images in the training database.

[0047] FIG. 10C reflects the labeling process described in view of FIG. 5, where the at least one input 1005 includes an image of a face and, separately, an image of a visual device. The at least one input 1005 of FIG. 10C, or in another embodiment, the processed at least one input, can be labeled by a group of labelers 1045, which are expert labelers 1048. In an embodiment, to facilitate the labeler's task, the at least one input 1005 can be the processed at least one input. The processed at least one input can be a virtual try-on action 1052, where the image of the face and the image of the visual device are manipulated to make it appear as if the face is wearing the visual device. The labels of the expert labeler category 1048 can include a binary visual device fit 1041, an overall visual device match score 1042, and a benchmark visual device match score 1043, as described in FIG. 5. These labels, as determined by expert labeler categories 1048, can be deployed as "ground truth" or as training data during training of ML-VEST's neural network, and can define the output layer of the neural network. The above labeling process can be repeated for each combination of facial images "wearing" a vision device from among the multiple vision devices in the training database 852.

[0048] FIG. 10D reflects the labeling process described in view of FIG. 5, where the at least one input 1005 includes a face image and visual measurements 1049 associated with the face image. In an embodiment, to facilitate the labeler's task, the at least one input 1005 can be a processed at least one input, where the processed at least one input is a virtual try-on action 1052, where the face image and the visual device images of the subset of visual device images selected from the training database 1032 are manipulated and combined to make the face appear to be "wearing" the visual device. The labels of the expert labeler category 1048 can include a binary visual device fit 1041, an overall visual device match score 1042, and a visual device match score by criteria 1043, as described in FIG. 5. These labels, as determined by the expert labeler category 1048, can be deployed as "ground truth" or as training data during training of the neural network of ML-VEST, and can define the output layer of the neural network. In one example, the above labeling process is repeated for each virtual try-on action 1052 of facial images of people "wearing" a visual device from a subset of visual device images in the training database from visual device 1 or VE1 to VE2 to VE N.

[0049] Each of the above labeling schemes of the labeling process can be implemented within the training process of ML-VEST introduced in Figure 4A. In particular, with reference to the use case described with respect to Figure 3, an implementation of ML-VEST can proceed as shown in Figures 11A-11R. Where appropriate in this specification and drawings, "vision equipment" is abbreviated as "VE," which can be understood to be interchangeable.

[0050] Referring to FIG. 11A, which is an exemplary embodiment of Case (1), at least one input 1105 can be passed to an input preparation process 1110 before being input to a neural network 1115. The at least one input 1105 can include a face image and a visual device image, where the visual device image is provided together with the face image or selected from a plurality of visual devices in a database. The input preparation process 1110 can include image processing or manual measurement to derive morphological features 1153 from the face image and visual device attributes 1151 from the visual device image, according to an embodiment. Furthermore, the visual device attributes 1151 can be obtained upon request from a database. The processed at least one input can be sent to an input layer 1116 of the neural network 1115, where the neural network 1115 is applied. The structure of the neural network 1115 can include data preparation (including homogenization, regularization, etc.) and a fully connected neural network and / or a convolutional + fully connected neural network. During the labeling process, as defined by a group of labelers, the output layer 1117 of the neural network 1115 reflects the prediction of the neural network 1115 about the labels. This prediction can be a fitness measure 1120 generated by the neural network 1115 for at least one combination of inputs 1105.

[0051] As above, referring now to FIG. 11B, an exemplary embodiment of case (1), at least one input 1105 can be passed to an input preparation process 1110 before being input to a neural network 1115. The at least one input 1105 can include an image of a face wearing a vision device. The input preparation process 1110 can include image processing or manual measurement to derive morphological features 1153 from the image of the face and the image of the vision device, and vision device attributes 1151 from the image of the vision device, respectively, according to an embodiment. The processed at least one input can be sent to an input layer 1116 of the neural network 1115, where the neural network 1115 is applied. The output layer 1117 reflects the prediction of the neural network 1115 for the labels, as defined by a group of labelers. This prediction is a fitness measure 1120 generated by the neural network 1115 for each combination of at least one input 1105.

[0052] FIG. 11C, an exemplary embodiment of Case (2), provides a schematic diagram of ML-VEST, where at least one input 1105 includes a face image and a visual device image, where the visual device image is provided together with the face image or selected from a plurality of visual devices in a database. The face image can be passed directly to the neural network 1115. As before, the visual device image can be passed to an input preparation process 1110 before being sent to the neural network 1115. The structure of the neural network 1115 can include data preparation (including homogenization, regularization, etc.) and a fully connected neural network and / or a convolutional + fully connected neural network. To that end, the visual device image can be prepared through image processing and manual measurement to generate visual device attributes 1151. Furthermore, said visual device attributes 1151 can be obtained through a request from a database. Unlike the previous embodiment, the input preparation process 1110 applied to the face image can be external to the neural network 1115 or integrated with the neural network 1115. For example, a convolutional neural network 1112 may be applied to a face image to perform feature extraction and prepare the image for input to an input layer of the neural network 1115 where it meets the processed at least one input of the image of the vision device. After preparing both at least one 1105, the processed at least one input can be sent to the input layer of the neural network 1115 where the neural network 1115 is applied. The output layer 1117 reflects the prediction of the neural network 1115 for the labels, as indicated by the group of labelers. This prediction is the fitness measure 1120 generated by the neural network 1115 for each combination of at least one input 1105.

[0053] FIG. 11D, an exemplary embodiment of Case (3), provides a schematic diagram of ML-VEST, where at least one input 1105 includes a face image and a visual device image, where the visual device image is provided together with the face image or selected from a plurality of visual devices in a database. The visual device image can be passed directly to the neural network 1115. As before, the facial image can be passed to an input preparation process 1110 before being sent to the neural network 1115. To that end, the facial image can be prepared through image processing and manual measurements to generate morphological features 1153. Unlike the previous embodiment, the input preparation process 1110 applied to the visual device image can be external to the neural network 1115 or can be integrated with the neural network 1115. For example, a convolutional neural network 1112 can be applied to the visual device image to perform feature extraction and prepare the image for input to the input layer of the neural network 1115 (where it meets the processed input image of the face). After preparing both at least one input 1105, the processed at least one input can be sent to an input layer of a neural network 1115 where the neural network 1115 is applied. The structure of the neural network 1115 can include data preparation (including homogenization, regularization, etc.) and a fully connected neural network and / or a convolutional + fully connected neural network. The output layer 1117 reflects the prediction of the neural network 1115 about the labels, as indicated by the group of labelers. This prediction is the fitness measure 1120 generated by the neural network 1115 for each combination of at least one input 1105.

[0054] FIG. 11E, an exemplary embodiment of Case (4), provides a schematic diagram of ML-VEST, where at least one input 1105 includes a face image and a visual device image, where the visual device image is provided together with the face image or selected from a plurality of visual devices in a database. At least one input 1105 can be passed directly to a neural network 1115, where convolution is performed. As before, the input preparation process 1110 applied to the at least one input can be external to the neural network 1115 or can be integrated with the neural network 1115. For example, a convolutional neural network 1112 can be applied to at least one input 1105 including a visual device image and a face image to perform feature extraction and prepare the image for input to an input layer of the neural network 1115. After preparing both at least one input 1105 by convolution, the processed at least one input can be sent to an input layer of the neural network 1115, where the neural network 1115 is applied. The structure of the neural network 1115 may include data preparation (including homogenization, regularization, etc.) and a fully connected neural network and / or a convolutional + fully connected neural network. The output layer reflects the prediction of the neural network 1115 about the labels, as indicated by the group of labelers. This prediction is the fitness measure 1120 generated by the neural network 1115 for each combination of at least one input 1105.

[0055] Referring to FIG. 11F, which is an exemplary embodiment of Case (5), and considering FIG. 11A, the at least one input 1105 can be passed to an input preparation process 1110 before being input to the neural network 1115. The at least one input 1105 can include a face image and a visual device image, where the visual device image is provided together with the face image or is selected from a plurality of visual devices in a database. In addition, the at least one input 1105 can include visual measurements 1149 corresponding to the face image. The input preparation process 1110 can include image processing or manual measurements to derive morphological features 1153 from the face image and visual device attributes 1151 from the visual device image, according to an embodiment. Furthermore, the visual device attributes 1151 can be obtained upon request from a database. The processed at least one input can be sent to an input layer 1116 of the neural network 1115, where the neural network 1115 is applied. During the labeling process, the output layer 1117 of the neural network 1115 reflects the prediction of the neural network 1115 about the labels, as defined by a group of labelers. In one example, the group of labelers can be expert labelers. This prediction can be a fitness measure 1120 generated by the neural network 1115 for at least one combination of inputs 1105.

[0056] With reference to FIG. 11G, an exemplary embodiment of Case (5), and considering FIG. 11B, at least one input 1105 can be passed to an input preparation process 1110 before being input to the neural network 1115. The at least one input 1105 can include an image of a face wearing a visual device. Furthermore, the at least one input 1105 can include visual measurements 1149 corresponding to the image of the face wearing the visual device. The input preparation process 1110 can include image processing or manual measurements to derive morphological features 1153 from the image of the face and the image of the visual device and visual device attributes 1151 from the image of the visual device, respectively, according to an embodiment. The processed at least one input together with the visual measurements 1149 can be sent to an input layer 1116 of the neural network 1115, where the neural network 1115 is applied. The output layer 1117 reflects the predictions of the neural network 1115 about the labels, as defined by a group of labelers, in an embodiment expert labelers. The prediction is a measure of fitness 1120 generated by the neural network 1115 for each combination of at least one input 1105 .

[0057] FIG. 11H, an exemplary embodiment of Case (6), provides a schematic diagram of ML-VEST in which at least one input 1105 includes a face image and a visual device image, and the visual device image is provided together with the face image or selected from a plurality of visual devices in a database, in consideration of FIG. 11C. The face image of a person having a facial image can be directly passed to the neural network 1115. In addition, the at least one input 1105 includes visual measurements 1149 corresponding to the face image. As before, the visual device image can be passed to an input preparation process 1110 before being sent to the neural network 1115. To that end, the visual device image can be prepared through image processing and manual measurements to generate visual device attributes 1151. Furthermore, said visual device attributes 1151 can be obtained through a request from a database. Unlike the previous embodiment, the input preparation process 1110 applied to the face image can be external to the neural network 1115 or can be integrated with the neural network 1115. For example, a convolutional neural network 1112 may be applied to a face image to perform feature extraction and prepare the image for input to an input layer of the neural network 1115 (where it meets the processed at least one input of the image of the visual instrument). After preparing both at least one 1105, the processed at least one input can be sent along with visual measures 1149 to the input layer of the neural network 1115 where the neural network 1115 is applied. The output layer 1117 reflects the prediction of the neural network 1115 for the label, as indicated by a group of labelers, in an embodiment expert labelers. This prediction is the fitness measure 1120 generated by the neural network 1115 for each combination of at least one input 1105.

[0058] Considering FIG. 11D, FIG. 11I, an exemplary embodiment of Case (7), provides a schematic diagram of ML-VEST, where at least one input 1105 includes a face image and a visual device image, where the visual device image is provided together with the face image or selected from a plurality of visual devices in a database. The visual device image can be passed directly to the neural network 1115. In addition, the at least one input 1105 includes visual measurements 1149 corresponding to the face image. As before, the face image can be passed to an input preparation process 1110 before being sent to the neural network 1115. To that end, the face image can be prepared via image processing and manual measurements to generate morphological features 1153. Unlike the previous embodiment, the input preparation process 1110 applied to the visual device image can be external to the neural network 1115 or can be integrated with the neural network 1115. For example, a convolutional neural network 1112 may be applied to the image of the vision device to perform feature extraction and prepare the image for input to an input layer of the neural network 1115 (where it meets the processed input image of the face). After preparing both of the at least one input 1105, the processed at least one input can be sent to the input layer of the neural network 1115, where the neural network 1115 is applied. The output layer 1117 reflects the prediction of the neural network 1115 for the label, as indicated by a group of labelers, in an embodiment expert labelers. This prediction is the fitness measure 1120 generated by the neural network 1115 for each combination of the at least one input 1105.

[0059] FIG. 11J, an exemplary embodiment of Case (8), provides a schematic diagram of ML-VEST in which at least one input 1105 includes a face image and a visual device image, and the visual device image is provided along with the face image or selected from a database of multiple visual devices, in consideration of FIG. 11E. The at least one input 1105 can be passed directly to the neural network 1115, where the convolution is performed. In addition, the at least one input 1105 can include visual measurements 1149 corresponding to the face image. As before, the input preparation process 1110 applied to the at least one input 1105 can be external to the neural network 1115 or can be integrated with the neural network 1115. For example, a convolutional neural network 1112 can be applied to the at least one input 1105 including the visual device image and the face image to perform feature extraction and prepare the image for input to the input layer of the neural network 1115. After preparing both at least one input 1105 by convolution, the processed at least one input together with visual measures 1149 can be sent to an input layer of a neural network 1115 where the neural network 1115 is applied. The output layer 1117 reflects the prediction of the neural network 1115 about the labels, as indicated by a group of labelers, in an embodiment expert labelers. This prediction is the fitness measure 1120 generated by the neural network 1115 for each combination of at least one input 1105.

[0060] 11K reflects a schematic diagram of ML-VEST in which at least one input 1105 includes a facial image, in view of FIG. 8E. In such a process reflecting Example (9), the at least one input 1105 may be passed to a neural network 1115 having an architecture that allows the at least one input 1105 to be evaluated against each visual device of a subset 1132 of visual device images. In an embodiment, the at least one input 1105 may be passed to an input preparation process 1110 before being passed to the neural network 1115. To that end, the facial image may be prepared via image processing and manual measurement to generate morphological features 1153.

[0061] Unlike the previous embodiment, the subset 1132 of images of visual devices retrieved from the database 1155 is not provided to the neural network 1115 as the at least one input 1105. Instead, the neural network 1115 is applied to the at least one input 1105, the neural network 1115 being trained in part based on the subset 1132 of images of visual devices. According to an embodiment, in the context of training the neural network 1115, a pre-processing of each image of a visual device of the subset 1132 of images of visual devices from the database 1155 is required.

[0062] For each visual device i in the subset 1132 of visual device images from the database 1155, a statistical match score can be calculated in light of the morphological feature j derived for each image of the face. The statistical match score is the proportion p of labelers that (1) gave the same binary score (example binary score 0 or 1) to visual device image i for morphological feature j, (2) gave the same match score (example match score from 0 to N) for morphological feature j in the subset 1132 of visual device images from the database 1155, or (3) gave the same match score or had the same rank (example match score from 0 to N for each item) by each item of a criteria list specified for visual device image i for morphological feature j in the subset 1132 of visual device images from the database 1155.ji For each of the above situations, the proportion p of a given morphological feature j ji The following vectors associated with the morphological feature j of the face image can be obtained. One vector can be a vector of N binary values ​​{0,1}, each corresponding to a match of the morphological feature j of the face image with the image i of the subset 1132 of the images of the vision device. The second vector can be a vector of integer values ​​from 0 to X, each integer value corresponding to a match score of the image i of the subset 1132 of the images of the vision device to the face image. The third vector can be a vector of N lists l of M integer values ​​from 0 to X, each integer value of each list l corresponding to a match score of the image i of the subset 1132 of the images of the vision device to the face image with each rule of the set of M matching rules. Then, in view of the pre-processing described above, training can begin. At least one input to the training can be a morphological feature, and the neural network can be configured as a combinatorial neural network with a convolutional layer and a fully connected layer. In addition, the activation function can be a morphological feature j of the face image, including a rectified linear unit (ReLU), p ji The weights associated can be any kind of standard activation function. The associated vector containing the match information can be called the target vector. Training of the neural network can be successfully done on the entire target vector or on components of the target vector that specifically train select neurons.

[0063] Returning now to FIG. 11K, the output layer 1117 of the neural network 1115 reflects the neural network's 1115 prediction of the match measure 1120. The match measure 1120 can be a match score 1121, which can be one of the vectors described above. In particular, the match score 1121 can be (1) a vector of N binary values ​​{0,1}, each corresponding to a match of a morphological feature j of a facial image with an image i of a subset 1132 of N images of the vision device, (2) a vector of integer values ​​from 0 to X, each corresponding to a match score 1121 of an image i of the subset 1132 of images of the vision device to a facial image, or (3) a vector of N lists l of M integer values ​​from 0 to X, each integer value of each list l corresponding to a match score 1121 of an image i of the subset 1132 of images of the vision device to a facial image for each rule of a set of M match rules. In an embodiment, a prediction of the match score 1121 may be generated by the neural network 1115 for each combination of at least one input 1105 and a subset 1132 of images of visual devices from a database 1155, the database 1155 including consecutive images of visual devices "VE1", "VE2", and up to "VE N". In an embodiment, the database 1155 may be a preselected subset of images of visual devices available at a retail store or a preselected subset of images of visual devices previously identified as suitable for particular characteristics associated with the user.

[0064] FIG. 11L reflects a schematic diagram of ML-VEST in which at least one input 1105 includes a face image, in view of FIG. 8E. In such a process reflecting case (9), the at least one input 1105 can be passed to a neural network 1115 having an architecture that allows the at least one input 1105 to be evaluated in view of morphological features and corresponding visual device attributes associated with each visual device of the subset of visual device images. In an embodiment, the at least one input 1105 can be passed to an input preparation process 1110 before being passed to the neural network 1115. To that end, the facial image can be prepared through image processing and manual measurements to generate morphological features 1153. Unlike the previous embodiment, the visual device or visual device attributes associated with each visual device of the subset of visual device images are not provided to the neural network 1115 as the at least one input 1105. Instead, a neural network 1115 is applied to at least one input 1105, and the neural network 1115 is trained in part based on the morphological features and corresponding visual device attributes associated with each visual device of the subset of visual device images. According to an embodiment, in the context of training the neural network 1115, a pre-processing 1133 of each visual device image of the subset of visual device images from the database in view of the morphological features of the facial images is required. The pre-processing 1133 may include, for example, a pre-processing step of: ji coordinate information defining the best match for the morphological feature j, which is the best match for visual device i or VEi, and the proportion p of Np labelers that give the highest score to the coordinate pair {visual device i, morphological feature j}. ji Includes.

[0065] To this end, for each image of visual device i of the subset of images of the visual device, a statistical match score can be calculated in light of the morphological features of the facial image. The statistical match score is the proportion p of labelers that (1) associated visual device i with morphological feature j (binary score 1), (2) gave the same match score or had the same rank with visual device i of the subset of images of the visual device with respect to morphological feature j (example match scores from 0 to N), or (3) gave the same match score or had the same rank with respect to facial information j by each item of a specified criteria list with visual device i of the subset of images of the visual device (example match scores from 0 to N for each item). ji For simplicity, incomplete datasets can be ignored and it can be assumed that the labelers only gave a match score (0 to N) for each visual device i from the subset of visual device images for morphological feature j. Furthermore, for each entry of morphological feature j, only the visual device attribute with the highest score for each labeler is retained. In view of the above, we can compare each visual device i from the subset of visual device images for all morphological features F ji We can obtain a matrix that relates the morphological features F ji is the proportion of labelers p ji It may include the best match by

[0066] Then, in view of the above-mentioned pre-processing, training can begin. At least one input to the training can be morphological features and visual measurements, and the neural network can be configured as a combinatorial neural network with a convolutional layer and a fully connected layer. The fully connected layer is configured for embedding. The embedding layer 1118, which is a fully connected layer of D neurons, contains a vector representation of the morphological features in the vector space for each visual device i identified during pre-processing. Each cluster i of the D-dimensional vector space 1119 contained within the embedding layer 1118 represents a visual device, and each morphological feature can be represented by D vector coordinates.

[0067] During training, random sampling can be performed to randomly select a certain number of morphological feature pairs, and the morphological feature pairs are {F ki ,F li}. An example pair is F ki and F li is the ratio p ki and p li It is judged that the visual device i with ki ,p ki ) and f(F li ,p li ), where f is the activation function. ki and F ii is the proportion p ki and p li Then, backpropagation is performed by finding two activation functions f(F ki ,p ki ) and f(F li ,p li ), where f is the activation function.

[0068] Returning now to FIG. 11L, the output layer 1117 of the neural network 1115 reflects the prediction of the neural network 1115 on a fitness measure 1120. The fitness measure 1120 is a function of the morphological features F in a dimensional vector space 1119. ij The coordinates of the morphological feature F ij Post-processing of the coordinates of can include (1) calculating the centroid of each cluster i in the D-dimensional vector space 1119, and (2) calculating the distance between the output coordinates and the centroid of each cluster i, thereby generating a vector containing the rank of the visual device (centroid of each cluster i) from closest to furthest to the output coordinate. In an embodiment, the morphological feature F ijA prediction of the coordinates of may be generated by the neural network 1115 for each of the at least one input 1105 considered in light of the morphological features and corresponding visual device attributes of a subset of images of visual devices from a database on which the neural network 1115 was trained. In an embodiment, the database may be a preselected subset of images of visual devices available at a retail store or a preselected subset of images of visual devices previously identified as suitable for particular characteristics associated with the user.

[0069] FIG. 11M reflects a schematic diagram of ML-VEST where at least one input 1105 includes a facial image. In such a process reflecting Example (10), the at least one input 1105 can be passed to a neural network 1115 having an architecture that allows the at least one input 1105 to be evaluated against each visual device of a subset 1132 of visual device images. In an embodiment, the at least one input 1105 can be passed directly to the neural network 1115 where the convolution is performed. The convolution can be performed, for example, by a convolutional neural network 1112 applied to the at least one input 1105 including a facial image to perform feature extraction and prepare the facial image for input to an input layer of the neural network 1115.

[0070] Unlike the previous embodiment, the subset 1132 of images of visual devices retrieved from the database 1155 is not provided to the neural network 1115 as the at least one input 1105. Instead, the neural network 1115 is applied to the at least one input 1105, the neural network 1115 being trained in part based on the subset 1132 of images of visual devices. According to an embodiment, in the context of training the neural network 1115, a pre-processing of each image of a visual device of the subset 1132 of images of visual devices from the database 1155 is required.

[0071] For each visual device i in the subset 1132 of visual device images from the database 1155, a statistical match score can be calculated in light of the morphological feature j derived for each image of the face. The statistical match score is the proportion p of labelers that (1) gave the same binary score (example binary score 0 or 1) to visual device image i for morphological feature j, (2) gave the same match score (example match score from 0 to N) for morphological feature j in the subset 1132 of visual device images from the database 1155, or (3) gave the same match score or had the same rank (example match score from 0 to N for each item) by each item of a criteria list specified for visual device image i for morphological feature j in the subset 1132 of visual device images from the database 1155. ji For each of the above situations, the proportion p of a given morphological feature j ji The following vectors associated with the morphological feature j of the face image can be obtained. One vector can be a vector of N binary values ​​{0,1}, each corresponding to a match of the morphological feature j of the face image with the image i of the subset 1132 of the images of the vision device. The second vector can be a vector of integer values ​​from 0 to X, each integer value corresponding to a match score of the image i of the subset 1132 of the images of the vision device to the face image. The third vector can be a vector of N lists l of M integer values ​​from 0 to X, each integer value of each list l corresponding to a match score of the image i of the subset 1132 of the images of the vision device to the face image with each rule of the set of M matching rules. Then, in view of the pre-processing described above, training can begin. At least one input to the training can be a morphological feature, and the neural network can be configured as a combinatorial neural network with a convolutional layer and a fully connected layer. In addition, the activation function can be a morphological feature j of the face image, including a rectified linear unit (ReLU), p jiThe weights associated can be any kind of standard activation function. The associated vector containing the match information can be called the target vector. Training of the neural network can be successfully done on the entire target vector or on components of the target vector that specifically train select neurons.

[0072] Returning now to FIG. 11M, the output layer 1117 of the neural network 1115 reflects the neural network's 1115 prediction of a match measure 1120. The match measure 1120 can be a match score 1121, which can be one of the vectors described above. In particular, the match score 1121 can be (1) a vector of N binary values ​​{0,1}, each corresponding to a match of a morphological feature j of a facial image with an image i of a subset 1132 of N images of the vision device, (2) a vector of integer values ​​from 0 to X, each corresponding to a match score 1121 of an image i of the subset 1132 of images of the vision device to a facial image, or (3) a vector of N lists l of M integer values ​​from 0 to X, each integer value of each list l corresponding to a match score 1121 of an image i of the subset 1132 of images of the vision device to a facial image for each rule of a set of M match rules. In an embodiment, a prediction of the match score 1121 may be generated by the neural network 1115 for each combination of at least one input 1105 and a subset 1132 of images of visual devices from a database 1155, the database 1155 including consecutive images of visual devices "VE1", "VE2", and up to "VE N". In an embodiment, the database 1155 may be a preselected subset of images of visual devices available at a retail store or a preselected subset of images of visual devices previously identified as suitable for particular characteristics associated with the user.

[0073] FIG. 11N reflects a schematic diagram of ML-VEST where at least one input 1105 includes a facial image. In such a process reflecting Example (10), the at least one input 1105 can be passed to a neural network 1115 having an architecture that allows the at least one input 1105 to be evaluated against morphological features and corresponding visual device attributes associated with each visual device of a subset of visual device images. In an embodiment, the at least one input 1105 can be passed directly to the neural network 1115 where the convolution is performed. The convolution can be performed, for example, by a convolutional neural network 1112 applied to the at least one input 1105 including a facial image to perform feature extraction and prepare the facial image for input to an input layer of the neural network 1115.

[0074] Unlike the previous embodiment, the visual device attributes and corresponding morphological features associated with each visual device of the subset of visual device images are not provided to the neural network 1115 as at least one input 1105. Instead, the neural network 1115 is applied to the at least one input 1105, and the neural network 1115 is trained in part based on the morphological features and corresponding visual device attributes associated with each visual device of the subset of visual device images. According to an embodiment, in the context of training the neural network 1115, a pre-processing 1133 of each visual device image of the subset of visual device images from the database in view of the morphological features of the facial images is required. The pre-processing 1133 may include: ji coordinate information defining the best match for visual device i, face image j, and the proportion p of Np labelers that give the highest score to the coordinate pair {visual device i, face image j}. ji Includes.

[0075] To this end, for each image of visual device i in the subset of images of the visual device, a statistical match score can be calculated in light of the face images 1105 in the face images. The statistical match score is the proportion p of labelers that (1) associated visual device i with face image j (binary score 1), (2) gave the same match score or had the same rank for visual device i in the subset of images of the visual device with respect to face image j in the face images (example match scores 0 to N), or (3) gave the same match score or had the same rank for visual device i in the subset of images of the visual device with respect to face image j by each item of a specified criteria list (example match scores 0 to N for each item). ji For simplicity, the incomplete dataset can be ignored and it can be assumed that the labelers only gave a match score (0 to N) for each visual device i from the subset of visual device images to face image j. Furthermore, for each entry of face image j, only the visual device attributes with the highest scores for each labeler are retained. In view of the above, we can apply each visual device i from the subset of visual device images to all face images F ji We can obtain the matrix that relates to the matrix F ji is the proportion of labelers p ji It may include the best match by

[0076] Now, in light of the pre-processing described above, training can begin. At least one input to the training can be a face image and visual measurements, and the neural network can be configured as a combinatorial neural network with a convolutional layer and a fully connected layer. The fully connected layer is configured for embedding. The embedding layer 1118, which is a fully connected layer of D neurons, contains a vector representation of the face image in vector space for each visual device i identified during pre-processing. Each cluster i of the D-dimensional vector space 1119 contained within the embedding layer 1118 represents a visual device, and each face image can be represented by D vector coordinates.

[0077] During training, random sampling can be performed to randomly select a certain number of face image pairs, and the morphological feature pairs are {Fki ,F li}. An example pair is F ki and F li is the ratio p ki and p li It is judged that the visual device i with ki ,p ki ) and f(F li ,p li ), where f is the activation function. ki and F li is the proportion p ki and p li Then, backpropagation is performed by finding two activation functions f(F ki ,p ki ) and f(F li ,p li ), where f is the activation function.

[0078] Returning now to FIG. 11N, the output layer 1117 of the neural network 1115 reflects the neural network's 1115 prediction of a fit measure 1120. The fit measure 1120 is a function of the face image F ij The coordinates of the face image F ij Post-processing of the coordinates of can include (1) calculating the centroid of each cluster i in the D-dimensional vector space 1119, and (2) calculating the distance between the output coordinates and the centroid of each cluster i, thereby generating a vector containing the rank of the visual instruments (centroids of each cluster i) from closest to furthest to the output coordinate. ijA prediction of the coordinates of may be generated by the neural network 1115 for each of the at least one input 1105 in view of the morphological features and visual device attributes associated with each image of the subset of images of visual devices from the database. In an embodiment, the database may be a preselected subset of images of visual devices available at a retail store or a preselected subset of images of visual devices previously identified as suitable for particular characteristics associated with the user.

[0079] FIG. 11O reflects a schematic diagram of ML-VEST where at least one input 1105 includes a facial image. In such a process reflecting Example (11), the at least one input 1105 may be passed to a neural network 1115 having an architecture that allows the at least one input 1105 to be evaluated against each visual device of a subset 1132 of visual device images. In an embodiment, the at least one input 1105 may be passed to an input preparation process 1110 before being passed to the neural network 1115. To that end, the facial image may be prepared via image processing and manual measurements to generate morphological features 1153. In addition to the above, the at least one input 1105 may include visual measurements 1149 corresponding to the facial image.

[0080] Unlike the previous embodiment, the subset 1132 of images of visual devices retrieved from the database 1155 is not provided to the neural network 1115 as the at least one input 1105. Instead, the neural network 1115 is applied to the at least one input 1105, the neural network 1115 being trained in part based on the subset 1132 of images of visual devices. According to an embodiment, in the context of training the neural network 1115, a pre-processing of each image of a visual device of the subset 1132 of images of visual devices from the database 1155 is required.

[0081] For each visual device i in the subset 1132 of visual device images from the database 1155, a statistical match score can be calculated in light of the morphological feature j derived for each image of the face. The statistical match score is the proportion p of labelers that (1) gave the same binary score (example binary score 0 or 1) to visual device image i for morphological feature j, (2) gave the same match score (example match score from 0 to N) for morphological feature j in the subset 1132 of visual device images from the database 1155, or (3) gave the same match score or had the same rank (example match score from 0 to N for each item) by each item of a criteria list specified for visual device image i for morphological feature j in the subset 1132 of visual device images from the database 1155. ji For each of the above situations, the proportion p of a given morphological feature j ji The following vectors associated with the facial image can be obtained: One vector can be a vector of N binary values ​​{0,1}, each corresponding to a match of a morphological feature j of a facial image with an image i of the subset 1132 of N images of the visual device. The second vector can be a vector of integer values ​​from 0 to X, each integer value corresponding to a match score of an image i of the subset 1132 of images of the visual device to a facial image. The third vector can be a vector of N lists l of M integer values ​​from 0 to X, each integer value of each list l corresponding to a match score of an image i of the subset 1132 of images of the visual device to a facial image with each rule of the set of M matching rules. Then, in view of the pre-processing described above, training can begin. At least one input to the training can be a morphological feature, and the neural network can be configured as a combinatorial neural network with a convolutional layer and a fully connected layer. In addition, the activation function can include a rectified linear unit (ReLU), p jiThe weights associated can be any kind of standard activation function. The associated vector containing the match information can be called the target vector. Training of the neural network can be successfully done on the entire target vector or on components of the target vector that specifically train select neurons.

[0082] Returning now to FIG. 11O, the processed at least one input, along with the visual measure 1149, may be passed to an input layer of a neural network 1115. An output layer 1117 of the neural network 1115 reflects the prediction of the neural network 1115 in terms of a fit measure 1120. The fit measure 1120 may be a match score 1121, which may be one of the vectors described above. In particular, the match score 1121 can be (1) a vector of N binary values ​​{0,1}, each corresponding to a match of a morphological feature j of a facial image with an image i of a subset 1132 of N images of the visual device, (2) a vector of integer values ​​from 0 to X, each corresponding to a match score 1121 of an image i of the subset 1132 of N images of the visual device to a facial image, or (3) a vector of N lists l of M integer values ​​from 0 to X, each integer value of each list l corresponding to a match score 1121 of an image i of the subset 1132 of N images of the visual device to a facial image for each rule of a set of M match rules. In an embodiment, a prediction of the match score 1121 can be generated by the neural network 1115 for each combination of at least one input 1105 and a subset 1132 of images of the visual device from a database 1155, the database 1155 including consecutive images of the visual device from "VE1", "VE2", and up to "VE N". In an embodiment, the database 1155 may be a preselected subset of images of visual devices available at a retail store or a preselected subset of images of visual devices previously identified as suitable for particular characteristics associated with a user.

[0083] FIG. 11P reflects a schematic diagram of ML-VEST where at least one input 1105 includes a facial image. In such a process reflecting Example (11), the at least one input 1105 can be passed to a neural network 1115 having an architecture that allows the at least one input 1105 to be evaluated against morphological features and corresponding visual device attributes associated with each visual device of a subset of visual device images. In an embodiment, the at least one input 1105 can be passed to an input preparation process 1110 before being sent to the neural network 1115. To that end, the facial image can be prepared via image processing and manual measurements to generate morphological features 1153. Additionally, the at least one input 1105 can include visual measurements 1149 corresponding to the facial image.

[0084] Unlike the previous embodiment, the visual device attributes and corresponding morphological features associated with each visual device of the subset of visual device images are not provided to the neural network 1115 as at least one input 1105. Instead, the neural network 1115 is applied to the at least one input 1105, and the neural network 1115 is trained in part based on the morphological features and corresponding visual device attributes associated with each visual device of the subset of visual device images. According to an embodiment, in the context of training the neural network 1115, a pre-processing 1133 of each visual device image of the subset of visual device images from the database in view of the morphological features of the facial images is required. The pre-processing 1133 may include: ji coordinate information defining the best match for visual device i, the morphological feature j, and the proportion p of Np labelers that give the highest score to the coordinate pair {visual device i, morphological feature j}. ji Includes.

[0085] To this end, for each image of visual device i of the subset of images of the visual device, a statistical match score can be calculated in light of the morphological features of the facial image. The statistical match score is the proportion of labelers p that (1) associate visual device i with morphological feature j (binary score 1), (2) give the same match score or have the same rank for visual device i of the subset of images of the visual device with respect to morphological feature j of the facial image (example match scores from 0 to N), or (3) give the same match score or have the same rank for each item of a specified criteria list for visual device i of the subset of images of the visual device with respect to facial information j (example match scores from 0 to N for each item). ji For simplicity, incomplete datasets can be ignored and it can be assumed that the labelers only gave a match score (0 to N) for each visual device i from the subset of visual device images for morphological feature j. Furthermore, for each entry of morphological feature j, only the visual device attribute with the highest score for each labeler is retained. In view of the above, we can compare each visual device i from the subset of visual device images for all morphological features F ji We can obtain a matrix that relates the morphological features F ji is the proportion of labelers p ji It may include the best match by

[0086] Then, in view of the above-mentioned pre-processing, training can begin. At least one input to the training can be morphological features and visual measurements, and the neural network can be configured as a combinatorial neural network with a convolutional layer and a fully connected layer. The fully connected layer is configured for embedding. The embedding layer 1118, which is a fully connected layer of D neurons, contains a vector representation of the morphological features in the vector space for each visual device i identified during pre-processing. Each cluster i of the D-dimensional vector space 1119 contained within the embedding layer 1118 represents a visual device, and each morphological feature can be represented by D vector coordinates.

[0087] During training, random sampling can be performed to randomly select a certain number of morphological feature pairs, and the morphological feature pairs are {F ki ,F li}. An example pair is F ki and F li is the ratio p ki and p li It is judged that the visual device i with ki ,p ki ) and f(F li ,p li ), where f is the activation function. ki and F li is the proportion p ki and p li Then, backpropagation is performed by finding two activation functions f(F ki ,p ki ) and f(F li ,p li ), where f is the activation function.

[0088] Returning now to FIG. 11P, the output layer 1117 of the neural network 1115 reflects the prediction of the neural network 1115 on a fitness measure 1120. The fitness measure 1120 is a function of the morphological features F ij The coordinates of the morphological feature F ij Post-processing of the coordinates of can include (1) calculating the centroid of each cluster i in the D-dimensional vector space 1119, and (2) calculating the distance between the output coordinates and the centroid of each cluster i, thereby generating a vector containing the rank of the visual device (centroid of each cluster i) from closest to furthest to the output coordinate. In an embodiment, the morphological feature F ijA prediction of the coordinates of may be generated by the neural network 1115 for at least one input 1105 in view of the morphological features and corresponding visual device attributes of a subset of images of visual devices. In an embodiment, the database may be a preselected subset of images of visual devices available at a retail store or a preselected subset of images of visual devices previously identified as suitable for particular characteristics associated with the user.

[0089] FIG. 11Q reflects a schematic diagram of ML-VEST where at least one input 1105 includes a facial image. In such a process reflecting Example (12), the at least one input 1105 can be passed to a neural network 1115 having an architecture that allows the at least one input 1105 to be evaluated against each visual device of a subset 1132 of visual device images. In an embodiment, the at least one input 1105 can be passed directly to the neural network 1115 where the convolution is performed. The convolution can be performed, for example, by a convolutional neural network 1112 applied to the at least one input 1105 including the facial image to perform feature extraction and prepare the facial image for input to an input layer of the neural network 1115. Additionally, the at least one input 1105 can include visual measures 1149 corresponding to the facial image.

[0090] Unlike the previous embodiment, the subset 1132 of images of visual devices retrieved from the database 1155 is not provided to the neural network 1115 as the at least one input 1105. Instead, the neural network 1115 is applied to the at least one input 1105, the neural network 1115 being trained in part based on the subset 1132 of images of visual devices. According to an embodiment, in the context of training the neural network 1115, a pre-processing of each image of a visual device of the subset 1132 of images of visual devices from the database 1155 is required.

[0091] For each visual device i in the subset 1132 of visual device images from the database 1155, a statistical match score can be calculated in light of the morphological feature j derived for each image of the face. The statistical match score is the proportion p of labelers that (1) gave the same binary score (example binary score 0 or 1) to visual device image i for morphological feature j, (2) gave the same match score (example match score from 0 to N) for morphological feature j in the subset 1132 of visual device images from the database 1155, or (3) gave the same match score or had the same rank (example match score from 0 to N for each item) by each item of a criteria list specified for visual device image i for morphological feature j in the subset 1132 of visual device images from the database 1155. ji For each of the above situations, the proportion p of a given morphological feature j ji The following vectors associated with the morphological feature j of the face image can be obtained. One vector can be a vector of N binary values ​​{0,1}, each corresponding to a match of the morphological feature j of the face image with the image i of the subset 1132 of the images of the vision device. The second vector can be a vector of integer values ​​from 0 to X, each integer value corresponding to a match score of the image i of the subset 1132 of the images of the vision device to the face image. The third vector can be a vector of N lists l of M integer values ​​from 0 to X, each integer value of each list l corresponding to a match score of the image i of the subset 1132 of the images of the vision device to the face image with each rule of the set of M matching rules. Then, in view of the pre-processing described above, training can begin. At least one input to the training can be a morphological feature, and the neural network can be configured as a combinatorial neural network with a convolutional layer and a fully connected layer. In addition, the activation function can be a morphological feature j of the face image, including a rectified linear unit (ReLU), p jiThe weights associated can be any kind of standard activation function. The associated vector containing the match information can be called the target vector. Training of the neural network can be successfully done on the entire target vector or on components of the target vector that specifically train select neurons.

[0092] Returning now to FIG. 11Q, the output layer 1117 of the neural network 1115 reflects the neural network's 1115 prediction of a match measure 1120. The match measure 1120 can be a match score 1121, which can be one of the vectors described above. In particular, the match score 1121 can be (1) a vector of N binary values ​​{0,1}, each corresponding to a match of a morphological feature j of a facial image with an image i of a subset 1132 of N images of the vision device, (2) a vector of integer values ​​from 0 to X, each corresponding to a match score 1121 of an image i of the subset 1132 of images of the vision device to a facial image, or (3) a vector of N lists l of M integer values ​​from 0 to X, each integer value of each list l corresponding to a match score 1121 of an image i of the subset 1132 of images of the vision device to a facial image for each rule of a set of M match rules. In an embodiment, a prediction of the match score 1121 may be generated by the neural network 1115 for each combination of at least one input 1105 and a subset 1132 of images of visual devices from a database 1155, the database 1155 including consecutive images of visual devices "VE1", "VE2", and up to "VE N". In an embodiment, the database 1155 may be a preselected subset of images of visual devices available at a retail store or a preselected subset of images of visual devices previously identified as suitable for particular characteristics associated with the user.

[0093] FIG. 11R reflects a schematic diagram of ML-VEST where at least one input 1105 includes a facial image. In such a process reflecting Example (12), the at least one input 1105 can be passed to a neural network 1115 having an architecture that allows the at least one input 1105 to be evaluated against morphological features and corresponding visual device attributes associated with each visual device of a subset of visual device images. In an embodiment, the at least one input 1105 can be passed directly to the neural network 1115 where the convolution is performed. The convolution can be performed, for example, by a convolutional neural network 1112 applied to the at least one input 1105 including the facial image to perform feature extraction and prepare the facial image for input to an input layer of the neural network 1115. Additionally, the at least one input 1105 can include visual measures 1149 corresponding to the facial image.

[0094] Unlike the previous embodiment, the visual device attributes and corresponding morphological features associated with each visual device of the subset of visual device images are not provided to the neural network 1115 as at least one input 1105. Instead, the neural network 1115 is applied to the at least one input 1105, and the neural network 1115 is trained in part based on the morphological features and corresponding visual device attributes associated with each visual device of the subset of visual device images. According to an embodiment, in the context of training the neural network 1115, a pre-processing 1133 of each visual device image of the subset of visual device images from the database in view of the morphological features of the facial images is required. The pre-processing 1133 may include: ji coordinate information defining the best match for visual device i, face image j, and the proportion p of Np labelers that give the highest score to the coordinate pair {visual device i, face image j}. ji Includes.

[0095] To this end, for each image of visual device i in the subset of images of the visual device, a statistical match score can be calculated in light of the face images 1105 in the face images. The statistical match score is the proportion p of labelers that (1) associated visual device i with face image j (binary score 1), (2) gave the same match score or had the same rank for visual device i in the subset of images of the visual device with respect to face image j in the face images (example match scores 0 to N), or (3) gave the same match score or had the same rank for visual device i in the subset of images of the visual device with respect to face image j by each item of a specified criteria list (example match scores 0 to N for each item). ji For simplicity, the incomplete dataset can be ignored and it can be assumed that the labelers only gave a match score (0 to N) for each visual device i from the subset of visual device images to face image j. Furthermore, for each entry of face image j, only the visual device attributes with the highest scores for each labeler are retained. In view of the above, we can apply each visual device i from the subset of visual device images to all face images F ji We can obtain a matrix that relates the face image F ji is the proportion of labelers p ji It may include the best match by

[0096] Then, in view of the above-mentioned pre-processing, training can begin. At least one input to the training can be morphological features and visual measurements, and the neural network can be configured as a combinatorial neural network with a convolutional layer and a fully connected layer. The fully connected layer is configured for embedding. The embedding layer 1118, which is a fully connected layer of D neurons, contains a vector representation of the morphological features in the vector space for each visual device i identified during pre-processing. Each cluster i of the D-dimensional vector space 1119 contained within the embedding layer 1118 represents a visual device, and each morphological feature can be represented by D vector coordinates.

[0097] During training, random sampling can be performed to randomly select a certain number of morphological feature pairs, and the morphological feature pairs are {F ki ,F li}. An example pair is F ki and F li is the ratio p ki and p li It is judged that the visual device i with ki ,p ki ) and f(F li ,p li ), where f is the activation function. ki and F li is the proportion p ki and p li Then, backpropagation is performed by finding two activation functions f(F ki ,p ki ) and f(F li ,p li ), where f is the activation function.

[0098] Returning now to FIG. 11R, the output layer 1117 of the neural network 1115 reflects the prediction of the neural network 1115 on a fit measure 1120. The fit measure 1120 is a function of the face image F ij The coordinates of the face image F ij Post-processing of the coordinates of can include (1) calculating the centroid of each cluster i in the D-dimensional vector space 1119, and (2) calculating the distance between the output coordinates and the centroid of each cluster i, thereby generating a vector containing the rank of the visual instruments (centroids of each cluster i) from closest to furthest to the output coordinate. ijA prediction of the coordinates of may be generated by the neural network 1115 for each of the at least one input 1105 in view of the morphological features and corresponding visual device attributes of a subset of images of visual devices from the database. In an embodiment, the database may be a preselected subset of images of visual devices available at a retail store or a preselected subset of images of visual devices previously identified as suitable for particular characteristics associated with the user.

[0099] According to an embodiment of the present disclosure, the neural network of ML-VEST can be configured to determine a user's fit scale that reflects the ideal coordinates of the visual instrument. To this end, FIG. 12A reflects the labeling process used within ML-VEST, whose structure is generally similar to that described above. First, an input image 1205 can be received. The input image 1205 can be sent to an image preparation process 1210, which identifies the coordinates of facial landmarks. These facial landmarks can be identified by first calibrating the image to ensure scale distances, secondly detecting landmarks in the image of the person's face via traditional image processing or deep learning techniques to extract the scale coordinates of these landmarks, and thirdly regularizing these extracted coordinates with respect to anatomical reference points (e.g., the lowest point of the chin). These extracted and regularized coordinates can be sent as an input layer to the neural network training process 1235. As described above, this same image preparation process 1210 can be followed during the implementation of ML-VEST. The processed input image can then be passed to a labeling process 1240, where a group of labelers, specifically a category of expert labelers 1248, label the processed input image. In one example, the category of expert labelers can be eye care professionals assisted by 3D modeling experts, who can model the photographic facial image such that an ideal visual device is generated according to the morphological features of the person's face in the image. In one example, the model can be configured with scale coordinates corresponding to the internal and external contour landmarks of the ideal visual device. As before, these internal and external contour landmarks of the ideal visual device can be regularized to an anatomical reference point (e.g., the lowest point of the chin) and can be used as the output 1220 of the neural network. In other words, the above landmarks of the ideal visual device 1120 serve as training data during training and define the output layer of the neural network during the implementation of ML-VEST. As before, it can be appreciated that the training process 1235 can be repeated for each of a plurality of images of people's faces in the training database.

[0100] The above labeling scheme of the labeling process can be implemented within ML-VEST. In particular, the embodiment can be processed as shown in FIG. 12B.

[0101] Referring to FIG. 12B, the input image 1205 can be passed to an image preparation process 1210 before being passed to the neural network 1215. The input image 1205 can include an image of a human face. The image preparation process 1210 can include image processing to derive morphological features and landmarks of the image of the human face, according to an embodiment. The processed input image can be sent to an input layer of the neural network 1215, where the neural network 1215 is applied to the input image. The output layer reflects the prediction of the neural network 1215 regarding the labels, as indicated by a group of labelers. This prediction or fit measure 1220 reflects the ideal visual device coordinates generated by the neural network 1215 for the human face of the image. During the implementation of the trained neural network of FIG. 12B and FIG. 12A, which is clearly shown in ML-VEST, the output ideal visual device coordinates can be compared to the coordinates of multiple visual devices in a database, and the most highly correlated one is selected as the visual device suited to the user.

[0102] 13A-13F show an example implementation and components of the neural network of ML-VEST of the present disclosure.

[0103] As a review, several use cases of input to the neural network of ML-VEST can be considered: (1) morphological features and visual device attributes obtained from face images and visual device images or from face images wearing visual devices, (2) face images and visual device attributes, (3) morphological features and visual device attributes, (4) face images and visual device images, (5) morphological features, visual device attributes, and visual measurements, (6) face images, visual device attributes, and visual measurements, (7) morphological features, visual device images, and visual measurements, (8) face images, visual device images, and visual measurements, (9) morphological features, (10) face images, (11) morphological features and visual measurements, (12) face images and visual measurements.

[0104] In all cases except for cases (1), (4), (9), and (10), two heterogeneous input streams must be incorporated. To process face images and visual device images, a convolutional + fully connected neural network section can be used, as shown in FIG. 13A and detailed in FIG. 13B. For this, the neural network utilizes a succession of convolutional layers, each layer consisting of convolutional filters with various sizes, padding, strides, and depths, followed by an activation layer (e.g., ReLU, leaky ReLU) or a pooling filter (e.g., max pooling, average pooling). The last convolutional layer can then be vectorized, and each real number of the obtained vector can be processed through a fully connected layer, and the activation function can be selected from a group including ReLU, leaky ReLU, sigmoid, TanH, etc.

[0105] At this point, a subset of the nodes of the neural network remains unprocessed prior to final output. The "Data Preparation + Fully Connected" neural network section shown in FIG. 13C can be used to process data including real numbers and strings of strings representing, for example, morphological features, visual instrument attributes, and visual measurements. First, the data can be aggregated to form a vector of heterogeneous data. Then, the vector can be homogenized to obtain only real numbers. For this purpose, the string can be replaced with an integer corresponding to the order of the item in the string. For example, the color "yellow" can be replaced with "2" since "yellow" is the "2" item in a string of available colors that includes "blank", "yellow", "orange", "red", etc. Each characteristic can be replaced with the alphanumeric value of the item. Then, each real number of the obtained vector can be processed through a fully connected layer, with the processing being derived by an activation function selected from the group including ReLU, leaky ReLU, sigmoid, and TanH, among others.

[0106] After homogenization, there may still be unprocessed nodes. To combine the outputs of both the "convolution + fully connected" network section and the "data preparation + fully connected" network section, (1) the output vectors of each network section must be integrated to generate a unique vector, and (2) the unique vector must be processed through the "output preparation" network section, as shown in FIG. 13D. The "output preparation" network section may consist of a succession of fully connected layers whose activation functions are selected from a group including ReLU, leaky ReLU, sigmoid, and TanH, etc. The number and size of these fully connected layers may be based on the desired output. For example, if the output is unique, as is possible in cases (1)-(8), the last fully connected layer may consist of one node representing the probability (a real number between 0 and 1) that the visual instrument matches the face image given as input. In such cases, the probability can be (1) defined as a threshold (e.g., threshold=0.5) if the desired output is a (match) binary value, or can be scaled to fit a predefined scoring range (e.g., if the score is 0-10, the probability is multiplied by 10) if the desired output is a (match) score.

[0107] As shown in FIG. 13E, the output can be two or more nodes. If the outputs are multiple, but each output involves the same (and unique) visual device, as is possible in cases (1)-(8), where each output value is a match score linked to a specific criterion, there can be as many nodes in the final fully connected layer as there are scoring criteria, with each node representing the probability (a real number between 0 and 1) that the visual device matches the face image given as input with respect to criterion i. Then, if the desired output is a (match) score (e.g., if the score is between 1 and 10, the probability is multiplied by 10), the probability can be scaled to fit a predefined scoring range. If the outputs are multiple, but each output involves several visual devices, as is possible in cases (9)-(12), and shown in FIG. 13F, there can be as many nodes in the final fully connected layer as there are scoring criteria multiplied by the number of scoring visual devices, with each node representing the probability (a real number between 0 and 1) that the visual device matches the face image given as input with respect to criterion i. In such cases, the probability can be (1) defined as a threshold (e.g., threshold = 0.5) if the desired output is a (match) binary value, or (2) scaled to fit a predefined scoring range (e.g., if the score is 0-10, the probability is multiplied by 10) if the desired output is a (match) score.

[0108] Regarding case (1), since there are only processed features of the face image and the visual device image as input, there may be at least one “Data Preparation + Fully Connected” neural network section similar to that described above, and the output of the “Data Preparation + Fully Connected” neural network section may be processed as described above to arrive at the desired output.

[0109] Regarding case (4), since there are both facial images and visual device images as inputs, there may be at least two “convolutional + fully connected” neural network sections, one for each image, and the outputs of the two network sections may be combined in a similar manner as described above to arrive at the desired output.

[0110] For the exemplary vector embodiments of Case (9) (FIG. 11K) and Case (10) (FIG. 11M), there can be at least one “convolutional + fully connected” neural network section since there is only a face image as input.

[0111] For the exemplary coordinate-based embodiments of Case (10) (FIG. 11N) and Case (12) (FIG. 11R), since there is a face image as input along with visual measurements, there may be at least one “convolutional + fully connected” neural network section, where the fully connected layer includes the embedding.

[0112] 14 is a non-limiting example of an implementation of a training process 435 for training a neural network using training data. As discussed above, the training data may include a plurality of labeled input images or data from one or more sources including, for example, a training database connected via a wired or wireless connection.

[0113] At operation 1480 of process 435, an initial guess for the neural network coefficients is generated. For example, the initial guess can be based on prior knowledge of the data being collected and relevant indicators within the data. Further, the initial guess can be based on one of LeCun's initialization, Xavier's initialization, and Kaiming's initialization.

[0114] Operation 1481 of process 435 provides a non-limiting example of an optimization method for training a neural network. In operation 1481 of process 435, an error is calculated (e.g., using a loss or cost function) that represents a measure of difference (e.g., a distance measure) between the labeled data (i.e., ground truth) and the output data of the neural network applied in the current iteration of the neural network. The error may be calculated using any known cost function or distance measure between the training data. Furthermore, in certain implementations, the error / loss function may be calculated using one or more of a hinge loss and a cross-entropy loss. In one example, the loss function is calculated using the output of the neural network (S NN ) and labeled ground truth data (S AGT ), i.e.,

number

[0115] Furthermore, loss functions can be combined with regularization techniques to prevent the network from overfitting to specific instances represented in the training data. Regularization can help prevent overfitting in machine learning problems. If training is too long, assuming the model is expressive enough, the network will learn noise specific to that dataset, called overfitting. In the case of overfitting, the neural network will generalize poorly and will have high variance as the noise varies across datasets. The minimum total error occurs when the sum of bias and variance is minimal. Therefore, it is desirable to describe the data in the simplest possible way to reach a local minimum that maximizes the likelihood that the trained network represents a general solution rather than a solution specific to the noise in the training data. This goal can be achieved, for example, by stopping weight regularization, lasso regularization, ridge regularization, or elastic net regularization early.

[0116] In a particular embodiment, the neural network is trained using backpropagation. Backpropagation can be used to train neural networks and is used in conjunction with gradient descent optimization. During the forward pass, the algorithm calculates predictions of the network based on the current parameters Θ. These predictions are then input into a loss function, which compares them with the corresponding ground truth labels (i.e., the labeled data). During the backward pass, the model calculates the gradient of the loss function with respect to the current parameters, and then the parameters are updated by taking a step size of a predefined size in the direction in which the loss is minimized (e.g., accelerated methods such as the Nesterov momentum method and various adaptive methods can select a step size that converges more quickly to optimize the loss function).

[0117] The optimization method in which backpropagation is performed may use one or more of gradient descent, batch gradient descent, stochastic gradient descent, and mini-batch stochastic gradient descent. Additionally, the optimization method may be accelerated by using one or more momentum update techniques in the optimization method that increase the convergence rate of stochastic gradient descent in deep networks, including, for example, adaptive methods such as the Nesterov momentum technique or the Adagrad subgradient method, the Adadelta or RMSProp parameter update variants of the Adagrad method, and the Adam adaptive optimization technique. The optimization method may also apply a quadratic method by incorporating a Jacobian matrix into the update step.

[0118] Forward and backward passes can be run incrementally through each layer of the network. In the forward pass, execution begins by feeding inputs to the first layer, thereby producing input activations for the subsequent layer. This process is repeated until a loss function in the last layer is reached. During the backward pass, the last layer computes its own learnable parameters (if any) and gradients with respect to its own inputs, which act as the upstream derivative of the previous layer. This process is repeated until the input layer is reached.

[0119] Returning to the non-limiting example shown in Figure 14, operation 1482 of process 435 identifies an error change such that a function of the change in the network can be calculated (e.g., an error gradient) and this error change can be used to select the direction and step size of subsequent changes in the neural network weights / coefficients. Calculating the error gradient in this manner is consistent with certain implementations of gradient descent optimization. In certain other implementations, as will be appreciated by those skilled in the art, this operation can be omitted and / or replaced with another operation according to a different optimization algorithm (e.g., a non-gradient descent optimization algorithm such as a simulated annealing genetic algorithm).

[0120] In act 1483 of process 435, a new set of coefficients for the neural network is determined. For example, the weights / coefficients may be updated using the changes calculated in act 1482, such as in a gradient descent optimization or over-relaxation accelerated method.

[0121] In operation 1484 of process 435, new error values ​​are calculated using the updated weights / coefficients of the neural network.

[0122] In operation 1485 of process 435, a predefined stopping criterion is used to determine whether the training of the network is complete. For example, the predefined stopping criterion may evaluate whether the new error and / or the total number of iterations performed exceeds a predefined threshold. For example, the stopping criterion may be met if the new error falls below a predefined threshold or if a maximum number of iterations has been reached. If the stopping criterion is not met, the training process performed in process 435 returns to operation 1482 and continues back to the beginning of the iterative loop by repeating operation 1482 using new weights and coefficients (the iterative loop includes operations 1482, 1483, 1484, and 1485). If the stopping criterion is met, the training process performed in process 435 is complete.

[0123] FIG. 15A shows a flow diagram of an embodiment of process 435. FIG. 15A is general to any type of layer in a feed-forward artificial neural network (ANN), including a fully connected layer, for example, as shown in FIG. 11A-FIG. 11E. The ANN of the present disclosure may include a fully connected layer preceded by an image processing neural network of convolution, pooling, batch regularization, and activation, producing a flow diagram that is a combination of FIG. 15A and FIG. 15B, as will be understood by those skilled in the art. The implementation of process 435 shown in FIG. 15A also corresponds to applying the ANN of the present disclosure to each training data of the present disclosure.

[0124] In operation 1586, weights / coefficients corresponding to the connections between neurons (ie, nodes) are applied to each input corresponding to the processed input image data.

[0125] The weighted inputs are summed in operation 1587. If only the non-zero weights / coefficients connecting to a given neuron in the next layer are regionally localized in the processed input image data represented by the previous layer, then the combination of operations 1586 and 1587 is essentially the same as performing a convolution operation.

[0126] In operation 1588, each threshold is applied to the weighted sum of each neuron.

[0127] In process 1589, the weighting, summing, and thresholding steps are repeated for each successive layer.

[0128] Figure 15B shows a flow diagram of another embodiment of process 435 in which a convolutional neural network is applied during an image preparation step to prepare an input image for application of the ANN of the present disclosure, as discussed in Figures 11C-11E. Thus, the implementation of process 435 shown in Figure 15B corresponds to operating on input image data in a hidden layer using a non-limiting implementation of a convolutional neural network.

[0129] At operation 1590, the computation of the convolutional layer is performed as described above and in accordance with the understanding of convolutional layers by those skilled in the art.

[0130] Following the convolution, at operation 1591, batch regularization may be performed to control variation in the output of the previous layer, as will be appreciated by those skilled in the art.

[0131] Following batch regularization, activation is performed in operation 1592 according to the above description of activation and as understood by those skilled in the art. In one example, the activation function is a regularized activation function or, for example, ReLU, as described above.

[0132] In another embodiment, the ReLU layer of operation 1592 may be performed prior to the batch regularization layer of operation 1591.

[0133] In operation 1593, the output from the convolutional layer, following batch regularization and activation, is input to a pooling layer, which is performed in accordance with the above description of pooling layers and in accordance with the understanding of pooling layers by those skilled in the art.

[0134] In operation 1594, the steps of convolutional layers, pooling layers, batch regularization layers, and ReLU layers may be repeated in whole or in part for a predefined number of layers. Following (or intermixed with) the above layers, the output from the ReLU layer may be fed to a predefined number of ANN layers performed according to the description provided with respect to the ANN layers of FIG. 9A. The final output is the processed input image characteristics, as previously described.

[0135] For convolutional neural network architectures, convolutional layers are typically placed near the input layer, while fully connected layers that perform high-level inference are placed further into the architecture toward the loss function. Pooling layers are inserted after the convolutions to provide a reduction in the spatial extent of the filters, and thus a reduction in the amount of learnable parameters. Batch regularization layers discipline the perturbation of gradients to outliers and accelerate the learning process. Activation layers are also incorporated into various layers to introduce nonlinearity and allow the network to learn complex predictive relationships. Activation functions can be saturated activation functions (e.g., sigmoid or hyperbolic tangent activation functions) or regularized activation functions (e.g., ReLU, as discussed above).

[0136] FIG. 16 shows an example of the interconnections between layers in an ANN such as described in this disclosure. The ANN may include fully connected and, in view of FIG. 15B, convolutional, pooling, batch regularization, and activation layers, all of which are described above and below. In an embodiment, a convolutional neural network layer may be embedded within the ANN. Alternatively, the convolutional neural network may be placed in front of the ANN, with the output layer of the convolutional neural network partially defining the input layer of the ANN. The placement of the convolutional neural network relative to the ANN is as important as the convolutional neural network partially providing the processed input image to the input layer of the ANN.

[0137] Still referring to FIG. 16, FIG. 16 shows an example of a general ANN with N inputs, K hidden layers, and three outputs. Each layer is composed of nodes (also called neurons), which perform a weighted sum of the inputs and compare the result of the weighted sum with a threshold to generate an output. ANNs constitute a class of functions where the members of the class are obtained by varying the thresholds, the connection weights, or architectural details such as the number of nodes and / or the connections of the nodes. The nodes in an ANN can be called neurons (or neuron nodes), and the neurons can have interconnections between different layers of the ANN system. The simplest ANN has three layers and is called an autoencoder. The CNN of the present disclosure can have four or more layers of neurons, and can have the same number of output neurons as input neurons:

number

[0138] Mathematically, a function m(x) of a network of neurons can be further defined as a composition of other functions, n i (x). This can be conveniently represented as a network structure, with arrows indicating dependencies between variables, as shown in Figure 16. For example, an ANN is a nonlinear weighted sum of: m(x)=K(Σ i w i n i (x) can be used, where K (commonly called the activation function) is some predefined function, such as the hyperbolic tangent.

[0139] In Figure 16, neurons (i.e., nodes) are depicted as circles around a threshold function. In the non-limiting example shown in Figure 16, inputs are depicted as circles around a linear function, with arrows indicating directed communication between neurons. In certain implementations, the ANN is a feedforward network.

[0140] The disclosed ANN operates to accomplish a particular task by searching within a class of functions F and learning using a set of observations to find m*∈F that solves the particular task in some optimal sense (e.g., satisfies the stopping criteria used in act 1485 of process 435 described above). For example, in certain implementations, this can be accomplished by defining a cost function C:F→m such that for an optimal solution m*, C(m*)≦C(m)∀m∈F (i.e., no solution has a cost less than the cost of the optimal solution). The cost function C is a measure of how far a particular solution is from the optimal solution to the problem to be solved (e.g., error). The learning algorithm iteratively searches the solution space to find a function with the smallest possible cost. In certain implementations, the cost is minimized over a sample of data (i.e., training data).

[0141] Reference is now made to FIG. 17, which is a hardware illustration of ML-VEST according to an exemplary embodiment of the present disclosure.

[0142] In Figure 17, the ML-VEST includes a CPU 1760 that executes the processes described above. The ML-VEST can be a general-purpose computer or a specialized machine. In one embodiment, the ML-VEST becomes a specialized machine when the processor 1760 is programmed to execute visual device selection (particularly any of the processes discussed with reference to the disclosure above).

[0143] Alternatively or additionally, as one skilled in the art will appreciate, CPU 1760 may be implemented in an FPGA, ASIC, PLD, or using discrete logic circuitry. Further, CPU 1760 may be implemented as multiple processes working in parallel to execute instructions of the inventive process described above.

[0144] The ML-VEST also includes a network controller 1763, such as an Intel Ethernet PRO network interface card, that interfaces with a network 1775. As can be appreciated, the network 1775 can be a public network, such as the Internet, or a private network, such as a LAN or WAN network, or any combination thereof, including PSTN or ISDN sub-networks. The network 1775 can be wired, such as an Ethernet network, or wireless, such as a cellular network, including EDGE, 3G, and 4G wireless cellular systems. The wireless network can be Wi-Fi, Bluetooth, or any other wireless form of communication known in the art.

[0145] During the training process 435, input training images may be obtained from a training database 1736 that is wirelessly connected to the ML-VEST via a network 1775 or via a hardwired connection to the ML-VEST through a storage controller 1772. In an embodiment, the training database 1736 is a vision instrument database.

[0146] The ML-VEST further includes a display controller 1764, such as a graphics card or graphics adapter, that interfaces with a display 1765, such as a monitor. A general purpose I / O interface 1766 interfaces with a keyboard and / or mouse 1767 and a touch screen panel 1768 on or separate from the display 1765. The general purpose I / O interface also connects to a wide variety of peripherals 1769, including printers and scanners.

[0147] A sound controller 1770 is also provided in the ML-VEST to interface with a speaker / microphone 1771, thereby providing sound and / or music.

[0148] A generic storage controller 1772 connects the storage media disks 1762 to a communication bus 1773, which may be ISA, EISA, VESA, etc., that interconnects all components of the ML-VEST. A description of the general features and functions of the display 1765, keyboard and / or mouse 1767, and display controller 1764, storage controller 1772, network controller 1763, sound controller 1770, and generic I / O interface 1766 are omitted herein for the sake of brevity, as these features are known.

[0149] The exemplary circuit elements described in this disclosure may be substituted with other elements and may have different structures than the examples provided herein. Additionally, circuits configured to perform the features described herein may be embodied in multiple circuit units (e.g., chips), or the features may be combined into circuits on a single chipset.

[0150] The functions and features described herein may be performed by various distributed components of the system. For example, one or more processors may perform these system functions, and the processors are distributed across multiple components that communicate over a network. The distributed components may include one or more client and server machines that may share processing, as well as various human interface and communication devices (e.g., display monitors, smartphones, tablets, personal digital assistants (PDAs)). The network may be a private network, such as a LAN or WAN, or a public network, such as the Internet. Inputs to the system may be received via direct user input, or may be received remotely in real time or as a batch process. Additionally, some implementations may be implemented in modules or hardware that are not identical to those described. Thus, other implementations are within the scope of what may be claimed.

[0151] Obviously, many modifications and variations are possible in light of the above teachings and it is therefore to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein.

[0152] Embodiments of the present disclosure may also be described in the following description.

[0153] (1) An apparatus for determining a matched visual instrument, comprising: a processing circuit configured to: receive at least one input, the at least one input including an image of a human face; apply a neural network to the at least one input, the neural network generating at least one match measure for the at least one input; and determine a matched visual instrument based on the at least one match measure generated by the neural network, the at least one match measure corresponding to relative synchronization between the human face and the visual instrument.

[0154] (2) The apparatus of (1), wherein at least one input includes an image of a visual device.

[0155] (3) The apparatus of (1) or (2), wherein at least one input includes an image of a human face, the human being wearing a visual device.

[0156] (4) The device described in any one of (1) to (3), wherein the at least one input is at least one processed input, and the at least one processed input includes morphological features identified from an image of a human face.

[0157] (5) The device described in any one of (1) to (4), wherein the at least one input is at least one processed input, and the at least one processed input includes visual device attributes identified from an image of the visual device.

[0158] (6) The device described in any one of (1) to (5), wherein at least one input includes a visual measurement of the person, the visual measurement indicating the person's visual acuity.

[0159] (7) The apparatus of any one of (1) to (6), wherein the processing circuitry is further configured to train the neural network on a training database, the training database including a corpus of training images, the corpus of training images including images of people's faces and images of the visual device, and each combination of an image of the images of the people's faces and an image of the visual device is associated in the training database with at least one training fit measure assigned by a group of labelers.

[0160] (8) The apparatus of any one of (1) to (7), wherein the corpus of training images includes images of people wearing the visual device, and each image of the images of people wearing the visual device is associated in the training database with at least one training fit measure assigned by a group of labelers.

[0161] (9) The device of any one of (1) to (8), wherein the neural network includes an implicit input, the implicit input being a predefined set of visual instruments, and the at least one fit measure generated by the neural network is at least one match score between the at least one input and each one of the predefined set of visual instruments.

[0162] (10) To determine a matched visual device, the processing circuitry is further configured to select the at least one match score that is the largest, the at least one match score being one of a predefined set of visual devices that best matches the at least one input human face.

[0163] (11) The apparatus of any one of (1) to (10), wherein the maximum at least one match score is selected from a vector including at least one match score, each of the at least one match score in the vector corresponding to one of a predefined set of vision instruments, and the at least one match score is based on a proportion of labelers in a group of labelers that are assigned at least one match score of the same value.

[0164] (12) The device of any one of (1) to (11), wherein to determine a suitable visual device, the processing circuitry is further configured to calculate coordinates corresponding to at least one input, calculate a centroid of a cluster associated with each one of a predefined set of visual devices, calculate a distance between the coordinates and each centroid of the cluster, the distances being ranked in a vector, and select a cluster from among the clusters that minimizes the distance between the coordinates and each centroid of the cluster.

[0165] (13) An apparatus described in any one of (1) to (12), wherein a cluster associated with each one of a predefined set of visual instruments includes matching coordinates corresponding to at least one training input that maximizes at least one training matching score during training of the neural network, and the at least one training input includes morphological features of a human face.

[0166] (14) A method for determining a matched visual instrument, the method including: receiving, by a processing circuit, at least one input, the at least one input including an image of a human face; applying, by the processing circuit, a neural network to the at least one input, the neural network generating at least one match measure for the at least one input; and determining, by the processing circuit, a matched visual instrument based on the at least one match measure generated by the neural network, the at least one match measure corresponding to relative synchronization between the human face and the visual instrument.

[0167] (15) A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method for determining a matched vision device, the method including receiving at least one input, the at least one input including an image of a human face, applying a neural network to the at least one input, the neural network generating at least one match measure for the at least one input, and determining a matched vision device based on the at least one match measure generated by the neural network, the at least one match measure corresponding to relative synchronization between the human face and the vision device.

[0168] (16) The apparatus of any one of (1) to (13), wherein at least one training fit metric includes a partial assessment of the fit of the visual device to the human face.

[0169] (17) The apparatus according to any one of (1) to (13) and (16), wherein the group of labelers includes an ophthalmologist.

[0170] (18) The device of any one of (1) to (13), (16), and (17), wherein the processing circuitry is further configured to compare at least one fit measure to a predetermined threshold to determine a suitable visual device.

[0171] (19) The method of (14), wherein at least one input includes an image of a visual device.

[0172] (20) The method of any one of (14) or (19), wherein at least one input includes an image of a human face, the human being wearing a visual device.

[0173] (21) A method according to any one of (14), (19) and (20), wherein the at least one input is at least one processed input, and the at least one processed input includes morphological features identified from an image of a human face.

[0174] (22) A method according to any one of (14) and (19) to (21), wherein the at least one input is at least one processed input, and the at least one processed input includes visual device attributes identified from an image of the visual device.

[0175] (23) A method according to any one of (14) and (19) to (22), wherein at least one input includes a visual measurement of the person, the visual measurement being indicative of the person's visual acuity.

[0176] (24) The method of any one of (14) and (19) to (23), further comprising training the neural network with a training database by the processing circuitry, the training database including a corpus of training images, the corpus of training images including images of people's faces and images of the visual device, and each combination of an image of the images of the people's faces and an image of the visual device in the training database is associated with at least one training fit measure assigned by a group of labelers.

[0177] (25) A method according to any one of (14) and (19) to (24), wherein the corpus of training images includes images of people wearing the visual device, and each image of the images of people wearing the visual device is associated in the training database with at least one training fit measure assigned by a group of labelers.

[0178] (26) A method according to any one of (14) and (19) to (25), wherein the neural network includes an implicit input, the implicit input being a predefined set of visual instruments, and the at least one fit measure generated by the neural network is at least one match score between the at least one input and each one of the predefined set of visual instruments.

[0179] (27) A method according to any one of (14) and (19) to (26), further comprising selecting, by the processing circuitry, the at least one matching score that is the largest to determine a suitable visual device, the largest at least one matching score being one of a predefined set of visual devices that best matches the at least one input human face.

[0180] (28) The method of any one of (14) and (19) to (27), wherein the largest at least one match score is selected from a vector including at least one match score, each of the at least one match score in the vector corresponding to one of a predefined set of vision instruments, and the at least one match score is based on a proportion of labelers in a group of labelers that are assigned at least one match score of the same value.

[0181] (29) A method according to any one of (14) and (19) to (28), further comprising: calculating, by a processing circuit, coordinates corresponding to at least one input to determine a suitable visual device; calculating, by the processing circuit, centroids of clusters associated with each one of a predefined set of visual devices; calculating, by the processing circuit, distances between the coordinates and each centroid of the cluster, where the distances are ranked in a vector; and selecting, by the processing circuit, a cluster from among the clusters that minimizes the distance between the coordinates and each centroid of the cluster.

[0182] (30) A method according to any one of (14) and (19) to (29), wherein a cluster associated with each one of a predefined set of visual instruments includes matching coordinates corresponding to at least one training input that maximizes at least one training matching score during training of the neural network, and the at least one training input includes morphological features of a human face.

[0183] (31) The method of any one of (14) and (19) to (30), wherein at least one training fit measure includes a partial assessment of the fit of the visual device to the human face.

[0184] (32) The method according to any one of (14) and (19) to (31), wherein the group of labelers includes ophthalmologists.

[0185] (33) A method according to any one of (14) and (19) to (32), further comprising comparing, by a processing circuit, at least one suitability measure to a predetermined threshold to determine a suitable visual device.

[0186] (34) The method of (15), wherein at least one input includes an image of a visual device.

[0187] (35) The method of (15) or (34), wherein at least one input includes an image of a human face, the human being wearing a visual device.

[0188] (36) A method according to any one of (15), (34), and (35), wherein the at least one input is at least one processed input, and the at least one processed input includes morphological features identified from an image of a human face.

[0189] (37) A method according to any one of (15) and (34) to (36), wherein the at least one input is at least one processed input, and the at least one processed input includes visual device attributes identified from an image of the visual device.

[0190] (38) A method according to any one of (15) and (34) to (37), wherein at least one input includes a visual measurement of the person, the visual measurement being indicative of the person's visual acuity.

[0191] (39) The method of any one of (15) and (34) to (38), further comprising training the neural network with a training database, the training database including a corpus of training images, the corpus of training images including images of people's faces and images of the visual device, and each combination of an image of the images of the people's faces and an image of the visual device in the training database is associated with at least one training fit measure assigned by a group of labelers.

[0192] (40) A method according to any one of (15) and (34) to (39), wherein the corpus of training images includes images of people wearing the visual device, and each image of the images of people wearing the visual device is associated in the training database with at least one training fit measure assigned by a group of labelers.

[0193] (41) A method according to any one of (15) and (34) to (40), wherein the neural network includes an implicit input, the implicit input being a predefined set of visual instruments, and the at least one fit measure generated by the neural network is at least one match score between the at least one input and each one of the predefined set of visual instruments.

[0194] (42) A method according to any one of (15) and (34) to (41), further comprising selecting the at least one matching score that is the largest to determine a suitable visual device, the largest at least one matching score being one of a predefined set of visual devices that best matches the at least one input human face.

[0195] (43) The method of any one of (15) and (34) to (42), wherein the largest at least one match score is selected from a vector including at least one match score, each of the at least one match score in the vector corresponding to one of a predefined set of vision instruments, and the at least one match score is based on a proportion of labelers in a group of labelers that are assigned at least one match score of the same value.

[0196] (44) A method according to any one of (15) and (34) to (43), further comprising: calculating coordinates corresponding to at least one input to determine a suitable visual device; calculating centroids of clusters associated with each one of a predefined set of visual devices; calculating distances between the coordinates and each centroid of the cluster, the distances being ranked in a vector; and selecting from the clusters a cluster that minimizes the distance between the coordinates and each centroid of the cluster.

[0197] (45) A method according to any one of (15) and (34) to (44), wherein a cluster associated with each one of a predefined set of visual instruments includes matching coordinates corresponding to at least one training input that maximizes at least one training matching score during training of the neural network, and the at least one training input includes morphological features of a human face.

[0198] (46) The method of any one of (15) and (34) to (45), wherein at least one training fit measure includes a partial assessment of the fit of the visual device to the human face.

[0199] (47) The method according to any one of (15) and (34) to (46), wherein the group of labelers includes ophthalmologists.

[0200] (48) The method according to any one of (15) and (34) to (47), further comprising comparing at least one suitability measure to a predetermined threshold to determine a suitable visual device.

[0201] Thus, the foregoing discussion discloses and describes merely exemplary embodiments of the present invention. As will be understood by those skilled in the art, the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the present invention and other claims. The disclosure defines in part the scope of the following claim terms, including all readily discernible variations of the teachings herein, so that the spirit of the present invention is not dedicated to the public. [Explanation of symbols]

[0202] 100 Machine Learning Based Visual Instrument Selection Tools 105 Input 110 Input preparation process 115 Machine Learning Application Process 120 Fit measure 125 Visual Equipment Selection Process

Claims

1. An apparatus for determining a suitable visual instrument, comprising: A processing circuit is provided, The processing circuitry includes: receiving at least one input, the at least one input including an image of a human face; applying a neural network to the at least one input, the neural network generating at least one measure of fitness of the at least one input; determining the matched vision instrument based on the at least one match measure generated by the neural network; and [0023] the at least one fit measure corresponds to a relative synchronization providing a measure quantifying a fit between the person's face and a vision device; the neural network includes an implicit input, the implicit input being a predefined set of visual instruments, and the at least one fit measure generated by the neural network is at least one match score between the at least one input and a respective one of the predefined set of visual instruments; To determine the suitable vision instrument, the processing circuitry Selecting the at least one match score that is the largest. the highest at least one match score is the one of the predefined set of vision instruments that best matches the human face of the at least one input; the highest at least one match score is selected from a vector including the at least one match score, each of the at least one match score in the vector corresponding to one of the predefined set of vision instruments, and the at least one match score is based on a proportion of labelers of a group of labelers that are assigned the same value of the at least one match score.

2. The apparatus of claim 1 , wherein the at least one input comprises an image of a visual instrument.

3. The apparatus of claim 1 , wherein the at least one input includes an image of a face of the person, the person being wearing a vision instrument.

4. The apparatus of claim 1 , wherein the at least one input is at least one processed input, the at least one processed input including morphological features identified from an image of the person's face.

5. The apparatus of claim 2 , wherein the at least one input is at least one processed input, the at least one processed input including visual device attributes determined from the image of the visual device.

6. The apparatus of claim 1 , wherein the at least one input comprises a visual measurement of the person, the visual measurement indicative of the person's visual acuity.

7. The processing circuitry includes: further configured to train the neural network with a training database; 2. The apparatus of claim 1, wherein the training database includes a corpus of training images, the corpus of training images including images of faces of people and images of visual devices, and each combination of an image of the images of the faces of people and an image of the images of the visual device is associated in the training database with at least one training fit measure assigned by a group of labelers.

8. 8. The apparatus of claim 7, wherein the corpus of training images includes images of people wearing a vision device, and each image of the images of people wearing the vision device is associated in the training database with at least one training fit measure assigned by the group of labelers.

9. To determine the suitable vision instrument, the processing circuitry calculating a coordinate corresponding to said at least one input; calculating a centroid of a cluster associated with each one of said predefined set of vision instruments; calculating distances between the coordinates and each centroid of the cluster, the distances being ranked in a vector; selecting a cluster from among said clusters that minimizes the distance between said coordinates and each centroid of said cluster; The apparatus of claim 1 , further configured to:

10. 10. The apparatus of claim 9, wherein a cluster associated with each one of the predefined set of visual instruments includes match coordinates corresponding to at least one training input that maximizes at least one training match score during training of the neural network, the at least one training input including morphological features of the person's face.

11. 1. A method for determining a suitable vision instrument, comprising: receiving at least one input by a processing circuit, the at least one input including an image of a human face; applying, by the processing circuitry, a neural network to the at least one input, the neural network generating at least one measure of fitness of the at least one input; determining, by the processing circuitry, the matched vision instrument based on the at least one match measure generated by the neural network; Including, the at least one fit measure corresponds to a relative synchronization providing a measure quantifying a fit between the person's face and a vision device; selecting, by the processing circuitry, a highest at least one match score to determine the matched vision device, the highest at least one match score being one of a predefined set of vision devices that best matches the human face of the at least one input; the neural network includes an implicit input, the implicit input being a predefined set of visual instruments, and the at least one fit measure generated by the neural network is at least one match score between the at least one input and a respective one of the predefined set of visual instruments; the highest at least one match score is selected from a vector including the at least one match score, each of the at least one match score in the vector corresponding to one of the predefined set of vision instruments, and the at least one match score is based on a proportion of labelers of a group of labelers that are assigned the same value of the at least one match score.

12. A non-transitory computer readable storage medium storing computer readable instructions that, when executed by a computer, cause the computer to perform a method for determining a matched vision device, the method comprising: receiving at least one input, the at least one input including an image of a human face; applying a neural network to the at least one input, the neural network generating at least one measure of fitness of the at least one input; determining the matched vision instrument based on the at least one match measure generated by the neural network; and Including, the at least one fit measure corresponds to a relative synchronization providing a measure quantifying a fit between the person's face and a vision device; selecting a highest at least one match score to determine the matched vision device, the highest at least one match score being one of a predefined set of vision devices that best matches the human face of the at least one input; the neural network includes an implicit input, the implicit input being a predefined set of visual instruments, and the at least one fit measure generated by the neural network is at least one match score between the at least one input and a respective one of the predefined set of visual instruments; the highest at least one match score is selected from a vector including the at least one match score, each of the at least one match score in the vector corresponding to one of the predefined set of vision instruments, and the at least one match score is based on a proportion of labelers of a group of labelers that are assigned the at least one match score of the same value.

Images