System and method for predicting values of continuous output

Abstract

A computer-implemented method for learning a distance metric D in an input space according to a large margin nearest neighbor model adapted to an output having continuous values. The method comprises determining a transformation L for the input space, applying the transformation L to the input space, and learning the distance metric D from the transformed input space, wherein the transformation L is configured to correlate the distance between input data points in the input space with the differences and correlations between the values of the continuous output associated with the input data points.

Description

Technical Field

[0001] The present invention relates to a system and method for predicting values of continuous outputs, and to a system and method for learning a distance metric according to a large margin nearest neighbour (LMNN) model adapted to an output having continuous values.

Background Art

[0002] Machine learning tools and techniques are used to automate and improve the accuracy of decision-making. In particular, problems that require the analysis of multi-dimensional variables to derive patterns and metrics for the purpose of predicting an output for a given input can benefit from the analytical and learning capabilities of machine learning tools and techniques. An example of machine learning that can be used to predict an output for a given input is "supervised learning". Supervised learning provides an example of machine learning that relies on a training data, i.e., a use function that can be inferred from a labeled data set (composed of input-output pairs) related to a specific problem, in order to accurately predict the output to the problem for a given input. Therefore, it is important that the labeled data set from which the function can be inferred is maintained and organized in such a way as to enable an improved derivation of insights therefrom. Thus, by repeatedly analyzing, transforming, and updating the labeled data set available to the supervised learning model, the resulting decision-making, i.e., the accuracy of the output prediction, can be significantly improved.

[0003] The supervised learning method can include non-parametric models. That is, instead of learning parameters from the training data available for the model for subsequent use of these parameters for decision-making, rather, the model is such that the decisions made by the model are a direct function of the available training data. More specifically, it means that it can make a determination, that is, predict an output for a given input. This uses any function of the mapping function between a given input and the relevant output that can be derived from the available training data. Compared with parametric models where the number of parameters is fixed with respect to the amount of data, non-parametric models have fewer assumptions about the training data under a trade-off where the accuracy decreases but the ease of use increases. Therefore, non-parametric models are more versatile, reducing the need to build models, adjust various parameters, and make additional assumptions. However, to effectively use non-parametric models, the model needs to adequately organize the available training data.

[0004] An example of a supervised learning method for decision-making, that is, a non-parametric model that can be used to predict the output to a task for a given input, is the "K-nearest neighbor", KNN algorithm. In short, the KNN algorithm functions on the premise that objects with similar characteristics, for example, labeled outputs, such as input data points in the input space, exist in proximity. Therefore, the prediction of the output is based on identifying a predetermined number (K) of training samples, for example, input data points in the input space, that are closest in distance to the new input data point, that is, adjacent to the new input data point, and using the labels of the identified input data points, that is, the relevant outputs, to predict the output of the new input data point.

[0005] Therefore, the accuracy of non-parametric models such as the KNN algorithm highly depends on the distance metric used to calculate the distance between samples at input data points in the training data, for example, within the input space. Thus, in order to improve the accuracy of non-parametric models such as the KNN algorithm, the distance metric is learned for the training data available to the model, thereby making it possible to more accurately determine the relationship between a data input in the input space and other input data points, and as a result, to make more accurate and informative output predictions. The method of learning the distance metric is called "metric learning". The metric learning method transforms or converts the input space so that the algorithm applied later is more effective. The basic idea is to try to bring similar samples (with respect to the result, for example, with respect to the relevant output) closer to each other while keeping dissimilar samples well separated. As described above, non-parametric models highly depend on the distance between samples in the input space, and thus, their performance can be substantially improved by the judicious use of metric learning.

[0006] A known model for metric learning suitable for non-parametric models such as the KNN algorithm is the large margin nearest neighbor (LMNN) model. However, the known LMNN model is applicable only when the result of a given supervised learning model is discrete, that is, only for classification problems. The known LMNN model distorts the input space so that nearby samples within the input space have the same class. This improves the accuracy of any subsequent classification (i.e., discrete variable) prediction performed by a non-parametric model such as the KNN algorithm.

[0007] The known LMNN metric learning model cannot be applied when the result of the supervised learning model has continuous values. Therefore, in order to improve the accuracy of decision-making for regression problems, that is, for results with continuous values, it is desirable to learn a usable distance metric. SUMMARY OF THE INVENTION

[0008] As described above, the known LMNN metric learning model is limited to a supervised learning model used to predict discrete output values. In the development of the known LMNN model, the present application contemplates a method that can be adapted to provide a transformation of the input space labeled with continuous (i.e., non-discrete) output values by the known LMNN algorithm model and then learn a distance metric. By learning the distance metric in this way, the accuracy of predicting continuous outputs using non-parametric models such as the KNN algorithm can be improved.

[0009] Accordingly, the desired objective is to learn a distance metric according to a large margin nearest neighbor model adapted to an output having continuous values. In this case, there is a further desired objective of predicting the value of the continuous output.

[0010] According to one aspect of the present disclosure, a computer-implemented method for learning a distance metric D for an input space according to a large margin nearest neighbor model adapted to an output having continuous values is provided. The method has the following: a step of determining a transformation L for the input space, wherein the transformation L is configured to correlate the distances between input data points in the input space with the differences and correlations between the continuous output values associated with the input data points; a step of applying the transformation L to the input space; a step of learning the distance metric D from the transformed input space.

[0011] According to one aspect of the present disclosure, a computer-implemented method for predicting the value of a continuous output is provided. This method includes using the distance metric D learned for the input space.

[0012] According to another aspect of the present disclosure, a system is provided that includes one or more processors configured to execute the methods described herein.

[0013] According to another aspect of the present disclosure, there is provided a computer-readable medium including instructions for causing a computer to execute instructions for performing the methods described herein.

[0014] According to another aspect of the present disclosure, there is provided a computer program including instructions which, when executed by a computer, cause the computer to perform the steps of the methods described herein.

[0015] Optional features are described in the appended dependent claims.

[0016] These and other aspects of the present disclosure will be better recognized and understood when considered in conjunction with the following description and the accompanying drawings. The following description sets forth various embodiments of the present disclosure and numerous specific details thereof, given by way of illustration and not limitation. Many substitutions, modifications, additions, or rearrangements are possible within the scope of the present disclosure, and the present disclosure includes all such substitutions, modifications, additions, or rearrangements.

Brief Description of the Drawings

[0017] Embodiments of the present disclosure are described by way of example only with reference to the accompanying drawings:

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Best Mode for Carrying Out the Invention

[0018] In the following description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. However, it will be apparent to those skilled in the art that specific details are not required in order to practice the embodiments.

[0019] The methods and systems described herein can be used in the context of machine learning tools and techniques, particularly in the context of supervised learning used for the prediction of outputs having continuous values. Although embodiments of the present invention are described in the context of supervised learning, it will be apparent to those skilled in the art that other suitable machine learning tools and techniques may also be used to implement embodiments of the present invention.

[0020] FIG. 1 is a schematic diagram of a system for executing one or more of the methods disclosed herein. System 100 can include a client computing device 105 and a server 120 configured to be communicatively coupled via a network 110.

[0021] Network 110 is a wired or wireless network such as the Internet, an intranet, a LAN (Local Area Network), a WAN (Wide Area Network), an NFC (Near-field Communication) network, Bluetooth (registered trademark), infrared, radio frequency, a cellular network, or another type of network. It will be understood that network 110 may be a combination of multiple different types of wired or wireless networks.

[0022] Each client computing device 105 can be a smartphone, a tablet computer, a laptop computer, a computer, a personal data assistant, or any other type of mobile device having a hardware processor configured to process instructions and connected to one or more portions of network 110. Each client computing device 105 can have a graphical user interface configured to enable a user to interact with the processor of the client computing device 105.

[0023] Server 120 can include a physical computing device that resides at a specific location or can be deployed in a cloud computing network environment. In the present disclosure, "cloud computing" can be defined as a model that enables ubiquitous and convenient on-demand network access to a shared pool of configurable computing resources (such as networks, servers, storage, applications, and services) that are rapidly provisioned via virtualization, released with minimal management effort or service provider interaction, and then scaled accordingly. The cloud model can consist of various characteristics (such as on-demand self-service, broad network access, resource pooling, rapid elasticity, measured service, etc.), service models (such as Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS), and deployment models (such as private cloud, community cloud, public cloud, hybrid cloud, etc.). Server 120 can include any combination of one or more computer-usable or computer-readable media. In some embodiments, Server 120 can be configured to execute at least a portion of the methods disclosed herein. Server 120 can include one or more processing devices having a memory (such as read-only memory (ROM) and random access memory (RAM)) for storing processor-executable instructions and one or more processors for executing the processor-executable instructions. The processing device may include two or more processors, and the processors may operate in parallel or in a distributed manner. The processing device can execute an operating system or other software related to the methods disclosed herein.

[0024] FIG. 2 is a flowchart of a computer-implemented method 200 for learning a distance metric D in an input space by a large margin nearest neighbor model adapted to an output having continuous values, according to one embodiment of the present invention. Method 200 may be executed on any suitable processor, for example, a processor located on server 120.

[0025] As described above, the LMNN model provides a known metric learning algorithm for transforming an input space and learning a distance metric in the input space (i.e., on the input space), thereby improving the subsequent use of a nonparametric model for predicting discrete output variables for solving classification problems.

[0026] As will be appreciated by those skilled in the art, the term metric learning describes the process of determining a model-specific distance metric for a dataset. A distance metric describes a function that defines the distance between data points in a dataset as actual non-negative numerical values. A distance metric provides an indication of how close data points are to each other. Supervised learning methods often require a measure of the distance between data points. Although standard distance metrics are available, metric learning provides a means for determining a model-specific distance metric that can be subsequently used to improve the accuracy of predictions made by nonparametric models as compared to when standard distance metrics are used. By understanding the proximity, i.e., similarity, between data points in a dataset, e.g., input data points in an input space (each labeled with a corresponding output), an accurate prediction of the output corresponding to a new data input can be provided.

[0027] As used herein, the term input space describes a particular representation of samples by a user and / or a model, e.g., input data points. From this input space, labeled samples are values taken from the input space, i.e., possible combinations of input data points, each labeled with an associated output value.

[0028] A method 200 for learning a distance metric D for an input space according to a large margin nearest neighbor model adapted to an output having continuous values begins at step 210, where a transformation L is determined for the input space. The determined transformation L is configured to correlate the distances between input data points in the input space with the differences and correlations between the values of the continuous output associated with the input data points.

[0029] Referring now to FIG. 3, a schematic illustration of how an input space 312 can be transformed and the distance metric subsequently learned, according to an embodiment of the present invention, is shown. Three graphs 310, 320, and 330 are shown. Graph 310 shows an example of an input space 312 according to one embodiment of the present invention. The input space 312 can include one or more input data points 302, each of which can be characterized by one or more input dimensions (i.e., input variables). The input space 312 can be characterized by a vector space. In the example shown in graph 310, each input data point 302 is characterized by two input dimensions x1 and x2. As will be apparent to those skilled in the art, the input data points 302 need not be limited to two input dimensions and, instead, can have any integer number of input dimensions within the input space 312.

[0030] In one embodiment of the present invention, the input data points 302 that make up the input space 312 can include input data points that have been manually labeled with corresponding output values 304 and added to the input space 312 during the construction and / or calibration of a supervised learning model to which the input space 312 belongs. The input space 312 may also include input data points 302 for which previous predictions have been made during the runtime of the supervised learning model. The input data points 302 for which previous predictions have been made may be verified, for example manually, before being added to the input space 312, or may not be verified. Each input data point 302 within the input space 312 is labeled with an associated continuous output value 304. The continuous output value 304 associated with each input data point 302 characterizes the output predicted or labeled for the input associated with the input data point 302 with respect to a particular problem that the supervised learning model to which the input space 312 belongs is trying to solve.

[0031] In one embodiment of the present invention, the input data points 302 that make up the input space 312 are derived from normalized multi-dimensional inputs. The multi-dimensional inputs may be received by a supervised learning model during the construction of the model, at runtime, or at other times, and may be characterized by a plurality of discrete variables and / or continuous variables. To ensure that these multi-dimensional inputs can then be processed efficiently and economically, the multi-dimensional inputs can be normalized with respect to each dimension (i.e., variable) of the input using input conversion techniques. For example, categorical discrete variables may be One-Hot converted, ordinal discrete variables may be converted to integer values, continuous variables may be left as is, or rounded to a predetermined number of significant digits. It will be apparent to those skilled in the art that any number of input conversion techniques can be performed to normalize the multi-dimensional inputs, such that the multi-dimensional inputs are processed more efficiently and economically either before or after entering the input space 312.

[0032] In one embodiment of the present invention, one or more of the variables of the multi-dimensional input that can be received by the supervised learning model may include natural language text input. To ensure that such variables are properly processed, embodiments of the present invention can include an input transformation for the natural language text input variable. In one embodiment of the present invention, the input transformation of the natural language text can be performed by using a tokenization algorithm to split the whole words into smaller constituent parts. Then, a Latent Dirichlet Allocation (LDA) model can be fitted to the available samples, which are represented as a bag of words and / or a bag of tokens. Using some fitting parameters, the LDA model can be used to discover specific topics within the natural language text data input, and each topic is provided by a distribution over specific words and tokens. These distributions can then be used as features representing the samples.

[0033] One multi-dimensional input that can be received by the supervised learning model can include many different types of input variables. In one embodiment of the present invention, each variable (i.e., dimension) of the input can be transformed separately before all of the transformed variables are concatenated together to provide a sample input data point. Those skilled in the art will understand that the above types of variables and input transformation techniques are not exhaustive, and any number of types of suitable input variables and input transformation techniques can be used.

[0034] If it is determined that the sample input data point contains too many dimensions or more dimensions than necessary for subsequent processing, embodiments of the present invention can include selectively reducing the dimensions of the sample input data point using known dimensionality reduction techniques so that the sample input data point can be more appropriately input by the input space 312. As will be described in more detail below, dimensions can also be reduced when applying the transformation L to the input space 312.

[0035] Referring again to step 210 of method 200, a transformation L is determined for the input space 312. The determined transformation L is configured to correlate the distances between the input data points 302 within the input space 312 with the differences between the values of the continuous outputs 304 associated with the input data points 302.

[0036] As discussed, metric learning methods can be used to transform the input space so that a subsequent supervised learning model applied later is more effective. The known LMNN model attempts to transform the input space 312 such that neighboring input data points 302 have the same class. However, the present invention attempts to bring similar input data points 302 closer to each other in the input space 312 while keeping different input data points 302 sufficiently separated with respect to the output 204. In the context of the present invention, the output 304 associated with each input data point 302 within the input space 312 has continuous values. By performing the transformation in such a way, the learned subsequent distance metric D can make a more informative determination about the proximity of the data input points 302 within the input space 312, which can then be used to determine a more accurate prediction of the continuous output values corresponding to the received new data input.

[0037] In an embodiment of the present invention, the relationship between the distance metric D and the transformation L can be given by Equation 1 below:

[0038]

Equation

[0039] As discussed, the goal of metric learning in the context of the present disclosure is to learn a distance metric D for the input space 312, and to that end, similar input data points 302 are grouped closer together with respect to their associated continuous outputs 304, while dissimilar input data points 302 are sufficiently separated with respect to their associated continuous outputs 304. To achieve this, embodiments of the present invention can include determining a transformation L for the input space 312 by determining and minimizing a loss function with respect to the transformation L.

[0040] A loss function provides a means for characterizing the magnitude of the error in an algorithm. In other words, a loss function provides a means for determining how well a particular algorithm models a dataset. By calculating the loss function for a particular dataset belonging to the model, it is possible to determine the parameters that enable minimization of the amount of error, i.e., the amount of loss. By updating the loss function when all instances of the samples have been added to the dataset, the loss function can be used as a feedback mechanism for improving the accuracy of the algorithm being used.

[0041] In the context of the present invention, the loss function can be determined by considering each input data point 302 in the input space 312 and its relationship to other input data points 302 in the input space 312. A parameter (in this case, the transformation, L) that characterizes the relative distance between input data points 302 in the input space 312 (which should reflect their proximity with respect to the associated continuous output values 304) and weights that distance according to how well it reflects the relative differences within their associated continuous output values 304 can be determined to minimize the amount of error provided in the prediction of the associated supervised learning model. In the context of the present invention, the parameter is a transformation L that can be applied to the input space 312 to transform the input space 312, for example, a linear transformation, such that similar input data points 302 are grouped closer together with respect to their associated continuous output 304 while dissimilar input data points 302 are sufficiently separated with respect to their continuous output 304.

[0042] In one embodiment of the present invention, determining the loss function can include, for each input data point 302 in the input space 312, identifying a predetermined number of target and fake data from other input data points in the input space. When determining the loss function, each input data point 302 in the input space 312 is considered separately. For each input data point 302, the target is another input data point 302 that is desired to be close to the input data point under consideration, and the fake data is another input data point 302 that is desired to be far from the input data point under consideration.

[0043] Now, referring to graph 320 showing the same input space 312 and input data points 302 as graph 310, for a particular input data point 322 under consideration, a predetermined number of targets 324 and fake data 326 can be identified. For the sake of simplicity, in the example shown in graph 320, the predetermined number of targets 324 and fake data 326 selected are each three. It will be apparent to those skilled in the art that the predetermined number of targets 324 and fake data 326 can be more or less.

[0044] In one embodiment of the present invention, a predetermined number of targets 324 and false data 326 are identified from the 2X closest neighboring points with respect to the distance from the input data point 322 under consideration, and the targets 324 are the X neighboring points among the 2X closest neighboring points having the associated continuous output value 304 closest to the continuous output value 304 associated with the input data point 322 under consideration, and the false data 326 are the remaining X neighboring points, where X is a predetermined value.

[0045] In the context of the targets 324 and the false data 326, it is possible to determine a transformation L that distorts the input space 312 such that, by determining and minimizing a loss function with respect to the transformation L, the distance between each input data point 302 in the input space 312 and its target 324 is minimized, and the distance between each input data point 302 in the input space 312 and its false data 326 is maximized. When determining and minimizing the loss function with respect to the transformation L, the transformation L that can be determined does not necessarily have to be a square matrix. If there is a transformation L determined to be a non-square matrix, the dimension of the data to which the transformation is applied, i.e., the dimension of the input data points, is reduced. For example, input data points in an input space having a dimension of 3 can be reduced to have a dimension of 2 in the transformed input space. Of course, the dimension may be maintained by the transformation L.

[0046] As described above, when determining the loss function in one embodiment of the present invention, each term of the loss function can be weighted according to how well the relative distance between the input data points 302 in the input space 312 reflects the relative difference of those associated values of the continuous output 304. The terms of the loss function for which the input data points under consideration have a stronger positive correlation between the relative distance between the input data points 302 and the relative difference of the associated values of the continuous output 304 are weighted more strongly than these terms having a weaker correlation. That is, the loss function can be used to determine the transformation L that emphasizes the above-mentioned correlation.

[0047] In one embodiment of the present invention, the weighting can be calculated by weighting each term in the loss function according to the relative difference between the difference in the values of the continuous output 304 associated with the input data point 322 under consideration and the identified false data 326, and the difference in the values of the continuous output 304 associated with the input data point 322 under consideration and the identified target 324.

[0048] A possible example of a loss function that can be used is the following objective function J, which should be minimized with respect to the parameter L, i.e., the transformation:

[0049]

Equation

[0050] To minimize the objective function J, with respect to the transformation L, one of any number of known suitable minimization algorithms can be used. For example, using the gradient of the objective function J, the Limited Memory Broyden-Fletcher-Goldfarb-Shanno algorithm can be used, which can be calculated as follows:

[0051]

Equation

[0052] Referring back to method 200, following the determination of the transformation L for the input space 312, the next step 220 of method 200 includes applying the determined transformation L to the input space 312.

[0053] Referring to the example shown in Graph 330, the input spaces 310 and 320 shown in Graphs 310 and 320 are transformed by applying a transformation L to the input space 312 in order to generate a transformed input space 314. As can be seen from Graph 330, this transformed input space provides an updated composition of the input data points 302, which is intended to improve the organization of the input data points 302 so as to correlate the distances between the input data points 302 in the input space 314 with the differences between the values of the continuous outputs 304 associated with the input data points 302.

[0054] Referring back to Method 200, after applying the transformation L to the input space 312 to generate the transformed input space 314, the next step 230 of Method 200 includes learning a distance metric D from the transformed input space 314.

[0055] As described above with reference to Equation 1, the distance metric D can include a family of metrics over the transformed input space 314. In one embodiment of the present invention, learning the distance metric D from the transformed input space 314 can include calculating the Euclidean distance between the input data points 302 within the transformed input space 314. Referring to Graph 330, the distance 332 calculated between two input data points 302 within the input space 314 transformed according to the learned distance metric D is shown.

[0056] As explained in the background section, by learning a specific distance metric D for the input space, the accuracy of the predictions resulting from a non-parametric supervised learning model that attempts to evaluate distances within the input space using the learned distance metric D can be improved.

[0057] Regarding another aspect of the present disclosure, the relationship between the distance metric D learned for the input space and the prediction of the values of the continuous output corresponding to the received data input is explored.

[0058] Figure 4 is a flowchart of a computer-implemented method 400 for predicting values of continuous outputs according to one embodiment of the present invention. The method can include only predicting values of continuous outputs related to data inputs using a distance metric D learned about the input space. However, the method 400 shown in FIG. 4 provides an example including any additional optional steps. The method 400 can be executed on any suitable processor, for example, a processor located on server 120.

[0059] Method 400 begins at step 410 where a data input related to the input space is received. In an embodiment of the present invention, the received data input can be received during the design phase of a suitable supervised learning model, i.e., for the sole purpose of training the model. In other embodiments of the present invention, the received data input can be received during the deployment of the model, i.e., when the model is actually being used. In the context of the present disclosure, a data input related to the input space refers to a data input that is an input to a particular task for which an output value is predicted by a suitable supervised learning model, which is done by referring to an input space that is related to the task, i.e., includes input data points that were previously provided as inputs to the task. The method for providing the prediction is described in further detail below.

[0060] In one embodiment of the present invention, the received data input can include multi-dimensional variables. Thus, the received data input can have the same characteristics as those described above with reference to the multi-dimensional input from which the input data points 302 constituting the input space 312 are derived. The received data input can also be transformed, normalized, and dimensionally reduced in the same manner as described above. In one embodiment of the present invention, the multi-dimensional input can be received in tabular form.

[0061] Following the reception of the data input in step 410, the next step 420 of method 400 involves converting the data input. This step is a process of converting the data input using a transformation L determined when learning the distance metric D for the input space. In the context of the exemplary input spaces 312 and 314 described with reference to FIG. 3, the transformation L maps the input space 312 to the transformed input space 314. By performing this step, a correspondence between the format of the data input and the input space in which the distance metric D was learned can be achieved. As will be described later, this improves the accuracy of predicting the value of the continuous output associated with the data input related to the input space using the distance metric D learned for the input space.

[0062] Following the conversion of the data input in step 420, the next step 430 of method 400 involves predicting the value of the continuous output associated with the data input using the distance metric D learned for the input space. The distance metric D is learned in accordance with the embodiments of the present disclosure provided herein.

[0063] Predicting the value of the continuous output associated with the data input using the distance metric D learned for the input space is possible in the context of any suitable non-parametric model for supervised learning that relies on and benefits from the use of the distance metric learned for a given input space to improve the accuracy of predicting the output.

[0064] In an embodiment of the present invention, predicting the value of the continuous output associated with the data input using the distance metric D learned for the input space can include using the K-nearest neighbor, KNN, algorithm for the data input with respect to the input space in which the distance metric D was learned.

[0065] As discussed in the Background section, the KNN algorithm functions on the premise that input data points with similar characteristics, such as related continuous output values, exist in close proximity within the input space. Therefore, by learning an appropriate model-specific distance metric D across the input space, a more accurate determination of the similarity between input data points can be achieved for each respective related continuous output value.

[0066] Referring now to FIG. 5, a schematic illustration of how the K-nearest neighbor (KNN) algorithm can be used in accordance with an embodiment of the present invention is shown. The graph 500 shown in FIG. 5 represents the two-dimensional transformed input space 314 shown in the graph 330 of FIG. 3.

[0067] In one embodiment of the present invention, using the K-nearest neighbor (KNN) algorithm for data input with respect to the input space in which the distance metric D has been learned includes calculating, using the distance metric D, the distance from the data input to other input data points within the input space. Referring to FIG. 5, the received data input 502 is mapped with respect to the transformed input space 314. Using the distance metric D learned for the input space, the distance 506 from the received data input 502 to each of the other input data points 302 within the input space 314 can be calculated. This enables the determination of the input data points that constitute the K nearest neighbors to the data input 502 within the input space 214.

[0068] In one embodiment of the present invention, K may be a predetermined integer that is fixed regardless of the number of input data points 302 within the input spaces 312, 314. Alternatively, in other embodiments, K may be dynamically selected according to the number of data input points 302 within the input spaces 312, 314. For example, K may be larger for an input space having a larger number of input data points. In the example shown in the graph 500 of FIG. 5, K is selected to be 3. The three nearest neighbor points 504 to the data input 502 have a distance 506 between the data input 502 and each of the three nearest neighbor points 504.

[0069] In one embodiment of the present invention, predicting the value of a continuous output associated with data input 502 using the distance metric D learned for input space 314 can include calculating the average of the values of the continuous outputs associated with the K nearest neighbors 504 to the input data point 502 within input space 314. Referring to graph 500, K is selected as 3, and thus, the prediction of the value of the continuous output associated with data input 502 can be calculated by taking the average of the continuous output values associated with each of the three closest neighbors 504 to data input 502 within input space 314. For example, if the continuous output values associated with each of the three closest neighbors 504 are 176.4 cm, 186 cm, and 178.2 cm, respectively, the value of the continuous output associated with data input 502 is 180.2 cm, which is the average of the three numbers.

[0070] In an alternative embodiment of the present invention, predicting the value of a continuous output associated with data input 502 using the distance metric D learned for the input space can include calculating a weighted average of the values of the continuous outputs associated with the K nearest neighbors 504 to the input data point 502 within input space 314, where each term of the weighted average is weighted according to the reciprocal of the distance 506 of the neighbor 504 from the data input 502. By performing such weighting, the prediction of the value of the continuous output associated with data input 502 will take greater account of the degree of proximity between data input 502 and the nearest neighbor 504 within input space 314. Thus, the predicted value of the continuous output will be more influenced by the nearest neighbors 504 closer to data input 502 than by those farther away. This allows for a more accurate prediction to be made compared to when all K nearest neighbors have an equal influence on the predicted value.

[0071] As described above, a supervised learning model can infer a function and a metric from training data, i.e., labeled input data points in an input space, to predict an output for future data inputs. Therefore, by increasing the pool of labeled input data points available to the supervised learning model, the accuracy of the predictions made by the supervised learning model can be improved.

[0072] In one embodiment of the present invention, method 400 can further include the steps of verifying a predicted value of a continuous output associated with data input 502, labeling data input 502 with the verified predicted value, adding the labeled data input 502 to input space 314, and updating distance metric D. By verifying the prediction made prior to labeling and adding data input 502 to input space 314, it can be ensured that inaccurately labeled data inputs are filtered and / or manually corrected so as not to distort the accuracy of the input space. The verification can be performed manually or, for example, based on predefined heuristic rules. If the prediction associated with data input 502 is positively verified, data input 502 is labeled with the prediction and added to the input space. By adding to the pool of labeled input data points 302 available to the supervised learning model, it is guaranteed that input spaces 312, 314 are continuously improved for each input data point 302 processed and learned. This results in more accurate predictions.

[0073] By adapting a known LMNN model to an output having continuous values, a known supervised learning model that uses a nonparametric model to predict an output having continuous values can be improved in both accuracy and computational efficiency. As described above, by better understanding the proximity of input data points within the input space, more information is provided and accurate predictions can then be made. The improved distribution of input data points in the input space enabled by the method disclosed herein ensures that less effort is required in the nonparametric model part when making predictions. In accordance with the method disclosed herein, a model that attempts to predict an output having continuous values for a given related data input without learning a distance metric across the input space compensates for the lack of neighborhood influence provided by the disclosed adapted LMNN model by deriving additional insights and metrics. Thus, such known methods require more data as well as computational time and resources.

[0074] The methods and systems described herein can be used to predict an output having continuous values for a wide range of real-world problems across all areas of technology. As discussed, the methods and systems disclosed herein learn a distance metric D for the input space to provide a more improved organization of labeled input data points in the input space and thus provide a more accurate determination of the continuous output value for the received data input. In fact, the more labeled input data points obtained by a particular model, the more likely the resulting prediction will be accurate. Examples of input-output pairs for problems for which the value of a continuous output can be predicted are provided below. For each of these problems, each supervised learning model incorporates real-world labeled data and from there derives the resulting prediction in accordance with the methods and systems disclosed herein.

[0075] Example 1: Predict the remaining life of a drill bit.

[0076]

Table 1

[0077] Example 2: Determine the minimum required tire pressure for an HGV for a long - distance trip.

[0078]

Table 2

[0079] Example 3: Determine the amount of insulin required daily for a diabetic patient.

[0080]

Table 3

[0081] The term "comprises" includes "includes" and "consisting of". For example, a composition that "comprises" X may consist only of X, or may include something additional, e.g., X + Y.

[0082] Unless otherwise specified, each embodiment described in this specification can be combined with another embodiment described in this specification.

[0083] The methods described herein may be implemented in the form of a computer program including computer program code means adapted to perform all the steps of any of the methods described herein when the program is run on a computer, by software in machine-readable form on a tangible storage medium. The computer program may be embodied on a computer-readable medium. Examples of tangible (or non-transitory) storage media include disks, thumb drives, memory cards, etc., and do not include propagated signals. The software is suitable for execution on parallel processors or serial processors such that the method steps may be performed in any suitable order or simultaneously. It is recognized that firmware and software can be valuable and separately tradable commodities. It is intended to encompass software that runs or controls on "dumb" or standard hardware to perform the desired functions. It is also intended to encompass software that "describes" or defines the configuration of hardware, such as HDL (Hardware Description Language) software used to design a silicon chip or configure a field programmable chip, to perform the desired functions.

[0084] One skilled in the art will understand that the storage devices utilized to store program instructions may be distributed across a network. For example, a remote computer can store examples of the processes described as software. A local computer or terminal computer can access the remote computer and download some or all of the software to run the program. Alternatively, the local computer can download some of the software as needed, or some software instructions can be executed on the local terminal and the remote computer (or computer network). One skilled in the art will also understand, using conventional techniques known to those skilled in the art, that all or some of the software instructions can be executed by dedicated circuits such as DSPs, programmable logic arrays, etc.

[0085] It will be understood that the above advantages and benefits may relate to one embodiment or to several embodiments. Embodiments are not limited to those that solve any or all of the problems described or have any or all of the advantages and benefits described.

[0086] "An" refers to one or more of those items. The term "comprising" is used herein to mean including the identified method blocks or elements, but such blocks or elements do not include an exclusive list and the method or apparatus may include additional blocks or elements.

[0087] The steps of the methods described herein can be performed in any suitable order or, where appropriate, simultaneously. Further, individual blocks can be deleted from any of the methods without departing from the spirit and scope of the subject matter described herein. Any aspect of any of the examples described above can be combined with any aspect of any of the other examples described to form further examples without losing the desired effect. Any of the above-described modules can be implemented in hardware or software.

[0088] It is understood that the above description of the preferred embodiments is given by way of example only and that various modifications can be made by those skilled in the art. Although various embodiments have been described with a degree of particularity or with reference to one or more individual embodiments, those skilled in the art can make numerous changes to the disclosed embodiments without departing from the scope of the invention.

[0089] Embodiments of the present disclosure operate in numerous general-purpose or special computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with aspects of the present disclosure include mobile computing devices, personal computers (e.g., desktop computers), server computers, handheld or laptop devices, multiprocessor systems, game consoles, microprocessor-based systems, set-top boxes, programmable household appliances, mobile phones, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc., but are not limited thereto.

[0090] Any combination of one or more computer-usable or computer-readable media may be utilized. For example, the computer-readable media may include one or more of a portable computer diskette, hard disk, random access memory (RAM) device, read-only memory (ROM) device, erasable programmable read-only memory (EPROM or flash memory) device, portable compact disc read-only memory (CDROM), optical storage device, and magnetic storage device. The computer program code for carrying out operations of embodiments of the present disclosure may be written in any combination of one or more programming languages.

[0091] Flowcharts and block diagrams illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code that comprises one or more executable instructions for implementing the specified logical function. It should also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions. These computer program instructions may be stored in a computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instruction means for implementing the functions / operations specified in the flowchart and / or block diagram.

[0092] Embodiments of the invention The following is a list of embodiments of the present disclosure.

[0093] Embodiment 1: A computer-implemented method for learning a distance metric D for an input space according to a large margin nearest neighbor model adapted to an output having continuous values, the method comprising: determining a transformation L for the input space, the transformation L being configured to correlate distances between input data points in the input space with differences and correlations between values of continuous outputs associated with the input data points; applying the transformation L to the input space; learning the distance metric D from the transformed input space; and a method having the above steps.

[0094] Embodiment 2: The method according to Embodiment 1, wherein determining the transformation L of the input space includes determining and minimizing a loss function for the transformation L.

[0095] Embodiment 3: Determining the loss function is the method according to any of the preceding embodiments, including, for each input data point in the input space, identifying a predetermined number of target and false data from other input data points in the input space.

[0096] Embodiment 4: With respect to the distance from the target input data point, the predetermined number of target and false data are identified from the 2X closest neighboring points, the target being X of the 2X closest neighboring points having the associated continuous output value closest to the continuous output value associated with the target input data point, the false data being the remaining X neighboring points, and X being a predetermined value, the method according to any of the preceding embodiments.

[0097] Embodiment 5: Each term of the loss function is the relative difference between the difference in the value of the continuous output associated with the input data point under consideration and the identified false data, and the difference in the value of the continuous output associated with the input data point under consideration and the identified target, weighted according to any of the preceding embodiments.

[0098] Embodiment 6: The transformation L is configured to minimize the distance between each input data point in the input space and its target, and to maximize the distance between each input data point in the input space and its false data, the method according to any of the preceding embodiments.

[0099] Embodiment 7: Each term of the loss function is weighted according to how well the relative distance between input data points in the input space reflects the relative difference in their associated values of the continuous output, and terms of the loss function for which the input data point under consideration has a stronger positive correlation between the relative distance between input data points and the relative difference in their associated values of the continuous output are more heavily weighted, the method according to any of the preceding embodiments.

[0100] Embodiment 8: The method according to any of the foregoing embodiments, wherein learning the distance metric D from the transformed input space includes calculating the Euclidean distance between input data points in the transformed input space.

[0101] Embodiment 9: The method according to any of the foregoing embodiments, wherein the input data points are derived from a normalized multi-dimensional input.

[0102] Embodiment 10: The method according to any of the foregoing embodiments, wherein the input data points are labeled with associated continuous output values.

[0103] Embodiment 11: The method according to any of the foregoing embodiments, wherein the transformation L includes a linear transformation.

[0104] Embodiment 12: A computer-implemented method for predicting values of a continuous output, the method including predicting values of a continuous output associated with a data input using a distance metric D learned for the input space by a method according to any of the preceding embodiments.

[0105] Embodiment 13: The step of receiving the data input, wherein the data input is associated with the input space; The step of transforming the data input using the transformation L; The method according to any of the preceding embodiments, comprising:

[0106] Embodiment 14: The method according to any of the foregoing embodiments, wherein predicting values of a continuous output associated with a data input using a distance metric D learned for the input space includes using a K-nearest neighbor algorithm for the data input with respect to the input space in which the distance metric D was learned.

[0107] Embodiment 15: Using the K-nearest neighbor algorithm for a data input with respect to an input space in which a distance metric D has been learned includes calculating, using the distance metric D, the distances from the data input to other input data points in the input space, the method according to any of the preceding embodiments.

[0108] Embodiment 16: Predicting the value of a continuous output associated with the data input using a distance metric D learned for the input space includes calculating the average of the values of the continuous output associated with the K nearest neighbors to the input data point in the input space, where K is a predetermined integer, the method according to any of the preceding embodiments.

[0109] Embodiment 17: Predicting the value of a continuous output associated with the data input using a distance metric D learned for the input space includes calculating the weighted average of the values of the continuous output associated with the K neighbors closest to the input data point in the input space, where K is a predetermined integer and each term of the weighted average is weighted according to the reciprocal of the distance of the neighbor from the data input, the method according to any of the preceding embodiments.

[0110] Embodiment 18: The step of verifying the predicted value of the continuous output associated with the data input; The step of labeling the data input with the verified predicted value; The step of adding the labeled data input to the input space; The step of updating the distance metric D; The method according to any of the preceding embodiments having:

[0111] Embodiment 19: The method according to any of the preceding embodiments, wherein the data input is a multi-dimensional variable and each variable includes a discrete variable or a continuous variable.

[0112] Embodiment 20: The method according to any of the preceding embodiments, wherein the value of the continuous output is a numerical value.

[0113] Embodiment 21: A system comprising one or more processors configured to execute any one of the methods of Embodiments 1 to 20.

[0114] Embodiment 22: A computer-readable medium containing instructions for causing a computer to execute instructions according to any one of the methods of Embodiments 1 to 20.

[0115] Embodiment 23: A computer program containing instructions for causing a computer to execute any one of the steps of Embodiments 1 to 20 when the program is executed by the computer.

Claims

1. A computer implementation method for learning a distance metric D over an input space according to a large-margin nearest neighbor model adapted for an output having continuous values, A step of determining a transformation L for the input space, wherein the transformation L is configured to correlate the distance between input data points in the input space with the difference between the values ​​of a series of outputs associated with the input data points; Steps include applying the conversion L to the input space; A step of learning the distance metric D from the transformed input space; A method of having.

2. Determining the transformation L for the input space involves the step of determining and minimizing a loss function with respect to the transformation L. Determining the loss function involves, for each input data point in the input space, identifying a predetermined number of target and false data from other input data points in the input space. The predetermined number of targets and false data are identified from 2X nearest neighbors with respect to their distance from the input data point under consideration, the targets are the X nearest neighbors that are closest to the continuous output value associated with the input data point under consideration, and the false data are the remaining X nearest neighbors, where X is a predetermined value. The method according to claim 1.

3. The method according to claim 2, wherein minimizing the loss function with respect to the transformation L includes determining the gradient of the loss function.

4. The method according to claim 3, wherein minimizing the loss function with respect to the transformation L includes using a quadratic gradient-based optimization algorithm for the gradient of the loss function.

5. The method according to claim 3, wherein minimizing the loss function with respect to the transformation L includes using the restricted memory Broyden-Fletcher-Goldfarb-Shanno algorithm on the gradient of the loss function.

6. The method according to claim 2, wherein each term of the loss function is weighted according to the difference between the continuous output values ​​related to the input data point under consideration and the relative difference between that and the identified false data, and the difference between the continuous output values ​​related to the input data point under consideration and the identified target.

7. The method according to claim 2, wherein the transformation L is configured to minimize the distance between each input data point in the input space and its target, and to maximize the distance between each input data point in the input space and its false data.

8. Each term of the loss function is weighted according to how well the relative distances between input data points in the input space reflect the relative differences between their associated continuous output values. Of the terms in the loss function, those input data points under consideration that have a stronger positive correlation between the relative distance between the input data points and the relative difference between their associated continuous output values ​​are weighted more heavily. The method according to claim 2.

9. The method according to claim 1, wherein learning the distance metric D from the transformed input space includes calculating the Euclidean distance between input data points in the transformed input space.

10. A computer implementation method for predicting the value of a continuous output, A method comprising the step of predicting a continuous output value associated with a data input using a distance metric D learned over an input space, according to the method described in claim 1.

11. A step of receiving the data input, wherein the data input is associated with the input space; A step of converting the data input using the conversion L; The method according to claim 10, comprising:

12. The method of predicting a continuous output value associated with a data input using a distance metric D learned over an input space, comprising using a K nearest neighbor algorithm for the data input with respect to the input space from which the distance metric D was learned, according to claim 11.

13. The method of claim 12, wherein, with respect to the input space from which the distance metric D has been learned, using the K nearest neighbor algorithm for the data input includes using the distance metric D to calculate the distance from the data input to other input data points in the input space.

14. The method of claim 12, wherein predicting a continuous output value associated with the data input using a distance metric D learned over an input space includes calculating the average of the continuous output values ​​associated with the K closest neighbors to the input data point in the input space, where K is a predetermined integer.

15. The method of claim 12, wherein predicting a value of a continuous output associated with the data input using a distance metric D learned over an input space includes calculating a weighted average of the values ​​of the continuous outputs associated with the K continuous outputs closest to the input data point in the input space, where K is a predetermined integer, and each term of the weighted average is weighted according to the reciprocal of the distance of the adjacent point from the data input.

16. A step of verifying the predicted value of the continuous output related to the aforementioned data input; A step of labeling the data input with the verified predicted value; A step of adding the data input with the aforementioned label to the input space; A step of updating the distance metric D; The method according to claim 10, comprising:

17. A system comprising one or more processors configured to perform the method according to any one of claims 1 to 16.

18. A computer-readable medium comprising instructions for causing a computer to execute instructions by the method described in any one of claims 1 to 16.

19. A computer program that, when executed by a computer, includes instructions causing the computer to perform the steps described in any one of claims 1 to 16.

Images