System and method for predicting values of consecutive outputs

Abstract

A computer-implemented method for learning a distance metric D of an input space according to a large-interval nearest neighbor model adapted for outputs with continuous values. The method comprises: determining a transformation L of the input space, wherein the transformation L is configured to correlate a distance between respective input data points in the input space with a difference between values of consecutive outputs associated with the respective input data points; applying the transform L to the input space; and learning the distance metric D from the transformed input space.

Description

Technical Field

[0001] The present invention relates to systems and methods for predicting the value of a continuous output, and to systems and methods for learning a distance metric based on a large-margin nearest neighbor model suitable for outputs having continuous values. Background Art

[0002] Machine learning tools and techniques are increasingly being used to automate and improve the accuracy of decision making. In particular, problems that require the analysis of multidimensional variables in order to derive patterns and metrics in order to predict the output of a given input can benefit from the analysis and learning capabilities of machine learning tools and techniques. An example of a branch of machine learning that can be used to predict the output of a given input is the branch of "supervised learning". Supervised learning provides a branch of machine learning that relies on the use of functions, which can be inferred from training data, i.e., labeled data sets (consisting of input-output pairs) related to a specific problem, to accurately predict the output of the problem for a given input. Therefore, it is important that the labeled data sets from which functions can be inferred are maintained and organized in a manner that allows improved insights to be derived from them. Therefore, by iteratively analyzing, transforming, and updating labeled data sets that can be used for supervised learning models, the accuracy of the resulting decision making, i.e., output predictions, can be significantly improved.

[0003] Methods of supervised learning may include non-parametric models, i.e., models that do not learn parameters from the training data available to the model for subsequent use of these parameters to make decisions, but in which the decisions made by the model are direct functions of the available training data. More specifically, this means that any functional form of a mapping function between a given input and an associated output that can be derived from the available training data can be used to make decisions, i.e., outputs can be predicted for a given input. Relative to parametric models in which the number of parameters is fixed with respect to the amount of data, non-parametric models make fewer assumptions about the training data in a trade-off of reducing accuracy but increasing ease of use. Therefore, non-parametric models are more versatile and reduce the need to build models, adjust various parameters, and make additional assumptions. However, the effective use of non-parametric models requires that the training data available to the model be well organized.

[0004] An example of a non-parametric model that can be used in a supervised learning method for decision making, i.e., for predicting the output of a problem for a given input, is the "K Nearest Neighbors" KNN algorithm. In short, the KNN algorithm works on the premise that objects with similar properties, e.g., input data points in an input space, e.g., labeled outputs, exist in close proximity. Thus, prediction of the output is made based on identifying a predetermined number (K) of training samples (e.g., input data points in the input space) that are closest in distance to the new input data point (i.e., the neighbors of the new input data point), and using the labels (i.e., the associated outputs) of the identified input data points to predict the output of the new input data point.

[0005] Therefore, the accuracy of non-parametric models such as the KNN algorithm depends heavily on the distance metric used to calculate samples in the training data (e.g., input data points in the input space). Therefore, in order to improve the accuracy of non-parametric models (such as the KNN algorithm), a distance metric that can be used for the training data of the model can be learned, which allows more accurate determination of the relationship between the data input and other input data points in the input space, enabling more accurate and more informed output predictions. The process of learning a distance metric is called "metric learning". Metric learning methods distort or transform the input space so that the subsequently applied algorithm is more effective. The basic idea is to try to bring similar (in terms of results, e.g., associated outputs) samples closer together while keeping dissimilar samples well separated. As described above, non-parametric models rely heavily on the distance between samples in the input space, and therefore, their performance can be significantly improved by the judicious use of metric learning.

[0006] A known model for metric learning that is suitable for non-parametric models such as the KNN algorithm is the large margin nearest neighbor LMNN model. However, the known LMNN model is only applicable to the case where the results of a given supervised learning model are discrete, i.e., for classification problems. The known LMNN model operates to distort the input space so that neighboring samples in the input space have the same class. This improves the accuracy of any subsequent classification (i.e., discrete variable) predictions made by non-parametric models such as the KNN algorithm.

[0007] The known LMNN metric learning model is not suitable for the case where the results of the supervised learning model have continuous values. Therefore, it is desirable to learn a distance metric that can be used to improve the accuracy of decision making for regression problems, that is, it is desirable to learn a distance metric for results with continuous values. Summary of the invention

[0008] As described above, known LMNN metric learning models are limited to supervised learning models used to predict discrete output values. In developing known LMNN models, the present application considers how known LMNN algorithm models are adapted to provide a transformation of an input space labeled with continuous (i.e., non-discrete) output values, and then learn a distance metric from the input space. By learning the distance metric in this way, the accuracy of prediction of continuous outputs using non-parametric models such as KNN algorithms can be improved.

[0009] Therefore, the desired goal is to learn a distance metric based on a large-margin nearest neighbor model suitable for outputs with continuous values. Based on this, a further desired goal is to predict the value of the continuous output.

[0010] According to one aspect of the present disclosure, there is provided a computer-implemented method for learning a distance metric D about an input space based on a large-margin nearest neighbor model suitable for outputs with continuous values. The method comprises: determining a transformation L about the input space, wherein the transformation L is configured to relate the distance between each input data point in the input space to the difference between the values ​​of the continuous outputs associated with the each input data point; applying the transformation L to the input space; and learning the distance metric D from the transformed input space.

[0011] According to one aspect of the present disclosure, a computer-implemented method for predicting the value of a continuous output is provided. The method includes using a learned distance metric D on an input space.

[0012] According to another aspect of the present disclosure, a system comprising one or more processors is provided, wherein the one or more processors are configured to execute the methods described herein.

[0013] According to another aspect of the present disclosure, a computer-readable medium is provided, comprising instructions for causing a computer to execute the method described herein.

[0014] According to another aspect of the present disclosure, there is provided a computer program comprising instructions, which, when executed by a computer, causes the computer to perform the steps of the method described herein.

[0015] Optional features are set out in the attached dependent claims.

[0016] These and other aspects of the present disclosure will be better appreciated and understood when considered in conjunction with the following description and accompanying drawings. The following description, while indicating various embodiments of the present disclosure and many of its specific details, is given by way of illustration and not limitation. Many substitutions, modifications, additions, or reintegrations may be made within the scope of the present disclosure, and the present disclosure includes all such substitutions, modifications, additions, or reintegrations. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 A system for performing one or more methods disclosed herein is schematically illustrated; Figure 2 is a flowchart of a computer-implemented method for learning a distance metric D about an input space according to a large-margin nearest neighbor model suitable for outputs with continuous values ​​according to an embodiment of the present invention; Figure 3 schematically illustrates how an input space may be transformed and a distance metric subsequently learned according to an embodiment of the present invention; Figure 4 is a flow chart of a computer-implemented method for predicting the value of a continuous output according to an embodiment of the present invention; and Figure 5 It is schematically shown how the K-nearest neighbor algorithm may be used according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it will be apparent to one of ordinary skill in the art that these specific details are not required to implement these embodiments.

[0019] The methods and systems described herein can be used in the context of machine learning tools and techniques, in particular, in the context of supervised learning for predicting outputs with continuous values. Embodiments of the present invention will be described in the context of supervised learning, however, it will be apparent to those skilled in the art that other suitable machine learning tools and techniques may also be used to perform embodiments of the present invention.

[0020] Figure 1 1 is a schematic diagram of a system for performing one or more methods disclosed herein. System 100 may include a client computing device 105 and a server 120 configured to be communicatively coupled via a network 110 .

[0021] The network 110 may be a wired or wireless network, such as the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a near field communication (NFC) network, Bluetooth, infrared, radio frequency, a cellular network, or another type of network. It will be understood that the network 110 may be a combination of multiple different types of wired or wireless networks.

[0022] Each client computing device 105 may be a smartphone, tablet, laptop, computer, personal data assistant, or any other type of mobile device having a hardware processor configured to process instructions and connect to one or more portions of network 110. Each client computing device 105 may have a graphical user interface configured to allow a user to interact with the processor of the client computing device 105.

[0023] Server 120 may include a physical computing device resident at a specific location, or may be deployed in a cloud computing network environment. In the present disclosure, "cloud computing" may be defined as a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be quickly provisioned via virtualization and released with minimal management effort or service provider interaction, and then scaled accordingly. The cloud model may consist of various features (e.g., on-demand self-service, WAN access, resource pools, rapid elasticity, measured services, etc.), service models (e.g., software as a service (SaaS), platform as a service (PaaS), infrastructure as a service (IaaS), and deployment models (e.g., private cloud, community cloud, public cloud, hybrid cloud, etc.). The server 120 may include any combination of one or more computer-usable or computer-readable media. In some embodiments, the server 120 may be configured to perform at least a portion of the methods disclosed herein. The server 120 may include one or more processing devices having a memory (e.g., read-only memory (ROM) and random access memory (RAM)) for storing processor-executable instructions and one or more processors for executing processor-executable commands. The processing device may include two or more processors, and the processors may operate in parallel or in a distributed manner. The processing device may execute an operating system or other software associated with the methods disclosed herein.

[0024] Figure 2 1 is a flowchart of a computer-implemented method 200 for learning a distance metric D about an input space according to a large margin nearest neighbor model suitable for outputs with continuous values ​​according to an embodiment of the present invention. The method 200 can be executed on any suitable processor, for example, on a processor located on the server 120.

[0025] As described above, the large-margin nearest neighbor LMNN model provides a known metric learning algorithm for transforming an input space and learning a distance metric about (i.e., over) the input space, thereby improving the subsequent use of a nonparametric model to predict discrete output variables to solve classification problems.

[0026] As will be appreciated by those skilled in the art, the term metric learning describes the process of determining a model-specific distance metric for a data set. A distance metric describes a function that defines the distance between individual data points in a data set as a true non-negative numerical value. A distance metric provides an indication of how close individual data points are relative to each other. Supervised learning methods typically require measuring the distance between individual data points. Although standard distance metrics are available, metric learning provides a means for determining model-specific distance metrics that can be subsequently used to improve the accuracy of predictions made by non-parametric models relative to when standard distance metrics are employed. By understanding the proximity and, therefore, similarity between individual data points in a data set, such as individual input data points in an input space (each of which is labeled with a corresponding output), an accurate prediction of the output corresponding to a new data input can be provided.

[0027] The term input space as used herein describes a user and / or model specific representation of samples, such as input data points. From this input space, labeled samples are possible combinations of values, i.e., input data points, taken from the input space, each value being labeled with an associated output value.

[0028] The method 200 for learning a distance metric D about an input space according to a large-margin nearest neighbor model suitable for outputs having continuous values ​​begins at step 210, where a transformation L about the input space is determined. The determined transformation L is configured to relate distances between respective input data points in the input space to differences between values ​​of continuous outputs associated with the respective input data points.

[0029] Reference now Figure 3 , a schematic diagram of how to transform an input space 312 and subsequently learn a distance metric according to an embodiment of the present invention is shown. Three graphics 310, 320 and 330 are shown. Graphic 310 shows an example of an input space 312 according to an embodiment of the present invention. The input space 312 may include one or more input data points 302, each of which may be characterized by one or more input dimensions (i.e., input variables). The input space 312 may be characterized by a vector space. In the example shown in the graphic 310, each input data point 302 is characterized by two input dimensions x1 and x2. It is obvious to those skilled in the art that the input data point 302 need not be limited to two input dimensions, but may instead have an input dimension of any integer in the input space 312.

[0030] In an embodiment of the present invention, the input data points 302 that make up the input space 312 may include input data points that have been manually labeled with corresponding output values ​​304 and added to the input space 312 during the establishment and / or calibration of the supervised learning model to which the input space 312 belongs. The input space 312 may also include input data points 302 for which previous predictions have been made during the runtime of the supervised learning model. The input data points 302 for which previous predictions have been made may be subsequently verified, for example manually, or may not be verified before being added to the input space 312. Each input data point 302 in the input space 312 is labeled with an associated continuous output value 304. The continuous output value 304 associated with each input data point 302 represents the output of a specific problem that the supervised learning model to which the input space 312 belongs attempts to solve, which is predicted or otherwise labeled for the input associated with the input data point 302.

[0031] In an embodiment of the present invention, the input data points 302 that constitute the input space 312 are derived from a standardized multidimensional input. The multidimensional input can be received by a supervised learning model during the establishment of the model, at runtime, or in other situations, and can be characterized by multiple discrete and / or continuous variables. In order to ensure that these multidimensional inputs can be subsequently processed efficiently and economically, input transformation techniques can be used to standardize the multidimensional inputs with respect to each dimension (i.e., variable) of the input. For example, categorical discrete variables can be one-hot transformed, ordinal discrete variables can be converted to integer values, and continuous variables can remain intact or can be rounded to a predetermined number of significant figures. It is obvious to those skilled in the art that any number of input transformation techniques can be performed to standardize the multidimensional inputs so that they are processed more efficiently and economically before or after they enter the input space 312.

[0032] In an embodiment of the present invention, one or more variables of the multidimensional input that can be received by the supervised learning model can include natural language text input. In order to ensure that these variables are properly processed, an embodiment of the present invention may include an input transformation for the natural language text input variable. In an embodiment of the present invention, the input transformation of the natural language text can be performed by decomposing all words into smaller components using a tokenization algorithm. Then, the latent Dirichlet allocation LDA model can be fitted to the available sample, wherein the sample is represented as a bag of words and / or a bag of word. Using several fitting parameters, the LDA model can then be used to discover specific topics in the natural language text data input, wherein each topic is provided by the distribution on certain words and word units. Then, these distributions can be used as features representing the sample.

[0033] A multidimensional input that may be received by a supervised learning model may include many different types of input variables. In an embodiment of the present invention, each variable (i.e., dimension) of the input may be transformed separately before all transformed variables are concatenated together to provide sample input data points. Those skilled in the art will appreciate that the above variable types and input transformation techniques are not exhaustive, and any number of types of suitable input variables and input transformation techniques may be used.

[0034] If it is determined that the sample input data point includes too many dimensions or more dimensions than required for subsequent processing, embodiments of the present invention may include selectively reducing the dimensionality of the sample input data point using known dimensionality reduction techniques so that the sample input data point can better fit into the input space 312. As will be discussed in further detail below, dimensionality reduction may also be performed when the transformation L is applied to the input space 312.

[0035] Referring again to step 210 of method 200 , a transformation L is determined with respect to input space 312 . The determined transformation L is configured to correlate distances between respective input data points 302 in input space 312 with differences between values ​​of consecutive outputs 304 associated with the respective input data points 302 .

[0036] As discussed, metric learning methods can be used to transform the input space so that the subsequently applied supervised learning model is more effective. Known LMNN models attempt to transform the input space 312 so that nearby individual input data points 302 have the same class. However, the present invention is directed to attempting to bring individual input data points 302 in the input space 312 that are similar in terms of output 304 closer together, while keeping individual input data points 302 in the input space 312 that are not similar in terms of output 204 well separated. In the context of the present invention, the output 304 associated with each input data point 302 in the input space 312 has a continuous value. By performing the transformation in this manner, the subsequent distance metric D learned can provide a more intelligent determination of the proximity of the data input points 302 in the input space 312, which can then be used to determine a more accurate prediction of the continuous output value corresponding to the received new data input.

[0037] In an embodiment of the present invention, the relationship between the distance metric D and the transformation L may be given by the following equation 1: in and represents different input data points 302 in input vector spaces 312, 314 (where 314 represents the transformed input space, see graph 330), and where D L Denotes the distance metric D, ie, a family of metrics on the transformed input space 314 .

[0038] As discussed, in the context of the present disclosure, the goal of metric learning is to learn a distance metric D about the input space 312 for which input data points 302 that are similar with respect to their associated continuous outputs 304 are grouped closer together, while input data points 302 that are dissimilar with respect to their associated continuous outputs 304 are well separated. To achieve this, embodiments of the present invention may include determining a transformation L about the input space 312 by determining and minimizing a loss function with respect to the transformation L.

[0039] The loss function provides a means for characterizing the amount of error in an algorithm. In other words, the loss function provides a means for determining how well a particular algorithm models a set of data. By calculating the loss function for a particular data set belonging to the model, the parameters that enable the amount of error, i.e., the amount of loss, to be minimized can be determined. By updating the loss function as a sample is added to each instance of the data set, the loss function can be used as a feedback mechanism to improve the accuracy of the algorithm employed.

[0040] In the context of the present invention, the loss function can be determined by considering each input data point 302 in the input space 312 and its relationship to other input data points 302 in the input space 312. By characterizing the relative distances between the various input data points 302 in the input space 312 according to a parameter (in this case, a transformation L), which should reflect how close they are with respect to the associated continuous output values ​​304, and weighting the distances according to how well they reflect the relative differences in the associated continuous output values ​​304, a parameter can be determined that minimizes the amount of error provided in the predictions of the associated supervised learning model. In the context of the present invention, the parameter is a transformation L, such as a linear transformation, which can be applied to the input space 312 to transform the input space 312 so that input data points 302 that are similar with respect to their associated continuous outputs 304 are grouped closer together, while input data points 302 that are dissimilar with respect to their continuous outputs 304 are well separated.

[0041] In an embodiment of the invention, determining the loss function may include, for each input data point 302 in the input space 312, identifying a predetermined number of targets and impostors from other input data points in the input space. In determining the loss function, each input data point 302 in the input space 312 is considered individually. For each input data point 302, the targets are those other input data points 302 that are expected to be close to the input data point under consideration, and the impostors are those other input data points 302 that are expected to be far from the input data point under consideration.

[0042] Referring now to graph 320, which depicts the same input space 312 and input data point 302 as in graph 310, a predetermined number of targets 324 and impostors 326 may be identified for the particular input data point 322 under consideration. For simplicity, the predetermined number of targets 324 and impostors 326 selected for the example shown in graph 320 is three each. It will be apparent to one skilled in the art that the predetermined number of targets 324 and impostors 326 may be higher or lower.

[0043] In an embodiment of the present invention, a predetermined number of targets 324 and impostors 326 may be identified from the 2X nearest neighbors based on their distance from the input data point 322 under consideration, wherein the targets 324 are those X neighbors of the 2X nearest neighbors that have associated continuous output values ​​304 that are closest to the continuous output value 304 associated with the input data point 322 under consideration, and wherein the impostors 326 are the remaining X neighbors, and wherein X is a predetermined value.

[0044] In the context of the target 324 and the impostor 326, it is possible to determine a transformation L that distorts the input space 312 so as to minimize the distance between each input data point 302 in the input space 312 and its target 324 and maximize the distance between each input data point 302 in the input space 312 and its impostor 326 by determining and minimizing a loss function with respect to the transformation L. When determining and minimizing a loss function with respect to the transformation L, the transformation L that can be determined does not have to be a square matrix. If the determined transformation L is a non-square matrix, the dimensionality of the data (i.e., the input data points) to which the transformation is applied will be reduced. For example, an input data point in an input space having three dimensions can be reduced to having two dimensions in the transformed input space. Of course, the dimensionality can also be maintained by the transformation L.

[0045] As discussed, in embodiments of the present invention, when determining the loss function, each term in the loss function may be weighted according to how well the relative distances between the various input data points 302 in the input space 312 reflect the relative differences in the values ​​of their associated continuous outputs 304. Terms in the loss function that have a stronger positive correlation between the relative distances between the various input data points 302 and the relative differences in the values ​​of their associated continuous outputs 304 for the considered input data points 302 are weighted more heavily than those terms that have a weaker correlation. That is, the loss function may be used to determine a transformation L that strengthens the above-mentioned correlations.

[0046] In an embodiment of the present invention, the weighting may be calculated by weighting each term in the loss function according to the relative difference between the difference in values ​​of the continuous output 304 associated with the considered input data point 322 and the identified impostor 326 and the difference in values ​​of the continuous output 304 associated with the considered input data point 322 and the identified target 324.

[0047] A possible example of a loss function that can be used is the following objective function J, which is minimized with respect to the parameter L, i.e. the transformation: in: i, j, and l refer to the sample, target, and impostor, respectively; [a] + =max(0,a) is the articulation loss; ●x i is the position of sample i in the input space; ●y i is the label of sample i, i.e., the associated continuous output value; If sample i has target j, then η ij =1, otherwise 0; If sample i has an imposter l, then δ il =1, otherwise 0; and ·y ij =y i -y j .

[0048] To minimize the objective function J with respect to the transformation L, one of any number of known suitable minimization algorithms may be used. For example, a finite-memory Broyden–Fletcher–Goldfarb–Shannon algorithm may be used, using the gradient of the objective function J, which may be calculated as follows:

[0049] Referring back to the method 200 , after determining the transformation L with respect to the input space 312 , the next step 220 of the method 200 includes applying the determined transformation L to the input space 312 .

[0050] Referring to the example shown in graph 330, the input space 312 shown in graphs 310 and 320 has been transformed by applying a transformation L to the input space 312 to produce a transformed input space 314. As can be seen from graph 330, this transformed input space provides an updated composition of the input data points 302 that seeks to improve the organization of the input data points 302 so as to correlate the distances between individual input data points 302 in the input space 314 with the differences between the values ​​of successive outputs 304 associated with the individual input data points 302.

[0051] Referring back to method 200 , after applying the transformation L to input space 312 to produce transformed input space 314 , the next step 230 of method 200 includes learning a distance metric D from transformed input space 314 .

[0052] As discussed above with reference to Equation 1, the distance metric D may include a family of metrics on the transformed input space 314. In an embodiment of the present invention, learning the distance metric D from the transformed input space 314 may include calculating the Euclidean distance between each input data point 302 in the transformed input space 314. Referring to graph 330, a distance 332 calculated between two input data points 302 in the transformed input space 314 according to the learned distance metric D is shown.

[0053] As discussed in the Background section, the accuracy of predictions obtained from non-parametric supervised learning models can be improved by learning a specific distance metric D about the input space, which seeks to use the learned distance metric D to evaluate distances in the input space.

[0054]

[0066] With respect to another aspect of the present disclosure, the relationship between the learned distance metric D about the input space and the prediction of the value of the continuous output corresponding to the received data input will be explored.

[0055] Figure 4 is a flow chart of a computer-implemented method 400 for predicting the value of a continuous output according to an embodiment of the present invention. The method may simply include using the learned distance metric D about the input space to predict the value of the continuous output associated with the data input. However, Figure 4 The method 400 depicted in FIG. 4 provides an example including additional optional steps. The method 400 may be executed on any suitable processor, for example, on a processor located on the server 120 .

[0056] Method 400 begins at step 410, where data input associated with an input space is received. In embodiments of the present invention, the received data input may be received during the design phase of an appropriate supervised learning model, i.e., solely for the purpose of training the model. In other embodiments of the present invention, the received data input may be received during the deployment of the model, i.e., when the model is used in practice. In the context of the present disclosure, data input associated with an input space refers to data input as input to a particular problem, for which an output value is predicted by a suitable supervised learning model by reference to an input space including input data points associated with the problem, i.e., previously provided as input to the problem. Methods for providing the predictions are discussed in further detail below.

[0057] In an embodiment of the present invention, the received data input may include multidimensional variables. Therefore, the received data input may have the same properties as the above-mentioned data input with reference to the multidimensional input from which the input data points 302 constituting the input space 312 are derived. The received data input may also be transformed, standardized, and dimensionally reduced in the manner described above. In an embodiment of the present invention, the multidimensional input may be received in a tabular form.

[0058] After receiving the data input in step 410, the next step 420 of method 400 includes transforming the data input. This step describes the process of transforming the data input using a transformation L, which is determined when learning a distance metric D about the input space. Figure 3 In the context of the example input spaces 312 and 314 discussed above, the transformation L maps the input space 312 to the transformed input space 314. By performing this step, a correspondence can be achieved between the form of the data input and the input space for which the distance metric D has been learned. As will be discussed later, this improves the accuracy of using the learned distance metric D for the input space to predict the value of a continuous output associated with the data input associated with the input space.

[0059] After the transformation of the data input in step 420, the next step 430 of method 400 includes predicting the value of the continuous output associated with the data input using the learned distance metric D about the input space. The distance metric D is learned according to embodiments of the present disclosure provided herein.

[0060] Prediction of the value of a continuous output associated with a data input using a learned distance metric D about the input space is possible in the context of any suitable non-parametric model for supervised learning that relies on and will benefit from the use of a learned distance metric about a given input space to improve the accuracy of the predicted output.

[0061] In an embodiment of the present invention, using the learned distance metric D about the input space to predict the value of a continuous output associated with the data input may include employing a K-nearest neighbor (KNN) algorithm for the data input with respect to the input space for which the distance metric D has been learned.

[0062] As discussed in the Background section, the KNN algorithm works under the premise that input data points with similar properties, such as associated continuous output values, exist in close proximity in the input space. Therefore, by learning an appropriate model-specific distance metric D on the input space, a more accurate determination of the similarity between individual input data points with respect to their respective associated continuous output values ​​can be achieved.

[0063] Reference now Figure 5 , a schematic diagram showing how the K nearest neighbor KNN algorithm is used according to an embodiment of the present invention. Figure 5 The graph 500 shown in FIG. Figure 3 The two-dimensional transformed input space 314 is shown in FIG. 330 .

[0064] In one embodiment of the present invention, with respect to an input space for which a distance metric D has been learned, employing a K-nearest neighbor (KNN) algorithm for a data input includes using the distance metric D to calculate distances from the data input to other input data points in the input space. Figure 5 , the received data input 502 is mapped to the transformed input space 314. Using the learned distance metric D about the input space, the distance 506 from the received data input 502 to each other input data point 302 in the input space 314 can be calculated. This allows the input data points that constitute the K nearest neighbors of the data input 502 in the input space 214 to be determined.

[0065] In an embodiment of the present invention, K may be a predetermined integer that is fixed regardless of the number of input data points 302 in the input spaces 312, 314. Alternatively, in other embodiments, K may be dynamically selected based on the number of data input points 302 in the input spaces 312, 314. For example, K may be larger for input spaces with a greater number of input data points. Figure 5 In the example shown in graph 500 , K is selected to be three. The three nearest neighbors 504 of the data input 502 have a distance 506 between the data input 502 and each of the three nearest neighbors 504 .

[0066] In an embodiment of the present invention, predicting the value of a continuous output associated with a data input 502 using the learned distance metric D about the input space 314 may include calculating an average of the values ​​of the continuous output associated with the K nearest neighbors 504 of the input data point 502 in the input space 314. Referring to the graph 500, K has been selected to be three, and thus a prediction of the value of the continuous output associated with the data input 502 may be calculated by taking the average of the continuous output values ​​associated with each of the three nearest neighbors 504 of the data input 502 in the input space 314. For example, if the continuous output values ​​associated with each of the three nearest neighbors 504 are 176.4 cm, 186 cm, and 178.2 cm, respectively, then the continuous output value associated with the data input 502 will be 180.2 cm, i.e., the average of the three values.

[0067] In an alternative embodiment of the present invention, using the learned distance metric D about the input space to predict the value of the continuous output associated with the data input 502 may include calculating a weighted average of the values ​​of the continuous output associated with the K nearest neighbors 504 of the input data point 502 in the input space 314, wherein each term of the weighted average is weighted according to the inverse of the distance 506 of the neighbor 504 from the data input 502. By performing such weighting, the prediction of the value of the continuous output associated with the data input 502 takes more account of the proximity between the data input 502 and the nearest neighbor 504 in the input space 314. Therefore, the nearest neighbor 504 that is closer to the data input 502 has a greater impact on the predicted value of the continuous output than the nearest neighbor 504 that is further away, thereby allowing for a more accurate prediction relative to when all K nearest neighbors have an equal impact on the predicted value.

[0068] As mentioned above, supervised learning models can infer functions and metrics from training data, i.e., labeled input data points in the input space, to predict outputs for future data inputs. Therefore, by increasing the pool of labeled input data points available to supervised learning models, the accuracy of predictions made by supervised learning models can be improved.

[0069] In an embodiment of the present invention, the method 400 may also include the following steps: verifying the predicted value of the continuous output associated with the data input 502, marking the data input 502 with the verified predicted value, adding the marked data input 502 to the input space 314 and updating the distance metric D. By verifying the predicted value made before marking and adding the data input 502 to the input space 314, it is possible to ensure that the inaccurately marked data input is filtered out and / or manually corrected so as not to distort the accuracy of the input space. The verification can be performed manually or in other ways, for example based on predetermined heuristic rules. If the prediction associated with the data input 502 is positively verified, the data input 502 can be marked with the prediction and added to the input space. By adding to the pool of labeled input data points 302 that can be used for supervised learning models, it is ensured that the input space 312, 314 is continuously improved for each input data point 302 processed and learned. This results in more accurate predictions.

[0070] By adapting the known LMNN model to outputs with continuous values, known supervised learning models that use non-parametric models to predict outputs with continuous values ​​can be improved in terms of both accuracy and computational efficiency. As described above, by having a greater understanding of the proximity of input data points in the input space, more reliable and more accurate predictions can then be made. The improved distribution of input data points in the input space achieved by the method disclosed herein ensures that less effort is required on the part of the non-parametric model when making predictions. Without learning the distance metric on the input space according to the method disclosed herein, a model that attempts to predict a continuous valued output for a given associated data input to a problem will need to derive additional insights and metrics to compensate for the lack of proximity effects provided by the disclosed adapted LMNN model. Therefore, these known methods require larger amounts of data and computing time and resources.

[0071] The methods and systems described herein can be used to predict outputs with continuous values ​​for a wide range of real-world problems in all technical fields. As discussed, the methods and systems disclosed herein learn about a distance metric D of the input space to provide a more improved organization of labeled input data points in the input space, thereby providing a more accurate determination of the continuous output values ​​of the received data inputs. In fact, the more labeled input data points obtained by a particular model, the more accurate the resulting predictions may be. Examples of input-output pairs of problems that can predict the value of a continuous output are provided below. For each of these problems, the corresponding supervised learning model will use real-world labeled data to derive its result predictions from the data according to the methods and systems disclosed herein.

[0072] Example 1: Predict the remaining life of a drill bit.

[0073] Example 2: Determining the minimum required tire pressure for a heavy goods vehicle (HGV) for long distance driving.

[0074] Example 3: Determine the amount of insulin a diabetic patient needs each day.

[0075] The term "comprising" encompasses "including" and "consisting of," for example, a composition "comprising" X may consist of X alone, or may include something else, such as X+Y.

[0076] Unless stated otherwise, each embodiment described herein can be combined with another embodiment described herein.

[0077] The methods described herein can be performed by software in a machine-readable form on a tangible storage medium, for example, in the form of a computer program, the computer program including a computer program code device, when the program is run on a computer, the computer program code device is suitable for performing all the steps of any method described herein, and wherein the computer program can be contained on a computer-readable medium. Examples of tangible (or non-transitory) storage media include disks, thumb drives, memory cards, etc., and do not include propagation signals. The software can be suitable for execution on a parallel processor or a serial processor, so that the method steps can be executed in any suitable order or simultaneously. This confirms that firmware and software can be valuable and separately tradable commodities. It is intended to include software that runs on or controls "non-intelligent" or standard hardware to perform the desired functions. It is also intended to include software that "describes" or defines hardware configurations, such as HDL (hardware description language) software, such as software used to design silicon chips or for configuring general-purpose programmable chips to perform desired functions.

[0078] Those skilled in the art will recognize that storage devices for storing program instructions can be distributed over a network. For example, a remote computer can store examples of processes described as software. A local or terminal computer can access the remote computer and download some or all of the software to run the program. Alternatively, the local computer can download pieces of software as needed, or execute certain software instructions at a local terminal while executing certain software instructions at a remote computer (or computer network). Those skilled in the art will also recognize that all or part of the software instructions can be executed by a dedicated circuit, such as a DSP, a programmable logic array, etc., by utilizing conventional techniques known to those skilled in the art.

[0079] It should be understood that the benefits and advantages described above may relate to one embodiment or may relate to several embodiments. The embodiments are not limited to those that solve any or all of the problems described, or those that have any or all of the benefits and advantages described.

[0080] Any reference to 'an' item refers to one or more of those items. The term 'comprising' is used herein to indicate including the identified method blocks or elements, but such blocks or elements do not comprise an exclusive list and the method or apparatus may include additional blocks or elements.

[0081] The steps of the methods described herein may be performed in any suitable order, or simultaneously where appropriate. In addition, individual boxes may be deleted from any method without departing from the spirit and scope of the subject matter described herein. The aspects of any of the above examples may be combined with the aspects of any other examples described to form additional examples without losing the effects sought. Any of the above modules may be implemented in hardware or software.

[0082] It should be understood that the above description of the preferred embodiment is given as an example only, and various modifications may be made by those skilled in the art. Although various embodiments have been described above with a certain degree of particularity or with reference to one or more separate embodiments, those skilled in the art may make many changes to the disclosed embodiments without departing from the scope of the present invention.

[0083] Embodiments of the present disclosure can operate with numerous general or special computing system environments or configurations. Examples of known computing systems, environments and / or configurations that may be applicable to various aspects of the present disclosure include, but are not limited to, mobile computing devices, personal computers (e.g., desktop computers), server computers, handheld or laptop devices, multiprocessor systems, game consoles, microprocessor-based systems, set-top boxes, programmable consumer electronics, mobile phones, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0084] Any combination of one or more computer-usable or computer-readable media may be utilized. For example, a computer-readable medium may include one or more of a portable computer disk, a hard disk, a random access memory (RAM) device, a read-only memory (ROM) device, an erasable programmable read-only memory (EPROM or flash memory) device, a portable compact disk read-only memory (CDROM), an optical storage device, and a magnetic storage device. The computer program code for performing the operations of the embodiments of the present disclosure may be written in any combination of one or more programming languages.

[0085] The flowcharts and block diagrams illustrate the possible implementation architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, segment, or portion of a code, which includes one or more executable instructions for implementing a specified logical function. It will also be noted that each box of the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented by a combination of a dedicated hardware-based system or dedicated hardware and computer instructions that performs a specified function or action. These computer program instructions may also be stored in a computer-readable medium, which may guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable medium produce an article of manufacture including an instruction device that implements the functions / actions specified in the flowchart and / or block diagram. Embodiments of the present invention The following is a list of embodiments of the present disclosure: Embodiment 1: A computer-implemented method for learning a distance metric D about an input space according to a large-margin nearest neighbor model suitable for outputs with continuous values, the method comprising determining a transformation L with respect to the input space, wherein the transformation L is configured to correlate distances between respective input data points in the input space with differences between values ​​of consecutive outputs associated with the respective input data points; applying the transformation L to the input space; and The distance metric D is learned from the transformed input space. Embodiment 2: The method according to embodiment 1, wherein determining the transformation L with respect to the input space comprises determining and minimizing a loss function with respect to the transformation L. Embodiment 3: The method according to any preceding embodiment, wherein determining the loss function comprises, for each input data point in the input space, identifying a predetermined number of targets and impostors from other input data points in the input space. Embodiment 4: A method according to any of the preceding embodiments, wherein the predetermined number of targets and impostors are identified from 2X nearest neighbors in terms of distance from the considered input data point, wherein the targets are those X neighbors among the 2X nearest neighbors having associated continuous output values ​​that are closest to the continuous output value associated with the considered input data point, wherein the impostors are the remaining X neighbors, and wherein X is a predetermined value. Embodiment 5: A method according to any of the preceding embodiments, wherein each term in the loss function is weighted according to the relative difference between the difference between the value of the continuous output associated with the considered input data point and the identified impostor and the difference between the value of the continuous output associated with the considered input data point and the identified target. Embodiment 6: A method according to any preceding embodiment, wherein the transformation L is configured to minimize the distance between each input data point in the input space and its target, and to maximize the distance between each input data point in the input space and its imposter. Embodiment 7: A method according to any of the preceding embodiments, wherein each term in the loss function is weighted according to how well the relative distances between the individual input data points in the input space reflect the relative differences in the values ​​of their associated continuous outputs, wherein terms in the loss function that have a stronger positive correlation between the relative distances between the individual input data points and the relative differences in the values ​​of their associated continuous outputs are weighted more heavily. Embodiment 8: A method according to any preceding embodiment, wherein learning the distance metric D from the transformed input space comprises calculating the Euclidean distance between each input data point in the transformed input space. Embodiment 9: A method according to any preceding embodiment, wherein the individual input data points are derived from a standardized multi-dimensional input. Embodiment 10: The method according to any preceding embodiment, wherein each input data point is labeled with an associated continuous output value. Embodiment 11: A method according to any preceding embodiment, wherein the transformation L comprises a linear transformation. Embodiment 12: A computer-implemented method for predicting the value of a continuous output, the method comprising predicting the value of the continuous output associated with a data input using a distance metric D about an input space learned according to the method of any preceding embodiment. Embodiment 13: The method according to any of the preceding embodiments, comprising: receiving the data input, wherein the data input is associated with the input space; The data input is transformed using the transformation L. Embodiment 14: A method according to any of the preceding embodiments, wherein using the learned distance metric D about the input space to predict the value of a continuous output associated with a data input comprises employing a K-nearest neighbor algorithm for the data input with respect to the input space for which the distance metric D has been learned. Embodiment 15: A method according to any of the preceding embodiments, wherein, with respect to an input space for which a distance metric D has been learned, employing a K-nearest neighbor algorithm for the data input includes using the distance metric D to calculate the distance from the data input to other input data points in the input space. Embodiment 16: A method according to any of the preceding embodiments, wherein using the learned distance metric D about the input space to predict the value of the continuous output associated with the data input includes calculating the average of the values ​​of the continuous output associated with the K nearest neighbors of the input data point in the input space, wherein K is a predetermined integer. Embodiment 17: A method according to any of the preceding embodiments, wherein using the learned distance metric D about the input space to predict the value of the continuous output associated with the data input comprises calculating a weighted average of the values ​​of the continuous output associated with the K nearest neighbors of the input data point in the input space, wherein K is a predetermined integer, and wherein each term of the weighted average is weighted according to the inverse of the distance of the neighbor from the data input. Embodiment 18: The method according to any of the preceding embodiments, comprising: verifying predicted values ​​of continuous outputs associated with said data inputs; labeling the data input with a validated predictive value; adding the labeled data input to the input space; and Update the distance metric D. Embodiment 19: A method according to any of the preceding embodiments, wherein the data input is a multi-dimensional variable, wherein each variable comprises a discrete or continuous variable. Embodiment 20: A method according to any preceding embodiment, wherein the value of the continuous output is a numerical value. Embodiment 21: A system comprising one or more processors, wherein the one or more processors are configured to perform the method of any one of embodiments 1 to 20. Embodiment 22: A computer-readable medium comprising instructions for causing a computer to execute the method according to any one of Embodiments 1 to 20. Embodiment 23: A computer program comprising instructions, when the program is executed by a computer, the instructions cause the computer to perform the steps of any one of Embodiments 1 to 20.

Claims

1. A computer-implemented method for learning a distance metric D about an input space according to a large-margin nearest neighbor model suitable for outputs having continuous values, the method comprising determining a transformation L with respect to the input space, wherein the transformation L is configured to correlate distances between respective input data points in the input space with differences between values ​​of consecutive outputs associated with the respective input data points; Applying the transformation L to the input space; as well as The distance metric D is learned from the transformed input space.

2. The method according to claim 1, wherein: Determining a transformation L with respect to the input space includes determining and minimizing a loss function with respect to the transformation L, Wherein determining the loss function comprises: for each input data point in the input space, identifying a predetermined number of targets and impostors from other input data points in the input space, and wherein the predetermined number of targets and impostors are identified from the 2X nearest neighbors in terms of distance from the input data point under consideration, wherein the targets are those X neighbors of the 2X nearest neighbors having associated continuous output values ​​that are closest to the continuous output value associated with the input data point under consideration, wherein the impostors are the remaining X neighbors, and wherein X is a predetermined value.

3. The method according to claim 2, wherein: Minimizing the loss function with respect to the transformation L comprises determining the gradient of the loss function.

4. The method according to claim 3, wherein: Minimizing the loss function with respect to the transformation L includes using a second-order gradient-based optimization algorithm on the gradient of the loss function.

5. The method according to claim 3, wherein: Minimizing the loss function with respect to the transformation L includes using a limited-memory Bleuden-Fletcher-Goldfarb-Shannon algorithm on the gradient of the loss function.

6. The method according to claim 2, wherein: Each term in the loss function is weighted according to the relative difference between the difference between the value of the continuous output associated with the considered input data point and the identified impostor and the difference between the value of the continuous output associated with the considered input data point and the identified target.

7. The method according to any one of claims 2 to 6, wherein: The transformation L is configured to minimize the distance between each input data point in the input space and its target, and to maximize the distance between each input data point in the input space and its imposter.

8. The method according to any one of claims 2 to 7, wherein: Each term in the loss function is weighted according to how well the relative distances between individual input data points in the input space reflect the relative differences in the values ​​of their associated continuous outputs, wherein terms in the loss function that have a stronger positive correlation between the relative distances between the individual input data points and the relative differences in the values ​​of their associated continuous outputs are weighted more heavily.

9. A method according to any preceding claim, wherein: Learning the distance metric D from the transformed input space includes computing the Euclidean distance between each input data point in the transformed input space.

10. A computer-implemented method for predicting the value of a continuous output, the method comprising using a distance metric D on an input space learned according to the method of any one of claims 1 to 9 to predict the value of the continuous output associated with a data input.

11. The method according to claim 10, comprising: receiving the data input, wherein the data input is associated with the input space; The data input is transformed using the transformation L.

12. The method according to claim 11, wherein: Using the learned distance metric D about an input space to predict a value of a continuous output associated with a data input includes employing a K-nearest neighbor algorithm for the data input with respect to the input space for which the distance metric D has been learned.

13. The method according to claim 12, wherein: With respect to the input space for which the distance metric D has been learned, employing a K-nearest neighbor algorithm for the data input includes using the distance metric D to calculate distances from the data input to other input data points in the input space.

14. The method according to claim 12 or 13, wherein: Predicting the value of the continuous output associated with the data input using the learned distance metric D about the input space includes calculating the average of the values ​​of the continuous output associated with K nearest neighbors of the input data point in the input space, where K is a predetermined integer.

15. The method according to claim 12 or 13, wherein: Using the learned distance metric D about the input space to predict the value of the continuous output associated with the data input includes calculating a weighted average of the values ​​of the continuous output associated with K nearest neighbors of the input data point in the input space, where K is a predetermined integer, and wherein each term in the weighted average is weighted according to the inverse of the distance of the neighbor from the data input.

16. The method according to claim 10, comprising: verifying predicted values ​​of continuous outputs associated with said data inputs; labeling the data input with a validated predictive value; adding the labeled data input to the input space; as well as Update the distance metric D.

17. A system comprising one or more processors configured to perform the method according to any one of claims 1 to 16.

18. A computer-readable medium comprising instructions for causing a computer to execute the method according to any one of claims 1 to 16.

19. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the steps of any one of claims 1 to 16.

Images