Online training and updating of factorization machines using alternating least squares optimization

Abstract

Embodiments of the present disclosure relate to online training and updating of factorization machines using alternating least squares optimization. Techniques for training a factorization machine (FM) using streaming alternating least squares (ALS) optimization are disclosed. A method of implementing the techniques according to an embodiment includes receiving a data point including a feature vector and an associated target value. The feature vector includes a user identifier, a topic identifier, and a context. The target value identifies the user's opinion of the topic. The method also includes applying the FM to the feature vector to generate an estimate of the target value, and updating the parameters of the FM for training of the FM. The parameter update is based on the application of streaming ALS optimization to the following: the data point; the estimate of the target value; and an updated sum of intermediate calculation terms generated by applying the streaming ALS optimization to previously received data points associated with a previous parameter update of the FM.

Description

Technical Field

[0001] The present disclosure relates to factorization machines configured to generate context-aware recommendations, and more particularly to techniques for training and updating factorization machines using flow-mode alternating least squares optimization. Background Art

[0002] Context-aware recommendation systems are increasingly used to provide personalized recommendations to users or predict user reactions to a given topic or situation. Many such recommendation systems employ factorization machines, which are typically trained in batch mode, that is, prediction models are trained on large data sets every few hours or days. In these systems, trained models may become stale and fail to capture trends that evolve in real time. This can be particularly a problem in use cases such as sports or news, where it is important to react quickly to changing trends to make real-time recommendations based on current viewing patterns observed on a large number of users. Another disadvantage of batch mode training is the large amount of processing time and memory storage required to process large data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1 is a block diagram of a system for training and updating a factorization machine configured in accordance with an embodiment of the present disclosure;

[0004] Figure 2 shows a matrix format for training data including feature vectors and target values ​​according to an embodiment of the present disclosure;

[0005] Figure 3 shows an example training data matrix according to an embodiment of the present disclosure;

[0006] Figure 4 is a more detailed block diagram of a flow-mode alternating least squares optimization module configured in accordance with an embodiment of the present disclosure;

[0007] Figure 5 illustrates the operation of a trained factorization machine configured in accordance with an embodiment of the present disclosure;

[0008] Figure 6 is a flowchart illustrating a method for training a factorization machine according to an embodiment of the present disclosure; and

[0009] Figure 7 is a block diagram schematically illustrating a computing platform configured according to an embodiment of the present disclosure and configured to perform any of the techniques variously described in this disclosure. DETAILED DESCRIPTION

[0010] As previously mentioned, using existing batch mode training techniques to generate context-aware predictions and recommendations using factorization machines (FMs) presents many problems because the trained models may become stale and may not capture trends that evolve in real time. In addition, batch mode training may require a large amount of processing time and memory storage requirements to process large data sets in each batch training session. To this end, it is desirable to have a system that can train and update the FM in a more efficient, incremental, and continuous manner based on each newly received data point from the data stream, without the need to perform extensive recalculations based on a large amount of previously received historical data. Such a system (which may be referred to herein as an "online" training system) can enable the FM to capture dynamic information from a streaming data source in a real-time manner by performing a single processing pass on each data point while the data point is being received.

[0011] Therefore, a technique for online training and updating a FM using a stream-mode alternating least squares (ALS) optimization is disclosed, which updates the parameters of the FM upon receiving each new data point. In more detail, and in accordance with an embodiment of the present disclosure, a FM training system is configured to receive a data point from a data stream. The data point includes a feature vector and an associated target value. The feature vector includes a user identifier, a topic identifier (e.g., a movie, a book, a web page), and a context (e.g., a plurality of associated information). The context may include additional information, such as, for example, the platform through which the user experiences the topic, the user's location, the user's emotions, and / or the identifiers of the user's companions present during the experience, to name just a few example context parameters. Additional context parameters may include: any form of user metadata (e.g., demographics, etc.), metadata associated with the topic (e.g., category, genre, author, etc.), and data associated with the session (e.g., device, browser, date, time of day, etc.). The target value identifies the user's opinion of the topic, such as, for example, a level of approval or disapproval. The system is further configured to apply the FM model to the feature vector to generate an estimate of the target value, and to calculate an error term as the difference between the actual target value and the estimate. The system is configured to then update the parameters of the FM to further train the FM based on the new (current) data point. The parameter update is accomplished by applying a stream-mode ALS optimization to the current data point, the error term, and an updated cumulative sum of previously calculated intermediate terms, the previously calculated intermediate terms being generated by the stream-mode ALS optimization in conjunction with previous parameter updates associated with previously received data points (as will be described in more detail below).

[0012] In an example use case, a trained FM is applied to a hypothetical feature vector to predict a target value associated with the hypothetical feature vector. Thus, a trained FM can be used to predict or estimate a user's reaction to a new topic or situation, and to generate recommendations to the user based on the prediction. Such predictions can be used for contextual advertising purposes, including predicting user web page views and mouse clicks in a given context, and targeting the context that will generate the most views and clicks. Recommendations can include, for example, automatic generation of products and personalized playlists (including movies, videos, music, books), as well as products, advertisements, events, identification of potential friends, etc.

[0013] Thus, the foregoing framework provides a system for implementing online training of factorization machines using streaming-mode alternating least squares optimization in a relatively efficient manner, compared to existing batch-mode training methods that operate on large amounts of data at relatively infrequent intervals. The disclosed techniques improve prediction performance by enabling the factorization machine to quickly adapt to changing trends in streaming data. Many configurations and variations will be apparent in light of this disclosure.

[0014] System Architecture

[0015] Figure 1 is a block diagram of a system 100 for training and updating a factorization machine configured in accordance with an embodiment of the present disclosure. It can be seen that the system 100 includes a streaming mode ALS optimization training and update module 120, a factorization machine 140, and a batch mode ALS optimization training module 160, the operation of which will be described in more detail below. In one embodiment, and in a high-level description, at a given moment, a new data point 115 including a feature vector x and an associated target value y is received by the system 100 from an online streaming data source 110. The new data point 115 becomes the current data point. The factorization machine 140, which implements a type of low-rank multilinear modeling, is configured to: generate an estimated target value based on the current feature vector x (i.e., the feature vector of the current data point) 170. The error term can then be calculated as the difference between the received target value y and the estimated target value The difference between. The stream mode ALS optimization training and update module 120 is configured to provide updated parameters 130 to the FM to incrementally improve the training of the FM based on the current data point, the calculated error, and the cumulative sum of previous intermediate ALS calculations. This embodiment provides a continuous learning framework. The FM parameters to be updated include: a global deviation parameter that captures the overall deviation of the data, a scalar weight associated with each feature vector element for capturing the relative importance of the element, and an interaction parameter that captures the relationship between the elements, as will be described in more detail below. The batch mode ALS optimization training module 160 is configured to pre-train or bootstrap a FM with an initial parameter set 150, for example using a batch mode version of ALS optimization, as will be described below.

[0016] In some embodiments, data point 115 may include feature vector x without associated target value y. In this case, the target value may be predicted by FM using current FM parameters. The true target value may be provided as feedback from the user at a future time, or otherwise derived or inferred from feedback from the user at a future time, at which point the error between the predicted target value and the true target value may be calculated, and the error may be used to update FM parameters. For example, in some embodiments, the true target value may be inferred from measurements of the time spent on a web page, mouse clicks, purchases, viewings, and other such activities of the user. In a more general sense, the true target value is based on feedback from the user, whether the feedback is provided directly by the user, derived from the input provided by the user, or inferred according to the action or inaction of the user. In some such cases, note that the input inferred from the user may be based on omitting or not providing affirmative input in a timely manner.

[0017] Before engaging in a more detailed explanation of FM and ALS optimization techniques, Figure 2 and Figure 3 A description of the training data is provided for consideration. Figure 2 A matrix format 200 for training data including feature vectors and target values ​​according to an embodiment of the present disclosure is shown. Figure 3 An example of a training data matrix populated with sample values ​​300 according to an embodiment of the present disclosure is provided.

[0018] First go to Figure 2, each row of the matrix S200 represents a data point including a feature vector x202 and an associated target value y212. M data points are shown in the matrix S, each data point being received at a given time t, where the top row (1) represents the current data point, the second row (2) represents the previous data point, and so on. In practice, there may be millions or more data points (rows) received from the data stream, and data points may be received at a rate of one data point per second, or more frequently, depending on the application.

[0019] Each feature vector x consists of multiple elements x i . The number of elements is called the dimension (n) of the feature vector. In this example, the first four elements encode the identity 204 of the user (U1-U4). The next three elements encode the subject as the identification 206 of the movie (F1-F3) watched by the user. The subsequent three elements encode the platform 208 (P1-P3) (e.g., TV, iPad, iPhone) on which the movie was watched, and the last two elements encode the location 210 (L1, L2) where the movie was watched. Therefore, in this example, the dimension n is 12. In this example, the target y212 encodes the numerical rating or ranking of the movie provided by the identified user. It should be understood that this is a simplified example, and in practice, the number of users, topics, platforms, and locations can be relatively large, and additional or different contextual data can be provided, such as, for example, date and time, indication of user emotions, identity of the user's companions participating in the event / experience, etc. In addition, the subject is not limited to movies, but can include almost anything, including music, television, news, sports, books, web pages, etc.

[0020] Now turn to Figure 3 , shows a populated S matrix 300 with illustrative sample values. The first eigenvector X (1) The user field 204 of (e.g., the current data point) is encoded as {1, 0, 0, 0} to identify the user as U 1 Similarly, for the first feature vector, the movie field 206 is encoded as {1, 0, 0} to identify the movie F 1 , the platform field 208 is encoded as {0, 1, 0} to identify the platform P 2 , and the position field 210 is encoded as {0, 1} to identify the position L 2. Each row typically associates one user with some combination of one movie on one platform at one location. The target field 212 indicates that User 1 rated Movie 1 as 0.9 on a normalized scale from 0 to 1, which can be said to be very favorable. Additional data points are shown. For example, the feature vector of the Mth data point indicates that User 4 watched Movie 1 on Platform 1 at Location 2, and the associated target value indicates a rating of 0.75.

[0021] As previously mentioned, the S matrix can become very large, which is one of the disadvantages of computing FM parameters using batch-mode ALS optimization, which operates on the entire S matrix (or a large portion thereof), which represents data collected over an extended period of time at periodic, infrequent intervals (e.g., hours or days). In contrast, streaming-mode ALS optimization operates only on the current data point, combined with storage of a relatively small cache of cumulative sums of intermediate calculations generated from previous data points, as will be described in more detail below.

[0022] Return now Figure 1 , and as previously described, the factorization machine 140 is configured to generate an estimated target value based on the current feature vector x of the current data point 115 170. In one embodiment, the estimation may be described by the following equation:

[0023]

[0024] where x i (t) and x j (t) are the i-th and j-th elements of the feature vector x at the current time t, is the estimated target value at the current time t, and {w 0 (t-1), w i (t-1), v i (t-1), v j (t-1)} are the parameters of FM at time t-1 calculated (updated) from the previous data point, as explained in more detail below. The global deviation parameter w 0 A scalar weight w associated with each eigenvector element captures the overall bias of the data. i captures the relative importance of this element, and the interaction parameter vector v i , v j captures the relationship between elements i and j. Each interaction parameter vector v i , v jhas length k (also called rank), and thus the total size of the interaction parameter vector is n × k. The rank is determined heuristically and is typically chosen to be a small value in the range of 4 to 8. The <, > operators in the above equation represent the vector V i and v j The dot product between .

[0025] Figure 4 1 is a more detailed block diagram of a flow mode ALS optimization training and updating module 120 configured according to an embodiment of the present disclosure. It can be seen that the module 120 includes a global bias parameter update module 410, a linear weight parameter update module 420, and an interactive parameter update module 430, configured to generate updated parameters 130 of the FM. The module 120 also includes an error calculation module 460 and a memory cache 440.

[0026] The memory cache 440 is configured to store the currently updated parameters 130 (as well as the running sum of the intermediate calculations 450, as described below) for use in calculating and updating the parameters based on the next data point received.

[0027] The error calculation module 460 is configured to calculate the error term e(t) as the estimated target value The difference between 170 and the received target value y of data point 115:

[0028]

[0029] This error term is provided to modules 410, 420 and 430 for calculation of FM parameters.

[0030] The global bias parameter update module 410 is configured to calculate an update to the global bias parameter of the FM. In some embodiments, the calculation may be performed according to the following equation:

[0031]

[0032] where w 0 (t) is the updated global bias parameter, w 0 (t-1) is the previous global bias parameter, e(t) is the error term, and |S(t)| is the number of data points that have been processed so far, which is equivalent to the number of rows in the S matrix when all data points are stored in the S matrix (e.g., batch mode processing is the case). In addition, λ(w 0 ) is a global bias regularization parameter which is chosen heuristically to reduce overfitting of the data.

[0033] The linear weight parameter updating module 420 is configured to calculate n linear weight parameters w for FMl In some embodiments, the calculation can be performed according to the following equation:

[0034]

[0035] where w l (t) is the updated linear weight parameter of the first element of x (the current feature vector), w l (t-1) is the corresponding previous linear weight parameter, e(t) is the error term, and x l (t) is the first element of x. i∈S(t) x l (i) 2 is the first eigenvector of x 1 The cumulative sum of the squares of the elements over time, where S(t) is the set of all received data points up to and including time t. This term is one of the "cumulative sums of the calculated intermediate ALS terms" mentioned earlier. In addition, λ(w l ) is a regularization parameter, which is heuristically chosen to reduce the 1 Overfitting of the data with linear weights.

[0036] The interaction parameter updating module 430 is configured to calculate n interaction parameter vectors v for FM l (each with a length of k). In some embodiments, this calculation can be performed according to the following equation:

[0037]

[0038]

[0039] where v l，f (t) is the updated interaction parameter vector v l The fth element of l is associated with the first element of x (the current eigenvector), and v l，f (t-1) is the corresponding previous interaction parameter vector. Consistent with the previous notation, x l (t) is the first element of the current feature vector x, x i (t) is the i-th element of the current eigenvector x, and e(t) is the error term. i∈S(t) h l，f (i) 2 is the function h defined l，f (t) is the cumulative sum of the squares over time, which is another of the “cumulative sums of the calculated intermediate ALS terms” mentioned earlier. In addition, λ(v l，f) is a regularization parameter that is heuristically chosen to reduce overfitting of the data for the (l, f)th interaction parameter.

[0040] It should be understood that the intermediate calculation term ∑ i∈S(t) x l (i) 2 and∑ i∈S(t) h l，f (i) 2 The cumulative sum provides an efficient mechanism for capturing an approximation of relevant features of historical data (e.g., previous data points), eliminating the requirement to store and reprocess all historical data as each new streaming data point is received. Thus, in some embodiments, the memory cache 440 only needs to be large enough to store ∑ i∈S(t) x l (i) 2 n values ​​of ∑ i∈s(t) h l，f (i) 2 In some embodiments, the memory cache 440 may also be used to store previously calculated parameters w 0 (t-1), w l (t-1), and v l，f (t-1). Therefore, the storage cost (eg, the size of the memory cache) is independent of the number of data points.

[0041] Return to Figure 1 , initialize FM 140 before operating in online streaming data mode. Batch mode ALS optimization training module 160 is configured to perform this initialization or pre-training, using batch mode ALS optimization applied to a selected subset of training data, or using other known techniques in accordance with the present disclosure, including using random initialization values. In some embodiments, for example, a pre-trained model from operations performed the day before can be used to guide today's model.

[0042] Figure 5 Operations 500 of a trained factorization machine 540 configured in accordance with an embodiment of the present disclosure are illustrated. The FM 540, which is continuously trained on each newly received data point as previously described, can be used to generate or provide recommendations y 520 based on a hypothetical feature vector z 510. For example, a hypothetical feature vector may include a user, topic, and / or context combination that has not been previously seen in the streaming data. However, the trained FM 540 can generate context-aware estimates of a user's reaction or rating to a new topic based on previously captured data, based on learned relationships between the user and other users, and ratings that other users have given to the topic or similar topics in similar contexts.

[0043] In some embodiments, this calculation may be performed according to the following equation:

[0044]

[0045] where z i and z j are the elements of the hypothetical eigenvector z, is an estimate or forecast, and {w 0 , w, v} are the current parameters of the trained FM.

[0046] method

[0047] Figure 6 6 is a flow chart showing a method 600 for online training of a factorization machine using streaming ALS optimization according to an embodiment of the present disclosure. The streaming ALS technique described herein incrementally adjusts and improves the FM, for example in real time, by updating the training based on each newly received data point in the data point stream. As can be seen, the method is based on Figure 1 and Figure 4 The method is described with reference to the configuration of the stream mode ALS optimization training and update module 120, the factorization machine 140, and the batch mode ALS optimization training module 160. However, as will be understood in light of the present disclosure, any number of module configurations may be used to implement the method. It should also be noted that the various functions described in the method need not be assigned to the specific example modules shown. To this end, the exemplary method depicted is provided to give an exemplary embodiment and is not intended to limit the method to any particular physical or structural configuration.

[0048] At operation 610, the method begins by receiving a current data point from a data point stream. For training purposes, a data point includes a feature vector and an associated target value. The feature vector includes a user identifier, a topic identifier (e.g., movie, book, web page), and a context. The context may include additional information, such as, for example, the platform on which the user experienced the topic, the user's location, the user's emotions, and / or the identifiers of the user's peers. The target value identifies the user's opinion or inclination regarding the topic and may be expressed or represented in any suitable manner, such as a numerical value.

[0049] The method continues by applying FM to the feature vector to generate an estimate of the target value at operation 620. The difference between the estimated target value and the actual target value is calculated and used as an error term.

[0050] At operation 630, the parameters of the FM are updated to further train the FM. The update is based on the application of a flow-mode alternating least squares (ALS) optimization to the current data point, the error term, and an updated running sum of intermediate calculation terms, the intermediate calculation terms being generated in association with the application of the flow-mode ALS optimization to previously received data points to perform a previous parameter update of the FM. In some embodiments, the parameters to be updated include global bias parameters, linear weight parameters, and interaction parameters, as previously described.

[0051] In some embodiments, additional operations are performed. For example, in one embodiment, the trained FM is applied to a hypothetical feature vector to generate a predicted target value associated with the hypothetical feature vector. Thus, the trained FM can be used to predict or estimate a user's reaction to a new topic or situation, and to generate recommendations to the user based on the predictions. Such recommendations can include, for example, products, playlists, or targeted advertisements.

[0052] Additionally, in some embodiments, the FM may be initialized using a batch-mode application of ALS optimization to a historical set of feature vectors and associated target values.

[0053] Example Platform

[0054] Figure 7 is a block diagram schematically illustrating a computing platform or device 700 for performing any of the various techniques described in this disclosure. For example, in some embodiments, for online training and updating Figure 1 , Figure 4 and Figure 5 a system of factorization machines (FM) or any part thereof, and Figure 7 The method or any part thereof is implemented in a computing platform 700. In some embodiments, the computing platform 700 is a computer system, such as a workstation, a desktop computer, a server, a laptop computer, a handheld computer, a tablet computer (e.g., an iPad tablet computer), a mobile computing or communication device (e.g., an iPhone mobile communication device, an Android mobile communication device, etc.), or other forms of computing or telecommunications devices that are capable of communication and have sufficient processor power and memory capacity to perform the operations described in the present disclosure. In some embodiments, a distributed computing system including a plurality of such computing devices is provided.

[0055] The computing platform 700 includes one or more storage devices 790 and / or non-transitory computer-readable media 730 encoded with one or more computer-executable instructions or software for implementing the various described techniques in the present disclosure. In some embodiments, the storage device 790 includes a computer system memory or random access memory, such as a durable disk storage device (e.g., any suitable optical or magnetic durable storage device, including RAM, ROM, flash memory, USB drive or other semiconductor-based storage medium), a hard drive, a CD-ROM or other computer-readable medium for storing data and computer-readable instructions and / or software for implementing various embodiments as taught in the present disclosure. In some embodiments, the storage device 790 also includes other types of memory or a combination thereof. In one embodiment, the storage device 790 is disposed on the computing platform 700. In another embodiment, the storage device 790 is disposed separately or away from the computing platform 700. The non-transitory computer-readable medium 730 includes, but is not limited to, one or more types of hardware memory, non-transitory tangible media (e.g., one or more magnetic storage disks, one or more optical disks, one or more USB flash drives), etc. In some embodiments, a non-transitory computer-readable medium 730 included in the computing platform 700 stores computer-readable and computer-executable instructions or software for implementing various embodiments. In one embodiment, the computer-readable medium 730 is provided on the computing platform 700. In another embodiment, the computer-readable medium 730 is provided separately from or remotely from the computing platform 700.

[0056] The computing platform 700 also includes at least one processor 710 for executing computer-readable and computer-executable instructions or software stored in the storage device 790 and / or the non-transitory computer-readable medium 730 and other programs for controlling the system hardware. In some embodiments, virtualization is employed in the computing platform 700 so that the infrastructure and resources in the computing platform 700 are dynamically shared. For example, a virtual machine is set up to handle a process running on multiple processors so that the process appears to use only one computing resource instead of multiple computing resources. In some embodiments, multiple virtual machines are used with one processor.

[0057] As can be further seen, interconnect 705 is also provided to allow communication between the various components listed above and / or other components not shown. Computing platform 700 may be coupled to a network 750 (e.g., a local area network or a wide area network such as the Internet) via network interface circuitry 740 to allow communication with other computing devices, platforms, resources, clients, and Internet of Things (IoT) devices.

[0058] In some embodiments, the user interacts with the computing platform 700 through an input / output system 760, which interfaces with devices such as a keyboard and mouse 770 and / or a display element (screen / display) 780, which provides a user interface to accept user input and provide feedback and guidance to the user. In some embodiments, the computing platform 700 includes other I / O devices (not shown) for receiving input from the user, such as a pointing device or touchpad, or any suitable user interface. In some embodiments, the computing platform 700 includes other suitable conventional I / O peripherals. The computing platform 700 may include and / or be operably coupled to various suitable devices for performing one or more aspects of the various descriptions in the present disclosure.

[0059] In some embodiments, computing platform 700 runs an operating system (OS) 720, such as any version of the Microsoft Windows operating system, different versions of Unix and Linux operating systems, any version of MacOS for Macintosh computers, any embedded operating system, any real-time operating system, any open source operating system, any proprietary operating system, any operating system for mobile computing devices, or any other operating system capable of running on computing platform 700 and performing the operations described in the present disclosure. In one embodiment, the operating system runs on one or more cloud computer instances.

[0060] It will be understood from this disclosure that the various modules and components of the system, including the stream mode ALS optimization training and update module 120, the factorization machine 140 and the batch mode ALS optimization training module 160, and submodules thereof, can be implemented in software, such as a set of instructions (e.g., HTML, XML, C, C++, object-oriented C, JavaScript, Java, BASIC, etc.) encoded on any computer-readable medium or computer program product (e.g., a hard drive, server, optical disk, or other suitable non-transitory memory or memory bank), which when executed by one or more processors causes the various methods provided in this disclosure to be implemented. It will be appreciated that in some embodiments, the various functions and data conversions performed by the user computing system as described in this disclosure can be performed by similar processors and / or databases having different configurations and arrangements, and the depicted embodiments are not intended to be limiting. The various components of this example embodiment including the computing platform 700 can be integrated into, for example, one or more desktop or laptop computers, workstations, tablet computers, smart phones, game consoles, set-top boxes, or other such computing devices. Other components and modules typical of a computing system are not shown, such as, for example, co-processors, processing cores, graphics processing units, touch pads, touch screens, etc., but will be apparent.

[0061] In other embodiments, the functional components / modules are implemented in hardware such as gate-level logic (e.g., FPGA) or application-specific semiconductors (e.g., ASIC). Other embodiments are implemented using a microcontroller with multiple input / output ports for receiving and outputting data and multiple embedded routines for performing the functions described in the present disclosure. In a more general sense, it will be apparent that any suitable combination of hardware, software, and firmware may be used.

[0062] Example Embodiments

[0063] Many example embodiments will be apparent, and the features described herein may be combined in any number of configurations.

[0064] Example 1 includes a method for training a factorization machine (FM), the method comprising: receiving, by a processor-based system, a current data point, the current data point comprising a feature vector, wherein the feature vector comprises a user identification, a topic identification, and a context; applying, by the processor-based system, the FM to the feature vector to generate an estimated target value, the estimated target value identifying the user's opinion on the topic; and updating, by the processor-based system, parameters of the FM for training of the FM, the update being based on the application of a streaming alternating least squares (ALS) optimization to: the current data point, an estimate of the target value, a true target value based on feedback from the user, and an updated sum of intermediate calculation terms generated by applying the streaming ALS optimization to previously received data points associated with previous parameter updates of the FM.

[0065] Example 2 includes the subject matter of Example 1, wherein the current data point includes a true target value.

[0066] Example 3 includes the subject matter of Example 1 or 2, further comprising: applying the trained FM to a hypothetical feature vector to generate a predicted target value associated with the hypothetical feature vector; and generating a context-aware recommendation based on the predicted target value.

[0067] Example 4 includes the subject matter of any of Examples 1 to 3, wherein the context-aware recommendation includes one or more of a playlist, a product, an advertisement, an event, and an identification of a friend.

[0068] Example 5 includes the subject matter of any one of Examples 1 to 4, wherein the context comprises one or more of: a location of the user, a mood of the user, a platform associated with the subject matter, and identification of peers of the user.

[0069] Example 6 includes the subject matter of any one of Examples 1 to 5, wherein the parameters of the FM include a global bias parameter, a plurality of linear weight parameters, and a plurality of interaction parameters.

[0070] Example 7 includes the subject matter of any of Examples 1 to 6, further comprising initializing the FM based on a batch mode application of the ALS optimization to a historical set of feature vectors and associated target values.

[0071] Example 8 is a system for performing training of a factorization machine (FM), the system comprising: one or more processors; a FM that is controllable, or executable, or both controllable and executable by the one or more processors, and the FM is configured to: generate an estimated target value based on a feature vector, wherein the feature vector includes a user identifier, a topic identifier, and a context, and wherein the target value identifies the user's opinion of the topic, and the feature vector and the target value are associated with a currently received data point; and a streaming alternating least squares (ALS) optimization module that is controllable, or executable, or both controllable and executable by the one or more processors, and the streaming ALS optimization module is configured to update parameters of the FM based on: the currently received data point, an estimate of the target value, and an updated sum of intermediate calculation terms generated by the streaming ALS optimization module operating on previously received data points associated with previous parameter updates of the FM for training of the FM.

[0072] Example 9 includes the subject matter of Example 8, wherein the trained FM is further configured to generate a predicted target value associated with the hypothetical feature vector.

[0073] Example 10 includes the subject matter of Example 8 or 9, wherein the trained FM is further configured to generate context-aware recommendations based on the predicted target value, the context-aware recommendations comprising one or more of playlists, products, advertisements, events, and identification of friends.

[0074] Example 11 includes the subject matter of any one of Examples 8 to 10, wherein the context comprises one or more of: a location of the user, a mood of the user, a platform associated with the subject matter, and identification of peers of the user.

[0075] Example 12 includes the subject matter of any one of Examples 8 to 11, wherein the parameters of the FM include a global bias parameter, a plurality of linear weight parameters, and a plurality of interaction parameters.

[0076] Example 13 includes the subject matter of any of Examples 8 to 12, further comprising a batch-mode ALS optimization module configured to initialize the FM based on a batch-mode application of ALS optimization to a historical set of feature vectors and associated target values.

[0077] Example 14 is a non-transitory computer program product having instructions encoded thereon that, when executed by one or more computer processors, cause the one or more computer processors to perform a process comprising: receiving a current data point, the current data point comprising a feature vector and an associated target value, wherein the feature vector comprises a user identification, a topic identification, and a context, and wherein the target value identifies the user's opinion of the topic; applying a FM to the feature vector to generate an estimate of the target value; and updating parameters of the FM for training of the FM, the update being based on the application of a flow-mode alternating least squares (ALS) optimization to the current data point, the estimate of the target value, and an updated sum of intermediate calculation items generated by applying the flow-mode ALS optimization to previously received data points associated with previous parameter updates of the FM.

[0078] Example 15 includes the subject matter of Example 14, further comprising applying the trained FM to a hypothetical feature vector to generate a predicted target value associated with the hypothetical feature vector.

[0079] Example 16 includes the subject matter of Example 14 or 15, further comprising: generating context-aware recommendations based on the predicted target value.

[0080] Example 17 includes the subject matter of any of Examples 14 to 16, wherein the context-aware recommendation comprises one or more of a playlist, a product, an advertisement, an event, and an identification of a friend.

[0081] Example 18 includes the subject matter of any one of Examples 14 to 17, wherein the context comprises one or more of: a location of the user, a mood of the user, a platform associated with the subject matter, and identification of peers of the user.

[0082] Example 19 includes the subject matter of any one of Examples 14 to 18, wherein the parameters of the FM include a global bias parameter, a plurality of linear weight parameters, and a plurality of interaction parameters.

[0083] Example 20 includes the subject matter of any of Examples 14 to 19, further comprising initializing the FM based on a batch mode application of the ALS optimization to a historical set of feature vectors and associated target values.

[0084] The foregoing description of the exemplary embodiments of the present disclosure has been presented for the purpose of illustration and description. It is not intended to be exhaustive or to limit the present disclosure to the exact form disclosed. Many modifications and variations are possible in light of the present disclosure. It is intended that the scope of the present disclosure is not limited by the present specific embodiments, but rather by the appended claims.

Claims

1. A method for training a factorization machine (FM), the method comprising: receiving, by a processor-based system, a current data point, the current data point comprising a feature vector, wherein the feature vector includes a user identifier, a subject identifier and a context; applying, by the processor-based system, the FM to the feature vector to generate an estimated target value, the estimated target value identifying the user's opinion of the topic; as well as Updating, by the processor-based system, parameters of the FM for training of the FM, the updating being based on application of a flow-mode alternating least squares (ALS) optimization to: The current data point, An estimate of the target value, a true target value based on feedback from the user, and An updated sum of intermediate computation terms generated by applying the streaming ALS optimization to previously received data points associated with previous parameter updates of the FM.

2. The method of claim 1, wherein the current data point comprises the true target value.

3. The method according to claim 1, further comprising: applying the trained FM to a hypothetical feature vector to generate a predicted target value associated with the hypothetical feature vector; and generating context-aware recommendations based on the predicted target values. 4 . The method of claim 3 , wherein the context-aware recommendations include one or more of playlists, products, advertisements, events, and identification of friends. 5 . The method of claim 1 , wherein the context comprises one or more of: the user's location, the user's mood, a platform associated with the topic, and identification of the user's peers. The method according to claim 1 , wherein the parameters of the FM include a global bias parameter, a plurality of linear weight parameters, and a plurality of interaction parameters.

7. The method according to claim 1, further comprising: The FM is initialized based on a batch mode application of the ALS optimization to a historical set of feature vectors and associated target values.

8. A system for performing training of a factorization machine (FM), the system comprising: one or more processors; a FM that is controllable, or executable, or both controllable and executable by the one or more processors, and the FM is configured to: generate an estimated target value based on a feature vector, wherein the feature vector includes a user identifier, a topic identifier, and a context, and wherein the target value identifies the user's opinion on the topic, the feature vector and the target value being associated with a currently received data point; and a flow-mode alternating least squares (ALS) optimization module, which is controllable, or executable, or both controllable and executable by the one or more processors, and the flow-mode ALS optimization module is configured to update parameters of the FM for training of the FM based on: The currently received data point, an estimate of the target value, and An updated sum of intermediate computation terms generated by the streaming ALS optimization module operating on previously received data points associated with previous parameter updates of the FM.

9. The system of claim 8, wherein the trained FM is further configured to generate predicted target values ​​associated with hypothetical feature vectors.

10. The system of claim 9, wherein the trained FM is further configured to generate context-aware recommendations based on the predicted target value, the context-aware recommendations comprising one or more of playlists, products, advertisements, events, and identification of friends.

11. The system of claim 8, wherein the context comprises one or more of: the user's location, the user's mood, a platform associated with the topic, and identification of the user's peers.

12. The system of claim 8, wherein the parameters of the FM include a global bias parameter, a plurality of linear weight parameters, and a plurality of interaction parameters.

13. The system of claim 8, further comprising a batch-mode ALS optimization module configured to initialize the FM based on a batch-mode application of the ALS optimization to a historical set of feature vectors and associated target values.

14. A non-transitory computer program product having encoded thereon instructions which, when executed by one or more computer processors, cause the one or more computer processors to perform a process comprising: receiving a current data point, the current data point comprising a feature vector and an associated target value, wherein the feature vector comprises a user identification, a topic identification, and a context, and wherein the target value identifies an opinion of the user with respect to the topic; applying FM to the feature vector to generate an estimate of the target value; and The parameters of the FM are updated for training of the FM, the updating being based on application of a flow-mode alternating least squares (ALS) optimization to: The current data point, an estimate of the target value, and An updated sum of intermediate computation terms generated by applying the streaming ALS optimization to previously received data points associated with previous parameter updates of the FM.

15. The non-transitory computer program product of claim 14, further comprising: The trained FM is applied to a hypothetical feature vector to generate a predicted target value associated with the hypothetical feature vector.

16. The non-transitory computer program product of claim 15, further comprising: A context-aware recommendation is generated based on the predicted target value.

17. The non-transitory computer program product of claim 16, wherein the context-aware recommendations include one or more of playlists, products, advertisements, events, and identification of friends.

18. The non-transitory computer program product of claim 14, wherein the context comprises one or more of: a location of the user, a mood of the user, a platform associated with the topic, and an identification of a companion of the user.

19. The non-transitory computer program product of claim 14, wherein the parameters of the FM include a global bias parameter, a plurality of linear weight parameters, and a plurality of interaction parameters.

20. The non-transitory computer program product of claim 14, further comprising: The FM is initialized based on a batch mode application of the ALS optimization to a historical set of feature vectors and associated target values.

Images