Stream data classification method and device based on bandit feedback online learning

Abstract

The present application relates to a method and apparatus for stream data classification based on online learning of bandit feedback. The method comprises: obtaining a stream data sequence, constructing a probability distribution function for each data in the stream data sequence to belong to different categories, sampling a label from the probability distribution function as a predicted label for the current stream data, and obtaining predicted bandit feedback based on the predicted label. For the classifier parameters updated at the last moment, an instantaneous loss function is constructed based on the shared feature part of the current stream data, the predicted label, and the predicted bandit feedback, and the cumulative loss is calculated. An online passive-attack learning strategy is used to construct an optimization problem for updating the shared parameters and the newly added parameters of the classifier. By solving the optimization problem and the cumulative loss, the parameters of the optimized classifier at the current moment are obtained, and the stream data is classified online using the classifier with the optimized parameters. This method can solve the classification problem when the feature dimension in the bandit feedback increases and the data scale increases.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for stream data classification based on bandit feedback online learning. Background Art

[0002] Online learning assumes that the feature space of the data stream is fixed and the true labels of the observed samples are always available. However, in real applications, the data stream acquisition environment is often dynamic, resulting in a simultaneous increase in both the amount of data and the number of feature dimensions. Furthermore, only bandit feedback about the data label information may be obtained. In other words, only whether the model's prediction of the data label is correct is known, without knowing the true label of the data.

[0003] For example, in a personalized recommendation system, given a user's profile, the system recommends potentially suitable products to that user. The user then provides feedback based on the recommendations, typically in the form of actual user behavior or specific purchases. The system has no idea what will happen if other products become available. Furthermore, as user behavior accumulates, the user profile continues to expand and become more detailed, which means that the amount of data and feature space continues to expand.

[0004] The practical problems of online learning based on incremental feature space and bandit feedback are difficult to solve because, compared with traditional learning paradigms, the data volume, feature space, and label information of the data stream all change simultaneously. Currently, there are already literatures exploring learning from bandit feedback to solve data streams with partial class label information. And more recently, a small number of literatures have studied how to learn from data streams with dynamic feature spaces to adapt to the situation where the feature space of the acquired samples is evolving. However, studying the above two challenges from a single perspective cannot effectively solve the problem of online learning where feature space amplification and bandit feedback occur simultaneously, because when designing the model, they either only consider the bandit feedback learning problem or only consider the feature space evolution learning problem, without considering both problems simultaneously. Summary of the Invention

[0005] Based on this, it is necessary to provide a stream data classification method and device based on bandit feedback online learning to address the above technical problems.

[0006] A streaming data classification method based on bandit feedback online learning, the method comprising:

[0007] Get the stream data sequence;

[0008] Constructing a probability distribution function of each data in the stream data sequence belonging to different categories;

[0009] Sampling labels from the probability distribution function as predicted labels for current stream data, and obtaining predicted bandit feedback based on the predicted labels;

[0010] For the classifier parameters updated at the last moment, construct an instantaneous loss function according to the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback, and calculate the cumulative loss;

[0011] An online passive-aggressive learning strategy is used to construct an optimization problem that converts the current moment classifier parameter update into the current moment classifier shared parameter update and the current moment classifier new parameter update; the current moment classifier parameters include the current moment classifier shared parameters and the current moment classifier new parameters, and the current moment classifier shared parameters inherit the previous moment classifier parameters;

[0012] By solving the optimization problem and the cumulative loss, the parameters of the optimized classifier at the current moment are obtained, and the streaming data is classified online using the classifier with the optimized parameters.

[0013] In one embodiment, the method further includes: constructing a probability distribution function for each data in the stream data sequence belonging to different categories:

[0014]

[0015] in, Represents the shared feature part of the current stream data, represents the classifier parameters of the jth label updated at time t-1, M represents the label set of the data stream, Represents the predicted label of the shared feature part, Represents the probability distribution function of each data in the stream data sequence belonging to different categories, γ represents the exploration parameter, The value of 1 if yes, 0 otherwise.

[0016] In one embodiment, the method further includes: based on the predicted label Get Predicted Bandit Feedback , where y t represents the true label.

[0017] In one embodiment, the method further includes: constructing an instantaneous loss function based on the classifier parameters updated at the last moment, the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback:

[0018]

[0019] in, represents the instantaneous loss function.

[0020] In one embodiment, the method further includes: using an online passive-aggressive learning strategy to construct an optimization problem for converting the current moment classifier parameter update into the current moment classifier shared parameter update and the current moment classifier new parameter update:

[0021]

[0022]

[0023] The optimization problem is: in, represents the classifier parameters at time t, represents the shared parameters of the classifier at time t, Indicates that the classifier adds new parameters at time t, represents the classifier parameters at time t-1, and the constraints are:

[0024] In one embodiment, the method further includes: introducing a slack variable into the optimization problem based on a soft margin strategy, so as to obtain an updated optimization problem:

[0025]

[0026]

[0027] in C>0 is an equilibrium parameter, ξ i represents the i-th slack variable.

[0028] A streaming data classification device based on bandit feedback online learning, the device comprising:

[0029] A distribution function construction module is used to obtain a stream data sequence and construct a probability distribution function on whether each data in the stream data sequence belongs to a different category;

[0030] a loss construction module, configured to sample a label from the probability distribution function as a predicted label for the current stream data, and obtain a predicted bandit feedback based on the predicted label; construct an instantaneous loss function based on the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback for the current classifier parameters, and calculate the cumulative loss;

[0031] An optimization problem construction module is used to construct an optimization problem for converting a current moment classifier parameter update into a current moment classifier shared parameter update and a current moment classifier newly added parameter update using an online passive-aggressive learning strategy; the current moment classifier parameters include the current moment classifier shared parameters and the current moment classifier newly added parameters, and the current moment classifier shared parameters inherit the classifier parameters of the previous moment;

[0032] The online classification module is used to obtain the parameters of the optimized classifier at the current moment by solving the optimization problem and the cumulative loss, and perform online classification on the streaming data using the classifier with optimized parameters.

[0033] In one embodiment, the distribution function construction module is further configured to construct a probability distribution function for each data in the stream data sequence belonging to different categories:

[0034]

[0035] in, Represents the shared feature part of the current stream data, represents the classifier parameters of the jth label updated at time t-1, M represents the label set of the data stream, Represents the predicted label of the shared feature part, Represents the probability distribution function of each data in the stream data sequence belonging to different categories, γ represents the exploration parameter, The value of 1 if yes, 0 otherwise.

[0036] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0037] Get the stream data sequence;

[0038] Constructing a probability distribution function of each data in the stream data sequence belonging to different categories;

[0039] Sampling labels from the probability distribution function as predicted labels for current stream data, and obtaining predicted bandit feedback based on the predicted labels;

[0040] For the classifier parameters updated at the last moment, construct an instantaneous loss function according to the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback, and calculate the cumulative loss;

[0041] An online passive-aggressive learning strategy is used to construct an optimization problem that converts the current moment classifier parameter update into the current moment classifier shared parameter update and the current moment classifier new parameter update; the current moment classifier parameters include the current moment classifier shared parameters and the current moment classifier new parameters, and the current moment classifier shared parameters inherit the previous moment classifier parameters;

[0042] By solving the optimization problem and the cumulative loss, the parameters of the optimized classifier at the current moment are obtained, and the streaming data is classified online using the classifier with the optimized parameters.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0044] Get the stream data sequence;

[0045] Constructing a probability distribution function of each data in the stream data sequence belonging to different categories;

[0046] Sampling labels from the probability distribution function as predicted labels for current stream data, and obtaining predicted bandit feedback based on the predicted labels;

[0047] For the classifier parameters updated at the last moment, construct an instantaneous loss function according to the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback, and calculate the cumulative loss;

[0048] An online passive-aggressive learning strategy is used to construct an optimization problem that converts the current moment classifier parameter update into the current moment classifier shared parameter update and the current moment classifier new parameter update; the current moment classifier parameters include the current moment classifier shared parameters and the current moment classifier new parameters, and the current moment classifier shared parameters inherit the previous moment classifier parameters;

[0049] By solving the optimization problem and the cumulative loss, the parameters of the optimized classifier at the current moment are obtained, and the streaming data is classified online using the classifier with the optimized parameters.

[0050] The aforementioned streaming data classification method, apparatus, computer device, and storage medium based on online learning using bandit feedback address the increasing number of categories and features as the feature dimensionality of streaming data increases. First, based on the probability distribution function of the data belonging to different categories, the optimal predicted label of the sample is inferred using information from the bandit feedback. Then, an instantaneous loss function is constructed based on the bandit feedback and the inferred optimal predicted label. This optimization problem and the loss function are used to train a highly dynamic multi-classifier. This multi-classifier can achieve accurate classification even as the feature dimensionality and data size increase. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 FIG2 is an application scenario diagram of stream data classification based on bandit feedback online learning in one embodiment;

[0052] Figure 2 1 is a flow chart of a method for stream data classification based on bandit feedback online learning in one embodiment;

[0053] Figure 3 1 is a structural block diagram of a stream data classification device based on bandit feedback online learning in one embodiment;

[0054] Figure 4 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0056] The stream data classification method based on bandit feedback online learning provided by the present application can be applied in a highly dynamic environment, specifically in a recommendation system, where the stream data sequence is a data sequence obtained by collecting user profiles at set time intervals. It can be seen that the profile is constantly increasing with the user's usage time. This leads to an increase in the characteristic dimension of the data. Similarly, with the update of recommended objects in the recommendation system, the current traditional classifiers can no longer meet the needs of user recommendations under high dynamics. In addition, in a stream classification system, the stream data received at different times is different. As the network conditions change, new stream data is generated. Traditional classifiers can no longer meet the needs of stream data classification under high dynamics. Therefore, it can be seen that as long as the characteristic dimension of the stream data increases and the data scale increases in a highly dynamic environment, it can be classified by the technical solution of the present invention.

[0057] In order to solve the above problems, a streaming data classification method based on bandit feedback online learning is proposed. First, consider the classification model of M categories, {x t |t=1,...,T} represents the sample obtained at time t, where It is a d t dimensional vector, d t-1 ≤d t [M] = {1,...,M} represents the label set of the data stream. Let is the classifier parameter constructed at time t-1, where is a zero vector. Figure 1 It can be seen that at time t, if the received sample x t As the feature dimension increases, the features will be divided into two parts: t-1 The shared features of the sample x t For better description, the shared features are represented as The newly added features are expressed as That is to say, This representation also works for other vectors, such as classifier parameters

[0058] In addition, Expressed as x t+1 The feature space is mapped to x t Therefore, the vector The elements in are vector x t+1 A subset of the elements in x t The elements in are in the same feature space. Similarly, the vector The elements in are vector x t+1 A subset of the elements in , but these elements are in x t outside the characteristic space of the elements in . Therefore, we have:

[0059] In one embodiment, Figure 2 As shown, a streaming data classification method based on bandit feedback online learning is provided, which includes the following steps:

[0060] Step 202: Obtain a stream data sequence.

[0061] Streaming data sequences refer to data acquired in continuous time. When performing classifier classification, the effective information contained in them is the characteristics of the data. Generally speaking, classifiers with fixed-dimensional features are currently used. In this way, they can be trained according to this paradigm using a large amount of data to maintain high accuracy. However, once the feature dimension and data scale change, traditional methods will not be able to solve the above problems, that is, accurate online learning and classification cannot be achieved.

[0062] It is worth noting that for the recommendation system, each data in the stream data sequence is the profile of the user at different times. For the stream classification system, it is the data in the stream at different times. The present invention only considers the situation where the stream has increments. It can be seen that it can also be applied in other highly dynamic environments and is no longer limited here.

[0063] Step 204: construct a probability distribution function for each data in the stream data sequence belonging to different categories.

[0064] In traditional classifier learning, learning is performed based on the true labels of samples. However, in bandit learning, only one bandit feedback can be obtained, and when the bandit feedback is 1, the classifier is updated using the true label. In the present invention, due to the increase in feature dimension, if all valid samples need to be utilized, a probability distribution can be constructed on different categories before obtaining the bandit feedback.

[0065] Step 206: sampling labels from the probability distribution function as predicted labels for the current stream data, and obtaining predicted bandit feedback based on the predicted labels.

[0066] Step 208 : For the classifier parameters updated at the last moment, an instantaneous loss function is constructed according to the shared feature part of the current stream data, the predicted label, and the predicted bandit feedback, and the cumulative loss is calculated.

[0067] Step 210 , adopting an online passive-aggressive learning strategy to construct an optimization problem of converting the current moment classifier parameter update into the current moment classifier shared parameter update and the current moment classifier new parameter update.

[0068] The classifier parameters at the current moment include the classifier shared parameters at the current moment and the classifier newly added parameters at the current moment. The classifier shared parameters at the current moment inherit the classifier parameters at the previous moment.

[0069] In step 212 , the optimized parameters of the classifier at the current moment are obtained by solving the optimization problem and the cumulative loss, and the classifier with the optimized parameters is used to perform online classification on the streaming data.

[0070] The aforementioned streaming data classification method based on online learning with bandit feedback addresses the increasing number of categories as the feature dimensionality of streaming data increases. First, based on the probability distribution function of the data belonging to different categories, the optimal predicted label of the sample is inferred using information from the bandit feedback. Then, an instantaneous loss function is constructed based on the bandit feedback and the inferred optimal predicted label. This optimization problem, the cumulative loss solution, and the loss function are used to train a highly dynamic multi-classifier. This multi-classifier can achieve accurate classification even as the feature dimensionality and data size increase.

[0071] In one embodiment, the probability distribution function of each data in the stream data sequence belonging to different categories is constructed as follows:

[0072]

[0073] in, Represents the shared feature part of the current stream data, represents the classifier parameters of the jth label at time t-1, M represents the label set of the data stream, Represents the predicted label of the shared feature part, Represents the probability distribution function of each data in the stream data sequence belonging to different categories, γ represents the exploration parameter, The value of 1 if yes, 0 otherwise.

[0074] Specifically, the traditional technology is to obtain labels by inputting samples into the classifier, and then use the predicted labels to train the classifier, which can be expressed as In the present invention, the distribution Randomly sample a label from Take it as the model for sample x t The predicted label of .

[0075] In one embodiment, based on the predicted label Get Predicted Bandit Feedback Among them, y t Represents the true label and is prepared for further loss function construction.

[0076] In one embodiment, for the current classifier parameters, according to the shared feature part of the current stream data, the predicted label and the predicted bandit feedback, an instantaneous loss function is constructed as follows:

[0077]

[0078] in, Represents the instantaneous loss function. The total loss at time t is

[0079] In order to further utilize x t and bandit feedback To update each A new learning strategy is designed based on online passive-aggressive.

[0080] Specifically, the online passive-aggressive learning strategy is used to construct the optimization problem of converting the current moment classifier parameter update into the current moment classifier shared parameter update and the current moment classifier new parameter update:

[0081]

[0082] The optimization problem is: in, represents the classifier parameters at time t, represents the shared parameters of the classifier at time t, Indicates that the classifier adds new parameters at time t, represents the classifier parameters updated at time t-1, with the following constraints:

[0083] In this embodiment, at time t, the updated parameters It consists of two parts, namely in The role of is to update the classification weights corresponding to the shared features and inherit Information in The function of is to update the classification weights corresponding to the newly added features.

[0084] From the above optimization problem, it can be seen that the present invention attempts to and Stay consistent, so that Inherit more historical information. Constraints are added to prevent overfitting.

[0085] In another embodiment, it can be seen from the above objective function that the method of the present invention requires the updated model to completely and correctly predict the current sample x t This makes it particularly sensitive to data noise. In order to alleviate this problem, a soft margin strategy with a slack variable ξ is introduced into the above objective function as follows:

[0086]

[0087] in C>0 is an equilibrium parameter, ξ i represents the i-th slack variable.

[0088] In essence, the present invention proposes two methods, one is the algorithm LIFBF without introducing slack variables, and the other is the algorithm LIFBF-V after introducing slack variables.

[0089] For the algorithm LIFBF, the solution process is as follows:

[0090] The Lagrangian function of the objective function (1) can be written as:

[0091]

[0092] in is the Lagrange multiplier. The KKT conditions for the optimization problem (1) are as follows:

[0093]

[0094] To determine The specific form of is deduced as follows: Sometimes, there are

[0095]

[0096] say The corresponding category is the support class, and its set is denoted as S t Assume S t It is known that due to Formula (3) can be written as:

[0097]

[0098] Will and Substituting the expression into the above equation, we can deduce:

[0099]

[0100] To S t Sum all m in , we have:

[0101]

[0102] There are two situations at this time:

[0103] Situation 1: When season have:

[0104]

[0105] From the above equation, we can have:

[0106]

[0107] because when When there is z t =2, otherwise it is 0. Therefore, we can get as well as The above equation can be written as:

[0108]

[0109] Use δ (m) express Will and Substituting the value of into the above equation, we have:

[0110]

[0111] Scenario 2: When Sometimes, there are Similar to case 1, we have:

[0112]

[0113] For the support set S t There are the following methods to establish :

[0114] Assume that at time t S t ≠φ, Arrange in descending order, let σ(m) represent the mth class after sorting, let Then σ(k) belongs to the support set S t The necessary and sufficient conditions are:

[0115]

[0116] Thus, the solution is completed.

[0117] For the optimization results of LIFBF-V and the support set S t How to determine.

[0118] Situation 1: When season have

[0119]

[0120] Scenario 2: When Sometimes, there are but

[0121]

[0122] For the LIFBF-V support set S t The method for determining is as follows:

[0123] Assume that at time t S t ≠φ, Arrange in descending order, let σ(m) represent the mth class after sorting, let Then σ(k) belongs to the support set S t The necessary and sufficient conditions are:

[0124]

[0125] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0126] In one embodiment, Figure 3 As shown, a streaming data classification device based on bandit feedback online learning is provided, including: a distribution function construction module 302, a loss construction module 304, an optimization problem construction module 306 and an online classification module 308, wherein:

[0127] A distribution function construction module 302 is used to obtain a stream data sequence and construct a probability distribution function for each data in the stream data sequence belonging to different categories;

[0128] The loss construction module 304 is configured to sample a label from the probability distribution function as a predicted label for the current stream data, and obtain predicted bandit feedback based on the predicted label; construct an instantaneous loss function based on the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback for the classifier parameters updated at the last moment, and calculate the cumulative loss;

[0129] An optimization problem construction module 306 is configured to construct an optimization problem for converting a current moment classifier parameter update into a current moment classifier shared parameter update and a current moment classifier newly added parameter update using an online passive-aggressive learning strategy; the current moment classifier parameters include the current moment classifier shared parameters and the current moment classifier newly added parameters, and the current moment classifier shared parameters inherit the classifier parameters updated at the previous moment;

[0130] The online classification module 308 is configured to obtain the parameters of the optimized classifier at the current moment by solving the optimization problem and the cumulative loss, and perform online classification on the streaming data using the classifier with the optimized parameters.

[0131] In one embodiment, the distribution function construction module 302 is further configured to construct a probability distribution function for each data in the stream data sequence belonging to different categories:

[0132]

[0133] in, Represents the shared feature part of the current stream data, represents the classifier parameters of the jth label at time t-1, M represents the label set of the data stream, Represents the predicted label of the shared feature part, Represents the probability distribution function of each data in the stream data sequence belonging to different categories, γ represents the exploration parameter, The value of 1 if yes, 0 otherwise.

[0134] In one embodiment, the loss construction module 304 is further configured to: Get Predicted Bandit Feedback Among them, y t represents the true label.

[0135] In one embodiment, the loss construction module 304 is further configured to construct an instantaneous loss function based on the classifier parameters updated at the last moment, the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback:

[0136]

[0137] in, represents the instantaneous loss function.

[0138] In one embodiment, the optimization problem construction module 306 is further configured to use an online passive-aggressive learning strategy to construct an optimization problem for converting a current moment classifier parameter update into a current moment classifier shared parameter update and a current moment classifier newly added parameter update:

[0139]

[0140]

[0141] The optimization problem is: in, represents the classifier parameters at time t, represents the shared parameters of the classifier at time t, Indicates that the classifier adds new parameters at time t, represents the classifier parameters at time t-1, and the constraints are:

[0142] In one embodiment, the optimization problem construction module 306 is further configured to introduce slack variables into the optimization problem based on a soft margin strategy, so that the updated optimization problem is:

[0143]

[0144]

[0145] in C>0 is an equilibrium parameter, ξ i represents the i-th slack variable.

[0146] Regarding the specific limitations of the stream data classification device based on bandit feedback online learning, please refer to the limitations of the stream data classification method based on bandit feedback online learning above, which will not be repeated here. Each module in the above-mentioned stream data classification device based on bandit feedback online learning can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0147] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a stream data classification method based on bandit feedback online learning is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0148] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0149] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0151] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0152] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0153] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A streaming data classification method based on bandit feedback online learning, characterized in that The method comprises: Get the stream data sequence; Constructing a probability distribution function of each data in the stream data sequence belonging to different categories; Sampling labels from the probability distribution function as predicted labels for current stream data, and obtaining predicted bandit feedback based on the predicted labels; For the classifier parameters updated at the last moment, construct an instantaneous loss function according to the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback, and calculate the cumulative loss; An online passive-aggressive learning strategy is used to construct an optimization problem that converts the current moment classifier parameter update into the current moment classifier shared parameter update and the current moment classifier new parameter update. The classifier parameters to be updated at the current moment include the current moment classifier shared parameters and the current moment classifier new parameters. The current moment classifier shared parameters inherit the classifier parameters at the previous moment. The optimization problem is: in, represents the classifier parameters to be updated at time t, represents the shared parameters of the classifier to be updated at time t, represents the new parameters of the classifier to be updated at time t, represents the classifier parameters updated at time t-1; By solving the optimization problem and the cumulative loss, the parameters of the optimized classifier at the current moment are obtained, and the streaming data is classified online by the classifier with the optimized parameters.

2. The method according to claim 1, characterized in that Constructing a probability distribution function for each data in the stream data sequence belonging to different categories, including: The probability distribution function of each data in the stream data sequence belonging to different categories is constructed as follows: in, Represents the shared feature part of the current stream data, represents the classifier parameters of the jth label at time t, M represents the label set of the data stream, Represents the predicted label of the shared feature part, Represents the probability distribution function of each data in the stream data sequence belonging to different categories, γ represents the exploration parameter, The value of 1 if yes, 0 otherwise.

3. The method according to claim 2, characterized in that The predicted bandit feedback is obtained based on the predicted label, including: Based on the predicted label Get Predicted Bandit Feedback Among them, y t represents the true label.

4. The method according to claim 3, characterized in that For the classifier parameters updated at the last moment, an instantaneous loss function is constructed according to the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback, including: For the classifier parameters updated at the last moment, according to the shared feature part of the current stream data, the predicted label and the predicted bandit feedback, the instantaneous loss function is constructed as follows: in, represents the instantaneous loss function.

5. The method according to claim 4, characterized in that The constraints of the optimization problem are:

6. The method according to claim 5, characterized in that The method further comprises: Based on the soft margin strategy, slack variables are introduced into the optimization problem, and the updated optimization problem is obtained as follows: in C>0 is an equilibrium parameter, ξ i represents the i-th slack variable.

7. A streaming data classification device based on bandit feedback online learning, characterized in that The device comprises: A distribution function construction module is used to obtain a stream data sequence and construct a probability distribution function on whether each data in the stream data sequence belongs to a different category; a loss construction module, configured to sample a label from the probability distribution function as a predicted label for the current stream data, and obtain predicted bandit feedback based on the predicted label; construct an instantaneous loss function based on the shared feature portion of the current stream data, the predicted label, and the predicted bandit feedback for the classifier parameters updated at the last moment, and calculate the cumulative loss; An optimization problem construction module is used to construct an optimization problem for converting the current moment classifier parameter update into the current moment classifier shared parameter update and the current moment classifier new parameter update using an online passive-aggressive learning strategy; the current moment classifier parameters include the current moment classifier shared parameters and the current moment classifier new parameters, and the current moment classifier shared parameters inherit the previous moment classifier parameters; the optimization problem is: in, represents the classifier parameters to be updated at time t, represents the shared parameters of the classifier to be updated at time t, represents the new parameters of the classifier to be updated at time t, represents the classifier parameters updated at time t-1; The online classification module is used to obtain the parameters of the optimized classifier at the current moment by solving the optimization problem and the cumulative loss, and perform online classification on the streaming data using the classifier with optimized parameters.

8. The device according to claim 7, characterized in that The distribution function construction module is also used to construct the probability distribution function of each data in the stream data sequence belonging to different categories: in, Represents the shared feature part of the current stream data, represents the classifier parameters of the jth label at time t, M represents the label set of the data stream, Represents the predicted label of the shared feature part, Represents the probability distribution function of each data in the stream data sequence belonging to different categories, γ represents the exploration parameter, The value of 1 if yes, 0 otherwise.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images