Optimal kernel combination method for large-scale multi-label data stream classification

Abstract

The invention discloses an optimal kernel combination method for large-scale multi-label data stream classification in the technical field of machine learning, and the method comprises the steps: introducing a random Fourier feature RFF technology, and enabling a plurality of predefined kernel functions to be explicitly mapped into fixed-dimension feature vectors; linear increase of storage and calculation overhead along with data volume caused by kernel matrix implicit calculation is effectively avoided; the learning robustness is improved by adopting a fixed label threshold strategy; an improved online updating mechanism is provided, the problem of structural risk minimization of a multi-label kernel classifier and a combination coefficient thereof is directly solved, the combination coefficient is updated through index gradient descent, and an upper bound is constructed by using a Jensen inequality to realize independent and efficient online gradient descent updating of each single-kernel classifier, so that the classifiers and the combination coefficient are synchronously optimized, and the robustness of the multi-label kernel classifier is improved. Complementary advantages of different kernels are fully mined, multiple rounds of learning of the same sample are supported, and the method has higher prediction accuracy and expansibility and is suitable for large-scale data stream scenes such as text classification and image annotation.

Description

Technical Field

[0001] This invention relates to the fields of machine learning, multi-label classification, and data stream classification, and particularly to an optimal kernel combination method. Background Technology

[0002] With the rapid development of the internet, the real world generates massive and continuously growing data streams. In applications such as text classification, image annotation, network security monitoring, and recommender systems, a single sample often has multiple category labels simultaneously. Therefore, the demand for multi-label classification models in this streaming environment is increasing. In multi-label classification tasks, after receiving multi-label samples in the form of a data stream, the model needs to identify all associated labels from a predefined label set, which places more stringent demands on the model's learning capabilities.

[0003] In multi-label classification tasks, traditional offline learning methods typically require rebuilding and retraining the model after receiving new data. This not only incurs high computational costs but also fails to meet the demands of current applications for real-time response and continuous prediction. In contrast, online learning methods can progressively update the model as data streams arrive, making them more suitable for real-time multi-label prediction. However, practical applications often face highly nonlinear prediction scenarios, making it difficult for simple linear models to achieve ideal results. Introducing kernel methods can effectively capture complex nonlinear structures, but this brings new challenges: how to choose a suitable kernel function. A suitable kernel function can significantly improve prediction performance, while an inappropriate kernel selection may lead to a decline in model performance. Existing online kernel methods for multi-label classification can be divided into single-kernel methods and multi-kernel methods. Single-kernel methods rely on a pre-selected kernel function, implicitly mapping the input data to a new high-dimensional feature space where samples are approximately linearly separable, thus allowing the application of linear classification algorithms. Within this category of methods, a family of Online Passive-Aggressive Multilabel Classification Algorithms (PAML) has been proposed, extending the margin-based learning framework of Support Vector Machines (SVM) to multilabel classification. Their learning objective is to maximize the margin of multilabel classification. Subsequently, First-order Adaptive Label Thresholding Algorithms (FALT) and First-order Fixed Label Thresholding Algorithms (FFLT) have been proposed. These algorithms introduce different label thresholding strategies and reformulate the objective of maximizing the classification margin as an unconstrained problem, thereby achieving joint optimization of all label score predictors and threshold predictors. The main limitation of the aforementioned online single-core methods is that they rely on prior knowledge of the specific task or require intensive cross-validation to pre-select a kernel function. However, in online learning scenarios, when the global data structure is unknown, this approach can easily lead to suboptimal kernel function selection. In contrast, multi-core methods employ a data-driven approach, selecting appropriate kernel functions or combining different kernel classifiers during the online learning process. In this category, a recent proposal is the Online Kernel Selection for Online Multi-Label Classification (OKS-OMC), which extends the FALT algorithm to the setting of multi-kernel learning. It constructs a joint optimization problem concerning multi-label kernel classifiers and their combination coefficients, and approximates the solution by combining FALT with the Hedge algorithm.OKS-OMC has been shown to converge to the best fixed single-core multi-label classifier selected ex post. However, OKS-OMC suffers from two key drawbacks: first, its regrettable border indicates that it primarily functions as an online kernel selector, failing to fully utilize the complementary information of multiple kernels; second, it faces the "kernel curse" because the size of each predictor that makes up the multi-label model grows linearly with the amount of data, thus impairing the scalability required for online learning. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an optimal kernel combination method for large-scale multi-label data stream classification. It utilizes the Random Fourier Feature (RFF) technique to circumvent the kernel curse problem; enhances the robustness of learning in high-dimensional feature space through a fixed label thresholding strategy; and achieves synchronous updates of classifiers and coefficients in an online environment by directly solving the joint optimization problem of multi-label kernel classifiers and their combination coefficients, thereby reaching the optimal combination of multiple kernel classifiers.

[0005] The objective of this invention is achieved as follows:

[0006] An optimal kernel combination method for classification of large-scale multi-label data streams includes the following steps:

[0007] Step 1) in the In the next round, the online prediction program receives a multi-label feature vector to be predicted. ;

[0008] Step 2) Use the Random Fourier Feature Mapping (RFF) subroutine to... Mapped to respectively From a feature space, we obtain the mapped feature vectors;

[0009] Step 3) Use the latest multi-label multi-core classifier To predict The relevant tag set, under the convex composition framework, yields... Related tag set;

[0010] Step 4) After prediction, obtain the multi-label feature vector. Real relevant tag set Subsequently, an online update subroutine for the multi-label multi-core classifier is executed. This subroutine obtains the iterative update formula for the classifier and combination coefficients by solving a structural risk minimization problem, thereby obtaining the multi-label multi-core classifier for the next round. ;

[0011] Step 5) Repeat steps 1)-4) until the data stream is used up, thus completing the online learning process of the multi-label multi-core classifier.

[0012] Furthermore, the mapped feature vectors obtained in step 2) are respectively , , where, for any , It is obtained using RFF technology and kernel function The relevant approximate nonlinear eigenmaps satisfy , The number of random Fourier components sampled for RFF.

[0013] Furthermore, in step 2), the RFF feature mapping subroutine is a technique that approximates a translation-invariant kernel function to find an explicit feature map. Its dimensions are controlled by the user, making Approximately equal to kernel function If a certain kernel function satisfies = ,in Then the function is called a translation-invariant kernel; given a translation-invariant kernel Its Fourier transform is denoted as , is represented as:

[0014] ,

[0015] Bochner's theorem guarantees that when After appropriate scaling It can be viewed as a probability distribution, according to the inverse Fourier transform theorem, It can be derived from its Fourier transform The result of the recovery process is:

[0016]

[0017] in, Expressing mathematical expectation, the last equation is obtained by eliminating the imaginary part. Obtained; further obtained:

[0018] ,

[0019] Where b is a random phase, following the pattern [ Uniform distribution on ] ;definition Then the above equation becomes

[0020] ,

[0021] This mathematical expectation can be approximated by the sample average, that is, by using... D independent samples were randomly selected from the middle. and from D random phases are sampled from the middle. ,but

[0022]

[0023] in

[0024] (1)

[0025] The RFF feature mapping subroutine simplifies the learning problem of kernel models into a problem with dimension 1. This addresses the problem of learning linear classifiers in a vector space, thus combining the efficiency of linear learning algorithms with the expressive power of nonlinear kernel functions.

[0026] Furthermore, step 3) specifically involves: using the latest multi-label multi-core classifier. To predict The relevant tag set, among which

[0027] ,

[0028] ,

[0029] ,

[0030] yes A vector consisting of the combined coefficients of each kernel classifier. Let be a probabilistic simplex, defined as ,for For the first The kernel classifier regarding the first... A score predictor for each label; the multi-label classifier is composed of... A single-core multi-label classifier and the corresponding combination coefficients Composition; prediction within a convex composite framework The relevant tag set is:

[0031] ,

[0032] in It is a predefined fixed threshold.

[0033] Furthermore, the multi-label multi-core classifier in step 4). The steps for the online update subroutine are as follows:

[0034] 4-1) Based on the prediction rules, derive the corresponding loss function to evaluate the performance of the multi-label, multi-kernel model:

[0035]

[0036] Within the framework of structural risk minimization, find This minimizes the following regularization loss;

[0037]

[0038] in

[0039] ,

[0040] ,

[0041] For regularization terms, It is a predefined regularization coefficient. It is a matrix The F-norm, It is a vector 2-norm;

[0042] 4-2) Follow these two steps to convert the multi-label multi-core classifier Updated to :

[0043] 4-2-1) Fixed for By minimizing the original objective function ), thus obtaining new combination coefficients. ;

[0044] 4-2-2) Fixed for By minimizing the original objective function The upper realm obtained .

[0045] Furthermore, in step 4-1), in the fixed for Afterwards, Perform gradient descent; because Constrained on a probabilistic simplex, therefore exponential gradient descent is applied to maintain feasibility; definition , record it The gradient at is The p-th component of the gradient The calculation formula is:

[0046]

[0047] in, = , , Represents an indicator function, under the condition At the time of its establishment, If not true, ;

[0048] We can obtain the exponential gradient formula. The update formula for each element is:

[0049] , (3)

[0050] in This is the learning rate for this update.

[0051] Furthermore, in step 4-2), in the fixed for After that, consider The following upper bound:

[0052]

[0053] in, For single-core model The loss suffered is defined as

[0054]

[0055]

[0056]

[0057] The above inequality is expressed by applying the formula to convex functions. The above is obtained using Jensen's inequality, that is Furthermore, define:

[0058]

[0059] get The upper bound is P single-core losses. A convex combination:

[0060]

[0061] According to the above formula, minimizing the upper bound function is equivalent to applying it to each... Independent Minimization For each Perform online gradient descent to update; at this time Updates and Irrelevant; the specific process is as follows;

[0062] Initially, set That is, for any arbitrarily ,set up The following update formula can be obtained based on the online gradient descent method:

[0063] ,

[0064] in for abbreviation, yes about The subgradient in The specific value at that location, It is a predefined learning rate; calculated

[0065] , (4)

[0066] in The representation is as follows:

[0067] ,

[0068] in , , bring in The iterative update formula for each single-core multi-label model is obtained as follows:

[0069] , (5)

[0070] For all kernels Applying this formula sequentially, we obtain... Thus obtain ;

[0071] Formulas (3) and (5) are allowed to be executed in each round. L Second-rate.

[0072] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0073] Compared with existing online multi-label classification methods, the method of this invention overcomes the "kernel curse" problem, significantly saves computational costs, and utilizes the complementary advantages of different kernel functions to construct a prediction model, greatly improving the online multi-label prediction performance of kernel methods. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0075] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0076] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0077] In this invention, in an online learning scenario, data arrives continuously in the form of a stream, denoted as... ,in, It is the eigenvector of dimension d that arrives at time t. The set of related labels associated with this sample. Represents the set of all possible labels. L The total number of tags. The method of this invention learns from a predefined set of kernel functions during the learning process. The adaptive combination of kernel classifiers is based on different kernel functions, where P is the number of kernel functions to choose from.

[0078] like Figure 1 The optimal kernel combination method shown includes the following steps for large-scale multi-label data stream classification:

[0079] (1) in the In this round, the program of the present invention receives a multi-label feature vector to be predicted. .

[0080] (2) Using the Random Fourier Features (RFF) subroutine to... Mapped to respectively Given a feature space, the mapped feature vectors are as follows: , , where, for any , It is obtained using RFF technology and kernel function The relevant approximate nonlinear eigenmaps satisfy , for The number of random Fourier components sampled.

[0081] (3) Use the latest multi-label classifier To predict The relevant tag set, among which

[0082] ,

[0083] ,

[0084] ,

[0085] yes A vector consisting of the combined coefficients of each kernel classifier. Let be a probabilistic simplex, defined as ,for For the first The kernel classifier regarding the first... A score predictor for each label. That is, this multi-label classifier consists of... A single-core multi-label classifier and the corresponding combination coefficients Composition. Within the convex composite framework, prediction... The relevant tag set is:

[0086] ,

[0087] in It is a predefined fixed threshold.

[0088] (4) After the prediction is completed, obtain the instance. Real relevant tag set Subsequently, an online update subroutine for the multi-label multi-core classifier is executed. This subroutine obtains the iterative update formula for the classifier and combination coefficients by solving a structural risk minimization problem, thereby obtaining the multi-label multi-core classifier for the next round. .

[0089] (5) Repeat steps (1) to (4) until the data stream is used up, thus completing the online learning process of the model.

[0090] According to the above process, the method of the present invention consists of two subroutines: Feature mapping subroutine and multi-label multi-kernel classifier The online update subroutine.

[0091] The RFF eigenmap subroutine is a technique that approximates a translation-invariant kernel function. Its core idea is to find an explicit eigenmap. Its dimensions are controlled by the user, making Approximately equal to kernel function If a certain kernel function satisfies = ,in Then this function is called a translation-invariant kernel. Given a translation-invariant kernel... Its Fourier transform is denoted as , can be represented as:

[0092] ,

[0093] Bochner's theorem guarantees that when After appropriate scaling This can be viewed as a probability distribution. According to the inverse Fourier transform theorem, It can be derived from its Fourier transform The result of the recovery process is:

[0094]

[0095] in, Expressing mathematical expectation, the last equation is achieved by eliminating the imaginary part. Therefore, it can be obtained.

[0096]

[0097] ,

[0098] Where b is a random phase, following the pattern [ Uniform distribution on ] .definition Then the above equation becomes

[0099] .

[0100] This mathematical expectation can be approximated by the sample average. (This is achieved by...) D independent samples were randomly selected from the middle. , and from D random phases are sampled from the middle. ,but

[0101]

[0102] in

[0103] (1)

[0104] Although RFF (Reference Function Forwarding) was initially designed for translation-invariant kernels, it has now been successfully extended to other kernel functions, including inverse cosine kernels and polynomial kernels. Using RFF, the learning problem of kernel models can be simplified to a problem with dimension 1. This paper addresses the learning problem of linear classifiers in vector space, thus perfectly combining the efficiency of linear learning algorithms with the expressive power of nonlinear kernel functions, effectively overcoming the kernel curse problem.

[0105] Multi-label multi-core classifier The online update subroutine is the core innovation of this invention. Multi-label multi-core classifier. The online update subroutine is designed to solve a problem related to multi-label models. and its combination coefficients The problem of minimizing structural risks.

[0106] To address this issue, a corresponding loss function is first derived based on the prediction rules of this invention to evaluate the performance of the multi-label, multi-kernel model:

[0107]

[0108] Within the framework of structural risk minimization, we hope to find This minimizes the following regularization loss.

[0109]

[0110] in, To optimize the objective function;

[0111] ,

[0112] For regularization terms, It is a predefined regularization coefficient. It is a matrix The F-norm, It is a vector 2-norm.

[0113] The above optimization problem is about It is convex, regarding It is also convex, but regarding It is nonconvex. A common approach to solving nonconvex optimization problems is to use a generalized alternating minimization strategy, which involves alternately fixing one variable and optimizing the other. However, this method converges extremely slowly in online environments. Therefore, this invention improves upon it by implementing a multi-label, multi-kernel classifier in the following two steps. Updated to :

[0114] Step a: Fix for By minimizing the original objective function , and obtain new combination coefficients ;

[0115] Step b: Fix for By minimizing the original objective function The upper realm obtained .

[0116] In step a, in the fixed for Later, due to the matter The marginal minimization problem has no closed-form solution, therefore... Perform gradient descent. Because... Constrained on a probabilistic simplex, exponential gradient descent is applied to maintain feasibility. Definition , record it The gradient at is The p-th component of the gradient The calculation formula is:

[0117]

[0118] in, = , , Represents an indicator function, under the condition At the time of its establishment, If not true, .

[0119] We can obtain the exponential gradient formula. The update formula for each element is:

[0120] , (3)

[0121] in This is the learning rate for this update.

[0122] In step b, at the fixed for After that, consider The following upper bound:

[0123]

[0124] in, For single-core model The loss suffered is defined as

[0125]

[0126]

[0127]

[0128] The above inequality is expressed by applying the formula to convex functions. Using Jensen's inequality (i.e. This is obtained from [the previous sentence]. Further, define [the following].

[0129]

[0130] It can be obtained The upper bound is P single-core losses. A convex combination:

[0131]

[0132] According to the above formula, minimizing the upper bound function is equivalent to applying it to each... Independent Minimization Since this marginal optimization problem also lacks a closed-form solution, it is possible to solve for each... Update using online gradient descent. Updates and It's irrelevant. The specific process is as follows.

[0133] Initially, set That is, for any arbitrarily ,set up Based on the online gradient descent method, the following update formula can be obtained:

[0134] ,

[0135] in for abbreviation, yes about The subgradient in The specific value at that location, It is a predefined learning rate. Calculated...

[0136] , (4)

[0137] in The representation is as follows:

[0138] ,

[0139] in , , bring in The iterative update formula for each single-core multi-label model is obtained as follows:

[0140] , (5)

[0141] For all kernels Applying this formula sequentially, we obtain... Thus obtain .

[0142] Since learning from a single sample multiple times helps improve performance, equations (3) and (5) are allowed to be executed in each round. Second-rate.

[0143] In summary, the multi-label multi-core classifier of the present invention... The steps for the online update subroutine are as follows:

[0144] (1) Calculate according to formula (2) Each component Update the combination coefficients using formula (3) to obtain ;

[0145] (2) For Update sequentially according to formula (5) Finally obtained ;

[0146] (3) Execute steps (1) and (2) a total of L times to make the sample , It is learned L times and improved. and .

[0147] The online kernel combination method for multi-label classification of large-scale data streams proposed in this invention is named OMKC (Optimal Multiple Kernel Combination algorithm). This name reflects the ability of the method to optimally integrate multiple single-kernel multi-label classifiers.

[0148] Before running the method of this invention, the hyperparameters of the algorithm need to be determined in advance, including the RFF sampling dimension. Regularization coefficient Classifier learning rate Combination coefficients and learning rate .generally A smaller value should be chosen, for example or Moreover, the more samples in the data stream, the better. The smaller the value, the better. (Classifier learning rate) The learning rate can be selected from {0.25, 0.5}, which represents the combined coefficients. It can be set to 0.25.

[0149] To make the technical solutions and advantages of the present invention clearer, typical embodiments of the present invention are given below.

[0150] When the predefined kernel set used in this invention is the Laplace kernel, the expression of the Laplace kernel is: ,in Represents the L1 norm. This is achieved by varying the kernel parameters. Different values ​​of can yield different Laplace kernels. Since the Laplace kernel is a translation-invariant kernel, its Fourier transform... equal The product of one-dimensional Cauchy distributions, where the location parameter of each Cauchy distribution is 0 and the scale parameter is also 0. Therefore, when calculating the RFF eigenvector corresponding to the Laplace kernel, it is necessary to randomly sample from the Cauchy distribution. Each value constitutes Random sampling Finally, the Fourier component was obtained. .

[0151] When the predefined kernel set used in this invention is a polynomial kernel defined on a unit sphere, the polynomial kernel has the following form: By changing the order Different polynomial kernels can be obtained. Since the polynomial kernel is not a translation-invariant kernel, a valid probability distribution can be obtained using the Spherical Random Fourier Features technique. Make its inverse Fourier transform approximately equal to the original kernel function, and then according to Perform sampling.

[0152] The specific implementation steps of this invention are as follows:

[0153] 1. Set the hyperparameters required for this method to run: , , , The settings are based on the dataset size; when the dataset size... , Set as When the dataset size , Set as Number of random Fourier components in RFF sampling The kernel size should be set according to the size of each dataset, and is generally set large enough to ensure the accuracy of the kernel function approximation.

[0154] 2. When the kernel set used in this method is a polynomial kernel, the order of the polynomial kernel is set. The set of values ​​is ,get A polynomial kernel. When the kernel set used in this method is a Laplace kernel, the Laplace kernel is set... The set of values ​​is , thus obtaining 9 Laplace nuclei.

[0155] 3. Initialization: For ,initialization for ;

[0156] 4. For Please follow these steps:

[0157] 4.1. Receive a feature vector ;

[0158] 4.2. Based on the type of kernel function used, call the corresponding RFF feature mapping program to calculate the mapped feature vector. The details are as follows:

[0159] 4.2.1. For each kernel function in the predefined kernel set, calculate... and according to Sampling was performed to obtain Fourier components and from Sampling A random phase is generated, and then an explicit feature map is constructed according to formula (1). .

[0160] 4.3. Use the latest multi-label kernel classifier. predict The relevant tag set, the prediction formula is: ;

[0161] 4.4. Receiving Real relevant tag set ;

[0162] 4.5. Perform online updates to the multi-label, multi-core classifier, as follows:

[0163] 4.5.1. Calculate according to formula (2) Each component Update the combination coefficients using formula (3) to obtain .

[0164] 4.5.2. For ,as well as Update sequentially according to formula (5) Finally obtained .

[0165] 4.5.3. Perform steps 4.5.1 and 4.5.2 a total of L times to obtain samples. , Learned L times;

[0166] 4.6. Waiting to obtain the first... Return to step 4.1 for the multi-labeled data. If no new data is received, exit the loop.

[0167] Example 1

[0168] The Tmc2007-500 dataset was selected to validate its effectiveness on large-scale data streams. This dataset is a large-scale dataset containing 28,596 texts and 22 different labels. Each text sample consists of a... The dataset is represented using 500-dimensional feature vectors. Seven polynomial kernels of varying orders are used on this dataset. The set of values ​​is RFF sampling dimension Set to 8* The hyperparameters for this method are set as follows: , , , Five percent of the data in this dataset is reserved for finding the optimal hyperparameters for the comparison algorithm.

[0169] Example 2

[0170] The Scene dataset was selected, an image dataset containing 2407 images labeled into up to six categories: beach, sunset, autumn leaves, field, mountain, and city. Each image is represented by a... The dataset is represented by 294-dimensional feature vectors. Nine Laplacian kernels are used on this dataset, with kernel parameters... The set of values ​​is RFF feature sampling dimension 16* The hyperparameters for this method are set as follows: , , , Five percent of the data in this dataset is reserved for finding the optimal hyperparameters for the comparison algorithm.

[0171] To verify the effectiveness and superiority of the method proposed in this invention, the following were selected as comparison methods: ELM-OMLL, an online multi-label learning algorithm based on Extreme Learning Machine; FFLT, an online single-core multi-label method using RFF feature mapping technology; OKS-OMC, an online kernel combination method; and PAML-I, an online single-core multi-label method.

[0172] When running the comparison algorithm, 5% of the samples were used to optimize its hyperparameters, and the remaining samples were used to evaluate the online prediction performance of each method. This invention uses seven evaluation metrics to assess the performance of multi-label prediction: precision (Psn), recall (Rcal), F1 measure, MacroF1, MicroF1, Hamming loss (Hl), and ranking loss (Rl). The specific calculation methods for each metric are as follows:

[0173] , ,

[0174] ,

[0175] ,

[0176] Where N is the number of samples used for evaluation. To measure the symmetric difference between two sets, The predictive model is represented as an instance. The real-value score assigned to label j. represents the number of true positives, false positives, and false negatives for label j, respectively. For the first five metrics, higher values ​​indicate better performance. For the last two metrics, lower values ​​indicate good performance.

[0177] The values ​​for the above metrics were all calculated online. Specifically, to simulate the data flow environment, the data in the entire dataset was randomly shuffled, and it was assumed that the data arrived sequentially. When a new instance... Upon arrival, the online learner uses its current model to predict... All relevant tags, and update each evaluation indicator based on the prediction result; when Real relevant tag set After the discovery, the learning device was used. , The predictive model is updated to make better predictions in the next round. This process is repeated until the data stream is exhausted, and the final values ​​of the seven performance metrics are recorded when the data stream is exhausted. To ensure the reliability of the experimental results, all methods are run 10 times, with a different random sorting of samples in the dataset each time. The mean and standard deviation of the evaluation metrics obtained after these 10 runs are recorded in Tables 1 and 2.

[0178] Table 1 shows the online performance metrics [%] and standard deviation (in parentheses) of the five algorithms on the Tmc2007-500 dataset. The best results for each metric are shown in bold.

[0179]

[0180] Table 2 shows the online performance metrics [%] and standard deviation (in parentheses) of the five algorithms on the Scene dataset. The best results for each metric are shown in bold.

[0181]

[0182] As shown in Table 1, on the Tmc2007-500 dataset, OKMC achieves the best results except for ranking loss (Rl), and only slightly differs from the best result in ranking loss. Table 2 shows that OKMC achieves the best results in all metrics except for recall (Rcal) and Hamming loss (H1), where it only slightly differs from the best. Considering the values ​​of the method across these seven metrics on both datasets, OKMC demonstrates the best overall online performance. OKMC's performance advantage lies in its comprehensive consideration of different kernel function suggestions, leveraging the complementary advantages of different kernel functions.

[0183] The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. An optimal kernel combination method for large-scale multi-label data stream classification, characterized in that, Includes the following steps: Step 1) in the In the next round, the online prediction program receives a multi-label feature vector to be predicted. ; Step 2) Use the Random Fourier Feature Mapping (RFF) subroutine to... Mapped to respectively From a feature space, we obtain the mapped feature vectors; Step 3) Use the latest multi-label multi-core classifier To predict The relevant tag set, under the convex composition framework, yields... Related tag set; Step 4) After prediction, obtain the multi-label feature vector. Real relevant tag set Subsequently, an online update subroutine for the multi-label multi-core classifier is executed. This subroutine obtains the iterative update formula for the classifier and combination coefficients by solving a structural risk minimization problem, thereby obtaining the multi-label multi-core classifier for the next round. ; Step 5) Repeat steps 1)-4) until the data stream is used up, thus completing the online learning process of the multi-label multi-core classifier.

2. The optimal kernel combination method for large-scale multi-label data stream classification according to claim 1, characterized in that, In step 2), the mapped feature vectors are respectively , , where, for any , It is obtained using RFF technology and kernel function The relevant approximate nonlinear eigenmaps satisfy , The number of random Fourier components sampled for RFF.

3. The optimal kernel combination method for large-scale multi-label data stream classification according to claim 2, characterized in that, In step 2), the RFF feature mapping subroutine is a technique that approximates a translation-invariant kernel function to find an explicit feature map. Its dimensions are controlled by the user, making Approximately equal to kernel function If a certain kernel function satisfies = ,in Then the function is called a translation-invariant kernel; given a translation-invariant kernel Its Fourier transform is denoted as , is represented as: ， Bochner's theorem guarantees that when After appropriate scaling It can be viewed as a probability distribution, according to the inverse Fourier transform theorem, It can be derived from its Fourier transform The result of the recovery process is: ； in, Expressing mathematical expectation, the last equation is obtained by eliminating the imaginary part. Obtained; further obtained: ， Where b is a random phase, following the pattern [ Uniform distribution on ] ;definition Then the above equation becomes ， This mathematical expectation can be approximated by the sample average, that is, by using... D independent samples were randomly selected from the middle. and from D random phases are sampled from the middle. ,but in （1） The RFF feature mapping subroutine simplifies the learning problem of kernel models into a problem with dimension 1. This addresses the problem of learning linear classifiers in a vector space, thus combining the efficiency of linear learning algorithms with the expressive power of nonlinear kernel functions.

4. The optimal kernel combination method for large-scale multi-label data stream classification according to claim 3, characterized in that, Step 3) Specifically: Use the latest multi-label multi-core classifier To predict The relevant tag set, among which ， ， ， yes A vector consisting of the combined coefficients of each kernel classifier. Let be a probabilistic simplex, defined as ,for For the first The kernel classifier regarding the first... A score predictor for each label; This multi-label classifier is composed of A single-core multi-label classifier and the corresponding combination coefficients Composition; prediction within a convex composite framework The relevant tag set is: ， in It is a predefined fixed threshold.

5. The optimal kernel combination method for large-scale multi-label data stream classification according to claim 4, characterized in that, Step 4) Multi-label multi-core classifier The steps for the online update subroutine are as follows: 4-1) Based on the prediction rules, derive the corresponding loss function to evaluate the performance of the multi-label, multi-kernel model: ； Within the framework of structural risk minimization, find This minimizes the following regularization loss; ； in ， ， For regularization terms, It is a predefined regularization coefficient. It is a matrix The F-norm, It is a vector 2-norm; 4-2) Follow these two steps to convert the multi-label multi-core classifier Updated to : 4-2-1) Fixed for By minimizing the original objective function ), thus obtaining new combination coefficients. ; 4-2-2) Fixed for By minimizing the original objective function The upper realm obtained .

6. The optimal kernel combination method for large-scale multi-label data stream classification according to claim 5, characterized in that, In step 4-1), in the fixed for Afterwards, Perform gradient descent; because Constrained on a probabilistic simplex, therefore exponential gradient descent is applied to maintain feasibility; definition , record it The gradient at is The p-th component of the gradient The calculation formula is: ； in, = , , Represents an indicator function, under the condition At the time of its establishment, If not true, ; We can obtain the exponential gradient formula. The update formula for each element is: ， （3） in This is the learning rate for this update.

7. The optimal kernel combination method for large-scale multi-label data stream classification according to claim 6, characterized in that, In step 4-2), in the fixed for After that, consider The following upper bound: ； in, For single-core model The loss suffered is defined as ； The above inequality is expressed by applying the formula to convex functions. The above is obtained using Jensen's inequality, that is Furthermore, define: ； get The upper bound is P single-core losses. A convex combination: ； According to the above formula, minimizing the upper bound function is equivalent to applying it to each... Independent Minimization For each Perform online gradient descent to update; at this time Updates and Irrelevant; the specific process is as follows; Initially, set That is, for any arbitrarily ,set up The following update formula can be obtained based on the online gradient descent method: ， in for abbreviation, yes about The subgradient in The specific value at that location, It is a predefined learning rate; calculated ， （4） in The representation is as follows: ， in , , bring in The iterative update formula for each single-core multi-label model is obtained as follows: ， （5） For all kernels Applying this formula sequentially, we obtain... Thus obtain ; Formulas (3) and (5) are allowed to be executed in each round. Second-rate.

Images