Online adaptive kernel selection method for image multi-label data stream classification

Abstract

The invention discloses an online adaptive kernel selection method for image multi-label data stream classification in the technical field of machine learning. The method comprises the following steps: a program receives a feature vector of a to-be-identified image; respectively mapping the feature vectors to a feature space by using a feature mapping subprogram to obtain mapped feature vectors; using a current latest multi-label kernel classifier to predict a related label set of the feature vector in real time; after the prediction is finished, an online updating subprogram of the multi-label kernel classifier is entered, and the subprogram obtains an iterative updating formula of the multi-label kernel classifier, so that the multi-label kernel classifier of the next round is obtained; waiting for the next round of image feature vector; according to the method, kernel function selection can be executed in the online learning process, meanwhile, online learning of the multi-label kernel classifier is carried out, and the online multi-label prediction performance of an existing online kernel method is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of machine learning, image multi-label classification and data stream classification, and in particular to an online adaptive kernel selection method. Background Art

[0002] Image multi-label data stream classification is an important task in the field of computer vision and machine learning. It involves real-time processing and analysis of all labels associated with an image. It is widely used in intelligent monitoring systems, social media content analysis and other fields. In the image multi-label data stream classification problem, images continuously arrive in the form of data streams, and each image may be associated with multiple labels. The task is to identify all labels associated with the object from a predefined label set in real time after each image arrives.

[0003] Traditional offline multi-label classification methods often need to retrain new classification models after new images arrive, which has a high training cost and is difficult to meet the needs of real-time recognition. In comparison, online methods can incrementally update classification models and make real-time predictions, so they play an important role in data stream classification. Among online methods, kernel methods have attracted extensive attention and research because they can significantly enhance the ability of methods to handle complex nonlinear prediction problems. In online kernel methods, the choice of kernel is crucial. A good choice can bring successful task results, while a bad choice will make the task more challenging.

[0004] In the field of multi-label classification, recently proposed online kernel methods include the PAML (Passive Aggressive Multi-Label classification) algorithm and the FALT (First-order Adaptive Label Thresholding) algorithm. For any input instance, the ideal goal of these two algorithms is to rank the score of the instance's relevant label above the label threshold and ensure that the label threshold is higher than the score of the irrelevant label, so as to separate the relevant and irrelevant labels of the instance by comparing with the label threshold. This goal is formalized as an optimization problem with L constraints. Solving this problem directly poses significant computational challenges. Therefore, PAML solves a simplified problem in which only the two constraints that the current classifier violates the most are considered. It then uses the KKT condition to derive a closed-form update solution for this simplified problem. In contrast, FALT converts the original constrained problem into an unconstrained problem, which can be easily solved using the online gradient descent method.

[0005] The above-mentioned online kernel methods that can be used for image multi-label data stream classification generally have a defect, that is, they adopt offline kernel function selection strategies. These strategies either rely on prior knowledge to predefine a kernel function, or select a suitable kernel from a set of kernels with the help of available data. However, in practical applications, there is often a lack of sufficient prior knowledge. In addition, in the context of data streams, data arrives sequentially and step by step, and the global data structure is unknown before starting online learning. Therefore, the data available for kernel selection is usually very limited. This situation will cause the offline kernel selection strategy to select suboptimal kernels, thereby affecting the performance of online multi-label prediction. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides an online adaptive kernel selection method for image multi-label data stream classification, which solves the above-mentioned defects of the existing methods, can perform kernel function selection in the process of online learning, and simultaneously perform online learning of multi-label kernel classifiers, in the hope of quickly identifying the globally optimal kernel function from a possibly large predefined kernel function pool, effectively avoiding the use of suboptimal kernels, and achieving performance as good as the best fixed single-kernel multi-label model known only after the fact, greatly improving the online multi-label prediction performance of existing online kernel methods.

[0007] The object of the present invention is achieved by: an online adaptive kernel selection method for image multi-label data stream classification, comprising the following steps:

[0008] Step 1) In round t, the program receives a feature vector of an image to be recognized

[0009] Step 2) Use the feature mapping subroutine to transform x t Map them to P feature spaces respectively and get the mapped feature vectors Among them, for any p∈{1,2,…,P}, Φ p (·) is the kernel function K p The implied nonlinear feature map, d p is the dimension of the feature space after the pth mapping;

[0010] Step 3) Use the latest multi-label kernel classifier To predict x in real time t The set of related tags, where And for any p∈{1,2,…,P}, μ t =[μ 1,t ,…,μ P,t ] T ∈Δ, and The multi-label kernel classifier consists of P single-kernel multi-label classifiers W p,t And the corresponding combination coefficient μ p,t Composed, and each single-core multi-label classifier W p,t It contains L label score predictors and 1 label threshold predictor; predict x t The relevant tag sets are:

[0011]

[0012] Step 4) After the prediction is completed, x t The real relevant label set Y t is revealed, and then enters the online update subroutine of the multi-label kernel classifier, which obtains the iterative update formula of the multi-label kernel classifier by solving a joint optimization problem involving P single-core multi-label classifiers and their combination coefficients, thereby obtaining the multi-label kernel classifier for the next round

[0013] Step 5) Return to step 1) and wait for the image feature vector of the t+1th round; if no new feature vector is passed in, exit the program.

[0014] Furthermore, before step 1), a multi-label image data stream {(x 1 ,Y 1 ),(x 2 ,Y 2 ),…,(x T ,Y T )},in It is a d-dimensional real-valued vector obtained after feature extraction of an image arriving at time t. is with x t A collection of related tags. represents the set of all possible labels, L is the total number of labels; a predefined kernel function set {K 1 ,K 2 ,...,K P} as input, where P represents the number of predefined kernel functions.

[0015] Furthermore, for any eigenvector And any p∈{1,2,…,P}, the kernel function K p Satisfy the condition K p (x i ,x j )≤1.

[0016] Furthermore, in step 3) the multi-label kernel classifier The iterative update formula is derived as follows:

[0017] First, define the following loss function to evaluate the multi-label kernel classifier Multi-label classification performance on samples (x, Y):

[0018]

[0019] Among them, |Y| represents the size of set Y, represents a set of labels that are unrelated to x. According to the regularization framework in statistical learning, we find the one that minimizes the regularization loss. That is, solve the following problem:

[0020]

[0021] in, is a regularization term, λ>0 is a predefined regularization coefficient, ||W p || F is the matrix W p The F-norm of is a vector 2-norm of ;

[0022] Then, the following loss function is defined to evaluate the single-core multi-label model W p In the sample (Φ p Classification performance on (x), Y):

[0023]

[0024] Since, μ=[μ 1 ,…,μ P ]∈Δ is a probability distribution, and the function g(z)=max{0,1-z} is a convex function. Using Jensen’s inequality, we can get In the original optimization objective, replace L with its upper bound and rearrange it to obtain the following optimization problem:

[0025]

[0026] in, represents the loss of the regularized single-core model;

[0027] Finally, we solve the above two subproblems. The first subproblem involves finding an optimal single-core multi-label model in each mapped feature space, whose goal is to minimize the regularized single-core loss on all samples; the second subproblem aims to determine the optimal combination coefficient to minimize the weighted average of the losses generated by these optimal single-core multi-label models.

[0028] Furthermore, the first subproblem is solved incrementally using the online gradient descent method:

[0029] Initially, set That is, for any p∈{1,2,…,P}, any i∈{1,2,…,L+1}, set When round t ≥ 1, we can get

[0030]

[0031] Among them, f t (W p ) is f(W p ;(Φ p (x t ),Y t )) is an abbreviation of Yes t (W p )about The subgradient of W p,t The specific value at which η<1 / λ is the predefined learning step size; the calculation is and

[0032]

[0033] in here represents an indicator function. When condition h is met, When it is not established, Bring in The iterative update formula for each single-core multi-label model is obtained:

[0034]

[0035] For all p=1,2,…,P, apply this formula in turn to get W 1,t+1 ,W 2,t+1 ,…,W P,t+1 , thus obtaining

[0036] Furthermore, the expert prediction algorithm Hedge algorithm is used to incrementally solve the second sub-problem:

[0037] is the unstandardized combination coefficient vector, μ t =[μ 1,t ,μ 2,t ,…,μ P,t ] T is the standardized combination coefficient vector, that is, here Representation vector 1-norm of; initially, When round t ≥ 1, first calculate each single-core multi-label classification model W p,t Loss incurred t (W p,t ), and then μ is obtained according to the following formula t+1 :

[0038]

[0039] where β>0 is the step size of this update, is a vector ; In each round, update formulas (1) and (2) are allowed to be executed L times.

[0040] Furthermore, in step 3) the multi-label kernel classifier The online update subroutine steps are as follows:

[0041] 3-1) In round t, the complete image data information (x t ,Y t );

[0042] 3-2) For p = 1, 2, ..., P, use formula (1) to calculate W 1,t+1 ,W 2,t+1 ,…,W P,t+1 , thus obtaining

[0043] 3-3) For p = 1, 2, ..., P, calculate f in turn t (W p,t ), calculated using formula (2) Thus we get

[0044] 3-4) Execute steps 3-2) and 3-3) for L times, so that the sample (x t ,Y t ) is learned L times and improved and

[0045] 3-5) Use formula (3) to calculate μ t+1 , and finally get the next round of multi-label kernel classifier

[0046] Furthermore, before running this method, it is necessary to determine the hyperparameters of the algorithm in advance, including the regularization coefficient λ, the learning step size η, and the learning step size β.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] Compared with the existing kernel-based online multi-label classification method, the method of the present invention eliminates the cumbersome offline kernel selection procedure, can ensure that the globally optimal kernel function is found from a predefined kernel function set during the online learning process, and the optimal kernel classifier is learned, which greatly improves the online multi-label prediction performance of the kernel method. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0050] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Given an image multi-label data stream {(x 1 ,Y 1 ),(x 2 ,Y 2 ),…,(x T ,Y T )},in It is a d-dimensional real-valued vector obtained after feature extraction of an image arriving at time t. is with x t A collection of related tags. represents the set of all possible labels, and L is the total number of labels. The method of the present invention requires the use of a predefined kernel function set {K 1 ,K 2 ,...,K P} as input, where P represents the number of predefined kernel functions.

[0053] An online adaptive kernel selection method for image multi-label data stream classification comprises the following steps:

[0054] Step 1) In round t, the program of the present invention receives a feature vector of an image to be identified

[0055] Step 2) Use the feature mapping subroutine to transform x t Map them to P feature spaces respectively and get the mapped feature vectors Among them, for any p∈{1,2,…,P}, Φ p (·) is the core K p The implied nonlinear feature map, d p is the dimension of the feature space after the pth mapping.

[0056] Step 3) Use the latest multi-label kernel classifier To predict x in real time t The set of related tags, where And for any p∈{1,2,…,P}, μ t =[μ 1,t ,…,μ P,t ] T ∈Δ, and That is to say, the multi-label kernel classifier is composed of P single-kernel multi-label classifiers W p,t And the corresponding combination coefficient μ p,t Composed, and each single-core multi-label classifier W p,t It contains L label score predictors and 1 label threshold predictor. Predict x t The relevant tag sets are:

[0057] Step 4) After the prediction is completed, x t The real relevant label set Y t is revealed, and then enters the online update subroutine of the multi-label kernel classifier, which obtains the iterative update formula of the multi-label kernel classifier by solving a joint optimization problem involving P single-core multi-label classifiers and their combination coefficients, thereby obtaining the multi-label kernel classifier for the next round

[0058] Step 5) Return to step 1) and wait for the image feature vector of the t+1th round. If no new feature vector is passed in, exit the program.

[0059] According to the above process, the method of the present invention consists of two subroutines: a feature mapping subroutine and a multi-label kernel classifier. The feature mapping subroutine depends on the type of kernel function used. The method of the present invention is applicable to many common types of kernel functions. In fact, the only requirement for kernel functions is that they must be bounded. Specifically, for any feature vector And any p∈{1,2,…,P}, kernel function Kp The condition K must be met p (x i ,x j )≤1. This requirement is true for many kernel functions, such as Gaussian kernels, Laplace kernels, exponential chi-square kernels, and isolation kernels. If the kernel used in the method of the present invention has a feature map with extremely high or even infinite dimensions, such as a Gaussian kernel, there is no need to execute the feature mapping subroutine, and the method directly jumps to the online prediction stage to calculate the prediction results using the common "kernel trick". If the kernel used in the method of the present invention has a feature map with limited dimensions, such as an isolation kernel, the existing feature mapping method for this type of kernel can be used to map x. t Mapping is done.

[0060] Multi-label kernel classifier The online update subroutine of is the core innovation of the present invention. The iterative update formula first defines the following loss function to evaluate the multi-label kernel classifier Multi-label classification performance on samples (x, Y):

[0061]

[0062] Where |Y| represents the size of set Y, represents a set of labels that are independent of x. According to the regularization framework in statistical learning, we find the one that minimizes the regularization loss. That is, solve the following problem:

[0063]

[0064] in is a regularization term, λ>0 is a predefined regularization coefficient, ||W p || F is the matrix W p The F-norm of is a vector The 2-norm of .

[0065] The above optimization problem is about is convex, and is also convex with respect to μ, but is non-convex. Since it is difficult to efficiently solve this non-convex optimization problem, we turn to find a proxy target for the optimization goal. Define the following loss function to evaluate the single-core multi-label model W p In the sample (Ф p Classification performance on (x), Y):

[0066]

[0067] Since μ=[μ 1 ,…,μ P ]∈Δ is a probability distribution, and the function g(z)=max{0,1-z} is a convex function. Using Jensen’s inequality, we can get In the original optimization goal, Replace it with its upper bound and rearrange it to get the following optimization problem:

[0068]

[0069] in represents the loss of the regularized single-core model.

[0070] The advantage of this optimization problem is that it decomposes the learning task into two simpler subproblems, each with its own independent objective. At the same time, this decomposition method still ensures that meaningful regret performance bounds can be derived. The first decomposed subproblem involves finding an optimal single-core multi-label model in each mapped feature space, with the goal of minimizing the regularized single-core loss over all samples. The second subproblem aims to determine the optimal combination coefficients to minimize the weighted average of the losses produced by these optimal single-core multi-label models.

[0071] The first subproblem is solved incrementally using online gradient descent. Initially, we set That is, for any p∈{1,2,…,P}, any i∈{1,2,…,L+1}, set When round t ≥ 1, we can get

[0072]

[0073] where f t (W p ) is f(W p ;(Φ p (x t ),Y t )) is an abbreviation of Yes t (W p )about The subgradient of W p,t The specific value of η<1 / λ is the predefined learning step size. and

[0074]

[0075] in here represents an indicator function. When condition h is met, When it is not established, Bring in The iterative update formula for each single-core multi-label model is obtained:

[0076]

[0077] For all p=1,2,…,P, apply this formula in turn to get W 1,t+1 ,W 2,t+1 ,…,W P,t+1 , thus obtaining

[0078] The second sub-problem is solved incrementally using the expert prediction algorithm Hedge algorithm. is the unstandardized combination coefficient vector, μ t =[μ 1,t ,μ 2,t ,…,μ P,t ] T is the standardized combination coefficient vector, that is, here Representation vector 1-norm of . Initially, When round t ≥ 1, first calculate each single-core multi-label classification model W p,t Loss incurred t (W p,t ), and then μ is obtained according to the following formula t+1 :

[0079]

[0080] where β>0 is the step size of this update, is a vector 1-norm of . Note that the Hedge algorithm requires the loss function to be bounded to obtain sublinear regret. By updating formula (1), the method of the present invention can ensure that all single-core multi-label models W p,t lies in a bounded convex set, which further ensures that the loss f t (W p,t ) is bounded, paving the way for the use of the Hedge algorithm. Finally, considering that learning multiple times on a sample helps improve performance, in each round, the update formulas (1) and (2) are allowed to be executed L times.

[0081] In summary, the multi-label kernel classifier of the method of the present invention The online update subroutine steps are as follows:

[0082] (1) In round t, the complete image data information (x t ,Y t );

[0083] (2) For p = 1, 2, ..., P, use formula (1) in sequence to calculate W 1,t+1 ,W 2,t+1 ,…,W P,t+1 , thus obtaining

[0084] (3) For p = 1, 2, ..., P, calculate f in turn t (W p,t ), calculated using formula (2) Thus we get

[0085] (4) Execute steps (2) and (3) for L times, so that the sample (x t ,Y t ) is learned L times and improved and

[0086] (5) Using formula (3) to calculate μ t+1 , and finally get the next round of multi-label kernel classifier

[0087] Thus, the online adaptive kernel selection method for image multi-label data stream classification proposed by the present invention is obtained, which is named OKS-OMC (Online Kernel Selection for Online Multi-label Classification).

[0088] Before running the method of the present invention, it is necessary to determine the hyperparameters of the algorithm in advance, including the regularization coefficient λ, the learning step length η, and the learning step length β. Usually, λ should be selected to be a small value, such as λ = 10 -6 Or λ=10 -8 , and the more samples there are in the data stream, the smaller the value of λ should be. The value of step size η varies from dataset to dataset and needs to be carefully selected. The appropriate range of step size β is [0,0.5].

[0089] Two typical embodiments of the present invention are given below, which are described using different kernel function types.

[0090] Example 1

[0091] When the predefined kernel set {K 1 ,K 2 ,...,K P} is an isolated core, the implementation steps of the method are as follows:

[0092] 1. Set the hyperparameters required for this method to run: λ = 10 -8 , η = 0.5, β = 0.25;

[0093] 2. Set the hyperparameters of the isolation kernel: reserve the first 20% of the data from the multi-label data stream to create the isolation kernel, record N 0 is the number of reserved samples. For any p∈{1,2,…,P}, when calculating the isolation kernel K p When the feature map is 0 Randomly sample ψ from the reserved samples p points, sample 2*d times, and save these 2*d sample subsets as S p,1 ,…S p,k ,…S p,2*d ; The method of the present invention uses P = Q-4 isolation cores as input, where For P cores, ψ 1 ,ψ 2 ,···,ψ P The values ​​are 2 5 ,2 6 ,···,2 Q ;

[0094] 3. Initialization operation: command

[0095] 4. For t=1,2,…,T, proceed as follows:

[0096] 4.1. Receive a feature vector

[0097] 4.2. Call the existing isolation kernel feature mapping program to calculate the mapped feature vector Φ 1 (x t ),…Φ p (x t ),…Φ P (x t ), as follows:

[0098] 4.2.1. For p = 1, 2, ..., P, and k = 1, 2, ..., 2*d, calculate x t With the sample subset S p,k The point closest to the European distance is x t Encoded as a ψ p dimensional one-hot vector This vector takes the value of 1 only at the position corresponding to the closest point, and takes the value of 0 at all other positions, so we can get

[0099] 4.3. Using the latest multi-label kernel classifier Predict x t The relevant label set of

[0100] 4.4. Receive x t The real relevant label set Y t ;

[0101] 4.5. Perform online update on the multi-label kernel classifier as follows:

[0102] 4.5.1. For p = 1, 2, ..., P, and i = 1, 2, ..., L + 1, calculate for

[0103]

[0104] in, Thus updating the p-th single-core multi-label classification model W p,t The i-th predictor in :

[0105]

[0106] 4.5.2. For p = 1, 2, ..., P, calculate the p-th single-core multi-label classification model W in turn p,t The losses suffered And use this loss to update the unstandardized combination coefficients:

[0107] 4.5.3. Execute steps 4.5.1. and 4.5.2. for L times, that is, the sample (x t ,Y t ) is learned L times;

[0108] 4.5.4. Update the standardized combination coefficients:

[0109] 4.6. Wait for the multi-label data of the t+1th round and return to step 4.1. If no new data is input, exit the loop.

[0110] The Mediamill dataset is used to verify the effectiveness of the proposed method when using isolation kernels. This dataset is a multimedia dataset for general video indexing, which contains 85 hours of international TV news data. There are a total of L = 101 labels, and each video instance is represented as a 120-dimensional feature vector consisting of numerical features. There are a total of 43,907 video instances. 20% of the data from this dataset is reserved for creating isolation kernels, so N 0=8781, the number of isolation cores used is P=8, and the number of points sampled each time by the 8 isolation cores is 2 5 ,2 6 ,···,2 12 , a total of 2*120=240 samples were collected. The hyperparameters for the method were set to: λ=10 -8 , η=0.5, β=0.25.

[0111] Example 2

[0112] When the predefined kernel set {K 1 ,K 2 ,...,K P} is a Gaussian kernel, where the Gaussian kernel is represented by By varying the kernel width δ 2 Different Gaussian kernels can be obtained by changing the value of . In this case, the implementation steps of this method are as follows:

[0113] 1. Set the hyperparameters required for this method to run: λ = 10 -6 , η = 0.5, β = 0.25;

[0114] 2. Set the Gaussian kernel width δ 2 The value set is {2 -6 ,2 -5 ,…,2 6}, and 13 Gaussian kernels are obtained as the predefined kernel set of this method;

[0115] 3. Initialization: command for and initialization The support vector set is an empty set;

[0116] 4. For t=1,2,…,T, proceed as follows:

[0117] 4.1. Receive a feature vector

[0118] 4.2. Using the latest multi-label kernel classifier Predict x t The relevant label set of , the prediction formula is Note that for and When t = 1, directly When t>1, calculate Need to use the kernel trick:

[0119]

[0120] here express The set of support vectors for is an element in the set, representing a support vector and its corresponding coefficient;

[0121] 4.3. Receive x t The real relevant label set Y t ;

[0122] 4.4. Update the multi-label kernel classifier online as follows:

[0123] 4.4.1. For p = 1, 2, ..., P, and i = 1, 2, ..., L + 1, calculate for

[0124]

[0125] in, Thus updating the p-th single-core multi-label classification model W p,t The i-th predictor in The support vector set

[0126] (a) Multiply the coefficients corresponding to all support vectors in by (1-λη);

[0127] (b) If Then x t and its coefficient join in This is equivalent to Thus we get The support vector set

[0128] The above two steps (a) and (b) are equivalent to the following update formula:

[0129]

[0130] 4.4.2. For p = 1, 2, ..., P, calculate the p-th single-core multi-label classification model W in turn p,t The losses suffered And use this loss to update the unstandardized combination coefficients:

[0131] 4.4.3. Execute steps 4.5.1. and 4.5.2. for L times, that is, the sample (x t ,Y t ) is learned L times;

[0132] 4.4.4. Update the standardized combination coefficients:

[0133] 4.5. Wait for the multi-label data of the t+1th round to be obtained and return to step 4.1. If no new data is input, exit the loop.

[0134] Note that when using a Gaussian kernel, each component predictor in the multi-label kernel classifier It cannot be directly represented by a vector, but only by a support vector set To express indirectly.

[0135] The Scene dataset is an image dataset containing 2407 images, which are labeled into up to 6 categories: beach, sunset, autumn leaves, field, mountain and city. Each image is described by 294 visual numerical features, thus corresponding to a 294-dimensional feature vector. On this dataset, 13 Gaussian kernels are used, with Gaussian kernel width δ 2 The value set is {2 -6 ,2 -5 ,…,2 6}. The hyperparameters for this method are set to: λ = 10 -6 , η = 0.5, β = 0.25. 5% of the data in the dataset is reserved to find the optimal hyperparameters for the comparison algorithm.

[0136] In order to verify the effectiveness and superiority of the method proposed in this paper, two online multi-label learning algorithms based on extreme learning machines, OSML-ELM and ELM-OMLL, an online single-core multi-label method FALT, and an online single-core multi-label method PAML-I were selected as comparison methods.

[0137] When running the comparison algorithm, use the reserved N 0 We use 7 evaluation indicators to evaluate the performance of multi-label prediction: precision (Psn), recall (Rcal), F1-measure, MacroF1, MicroF1, Hamming loss (Hl), and ranking loss (Rl). The specific calculation method of each indicator is as follows:

[0138]

[0139] Where N is the number of samples used for evaluation, the operator Δ is used to measure the symmetric difference between two sets, and h(x i ,j) indicates that the prediction model is instance x i The real-valued score assigned to label j, tp j 、fp j 、fnj They represent the number of true positives, false positives, and false negatives for label j, respectively. For the first five metrics, higher values ​​indicate better performance. For the last two metrics, lower values ​​indicate better performance.

[0140] The values ​​of the above indicators are all calculated online. Specifically, in order to simulate the data flow environment, the data in the entire data set are randomly shuffled and it is assumed that the data arrives in sequence. t When it arrives, the online learner uses its current model to predict x t All relevant labels of , and update each evaluation index according to the prediction result; when x t The real relevant label set Y t After being revealed, the learner uses (x t ,Y t ) to update its prediction model so that it can make better predictions in the next round. Repeat this process until the data stream is exhausted, and record the final values ​​of the seven performance indicators when the data stream is exhausted. In order to ensure the reliability of the experimental results, all methods are run 10 times, and the random order of samples in the data set is different each time. The average values ​​and standard deviations of the evaluation indicators obtained after these 10 runs are recorded in Tables 1 and 2.

[0141] Table 1 Online performance indicators [%] and standard deviations (in brackets) of five algorithms on the Mediamill dataset. The best results on each indicator are shown in bold.

[0142]

[0143] Table 2 Online performance indicators [%] and standard deviations (in brackets) of the five algorithms on the Scene dataset. The best results on each indicator are shown in bold.

[0144]

[0145] In summary, it can be seen from Table 1 that on the Mediamill dataset, OKS-OMC achieved the best results in F1-measure, MacroF1 and MicroF1 indicators, and only slightly differed from the best results in Hamming loss (Hl) and ranking loss (Rl). It can be seen from Table 2 that on the Scene dataset, OKS-OMC has the best values ​​in all 7 indicators. On these two datasets, when the values ​​of the methods on these 7 indicators are considered comprehensively, it is found that the overall online performance of OKS-OMC is the best. The performance advantage of OKS-OMC is due to its ability to dynamically track the optimal kernel during the online learning process to find the global optimal kernel, while the single-kernel methods FALT and PAML-I have a higher risk of using a suboptimal kernel due to the limited data used for kernel selection, thus affecting their online performance.

[0146] The above embodiments are only used to help understand the method and core idea of ​​the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. An online adaptive kernel selection method for image multi-label data stream classification, characterized in that: The following steps are involved: Step 1) In round t, the program receives a feature vector of an image to be recognized Step 2) Use the feature mapping subroutine to transform x t Map them to P feature spaces respectively and get the mapped feature vectors Among them, for any p∈{1,2,…,P}, Φ p (·) is the kernel function K p The implied nonlinear feature map, d p is the dimension of the feature space after the pth mapping; Step 3) Use the latest multi-label kernel classifier To predict x in real time t The set of related tags, where And for any p∈{1,2,…,P}, μ t =[μ 1,t ,…,μ P,t ] T ∈Δ, and The multi-label kernel classifier consists of P single-kernel multi-label classifiers W p,t And the corresponding combination coefficient μ p,t Composed, and each single-core multi-label classifier W p,t It contains L label score predictors and 1 label threshold predictor; predict x t The relevant tag sets are: Step 4) After the prediction is completed, x t The real relevant label set Y t is revealed, and then enters the online update subroutine of the multi-label kernel classifier, which obtains the iterative update formula of the multi-label kernel classifier by solving a joint optimization problem involving P single-core multi-label classifiers and their combination coefficients, thereby obtaining the multi-label kernel classifier for the next round Step 5) Return to step 1) and wait for the image feature vector of the t+1th round; if no new feature vector is passed in, exit the program.

2. The online adaptive kernel selection method for image multi-label data stream classification according to claim 1, characterized in that: Step 1) requires a given image multi-label data stream {(x1, Y1), (x2, Y2), …, (x T ,Y T )},in It is a d-dimensional real-valued vector obtained after feature extraction of an image arriving at time t. is with x t A collection of related tags. represents the set of all possible labels, L is the total number of labels; a predefined kernel function set {K1, K2, ..., K P } as input, where P represents the number of predefined kernel functions.

3. The online adaptive kernel selection method for image multi-label data stream classification according to claim 2, characterized in that: For any eigenvector And any p∈{1,2,…,P}, the kernel function K p Satisfy the condition K p (x i ,x j )≤1.

4. An online adaptive kernel selection method for image multi-label data stream classification according to any one of claims 1 to 3, characterized in that: Multi-label kernel classifier in step 3) The iterative update formula is derived as follows: First, define the following loss function to evaluate the multi-label kernel classifier Multi-label classification performance on samples (x, Y): Among them, |Y| represents the size of set Y, represents a set of labels that are unrelated to x. According to the regularization framework in statistical learning, we find the one that minimizes the regularization loss. That is, solve the following problem: in, is a regularization term, λ>0 is a predefined regularization coefficient, ||W p || F is the matrix W p The F-norm of is a vector 2-norm of ; Then, the following loss function is defined to evaluate the single-core multi-label model W p In the sample (Φ p Classification performance on (x), Y): Since, μ=[μ1,…,μ P ]∈Δ is a probability distribution, and the function g(z)=max{0,1-z} is a convex function. Using Jensen’s inequality, we can get In the original optimization goal, Replace it with its upper bound and rearrange it to get the following optimization problem: in, represents the loss of the regularized single-core model; Finally, we solve the above two subproblems. The first subproblem involves finding an optimal single-core multi-label model in each mapped feature space, whose goal is to minimize the regularized single-core loss on all samples; the second subproblem aims to determine the optimal combination coefficient to minimize the weighted average of the losses generated by these optimal single-core multi-label models.

5. The online adaptive kernel selection method for image multi-label data stream classification according to claim 4, characterized in that: Solve the first subproblem incrementally using online gradient descent: Initially, set That is, for any p∈{1,2,…,P}, any i∈{1,2,…,L+1}, set When round t ≥ 1, we can get Among them, f t (W p ) is f(W p ;(Φ p (x t ),Y t )) is an abbreviation of It is B t (W p )about The subgradient of W p,t The specific value at which η<1 / λ is the predefined learning step size; the calculation is and in here represents an indicator function. When condition h is met, When it is not established, Bring in The iterative update formula for each single-core multi-label model is obtained: For all p=1,2,…,P, apply this formula in turn to get W 1,t+1 ,W 2,t+1 ,…,W P,t+1 , thus obtaining 6. The online adaptive kernel selection method for image multi-label data stream classification according to claim 5, characterized in that: The second sub-problem is solved incrementally using the expert prediction algorithm Hedge algorithm: is the unstandardized combination coefficient vector, μ t =[μ 1,t ,μ 2,t ,…,μ P,t ] T is the standardized combination coefficient vector, that is, here Representation vector 1-norm of; initially, When round t ≥ 1, first calculate each single-core multi-label classification model W p,t Loss incurred t (W p,t ), and then μ is obtained according to the following formula t+1 : where β>0 is the step size of this update, is a vector ; In each round, update formulas (1) and (2) are allowed to be executed L times.

7. The online adaptive kernel selection method for image multi-label data stream classification according to claim 6, characterized in that: Multi-label kernel classifier in step 3) The online update subroutine steps are as follows: 3-1) In round t, the complete image data information (x t ,Y t ); 3-2) For p = 1, 2, ..., P, use formula (1) to calculate W 1,t+1 ,W 2,t+1 ,…,W P,t+1 , thus obtaining 3-3) For p = 1, 2, ..., P, calculate f in turn t (W p,t ), calculated using formula (2) Thus we get 3-4) Execute steps 3-2) and 3-3) for L times, so that the sample (x t ,Y t ) is learned L times and improved and 3-5) Use formula (3) to calculate μ t+1 , and finally get the next round of multi-label kernel classifier 8. The online adaptive kernel selection method for image multi-label data stream classification according to claim 7, characterized in that: Before running this method, you need to determine the algorithm's hyperparameters in advance, including the regularization coefficient λ, the learning step size η, and the learning step size β.

Images