Multi-task collaborative classification method based on subspace segmentation
Through the subspace segmentation mechanism and alternating iterative optimization, the negative transfer problem in multi-task learning is solved, efficient joint training of image classification tasks is achieved, and the classification accuracy and generalization performance are improved.
Patent Information
- Application Number
- CN202510876058.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-10-17
AI Technical Summary
Existing multi-task learning methods are prone to negative transfer problems when dealing with unrelated tasks, which affects learning performance. In particular, traditional clustering methods are not applicable and have high computational costs on high-dimensional datasets.
A multi-task collaborative classification method based on subspace segmentation is adopted. Principal component analysis or downsampling is used for preprocessing, and the image classification task data is placed in the same dimension. The subspace segmentation mechanism is used to divide related tasks into the same group for joint training. The optimal projection matrix and offset vector are obtained through low-rank structure constraints and alternating iterative optimization.
Effectively handle high-dimensional projection parameters, improve knowledge transfer efficiency, avoid negative transfer, and achieve better classification performance and more efficient learning process.
Smart Images
Figure CN120808005A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of image recognition and classification and machine learning, and particularly relates to a multi-task collaborative classification method based on subspace segmentation, which aims to solve the negative transfer problem caused by unrelated tasks in the multi-task collaborative learning process, thereby improving the classification accuracy and generalization performance of each learning task. BACKGROUND
[0002] Multi-task learning method (MTL) as a variant of inductive transfer learning has been more and more widely concerned in recent years, which effectively solves the problem of learning performance decline of algorithm model caused by task diversification and data missing. Multi-task learning method assumes that when related tasks with different purposes are jointly learned, the tasks with correlation will benefit from the knowledge transfer of other tasks under the shared structural information and representation, thereby improving the learning performance. Due to this feature, multi-task learning method has been widely used in many scientific fields and has become a new topic in the field of machine learning and artificial intelligence, such as computer vision and medical image.
[0003] For multi-task learning method, it aims to search and measure the correlation between tasks to build shared information or representation, so that most tasks can improve the generalization performance in the joint learning process. Traditional multi-task learning methods can be divided into three categories under different working mechanisms, including multi-task learning methods based on task relationship, decomposition and feature subspace learning. The multi-task learning method based on task relationship usually uses the similarity or task covariance relationship between tasks as a measurement index for correlation learning. The multi-task learning method based on decomposition is to decompose the training parameter matrix of all tasks into the sum of two or more matrices, and to represent and measure the correlation between learning tasks by regularizing the decomposition matrix constraint. For the above two methods, although good performance can be achieved in processing multi-task learning problems, since the correlation between tasks is usually given by model assumption or prior information, they cannot be effectively applied to real scenarios. Therefore, the feature subspace learning method is proposed to solve these problems.
[0004] The present application is a multi-task learning model based on a feature subspace learning method. For the feature subspace learning method, in order to learn and measure the correlation between tasks, there are mainly two definition methods. One way is to learn the optimal structural information between tasks by assuming that the joint learning tasks share an explicit common representation. Based on this, the literature [8] proposes a structured learning paradigm (ASO), which learns the explicit common representation by placing all the projection parameters of the learning tasks in the same parameter prediction space. In addition, the literature designs a convex optimization method to efficiently learn to capture this display of shared information representation structure. Another way of defining the correlation assumes that the projection parameters from the learning tasks share a common but implicit information structure. From this, some constraints, such as sparse constraints and low-rank constraints, can be designed to constrain the learning of the projection parameter space, so as to obtain the implicit shared structure between tasks.
[0005] However, for most current multi-task learning algorithms, they all assume that all learning tasks are related and easy to be jointly trained. However, in real scenarios, this assumption is not always true. When there are unrelated tasks in the joint training learning tasks, it will cause the negative transfer problem. This means that when unrelated or noisy tasks participate in the joint training process, the learning performance of some tasks will not only not be improved, but will be greatly reduced. For example, for different types of data sets: face image data set and meteorological data of a certain region, there is no correlation between them. If the classification or regression tasks on these two types of data sets are forced into the joint learning process, the knowledge transfer between the two types of tasks is of no help to each other, and the strong joint learning constraint will damage the learning structure of each task itself.
[0006] Although this problem exists in multi-task learning methods, few relevant literature focuses on this problem. Jacob et al. designed a clustered multi-task learning method (Clustered-MTL method, CMTL), which clusters and divides all learning tasks into several groups, so that the learning tasks in the same group are highly correlated with each other. Then, Zhou et al. introduced the idea of k-means clustering into multi-task learning and proposed a convex relaxation variant model of the CMTL method. However, in real situations, the projection parameters of general learning tasks are usually high-dimensional, such as regression and classification tasks. Because high-dimensional data has the characteristics of sparsity and feature redundancy, traditional clustering methods such as k-means are not suitable for processing this type of data. In addition, for high-dimensional data sets, traditional clustering methods not only cannot effectively define a distance function suitable for this type of data, but also have very high consumption in terms of time and computational cost. SUMMARY
[0007] (I) Technical problems to be solved
[0008] The technical problem to be solved by the present application is how to provide a multi-task collaborative classification method based on subspace partitioning.
[0009] (II) Technical solutions
[0010] To solve the above technical problems, the present application provides a multi-task collaborative classification method based on subspace partitioning, which comprises the following steps:
[0011] Step 1: Under the given different types of image classification tasks, perform dimension reduction preprocessing operation on the data set of each image classification task respectively, and place the data sample dimensions of different image classification tasks on the same dimension; further, randomly select 40% of the samples from the preprocessed initial data set of each image classification task to construct a training data set, which is used to train the classification performance of the proposed algorithm; the remaining samples are used as test data set to verify the performance of the trained algorithm;
[0012] Step 2: Introduce all image classification tasks and corresponding training data sets into the multi-task collaborative learning framework based on subspace partitioning, divide related tasks and class attributes in all image classification tasks into the same group through subspace partitioning mechanism, and perform joint training within the group under low-rank structure constraint; after obtaining the optimal shared information within the group, further update the projection matrix and offset vector of different tasks, and continuously optimize the objective function through alternating iteration process to learn the optimal task projection matrix, i.e. the optimal classifier;
[0013] Finally, apply the optimal classification model learned by each task to the corresponding test data set to obtain the predicted classification result, and compare it with the real label.
[0014] In step 1, the proposed dimension reduction preprocessing operation uses principal component analysis method or down-sampling method.
[0015] In step 2, the step 2 comprises:
[0016] Step 2A: Construct a multi-task collaborative learning framework based on subspace partitioning, whose model is as follows:
[0017]
[0018] Here, and represent the training sample set and the corresponding sample label set of the tth image classification task; d represents the data dimension, n t represents the number of training samples in the image classification task t, c t represents the sample classification class in task t; for denote the projection matrix to be learned for each classification task, and the overall joint projection matrix is W = [W1, W2,..., WT]T. T ]∈R d×c , and c = ∑ t c t ; T is the total number of all classification tasks;
[0019] The first term of the objective function in the multi-task co-learning framework model (1) represents the regression loss function of multi-class support vector machine (SVM), and the second term is the low-rank regularization term of the joint projection matrix of each group of tasks, which is realized by the Schatten p norm, and λ represents the penalty parameter; in addition, the summation constraint condition in the original optimization problem can ensure that each task can be divided into and only into one group;
[0020] For the t-th classification task, the mathematical expression of the regression loss function of multi-class support vector machine (SVM) can be written as:
[0021]
[0022] where b t and M t ≥ 0 represent the offset vector and the offset matrix, respectively, the elements in the column vector 1 t are all 1, the symbol represents the Hadamard product; here, the Frobenius norm is directly used instead of its square term to improve the robustness of the model to noisy data;
[0023] Step 2B: for the above multi-task co-learning framework model (1), the following algorithm process is used to solve it to learn the optimal joint projection matrix;
[0024] Step 2B1: according to the re-weighted optimization method, the multi-task co-learning framework model (1) can be equivalently transformed into the following optimization problem to solve it:
[0025]
[0026] For convenience of calculation and writing, here let and have For problem (3), where the weight v t and the intermediate variable D i are related to the current updated solution W and G i , respectively, and the calculation formulas are as follows:
[0027]
[0028] Step 2B2: randomly initialize the group indication matrix According to the total number of multi-classification tasks and the corresponding classification information, the penalty parameter λ and the number of groups k are set;
[0029] Step 2B3: independently pre-train each multi-classification task to initialize the projection matrix W t and the offset vector b t and the offset matrix M t ;
[0030] Step 2B4: calculate the intermediate variables and
[0031] Step 2B5: fix the joint projection matrix W, update the group indicator matrix
[0032]
[0033] wherein the elements in the optimal solution G i can be solved by the following equation:
[0034]
[0035] Here, the t-th diagonal element of the matrix G i is g ti , the t-th diagonal element of the matrix A i is a ti , and A i = W T D i W; therefore, for all group indicator matrices can be calculated by the corresponding given matrix through equation (6);
[0036] Step 2B6: update the joint projection matrix W = [W1, W2,..., W T ], update each column thereof respectively according to the following equation:
[0037]
[0038] Step 2B7: calculate and update the offset vector b t and the offset matrix M t of each task respectively according to the following equation:
[0039]
[0040] Step 2B8: calculate the objective function value after each update iteration according to the objective function in equation (3), and if the difference between the objective function values calculated after the previous two iterations is not greater than 10 -6If yes, stop iteration, and the final optimal joint projection matrix W is obtained; otherwise, return to step 3.4 to update and calculate again.
[0041] Step 2C: After joint training, for each classification task, the corresponding optimal projection matrix W is obtained t and the optimal offset vector b t ; and the continuous prediction result is obtained by the classification model on the test data set; for the prediction result of each sample, that is, the column vector Since it is continuous, the max() function is used here to discretize it, that is, by setting the element with the maximum value in the label vector to 1 and the remaining elements to 0, the final classification result is obtained.
[0042] Step 1: First, under different given image classification tasks, the data of each task is preprocessed, and principal component analysis technology or downsampling method is used to place the data samples in different tasks in the same dimension. Since the method proposed in the application is applied to a multi-classification problem, part of the samples in the data corresponding to the task are selected as the training set to learn the parameters of the classification model, and the remaining data is used as the test set to verify the classification performance of the model.
[0043] Step 2: All classification tasks and corresponding training data sets are introduced into the multi-task collaborative learning framework based on subspace partitioning proposed in the application, and through the subspace partitioning mechanism, related tasks and class attributes in all classification tasks are divided into the same group, and joint training is performed in the group under the low-rank structure constraint. After obtaining the optimal shared information in the group, the projection matrix and the offset vector of different tasks are further updated, and the optimal task projection matrix, that is, the optimal classifier, is learned by continuously optimizing the objective function through the alternating iteration process. Finally, the optimal classification model learned by each task is applied to the corresponding test data set to obtain the predicted classification result, which is compared with the real label.
[0044] (Three) beneficial effects
[0045] To solve the negative transfer problem in the multi-task learning process and overcome the shortcomings of the existing feature subspace learning method, the application proposes a multi-task collaborative learning method based on subspace partitioning to process image multi-classification problems. The method can not only adaptively judge the correlation between tasks for knowledge sharing, but also can perform multi-task reasoning in the joint training process. In addition, combined with the subspace learning mechanism, the proposed model can effectively process the high-dimensional projection parameters in the learning task and improve the knowledge transfer efficiency. Finally, the application proposes a simple and efficient optimization algorithm based on the reweighting method to solve the proposed model.
[0046] The main contribution of the present application is that, due to the introduction of the subspace segmentation mechanism, the proposed model can not only effectively handle the high-dimensional projection parameters between tasks in solving the problem of negative knowledge transfer, but also can simultaneously perform independent multi-task reasoning on the class group subspace. In addition, the present application designs an efficient optimization algorithm based on reweighting, so that the proposed model can converge more efficiently and obtain better classification results when dealing with large-scale multi-task learning. BRIEF DESCRIPTION OF DRAWINGS
[0047] Figure 1 FIG. 1 is a schematic diagram of a subspace segmentation-based multi-task collaborative learning framework for clustering and segmenting tasks during joint training of different classification tasks.
[0048] The training learning process of multiple classification tasks is included in the figure. The proposed multi-task multi-classification model first divides the learning tasks according to the correlation through the subspace clustering mechanism during the training process, and then performs joint training of the learning tasks within the group to realize positive knowledge transfer, so that different learning tasks can learn better classification models and avoid negative knowledge transfer.
[0049] Figure 2 The technical solution flowchart. DETAILED DESCRIPTION
[0050] In order to make the purpose, content, and advantages of the present application clearer, the specific embodiments of the present application are described in further detail below in combination with the drawings and examples.
[0051] To solve the above technical problems, the present application provides a multi-task collaborative classification method based on subspace segmentation, which comprises the following steps:
[0052] Step 1: Under the given different types of image classification tasks, perform dimensionality reduction preprocessing operation on the data set of each image classification task respectively, and place the data sample dimensions of different image classification tasks on the same dimension. Further, 40% of the samples are randomly selected from the preprocessed initial data set of each image classification task to construct a training data set, which is used to train the classification performance of the proposed algorithm. The remaining samples are used as test data set to verify the performance of the trained algorithm.
[0053] Step 2: Introduce all image classification tasks and corresponding training data sets into the multi-task collaborative learning framework based on subspace segmentation, divide related tasks and class attributes in all image classification tasks into the same group through the subspace segmentation mechanism, and perform joint training within the group under the low-rank structure constraint. After obtaining the optimal shared information within the group, further update the projection matrix and offset vector of different tasks, and continuously optimize the objective function through the alternating iteration process to learn the optimal task projection matrix, i.e. the optimal classifier.
[0054] Finally, the optimal classification model learned by each task is applied to the corresponding test dataset to obtain the predicted classification results, which are compared with the real labels.
[0055] In step 1, the dimensionality reduction preprocessing operation uses principal component analysis or downsampling.
[0056] In step 2, the method comprises:
[0057] Step 2A: Construct a multi-task collaborative learning framework based on subspace partitioning, whose model is as follows:
[0058]
[0059] Here, and represent the training sample set and the corresponding sample label set of the tth image classification task, respectively; d represents the data dimension, n t represents the number of training samples in the image classification task t, c t represents the sample classification category in task t; for represents the projection matrix to be learned by each classification task, and the overall joint projection matrix is W=[W1,W2,...,W T ]∈R d×c , and c=∑ t c t ; T is the total number of classification tasks;
[0060] For the multi-task collaborative learning framework model (1), the first term of the objective function represents the regression loss function of multi-class support vector machine (SVM), and the second term is the low-rank regularization term of the joint projection matrix of each group task, which is realized through the Schatten p norm, and λ represents the penalty parameter; In addition, the summation constraint condition in the original optimization problem can ensure that each task can be divided into and only into one group;
[0061] For the tth classification task, the mathematical expression of the regression loss function of multi-class support vector machine (SVM) can be written as:
[0062]
[0063] Where b t and M t ≥0 represent the offset vector and the offset matrix, respectively, and the elements in the column vector 1 t are all 1, and the symbol represents the Hadamard product; Here, the Frobenius norm is directly used instead of its square term to improve the robustness of the model to noisy data;
[0064] Step 2B: For the above multi-task collaborative learning framework model (1), the following algorithmic procedure is used to solve it to learn the optimal joint projection matrix;
[0065] Step 2B1: According to the re-weighted optimization method, the multi-task collaborative learning framework model (1) can be equivalently transformed into the following optimization problem to solve it:
[0066]
[0067] For convenience of calculation and writing, here let and have For the problem (3), where the weight v t and the intermediate variable D i are related to the current updated solution W and G i respectively, their calculation formulas are respectively:
[0068]
[0069] Step 2B2: Randomly initialize the group indicator matrix According to the total number of multi-classification tasks and the corresponding classification information, the penalty parameter λ and the number of groups k are set;
[0070] Step 2B3: Independently pre-train each multi-classification task to initialize the projection matrix W t and the offset vector b t and the offset matrix M t in the task;
[0071] Step 2B4: Calculate the intermediate variables and
[0072] Step 2B5: Fix the joint projection matrix W, update the group indicator matrix
[0073]
[0074] Where the elements in the optimal solution G i can be solved by the following equation:
[0075]
[0076] Here, the t-th diagonal element of the matrix G i is g ti , and the t-th diagonal element of the matrix A i is represented as a ti , and A i =WT D i W; therefore, for all group indicator matrices may be obtained by corresponding given matrix is calculated by formula (6);
[0077] Step 2B6: update the joint projection matrix W = [W1, W2,..., W T ], update each column thereof respectively according to the following formula:
[0078]
[0079] Step 2B7: calculate the offset vector b t and the offset matrix M t of each task respectively according to the following formula:
[0080]
[0081] Step 2B8: calculate the objective function value after each update iteration according to the objective function in formula (3), if the difference between the objective function values calculated after the previous two iterations is not greater than 10 -6 , stop iteration, and obtain the final optimal joint projection matrix W; otherwise, return to step 3.4 to perform update calculation again;
[0082] Step 2C: after joint training, for each classification task, the corresponding optimal projection matrix W t and the optimal offset vector b t may be obtained; through the classification model on the test data set, the continuous prediction result can be obtained; for the prediction result of each sample, that is, the column vector Since it is continuous, the max() function is used here to perform discretization operation, that is, by letting the element with the maximum value in the label vector be 1 and the remaining elements be 0, the final classification result can be obtained.
[0083] Embodiment 1
[0084] This embodiment includes:
[0085] Step 1: first, under the given different image classification tasks, the data thereof is preprocessed respectively, and principal component analysis technology or down-sampling method is used to place the data samples in different tasks on the same dimension. Since the method of the present application is applied to multi-classification problems, part of the samples in the data corresponding to the task are selected as the training set to learn the parameters of the classification model, and the remaining data is used as the test set to verify the classification performance of the model.
[0086] Step 2: All classification tasks and corresponding training data sets are introduced into the proposed multi-task collaborative learning framework based on subspace partitioning. Through the subspace partitioning mechanism, related tasks and class attributes in all classification tasks are divided into the same group, and joint training is performed within the group under the low-rank structure constraint. After obtaining the optimal shared information within the group, the projection matrix and offset vector of different tasks are further updated, and the objective function is continuously optimized through an alternating iteration process to learn the optimal task projection matrix, i.e., the optimal classifier. Finally, the optimal classification model learned by each task is applied to the corresponding test data set to obtain the predicted classification result, which is compared with the true label.
[0087] Embodiment 2
[0088] The present application proposes a multi-task collaborative learning model based on subspace partitioning, and the main implementation process is as follows:
[0089] 1. Multi-task collaborative learning model based on subspace partitioning
[0090] Given T multi-classification tasks, the t-th task contains: training sample set and corresponding sample label The traditional multi-task learning model learns to search for the joint projection matrix W ∈ R d×c with the optimal low-rank structure by minimizing the training regression loss of each task and the low-rank relationship of the task projection matrix. However, in order to deal with the problem of negative knowledge transfer, the present application proposes a new subspace partitioning strategy under the traditional multi-task learning model, which divides related tasks into the same group for separate joint training by introducing an indicator matrix Here, each indicator matrix G i is a diagonal matrix. For tasks within the same group, the present application uses a low-rank structure constraint method to learn the optimal shared structure of the group joint projection matrix WG i . Therefore, the mathematical expression formula of the proposed initial model can be written as:
[0091]
[0092] Here, represents the projection matrix needed to be learned by each classification task, and the overall joint projection matrix is W = [W1, W2, L, W T ] ∈ R d×c , and c = ∑ t c t . The first term of the objective function in problem (10) represents the regression loss function of the T multi-classification tasks, and the second term is the joint projection matrix WG iwhere λ denotes the penalty parameter. In addition, the summation constraint can ensure that each task can be assigned to and only to one group. In order to apply the proposed model to the multi-classification task, a self-weight learning strategy is designed as the loss function of the multi-classification task based on the multi-classification support vector machine (SVM) model proposed in the literature
[14] , and the formula can be written as:
[0093]
[0094] where b t and M t ≥0 denote the offset vector and the offset matrix, respectively, the elements in the column vector 1 t are all 1, and the symbol denotes the Hadamard product. By introducing the offset term , the tolerance of the model to samples between different classes can be increased, the class interval can be expanded, and thus the classification performance of the model can be improved.
[0095] For the regularization term shown in the second term of the problem (10), the low-rank structure relationship between tasks in each group is learned by using the matrix rank function, i.e., rank(WG i ). However, directly solving the rank function is an NP-hard problem. Therefore, as the optimal convex envelope of the rank function
[15] , the trace norm is proposed as the relaxation term of the rank function. However, introducing the regularization term based on the trace norm in the problem (10) can lead to the generation of trivial solutions. Therefore, the Schatten p norm, which is a more general matrix norm, is introduced, and the mathematical expression is as follows:
[0096]
[0097] It can be seen that when p = 1, the above formula is equivalent to the trace norm, and when the parameter p approaches zero infinitely, the above formula is equivalent to the rank function. Therefore, by introducing the Schatten p norm, the proposed model can become a general framework to better handle the multi-task multi-classification problem. In combination with the Schatten p norm definition and the initial model (10) proposed above, the final optimization model framework can be written in the following form:
[0098]
[0099] For the above problem, since its conditional constraint is discrete, directly solving it is also a very difficult task. Therefore, an efficient and easy-to-implement algorithm is designed to solve the optimization model proposed above.
[0100] 2. Solution algorithm design
[0101] For the optimization problem (4), the present disclosure proposes an alternating iterative algorithm based on reweighting to solve it. According to the derivation rule and the definition of the matrix Schatten p-norm, under the given classification loss function such as equation (11), the optimization problem (13) can be transformed into the following optimization problem by using the reweighting method to solve it:
[0102]
[0103] where the weight v t is related to the current updated solution W and G i , and the intermediate variable D i , and the calculation formula is as follows:
[0104]
[0105] Here, let and have For the problem (14), the alternating optimization algorithm can be used to solve the variables W and and update the intermediate variables and
[0106] In the case of fixing the joint projection matrix W, updating the group indicator matrix , the objective function and the constraint condition of the optimization problem (14) can be simplified as:
[0107]
[0108] Based on the commutativity of the matrix trace and the indicator matrix Therefore, the above problem can be equivalently transformed as:
[0109]
[0110] Here, A i = W T D i W. Since are all diagonal elements of the cxc matrix, and the elements are either 0 or 1. Therefore, the problem (17) can be transformed into the following optimization problem based on scalar:
[0111]
[0112] where the t-th diagonal element of the matrix G i is g ti , and a ti represents the t-th diagonal element of the matrix A i . Then the elements in the optimal solution G i can be solved by the following equation:
[0113]
[0114] Therefore, for all group indicator matrices whose element values can be given by corresponding given matrices are calculated by formula (19).
[0115] In the fixed group indicator matrices , the projection matrix W = [W1, W2,..., W T ] and the offset vector b t and the offset matrix M t corresponding to each task are updated.
[0116]
[0117] For the first term of problem (20), it can be independently decomposed into the following vector form:
[0118]
[0119] Here, the vectors and are the jth column of the projection matrix W t and the matrix H t , respectively. When the projection vector belongs to the ith group, then combined with equation (21), problem (20) can be independently decomposed into a convex optimization problem about the projection vector , as follows:
[0120]
[0121] Since the above problem is a convex optimization problem about , the optimal solution calculation formula can be obtained by derivation as
[0122]
[0123] By updating each column vector , the final optimal joint projection matrix W can be obtained.
[0124] For the update of the offset vector b t and the offset matrix M t in each classification task, it can be seen that it is only related to the first term in problem (20), and the update of these two variables by each task is independent, so problem (20) can be simplified as:
[0125]
[0126] It can be seen that problem (24) is about the variable bt and M t are convex optimization problems, so their optimal solutions can be obtained by the following calculation formula:
[0127]
[0128] Here, the function (M t ) + represents the non-negative operation on the elements in the matrix. By the calculation formula (25) and (26), the offset vector b t and the offset matrix M t of each classification task can be obtained.
[0129] In summary, the above solving process, the present application designs the above algorithm to alternately iteratively solve the optimization problem (13) until convergence, so that the joint projection matrix with optimal low-rank structure between related tasks in the group can be obtained, so that each task can learn the optimal projection matrix, and the influence of the irrelevant task knowledge negative transfer problem is avoided, and the generalization performance of all learning tasks is further robustly improved. After obtaining the optimal projection matrix W t of each classification task, the classification operation can be directly performed on the test data set of the corresponding task to obtain the predicted classification result.
[0130] The above only describes the preferred embodiments of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, and these improvements and modifications should be considered as the protection scope of the present application.
Claims
1. A multi-task collaborative classification method based on subspace segmentation, characterized in that: The method comprises the following steps: Step 1: Given different types of image classification tasks, perform dimensionality reduction preprocessing on the datasets of each image classification task, placing the data samples of different image classification tasks on the same dimension. Furthermore, 40% of the samples are randomly selected from the preprocessed initial dataset of each image classification task to construct a training dataset to train the classification performance of the proposed algorithm. The remaining samples are used as a test dataset to verify the performance of the trained algorithm. Step 2: All image classification tasks and corresponding training datasets are introduced into a multi-task collaborative learning framework based on subspace segmentation. Through the subspace segmentation mechanism, all related tasks and class attributes in the image classification task are divided into the same group, and joint training within the group is performed under the constraint of low-rank structure. After obtaining the optimal shared information within the group, the projection matrix and offset vector of different tasks are further updated, and the objective function is continuously optimized through an alternating iterative process to learn the optimal task projection matrix, that is, the optimal classifier. Finally, the optimal classification model learned for each task is applied to the corresponding test dataset to obtain the predicted classification results and compared with the true labels.
2. The multi-task collaborative classification method based on subspace segmentation according to claim 1, characterized in that: In step 1, the proposed dimensionality reduction preprocessing operation uses a principal component analysis method or a downsampling method.
3. The multi-task collaborative classification method based on subspace segmentation according to claim 1, characterized in that: The step 2 includes: Step 2A: Construct a multi-task collaborative learning framework based on subspace segmentation. The model is as follows: here, and They represent the training sample set and the corresponding sample label set of the t-th image classification task respectively; d represents the data dimension, n t represents the number of training samples in the image classification task t, c t Represents the sample classification category in task t; for Represents the projection matrix to be learned for each classification task, and the overall joint projection matrix is W = [W1, W2, ..., W T ]∈R d×c , and c=∑ t c t ; T is the total number of all classification tasks; For the multi-task collaborative learning framework model (1), the first term of the objective function represents the regression loss function of the multi-classification support vector machine, and the second term is the low-rank regularization term of the joint projection matrix of each task in each group, which is implemented by the Schattenp norm, and λ represents the penalty parameter; In addition, the summation constraint in the original optimization problem can ensure that each task can be divided into and only into one group; For the t-th classification task, the mathematical expression of the regression loss function of the multi-classification support vector machine can be written as: Among them, b t and M t ≥0 represents the offset vector and offset matrix, column vector 1 t The elements in are all 1, and the symbol Represents the Hadamard product; here, the Frobenius norm is used directly instead of its square term to improve the robustness of the model to noisy data; Step 2B: For the multi-task collaborative learning framework model (1), solve it according to the following algorithm process to learn the optimal joint projection matrix; Step 2B1: According to the reweighted optimization method, the multi-task collaborative learning framework model (1) can be equivalently transformed into the following optimization problem for solution: For the convenience of calculation and writing, And there is For problem (3), where the weight v t and the intermediate variable D i and the current updated solutions W and G respectively i The calculation formulas are: Step 2B2: Randomly initialize the group indicator matrix Set the penalty parameter λ and the number of groups k according to the total number of multi-classification tasks and the corresponding classification information; Step 2B3: Pre-train each multi-classification task independently to adjust the projection matrix W in the task t and the offset vector b t and the offset matrix M t Initialize; Step 2B4: Calculate the intermediate variables according to formula (4) and Step 2B5: Fix the joint projection matrix W and update the group indicator matrix as follows Among them, the optimal solution G i The elements in can be solved by the following equations: Here, the matrix G i The tth diagonal element of is g ti , matrix A i The t-th diagonal element of ti , and A i =W T D i W; therefore, for all group indicator matrices The corresponding matrix can be given by Calculated by formula (6); Step 2B6: Update the joint projection matrix W = [W1, W2, ..., W T ], update each column as follows: Step 2B7: Calculate and update the offset vector b of each task according to the following formula: t and the offset matrix M t : Step 2B8: Calculate the objective function value after each update iteration according to the objective function in formula (3). If the difference between the objective function values calculated after two iterations is not greater than 10 -6 , then stop the iteration and get the final optimal joint projection matrix W; otherwise return to step 3.4 and perform the update calculation again; Step 2C: After joint training, for each classification task, the corresponding optimal projection matrix W can be obtained t And the optimal offset vector b t ; On the test data set, through the classification model To obtain continuous prediction results; for the prediction result of each sample, that is, the column vector Since it is continuous, the max() function is used here to discretize it, that is, by setting the element with the largest value in the label vector to 1 and the remaining elements to 0, the final classification result can be obtained.
4. The multi-task collaborative classification method based on subspace segmentation according to claim 3, characterized in that: In formula (1), and They represent the training sample set and the corresponding sample label set of the t-th image classification task respectively; d represents the data dimension, n t represents the number of training samples in the image classification task t, c t Represents the sample classification category in task t; for Represents the projection matrix to be learned for each classification task, and the overall joint projection matrix is W = [W1, W2, ..., W T ]∈R d×c , and c=∑ t c t ; T is the total number of all classification tasks.
5. The multi-task collaborative classification method based on subspace segmentation according to claim 3, characterized in that: In formula (2), b t and M t ≥0 represents the offset vector and offset matrix, column vector 1 t The elements in are all 1, and the symbol represents the Hadamard product.
6. The multi-task collaborative classification method based on subspace segmentation according to claim 3, characterized in that: In formula (2), the Frobenius norm is used directly instead of its square term to improve the robustness of the model to noisy data.
7. The multi-task collaborative classification method based on subspace segmentation according to claim 3, characterized in that: In formula (6), the matrix G i The tth diagonal element of is g ti , matrix A i The t-th diagonal element of ti , and A i =W T D i W.
8. The multi-task collaborative classification method based on subspace segmentation according to claim 3, characterized in that: The method belongs to the technical fields of image recognition and classification and machine learning.
9. The multi-task collaborative classification method based on subspace segmentation according to claim 3, characterized in that: The method is used to solve the negative transfer problem caused by irrelevant tasks in the collaborative learning process of multiple classification tasks, thereby improving the classification accuracy and generalization performance of each learning task.
10. The multi-task collaborative classification method based on subspace segmentation according to claim 3, characterized in that: The method can not only adaptively identify the correlation between tasks for knowledge sharing, but also perform multi-task reasoning during joint training.