A semi-supervised multi-label learning method based on SPUM data

Abstract

The present invention discloses a semi-supervised multi-label learning method based on SPUM data. Consider single positive and unlabeled multi-label data (SPUM) data in which only one label is clearly marked as positive among multiple labels of data and the rest of the labels are unlabeled, and obtain the global optimal solution through information transmission when the unlabeled data and such SPUM data are scattered in different data nodes. An instant label prediction loss function based on the constraint of the number of positive classes is adopted, so that the algorithm can learn a large amount of information by constraining the number of positive classes and alternately optimizing label prediction and model output in the scenario where only a single label of part of the data is marked as a positive example, thereby greatly reducing the cost of the labeling task. In a distributed network with limited resources such as communication bandwidth, an iterative strategy based on event triggering is adopted, so that the gradient information amount is allowed to be transmitted only after reaching a certain level, which greatly reduces the network burden while maintaining performance.

Description

Technical Field

[0001] The present invention relates to a semi-supervised multi-label learning method based on SPUM data, which is an algorithm that can learn a good classification model on unlabeled data and multi-label data with only a single positive label, and this algorithm can learn the globally optimal classification model through only a small amount of information transfer between nodes. Background Art

[0002] In actual multi-label learning, a data may have multiple labels that need to be predicted. For example, in the field of target detection, a picture usually has multiple targets, and in text sentiment analysis, a text will also have multiple emotions. The current advanced multi-label classification algorithms usually require each label of each data to be manually annotated with +1 or 0 to indicate whether the class exists or not, but this is usually a tedious task so that it is expensive to fully annotate the data. If multi-label classification information can be learned when only a single label of part of the data is annotated and the other part of the data is completely unannotated, then the tedious multi-label annotation task will become a much simpler task that only requires a single positive example annotation for a small amount of data, which undoubtedly greatly reduces the cost of manual annotation. At the same time, in modern data storage forms, data is usually stored in different data nodes, and considering data privacy issues and communication cost issues, it is not expected to transmit original data between data nodes. How to obtain the global optimal classification model by only transmitting a small amount of information between distributed nodes is an important goal of distributed learning algorithms. Summary of the invention

[0003] In order to overcome the deficiency that the prior art is difficult to handle data conditions where only a single label of part of the data is annotated in a distributed scenario, the purpose of the present invention is to provide a semi-supervised multi-label learning method based on SPUM data.

[0004] The scenario discussed in this invention is based on a A distributed network undirected graph with nodes distributed in different geographical locations ,in represents a node set, Represents the set of numbers of neighbor nodes of each node, Representation Node The edge set of its neighbor nodes. Each distributed node stores a multi-label data set, which is recorded as , where each data Both labels and record the label set as . The data set at each node In addition to some SPUM datasets with single positive labels, There is still a considerable proportion of datasets that are not labeled. In the SPUM data, The dimension label vector has only one label annotated to it, and we use the observation vector represents this incompletely annotated label vector, then SPUM data It can be described by the formula:

[0005] (1);

[0006] where ∅ is the unknown label (i.e. it could be 1 or 0). Definition is the model output, where For a single data, Representative Node For simplicity of algorithm representation, we will use represent Multi-label learning is common in the field of target detection, and is used to detect multiple objects that may appear in an image. Therefore, this algorithm uses ResNet50, which is commonly used in target detection algorithms, as the basic framework to extract features from images, and uses the calculation loss function at the feature parameter layer and optimizes the parameters through gradient backpropagation.

[0007] The problem to be solved by the present invention is that in the above-mentioned distributed scenario, the loss function is calculated by the unlabeled data and SPUM data of each node, and the gradient information of the node is obtained by reverse calculation, and then transmitted in the network through the distributed strategy, so that each node finally obtains the globally optimal model parameters. The distributed semi-supervised multi-label learning algorithm based on SPUM data includes the following steps:

[0008] 1) Obtain SPUM data of each node;

[0009] 2) Calculate the loss function of each node and propose a global loss function;

[0010] 3) Through the distributed gradient calculation method, the gradient of each node is transferred to the neighboring nodes and integrated;

[0011] 4) Based on the event trigger mechanism, reduce the communication volume between nodes to reduce the communication cost;

[0012] 5) Obtain the global optimal model parameters through multiple iterations.

[0013] As a preferred technical means: In step 2), use the positive class number constraint to set an expected value for the number of positive labels for each sample output by the algorithm prediction , so that it can use the information of the positive observed labels to calculate This constraint on the number of positive labels is called positive number regularization (PNR), denoted as , expressed as:

[0014] (2)

[0015] in It represents the statistical expectation of the number of positive labels output by the algorithm. First, consider the labeled data set , you can use Obtain positive information, and at the same time, use the positive class quantity constraint to obtain the positive class quantity constraint loss function of the labeled data part as , expressed as:

[0016] (3);

[0017] The algorithm output of a single node is stacked. , the observation vector stack is . Also consider the semi-supervised case, that is, for an unlabeled dataset where only unobserved labels can be used , use the "false negative" strategy to impose negative losses on all labels of unlabeled data In order to obtain a large amount of negative information and use label smoothing to reduce the noise generated by the "false negative" strategy, and at the same time use the positive number constraint to obtain the positive number constraint loss function of the unlabeled part, recorded as , and express it as:

[0018] (4);

[0019] in Set to , in order to comprehensively consider the labeled part and the unlabeled part, the positive class quantity constraint loss function of each node is obtained as follows:

[0020] (5);

[0021] Despite the above l The PNR loss function uses As a constraint on the number of positive classes, a certain implicit assumption about the unobserved label output is implemented, so that certain information can be obtained from it, but this implicit assumption is only for the expected number of output positive labels, making its constraint weak. For this purpose, an implicit assumption directly for the predicted value of the output label value is used, real-time label estimation (RLE). After obtaining the feature vector through the neural network, two sets of parameters are used and Calculate the algorithm output separately and a label prediction output Both are sent to the loss function for subsequent gradient backpropagation and optimization. On this basis, the label prediction vector of a single data is recorded as , the label prediction matrix of all data is recorded as With strong assumptions about the instant label prediction, we can use the fully supervised cross entropy loss function to obtain more information. Combining the strong assumption based on label prediction with the weak assumption based on the expected number of positive labels (5) above, and supplemented by the manifold constraint, we get the preliminary RLE loss function: ,Right now:

[0022] (6);

[0023] in is the gradient interruption function, which will prevent the gradient from being returned, making Only used as an immediate prediction label; ; The last manifold constraint is used to optimize the algorithm output The above preliminary RLE loss function The algorithm can be prompted to output based on the number of positive classes. To get closer to the predicted label .

[0024] So how to optimize label prediction Parameters We only need to predict the label in equation (6) With the algorithm output By exchanging the positions of ,Right now:

[0025] (11);

[0026] The last manifold constraint term here is different from equation (6) and is used to optimize the algorithm output , but to optimize the label prediction output .

[0027] Will and By combining them, we can get the final RLE loss function suitable for SPUM learning, and propose the global optimization problem on the network as:

[0028] (7);

[0029] This allows the output function to be trained simultaneously Parameters And label prediction Parameters , and finally select the parameters that perform best in the test set or As the algorithm output.

[0030] As a preferred technical means: In step 4), we will define an event and let it determine the local node

[0031] Parameter estimation matrix Will it be transmitted to neighboring nodes by the network? is the temporary parameter estimation matrix allowed to be transmitted last time, For Node The last time a transmission was allowed. Define an event as Compared to The rate of change of is the greater the rate of change, the greater the amount of information, and it is considered worthy of transmission when the rate of change is greater than the threshold δ, that is:

[0032] (10);

[0033] When the node is allowed to pass information to neighboring nodes. If it is triggered, then the trigger event time point Will be updated to , the corresponding temporary parameter estimation matrix allowed to be transmitted will also be replaced by And transmit it to neighboring nodes.

[0034] The beneficial effects of the present invention are as follows: the instant label prediction loss function based on the positive class number constraint is adopted, so that even in the scenario where only a single label of a part of the data is labeled as a positive example, the algorithm can learn a lot of information by constraining the number of positive classes and alternately optimizing label prediction and model output, which greatly reduces the cost of the labeling task. The scale of may be very large. If it is transmitted to neighboring nodes in each iteration, it will cause a large communication cost. In order to reduce the communication burden between neighboring nodes, we adopt an event-triggered iteration strategy, which allows the gradient information to be transmitted only after it reaches a certain level, greatly reducing the network burden while maintaining performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG1 is a flow chart of the method of the present invention.

[0036] FIG. 2 and FIG. 3 are respectively intuitive effect diagrams of the event triggering mechanism of the present invention.

[0037] FIG. 4 is a diagram of the present invention.Pascal Voc Performance comparison on the dataset compared to the fully supervised algorithm. DETAILED DESCRIPTION

[0038] The technical solution of the algorithm of the present invention is further described in detail below in conjunction with the accompanying drawings. Figure 1 As shown, the present invention comprises the following steps:

[0039] 1) Obtain the data of each node and obtain the SPUM data of each node;

[0040] 2) Calculate the loss function of each node, calculate the loss function of each node and propose a global optimization problem;

[0041] 3) Distributed optimization: Through the distributed gradient calculation method, the global optimization problem mentioned in step 2) is solved by transferring the gradient of each node to the neighboring nodes and fusing them;

[0042] 4) Event triggering: Calculate the change rate of the temporary parameter estimate to determine whether it is greater than the trigger threshold. If it is greater than the trigger threshold, the gradient transfer event is triggered, thereby reducing the transmission frequency between nodes in step 3) to reduce communication costs;

[0043] 5) Repeat the above steps multiple times to obtain the global optimal model parameters.

[0044] The following is a further description of some of the steps:

[0045] 1. Step 2) Loss function calculation

[0046] Use the positive class number constraint to set an expected value for the number of positive labels for each sample output by the algorithm prediction , so that it can use the information of the positive observed labels to calculate This constraint on the number of positive labels is called positive number regularization (PNR), denoted as , expressed as:

[0047] (2);

[0048] in It represents the statistical expectation of the number of positive labels output by the algorithm. First, consider the labeled data set , you can use Obtain positive information, and at the same time, use the positive class quantity constraint to obtain the positive class quantity constraint loss function of the labeled data part as , expressed as:

[0049] (3);

[0050] The algorithm output of a single node is stacked. , the observation vector stack is . Also consider the semi-supervised case, that is, for an unlabeled dataset where only unobserved labels can be used , use the "false negative" strategy to impose negative losses on all labels of unlabeled data In order to obtain a large amount of negative information and use label smoothing to reduce the noise generated by the "false negative" strategy, and at the same time use the positive number constraint to obtain the positive number constraint loss function of the unlabeled part, recorded as , and express it as:

[0051] (4);

[0052] in Set to , in order to comprehensively consider the labeled part and the unlabeled part, the positive class quantity constraint loss function of each node is obtained as follows:

[0053] (5);

[0054] Despite the above l The PNR loss function uses As a constraint on the number of positive classes, a certain implicit assumption about the unobserved label output is implemented, so that certain information can be obtained from it, but this implicit assumption is only for the expected number of output positive labels, making its constraint weak. For this purpose, an implicit assumption directly for the predicted value of the output label value is used, real-time label estimation (RLE). After obtaining the feature vector through the neural network, two sets of parameters are used and Calculate the algorithm output separately and a label prediction output Both are sent to the loss function for subsequent gradient backpropagation and optimization. On this basis, the label prediction vector of a single data is recorded as , the label prediction matrix of all data is recorded as With strong assumptions about the instant label prediction, we can use the fully supervised cross entropy loss function to obtain more information. Combining the strong assumption based on label prediction with the weak assumption based on the expected number of positive labels (5) above, and supplemented by the manifold constraint, we get the preliminary RLE loss function: ,Right now:

[0055] (6);

[0056] in is the gradient interruption function, which will prevent the gradient from being returned, making Only used as an immediate prediction label; ; The last manifold constraint is used to optimize the algorithm output The above preliminary RLE loss function The algorithm can be prompted to output based on the number of positive classes. To get closer to the predicted label .

[0057] So how to optimize label prediction Parameters We only need to predict the label in equation (6) With the algorithm output By exchanging the positions of ,Right now:

[0058] (11);

[0059] The last manifold constraint term here is different from equation (6) and is used to optimize the algorithm output , but to optimize the label prediction output .

[0060] Will and By combining them, we can get the final RLE loss function suitable for SPUM learning, and propose the global optimization problem on the network as:

[0061] (7);

[0062] This allows the output function to be trained simultaneously Parameters And label prediction Parameters , and finally select the parameters that perform best in the test set or As the algorithm output.

[0063] 2. Step 3) Distributed Optimization

[0064] After obtaining the global optimization problem as shown in equation (7), this section will consider how to find the global optimal solution with only a small amount of information transmission. Define the connection matrix is a digital distributed network, where if the node and nodes If there is no undirected connection between Taking the global optimization problem (7) as an example, each node has its own independent parameters and .use Represents the parameter vector of the local node. Based on the undirected graph network and connection matrix mentioned above , we can use distributed gradient decent (DGD) to fuse the temporary gradient estimation matrix between nodes, thereby solving the distributed optimization problem expressed in equation (7).

[0065] The specific approach is to perform the following iterative steps for each data node:

[0066] (8a), (8b);

[0067] in For the The local parameter vector at iteration is the loss function In the The iteration about The derivative of ; For the local node The temporary parameter estimation matrix at the iteration; is a learning rate hyperparameter that varies over time and satisfies the vanishing condition:

[0068] (9);

[0069] If the entire network is strongly connected, the information between each node can be communicated and transmitted between the networks, and finally a consistent optimum is achieved. In formula (8), formula (8a) represents the update process of the temporary parameter estimation matrix of the local node. After the local node is updated, its temporary parameter estimation matrix will be transmitted to the neighboring nodes. Each node then performs a weighted combination according to the temporary parameter estimation matrices transmitted by the neighboring nodes through formula (8b) to obtain the final local parameter vector While transmitting key information, it does not involve the mutual transmission of original data, effectively protecting the privacy of each node.

[0070] 3. Step 4) Event triggering mechanism

[0071] We will define an event that determines the parameter estimation matrix for the local node Will it be transmitted to neighboring nodes by the network? is the temporary parameter estimation matrix allowed to be transmitted last time, For Node The last time a transmission was allowed. Define an event as Compared to The rate of change of is the greater the rate of change, the greater the amount of information, and it is considered worthy of transmission when the rate of change is greater than the threshold δ, that is:

[0072] (10);

[0073] When the node is allowed to pass information to neighboring nodes. If it is triggered, then the trigger event time point Will be updated to , the corresponding temporary parameter estimation matrix allowed to be transmitted will also be replaced by And transmit it to neighboring nodes. Example

[0074] We use mean average precision (mAP) as the main indicator for evaluating algorithm performance. In addition, in order to explore the performance of the algorithm under different annotation rates, we define an indicator LTR to express the ratio of labeled data to total data, namely:

[0075] (12);

[0076] Each labeled data is a single positive unlabeled multi-label data.

[0077] We first use a dataset containing 5717 training images, 5823 test images, and 20 class labels. Pascal Voc The data set is used for experimental simulation. To demonstrate the effect of the event triggering strategy, the nodes at training rounds 1-100 are intercepted. Compared to The change rate of and the triggering of each node event, i.e., the event of (10). The change rate is shown in Figure 2, and Figure 3 is the When each node satisfies the triggering condition of the event, that is, formula (10), we can see from the figure that with the increase of the number of training rounds, the number of nodes that trigger events and need to transmit information will decrease, which greatly reduces the transmission burden of the entire network and confirms the superiority of the event triggering algorithm.

[0078] For this algorithm The performance of different LTRs with an interval of 0.1 is compared in the range. At the same time, as a comparison, our simulation still needs to be complete. L Labels The loss function is fully annotated Pascal Voc The performance test was conducted on the dataset, and it was only for the labeled data calculate The loss function is optimized. The result is shown in Figure 4, where the horizontal axis is the number of observed labels available in the entire training data set. LTR∗20∗ N , The number of positive observation labels available for the loss function is LTR∗ N As can be seen from the figure, compared with all the required labels For example, the loss function based on SPUM data proposed in this invention is Only a very small number of observed labels is required to achieve classification performance close to that of the proposed method.

[0079] The distributed semi-supervised multi-label learning method based on SPUM data shown above is a specific embodiment of the present invention, which has embodied the essential characteristics and significant progress of the present invention. According to actual use needs and under the guidance of the present invention, equivalent modifications can be made to its method, structure, etc., which are all within the protection scope of this scheme.

Claims

1. A semi-supervised multi-label learning method based on SPUM data, characterized by: In the distributed scenario, the loss function is calculated from the unlabeled data and SPUM data of each node, and the gradient information of the node is obtained by reverse calculation. After that, it is transmitted in the network through the distributed strategy, and finally each node obtains the global optimal model parameter θ * ; The distributed scenario is based on a distributed network undirected graph G = {J, B} with J nodes distributed in different geographical locations, where J = {1, 2, ..., J} represents a node set, and B = {B j }, j∈J represents the number set of neighbor nodes of each node, ε={(j,b)},j∈J,b∈B j Represents the edge set between node j and its neighbor nodes; each distributed node stores a multi-label data set, denoted as N j ={1,2,...,N j }, Each data x n ,n∈N j There are L labels and the label set is recorded as L = {1, 2, ..., L}; in the data set N of each node j In addition to some SPUM datasets with single positive labels, There is still a considerable proportion of datasets that are not labeled. In the SPUM data, the L-dimensional label vector has only one label annotated, using the observation vector z n represents this incompletely labeled label vector, then SPUM data (x n ,z n ),n∈N j Described by the formula: in is an unknown label, that is, it may be 1 or 0, and define f(θ j |x n ) is the model output, where x n is a single data, θ j Represents the model parameters of node j; for the simplicity of algorithm representation, f is used n represents f(θ j |x n ); Here are the steps: 1) Obtain the data of each node and obtain the SPUM data of each node; 2) Calculate the loss function of each node, calculate the loss function of each node and propose a global optimization problem; 3) Distributed optimization: Through the distributed gradient calculation method, the global optimization problem mentioned in step 2) is solved by transferring the gradient of each node to the neighboring nodes and fusing them; 4) Event triggering, calculating the change rate of the temporary parameter estimate to determine whether it is greater than the trigger threshold. If it is greater than the trigger threshold, the gradient transfer event is triggered, thereby reducing the transmission frequency between nodes in step 3) to reduce communication costs; 5) Repeat the above steps for multiple times to obtain the global optimal model parameters; In step 2), the loss function is designed by first adopting the positive class number constraint, setting an expected value K for the number of positive labels of each sample output by the algorithm prediction, and this constraint on the number of positive labels is called the positive class number constraint (positive numbers regularization, PNR), recorded as It is expressed as: in Represents the statistical expectation of the number of positive labels output by the algorithm. First, consider the labeled data set Use log(f ni ) to obtain positive information, and at the same time, the positive class quantity constraint is used to obtain the positive class quantity constraint loss function of the labeled data part, which is recorded as l PNR l , expressed as: The algorithm output of a single node is stacked. The observation vector stack is At the same time, consider the semi-supervised case, that is, for unlabeled datasets where only unobserved labels can be used Use the "false negative" strategy to add a negative loss log(1-f ni ) to obtain a large amount of negative information and use label smoothing to reduce the noise generated by the "false negative" strategy. At the same time, the positive number constraint is used to obtain the positive number constraint loss function of the unlabeled part, which is recorded as l PNR ul , and express it as: where ε is set to In order to comprehensively consider the labeled and unlabeled parts, the positive class quantity constraint loss function of each node is obtained as follows: Although the above lPNR loss function adopts As a constraint on the number of positive classes, a certain implicit assumption about the unobserved label output is implemented, so that certain information can be obtained from it, but this implicit assumption is only for the expected number of output positive labels, making its constraint weaker; for this purpose, an implicit assumption directly for the predicted value of the output label value is used, real-time label estimation (RLE); after obtaining the feature vector through the neural network, two sets of parameters θ are used j and Calculate the algorithm output f(θ j |x n ) and a label prediction output And send both to the loss function for subsequent gradient backpropagation and optimization; on this basis, the label prediction vector of a single data is recorded as g n , the label prediction matrix of all data is recorded as With strong hypothetical instant label prediction, we can use the fully supervised cross entropy loss function to obtain more information. We combine the strong hypothesis based on label prediction with the weak hypothesis based on the expected number of positive labels (Formula (5)), and use the manifold constraint to obtain the preliminary RLE loss function: Right now: where sg(·) is the gradient interruption function, which will prevent the gradient from being propagated back so that g n Only used as an immediate prediction label; The last manifold constraint is used to optimize the algorithm output f n ; The above preliminary RLE loss function The algorithm can be prompted to output f based on the positive number constraint n To get close to the predicted label g n ; while optimizing label prediction Parameters It is necessary to predict the label in formula (6) With the algorithm output By exchanging the positions of Then and By combining them, we can get the final RLE loss function suitable for SPUM learning, and propose the global optimization problem on the network as: In this way, the output function f(θ j |x n ) parameter θ j And label prediction Parameters Finally, the parameter θ that performs best in the test set is selected j or As the algorithm output.

2. The method according to claim 1, characterized in that: In step 4), after obtaining the global optimization problem as shown in formula (7), consider how to find the global optimal solution with only a small amount of information transmission, and define the connection matrix C = [C jb ] is a numerical distributed network, where if there is no undirected connection between node j and node b, then C jb =0; In the global optimization problem (7), each node has its own independent parameter θ j and use Represents the parameter vector of the local node; based on the undirected graph network and the connection matrix C, the distributed gradient decent (DGD) is used to fuse the temporary gradient estimation matrix between nodes, thereby solving the distributed optimization problem represented by equation (7); the specific approach is to perform the following iterative steps for each data node: Where W j (t) is the local parameter vector at the tth iteration and is the loss function l RLE At the tth iteration, about W j The derivative of (t) and Ψ j (t) is the temporary parameter estimation matrix of the local node at the tth iteration; η(t)∈(0,1] is a learning rate hyperparameter that changes over time and satisfies the vanishing condition: Then if the entire network is strongly connected, the information between each node can be communicated and transmitted between the networks, and finally a consistent optimum is achieved; Formula (8a) represents the update process of the temporary parameter estimation matrix of the local node. After the local node is updated, its temporary parameter estimation matrix will be transmitted to the neighboring nodes. Each node then performs a weighted combination according to the temporary parameter estimation matrices transmitted by the neighboring nodes through Formula (8b) to obtain the final local parameter vector W j (t+1); while transmitting key information, it does not involve the mutual transmission of original data, effectively protecting the privacy of each node.

3. The method according to claim 1, characterized in that: In step 4), the temporary parameter estimation matrix Ψ is calculated j (t) Compared to the last trigger time The temporary parameter estimation matrix The rate of change of is considered to be worthy of transmission when the rate of change is greater than the threshold δ, that is: , node j is allowed to pass information to neighboring nodes; if this event is triggered on node j, then the triggering event time point Will be updated to t, the corresponding temporary parameter estimation matrix allowed to be transmitted Will also be replaced by j (t) and transmit it to neighboring nodes.

Images