A small sample classification method and device based on random projection metric space

Abstract

The present invention discloses a small sample classification method and device based on a random projection metric space, relating to the field of natural language processing technology. The method comprises: obtaining multiple tasks to be classified; inputting the multiple tasks into a classification model based on the random projection metric space; and obtaining classification results for the multiple tasks based on the multiple tasks and the classification model based on the random projection metric space. The present invention constructs a random metric space adapted to specific tasks, constructs a metric space based on task features by fine-tuning the position of sample vectors in the metric space, and specifically uses random vectors to learn the metric space for each task, thereby resolving the problem that the general practice of using a universal metric space is not well suited for all tasks and lacks applicability.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a small sample classification method and device based on random projection metric space. Background Art

[0002] Few-shot learning can learn from a small amount of data, making it applicable to many fields lacking sufficient labeled data (Chen et al., 2019; Wang et al., 2020; Garcia and Bruna, 2017). Few-shot learning has become a hot topic in natural language processing research. Few-shot learning techniques are used in various fields of natural language processing. For text classification, few-shot learning (Yu et al., 2018; Geng et al., 2019; Bao et al., 2019) can learn from a small amount of labeled data and transfer it to new classification tasks.

[0003] Many scholars have proposed solutions to the many difficulties that arise in the field of small-sample learning research. In response to the diversity of small-sample text classification tasks and the potential noise in samples, Gao T et al. (Gao et al., 2019) used hierarchical attention to address this issue, thereby improving the robustness of the model. Obamuyide et al. (Obamuyide and Vlachos, 2019) used supervised classification problems as an example of meta-learning and proposed a model-independent meta-learning solution. Xie et al. (Xie et al., 2020) used heterogeneous graph networks and adversarial training to reduce the model's sensitivity to noisy samples.

[0004] Metric-based meta-learning methods predict labels by learning a metric space and using a distance space. Generally speaking, meta-learning requires training on multiple tasks and uniformly projecting these subtasks into the same metric space (Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018). Early research focused on learning a more optimal metric space (e.g., obtaining a better embedding layer) (Vinyals et al., 2016) and exploring how to learn a more optimal distance space (Sung et al., 2018). While using the same metric space allows for the discovery of common features across multiple tasks, the difficulty of finding these features varies across tasks (Bao et al., 2019; Sui et al., 2020). When the sample distributions of different subtasks overlap or differ, optimization results in poor results. Summary of the Invention

[0005] The present invention addresses the problem that the same metric space cannot be applied to different tasks and the projections of different tasks during classification easily overlap in the same metric space, resulting in a decrease in classification accuracy.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In one aspect, the present invention provides a small sample classification method based on a random projection metric space, the method being implemented by an electronic device and comprising:

[0008] S1. Obtain multiple tasks to be classified.

[0009] S2. Input multiple tasks into the classification model based on random projection metric space.

[0010] S3. According to the multiple tasks and the classification model based on the random projection metric space, classification results of the multiple tasks are obtained.

[0011] Optionally, the classification results of the multiple tasks obtained in S3 according to the multiple tasks and the classification model based on the random projection metric space include:

[0012] S31. Obtain a support set and a query set for each of the multiple tasks.

[0013] S32. Perform feature extraction on the support set and the query set according to the embedding layer of the classification model based on the random projection metric space to obtain feature information of the support set and the query set.

[0014] S33. Construct a metric space for each task respectively.

[0015] S34. Based on the metric space, support set, and feature information of the query set of each task, classification results of multiple tasks are obtained.

[0016] Optionally, constructing a metric space for each task in S33 includes:

[0017] Random vector projection learning is performed on each task separately to obtain the metric space of each task.

[0018] Optionally, random vector projection learning for each task includes:

[0019] S331, using two independent single-layer perceptrons, respectively generate parameters γ according to random vectors predict and β predict .

[0020] S332: Determine whether the difference between the characteristics of each task and the characteristics of the existing task exceeds a threshold; if so, modify the sample characteristics in the task.

[0021] S333, using the weight generator, generate the normalized parameter γ according to the random vector predict and β predict .

[0022] Optionally, the parameter γ in S331 predict and β predict , as shown in the following formulas (1) and (2):

[0023]

[0024]

[0025] Among them, Δγ predict is the normalization parameter before adjustment, c is a random vector; It is a single-layer perceptron.

[0026] Optionally, in S332, the sample features in the task are modified as shown in the following equations (3)-(5):

[0027]

[0028] γ predict =Δγ predict +1 (4)

[0029]

[0030] Among them, γ penalty and β penalty is the Δγ generated by predict and β predict Perform L2 regularization, formula (4) is to prevent γ predict The presence of 0 elements in , causes the features of some dimensions of the sample vector x to become invalid.

[0031] Optionally, the normalized parameter γ generated in S333 is predict and β predict , as shown in the following equations (6)-(8):

[0032]

[0033]

[0034]

[0035] Among them, x is the original task feature; x′ is the adjusted task feature; D trainis the training set; ⊙ is the Hadamard product.

[0036] On the other hand, the present invention provides a small sample classification device based on a random projection metric space, which is used to implement a small sample classification method based on a random projection metric space, and the device includes:

[0037] The acquisition module is used to obtain multiple tasks to be classified.

[0038] The input module is used to input multiple tasks into the classification model based on random projection metric space.

[0039] The output module is used to obtain classification results of multiple tasks according to the multiple tasks and the classification model based on the random projection metric space.

[0040] Optionally, the output module is further configured to:

[0041] S31. Obtain a support set and a query set for each of the multiple tasks.

[0042] S32. Perform feature extraction on the support set and the query set according to the embedding layer of the classification model based on the random projection metric space to obtain feature information of the support set and the query set.

[0043] S33. Construct a metric space for each task respectively.

[0044] S34. Based on the metric space, support set, and feature information of the query set of each task, classification results of multiple tasks are obtained.

[0045] Optionally, the output module is further configured to:

[0046] Random vector projection learning is performed on each task separately to obtain the metric space of each task.

[0047] Optionally, the output module is further configured to:

[0048] S331, using two independent single-layer perceptrons, respectively generate parameters γ according to random vectors predict and β predict .

[0049] S332: Determine whether the difference between the characteristics of each task and the characteristics of the existing task exceeds a threshold; if so, modify the sample characteristics in the task.

[0050] S333, using the weight generator, generate the normalized parameter γ according to the random vector predict and β predict .

[0051] Optionally, the parameter γ predict and βpredict , as shown in the following formulas (1) and (2):

[0052]

[0053]

[0054] Among them, γ predict is the normalization parameter before adjustment, c is a random vector; It is a single-layer perceptron.

[0055] Optionally, the sample features in the task are modified as shown in the following equations (3)-(5):

[0056]

[0057] γ predict =Δγ predict +1 (4)

[0058]

[0059] Among them, γ penalty and β penalty is the Δγ generated by predict and β predict Perform L2 regularization, formula (4) is to prevent γ predict The presence of 0 elements in , causes the features of some dimensions of the sample vector x to become invalid.

[0060] Optionally, generate the normalized parameter γ predict and β predict , as shown in the following equations (6)-(8):

[0061]

[0062]

[0063]

[0064] Among them, x is the original task feature; x′ is the adjusted task feature; D train is the training set; ⊙ is the Hadamard product.

[0065] On the one hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned small sample classification method based on random projection metric space.

[0066] On the one hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned small sample classification method based on random projection metric space.

[0067] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0068] In the above scheme, after using the metric space based on random projection, the performance of the three models on both datasets was improved, which shows that the method of using random vectors to learn the metric space for each task is better than the method of constructing a universal metric space.

[0069] Our approach is to use random vectors to construct a metric space for each task, and the results show that the results are significantly improved compared to the baseline, indicating that better classification results can be learned in the randomly projected metric space.

[0070] It's worth noting that on the FewRel dataset, the Siamese network performed the worst of the three models. However, after optimizing the metric space, its accuracy improved significantly, surpassing the relational network in some cases. This may be because the universal metric space limits the performance of the Siamese network. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0072] Figure 1 This is a flowchart of a small sample classification method based on random projection metric space provided by an embodiment of the present invention;

[0073] Figure 2 This is an overview diagram of a small sample classification method based on random projection metric space provided by an embodiment of the present invention;

[0074] Figure 3 This is a comparison diagram of projections into the same metric space and different metric spaces provided by an embodiment of the present invention;

[0075] Figure 4 1 is a graph showing experimental results of three models provided by an embodiment of the present invention;

[0076] Figure 5 This is a block diagram of a small sample classification device based on random projection metric space provided by an embodiment of the present invention;

[0077] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0078] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0079] like Figure 1 As shown, the embodiment of the present invention provides a small sample classification method based on random projection metric space, which can be implemented by an electronic device. Figure 1 The flowchart of the small sample classification method based on random projection metric space is shown. The processing flow of this method may include the following steps:

[0080] S1. Obtain multiple tasks to be classified.

[0081] S2. Input multiple tasks into the classification model based on random projection metric space.

[0082] S3. According to the multiple tasks and the classification model based on the random projection metric space, classification results of the multiple tasks are obtained.

[0083] Optionally, the above step S3 may include the following steps S31-S33:

[0084] S31. Obtain a support set and a query set for each of the multiple tasks.

[0085] S32. Perform feature extraction on the support set and the query set according to the embedding layer of the classification model based on the random projection metric space to obtain feature information of the support set and the query set.

[0086] S33. Construct a metric space for each task respectively.

[0087] In one feasible implementation, an overview diagram of a small sample classification method based on random projection metric space is shown as follows: Figure 2 shown.

[0088] Furthermore, the present invention proposes to construct a different metric space for each task to solve the problems of the prior art. Specifically, each task is learned through random vector projection and a random metric space is obtained. The random projection in the high-dimensional space maps the metric spaces of different tasks to different spaces, thereby improving the performance of the classification model.

[0089] Optionally, constructing a metric space for each task in S33 includes:

[0090] Random vector projection learning is performed on each task separately to obtain the metric space of each task.

[0091] Optionally, random vector projection learning for each task includes:

[0092] S331, using two independent single-layer perceptrons, respectively generate parameters γ according to random vectors predict and β predict .

[0093] Optionally, the parameter γ in S331 predict and β predict , as shown in the following formulas (1) and (2):

[0094]

[0095]

[0096] Among them, Δγ predict is the normalization parameter before adjustment, c is a random vector; It is a single-layer perceptron.

[0097] In one feasible implementation, due to the transformation parameter γ predict The corresponding elements are multiplied (Hadamard product) with the sample vector x. In order to prevent the generated transformation parameter γ predict The presence of 0 elements in , which results in the invalidation of the features of some dimensions of the sample vector x, thus affecting the measurement results. In the specific implementation, this application adopts the offset strategy proposed by Perez et al. That is, γ predict =1+Δγ predict , select the offset Δγ that generates the parameter γ predict .

[0098] S332: Determine whether the difference between the characteristics of each task and the characteristics of the existing task exceeds a threshold; if so, modify the sample characteristics in the task.

[0099] Optionally, in S332, the sample features in the task are modified as shown in the following equations (3)-(5):

[0100]

[0101] γ predict =Δγ predict +1 (4)

[0102]

[0103] Among them, γ penalty and β penalty is the Δγ generated by predict and β predict Perform L2 regularization, formula (4) is to prevent γ predictThe presence of 0 elements in , causes the features of some dimensions of the sample vector x to become invalid.

[0104] In a feasible implementation, since the model itself can mine effective universal features, in order to accelerate and stabilize the training process, the present invention chooses to modify the sample features in the task only when the current task features are significantly different from the features of the previously encountered tasks, that is, to execute the above formulas (3)-(5) on the generated Δ.

[0105] S333, using the weight generator, generate the normalized parameter γ according to the random vector predict and β predict .

[0106] Optionally, the normalized parameter γ generated in S333 is predict and β predict , as shown in the following equations (6)-(8):

[0107]

[0108]

[0109]

[0110] Among them, x is the original task feature; x′ is the adjusted task feature; D train is the training set; ⊙ is the Hadamard product.

[0111] In one feasible implementation, drawing on the method of using text questions to adjust image features extracted by a feature extractor in a visual question-answering system (Perez et al., 2018), the present invention uses random vectors to adjust the feature positions of samples in the feature space. Specifically, the present invention uses a weight generator to generate a normalized parameter sum based on the random vector, i.e., employing conditional normalization, as shown in equations (6)-(8) above.

[0112] S34. Based on the metric space, support set, and feature information of the query set of each task, classification results of multiple tasks are obtained.

[0113] Furthermore, the experimental process includes:

[0114] FewRel is a small-sample relation classification dataset constructed by Han et al. (Han et al., 2018). It contains 100 relations, each with 700 samples. The experiment repartitioned multiple publicly available categories in a ratio of 50:14:16 to construct training, validation, and test sets. HuffPost is a news headline classification dataset collected from the HuffPost website. It contains 41 categories, each with 900 samples. This paper divides the categories of this dataset into training, validation, and test sets in a ratio of 26:7:8.

[0115] The present invention uses random vectors to learn different metric spaces for each task; the evaluation indicator of model performance adopts accuracy.

[0116] This paper selects three metric-based models as benchmark models, namely: Siamese networks (Koch et al., 2015), prototype networks (Snell et al., 2017), and relational networks (Sung et al., 2018).

[0117] In the meta-training process, the present invention adopts the HighWay (Snelletal., 2017) strategy of the prototype network. On the FewRel dataset, 30-way1-shot is selected for single-sample (1-shot) meta-training, and 5-way1-shot and 10-way1-shot meta-validation and meta-testing are performed. 20-way5-shot is selected for small-sample (5-shot) meta-training, and 5-way5-shot and 10-way5-shot meta-validation and meta-testing are performed. On the Huffpost dataset, 15-way1-shot is selected for single-sample (1-shot) meta-training, and 5-way1-shot meta-validation and meta-testing are performed. 10-way5-shot is selected for small-sample (5-shot) meta-training, and 5-way5-shot meta-validation and meta-testing are performed.

[0118] like Figure 3 As shown in the figure, this invention views classification as a mapping in a high-dimensional space. When using the same metric space, it is difficult to learn a mapping that can simultaneously separate all samples projected onto the same metric space. In the figure on the right, each task has its own metric space, which makes it easier to classify samples.

[0119] like Figure 4 The following table shows the experimental results of the three models. +random indicates that a random vector is used to project the metric space of each task. The numbers in brackets represent the improved accuracy compared to the baseline model.

[0120] This paper constructs a randomly projected metric space for each task by fine-tuning the position of sample vectors in the metric space. Specifically, the method uses random vectors to learn the metric space for each task. Experiments using this random vector approach on three metric space models revealed improved performance on both datasets for all three models after using the randomly projected metric space. This demonstrates that using random vectors to learn the metric space for each task is superior to constructing a universal metric space.

[0121] In the embodiment of the present invention, after using the metric space based on random projection, the performance of the three models on both datasets was improved, which shows that the method of using random vectors to learn the metric space of each task is better than the method of constructing a universal metric space.

[0122] Our approach is to use random vectors to construct a metric space for each task, and the results show that the results are significantly improved compared to the baseline, indicating that better classification results can be learned in the randomly projected metric space.

[0123] It's worth noting that on the FewRel dataset, the Siamese network performed the worst of the three models. However, after optimizing the metric space, its accuracy improved significantly, surpassing the relational network in some cases. This may be because the universal metric space limits the performance of the Siamese network.

[0124] like Figure 5 As shown, an embodiment of the present invention provides a small sample classification device 500 based on a random projection metric space. The device 500 is used to implement a small sample classification method based on a random projection metric space. The device 500 includes:

[0125] The acquisition module 510 is used to acquire a plurality of tasks to be classified.

[0126] The input module 520 is used to input multiple tasks into the classification model based on the random projection metric space.

[0127] The output module 530 is configured to obtain classification results of the multiple tasks according to the multiple tasks and the classification model based on the random projection metric space.

[0128] Optionally, the output module 530 is further configured to:

[0129] S31. Obtain a support set and a query set for each of the multiple tasks.

[0130] S32. Perform feature extraction on the support set and the query set according to the embedding layer of the classification model based on the random projection metric space to obtain feature information of the support set and the query set.

[0131] S33. Construct a metric space for each task respectively.

[0132] S34. Based on the metric space, support set, and feature information of the query set of each task, classification results of multiple tasks are obtained.

[0133] Optionally, the output module 530 is further configured to:

[0134] Random vector projection learning is performed on each task separately to obtain the metric space of each task.

[0135] Optionally, the output module 530 is further configured to:

[0136] S331, using two independent single-layer perceptrons, respectively generate parameters γ according to random vectors predict and β predict .

[0137] S332: Determine whether the difference between the characteristics of each task and the characteristics of the existing task exceeds a threshold; if so, modify the sample characteristics in the task.

[0138] S333, using the weight generator, generate the normalized parameter γ according to the random vector predict and β predict .

[0139] Optionally, the parameter γ predict and β predict , as shown in the following formulas (1) and (2):

[0140]

[0141]

[0142] Among them, Δγ predict is the normalization parameter before adjustment, c is a random vector; It is a single-layer perceptron.

[0143] Optionally, the sample features in the task are modified as shown in the following equations (3)-(5):

[0144]

[0145] γ predict =Δγ predict +1 (4)

[0146]

[0147] Among them, γ penalty and β penalty is the Δγ generated by predict and βpredict Perform L2 regularization, formula (4) is to prevent γ predict The presence of 0 elements in , causes the features of some dimensions of the sample vector x to become invalid.

[0148] Optionally, generate the normalized parameter γ predict and β predict , as shown in the following equations (6)-(8):

[0149]

[0150]

[0151]

[0152] Among them, x is the original task feature; x′ is the adjusted task feature; D train is the training set; ⊙ is the Hadamard product.

[0153] In the embodiment of the present invention, after using the metric space based on random projection, the performance of the three models on both datasets was improved, which shows that the method of using random vectors to learn the metric space of each task is better than the method of constructing a universal metric space.

[0154] Our approach is to use random vectors to construct a metric space for each task, and the results show that the results are significantly improved compared to the baseline, indicating that better classification results can be learned in the randomly projected metric space.

[0155] It's worth noting that on the FewRel dataset, the Siamese network performed the worst of the three models. However, after optimizing the metric space, its accuracy improved significantly, surpassing the relational network in some cases. This may be because the universal metric space limits the performance of the Siamese network.

[0156] Figure 6 6 is a schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present invention. The electronic device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPUs) 601 and one or more memories 602. The memories 602 store at least one instruction, which is loaded and executed by the processor 301 to implement the following small sample classification method based on random projection metric space:

[0157] S1. Obtain multiple tasks to be classified.

[0158] S2. Input multiple tasks into the classification model based on random projection metric space.

[0159] S3. According to the multiple tasks and the classification model based on the random projection metric space, classification results of the multiple tasks are obtained.

[0160] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions. The instructions are executable by a processor in a terminal to implement the above-described small sample classification method based on a random projection metric space. For example, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0161] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0162] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A small sample classification method based on random projection metric space, which is applied to text classification and is characterized by: The method comprises: S1. Obtain multiple tasks to be classified; S2. inputting the multiple tasks into a classification model based on random projection metric space; S3. Obtaining classification results of the multiple tasks according to the multiple tasks and the classification model based on the random projection metric space; The step S3 of obtaining classification results of the multiple tasks according to the multiple tasks and the classification model based on the random projection metric space includes: S31, obtaining a support set and a query set for each of the multiple tasks; S32. Perform feature extraction on the support set and the query set according to the embedding layer of the classification model based on the random projection metric space to obtain feature information of the support set and the query set; S33, construct a metric space for each task respectively; S34, obtaining classification results of multiple tasks based on the metric space of each task, the support set, and the feature information of the query set; The step S33 of constructing a metric space for each task includes: Perform random vector projection learning on each task separately to obtain the metric space of each task; The random vector projection learning for each task includes: S331, using two independent single-layer perceptrons, respectively generate parameters γ according to random vectors predict and β predict ; S332: Determine whether the difference between the characteristics of each task and the characteristics of the existing task exceeds a threshold; if so, modify the sample characteristics in the task; S333, using the weight generator, generate the normalized parameter γ according to the random vector predict and β predict .

2. The method according to claim 1, characterized in that The parameter γ in S331 predict and β predict , as shown in the following formulas (1) and (2): Among them, △γ predict is the normalization parameter before adjustment, c is a random vector; It is a single-layer perceptron.

3. The method according to claim 2, characterized in that The sample features in the task are modified in S332 as shown in the following equations (3)-(5): c predict =△γ predict +1 (4) Among them, γ penalty and β penalty is the △γ generated by predict and β predict Perform L2 regularization, formula (4) is to prevent γ predict The presence of 0 elements in , causes the features of some dimensions of the sample vector x to become invalid.

4. The method according to claim 1, wherein The normalized parameter γ generated in S333 predict and β predict , as shown in the following equations (6)-(8): Among them, x is the original task feature; x′ is the adjusted task feature; D train is the training set; ⊙ is the Hadamard product.

5. A small sample classification device based on random projection metric space, which is applied to text classification, characterized in that: The device comprises: An acquisition module is used to obtain multiple tasks to be classified; An input module, configured to input the plurality of tasks into a classification model based on a random projection metric space; An output module, configured to obtain classification results of the multiple tasks according to the multiple tasks and the classification model based on the random projection metric space; Obtaining classification results of the multiple tasks according to the multiple tasks and the classification model based on the random projection metric space includes: S31, obtaining a support set and a query set for each of the multiple tasks; S32. Perform feature extraction on the support set and the query set according to the embedding layer of the classification model based on the random projection metric space to obtain feature information of the support set and the query set; S33, construct a metric space for each task respectively; S34, obtaining classification results of multiple tasks based on the metric space of each task, the support set, and the feature information of the query set; The step S33 of constructing a metric space for each task includes: Perform random vector projection learning on each task separately to obtain the metric space of each task; The random vector projection learning for each task includes: S331, using two independent single-layer perceptrons, respectively generate parameters γ according to random vectors predict and β predict ; S332: Determine whether the difference between the characteristics of each task and the characteristics of the existing task exceeds a threshold; if so, modify the sample characteristics in the task; S333, using the weight generator, generate the normalized parameter γ according to the random vector predict and β predict .

Images