Short text labeling method and device

Abstract

The present application discloses a short text labeling method and device, the method comprising: preliminarily clustering samples according to the similarity between samples, labeling the samples using a label library composed of words extracted from the samples, thereby improving the uniformity of the sample labels; when labeling the samples, selecting multiple samples with low similarity from each cluster for labeling, thereby improving the text labeling efficiency of the samples; updating the clusters according to the labeling results of each round, dynamically controlling the fission degree of each cluster, and when the labeled samples in the jth cluster contain L different labels, clustering and updating the jth cluster according to the L labels based on the similarity between the unlabeled samples in the jth cluster and the L labels, where L≥2; obtaining the final labeled samples when the clustering is stable, thereby improving the label coverage in the labeled samples and thereby improving the quality of the labeled samples.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to a method and device for labeling short text. Background Art

[0002] With the tightening of financial regulation and the rapid development of financial business integration, the digital transformation of commercial banks has become an inevitable trend. Digital transformation is driving the rapid development of banking businesses, and the urgent need for commercial banks to upgrade their compliance management capabilities has made the development of intelligent compliance management systems imperative. External regulatory rules, external regulations, and internal rules and regulations, as important compliance data assets, are the primary source for building a compliance labeling system. Effective text annotation of these rules and regulations is crucial for the subsequent digital development of the labeling system.

[0003] In related technologies, the method of labeling text is to label samples manually and randomly. However, this method often results in too many samples under a certain label, too few samples under other labels, or even no samples at all. The labeled samples finally obtained cannot cover all labels, and the sample quality is low, which affects the precision and recall of subsequent text classification through machine learning models. Summary of the Invention

[0004] In view of this, the first aspect of the present application provides a short text labeling method, comprising:

[0005] Dividing the samples into M clusters by calculating the similarity between the samples;

[0006] In the i-th round of labeling, when labeling the j-th cluster, unlabeled samples with similarity lower than the labeling threshold are selected from the j-th cluster as samples to be labeled; wherein, in the j-th cluster, the labeled samples have the same label, j≤M;

[0007] Labeling the sample to be labeled using a label in a label library, and updating the labeled sample in the j-th cluster according to the labeling result of the sample to be labeled; wherein the label library is generated based on the words in the sample;

[0008] When the labeled samples in the j-th cluster contain L different labels, clustering the j-th cluster according to the L labels based on the similarity between the unlabeled samples in the j-th cluster and the L labels, where L≥2;

[0009] Update the M value according to the number of clusters after the i-th round of marking, and determine whether the M value after the i-th round of marking is the same as the M value after the i-1-th round of marking;

[0010] If yes, output the labeled sample;

[0011] If not, enter the i+1th round of marking.

[0012] Preferably, when the labeled samples in the j-th cluster contain L different labels, after clustering the j-th cluster according to the L labels based on the similarity between the unlabeled samples in the j-th cluster and the L labels, and before updating the M value according to the number of clusters after the i-th round of labeling, and determining whether the M value after the i-th round of labeling is the same as the M value after the i-1-th round of labeling, the method further includes:

[0013] Among the clusters obtained after the i-th round of labeling, when there are Q clusters in which the labels of the labeled samples are the same, the Q clusters are aggregated, where Q≥2.

[0014] Preferably, in the i-th round of labeling, when labeling the j-th cluster, before selecting the unlabeled samples with similarity lower than the labeling threshold from the j-th cluster as samples to be labeled, the method further includes:

[0015] According to the jth label priority, K unlabeled samples are selected from the jth cluster; wherein, in the jth cluster, the label of the labeled sample is the jth label corresponding to the jth label priority, and the K value is positively correlated with the jth label priority;

[0016] In the i-th round of labeling, when labeling the j-th cluster, selecting the unlabeled samples with similarity lower than the labeling threshold from the j-th cluster as samples to be labeled includes:

[0017] In the i-th round of labeling, when labeling the j-th cluster, according to the j-th label priority, K unlabeled samples with similarities lower than the labeling threshold are selected from the j-th cluster as samples to be labeled.

[0018] Preferably, the j-th tag priority is calculated by the following formula:

[0019]

[0020] Among them, the P j is the jth label priority in the i-th round of labeling; j is the number of samples in the jth cluster; N is the number of samples in all clusters; n is the number of labeled samples in all clusters; j is the number of labeled samples in the j-th cluster; Min is the minimum similarity between samples in the j-th cluster.

[0021] Preferably, the tag library is generated in the following manner:

[0022] Segmenting the sample according to a financial segmentation dictionary to obtain a first segmentation sample;

[0023] According to a stop word dictionary, the stop words in the first segmentation sample are removed to obtain the second segmentation sample;

[0024] Segmenting the sentences in the second word segmentation sample using a sliding window and combining words to obtain a potential word library set;

[0025] The scores of all words in the potential word library are calculated according to word frequency, word length and word occurrence position, and the words whose scores reach the tag score threshold are combined into the tag library.

[0026] Preferably, the score of the word is calculated as follows:

[0027]

[0028] Among them, the Score t is the score of the tth word, the a t is the number of times the t-th word appears in all samples, and the a all is the total number of words in all samples; d is the total number of samples, is the total number of samples containing the t-th word; t is the length of the tth word, the max len is the maximum word length of all words; Represents the average value of the total number of words in all samples. t is the average value of the position of the t-th word in sample X.

[0029] A second aspect of the present application provides a short text labeling device, comprising:

[0030] A sample clustering unit, configured to: divide the samples into M clusters by calculating the similarity between the samples;

[0031] a sample selection unit, configured to: during the i-th round of labeling, when labeling the j-th cluster, select unlabeled samples with a similarity lower than a labeling threshold from the j-th cluster as samples to be labeled; wherein, in the j-th cluster, the labeled samples have the same label, j≤M;

[0032] a labeling unit, configured to label the sample to be labeled using labels in a label library, and update the labeled sample in the j-th cluster according to the labeling result of the sample to be labeled; wherein the label library is generated based on the words in the sample;

[0033] The sample clustering unit is further configured to: when the labeled samples in the j-th cluster contain L different labels, update the j-th cluster according to the L labels based on the similarity between the unlabeled samples in the j-th cluster and the L labels, where L≥2;

[0034] The sample clustering unit is further configured to update the M value according to the number of clusters after the i-th round of marking, and determine whether the M value after the i-th round of marking is the same as the M value after the i-1-th round of marking;

[0035] an output unit, configured to: if yes, output the labeled sample;

[0036] The marking unit is further configured to: if not, enter the (i+1)th round of marking.

[0037] Preferably, the sample clustering unit is further used for:

[0038] Among the clusters obtained after the i-th round of labeling, when there are Q clusters in which the labels of the labeled samples are the same, the Q clusters are aggregated, where Q≥2.

[0039] Preferably, the sample selection unit is further configured to:

[0040] According to the jth label priority, K unlabeled samples are selected from the jth cluster; wherein, in the jth cluster, the label of the labeled sample is the jth label corresponding to the jth label priority, and the K value is positively correlated with the jth label priority;

[0041] The sample selection unit is further configured to:

[0042] In the i-th round of labeling, when labeling the j-th cluster, according to the j-th label priority, K unlabeled samples with similarities lower than the labeling threshold are selected from the j-th cluster as samples to be labeled.

[0043] Preferably, the device further comprises:

[0044] A label library generating unit is used to: segment the sample according to a financial segmentation dictionary to obtain a first segmentation sample;

[0045] The tag library generating unit is further configured to: remove stop words from the first segmentation sample according to a stop word dictionary to obtain the second segmentation sample;

[0046] The tag library generating unit is further configured to: segment the sentences in the second word segmentation sample by a sliding window, and perform word combination to obtain a potential word library set;

[0047] The tag library generating unit is further configured to calculate scores of all words in the potential lexicon according to word frequency, word length, and word occurrence position, and form the tag library with words whose scores reach a tag score threshold.

[0048] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: the present application preliminarily clusters the samples according to the similarity between the samples, and labels the samples through a label library composed of words extracted from the samples, thereby improving the uniformity of the sample labels; when labeling the samples, multiple samples with low similarity are selected from each cluster for labeling, thereby improving the text labeling efficiency of the samples; the clusters are updated according to the results of each round of labeling, and the fission degree of each cluster is dynamically controlled. When the clusters are stable, the final labeled samples are obtained, thereby improving the coverage of labels in the labeled samples, and thereby improving the quality of the labeled samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Flowchart provided for the embodiment of this application;

[0050] Figure 2 A flowchart for generating a tag library provided in an embodiment of the present application;

[0051] Figure 3 Schematic diagram of the compliance labeling system provided in the embodiment of this application;

[0052] Figure 4 A schematic block diagram of a short text labeling method provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of a sliding window provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the forward mechanism of the sample pre-classification unit provided in an embodiment of the present application;

[0055] Figure 7 Schematic diagram of the sample pre-classification unit feedback mechanism provided in an embodiment of the present application;

[0056] Figure 8 A diagram showing the working mechanism of the text annotation unit provided in an embodiment of the present application;

[0057] Figure 9A diagram showing the working mechanism of the tag priority calculation unit provided in an embodiment of the present application;

[0058] Figure 10 A diagram showing the working mechanism of the label review unit provided in an embodiment of the present application;

[0059] Figure 11 A schematic diagram of the device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of protection of the present application.

[0061] Step 101: Divide the samples into M clusters by calculating the similarity between the samples;

[0062] The samples used in this application are text data samples obtained by segmenting short text articles. By calculating the similarity between samples, the samples are divided into M clusters, and samples with higher similarity are grouped into one cluster. Among them, before the sample is marked, the M value can be set randomly by the program or by a technician; the similarity between samples can be calculated using formulas such as the cosine distance or Euclidean distance of the sample vectors. The similarity is negatively correlated with the distance. The greater the similarity between two samples, the smaller the distance between the two samples.

[0063] Step 102: During the i-th round of labeling, when labeling the j-th cluster, unlabeled samples with similarity lower than the labeling threshold are selected from the j-th cluster as samples to be labeled;

[0064] In the jth cluster, the labeled samples have the same label, and the jth cluster is one of the M clusters. In the labeling process, the samples used in this application are composed of labeled samples and unlabeled samples.

[0065] Among them, labeling refers to text annotation, which is the process of defining text according to its content. In this application, it refers to summarizing the text content, and the final summary result is called a label.

[0066] In each round of labeling, a labeling threshold is set, and the labeling threshold is used to select the most representative sample from each of the M clusters, that is, the sample with the lowest similarity with other samples. When the similarity between a certain sample and other samples in the j-th cluster is lower than the labeling threshold, it is considered that the similarity between the sample and the other samples is low, and the sample needs to be labeled to determine whether to continue to keep the sample in the current cluster. It should be noted that the similarity between the samples can be the average value obtained after calculating the similarity between a certain sample and other samples respectively, or it can be other values ​​that can reflect the degree of similarity between the sample and other samples, which does not affect the implementation of the embodiments of the present application.

[0067] In a possible implementation, before step 102, the following steps may be further included:

[0068] According to the jth label priority, select K unlabeled samples from the jth cluster;

[0069] Among them, in the jth cluster, the label of the labeled sample is the jth label corresponding to the jth label priority, and the number K of unlabeled samples selected from the jth cluster is positively correlated with the jth label priority.

[0070] Then step 102 can be implemented as follows: during the i-th round of labeling, when labeling the j-th cluster, first determine the number K of unlabeled samples that need to be selected from the j-th cluster based on the j-th label priority; then, based on the similarity between samples, select K unlabeled samples from the j-th cluster whose similarity is lower than the labeling threshold as samples to be labeled.

[0071] In addition, the j-th label priority can be calculated as follows:

[0072]

[0073] Among them, P j N is the jth label priority in the i-th round of labeling; j is the number of samples in the jth cluster; N is the number of samples in all clusters; n is the number of labeled samples in all clusters; n j is the number of labeled samples in the j-th cluster; Min is the minimum similarity between samples in the j-th cluster.

[0074] By calculating the label priority and dynamically adjusting the label priority of the cluster where the unlabeled samples are located according to the data distribution of the labeled samples, the sample distribution under each label is balanced, and the probability of uneven distribution of samples under each label due to the randomness of sample sampling is reduced, making the distribution of labeled samples more consistent with the actual classification ratio.

[0075] Step 103: Use the tags in the tag library to label the samples to be labeled, and update the labeled samples in the j-th cluster according to the labeling results of the samples to be labeled;

[0076] In a possible implementation, the tag library can be generated in the following way:

[0077] Step 201: Segment the samples according to the financial word segmentation dictionary to obtain the first segmented samples;

[0078] In a possible implementation, for the text samples segmented by the document structure recognition technology, the Jieba word segmentation tool can be used in combination with the financial word segmentation dictionary to segment the samples, so as to better apply to the application scenarios described in this application.

[0079] Step 202: Remove the stop words in the first segmented samples according to the stop word dictionary to obtain the second segmented samples;

[0080] In the first segmented samples obtained by word segmentation, there may be words that are meaningless for tag extraction, such as "de", "le", etc. In order to better apply to the application scenarios described in this application, a stop word dictionary with financial characteristics can also be introduced to filter the first segmented samples and remove the meaningless stop words therein.

[0081] Step 203: Segment the sentences in the second segmented samples through a sliding window and perform word combination to obtain a potential word library set;

[0082] Through the sliding window, set an appropriate window length to segment the sentences in the second segmented samples. When combining the segmented words, the N-gram mechanism can be used to combine the words to obtain a potential word library set generated by all word combinations.

[0083] Step 204: Calculate the scores of all words in the potential word library set according to the word frequency, word length, and the position where the words appear, and form the tag library with the words whose scores reach the tag score threshold.

[0084] The calculation rules for the word scores in the potential word library set are as follows:

[0085]

[0086] Among them, Score t is the score of the t-th word, a t is the number of times the t-th word appears in all samples, a all is the total number of words in all samples, represents the word frequency; C d is the total number of samples, is the total number of samples containing the t-th word, Indicates the importance of the t-th word, which means that when it appears frequently in sample X and rarely in other samples, it is more representative of sample X; len t is the length of the tth word, max len is the maximum word length of all words, Indicates the influence of word length. Generally, the longer the word, the more information it contains and the more it can represent sample X as a keyword. Mean represents the average of the total number of words in all samples. t is the average value of the position of the t-th word in sample X, Indicates the influence of the word position on the score. Generally, the closer the word is to the front, the more representative it is of sample X.

[0087] By extracting words from samples to generate a tag library, and using the words in the tag library that best match the samples to be labeled to label the text, the problem of difficulty in defining tags when manually annotating text is solved; based on the sliding window to extract potential discontinuous word tags, the problem that traditional keyword extraction tools are difficult to extract discontinuous combination words in position is solved.

[0088] Step 104: When the labeled samples in the jth cluster contain L different labels, based on the similarity between the unlabeled samples in the jth cluster and the L labels, the jth cluster is clustered and updated according to the L labels;

[0089] When the labels of the labeled samples in the jth cluster are different, the similarity between the unlabeled samples in the jth cluster and the L labels is calculated, and each unlabeled sample is divided into the cluster where the label with the highest similarity is located to complete the update of the jth cluster.

[0090] In a possible implementation, after step 103 and before step 104, the following steps may be further included:

[0091] In the clusters obtained after the i-th round of labeling, if there are Q clusters with the same labels of labeled samples, the Q clusters are aggregated, where Q ≥ 2.

[0092] By aggregating clusters with the same labels and dynamically adjusting the number of clusters, label redundancy is avoided and the efficiency of sample annotation is improved.

[0093] Step 105: Update the M value according to the number of clusters after the i-th round of marking, and determine whether the M value after the i-th round of marking is the same as the M value after the i-1-th round of marking;

[0094] Step 106: If yes, output the labeled sample;

[0095] Step 107: If not, enter the (i+1)th round of marking.

[0096] After the i-th round of labeling is completed, the number of clusters M is updated, and the number of clusters in the i-th round is compared with the number of clusters in the i-1-th round. If they are the same, the clustering is considered stable and all labels have been marked, and the labeled samples are output; if they are different, the clustering is considered unstable, and unlabeled samples need to be selected from each cluster for labeling until the clustering is stable.

[0097] In a possible implementation, after the sample clustering tends to be stable, the unlabeled samples in the j-th cluster may be labeled according to the labels of the labeled samples in the j-th cluster.

[0098] The samples are preliminarily clustered by calculating the similarity between samples, and the samples are labeled using a tag library generated by words extracted from the samples, thereby improving the uniformity of the sample labels; when labeling the samples, samples with lower similarity are selected from each cluster for labeling, thereby improving the efficiency of text labeling for the samples; the clusters are updated according to the results of each round of labeling, and the fission degree of each cluster is dynamically controlled. When the clusters are stable, the final labeled samples are obtained, thereby improving the coverage of labels in the labeled samples and thus improving the quality of the labeled samples.

[0099] The following describes the scenario embodiments provided by this application in conjunction with specific application scenarios.

[0100] In order to solve the problems of internalizing external regulations, mapping internal and external regulations, and searching for rules and regulations, the digital transformation of banks needs to establish the following Figure 3 The compliance labeling system shown in the figure. For the internalization of external regulations and the mapping of internal and external regulations, newly added external rules and regulations can be classified according to their corresponding compliance labels, and internal regulations under these labels can be searched to determine whether the relevant regulations need to be modified or added. For the systematic learning of rules and regulations, the structured compliance labeling system helps staff to systematically and comprehensively understand the regulations under each label.

[0101] Figure 4 This is a schematic block diagram of the short text labeling method provided in an embodiment of the present application. Based on the functions to be implemented in this embodiment of the present application, a label extraction unit, a sample pre-classification unit, a text labeling unit, a label priority calculation unit, and a label review unit are provided. A text dataset is composed of all samples. A sample clustering library is generated based on the sample clustering results. A labeled sample library is generated based on the labeled samples. The labeled samples in the labeled sample library come from the sample clustering library.

[0102] It should be noted that in Figure 4In the schematic block diagram of the short text labeling method shown, each unit is only a description of the function of the embodiment of the present application, and is not a limitation of the device of the embodiment of the present application. Other descriptions are also possible as long as the functions of the embodiment of the present application can be realized.

[0103] Step 401: Input the text dataset into the tag extraction unit, and generate a predefined tag library based on the words extracted from each sample;

[0104] The process of using sliding windows to segment sentences and combining words through the N-gram mechanism to extract potential vocabulary sets is as follows: Figure 5 As shown. Figure 5 In the example, the sliding window has a length of 5 and a step size of 1. After each calculation, the sliding window advances one unit word. For the word segmentation results within the sliding window, the N-gram value is set to 3. Each time, 1-3 units of words are selected from the sliding window and combined to form the words in the potential vocabulary. The sliding window and N-gram mechanism can extract important words that are not continuous in word position, such as "corporate remittance" and "international business."

[0105] Step 402: inputting the text data set into the sample pre-classification unit for pre-classification;

[0106] In the sample pre-classification unit, before labeling begins, you can set the initial clustering value M, that is, the estimated value of the number of label classifications, and preliminarily divide the samples in the text data set into M clusters according to their similarity. One cluster is a cluster, and a sample cluster library is generated. Then, through the labeled sample library generated by the text labeling unit, the degree of cluster splitting in the sample cluster library is dynamically adjusted by the positive mechanism and feedback mechanism.

[0107] See also Figure 6 As shown in FIG, the forward mechanism of the sample pre-classification unit is to cluster the samples in the text dataset according to the clustering value to generate a sample cluster library.

[0108] See also Figure 7 As shown, the feedback mechanism of the sample pre-classification unit is to update the sample cluster library based on the annotated sample library generated by the text annotation unit. When samples from the same cluster in the sample cluster library have different labels in the annotated sample library, the same cluster is split, and the cluster value and sample cluster library are updated, where the cluster value is the number of clusters in the sample cluster library. When samples with the same label in the annotated sample library come from different clusters in the sample cluster library, the different clusters are aggregated, and the cluster value and sample cluster library are updated.

[0109] Step 403: Select unlabeled samples from the sample cluster library as samples to be labeled and input into the text labeling unit, and label the samples to be labeled according to the predefined label library, label priority and label library, and generate a labeled sample library based on the labeling results;

[0110] In the text annotation unit, the working mechanism for labeling samples is as follows: Figure 8 As shown. When the first round of labeling is performed, the text labeling unit extracts x samples with the lowest text similarity from each cluster in the sample cluster library, and labels them using the label in the predefined label library that best matches the sample; when the similarity between the label in the predefined label library and the sample is low, the text labeling unit can prompt the staff to select a label in the predefined label library or a custom label for the sample for pre-labeling. After the pre-labeling is completed, the text labeling unit determines whether the sample label comes from the predefined label library. If so, the pre-labeled sample is added to the labeled sample library as a labeled sample; if not, the aforementioned pre-labeled sample is sent to the label review unit, and the label review unit confirms whether the label in the pre-labeled sample is added to the label library. After the first round of labeling, the label review unit generates a label library based on the predefined label library for use in subsequent labeling.

[0111] When performing non-first-round labeling, since there is already data in the labeled sample library, the sample pre-classification unit dynamically adjusts the sample cluster library based on the labeled sample library, and the label priority is adjusted according to the results of the previous round of labeling. Therefore, in non-first-round labeling, you only need to input the total number of samples Y that need to be labeled in this round into the text labeling unit. The text labeling unit calculates the number of samples to be extracted from each cluster based on the label priority and the total number of samples Y, and takes out the corresponding number of samples for labeling.

[0112] Step 404: After each round of labeling is completed, the label priority calculation unit dynamically adjusts the label priority based on the labeling results of the existing data;

[0113] The calculation of the tag priority can refer to the calculation rule of the j-th tag priority in step 102. In addition, see Figure 9 As shown, in addition to calculating and updating the priorities of various tags, the working mechanism of the tag priority calculation unit also includes a manual intervention mechanism to fine-tune the tag priorities through professional experience to better meet the needs of actual business scenarios.

[0114] Step 405: The tag review unit updates the tag library to reduce the possibility that the tags in the tag library have different synonyms.

[0115] See also Figure 10As shown, in an embodiment of the present application, a label review unit reviews the predefined label library and whether there are synonymous words in the label library, and reviews the custom labels generated during the labeling process to confirm whether new labels need to be added to the label library to avoid generating redundant labels.

[0116] See also Figure 11 As shown, the device used in this application includes: a sample clustering unit 1101, a sample selection unit 1102, a marking unit 1103 and an output unit 1104.

[0117] The sample clustering unit 1101 is used to divide the samples into M clusters by calculating the similarity between the samples;

[0118] The sample selection unit 1102 is configured to select unlabeled samples with similarity lower than a labeling threshold from the jth cluster as samples to be labeled during the i-th round of labeling, when labeling the jth cluster; wherein, in the jth cluster, the labeled samples have the same label, j≤M;

[0119] The labeling unit 1103 is used to label the samples to be labeled using the labels in the label library, and update the labeled samples in the jth cluster according to the labeling results of the samples to be labeled; wherein the label library is generated based on the words in the samples;

[0120] The sample clustering unit 1101 is further configured to update the jth cluster according to the L labels based on the similarity between the unlabeled samples in the jth cluster and the L labels when the labeled samples in the jth cluster contain L different labels, where L≥2;

[0121] The sample clustering unit 1101 is further configured to update the M value according to the number of clusters after the i-th round of marking, and determine whether the M value after the i-th round of marking is the same as the M value after the i-1-th round of marking;

[0122] Output unit 1104, configured to: if yes, output the labeled sample;

[0123] The marking unit 1103 is further configured to: if not, enter the (i+1)th round of marking.

[0124] In a possible implementation, the sample clustering unit 1101 is further configured to:

[0125] In the clusters obtained after the i-th round of labeling, if there are Q clusters with the same labels of labeled samples, the Q clusters are aggregated, where Q ≥ 2.

[0126] In one possible implementation, the sample selection unit 1102 is further configured to select K unlabeled samples from the jth cluster according to the jth label priority; wherein, in the jth cluster, the label of the labeled sample is the jth label corresponding to the jth label priority, and the K value is positively correlated with the jth label priority;

[0127] The sample selection unit 1102 is further configured to select K unlabeled samples with similarities lower than a labeling threshold from the jth cluster as samples to be labeled according to the jth label priority during the i-th round of labeling when labeling the jth cluster.

[0128] In one possible implementation, Figure 11 The device shown may further include a calculation unit for calculating the j-th tag priority using the formula described in step 102 ; in addition, the calculation unit is further configured to calculate the score of the word using the formula described in step 204 .

[0129] In one possible implementation, Figure 11 The apparatus shown may further include a tag library generating unit, configured to:

[0130] Segment the sample according to the financial segmentation dictionary to obtain the first segmentation sample;

[0131] According to the stop word dictionary, the stop words in the first segmentation sample are removed to obtain the second segmentation sample;

[0132] Segment the sentences in the second word segmentation sample using a sliding window and combine words to obtain a potential vocabulary set;

[0133] The scores of all words in the potential vocabulary are calculated based on word frequency, word length and the position where the words appear, and the words whose scores reach the label score threshold are combined into a label library.

[0134] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0135] Although the operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or in a sequential order.Multitasking and parallel processing may be advantageous under certain circumstances.

[0136] It should be understood that the various steps described in the method embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0137] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0138] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0141] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0142] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), random access memory (English full name: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store computer programs.

[0143] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A short text labeling method, characterized in that: The method comprises: Dividing the samples into M clusters by calculating the similarity between the samples; In the i-th round of labeling, when labeling the j-th cluster, unlabeled samples with similarity lower than the labeling threshold are selected from the j-th cluster as samples to be labeled; wherein, in the j-th cluster, the labeled samples have the same label, j≤M; Labeling the sample to be labeled using a label in a label library, and updating the labeled sample in the j-th cluster according to the labeling result of the sample to be labeled; wherein the label library is generated based on the words in the sample; When the labeled samples in the j-th cluster contain L different labels, clustering the j-th cluster according to the L labels based on the similarity between the unlabeled samples in the j-th cluster and the L labels, where L≥2; Update the M value according to the number of clusters after the i-th round of marking, and determine whether the M value after the i-th round of marking is the same as the M value after the i-1-th round of marking; If yes, output the labeled sample; If not, enter the i+1th round of marking.

2. The method according to claim 1, characterized in that After updating the j-th cluster according to the L labels based on the similarity between the unlabeled samples in the j-th cluster and the L labels when the labeled samples in the j-th cluster contain L different labels, and before updating the M value according to the number of clusters after the i-th round of labeling, and determining whether the M value after the i-th round of labeling is the same as the M value after the i-1-th round of labeling, the method further includes: Among the clusters obtained after the i-th round of labeling, when there are Q clusters in which the labels of the labeled samples are the same, the Q clusters are aggregated, where Q≥2.

3. The method according to claim 1, characterized in that In the i-th round of labeling, when labeling the j-th cluster, before selecting the unlabeled samples with similarity lower than the labeling threshold from the j-th cluster as samples to be labeled, the method further includes: According to the jth label priority, K unlabeled samples are selected from the jth cluster; wherein, in the jth cluster, the label of the labeled sample is the jth label corresponding to the jth label priority, and the K value is positively correlated with the jth label priority; In the i-th round of labeling, when labeling the j-th cluster, selecting the unlabeled samples with similarity lower than the labeling threshold from the j-th cluster as samples to be labeled includes: In the i-th round of labeling, when labeling the j-th cluster, according to the j-th label priority, K unlabeled samples with similarities lower than the labeling threshold are selected from the j-th cluster as samples to be labeled.

4. The method according to claim 3, characterized in that The j-th tag priority is calculated using the following formula: Among them, the P j is the jth label priority in the i-th round of labeling; j is the number of samples in the jth cluster; N is the number of samples in all clusters; n is the number of labeled samples in all clusters; j is the number of labeled samples in the j-th cluster; Min is the minimum similarity between samples in the j-th cluster.

5. The method according to claim 1, wherein The tag library is generated in the following way: Segmenting the sample according to a financial segmentation dictionary to obtain a first segmentation sample; According to a stop word dictionary, the stop words in the first word segmentation sample are removed to obtain a second word segmentation sample; Segmenting the sentences in the second word segmentation sample using a sliding window and combining words to obtain a potential word library set; The scores of all words in the potential word library are calculated according to word frequency, word length and word occurrence position, and the words whose scores reach the tag score threshold are combined into the tag library.

6. The method according to claim 5, characterized in that The score of the word is calculated as follows: Among them, the Score t is the score of the tth word, the a t is the number of times the t-th word appears in all samples, and the a all is the total number of words in all samples; d is the total number of samples, is the total number of samples containing the t-th word; t is the length of the tth word, the max len is the maximum word length of all words; Represents the average value of the total number of words in all samples. t is the average value of the position of the t-th word in sample X.

7. A short text labeling device, characterized in that: The device comprises: A sample clustering unit, configured to: divide the samples into M clusters by calculating the similarity between the samples; a sample selection unit, configured to: during the i-th round of labeling, when labeling the j-th cluster, select unlabeled samples with a similarity lower than a labeling threshold from the j-th cluster as samples to be labeled; wherein, in the j-th cluster, the labeled samples have the same label, j≤M; a labeling unit, configured to label the sample to be labeled using labels in a label library, and update the labeled sample in the j-th cluster according to the labeling result of the sample to be labeled; wherein the label library is generated based on the words in the sample; The sample clustering unit is further configured to: when the labeled samples in the j-th cluster contain L different labels, update the j-th cluster according to the L labels based on the similarity between the unlabeled samples in the j-th cluster and the L labels, where L≥2; The sample clustering unit is further configured to update the M value according to the number of clusters after the i-th round of marking, and determine whether the M value after the i-th round of marking is the same as the M value after the i-1-th round of marking; an output unit, configured to: if yes, output the labeled sample; The marking unit is further configured to: if not, enter the (i+1)th round of marking.

8. The device according to claim 7, characterized in that The sample clustering unit is further used for: Among the clusters obtained after the i-th round of labeling, when there are Q clusters in which the labels of the labeled samples are the same, the Q clusters are aggregated, where Q≥2.

9. The device according to claim 7, characterized in that The sample selection unit is further configured to: According to the jth label priority, K unlabeled samples are selected from the jth cluster; wherein, in the jth cluster, the label of the labeled sample is the jth label corresponding to the jth label priority, and the K value is positively correlated with the jth label priority; The sample selection unit is further configured to: In the i-th round of labeling, when labeling the j-th cluster, according to the j-th label priority, K unlabeled samples with similarities lower than the labeling threshold are selected from the j-th cluster as samples to be labeled.

10. The device according to claim 7, characterized in that The device further comprises: A label library generating unit is used to: segment the sample according to a financial segmentation dictionary to obtain a first segmentation sample; The tag library generating unit is further configured to: remove stop words from the first word segmentation sample according to a stop word dictionary to obtain a second word segmentation sample; The tag library generating unit is further configured to: segment the sentences in the second word segmentation sample by a sliding window, and perform word combination to obtain a potential word library set; The tag library generating unit is further configured to calculate scores of all words in the potential lexicon according to word frequency, word length, and word occurrence position, and form the tag library with words whose scores reach a tag score threshold.

Images