Method, device and electronic device for multi-label classification of events with time series characteristics

Abstract

The present application provides a method, device and electronic device for multi-label classification of events with time series characteristics, which inputs unlabeled events into the current embedding model for embedding. The updated current embedding model can better extract features of unlabeled events, and then clusters the first vector representation obtained by embedding to obtain labeled data containing multiple first category labels and first unlabeled data. The attention mechanism is used to obtain the word label corresponding to each first category label and the self-label corresponding to the first unlabeled data. The classification label of each unlabeled event is determined based on the word label and the self-label, and the classification of the unlabeled event is completed. The classification method provided by the present application can show the evolution of the distribution of unlabeled events over time, can be applied to different fields and scenarios, and has good portability.

Description

Technical Field

[0001] The present application relates to the technical field of event classification, and in particular to a method, device, and electronic device for multi-label classification of events with time series characteristics. Background Art

[0002] With the continuous advancement of informatization, grid-based systems, and intelligent governance in social and grassroots governance, various social governance platforms collect massive amounts of social governance events daily. How to effectively utilize these collected events to identify the main contradictions in social governance, the connections between various events, and the changing trends of the main contradictions to be addressed over time, and how to resolve contradictions at their source based on current status and development trends, are urgent issues that need to be addressed in the current intelligent social governance landscape.

[0003] Related technologies primarily use text classification or clustering methods to analyze and mine a large number of social governance events. However, text classification methods use fixed classification labels, resulting in poor timeliness. While clustering methods offer timeliness, cluster labels are typically numerical. Labels generated using statistical learning methods cannot accurately describe the meaning of data within a cluster, resulting in poor classification results. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a method, device and electronic device for multi-label classification of events with time series characteristics.

[0005] Based on the above objectives, the first aspect of the present application provides a multi-label classification method for events with time series features, comprising:

[0006] Obtaining a plurality of current unlabeled events, inputting the plurality of unlabeled events into a current embedding model, and outputting a first vector representation corresponding to each of the unlabeled events via the current embedding model;

[0007] Clustering all the first vector representations using a clustering algorithm to obtain labeled data including a plurality of first category labels and first unlabeled data;

[0008] For each first category label, use the attention mechanism of the embedding model to perform word labeling on all labeled data corresponding to the first category label to obtain a word label corresponding to the first category label; and use the attention mechanism of the embedding model to perform word labeling on the first unlabeled data to obtain a self-label corresponding to the first unlabeled data;

[0009] A classification label for each of the unlabeled events is determined based on the word label and the self-label.

[0010] Optionally, the using the attention mechanism of the embedding model to perform word tagging on all labeled data corresponding to the first category label to obtain a word label corresponding to the first category label includes:

[0011] For each sentence in all the labeled data corresponding to the first category label, using the attention mechanism of the embedding model to obtain an attention value for each word in the sentence, selecting words corresponding to the first N attention values in descending order as a set of candidate words for the sentence; determining a set of candidate keywords corresponding to the sentence based on the set of candidate words, where N is a positive integer;

[0012] Constructing a keyword frequency table based on a set of candidate keywords corresponding to all sentences in all labeled data corresponding to the first category label;

[0013] The keywords corresponding to the first M word frequencies are selected from the keyword word frequency table in descending order of word frequency values as word labels corresponding to the first category label, where M is a positive integer.

[0014] Optionally, the using the attention mechanism of the embedding model to perform word tagging on the first unlabeled data to obtain self-labels corresponding to the first unlabeled data includes:

[0015] For each sentence in the first unlabeled data, the attention mechanism of the embedding model is used to obtain an attention value of each word in the sentence, and a self-label of the sentence is determined based on all the attention values.

[0016] Optionally, determining a classification label for each of the unlabeled events based on the word label and the self-label includes:

[0017] In response to the unlabeled event being the labeled data, selecting at least one keyword from the word labels of the first category label corresponding to the labeled data as a classification label for the unlabeled event; or,

[0018] In response to the unlabeled event being the first unlabeled data, the self-label corresponding to the first unlabeled data is used as a classification label for the unlabeled event.

[0019] Optionally, in response to the ratio of the number of the first unlabeled data to the number of the unlabeled events exceeding a first preset threshold within a preset time interval, the current classification model is trained based on all the first unlabeled data to obtain an updated current classification model, wherein the current classification model includes the current embedding model and a classification layer superimposed on the current embedding model.

[0020] Optionally, the training the current classification model based on all the first unlabeled data to obtain an updated current classification model includes:

[0021] Clustering all of the first unlabeled data using a clustering algorithm to obtain labeled data containing multiple second category labels;

[0022] Determining training sample data based on the labeled data including multiple second category labels and a second preset threshold;

[0023] The current classification model is trained based on the training sample data to obtain an updated current classification model.

[0024] Optionally, the second preset threshold includes a persistence threshold and a probability threshold, and determining the training sample data based on the labeled data including multiple second category labels and the second preset threshold includes:

[0025] The labeled data whose probability value exceeds the probability threshold and whose corresponding second category label persistence exceeds the persistence threshold is used as the training sample data.

[0026] Optionally, before clustering all the first vector representations using a clustering algorithm, the method further includes:

[0027] A dimensionality reduction algorithm is used to perform dimensionality reduction processing on the first vector representation.

[0028] Based on the same inventive concept, the second aspect of the present application provides a multi-label classification device for events with time series characteristics, comprising:

[0029] an embedding module configured to obtain a plurality of current unlabeled events, input the plurality of unlabeled events into a current embedding model, and output a first vector representation corresponding to each of the unlabeled events via the current embedding model;

[0030] a clustering module configured to cluster all the first vector representations using a clustering algorithm to obtain labeled data including a plurality of first category labels and first unlabeled data;

[0031] The tagging module is configured to, for each first category label, use the attention mechanism of the embedding model to perform word tagging on all labeled data corresponding to the first category label to obtain a word label corresponding to the first category label; and use the attention mechanism of the embedding model to perform word tagging on the first unlabeled data to obtain a self-label corresponding to the first unlabeled data;

[0032] A determination module is configured to determine a classification label for each of the unlabeled events based on the word label and the self-label.

[0033] Based on the same inventive concept, the third aspect of this application also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0034] From the above, it can be seen that the present application provides a method, device and electronic device for multi-label classification of events with time series characteristics, which inputs unlabeled events into the current embedding model for embedding. The updated current embedding model can better extract features of unlabeled events, and then clusters the first vector representation obtained by embedding to obtain labeled data containing multiple first category labels and first unlabeled data. The attention mechanism is used to obtain the word label corresponding to each first category label and the self-label corresponding to the first unlabeled data. The classification label of each unlabeled event is determined based on the word label and the self-label, and the classification of the unlabeled event is completed. Compared with the supervised learning algorithm, the classification method provided by the present application does not require manual labeling of data, which greatly reduces labor costs. At the same time, the multiple first category labels obtained by clustering are not limited to the number and type of traditional preset classification labels, and can be more accurately labeled according to the characteristics of the unlabeled event itself. The classification method provided by the present application can show the evolution of the distribution of unlabeled events over time, can be applied to different fields and scenarios, and has good portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] Figure 1 Schematic diagram of the process of the multi-label classification method of events with time series characteristics according to an embodiment of the present application;

[0037] Figure 2 A flowchart of a method for determining a word label according to an embodiment of the present application;

[0038] Figure 3 Schematic diagram of attention distribution of the embedding model before and after updating according to an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the process of training the current classification model according to an embodiment of the present application;

[0040] Figure 5 This is a flowchart of a multi-label classification method for events with time series features according to an embodiment of the present application;

[0041] Figure 6 This is a schematic diagram of the structure of a multi-label classification device for events with time series characteristics according to an embodiment of the present application;

[0042] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0044] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0045] According to the background technology, at present, the analysis and mining of a large number of social governance events mainly adopts text classification or clustering methods to classify events of the same type into one category, and then process them by category. However, text classification and clustering methods have certain defects. Among them, the defects of text classification methods are as follows: [1] The framework of text classification is mainly based on given classification labels, which cannot identify new categories of events and is not timely; [2] It is mainly based on supervised learning, requires a large amount of labeled data, and has high labor costs. The defects of text clustering methods are as follows: [1] Compared with text classification methods, although text clustering methods can identify new categories of events and are timely, they need to be calculated on all data each time. When the data accumulates to a large order of magnitude, a single clustering operation takes a long time and cannot be completed within a reasonable and acceptable time range; [2] The clusters obtained after clustering only have abstract digital labels. When converting them into meaningful text labels, statistical learning methods are generally used to generate labels based on the word frequency of the cluster event text. The obtained labels are poor in accuracy and cannot summarize the meaning of all the data in the cluster. The common defects of classification and clustering methods are: [1] Each event is forced to be classified into a category label, which has certain limitations in processing certain events that may have multiple category label attributes at the same time; [2] Each label is regarded as an independent individual, and the semantic association between labels is not taken into account.

[0046] In view of this, the present application addresses the deficiencies in the prior art and provides a method, device, and electronic device for event multi-label classification with temporal characteristics. This method integrates algorithms in the field of artificial intelligence natural language processing, such as sentence embedding algorithms based on pre-trained language models, dimensionality reduction algorithms, hierarchical clustering algorithms, and attention mechanisms, which effectively solves the limitation that the multi-category classification framework cannot label multiple category labels for the same event. It can show the changes in event distribution over time, has temporal characteristics, and does not require manual annotation of data. While liberating manpower, it also has good portability and can flexibly switch to different scenarios without relying on the corpus of the usage scenario. In response to the problem of the accuracy of the semantic description of the cluster in which the generated cluster labels are located, the present application adopts a method based on the attention mechanism to directly decode the latent space of the cluster to obtain the cluster labels. This method can directly learn the cluster features from the latent space, and establish a natural mapping between the potential semantic features of the category and the keywords. The keywords obtained in this way can be more consistent with the semantic features of the category in which they are located than traditional statistical machine learning algorithms such as tf-idf.

[0047] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0048] This application provides a multi-label classification method for events with time series characteristics, referring to Figure 1 , including the following steps:

[0049] Step 102: Obtain a plurality of current unlabeled events, input the plurality of unlabeled events into a current embedding model, and output a first vector representation corresponding to each of the unlabeled events via the current embedding model.

[0050] Specifically, unlabeled events are unclassified events. The current multiple unlabeled events can be multiple social governance events collected on the same day to be classified. Social governance events can also be events to be classified in other fields. There is no specific restriction on the field of the event here. The embedding model is pre-built and continuously updated. After obtaining multiple unlabeled events, the unlabeled events are input into the current embedding model obtained by the most recent update. The unlabeled events are feature extracted through the embedding model, and the first vector representation of each unlabeled event is output through the current embedding model.

[0051] Step 104: cluster all the first vector representations using a clustering algorithm to obtain labeled data containing multiple first category labels and first unlabeled data.

[0052] In some embodiments, before clustering all the first vector representations using a clustering algorithm, the method includes: using a dimensionality reduction algorithm to perform dimensionality reduction processing on the first vector representation. The first vector representation output by the current embedding model is a high-dimensional vector representation. Due to the sparsity and nearest neighbor characteristics of high-dimensional space, there are no data clusters in the high-dimensional space, and each piece of data will be regarded as an independent category. Therefore, directly clustering high-dimensional data is generally very poor, and the data needs to be reduced in dimensionality first. This embodiment uses the dimensionality reduction algorithm UMAP based on unified manifold approximation and projection to reduce the dimensionality of the obtained high-dimensional vector to obtain a low-dimensional sentence embedding vector.

[0053] Furthermore, the first vector representation that has undergone dimensionality reduction is clustered. In this embodiment, the density-based clustering algorithm HDBSCAN is used to cluster all the first vector representations to obtain labeled data containing multiple first category labels and first unlabeled data. Among them, the first category label is a digital label obtained by clustering through a clustering algorithm. For example, the digital label can be 1, 2, 3, etc. There are multiple first category labels, and the number is determined by the semantics of the unlabeled event itself. Each first category label corresponds to a cluster obtained by clustering, indicating that the data in the cluster are semantically similar. The first unlabeled data is the data marked as the "other" category in this clustering process. The data in the "other" category has a low similarity with other data and is assigned to the "other" category during the clustering process.

[0054] Step 106: For each first category label, use the attention mechanism of the embedding model to perform word labeling on all labeled data corresponding to the first category label to obtain a word label corresponding to the first category label; and use the attention mechanism of the embedding model to perform word labeling on the first unlabeled data to obtain a self-label corresponding to the first unlabeled data.

[0055] Specifically, the attention mechanism within the embedding model is used to perform word labeling on all labeled data corresponding to the first category label and the first unlabeled data. While outputting the first vector representation, the embedding model also outputs an attention value for each word in each event sentence. Keywords corresponding to each event sentence are calculated based on all attention values. The word labels for the first category label and the self-labels for the first unlabeled data are then determined based on the keywords in each sentence. Compared to traditional statistical machine learning algorithms, the attention mechanism can directly decode clustered word labels from the latent space of the cluster, making these word labels more consistent with the semantic characteristics of the category.

[0056] Step 108: Determine the classification label of each of the unlabeled events based on the word label and the self-label. The word label corresponds to the first category label. The word label may include several keywords, which are keywords that appear frequently in all events of the cluster corresponding to the first category label. This means that the cluster may correspond to multiple keywords. When determining the classification label of each event, at least one keyword is selected from the word label as the classification label of the unlabeled event, breaking the phenomenon that each event can only correspond to one label in the traditional classification method. If the unlabeled event is marked as the "other" category during the clustering process and there is no corresponding first category label, the attention mechanism is used to generate a self-label as the classification label of the unlabeled event.

[0057] Based on the above steps 102 to 108, it can be seen that the present application provides a multi-label classification method for events with time series characteristics, which inputs unlabeled events into the current embedding model for embedding. The updated current embedding model can better extract features of unlabeled events, and then clusters the first vector representation obtained by embedding to obtain labeled data containing multiple first category labels and first unlabeled data. The attention mechanism is used to obtain the word label corresponding to each first category label and the self-label corresponding to the first unlabeled data. The classification label of each unlabeled event is determined based on the word label and the self-label, and the classification of the unlabeled event is completed. Compared with the supervised learning algorithm, the classification method provided by the present application does not require manual labeling of data, which greatly reduces labor costs. At the same time, the multiple first category labels obtained by clustering are not limited to the number and type of traditional preset classification labels, and can be more accurately labeled according to the characteristics of the unlabeled events themselves. The classification method provided by the present application can show the evolution of the distribution of unlabeled events over time, can be applied to different fields and scenarios, and has good portability.

[0058] In some embodiments, reference Figure 2 , the attention mechanism of the embedding model is used to perform word labeling on all labeled data corresponding to the first category label to obtain a word label corresponding to the first category label, including:

[0059] Step 202: For each sentence in all the labeled data corresponding to the first category label, the attention mechanism of the embedding model is used to obtain the attention value of each word in the sentence, and the words corresponding to the first N attention values are selected in descending order as the candidate word set of the sentence; based on the candidate word set, the candidate keyword set corresponding to the sentence is determined, where N is a positive integer.

[0060] When the embedding model is used to output the first vector representation of the unlabeled event, the embedding model also outputs the attention value for each word in the event sentence. For example, the unlabeled event sentence input into the current embedding model is "Someone is walking his dog without a leash in the community", where the attention value of each word is, "small": 1.2826, "district": 1.2883, "there is": 4.0910, "person": 4.2800, "walk": 14.7049, "dog": 47.8593, "not": 4.2378, "tie": 8.2129, "rope": 7.3051. The words "walk" and "dog" have the highest attention values, followed by "tie" and "rope". Therefore, "walk", "dog", "tie", and "rope" are used as the candidate word set. In this embodiment, N is 4. Since the candidate word set includes the candidate words "walk", "dog", "tie", and "rope", the mapping to the keywords is "walk the dog" and "tie rope", that is, the keyword set includes the two words "walk the dog" and "tie rope".

[0061] Step 204: Construct a keyword frequency table based on the candidate keyword sets corresponding to all sentences in all the labeled data corresponding to the first category label. After obtaining the keyword set for each labeled data item in the first category label according to step 202, all keyword sets are merged and deduplicated to obtain a keyword frequency table. The frequency of each keyword in the frequency table is the number of times it appears in the entire keyword set.

[0062] Step 206: Select the top M keywords corresponding to the keyword frequency table in descending order of frequency as the word labels corresponding to the first category label, where M is a positive integer. For example, in this embodiment, M is 5, which means that the top five keywords with the highest frequency values are selected as the word labels corresponding to the first category label. For example, the selected word labels may be: dog walking / leash / dog / urination / dog shit, which means that the word labels of all the labeled data in the cluster corresponding to the first category label include 5 keywords.

[0063] Based on the above steps 202 to 206 , a word label corresponding to each first category label is determined.

[0064] In some embodiments, the using the attention mechanism of the embedding model to perform word tagging on the first unlabeled data to obtain self-labels corresponding to the first unlabeled data includes:

[0065] For each sentence in the first unlabeled data, the attention mechanism of the embedding model is used to obtain an attention value of each word in the sentence, and a self-label of the sentence is determined based on all the attention values.

[0066] Similar to the process of determining word labels above, self-labels are calculated by calculating the attention value of each word in the unlabeled event sentence. Words with high attention values in the sentence are used as a candidate word set. All words in the candidate word set are then mapped to keywords to obtain a keyword set. This keyword set is the self-label for the first unlabeled data.

[0067] In some embodiments, determining a classification label for each of the unlabeled events based on the word label and the self-label comprises:

[0068] In response to the unlabeled event being the labeled data, selecting at least one keyword from the word labels of the first category label corresponding to the labeled data as a classification label for the unlabeled event; or,

[0069] In response to the unlabeled event being the first unlabeled data, the self-label corresponding to the first unlabeled data is used as a classification label for the unlabeled event.

[0070] Specifically, after determining the word label corresponding to each first category label, if the unlabeled event is labeled data, it means that the data is assigned to a digital category during the clustering process and there is a corresponding first category label. Since the word label contains at least one keyword, not all keywords in the word label appear in the unlabeled event. Therefore, when determining the classification label for the unlabeled event, the keyword that appears in the unlabeled event is selected from the word label as the classification label of the unlabeled event. Similarly, the number of classification labels is at least one.

[0071] It should be noted that the unlabeled event may not contain any keywords in the word label. For example, the unlabeled event is "How do I apply for a dog license now?" In the clustering process, since the clustering is based on the first vector representation, the first vector representation contains implicit information about "dog" (the word "dog" exists in the unlabeled event), so the unlabeled event is assigned to a cluster. The word labels corresponding to the cluster are: walking the dog / leash / dog / urination / dog shit. The unlabeled event does not contain any keywords in the word label. In this case, the keyword of the unlabeled event can be determined as the classification label through the attention mechanism. The classification label of the unlabeled event sentence "How do I apply for a dog license now?" determined by the attention mechanism is "dog".

[0072] If the unlabeled event is the first unlabeled data, the self-label of the first unlabeled data is directly used as the classification label of the unlabeled event.

[0073] In some embodiments, in response to the ratio of the number of the first unlabeled data to the number of the unlabeled events exceeding a first preset threshold within a preset time interval, the current classification model is trained based on all the first unlabeled data to obtain an updated current classification model, wherein the current classification model includes the current embedding model and a classification layer superimposed on the current embedding model.

[0074] Specifically, the current embedding model is continuously updated, that is, the embedding model is updated once every period of time. The preset time interval can be 7 days, 10 days or a month, etc. It can be specifically determined based on the number and type of new unlabeled events added each day. If the number is large and changes rapidly, the preset time interval can be shortened to increase the update rate of the embedding model, so that the unlabeled events can be better extracted. Sentences with similar meanings are output with embedding vectors of similar distances, and sentences with different meanings are output with embedding vectors of farther distances. When the ratio of the total number of first unlabeled data within the preset time interval to the total number of unlabeled events within the preset time interval exceeds the first preset threshold, it means that within the preset time interval, the data assigned to the "other" category by the clustering algorithm has accumulated to a certain value, and the clustering effect of the clustering algorithm on these "other" data is not ideal, and the current embedding model needs to be adjusted. Exemplarily, the first preset threshold in this embodiment is 10%. The current classification model is trained using all the first unlabeled data within a preset time interval, and the parameters of the classification model are adjusted. The current classification model is a model formed by superimposing a classification layer on the current embedding model. By updating the parameters of the current classification model, the parameters of the current embedding model are also updated. After obtaining the updated classification model, the classification layer is removed to obtain the updated current embedding model. The attention distribution of the updated embedding model tends to give higher scores to words related to the current category, while the attention before the update tends to be dispersed to different keywords. Figure 3 The attention distribution of the embedding model before and after the update is shown. It can be clearly seen that before the update, the attention was scattered to each word of the unlabeled event. After the update, the attention was obviously more focused on the word "dog".

[0075] In some embodiments, the current classification model is trained based on all the first unlabeled data to obtain an updated current classification model, referring to Figure 4 , including the following steps:

[0076] Step 402: cluster all the first unlabeled data using a clustering algorithm to obtain labeled data containing multiple second category labels.

[0077] Specifically, all first unlabeled data within a preset time interval is clustered using the density-based clustering algorithm HDBSCAN. Those skilled in the art may select other density clustering algorithms based on actual needs, and no specific limitations are imposed herein. After clustering, labeled data containing multiple second category labels is obtained, where the second category labels are digital labels.

[0078] Step 404: Determine training sample data based on the labeled data containing multiple second category labels and the second preset threshold.

[0079] Since the labeled data needs to be used for subsequent classification model training, the quality requirements for the data are relatively high. Among all the categories finally output by the density clustering algorithm, there are some low-quality categories, and within each category, there are low-quality data. Therefore, data screening is required to retain only high-quality labeled data for training the classification model. While the clustering algorithm outputs the second category label for each data piece, it can also output the persistence of each cluster category and the probability that each data piece belongs to its cluster category. The persistence of each cluster category and the probability of belonging to its cluster category are compared with the second preset threshold respectively, and the labeled data that meets the second preset threshold is used as training sample data.

[0080] Step 406: Train the current classification model based on the training sample data to obtain an updated current classification model.

[0081] The selected high-quality training sample data is used to train the current classification model to obtain an updated current classification model.

[0082] In some embodiments, the second preset threshold includes a persistence threshold and a probability threshold, and determining the training sample data based on the labeled data including the plurality of second category labels and the second preset threshold includes:

[0083] The labeled data whose probability value exceeds the probability threshold and whose corresponding second category label persistence exceeds the persistence threshold is used as the training sample data.

[0084] When screening training sample data, it is necessary to use the second preset threshold for screening. The second preset threshold specifically includes a persistence threshold and a probability threshold. The persistence value is used to characterize the quality of the cluster. The data in high-quality clusters basically belong to the same category, while the data in low-quality clusters are more scattered and the categories are not uniform. The persistence exceeds the persistence threshold, indicating that the cluster is of high quality and the labeled data in the cluster is of high quality. The probability of labeled data represents the confidence level within the cluster. The higher the confidence level, the greater the possibility that the data belongs to the cluster. When the probability exceeds the probability threshold, it indicates that the data is of high quality. Data that meets both the persistence threshold and the probability threshold is used as training sample data to improve the quality of the training sample data and thereby improve the performance of the embedding model.

[0085] In some embodiments, the multi-label classification method for events with time series characteristics provided in this application can also be described in the following manner. Figure 5A flow chart of a multi-label classification method for events with time series characteristics is shown. When the time node is 0, the unlabeled initial data set 0 is labeled based on the unlabeled initial data set 0 through the classification model labeled data automatic generation module 10 to obtain labeled data with new labels. The unlabeled initial data set 0 can be understood as historical unlabeled data (equivalent to the historical unlabeled events in the aforementioned embodiment). The classification model labeled data automatic generation module 10 is used to cluster the data to generate digital labels, and screen high-quality data, and finally output labeled data with new labels. The labeled data with new labels are used to train the multi-category classification model to obtain the multi-category classification model 0. The unlabeled initial data set 0 is then classified by the multi-label automatic classification module 20 based on the multi-category classification model 0 to obtain the class label data 0 and self-label data 0 corresponding to the unlabeled initial data set 0. The multi-label automatic classification module 20 based on the multi-class classification model 0 uses the embedding layer of the multi-class classification model 0 to embed the unlabeled initial data set 0 to obtain the corresponding vector representation, and uses the self-attention mechanism to calculate the vector representation to obtain class label data 0 and self-label data 0. The class label data is equivalent to the word label in the aforementioned embodiment. The word label generation and self-label generation process are the same as the aforementioned embodiment and will not be repeated here. Within the range of time node 1 to time node n, for each newly added data set at time node 1, the corresponding class label data and self-label data are output by the multi-label automatic classification module 20 based on the multi-class classification model 0. When the self-label data between time node 1 and time node n accumulates to a certain number, that is, when the ratio of the accumulated number of self-label data to the number of unlabeled data exceeds a preset threshold, the self-label data set 1-n is annotated using the classification model annotation data automatic generation module 10 based on the self-label data set 1-n to obtain annotated data with new labels. The annotated data with new labels is used to train the current multi-class classification model to obtain an updated multi-class classification model 1. Within the time range from time node n+1 to time node n+m, for each new data set at each time node, the corresponding class label data and self-label data are output through the multi-label automatic classification module 30 based on the multi-category classification model 1, and then the above process is repeated to train the multi-category classification model and update the multi-category classification model. The update of the multi-category classification model is continuous. In each subsequent preset time range, if the ratio of the number of self-label data to the number of unlabeled data exceeds the preset threshold, the multi-category classification model is updated.

[0086] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0087] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0088] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an event multi-label classification device with time series characteristics.

[0089] refer to Figure 6 The multi-label classification device for events with time series features comprises:

[0090] The embedding module 602 is configured to obtain a plurality of current unlabeled events, input the plurality of unlabeled events into a current embedding model, and output a first vector representation corresponding to each of the unlabeled events via the current embedding model;

[0091] A clustering module 604 is configured to cluster all the first vector representations using a clustering algorithm to obtain labeled data including a plurality of first category labels and first unlabeled data;

[0092] The tagging module 606 is configured to, for each first category label, use the attention mechanism of the embedding model to perform word tagging on all labeled data corresponding to the first category label to obtain a word label corresponding to the first category label; and use the attention mechanism of the embedding model to perform word tagging on the first unlabeled data to obtain a self-label corresponding to the first unlabeled data;

[0093] The determination module 608 is configured to determine a classification label for each of the unlabeled events based on the word label and the self-label.

[0094] In some embodiments, the tagging module 606 is further configured to, for each sentence in all the labeled data corresponding to the first category label, use the attention mechanism of the embedding model to obtain an attention value of each word in the sentence, select the words corresponding to the first N attention values in descending order as a candidate word set for the sentence; determine a candidate keyword set corresponding to the sentence based on the candidate word set, where N is a positive integer;

[0095] Constructing a keyword frequency table based on a set of candidate keywords corresponding to all sentences in all labeled data corresponding to the first category label;

[0096] The keywords corresponding to the first M word frequencies are selected from the keyword word frequency table in descending order of word frequency values as word labels corresponding to the first category label, where M is a positive integer.

[0097] In some embodiments, the labeling module 606 is further configured to, for each sentence in the first unlabeled data, use the attention mechanism of the embedding model to obtain the attention value of each word in the sentence, and determine the self-label of the sentence based on all the attention values.

[0098] In some embodiments, the determination module 608 is further configured to, in response to the unlabeled event being the labeled data, select at least one keyword from the word labels of the first category label corresponding to the labeled data as a classification label for the unlabeled event; or

[0099] In response to the unlabeled event being the first unlabeled data, the self-label corresponding to the first unlabeled data is used as a classification label for the unlabeled event.

[0100] In some embodiments, an update module 610 is further included, which is configured to train the current classification model based on all the first unlabeled data in response to the ratio of the number of the first unlabeled data to the number of the unlabeled events exceeding a first preset threshold within a preset time interval to obtain an updated current classification model, wherein the current classification model includes the current embedding model and a classification layer superimposed on the current embedding model.

[0101] In some embodiments, the updating module 610 is further configured to cluster all the first unlabeled data using a clustering algorithm to obtain labeled data including multiple second category labels;

[0102] Determining training sample data based on the labeled data including multiple second category labels and a second preset threshold;

[0103] The current classification model is trained based on the training sample data to obtain an updated current classification model.

[0104] In some embodiments, the updating module 610 is further configured to use the labeled data whose probability value exceeds the probability threshold and whose corresponding second category label persistence exceeds the persistence threshold as the training sample data.

[0105] In some embodiments, before clustering all the first vector representations using a clustering algorithm, the method includes: performing dimensionality reduction processing on the first vector representations using a dimensionality reduction algorithm.

[0106] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0107] The device of the above embodiment is used to implement the corresponding multi-label classification method of events with time series characteristics in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0108] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and run on the processor. When the processor executes the program, it implements the multi-label classification method for events with time series characteristics described in any of the above embodiments.

[0109] Figure 7 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0110] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0111] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0112] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0113] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0114] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0115] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0116] The electronic device of the above embodiment is used to implement the corresponding multi-label classification method of events with time series characteristics in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0117] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the multi-label classification method for events with time series characteristics as described in any of the above embodiments.

[0118] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0119] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the multi-label classification method for events with time series characteristics as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0120] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0121] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0122] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0123] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A multi-label classification method for events with temporal features, characterized in that: include: Obtaining a plurality of current unlabeled events, inputting the plurality of unlabeled events into a current embedding model, and outputting a first vector representation corresponding to each of the unlabeled events via the current embedding model; Clustering all the first vector representations using a clustering algorithm to obtain labeled data including a plurality of first category labels and first unlabeled data; For each first category label, use the attention mechanism of the embedding model to perform word labeling on all labeled data corresponding to the first category label to obtain a word label corresponding to the first category label; and, using the attention mechanism of the embedding model to perform word tagging on the first unlabeled data to obtain self-labels corresponding to the first unlabeled data; determining a classification label for each of the unlabeled events based on the word label and the self-label; In response to a ratio of the amount of the first unlabeled data to the number of the unlabeled events exceeding a first preset threshold within a preset time interval, training the current classification model based on all of the first unlabeled data to obtain an updated current classification model, including: Clustering all of the first unlabeled data using a clustering algorithm to obtain labeled data containing multiple second category labels; determining training sample data based on the labeled data containing multiple second category labels and a second preset threshold; and training the current classification model based on the training sample data to obtain an updated current classification model; The current classification model includes the current embedding model and a classification layer superimposed on the current embedding model.

2. The method according to claim 1, characterized in that The use of the attention mechanism of the embedding model to perform word tagging on all labeled data corresponding to the first category label to obtain a word label corresponding to the first category label includes: For each sentence in all the labeled data corresponding to the first category label, using the attention mechanism of the embedding model to obtain an attention value for each word in the sentence, selecting words corresponding to the first N attention values in descending order as a set of candidate words for the sentence; determining a set of candidate keywords corresponding to the sentence based on the set of candidate words, where N is a positive integer; Constructing a keyword frequency table based on a set of candidate keywords corresponding to all sentences in all labeled data corresponding to the first category label; The keywords corresponding to the first M word frequencies are selected from the keyword word frequency table in descending order of word frequency values as word labels corresponding to the first category label, where M is a positive integer.

3. The method according to claim 1, characterized in that The step of using the attention mechanism of the embedding model to perform word tagging on the first unlabeled data to obtain self-labels corresponding to the first unlabeled data includes: For each sentence in the first unlabeled data, the attention mechanism of the embedding model is used to obtain an attention value of each word in the sentence, and a self-label of the sentence is determined based on all the attention values.

4. The method according to claim 1, wherein The determining of a classification label for each of the unlabeled events based on the word label and the self-label comprises: In response to the unlabeled event being the labeled data, selecting at least one keyword from the word labels of the first category label corresponding to the labeled data as a classification label for the unlabeled event; or, In response to the unlabeled event being the first unlabeled data, the self-label corresponding to the first unlabeled data is used as a classification label for the unlabeled event.

5. The method according to claim 1, wherein The second preset threshold includes a persistence threshold and a probability threshold, and determining the training sample data based on the labeled data including multiple second category labels and the second preset threshold includes: The labeled data whose probability value exceeds the probability threshold and whose corresponding second category label persistence exceeds the persistence threshold is used as the training sample data.

6. The method according to claim 1, characterized in that Before clustering all the first vector representations using a clustering algorithm, the method includes: A dimensionality reduction algorithm is used to perform dimensionality reduction processing on the first vector representation.

7. A multi-label classification device for events with time series characteristics, characterized in that: include: an embedding module configured to obtain a plurality of current unlabeled events, input the plurality of unlabeled events into a current embedding model, and output a first vector representation corresponding to each of the unlabeled events via the current embedding model; a clustering module configured to cluster all the first vector representations using a clustering algorithm to obtain labeled data including a plurality of first category labels and first unlabeled data; a labeling module configured to, for each first category label, use the attention mechanism of the embedding model to perform word labeling on all labeled data corresponding to the first category label to obtain a word label corresponding to the first category label; and, using the attention mechanism of the embedding model to perform word tagging on the first unlabeled data to obtain self-labels corresponding to the first unlabeled data; a determination module configured to determine a classification label for each of the unlabeled events based on the word label and the self-label; In response to a ratio of the amount of the first unlabeled data to the number of the unlabeled events exceeding a first preset threshold within a preset time interval, training the current classification model based on all of the first unlabeled data to obtain an updated current classification model, including: Clustering all of the first unlabeled data using a clustering algorithm to obtain labeled data containing multiple second category labels; determining training sample data based on the labeled data containing multiple second category labels and a second preset threshold; and training the current classification model based on the training sample data to obtain an updated current classification model; The current classification model includes the current embedding model and a classification layer superimposed on the current embedding model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images