Text clustering method, device, electronic device and storage medium

Abstract

The present application proposes a text clustering method, device, electronic device and storage medium, wherein the method comprises: determining similar events corresponding to the text in each event based on the acquired text to be processed and the corresponding entity features, and the acquired events in the event database and the corresponding entity features; determining the event described by the text based on the text and the corresponding similar events, and adding the text to the text set under the event described by the text. The method determines similar events corresponding to the text in each event based on the text to be processed and the corresponding entity features, and the events in the event database and the corresponding entity features, thereby determining the event described by the text and adding the text to the text set under the event described by the text, achieving the distinction between texts to be processed that are not much different, more accurately determining similar events corresponding to the text to be processed in the event database, and improving the accuracy of text clustering.

Description

Technical Field

[0001] The present application relates to the field of emergency response technology, and in particular to a text clustering method, device, electronic device and storage medium. Background Art

[0002] At present, in the field of emergency response, there are problems such as difficulty in integrating text information on social networks and too low text value density, which makes it impossible for emergency response work to quickly explore the situation of emergencies from the Internet, such as the spread of events and popularity trends.

[0003] In the related technology, the above problems are mainly solved by text clustering methods such as K-Means (K-means) clustering, density-based clustering method (DBSCAN), and maximum expectation (EM) clustering of Gaussian mixture model (GMM). However, the clustering models of the above text clustering methods are unsupervised clustering models, which do not attach great importance to the entity features of the training set text, resulting in that texts with little difference are not easy to distinguish and the clustering accuracy is low. Summary of the invention

[0004] The purpose of this application is to solve one of the above technical problems at least to a certain extent.

[0005] To this end, the first purpose of the present application is to propose a text clustering method, which determines similar events corresponding to the text in each event based on the text and corresponding entity features, and each event in an event database and its corresponding entity features, thereby determining the event described by the text and adding the text to the text collection described by the text, thereby achieving the distinction between texts to be processed that are not much different, more accurately determining similar events corresponding to the text to be processed in the event database, and improving the accuracy of text clustering.

[0006] The second objective of this application is to provide a text clustering device.

[0007] The third objective of the present application is to provide an electronic device.

[0008] A fourth objective of the present application is to provide a computer-readable storage medium.

[0009] To achieve the above-mentioned purpose, the first aspect of the present application proposes a text clustering method, including: obtaining a text to be processed and corresponding entity features; obtaining an event database, wherein the event database includes: each event and corresponding entity features, and a text collection under each event; determining similar events corresponding to the text in each event based on the text and corresponding entity features, the events and corresponding entity features; determining the event described by the text based on the text and the corresponding similar events; and adding the text to the text collection under the event described by the text.

[0010] The text clustering method of the embodiment of the present application obtains the text to be processed and the corresponding entity features; obtains an event database, wherein the event database includes: each event and the corresponding entity features, and a text set under each event; determines similar events corresponding to the text in each event based on the text and the corresponding entity features, each event and the corresponding entity features; determines the event described by the text based on the text and the corresponding similar events; and adds the text to the text set under the event described by the text. The method determines similar events corresponding to the text in each event based on the text and the corresponding entity features, each event in the event database and the corresponding entity features, thereby determining the event described by the text and adding the text to the text set described by the text, thereby achieving the distinction between texts to be processed that are not much different, more accurately determining similar events corresponding to the text to be processed in the event database, and improving the accuracy of text clustering.

[0011] To achieve the above-mentioned purpose, the second aspect of the present application proposes a text clustering device, including: a first acquisition module, used to acquire the text to be processed and the corresponding entity features; a second acquisition module, used to acquire an event database, wherein the event database includes: each event and the corresponding entity features, and a text collection under the each event; a first determination module, used to determine similar events corresponding to the text in each event based on the text and the corresponding entity features, the each event and the corresponding entity features; a second determination module, used to determine the event described by the text based on the text and the corresponding similar events; and an adding module, used to add the text to the text collection under the event described by the text.

[0012] The text clustering device of the embodiment of the present application obtains the text to be processed and the corresponding entity features; obtains an event database, wherein the event database includes: each event and the corresponding entity features, and a text set under each event; determines similar events corresponding to the text in each event according to the text and the corresponding entity features, the events and the corresponding entity features; determines the event described by the text according to the text and the corresponding similar events, and adds the text to the text set under the event described by the text. The device can determine similar events corresponding to the text in each event according to the text and the corresponding entity features, the events in the event database and the corresponding entity features, thereby determining the event described by the text and adding the text to the text set described by the text, achieving the distinction between texts to be processed that are not much different, more accurately determining similar events corresponding to the text to be processed in the event database, and improving the accuracy of text clustering.

[0013] To achieve the above-mentioned purpose, the third aspect embodiment of the present application proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the text clustering method as described above when executing the program.

[0014] In order to achieve the above-mentioned purpose, the fourth aspect of the present application proposes a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the text clustering method as described above is implemented.

[0015] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0017] Figure 1 A schematic diagram of a process of a text clustering method according to an embodiment of the present application;

[0018] Figure 2 A schematic diagram of determining the categorical relationship between a text and the most similar event using a long short-term memory neural network model according to an embodiment of the present application;

[0019] Figure 3 A schematic diagram of a flow chart of a text clustering method according to another embodiment of the present application;

[0020] Figure 4A schematic diagram of a structure for obtaining a coefficient of variation between text and events through a preset guided clustering model according to an embodiment of the present application;

[0021] Figure 5 A schematic diagram of the structure of a text clustering device provided in an embodiment of the present application;

[0022] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0024] The text clustering method, device, electronic device, and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.

[0025] Figure 1 The following is a flow chart of a text clustering method provided in an embodiment of the present application. Figure 1 As shown, it mainly includes the following steps:

[0026] Step 101, obtaining the text to be processed and the corresponding entity features.

[0027] In an embodiment of the present application, the text to be processed can be downloaded via the Internet, or obtained by accessing network platform data corresponding to the emergency event (such as an earthquake network platform).

[0028] It should be noted that the entity features corresponding to the text to be processed may include at least one of the following features: event time, event location and event type.

[0029] In the embodiment of the present application, the entity features corresponding to the text to be processed are different, and the methods of obtaining the corresponding entity features are also different.

[0030] As an example, when the entity features corresponding to the text to be processed include event time, the event time in the text can be extracted by inputting the text to be processed into a preset event time extraction model.

[0031] For example, the text to be processed can be input into a transformer model (such as a BERT model) to extract the event time in the text.

[0032] As another example, when the entity features corresponding to the text to be processed include the location of an event, the text can be segmented to obtain each word in the text; each word in the text is matched with a location in a preset location dictionary to obtain the location matching the word in the text; and the matched location is determined as the event location corresponding to the text.

[0033] That is to say, when the entity features corresponding to the text to be processed include the location of the event, the text can be segmented using a preset segmentation algorithm (such as a dictionary-based segmentation algorithm) to obtain each word in the text; then, each word in the text is matched with a location in a preset location dictionary, and the location that matches the word in the text is obtained, and the obtained location that matches the word in the text is used as the event location corresponding to the text.

[0034] As another example, when the entity features corresponding to the text to be processed include time types, the text is segmented to obtain each word in the text; the words associated with each event type in each word are obtained; the weights of each event type are determined based on the words associated with each event type in each word and the weights of the associated words; and the event type corresponding to the text is determined based on the weights of each event type.

[0035] That is to say, when the entity features corresponding to the text to be processed include time types, the text can be segmented by a preset segmentation algorithm (such as a segmentation algorithm based on a dictionary) to obtain each word in the text; and some words in the text that can represent event types are used as words associated with each event type. Then, according to the words associated with each event type in each word, and the weight of the associated words, the weight of each event type is determined. For example, the weight of each associated word can be set according to the degree of association between the associated words and the event type. For example, the higher the degree of association between the associated words and the event type, the greater the weight of the corresponding associated words; then, for each event type, the weights of the words associated with the event type are summed, and the summed result is used as the weight of the corresponding event type. Afterwards, according to the weights of each event type, the event type corresponding to the text is determined. For example, the weights of each event type can be compared, and the corresponding event type with a larger weight can be used as the event type corresponding to the text. Among them, it should be noted that the event type can be a natural disaster event, an accident disaster event, a public health event, a social security event, etc., and the event type can be classified according to actual needs, and this application does not make specific restrictions.

[0036] Step 102: Acquire an event database, wherein the event database includes: each event and corresponding entity features, and a text collection under each event.

[0037] In the embodiment of the present application, an event database may be pre-set. In order to ensure that the event database information is more comprehensive, the event database may include various events and corresponding entity features. Among them, the entity features corresponding to each event can be obtained by extracting the event time, event location, and event type of the event. In addition, the event database may also include a text set under each event, and the entity features corresponding to each event can also be selected by selecting the entity features corresponding to each text in the text set under each event, and the selected entity features are used as the entity features corresponding to the event.

[0038] Step 103: Determine similar events corresponding to the text in each event based on the text and the corresponding entity features, and each event and the corresponding entity features.

[0039] Optionally, the text and the corresponding entity features, each event and the corresponding entity features can be input into a preset guided clustering model to obtain the probability that the text belongs to each event; based on the probability that the text belongs to each event, similar events corresponding to the text in each event are determined. For details, please refer to the description of the subsequent embodiments.

[0040] Step 104: Determine the event described in the text based on the text and corresponding similar events.

[0041] Step 105: Add the text to the text collection under the event described by the text.

[0042] In an embodiment of the present application, text and corresponding similar events can be input into a long short-term memory neural network model to determine the probability that the similar event is the event described by the text; when the probability is greater than a preset probability threshold, the similar event is determined to be the event described by the text, and the text is added to the text collection under the event described by the text; when the probability is less than or equal to the preset probability threshold, a new event is generated based on the text, and the new event is updated to the event database.

[0043] That is to say, Figure 2 As shown, the text to be processed and the corresponding similar events can be input into a pre-trained long short-term memory neural network model. The long short-term memory neural network model can determine the probability that the similar event is the event described by the text to be processed. When the probability is greater than a preset probability threshold, the long short-term memory neural network model can output that the similar event is the event described by the text to be processed, and add the text to be processed to the text set under the event described by the text. When the probability is less than or equal to the preset probability threshold, the long short-term memory neural network model can generate a new event based on the text to be processed, and output that the text to be processed is a new event, and update the new event to the event database.

[0044] To better illustrate the above embodiment, the text clustering method of the embodiment of the present application can distinguish texts with little difference, more accurately determine similar events corresponding to the text to be processed in the event database, and improve the accuracy of text clustering.

[0045] In summary, based on the text and the corresponding entity features, and the events in the event database and the corresponding entity features, similar events corresponding to the text in each event are determined, thereby determining the event described by the text, and adding the text to the text set described by the text, thereby achieving the distinction between texts to be processed that are not much different, more accurately determining similar events in the event database corresponding to the text to be processed, and improving the accuracy of text clustering.

[0046] In order to improve the accuracy of text clustering, in the embodiment of the present application, Figure 3 As shown, Figure 3 A flowchart of another text clustering method provided in an embodiment of the present application. The text and the corresponding entity features, each event and the corresponding entity features can be input into a preset guided clustering model to obtain the probability that the text belongs to each event; based on the probability that the text belongs to each event, similar events corresponding to the text in each event are determined. Figure 1 Step 103 of the embodiment may include the following steps:

[0047] Step 401: input the text and the corresponding entity features, each event and the corresponding entity features into a preset guided clustering model to obtain the probability that the text belongs to each event.

[0048] In an embodiment of the present application, the text to be processed and the corresponding entity features, each event in the event database and the corresponding entity features can be input into a preset guided aggregation model, and the model can output the probability that the text to be processed belongs to each event. It should be noted that the preset guided aggregation model may include multiple regression network sub-models, and each regression network sub-model can be used to output a probability that the text belongs to each event.

[0049] Step 402: Determine similar events corresponding to the text in each event based on the probability that the text belongs to each event.

[0050] Optionally, for each event, multiple probabilities that the text belongs to the event are obtained; based on the multiple probabilities that the text belongs to the event, the average probability that the text belongs to the event is determined; based on the multiple probabilities that the text belongs to the event and the average probability, the coefficient of variation between the text and the event is determined; and the event with the smallest corresponding coefficient of variation is determined as a similar event corresponding to the text.

[0051] That is to say, since the preset guided clustering model may include multiple regression network sub-models, each regression network sub-model can be used to output a probability that the text belongs to each event. Therefore, for each event, each regression network sub-model in the preset guided clustering model can output a probability corresponding to the event that the text to be processed belongs to, thereby obtaining multiple probabilities that the text to be processed belongs to the event, then, the multiple probabilities are averaged, and the average value result is used as the average probability that the text to be processed belongs to the event; then, according to the multiple probabilities that the text to be processed belongs to the event and the average probability of belonging to the event, the sum of square differences between the multiple probabilities that the text to be processed belongs to the event and the average probability of belonging to the event is determined; according to the sum of square differences and the number of regression network sub-models in the preset guided clustering model, the coefficient of variation between the text to be processed and each event is calculated, and the event with the smallest coefficient of variation among the coefficients of variation between the text to be processed and each event is used as a similar event corresponding to the text to be processed. In this way, texts with little difference can be distinguished, similar events corresponding to the text to be processed in the event database can be more accurately determined, and the accuracy of text clustering can be improved.

[0052] For example, Figure 4 As shown, taking the preset guided aggregation model (Bagging integrated model) including regression network models A, B, and C as an example, the text to be processed and the corresponding entity features, each event in the event database and the corresponding entity features are input into the Bagging integrated model. For each event, the regression network model A outputs a probability Y1 that the text to be processed belongs to the event, the regression network model B outputs a probability Y2 that the text to be processed belongs to the event, and the regression network model C outputs a probability Y3 that the text to be processed belongs to the event. The average probability of the text to be processed belonging to the event is The coefficient of variation between the text to be processed and the event is Next, the coefficients of variation between the text to be processed and each event are compared respectively, and the event with the smallest coefficient of variation among the coefficients of variation between the text to be processed and each event is taken as a similar event corresponding to the text to be processed.

[0053] In summary, by inputting the text and the corresponding entity features, each event and the corresponding entity features into the preset guided clustering model, the probability that the text belongs to each event is obtained; according to the probability that the text belongs to each event, the similar events corresponding to the text in each event are determined. In this way, texts with little difference can be distinguished, and similar events corresponding to the text to be processed in the event database can be determined more accurately, thereby improving the accuracy of text clustering.

[0054] The text clustering method of the embodiment of the present application obtains the text to be processed and the corresponding entity features; obtains an event database, wherein the event database includes: each event and the corresponding entity features, and a text set under each event; determines similar events corresponding to the text in each event according to the text and the corresponding entity features, the events and the corresponding entity features; determines the event described by the text according to the text and the corresponding similar events, and adds the text to the text set under the event described by the text. The method determines similar events corresponding to the text in each event according to the text and the corresponding entity features, each event in the event database and the corresponding entity features, thereby determining the event described by the text and adding the text to the text set described by the text, achieving the distinction between texts to be processed that are not much different, more accurately determining similar events corresponding to the text to be processed in the event database, and improving the accuracy of text clustering.

[0055] Figure 5 The structure diagram of a text clustering device provided in the embodiment of the present application is shown in FIG. Figure 5 As shown, the text clustering device 600 includes: a first acquisition module 610 , a second acquisition module 620 , a first determination module 630 , a second determination module 640 , and an adding module 650 .

[0056] Among them, the first acquisition module 610 is used to obtain the text to be processed and the corresponding entity features; the second acquisition module 620 is used to obtain the event database, wherein the event database includes: each event and the corresponding entity features, and the text collection under each event; the first determination module 630 is used to determine the similar events corresponding to the text in each event based on the text and the corresponding entity features, each event and the corresponding entity features; the second determination module 640 is used to determine the event described by the text based on the text and the corresponding similar events; the adding module is used to add the text to the text collection under the event described by the text.

[0057] As a possible implementation method of an embodiment of the present application, the entity feature includes at least one of the following features: event time, event location and event type; the first acquisition module 610 is specifically used to input the text into a preset event time extraction model to extract the event time in the text; segment the text to obtain each word in the text; match each word in the text with a location in a preset location dictionary to obtain a location matching the word in the text; determine the matched location as the event location corresponding to the text; segment the text to obtain each word in the text; obtain the words associated with each event type in each word; for each event type, sum the weights of the words associated with the event type to obtain the weight of the event type; determine the event type corresponding to the text according to the weight of each event type.

[0058] As a possible implementation method of an embodiment of the present application, the first determination module 630 is specifically used to input the text and the corresponding entity features, each event and the corresponding entity features into a preset guided clustering model to obtain the probability that the text belongs to each event; based on the probability that the text belongs to each event, determine similar events corresponding to the text in each event.

[0059] As a possible implementation method of an embodiment of the present application, the guided clustering model includes: multiple regression network sub-models, each regression network sub-model is used to output a probability that the text belongs to each event; the first determination module 630 is also used to obtain multiple probabilities that the text belongs to the event for each event; based on the multiple probabilities that the text belongs to the event, determine the average probability that the text belongs to the event; based on the multiple probabilities that the text belongs to the event and the average probability, determine the coefficient of variation between the text and the event; and determine the event with the smallest corresponding coefficient of variation as a similar event corresponding to the text.

[0060] As a possible implementation method of an embodiment of the present application, the first determination module 630 is also used to determine the sum of squares of differences between multiple probabilities that the text belongs to the event and the average probability based on multiple probabilities that the text belongs to the event and the average probability; and calculate the coefficient of variation between the text and the event based on the sum of squares of differences and the number of regression network sub-models.

[0061] As a possible implementation method of an embodiment of the present application, the second determination module 640 is specifically used to input the text and the corresponding similar events into the long short-term memory neural network model to determine the probability that the similar event is the event described by the text; when the probability is greater than a preset probability threshold, determine that the similar event is the event described by the text; when the probability is less than or equal to the preset probability threshold, generate a new event based on the text, and update the new event to the event database.

[0062] The text clustering device of the embodiment of the present application obtains the text to be processed and the corresponding entity features; obtains an event database, wherein the event database includes: each event and the corresponding entity features, and a text set under each event; determines similar events corresponding to the text in each event according to the text and the corresponding entity features, each event and the corresponding entity features; determines the event described by the text according to the text and the corresponding similar events, and adds the text to the text set under the event described by the text. This method determines similar events corresponding to the text in each event according to the text and the corresponding entity features, each event in the event database and the corresponding entity features, thereby determining the event described by the text and adding the text to the text set described by the text, achieving the distinction between texts to be processed that are not much different, more accurately determining similar events corresponding to the text to be processed in the event database, and improving the accuracy of text clustering.

[0063] In order to implement the above embodiment, the present application also proposes an electronic device, Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes:

[0064] A memory 1001 , a processor 1002 , and a computer program stored in the memory 1001 and executable on the processor 1002 .

[0065] When the processor 1002 executes the program, the text clustering method provided in the above embodiment is implemented.

[0066] Furthermore, the electronic device further comprises:

[0067] The communication interface 1003 is used for communication between the memory 1001 and the processor 1002 .

[0068] The memory 1001 is used to store computer programs that can be executed on the processor 1002 .

[0069] The memory 1001 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0070] The processor 1002 is configured to implement the text clustering method described in the above embodiment when executing the program.

[0071] If the memory 1001, the processor 1002 and the communication interface 1003 are implemented independently, the communication interface 1003, the memory 1001 and the processor 1002 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0072] Optionally, in a specific implementation, if the memory 1001, the processor 1002 and the communication interface 1003 are integrated on a chip, the memory 1001, the processor 1002 and the communication interface 1003 can communicate with each other through an internal interface.

[0073] The processor 1002 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0074] In order to implement the above embodiment, the embodiment of the present application further proposes a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the text clustering method as in the above embodiment is implemented.

[0075] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0076] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0077] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0078] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0079] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0080] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0081] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0082] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A text clustering method, characterized in that: include: Acquire the text to be processed and the corresponding entity features, and determine the method for acquiring the entity features corresponding to the text according to the entity features corresponding to the text, wherein the entity features include at least one of the following features: event time, event location and event type; the method for acquiring the event time corresponding to the text is to input the text into a preset event time extraction model to extract the event time in the text; the method for acquiring the event location corresponding to the text is to segment the text to acquire each word in the text; match each word in the text with a location in a preset location dictionary to acquire a location matching the word in the text; determine the matched location as the event location corresponding to the text; the method for acquiring the event type corresponding to the text is to segment the text to acquire each word in the text; acquire the words associated with each event type in the words; for each event type, sum the weights of the words associated with the event type to obtain the weight of the event type; Determining the event type corresponding to the text according to the weights of the various event types; Acquire an event database, wherein the event database includes: each event and corresponding entity features, and a text set under each event; Determine similar events corresponding to the text among the events according to the text and the corresponding entity features, and the events and the corresponding entity features; Determine the event described by the text according to the text and the corresponding similar events; The text is added to the text collection under the event described by the text.

2. The method according to claim 1, characterized in that Determining similar events corresponding to the text among the events according to the text and the corresponding entity features, the events and the corresponding entity features, includes: Input the text and the corresponding entity features, the events and the corresponding entity features into a preset guided clustering model to obtain the probability that the text belongs to each event; According to the probability that the text belongs to each event, similar events corresponding to the text are determined among the events.

3. The method according to claim 2, characterized in that The guided aggregation model includes: a plurality of regression network sub-models, each regression network sub-model is used to output a probability that the text belongs to each event; The determining, according to the probability that the text belongs to each event, similar events corresponding to the text in each event includes: For each event, obtaining multiple probabilities that the text belongs to the event; Determining an average probability that the text belongs to the event based on multiple probabilities that the text belongs to the event; Determining a coefficient of variation between the text and the event based on a plurality of probabilities that the text belongs to the event and the average probability; The event with the smallest corresponding coefficient of variation is determined as a similar event corresponding to the text.

4. The method according to claim 3, characterized in that The step of determining the coefficient of variation between the text and the event according to the multiple probabilities that the text belongs to the event and the average probability includes: According to the multiple probabilities that the text belongs to the event and the average probability, determining the sum of squares of differences between the multiple probabilities that the text belongs to the event and the average probability; The coefficient of variation between the text and the event is calculated based on the sum of squares and the number of regression network sub-models.

5. The method according to claim 1, characterized in that Determining the event described by the text according to the text and the corresponding similar events includes: Inputting the text and the corresponding similar events into a long short-term memory neural network model to determine the probability that the similar event is the event described by the text; When the probability is greater than a preset probability threshold, determining that the similar event is the event described in the text; When the probability is less than or equal to a preset probability threshold, a new event is generated according to the text, and the new event is updated to the event database.

6. A text clustering device, characterized in that: include: The first acquisition module is used to acquire the text to be processed and the corresponding entity features, and determine the acquisition method of the entity features corresponding to the text according to the entity features corresponding to the text, wherein the entity features include at least one of the following features: event time, event location and event type; the event time corresponding to the text is acquired by inputting the text into a preset event time extraction model to extract the event time in the text; the event location corresponding to the text is acquired by segmenting the text to acquire each word in the text; each word in the text is matched with a location in a preset location dictionary to acquire a location matching the word in the text; the matched location is determined as the event location corresponding to the text; the event type corresponding to the text is acquired by segmenting the text to acquire each word in the text; the words associated with each event type are acquired from each word; for each event type, the weights of the words associated with the event type are summed to obtain the weight of the event type; Determining the event type corresponding to the text according to the weights of the various event types; A second acquisition module is used to acquire an event database, wherein the event database includes: each event and a corresponding entity feature, and a text set under each event; A first determination module, configured to determine similar events corresponding to the text among the events according to the text and the corresponding entity features, the events and the corresponding entity features; A second determination module, configured to determine the event described by the text according to the text and the corresponding similar events; The adding module is used to add the text to the text collection under the event described by the text.

7. The device according to claim 6, characterized in that The first determination module is specifically configured to: Input the text and the corresponding entity features, the events and the corresponding entity features into a preset guided clustering model to obtain the probability that the text belongs to each event; According to the probability that the text belongs to each event, similar events corresponding to the text are determined among the events.

8. The device according to claim 7, characterized in that The guided aggregation model includes: a plurality of regression network sub-models, each regression network sub-model is used to output a probability that the text belongs to each event; The first determining module is further configured to: For each event, obtaining multiple probabilities that the text belongs to the event; Determining an average probability that the text belongs to the event based on multiple probabilities that the text belongs to the event; Determining a coefficient of variation between the text and the event based on a plurality of probabilities that the text belongs to the event and the average probability; The event with the smallest corresponding coefficient of variation is determined as a similar event corresponding to the text.

9. The device according to claim 8, characterized in that The first determining module is further configured to: According to the multiple probabilities that the text belongs to the event and the average probability, determining the sum of squares of differences between the multiple probabilities that the text belongs to the event and the average probability; The coefficient of variation between the text and the event is calculated based on the sum of squares and the number of regression network sub-models.

10. The device according to claim 6, characterized in that The second determination module is specifically configured to: Inputting the text and the corresponding similar events into a long short-term memory neural network model to determine the probability that the similar event is the event described by the text; When the probability is greater than a preset probability threshold, determining that the similar event is the event described in the text; When the probability is less than or equal to a preset probability threshold, a new event is generated according to the text, and the new event is updated to the event database.

11. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the text clustering method according to any one of claims 1 to 5 when executing the program.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the text clustering method as described in any one of claims 1 to 5 is implemented.

Images