Event identification method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an event identification method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a plurality of texts containing event information of an event as first texts; based on a preset clustering algorithm, determining a text of which the represented event belongs to a to-be-processed category from each first text, and taking the determined text as a second text; inputting each second text and the first cue word into a large language model to obtain a first topic viewpoint of the text of the to-be-processed category; wherein the first cue word is used for indicating the topic viewpoint of the text input by the large language model; and determining a text matched with the first theme viewpoint from the second texts by utilizing a large language model, and taking the text as a third text representing the to-be-processed event. Therefore, the identification efficiency of the to-be-processed event can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an event recognition method, device, electronic device and storage medium. Background Art

[0002] With the development of network technology, the public can upload their opinions and views on designated events on relevant platforms through the Internet. For example, relevant departments can establish a problem feedback platform for the public, and accordingly, the public can upload the information of the problems to be reflected to the problem feedback platform. The problem information to be reflected may be an event that occurred in a certain area. However, the information received by the platform may be massive, and there may be a large amount of information on repeated events in the received information, and each piece of information may also contain invalid content (such as garbled characters) that is irrelevant to the event that the user needs to reflect. In one method, the received texts can be statistically analyzed manually one by one, and the text representing the event to be processed can be determined.

[0003] However, manually identifying events to be processed is inefficient. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide an event recognition method, device, electronic device and storage medium to improve the efficiency of identifying events to be processed. The specific technical solution is as follows:

[0005] In a first aspect of an embodiment of the present application, a method for event recognition is first provided, the method comprising:

[0006] Acquire multiple texts containing event information of an event as first text;

[0007] Determine, based on a preset clustering algorithm, from each first text, a text representing an event that belongs to the category to be processed, as the second text;

[0008] Input each second text and the first prompt word into the large language model to obtain a first topic view of the text of the category to be processed; wherein the first prompt word is used to indicate the topic view of the input text generated by the large language model;

[0009] By using the large language model, texts matching the first subject viewpoint are determined from the second texts as third texts representing the event to be processed.

[0010] In some embodiments, the method of determining, from each first text based on a preset clustering algorithm, texts representing events belonging to a category to be processed as second texts includes:

[0011] Encode each first text to obtain a first embedding vector of each first text;

[0012] Clustering the first embedding vector based on a preset clustering algorithm to obtain at least one vector cluster;

[0013] Selecting a vector cluster belonging to the category to be processed from the obtained at least one vector cluster;

[0014] The first text represented by the first embedding vector contained in the selected vector cluster is used as the second text.

[0015] In some embodiments, before encoding each first text to obtain a first embedding vector for each first text, the method further includes:

[0016] Inputting the length average, standard deviation, and third prompt word of each first text into the large language model to obtain a first parameter for dimensionality reduction processing and a second parameter for clustering;

[0017] The first parameter includes at least one of the following: the dimension of the embedded vector after dimensionality reduction, the minimum distance between each embedded vector after dimensionality reduction, and the number of adjacent points when constructing the neighborhood after dimensionality reduction; the second parameter includes at least one of the following: the minimum number of vectors contained in a vector cluster, the minimum number of vectors in the neighborhood of the center of a vector cluster; the third prompt word is used to instruct the large language model to generate parameters for dimensionality reduction processing and parameters for clustering based on the length average and standard deviation of the input text;

[0018] The encoding of each first text to obtain a first embedding vector of each first text includes:

[0019] Encode each first text, and perform dimensionality reduction processing on the encoding result according to the first parameter to obtain a first embedding vector of each first text;

[0020] The clustering of the first embedding vector based on a preset clustering algorithm to obtain at least one vector cluster includes:

[0021] According to the second parameter, the first embedding vectors are clustered based on a preset clustering algorithm to obtain at least one vector cluster.

[0022] In some embodiments, using the large language model to determine, from each second text, a text matching the first subject viewpoint as a third text representing the event to be processed, includes:

[0023] For each second text, input the second text and the first prompt word into the large language model to obtain a second topic view of each second text;

[0024] Calculating the similarity between the first topic view and the second topic view of each second text;

[0025] If the calculated similarity is greater than a preset similarity threshold, the second text is determined as the third text representing the event to be processed.

[0026] In some embodiments, using the large language model to determine, from each second text, a text matching the first subject viewpoint as a third text representing the event to be processed, includes:

[0027] For each second text, the second text, the first topic viewpoint and the fourth prompt word are input into the large language model to obtain the matching degree of the second text; wherein the fourth prompt word is used to indicate the matching degree between the text generated by the large language model and the topic viewpoint; if the matching degree of the second text is greater than a preset matching degree threshold, the second text is determined as the third text representing the event to be processed;

[0028] and / or,

[0029] The step of obtaining a plurality of texts containing event information of an event as a first text includes:

[0030] Get multiple texts containing event information of events;

[0031] Preprocessing each acquired text separately to obtain multiple first texts; wherein the preprocessing includes at least one of the following: denoising, stop word filtering and stem extraction;

[0032] and / or,

[0033] The method further comprises:

[0034] Based on the occurrence time of the event represented by the third text, a time series analysis algorithm is used to obtain an analysis result of the occurrence period of the events in the category to be processed.

[0035] In some embodiments, after using the large language model to determine texts matching the first topic viewpoint from each second text as third texts representing the event to be processed, the method further includes:

[0036] Inputting the third text and the second prompt word into the large language model to obtain the title of the text of the category to be processed; wherein the second prompt word is used to instruct the large language model to generate the title of the input text;

[0037] and / or,

[0038] Inputting the third text and the fifth prompt word into the large language model to obtain the cause of the event in the category to be processed; wherein the fifth prompt word is used to indicate the cause of the event represented by the input text generated by the large language model;

[0039] and / or,

[0040] Inputting the third text and the sixth prompt word into the large language model to obtain the risk index of the event in the category to be processed; wherein the sixth prompt word is used to instruct the large language model to generate the risk index of the event represented by the input text, and the risk index of an event includes at least one of the following: the amount involved in the event, the number of people involved in the event, and the area affected by the event;

[0041] and / or,

[0042] The third text and the seventh prompt word are input into the large language model to obtain the risk level of the event in the category to be processed; wherein the seventh prompt word is used to indicate the risk level of the event represented by the input text generated by the large language model, and the risk level of an event is determined based on the risk indicator of the event.

[0043] According to a second aspect of the present application, there is provided an event recognition device, the device comprising:

[0044] A first text acquisition module, used to acquire multiple texts containing event information of an event as a first text;

[0045] A second text determination module, used to determine, from each first text based on a preset clustering algorithm, text representing an event belonging to a category to be processed, as the second text;

[0046] A topic viewpoint generation module, used for inputting each second text and the first prompt word into the large language model to obtain the first topic viewpoint of the text of the category to be processed; wherein the first prompt word is used to indicate the topic viewpoint of the input text generated by the large language model;

[0047] A matching module is used to use the large language model to determine, from each second text, a text that matches the first subject viewpoint as a third text representing the event to be processed.

[0048] In some embodiments, the second text determination module includes:

[0049] An encoding submodule, used for encoding each first text to obtain a first embedding vector of each first text;

[0050] A clustering submodule, configured to cluster the first embedding vector based on a preset clustering algorithm to obtain at least one vector cluster;

[0051] A vector cluster selection submodule, used for selecting a vector cluster belonging to a category to be processed from the at least one vector cluster obtained;

[0052] The second text determination submodule is used to use the first text represented by the first embedding vector included in the selected vector cluster as the second text.

[0053] In some embodiments, the apparatus further comprises:

[0054] A parameter acquisition module, used for inputting the length average, standard deviation, and third prompt word of each first text into the large language model before encoding each first text to obtain the first embedding vector of each first text, so as to obtain the first parameter for dimensionality reduction processing and the second parameter for clustering;

[0055] The first parameter includes at least one of the following: the dimension of the embedded vector after dimensionality reduction, the minimum distance between each embedded vector after dimensionality reduction, and the number of adjacent points when constructing the neighborhood after dimensionality reduction; the second parameter includes at least one of the following: the minimum number of vectors contained in a vector cluster, the minimum number of vectors in the neighborhood of the center of a vector cluster; the third prompt word is used to instruct the large language model to generate parameters for dimensionality reduction processing and parameters for clustering based on the length average and standard deviation of the input text;

[0056] The encoding submodule is specifically used for:

[0057] Encode each first text, and perform dimensionality reduction processing on the encoding result according to the first parameter to obtain a first embedding vector of each first text;

[0058] The clustering submodule is specifically used for:

[0059] According to the second parameter, the first embedding vectors are clustered based on a preset clustering algorithm to obtain at least one vector cluster.

[0060] In some embodiments, the matching module is specifically used to:

[0061] For each second text, input the second text and the first prompt word into the large language model to obtain a second topic view of each second text;

[0062] Calculating the similarity between the first topic view and the second topic view of each second text;

[0063] If the calculated similarity is greater than a preset similarity threshold, the second text is determined as the third text representing the event to be processed.

[0064] In some embodiments, the matching module is specifically used to:

[0065] For each second text, the second text, the first topic viewpoint and the fourth prompt word are input into the large language model to obtain the matching degree of the second text; wherein the fourth prompt word is used to indicate the matching degree between the text generated by the large language model and the topic viewpoint;

[0066] If the matching degree of the second text is greater than a preset matching degree threshold, the second text is determined as a third text representing the event to be processed;

[0067] and / or,

[0068] The first text acquisition module is specifically used for:

[0069] Get multiple texts containing event information of events;

[0070] Preprocessing each acquired text separately to obtain multiple first texts; wherein the preprocessing includes at least one of the following: denoising, stop word filtering and stem extraction;

[0071] and / or,

[0072] The device also includes:

[0073] The cycle analysis module is used to obtain the analysis result of the occurrence cycle of the events in the category to be processed based on the occurrence time of the events represented by the third text and using a time series analysis algorithm.

[0074] In some embodiments, the apparatus further comprises:

[0075] a title generation module, used to input the third text and the second prompt word into the large language model to obtain the title of the text of the category to be processed; wherein the second prompt word is used to instruct the large language model to generate the title of the input text;

[0076] and / or,

[0077] a cause acquisition module, for determining, by using the large language model, from each second text a text matching the first theme viewpoint as a third text representing the event to be processed, inputting the third text and a fifth prompt word into the large language model to obtain the cause of the event in the category to be processed; wherein the fifth prompt word is used to indicate the cause of the event represented by the input text generated by the large language model;

[0078] and / or,

[0079] A risk indicator acquisition module is used to, after using the large language model to determine the text matching the first theme viewpoint from each second text as the third text representing the event to be processed, input the third text and the sixth prompt word into the large language model to obtain the risk indicator of the event in the category to be processed; wherein the sixth prompt word is used to indicate that the large language model generates the risk indicator of the event represented by the input text, and the risk indicator of an event includes at least one of the following: the amount involved in the event, the number of people involved in the event, and the area affected by the event;

[0080] and / or,

[0081] A risk level acquisition module is used to input the third text and a seventh prompt word into the large language model after determining the text matching the first theme viewpoint from each second text using the large language model as the third text representing the event to be processed, so as to obtain the risk level of the event in the category to be processed; wherein the seventh prompt word is used to indicate the risk level of the event represented by the input text generated by the large language model, and the risk level of an event is determined based on the risk indicator of the event.

[0082] According to a third aspect of the embodiments of the present application, an electronic device is provided, including:

[0083] Memory, used to store computer programs;

[0084] The processor is used to implement any of the above-mentioned event recognition methods when executing the program stored in the memory.

[0085] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned event recognition methods is implemented.

[0086] According to another aspect of the embodiments of the present application, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the event recognition methods described above.

[0087] Beneficial effects of the embodiments of the present invention:

[0088] Based on the event recognition method provided by the embodiment of the present application, the first text may contain texts representing different events, and the categories to which different events belong may be different. Accordingly, the electronic device can determine from the first text that the represented event belongs to multiple texts (i.e., second texts) of the category to be processed, and input each second text and the prompt word (i.e., the first prompt word) for indicating the theme viewpoint of the input text generated by the large language model to the large language model. Accordingly, the large language model can output the theme viewpoint (i.e., the first theme viewpoint) of the text of the category to be processed. The first theme viewpoint of the text of the category to be processed can summarize and briefly describe the event represented by the text of the category to be processed. Therefore, after obtaining the first theme viewpoint of the text of the category to be processed, the electronic device can also use the large language model to determine the third text matching the first theme viewpoint from each second text. In other words, the third text with a higher frequency of occurrence of the represented event can be determined from the first text. Correspondingly, the event represented by the third text is also the event to be processed. In this way, the efficiency of identifying the event to be processed can be improved.

[0089] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0091] Figure 1 A first flow chart of the event identification method provided in the embodiment of the present application;

[0092] Figure 2 A second flow chart of the event identification method provided in the embodiment of the present application;

[0093] Figure 3 A third flow chart of the event identification method provided in the embodiment of the present application;

[0094] Figure 4 A schematic diagram of a clustering result obtained by clustering a first embedding vector of a first text provided in an embodiment of the present application;

[0095] Figure 5 A fourth flow chart of the event identification method provided in the embodiment of the present application;

[0096] Figure 6 A structural diagram of an event recognition device provided in an embodiment of the present application;

[0097] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0098] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0099] With the development of Internet technology, the public can upload their opinions and views on designated events on relevant platforms through the Internet.

[0100] For example, relevant departments (such as the city's traffic management department) can establish a problem feedback platform for the public, and accordingly, the public can upload information about the problems they need to reflect to the problem feedback platform. The problem information to be reflected can be an event that occurred in a certain area.

[0101] For another example, an online media may establish a feedback platform for netizens, and netizens may upload information about events that they wish the media to intervene in to the feedback platform.

[0102] However, the platform may receive a huge amount of information, and there may be a large amount of information about repeated events in the received information. Each piece of information may also contain invalid content (such as garbled characters) that is irrelevant to the event that the user needs to reflect. In one method, the received texts can be statistically analyzed one by one by humans, and the text representing the event can be determined. Subsequently, the users of the platform (such as the above-mentioned relevant departments or online media, etc.) can understand the event information of the pending event based on the text representing the pending event and handle the pending event.

[0103] However, manually identifying events is inefficient.

[0104] The embodiment of the present application provides an event recognition method, which can be applied to an electronic device. For example, the electronic device can be a server of the above platform.

[0105] See also Figure 1 , Figure 1 The first flow chart of the event identification method provided in the embodiment of the present application comprises the following steps:

[0106] S101: Acquire multiple texts containing event information of an event as first texts.

[0107] S102: Determine, based on a preset clustering algorithm, from each first text a text representing an event belonging to a category to be processed, as a second text.

[0108] S103: Input each second text and the first prompt word into the large language model to obtain the first topic view of the text of the category to be processed.

[0109] The first prompt word is used to indicate the topic viewpoint of the input text generated by the large language model.

[0110] S104: Using the large language model, determine the text that matches the first theme viewpoint from each second text as the third text representing the event to be processed.

[0111] Based on the event recognition method provided by the embodiment of the present application, the first text may contain texts representing different events, and the categories to which different events belong may be different. Accordingly, the electronic device can determine from the first text that the represented event belongs to multiple texts (i.e., second texts) of the category to be processed, and input each second text and the prompt word (i.e., the first prompt word) for indicating the theme viewpoint of the input text generated by the large language model to the large language model. Accordingly, the large language model can output the theme viewpoint (i.e., the first theme viewpoint) of the text of the category to be processed. The first theme viewpoint of the text of the category to be processed can summarize and briefly describe the event represented by the text of the category to be processed. Therefore, after obtaining the first theme viewpoint of the text of the category to be processed, the electronic device can also use the large language model to determine the third text matching the first theme viewpoint from each second text. In other words, a third text with a higher frequency of occurrence of the represented event can be determined from the first text based on a preset clustering algorithm. Accordingly, the event represented by the third text is also the event to be processed. In this way, the efficiency of identifying the event to be processed can be improved.

[0112] With respect to step S101, the electronic device may obtain multiple texts from the user (such as the above-mentioned community owner, netizen, etc.), wherein the content of each text is used to describe an event, that is, the text contains event information of the event.

[0113] The text containing the event information of the event obtained by the electronic device may be uploaded by the user (such as the above-mentioned community owner, Internet user, etc.) through the above-mentioned platform, or may be collected in advance through grid investigation.

[0114] For example, the electronic device may obtain 100,000 original texts uploaded by users from January 2022 to December 2022. Take 23 original texts as examples, for example, original text 1 is: "The setting of the traffic light at the intersection of C Avenue and D Road is unreasonable." Original text 2 is: "The elevator in Community A is out of order." Original text 3 is: "The garbage classification in Community A is not implemented effectively." Original text 4 is: "The water supply in Community B is unstable." Original text 5 is: "Public transportation vehicles on Road N1 are delayed." Original text 6 is: "The construction site near Community A is noisy in the middle of the night." Original text 7 is: "Insufficient parking spaces in Shopping Mall A." Original text 8 is: text content 8. Original text 9 is: text content 9. Original text 10 is: text content 10. Original text 11 is: text content 11. Original text 12 is: text content 12. Original text 13 is: text content 13. Original text 14 is: text content 14. Original text 15 is: text content 15. Original text 16 is: text content 16. Original text 17 is: text content 17. Original text 18 is: text content 18. Original text 19 is: text content 19. Original text 20 is: text content 20. Original text 21 is: text content 21. Original text 22 is: text content 22. Original text 23 is: text content 23. Among them, one text content represents event information of one event.

[0115] In one implementation, the electronic device may directly use the acquired multiple original texts as the first text.

[0116] In some embodiments, the electronic device may also pre-process the multiple acquired original texts respectively to improve the quality of the obtained first text. Figure 2 , Figure 2 A second flow chart of the event recognition method provided in the embodiment of the present application, in Figure 1 On the basis of, step S101 comprises:

[0117] S1011: Acquire multiple texts containing event information of events.

[0118] S1012: Preprocess each acquired text respectively to obtain multiple first texts.

[0119] The preprocessing includes at least one of the following: denoising, stop word filtering and stemming.

[0120] In an embodiment of the present application, after obtaining multiple texts (i.e., original texts) containing event information of an event, the electronic device can preprocess each of the original texts obtained, and use the preprocessing results of each original text as the first text. The preprocessing includes at least one of the following: denoising, stop word filtering, and stem extraction.

[0121] In one implementation, a technician can preset various preprocessing operations required for each original text, as well as the execution order between the various preprocessing operations. In this application, the execution order between the various preprocessing operations is not limited.

[0122] For example, for each original text, the electronic device can first perform denoising processing on the original text, then perform stop word filtering on the obtained denoising result, and finally perform stemming on the obtained stop word filtering result to obtain the preprocessing result of the original text. For another example, for each original text, the electronic device can also first perform stop word filtering on the original text, then perform denoising on the obtained stop word filtering result, and finally perform stemming on the obtained denoising result to obtain the preprocessing result of the original text.

[0123] Correspondingly, the electronic device can perform denoising processing on the text to be denoised. That is, delete the noise words in the text. Among them, the noise words include: special characters, numbers, punctuation marks, etc. Among them, the text to be denoised can be the original text, or the text to be denoised can also be obtained by performing other preprocessing operations on the original text except for denoising processing.

[0124] In this way, the electronic device can clean the text, that is, delete the noise in the text. Since the noise in the text (i.e., special characters, numbers, punctuation marks, etc.) is usually irrelevant to the event represented by the text. Therefore, it is possible to reduce the proportion of invalid information in the obtained first text, and thus improve the quality of the first text. Furthermore, it is possible to improve the effect of subsequent classification.

[0125] The electronic device can perform stop word filtering on the text to be filtered. That is, delete the stop words in the text. Among them, the stop words can be words without actual meaning, and / or words that may affect the subsequent classification process. That is to say, the stop words are: words irrelevant to the event information of the event represented by the text. Correspondingly, a technician can preset stop words according to actual needs. For example, the stop words can be common articles, prepositions, conjunctions, etc., such as: "de", "shi", "zai", etc. Among them, the text to be filtered can be the original text, or the text to be filtered can also be obtained by performing other preprocessing operations on the original text except for stop word filtering processing.

[0126] In this way, the electronic device can remove the stop words in the text (i.e., common articles, prepositions, conjunctions, etc.), and remove the words that appear frequently in the text but are irrelevant to the event information of the event represented by the text. It is also possible to reduce the proportion of invalid information in the obtained first text, and thus improve the quality of the first text. Furthermore, reduce the computational complexity of subsequent clustering, and improve the efficiency and effect of subsequent event recognition to be processed.

[0127] The electronic device can perform stem extraction processing on the text to be standardized. That is, the text is segmented, and the continuous original text is divided into independent words or phrases to obtain the segmentation result of the text. Furthermore, the words with preset parts of speech can be extracted from the segmentation result of the text and represented in a unified text representation form as the stem extraction result of the text (also called the standardized result). For example, the electronic device can use a preset segmentation tool, such as the Jieba segmentation tool, to perform segmentation processing on the text and obtain the segmentation result of the text. The preset parts of speech can be nouns and verbs.

[0128] In this way, the electronic device can divide the text into the smallest semantic units (words). Since the nouns and verbs in the text are usually words with practical meanings, the original text is subjected to stem extraction processing to obtain the nouns and verbs in the text and represent them in a unified text representation form. This can further improve the quality of the first text and facilitate subsequent feature extraction and analysis.

[0129] For example, taking the order of denoising, stop word filtering and stem extraction as an example, the process of preprocessing the above original text 1 "The setting of the traffic light at the intersection of Avenue C and Road D is unreasonable" is: first, the original text 1 is denoised, and the denoising result obtained is "The setting of the traffic light at the intersection of Avenue C and Road D is unreasonable". Secondly, the stop word filtering result is performed on the obtained denoising result, and the stop word filtering result is "The setting of the traffic light at the intersection of Avenue C and Road D is unreasonable". Finally, the stop word filtering result is performed on the obtained stop word filtering result, and the preprocessing result of the original text is: "Avenue C, Road D, intersection, traffic light, setting, unreasonable". Among them, ", " in a preprocessing result is a word segmentation symbol, which is used to distinguish each word in the text, and the preprocessing result of each text actually obtained by the electronic device does not contain the word segmentation symbol.

[0130] Correspondingly, the electronic device can obtain the preprocessing result of each original text as the first text.

[0131] Based on the above processing, after obtaining each original text, the electronic device can perform pre-processing operations such as denoising, stop word filtering and stem extraction on each original text, and represent it in a unified text representation form. In this way, the proportion of invalid information in the first text obtained can be reduced, which can improve the quality of the first text, reduce the calculation amount of subsequent clustering, and improve the efficiency and effect of subsequent identification of pending events.

[0132] For step S102, the electronic device can cluster each first text according to the category to which the event represented by each first text belongs, and determine the first text whose represented event belongs to the category to be processed as the second text. Correspondingly, the events represented by other first texts in the first text except the second text do not belong to the category to be processed. Among them, the preset clustering algorithm can be the HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) algorithm, or the preset clustering algorithm can be the K-Means clustering algorithm.

[0133] The categories to be processed are selected from the categories to which the events represented by the original texts belong according to actual needs.

[0134] In some embodiments, each first text may be classified based on a preset topic modeling technology using text features (i.e., text embedding vectors) to determine the category to which the event represented by each first text belongs. The preset topic modeling technology may be: BERTopic, a topic modeling technology based on a pre-trained model BERT (Bidirectional Encoder Representations from Transformers, based on Transformer bidirectional encoder representations).

[0135] See also Figure 3 , Figure 3 A third flow chart of the event identification method provided in the embodiment of the present application, in Figure 1 On the basis of, step S102 comprises:

[0136] S1021: Encode each first text to obtain a first embedding vector for each first text.

[0137] S1022: Clustering the first embedding vector based on a preset clustering algorithm to obtain at least one vector cluster.

[0138] S1023: Select a vector cluster belonging to the category to be processed from the at least one obtained vector cluster.

[0139] S1024: Use the first text represented by the first embedded vector included in the selected vector cluster as the second text.

[0140] In an embodiment of the present application, for each first text, the electronic device may use a pre-trained language model to encode the first text to obtain an encoding result of the first text. The pre-trained language model may be BERT (Bidirectional Encoder Representations from Transformers), or the pre-trained language model may be TextCNN (Text Convolutional Neural Networks).

[0141] It can be understood that for a first text, the first text is composed of multiple words. Therefore, the electronic device can first perform word embedding processing, that is, convert the words in the first text into word vectors. Then, based on the word vectors corresponding to the first text, a vector representing the first text is constructed. That is, the electronic device can convert each first text into a high-dimensional vector for representation, and obtain an embedding vector of each first text. An embedding vector of a first text can represent the semantic information of the first text.

[0142] For example, for the original text 1, the first text corresponding to the original text 1 (which may be referred to as the first text 1) is: "C Avenue, D Road, intersection, traffic light, setting, unreasonable". Then, the electronic device may encode the first text 1 to obtain the encoding result of the first text 1, and the encoding result of the first text 1 is: [0.23, -0.11, ..., 0.76].

[0143] Similarly, for the original text 2 "An elevator in Community A has a malfunction", the first text corresponding to the original text 2 (which may be referred to as the first text 2) is: "Community A, elevator, malfunction". Furthermore, the electronic device may encode the first text 2 to obtain the encoding result of the first text 2, and the encoding result of the first text 2 is: [-0.34, 0.57, ..., -0.09].

[0144] Similarly, for the original text 3 "Community A's garbage classification is not implemented effectively", the first text (which can be called the first text 3) corresponding to the original text 2 is: "Community A, garbage, classification, implementation, ineffective". Then, the electronic device can encode the first text 3 to obtain the encoding result of the first text 3, and the encoding result of the first text 3 is: [0.12, 0.36, ..., -0.83]. By analogy, the electronic device can obtain the encoding result of each first text, and the specific acquisition process will not be repeated here.

[0145] In one implementation, after obtaining the encoding results of each first text, the electronic device can directly determine the encoding results of each first text as the first embedding vector of each first text. Accordingly, a first embedding vector of a first text can represent the semantic information of the first text.

[0146] In another implementation, the above step S1021 includes:

[0147] Each first text is encoded, and a dimension reduction process is performed on the encoding result according to the first parameter to obtain a first embedding vector of each first text.

[0148] In an embodiment of the present application, after obtaining the encoding result of each first text, for each encoding result of the first text, the electronic device can perform dimensionality reduction processing on the encoding result of the first text according to the first parameter based on a preset dimensionality reduction algorithm, and embed the high-dimensional embedding vector (i.e., the encoding result of the first text) into a low-dimensional feature space, thereby reducing the high-dimensional embedding vector to a lower dimension. Accordingly, the electronic device can determine the dimensionality reduction result corresponding to the encoding result of the first text as the first embedding vector of the first text. Accordingly, a first embedding vector of a first text can represent the semantic information of the first text. For example, the dimension of the first embedding vector can be two-dimensional or three-dimensional.

[0149] Among them, the preset dimensionality reduction algorithm can be: UMAP (Uniform Manifold Approximation and Projection) algorithm, or t-SNE (t-Distributed Stochastic Neighbor Embedding) algorithm.

[0150] The first parameter indicates: parameters used for dimensionality reduction. The first parameter includes at least one of the following: the dimension of the embedded vector after dimensionality reduction (which can be recorded as n_components), the minimum distance between each embedded vector after dimensionality reduction (which can be recorded as min_dist), and the number of neighboring points when constructing the neighborhood after dimensionality reduction (which can be recorded as n_neighbors).

[0151] Among them, the dimension of the embedded vector after dimensionality reduction (n_components) represents: the target dimension of the encoding result of each first text after dimensionality reduction. The minimum distance between each embedded vector after dimensionality reduction (min_dist) is used to control the minimum distance of data points in the low-dimensional feature space after dimensionality reduction, which can reflect the compactness of the vector cluster, where one data point corresponds to one first embedded vector. The number of neighboring points when constructing the neighborhood after dimensionality reduction (n_neighbors) is used to control the degree of preservation of the global structure. n_neighbors represents the number of neighbors considered for each data point when constructing the local neighborhood.

[0152] In one implementation, a technician may set the parameter values ​​of each first parameter in advance. For example, n_neighbors may be set to 15, n_components may be set to 2, and min_dist may be set to 0.1.

[0153] In another implementation, the electronic device can use the large language model to obtain the parameter value of the first parameter. The specific process of obtaining the first parameter will be described in the subsequent embodiments.

[0154] For example, for the encoding result of the first text 1 mentioned above (i.e., [0.23, -0.11, ..., 0.76]), the electronic device can obtain the dimensionality reduction result corresponding to the encoding result of the first text 1 based on a preset dimensionality reduction algorithm, as the first embedding vector of the first text 1. Similarly, for the encoding result of the first text 2 mentioned above (i.e., [-0.34, 0.57, ..., -0.09]), the electronic device can obtain the dimensionality reduction result corresponding to the encoding result of the first text 2 based on a preset dimensionality reduction algorithm, as the first embedding vector of the first text 2. By analogy, the electronic device can obtain the first embedding vector of each first text, and the specific acquisition process is not repeated here.

[0155] In this way, the electronic device can use a preset dimensionality reduction algorithm (such as UMAP) to perform dimensionality reduction processing on the encoding result of the first text, so that the dimension of the first embedding vector of each first text can be reduced, which can also reduce the amount of calculation and memory usage required for subsequent operations, reduce the computational complexity, and facilitate visualization and subsequent clustering analysis. In the subsequent clustering process, clustering using lower-dimensional data can improve the efficiency and effect of clustering.

[0156] For step S1022, after obtaining the first embedding vectors of each first text, the electronic device may cluster the first embedding vectors based on a preset clustering algorithm to obtain at least one vector cluster. The preset clustering algorithm may be a HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) algorithm, or the preset clustering algorithm may be a K-Means clustering algorithm.

[0157] Accordingly, since the first embedding vector of a first text can represent the semantic information of the first text, the electronic device can use a preset clustering algorithm to classify semantically similar text vectors (i.e., first embedding vectors) into the same vector cluster based on the distribution between the first embedding vectors of each first text. That is, in the feature space to which each first embedding vector belongs, a first embedding vector in a region with a higher density represents a first embedding vector in a vector cluster, and a first embedding vector in a region with a lower density represents noise.

[0158] It is understandable that in the process of clustering each first embedding vector based on the HDBSCAN algorithm, there is no need to pre-specify the number of vector clusters obtained. The HDBSCAN algorithm can automatically determine the number of vector clusters in the clustering result based on the density characteristics of the data (i.e., each first embedding vector). In this way, the method for identifying pending events provided in the embodiment of the present application is particularly suitable for application scenarios where the categories required to be divided into for the events represented by the first text are unknown.

[0159] In addition, if there is a first text that has a large semantic difference from other first texts, it can be marked as noise, that is, the HDBSCAN algorithm can effectively identify noise in the data, and thus can ensure the accuracy of the clustering results.

[0160] Thus, in the process of obtaining an embedding vector of a first text, the electronic device can convert a discrete, high-dimensional, sparse word vector representation into a continuous, low-dimensional, dense vector representation. Accordingly, the first embedding vector of each first text obtained can reflect the semantic relationship between words, which is convenient for subsequent clustering operations.

[0161] In some embodiments, step S1022 includes:

[0162] According to the second parameter, the first embedding vector is clustered based on a preset clustering algorithm to obtain at least one vector cluster.

[0163] In the embodiment of the present application, the second parameter represents: a parameter for clustering. The second parameter includes at least one of the following: the minimum number of vectors contained in a vector cluster (which can be recorded as min_cluster_size), the minimum number of vectors in the neighborhood of the center of a vector cluster (which can be recorded as min_samples), and the minimum distance between clusters (which can be recorded as cluster_selection_epsilon).

[0164] Among them, the minimum number of vectors contained in a vector cluster (min_cluster_size) indicates: the minimum number of data points required to form a vector cluster. Among them, one data point corresponds to one first embedded vector. It can be understood that min_cluster_size indicates the minimum size of a vector cluster. If the value of min_cluster_size is large, it may cause vector clusters containing fewer data points to be regarded as noise.

[0165] The minimum number of vectors in the neighborhood of the center of a vector cluster (min_samples), that is, the minimum number of neighbors around the center (i.e., the core point) of each vector cluster. min_samples is used to control the identification of noise in data points. Accordingly, if the value of min_samples is large, it means that the density of data points required to form a vector cluster is high, which may lead to an increase in the number of data points determined to be noise.

[0166] The minimum distance between clusters (cluster_selection_epsilon) is used to control the merging of vector clusters. That is, if the distance between two vector clusters is less than cluster_selection_epsilon, the two vector clusters will be merged.

[0167] In one implementation, a technician may set the parameter values ​​of each second parameter in advance. For example, min_cluster_size may be set to 5, min_samples may be set to none, and cluster_selection_epsilon may be set to 0.01.

[0168] In another implementation, the electronic device can use the large language model to obtain the parameter value of the second parameter. The specific process of obtaining the second parameter will be described in the subsequent embodiments.

[0169] Based on the above processing, in the process of clustering the first embedding vectors of each first text, the electronic device can control the clustering result by setting the parameters (i.e., the second parameters) for clustering. That is, the electronic device can control the number and size of vector clusters obtained based on the first embedding vectors of each first text according to actual needs. In this way, the flexibility of clustering can be guaranteed, and clustering can be performed according to actual needs.

[0170] For each vector cluster obtained according to the above steps S1021 to S1022, the electronic device may determine each first text represented by each first embedded vector included in the vector cluster as first text belonging to the same category.

[0171] That is, the electronic device can determine the classification result of the event represented by each first text according to the clustering result of the first embedding vector of each first text. Accordingly, the number and size of the vector clusters obtained in the clustering process are used to reflect: the classification granularity of classifying the event represented by the first text. That is, the technician can set the classification granularity when classifying the event by setting the clustering parameters.

[0172] It can be understood that, for the first text obtained, if the classification granularity of the event is finer, the number of vector clusters obtained by clustering is greater, that is, the number of categories to which the events represented by each first text belong is greater, and accordingly, the number of first embedded vectors contained in a vector cluster is smaller, and the number of first texts corresponding to the same category is smaller. If the classification granularity of the event is coarser, the number of vector clusters obtained by clustering is smaller, that is, the number of categories to which the events represented by each first text belong is smaller, and accordingly, the number of first embedded vectors contained in a vector cluster is larger, and the number of first texts corresponding to the same category is larger.

[0173] For example, taking the above-mentioned original text 1, original text 5, original text 9, original text 18, and original text 19 as examples, original text 1 is "The traffic light setting at the intersection of Bing Avenue and Ding Road is unreasonable"; original text 5 is "Public transportation vehicles on Line N1 are delayed"; original text 9 is text content 9; original text 18 is text content 18; original text 19 is text content 19.

[0174] In the case where the classification granularity of events is coarse, the first embedding vectors contained in a vector cluster (which can be recorded as vector cluster A) include: the first embedding vector of the first text (i.e., first text 1) corresponding to the original text 1, the first embedding vector of the first text (i.e., first text 5) corresponding to the original text 5, the first embedding vector of the first text (i.e., first text 9) corresponding to the original text 9, the first embedding vector of the first text (i.e., first text 18) corresponding to the original text 18, and the first embedding vector of the first text (i.e., first text 19) corresponding to the original text 19. At this time, the events represented by the first text 1, the first text 5, the first text 9, the first text 18, and the first text 19 belong to the same category (which can be recorded as: traffic).

[0175] For another example, when the classification granularity of events is relatively fine, the first embedding vectors contained in a vector cluster (which can be recorded as vector cluster B) include: the first embedding vector of the first text 1, the first embedding vector of the first text 5, the first embedding vector of the first text 9, and the first embedding vector of the first text 18. At this time, the events represented by the first text 1, the first text 5, the first text 9, and the first text 18 belong to the same category (which can be recorded as: city-transportation).

[0176] Accordingly, since one vector cluster obtained corresponds to one category, the electronic device can select a vector cluster from the at least one vector cluster obtained, and the category corresponding to the vector cluster is the category to be processed. Then, the first text represented by the first embedded vector contained in the selected vector cluster is used as the second text. That is, the second text is the first text representing the event belonging to the category to be processed.

[0177] In one implementation, for each vector cluster, the electronic device may determine the first first number of words (i.e., keywords) with the highest occurrence frequency in each first text corresponding to each first embedded vector included in the vector cluster to obtain a keyword set of the vector cluster. For example, the first number may be 2.

[0178] For example, when the first number is 2, for the above vector cluster A, in each first text corresponding to each first embedding vector contained in the vector cluster A, the top two words (i.e., keywords) with the highest frequency of occurrence are "road" and "congestion". Correspondingly, the keyword set of vector cluster A is: road-congestion.

[0179] In another implementation, before step S1023, the method further includes:

[0180] Step 1: For each vector cluster, based on a preset keyword extraction algorithm, multiple keywords are extracted from each first text corresponding to each first embedded vector contained in the vector cluster as a keyword set of the vector cluster.

[0181] Accordingly, step S1023 includes:

[0182] Based on the keyword set of each vector cluster, a vector cluster belonging to the category to be processed is selected from the at least one obtained vector cluster.

[0183] In the embodiment of the present application, a vector cluster corresponds to a category, that is, the electronic device can determine the first texts belonging to the same category based on a vector cluster. Accordingly, for each obtained vector cluster, the electronic device can extract multiple keywords from each first text corresponding to each first embedded vector contained in the vector cluster based on a preset keyword extraction algorithm as a keyword set of the vector cluster.

[0184] The preset keyword extraction algorithm may be: C-TF-IDF (Class-based Term Frequency-Inverse Document Frequency) algorithm. The C-TF-IDF algorithm is obtained by expanding the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm. The C-TF-IDF algorithm is used to extract representative keywords from each category of text.

[0185] In other words, the C-TF-IDF algorithm can identify the most representative words in the text of this category, not just the high-frequency words. In this way, the frequency of words in a specific category can be considered in the process of determining the keyword set of each vector cluster, which can reduce the interference of common words on the recognition results. If recognition is based on the TF-IDF algorithm, some words that frequently appear in multiple categories (i.e., common words) may be used as keywords. The C-TF-IDF algorithm suppresses the weight of common words by inverse category frequency, thereby improving the uniqueness and recognition of the keywords of each vector cluster extracted.

[0186] In one implementation, the above step 1 includes:

[0187] For each vector cluster, according to the third parameter and based on a preset keyword extraction algorithm, multiple keywords are extracted from each first text corresponding to each first embedded vector included in the vector cluster as a keyword set of the vector cluster.

[0188] In the embodiment of the present application, each first text corresponding to each first embedded vector contained in a vector cluster can be called a topic. Accordingly, a keyword set of a vector cluster is obtained, that is, a keyword set of the topic corresponding to the vector cluster is obtained.

[0189] The third parameter indicates: parameters for extracting keywords. The third parameter includes at least one of the following: the phrase range for extracting keywords (which can be recorded as n_gram_range), the number of keywords in the keyword set of each topic (which can be recorded as top_n_words), the minimum number of texts contained in each topic (which can be recorded as min_topic_size), the setting method of the number of topics (which can be recorded as nr_topics), and whether to use BM25 weighting for keyword extraction (which can be recorded as bm25_weighting).

[0190] Among them, the phrase range for extracting keywords (n_gram_range) is used to define the phrase range considered when extracting keywords. For example, if the value of the phrase range for extracting keywords is (1,2), it means that the extracted keywords can be single words and phrases consisting of two words. The number of keywords in the keyword set of each topic (top_n_words) indicates the number of keywords with higher weights extracted from each topic. The higher the weight of a keyword, the more important the keyword is relative to the topic. The minimum number of texts contained in each topic (min_topic_size) indicates the minimum value of the number of first texts contained in each topic, that is, the minimum size of the topic. When the setting method of the number of topics (nr_topics) is auto, it means that the number of topics is automatically optimized according to the data. When whether to use BM25 weights for keyword extraction (bm25_weighting) is False, it means that BM25 weights are not used. BM25 (Best Matching 25) is an information retrieval algorithm.

[0191] In one implementation, a technician may set the parameter values ​​of each third parameter in advance. For example, n_gram_range may be set to (1, 2), top_n_words may be set to 10, min_topic_size may be set to 10, nr_topics may be set to auto, and bm25_weighting may be set to False.

[0192] In another implementation, the electronic device can use the large language model to obtain the parameter value of the third parameter. The specific process of obtaining the third parameter will be described in the subsequent embodiments.

[0193] For example, for the above vector cluster A, the electronic device can extract multiple keywords from each first text corresponding to each first embedded vector contained in the vector cluster based on a preset keyword extraction algorithm as a keyword set of the vector cluster. For example, the keyword set of the vector cluster A obtained can be: road-congestion.

[0194] In this way, the electronic device can extract multiple keywords from each first text corresponding to each first embedded vector included in a vector cluster based on BERTopic as the keyword set of the vector cluster. In this way, the category corresponding to each vector cluster can be automatically determined.

[0195] Furthermore, after obtaining the keyword set of each vector cluster, the administrator of the relevant platform (such as the above-mentioned network media, community property) can select one from all vector clusters according to the keyword set of each vector cluster. Correspondingly, the electronic device can use the first text represented by the first embedded vector contained in the selected vector cluster as the second text.

[0196] Based on the above processing, the electronic device can process the first text to obtain the first embedding vector of each first text. The subtraction Chinese characters are converted into vectors for representation, and then the first embedding vectors can be clustered, so that the first text can be classified through the vector clusters obtained by clustering, and the text representing the event belonging to the category to be processed is determined as the second text.

[0197] See also Figure 4 , Figure 4 A schematic diagram of a clustering result obtained by clustering the first embedding vectors of the first text provided in an embodiment of the present application. If the number of the first texts is 21, that is, the number of the first embedding vectors is 21. Figure 4 In the figure, the area corresponding to each dotted circle corresponds to a vector cluster, and each black dot corresponds to the first embedding vector of a first text. The position of the black dot in the rectangular box indicates the position of the first embedding vector of the first text in the feature space. The first embedding vectors of the 21 first texts are clustered, and the clustering results are shown in Table 1.

[0198] Table 1

[0199]

[0200] Among them, a row of data in the table corresponds to a vector cluster, and the cluster number represents the number of the vector cluster. The text sequence number represents the sequence number of each first text corresponding to each first embedded vector in the vector cluster. The keyword represents the extracted keyword of the vector cluster.

[0201] The cluster number of the vector cluster represented by region 1 is 1. The vector cluster contains the first embedding vector of the first text with text numbers 1, 5 and 6. The extracted keyword of the vector cluster is transportation.

[0202] The cluster number of the vector cluster represented by region 2 is 2. The vector cluster contains the first embedding vector of the first text with text numbers 2, 7, 8 and 9. The keyword extracted from the vector cluster is environmental sanitation.

[0203] The cluster number of the vector cluster represented by region 3 is 3. The vector cluster contains the first embedding vector of the first text with text numbers 3, 10, 11, 14, 15 and 18. The extracted keyword of the vector cluster is public facilities.

[0204] The cluster number of the vector cluster represented by region 4 is 4. The vector cluster contains the first embedding vectors of the first texts with text numbers 4, 12, 13, 16, 17, 19 and 20. The extracted keyword of the vector cluster is food safety.

[0205] The vector cluster represented by region -1 has a cluster number of -1, and the vector cluster contains the first embedding vector of the first text with a text sequence number of 21. Since the first embedding vector is far away from other first embedding vectors, the first embedding vector can be determined as noise. Accordingly, the electronic device can set the keyword of the vector cluster as noise.

[0206] It can be understood that in the clustering results, the event represented by the first text corresponding to the first embedding vector determined to be noise has a low frequency of occurrence, and accordingly, the event corresponding to the noise is not an event to be processed in this application.

[0207] With respect to step S103, after obtaining the second text of the category to be processed, the electronic device can generate the theme viewpoint (i.e., the first theme viewpoint) of the second text of the category to be processed using the large language model. That is, the electronic device can input each second text and the first prompt word into the large language model (LLM).

[0208] Among them, the large language model can be ChatGPT (Chat Generative Pre-trained Transformer), QWEN (general meaning) model, etc. The large language model can learn the grammatical and semantic rules of natural language, and it can perform the corresponding text generation function according to the input prompt word. The prompt word refers to the initial text or instruction input into the model. In this application, the prompt word is an injected instruction input into the large language model, and the representation of the prompt word is not limited to words, but can also be sentences, pictures, URLs, etc.

[0209] The first prompt word is used to indicate the topic viewpoint of the input text generated by the large language model. For example, the first prompt word can be: "Please output the topic viewpoint of the multiple texts."

[0210] In one implementation, in the process of using the large language model to generate the first topic view of the second text of the category to be processed, the first prompt word may also include: content requirements for the first topic view.

[0211] For example, the content requirements for the first topic viewpoint may include at least one of the following: eliminating ambiguity and enhancing semantic association, etc.

[0212] Among them, eliminating ambiguity means: using a large language model to check possible ambiguity in the generated first topic opinion to ensure the clarity and accuracy of the generated first topic opinion.

[0213] Enhanced semantics includes: association expansion and context supplement. Association expansion means identifying the association between the first theme viewpoint and concepts such as events and people, and enriching the connotation of the first theme viewpoint. Context supplement means adding context information such as time, place, and reason to the generated first theme viewpoint to make the first theme viewpoint more complete.

[0214] The large language model can obtain the first topic viewpoint after deepening the semantic level through semantic analysis. In addition, according to actual needs, the required information (such as time, place, etc.) is added to the generated first topic viewpoint to enhance the depth and breadth of the first topic viewpoint.

[0215] Accordingly, the large language model can perform multi-angle analysis on the multiple second texts according to the actual needs represented by the first prompt word, generate and output the first theme view of the text of the category to be processed. Accordingly, the electronic device can obtain the semantically enhanced first theme view, the first theme view of the text of the category to be processed, and ensure the accuracy of subsequent analysis.

[0216] In this way, the large language model can conduct in-depth analysis of the text of the category to be processed and extract the first topic viewpoint that can represent the text of the category to be processed. In other words, the large language model can be used to automatically generate refined and accurate first topic viewpoints from a large amount of text, which can significantly improve the accuracy and richness of the description of the first topic viewpoint and provide more powerful support for subsequent processing.

[0217] For step S104, events represented by texts of the same category are the same event. The event represented by the second text is a to-be-processed event that is clustered and has a higher frequency of occurrence among all events represented by the first texts. It is understandable that in the second text of the to-be-processed category, there may be a small amount of text that has a low correlation with the first theme viewpoint or is of poor quality. Therefore, the electronic device can use the first theme viewpoint to screen the second text of the to-be-processed category to determine the text that matches the first theme viewpoint from each second text as the third text.

[0218] The electronic device may determine the third text matching the first theme viewpoint from each second text by using any one of the following methods 1 and 2.

[0219] Method 1:

[0220] In some embodiments, step S104 includes:

[0221] Step S1041: For each second text, the second text and the first prompt word are input into the large language model to obtain a second topic view of each second text.

[0222] Step S1042: Calculate the similarity between the first topic viewpoint and the second topic viewpoint of each second text.

[0223] Step S1043: If the calculated similarity is greater than a preset similarity threshold, the second text is determined as a third text representing the event to be processed.

[0224] In the embodiment of the present application, for each second text, the electronic device can use the large language model to generate the theme viewpoint of the second text (which can be called the second theme viewpoint). That is, the electronic device can input each second text and the first prompt word into the large language model. Correspondingly, the large language model can process the second text, generate and output the second theme viewpoint of the second text.

[0225] Furthermore, the electronic device may calculate the similarity between the first topic view and the second topic view of the second text.

[0226] For example, the electronic device can obtain the encoding result of the first topic viewpoint and the encoding result of the second topic viewpoint of the second text based on the above BERT (Bidirectional Encoder Representations from Transformers). Then, based on the distance between the encoding result of the first topic viewpoint and the encoding result of the second topic viewpoint of the second text, the similarity between the first topic viewpoint and the second topic viewpoint of the second text is calculated.

[0227] If the calculated similarity is greater than the preset similarity threshold, it indicates that the semantics of the second theme view representation of the second text is close to the semantics of the first theme view representation, which also indicates that the semantics of the second text representation is close to the semantics of the first theme view representation, that is, the second text matches the first theme view. Therefore, the electronic device can determine the second text as the third text.

[0228] If the calculated similarity is not greater than the preset similarity threshold, it indicates that the semantics of the second theme view representation of the second text is not close to the semantics of the first theme view representation, which also indicates that the semantics of the second text representation is not close to the semantics of the first theme view representation, that is, the second text does not match the first theme view. Therefore, the electronic device will not determine the second text as the third text.

[0229] Based on the above processing, the electronic device can use the large language model to generate the second theme view of each second text. Then, by matching the second theme view of each second text with the first theme view corresponding to the category to be processed, it is determined whether each second text matches the first theme view. In this way, the text with low relevance to the first theme view corresponding to the category to be processed or poor quality in the second text can be removed. This also improves the quality of the third text used to generate the title and improves the accuracy of the generated title.

[0230] Method 2:

[0231] In some embodiments, step S104 includes:

[0232] S104a: For each second text, the second text, the first topic viewpoint, and the fourth prompt word are input into the large language model to obtain a matching degree of the second text.

[0233] Among them, the fourth prompt word is used to indicate the matching degree between the text generated by the large language model and the topic viewpoint.

[0234] S104b: If the matching degree of the second text is greater than a preset matching degree threshold, the second text is determined as a third text representing the event to be processed.

[0235] In the embodiment of the present application, for each second text, the electronic device can use the large language model to determine the matching degree between the second text and the first topic view. That is, the electronic device can input the second text, the first topic view and the fourth prompt word into the large language model.

[0236] The fourth prompt word is used to indicate the matching degree between the text generated by the large language model and the first topic viewpoint. For example, the fourth prompt word may be: "Please output the matching degree between the text and the first topic viewpoint".

[0237] In one implementation, before using the large language model to generate the matching degree between each second text and the first topic viewpoint, the electronic device can also input into the large language model: multiple first sample texts, sample topic viewpoints, and labels of each first sample text, wherein the label of a first sample text represents the matching degree corresponding to the first sample text and the sample topic viewpoint.

[0238] In the embodiment of the present application, the multiple first sample texts include: first sample texts with high quality and high relevance to the sample theme and viewpoint, and first sample texts with low quality and low relevance to the sample theme and viewpoint. Accordingly, the matching degree of the label representation of the first sample text with high quality and high relevance to the sample theme and viewpoint is greater than the matching degree of the label representation of the first sample text with low quality and low relevance to the sample theme and viewpoint.

[0239] The quality of a text is determined based on the clarity of the text and the amount of information in the text.

[0240] The clarity of a text indicates how clearly and accurately the text expresses the events it represents. The amount of information a text contains indicates how much valuable information the text contains.

[0241] The relevance of a text to a topic indicates the degree to which the semantics of the text's content matches the semantics of the topic.

[0242] That is to say, the large language model can learn how to evaluate the matching degree between text and topic opinions from multiple dimensions (i.e., quality and relevance) based on multiple input first sample texts, sample topic opinions, and labels of each first sample text.

[0243] Accordingly, for each second text input into the large language model, the large language model may process the second text and the first topic viewpoint to generate a matching degree between the second text and the first topic viewpoint.

[0244] Furthermore, the electronic device can screen the second text based on a preset matching degree threshold. That is, if the matching degree of a second text is greater than the preset matching degree threshold, it indicates that the second text matches the first theme viewpoint, and the second text is suitable as a text for generating a title corresponding to the category to be processed, so the electronic device can determine the second text as the third text.

[0245] If the mismatch of a second text is greater than a preset matching threshold, it indicates that the second text is of poor quality or has a low relevance to the first theme viewpoint, and the second text is not suitable as the third text. For example, in the second text, the text of poor quality or low relevance to the first theme viewpoint may be: the text whose semantics is inconsistent with the semantics represented by the first theme viewpoint, or the text that is ambiguous and misleading.

[0246] Based on the above processing, the electronic device can use the large language model to verify the concept consistency, that is, use the large language model to verify each second text, determine whether the second text belongs to or effectively expresses the first theme viewpoint, and obtain the matching degree between the second text and the first theme viewpoint. Then, use the matching degree of each second text to filter out texts with low relevance to the first theme viewpoint or poor quality, so as to ensure the quality of the obtained third text.

[0247] In addition, by evaluating and screening the second text, the data quality can be improved, ensuring that the retained third text is highly relevant and of high quality, providing a reliable basis for subsequent processing. At the same time, filtering out texts with low relevance to the first topic or poor quality strengthens the consistency of the text in the third text and reduces the impact of noise on subsequent processing.

[0248] In some embodiments, after step S104, the method further includes:

[0249] The third text and the second prompt word are input into the large language model to obtain the title of the text of the category to be processed.

[0250] The second prompt word is used to instruct the large language model to generate a title for the input text.

[0251] In the embodiment of the present application, the electronic device can generate a title based on the third text as the title of the text of the category to be processed. That is, the electronic device can input the third text and the second prompt word into the large language model.

[0252] The second prompt word is used to instruct the large language model to generate the title of the input text. For example, the second prompt word may be: "Please output the titles of the multiple texts."

[0253] In one implementation, in the process of using the large language model to generate the title of the third text of the category to be processed, the second prompt word may also include: requirements for the title. For example, the requirements for the title may include a limit on the number of words in the title and content requirements represented by the title. For example, the content requirements represented by the title are used to limit: the title contains concepts such as the result of the event and characters.

[0254] Accordingly, the large language model can perform multi-angle analysis on the plurality of third texts according to the actual needs indicated by the second prompt word, generate and output the titles of the plurality of third texts. Accordingly, the electronic device can obtain the title as the title of the text of the category to be processed.

[0255] Based on the event recognition method provided by the present application, the electronic device can classify the acquired text based on the BERTopic technology. Since the BERTopic technology can realize model parameter adaptation, that is, automatically adjust the clustering parameters according to the data volume of the text, the clustering effect can be optimized and the clustering accuracy and stability can be improved.

[0256] Furthermore, the large language model is used to process texts of the same category (category to be processed) and generate titles for the texts of the category to be processed. The powerful semantic understanding ability of the large language model can be used to conduct in-depth semantic analysis of the acquired text, significantly improving the accuracy and efficiency of identifying events to be processed.

[0257] Compared with directly inputting all texts into a large language model and classifying and identifying events to be processed through the large language model, the event recognition method provided in the present application can improve the efficiency of identifying events to be processed and reduce the computational complexity of the large language model.

[0258] In addition, the event recognition method provided in the application introduces a concept verification and data screening link, that is, using a large language model to determine a third text that matches the first theme from each second text. In this way, high-quality, highly relevant data can be screened out, noise and irrelevant data can be eliminated, and the quality and consistency of the text used to generate the title can be guaranteed, thereby ensuring the accuracy of the obtained title.

[0259] In one embodiment, see Figure 5 , Figure 5 A fourth flow chart of the event recognition method provided in the embodiment of the present application, in Figure 3 On the basis of, before step S1021, the method further includes:

[0260] S1025: Inputting the length average, standard deviation, and third prompt word of each first text into the large language model to obtain a first parameter for dimensionality reduction processing and a second parameter for clustering.

[0261] Among them, the first parameter includes at least one of the following: the dimension of the embedded vector after dimensionality reduction, the minimum distance between each embedded vector after dimensionality reduction, and the number of adjacent points when constructing the neighborhood after dimensionality reduction; the second parameter includes at least one of the following: the minimum number of vectors contained in a vector cluster, the minimum number of vectors in the neighborhood of the center of a vector cluster; the third prompt word is used to instruct the large language model to generate parameters for dimensionality reduction processing and parameters for clustering based on the length mean and standard deviation of the input text.

[0262] Step S1021 includes:

[0263] S1021a: Encode each first text, and perform dimensionality reduction processing on the encoding result according to the first parameter to obtain a first embedding vector of each first text.

[0264] Step S1022 includes:

[0265] S1022a: Clustering the first embedding vector according to the second parameter based on a preset clustering algorithm to obtain at least one vector cluster.

[0266] In an embodiment of the present application, the electronic device can use a large language model to obtain a first parameter for dimensionality reduction processing and a second parameter for clustering.

[0267] Accordingly, the electronic device can calculate the average length and standard deviation of each first text, and then input the average length, standard deviation of each first text, and the third prompt word into the large language model.

[0268] The third prompt word is used to instruct the large language model to generate parameters for dimensionality reduction processing and parameters for clustering based on the average length and standard deviation of the input text.

[0269] In one implementation, before using a large language model to generate parameters for dimensionality reduction processing and parameters for clustering, the electronic device can also input into the large language model: the length average and standard deviation of multiple groups of second sample texts, and labels corresponding to each group of second sample texts; wherein a group of second sample texts includes multiple sample texts, and the labels corresponding to a group of second sample texts include: the parameter values ​​of each first parameter when the dimensionality reduction processing is performed on the embedding vector of the group of second sample texts, and the parameter values ​​of each second parameter when the embedding vector of the group of second sample texts is clustered.

[0270] In addition, the label corresponding to a group of second sample texts may further include: parameter values ​​of each third parameter when extracting keywords from the embedding vector of the group of second sample texts.

[0271] For example, the average length of a group of second sample texts is large, and the standard deviation is also large. This indicates that the data volume of the text in the group of second sample texts is large and the event information represented is complex. That is, the group of second sample texts contains a large number of texts, and the categories they belong to are diverse and subdivided. Accordingly, in each first parameter of the group of second sample texts, the minimum distance (min_dist) between each embedded vector after dimensionality reduction can be 0.1-0.3 to ensure that there is a certain separation between the obtained vector clusters; the number of neighboring points (n_neighbors) when constructing the neighborhood after dimensionality reduction can be increased, such as 30-50, to capture more global structures. In each second parameter of the group of second sample texts, the minimum number of vectors contained in a vector cluster (min_cluster_size) can be increased, such as 10-20, to avoid treating smaller vector clusters as noise. The minimum number of vectors in the neighborhood of the center of a vector cluster can be increased, and can be adjusted according to the data density to reduce the number of embedded vectors determined as noise. In each third parameter of the second sample text group, the minimum number of texts contained in each topic (min_topic_size) can be increased, such as 20-50, to ensure that the topic has sufficient text support. The setting method of the number of topics can be auto to automatically optimize the number of topics based on the algorithm. The number of keywords in the keyword set of each topic (top_n_words) can be increased, such as 15-20, to ensure that the obtained keywords are more comprehensive.

[0272] For example, the length average of a group of second sample texts is small, and the standard deviation is also small. This indicates that the data volume of the text in the group of second sample texts is small and the event information represented is simple. That is, the number of texts contained in the group of second sample texts is small, and the categories they belong to are few and obvious. Accordingly, in each first parameter of the group of second sample texts, the minimum distance (min_dist) between each embedded vector after dimensionality reduction can be unchanged or reduced, such as 0.0-0.1, to ensure that the obtained vector cluster is tighter; the number of neighboring points (n_neighbors) when constructing the neighborhood after dimensionality reduction can be reduced, such as 5-15, to emphasize the local structure. In each second parameter of the group of second sample texts, the minimum number of vectors contained in a vector cluster (min_cluster_size) can be reduced, such as 3-5, to capture more categories with a smaller range. The minimum number of vectors in the neighborhood of the center of a vector cluster can be reduced to allow more vectors to be clustered into a vector cluster. In each third parameter of the second sample text group, the minimum number of texts contained in each topic (min_topic_size) can be reduced, such as 5-10, to allow the text to be subdivided into multiple categories. The number of topics can be set to auto, or a smaller number of categories can be set according to actual needs. The number of keywords in the keyword set of each topic (top_n_words) can remain unchanged or decrease, such as 10-15.

[0273] Accordingly, after obtaining the first parameter for dimensionality reduction processing and the second parameter for clustering, the electronic device can encode each first text, and perform dimensionality reduction processing on the encoding result according to the first parameter to obtain the first embedding vector of each first text. Furthermore, according to the second parameter, the first embedding vector is clustered based on a preset clustering algorithm to obtain at least one vector cluster. The process of dimensionality reduction and clustering can refer to the relevant description of the above-mentioned step S1022 and step S1023, which will not be repeated here.

[0274] Based on the above processing, the large language model can learn the length average and standard deviation based on the text, determine the first parameter for dimensionality reduction processing, the second parameter for clustering, and the third parameter for extracting keywords. Accordingly, the electronic device can use the large language model to obtain the first parameter, the second parameter, and the third parameter suitable for processing the embedding vector of the first text, so that the first text can be classified according to actual needs, thereby improving the accuracy and consistency of the determined second text.

[0275] In some embodiments, after step S104, the method further includes:

[0276] The third text and the fifth prompt word are input into the large language model to obtain the cause of the event in the category to be processed.

[0277] The fifth prompt word is used to indicate the cause of the event represented by the input text generated by the large language model.

[0278] In the embodiment of the present application, since the events represented by each third text belong to the category to be processed, the electronic device can use the large language model to process the third text to obtain the cause of the event in the category to be processed.

[0279] That is, the electronic device may input the third text and the fifth prompt word into the large language model.

[0280] The fifth prompt word is used to indicate the cause of the event represented by the input text generated by the large language model. For example, the fifth prompt word may be: "Please output the cause of the event of this category".

[0281] Accordingly, the large language model can process and analyze each third text, obtain and output the cause of the event represented by each third text. Further, the electronic device can obtain the cause of the event in the category to be processed.

[0282] Based on the above processing, for events in the category to be processed, the electronic device can use the large language model to analyze the third text corresponding to the category to be processed, clarify the causes of the events in the category to be processed, and obtain the core contradictions and demands of the events in the category to be processed.

[0283] In some embodiments, after step S104, the method further includes:

[0284] The third text and the sixth prompt word are input into the large language model to obtain the risk index of the event in the category to be processed.

[0285] The sixth prompt word is used to indicate the risk indicator of the event represented by the input text generated by the large language model. The risk indicator of an event includes at least one of the following: the amount involved in the event, the number of people involved in the event, and the area affected by the event.

[0286] In the embodiment of the present application, since the events represented by each third text belong to the category to be processed, the electronic device can process the third text using the large language model to obtain the risk index of the events in the category to be processed. The risk index of an event includes at least one of the following: the amount involved in the event, the number of people involved in the event, and the area affected by the event.

[0287] That is, the electronic device may input the third text and the sixth prompt word into the large language model.

[0288] The sixth prompt word is used to indicate the risk index of the event represented by the input text generated by the large language model. For example, the sixth prompt word may be: "Please output the risk index of the event of this category".

[0289] Accordingly, the large language model can process and analyze each third text, obtain and output the risk index of the event represented by each third text. Furthermore, the electronic device can obtain the risk index of the event in the category to be processed.

[0290] It is understandable that the risk indicators of events of different categories may be different. For example, for natural disaster events, the risk indicators of events may be the number of people involved in the event, the area affected by the event, etc.

[0291] Based on the above processing, for events in the to-be-processed category, the electronic device can use the large language model to analyze the third text corresponding to the to-be-processed category, clarify the risk indicators of the events in the to-be-processed category, and facilitate subsequent analysis and processing.

[0292] In some embodiments, after step S104, the method further includes:

[0293] The third text and the seventh prompt word are input into the large language model to obtain the risk level of the event in the category to be processed.

[0294] The seventh prompt word is used to indicate the risk level of the event represented by the input text generated by the large language model. The risk level of an event is determined based on the risk index of the event.

[0295] In the embodiment of the present application, since the events represented by each third text belong to the category to be processed, the electronic device can process the third text using the large language model to obtain the risk level of the events in the category to be processed. The risk level of an event is determined based on the risk index of the event.

[0296] That is, the electronic device may input the third text and the seventh prompt word into the large language model.

[0297] The seventh prompt word is used to indicate the risk level of the event represented by the input text generated by the large language model. For example, the seventh prompt word may be: "Please determine the risk level of the event of this category based on the risk index of the event of this category."

[0298] Accordingly, the large language model can process and analyze each third text, and obtain and output the risk level of the event represented by each third text.

[0299] In one implementation, the large language model can combine real-time data and knowledge of the field to which the event belongs to dynamically analyze the identified risk indicators, evaluate the urgency and urgency of the event, and determine the risk level of the event in this category. For example, for events with high urgency and urgency, the risk level of the event in this category can be determined to be high; for events with low urgency and urgency, the risk level of the event in this category can be determined to be low.

[0300] Based on the above processing, the large language model can identify specific risk indicators for different types of events, and conduct dynamic risk analysis in combination with real-time data and domain knowledge to accurately assess the risk level of the event. This provides support for subsequent rapid response and resource scheduling for events with high risk levels, and reduces the impact of high-risk events.

[0301] In this way, high-risk events can be quickly identified. Subsequently, managers of relevant platforms can give priority to and handle events of this category, achieve continuous monitoring and analysis of data in relevant platforms, and achieve early warning and rapid response to high-risk events.

[0302] In some embodiments, the method further comprises:

[0303] Based on the occurrence time of the event represented by the third text, a time series analysis algorithm is used to obtain an analysis result of the occurrence period of the event in the category to be processed.

[0304] In the application embodiment, the electronic device can obtain the occurrence time of the event represented by the third text, and use the time series analysis algorithm to determine whether the event in the category to be processed occurs periodically. The time series analysis algorithm can be: a time series dense encoder (TiDE) for long-term time series prediction, or a long short-term memory network (LSTM, Long Short-Term Memory), etc.

[0305] If the events in the to-be-processed category occur periodically, the occurrence period (eg, year, month, week, or day, etc.) of the events in the to-be-processed category may be determined.

[0306] For example, the events represented by the third texts contained in a category to be processed are all events related to water supply pipeline failures in a certain area, and the occurrence time of the events in the category to be processed is concentrated in December of each year. Accordingly, the electronic device can use the time series analysis algorithm to obtain the occurrence cycle of the events in the category to be processed as years. Subsequently, the administrator of the relevant platform can inform the maintainer of the water supply pipeline to inspect and maintain the water supply pipeline before December of each year, replace aging water pipes, and stabilize the water supply pressure.

[0307] For another example, the events represented by the third texts contained in a pending category are all related to the insufficient number of service personnel in a certain area's traffic management department, and the occurrence time of the events in the pending category is concentrated on Mondays every week. Accordingly, the electronic device can use the time series analysis algorithm to obtain that the occurrence cycle of the events in the pending category is weeks. Subsequently, the administrator of the relevant platform can inform the traffic management department to increase the number of staff on duty on Mondays to provide corresponding services to visitors in a timely manner.

[0308] Based on the above processing, the electronic device can perform periodic analysis on the events in the category to be processed, evaluate the frequency and trend of the events in the category to be processed, determine the occurrence pattern and specific time period of the events, and provide a basis for preventing and handling the events in this category. Subsequently, the personnel involved in the event can proactively prevent the occurrence of the events, reduce the frequency of the events, reduce the scope of the events, improve the efficiency of event handling, and realize the transformation of the events from "first occur and then handle" to "handle before they occur" based on the analysis results of the occurrence cycle of the events in this category.

[0309] Based on the same inventive concept, the present application embodiment provides an event recognition device. Figure 6 , Figure 6 A structural diagram of an event recognition device provided in an embodiment of the present application, the device comprising:

[0310] A first text acquisition module 601 is used to acquire multiple texts containing event information of an event as a first text;

[0311] A second text determination module 602, configured to determine, from each first text, text representing an event belonging to a category to be processed based on a preset clustering algorithm, as a second text;

[0312] Theme viewpoint generation module 603 is used to input each second text and the first prompt word into the large language model to obtain the first theme viewpoint of the text of the category to be processed; wherein the first prompt word is used to indicate the theme viewpoint of the input text generated by the large language model;

[0313] The matching module 604 is used to use the large language model to determine the text that matches the first theme viewpoint from each second text as the third text representing the event to be processed.

[0314] In some embodiments, the second text determination module 602 includes:

[0315] An encoding submodule, used for encoding each first text to obtain a first embedding vector of each first text;

[0316] A clustering submodule, configured to cluster the first embedding vector based on a preset clustering algorithm to obtain at least one vector cluster;

[0317] A vector cluster selection submodule, used for selecting a vector cluster belonging to a category to be processed from the at least one vector cluster obtained;

[0318] The second text determination submodule is used to use the first text represented by the first embedding vector included in the selected vector cluster as the second text.

[0319] In some embodiments, the apparatus further comprises:

[0320] A parameter acquisition module, used for inputting the length average, standard deviation, and third prompt word of each first text into the large language model before encoding each first text to obtain the first embedding vector of each first text, so as to obtain the first parameter for dimensionality reduction processing and the second parameter for clustering;

[0321] The first parameter includes at least one of the following: the dimension of the embedded vector after dimensionality reduction, the minimum distance between each embedded vector after dimensionality reduction, and the number of adjacent points when constructing the neighborhood after dimensionality reduction; the second parameter includes at least one of the following: the minimum number of vectors contained in a vector cluster, the minimum number of vectors in the neighborhood of the center of a vector cluster; the third prompt word is used to instruct the large language model to generate parameters for dimensionality reduction processing and parameters for clustering based on the length average and standard deviation of the input text;

[0322] The encoding submodule is specifically used for:

[0323] Encode each first text, and perform dimensionality reduction processing on the encoding result according to the first parameter to obtain a first embedding vector of each first text;

[0324] The clustering submodule is specifically used for:

[0325] According to the second parameter, the first embedding vectors are clustered based on a preset clustering algorithm to obtain at least one vector cluster.

[0326] In some embodiments, the matching module 604 is specifically configured to:

[0327] For each second text, input the second text and the first prompt word into the large language model to obtain a second topic view of each second text;

[0328] Calculating the similarity between the first topic view and the second topic view of each second text;

[0329] If the calculated similarity is greater than a preset similarity threshold, the second text is determined as the third text representing the event to be processed.

[0330] In some embodiments, the matching module 604 is specifically configured to:

[0331] For each second text, the second text, the first topic viewpoint and the fourth prompt word are input into the large language model to obtain the matching degree of the second text; wherein the fourth prompt word is used to indicate the matching degree between the text generated by the large language model and the topic viewpoint;

[0332] If the matching degree of the second text is greater than a preset matching degree threshold, the second text is determined as a third text representing the event to be processed;

[0333] and / or,

[0334] The first text acquisition module 601 is specifically used for:

[0335] Get multiple texts containing event information of events;

[0336] Preprocessing each acquired text separately to obtain multiple first texts; wherein the preprocessing includes at least one of the following: denoising, stop word filtering and stem extraction;

[0337] and / or,

[0338] The device also includes:

[0339] The cycle analysis module is used to obtain the analysis result of the occurrence cycle of the events in the category to be processed based on the occurrence time of the events represented by the third text and using a time series analysis algorithm.

[0340] In some embodiments, the apparatus further comprises:

[0341] a title generation module, used to input the third text and the second prompt word into the large language model to obtain the title of the text of the category to be processed; wherein the second prompt word is used to instruct the large language model to generate the title of the input text;

[0342] and / or,

[0343] a cause acquisition module, for determining, by using the large language model, from each second text a text matching the first theme viewpoint as a third text representing the event to be processed, inputting the third text and a fifth prompt word into the large language model to obtain the cause of the event in the category to be processed; wherein the fifth prompt word is used to indicate the cause of the event represented by the input text generated by the large language model;

[0344] and / or,

[0345] A risk indicator acquisition module is used to use the large language model to determine the text matching the first theme viewpoint from each second text as the third text representing the event to be processed, and then input the third text and the sixth prompt word into the large language model to obtain the risk indicator of the event in the category to be processed; wherein the sixth prompt word is used to indicate that the large language model generates the risk indicator of the event represented by the input text, and the risk indicator of an event includes at least one of the following: the amount involved in the event, the number of people involved in the event, and the area affected by the event;

[0346] and / or,

[0347] A risk level acquisition module is used to input the third text and a seventh prompt word into the large language model after determining the text matching the first theme viewpoint from each second text using the large language model as the third text representing the event to be processed, so as to obtain the risk level of the event in the category to be processed; wherein the seventh prompt word is used to indicate the risk level of the event represented by the input text generated by the large language model, and the risk level of an event is determined based on the risk indicator of the event.

[0348] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701 , a communication interface 702 , a memory 703 and a communication bus 704 , wherein the processor 701 , the communication interface 702 , and the memory 703 communicate with each other via the communication bus 704 .

[0349] Memory 703, used for storing computer programs;

[0350] The processor 701 is used to implement the steps of any of the above event identification methods when executing the program stored in the memory 703.

[0351] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0352] The communication interface is used for communication between the above electronic device and other devices.

[0353] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0354] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0355] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above event identification methods are implemented.

[0356] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute any event identification method in the above embodiments.

[0357] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state disk (SSD), etc.

[0358] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0359] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0360] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An event recognition method, characterized in that: The method comprises: Acquire multiple texts containing event information of an event as first text; Determine, based on a preset clustering algorithm, from each first text, a text representing an event that belongs to the category to be processed, as the second text; Input each second text and the first prompt word into the large language model to obtain a first topic view of the text of the category to be processed; wherein the first prompt word is used to indicate the topic view of the input text generated by the large language model; By using the large language model, texts matching the first subject viewpoint are determined from the second texts as third texts representing the event to be processed.

2. The method according to claim 1, characterized in that The method of determining, from each first text, texts representing events belonging to the category to be processed based on a preset clustering algorithm as second texts includes: Encode each first text to obtain a first embedding vector of each first text; Clustering the first embedding vector based on a preset clustering algorithm to obtain at least one vector cluster; Selecting a vector cluster belonging to the category to be processed from the obtained at least one vector cluster; The first text represented by the first embedding vector contained in the selected vector cluster is used as the second text.

3. The method according to claim 2, characterized in that Before encoding each first text to obtain a first embedding vector of each first text, the method further includes: Inputting the length average, standard deviation, and third prompt word of each first text into the large language model to obtain a first parameter for dimensionality reduction processing and a second parameter for clustering; The first parameter includes at least one of the following: the dimension of the embedded vector after dimensionality reduction, the minimum distance between each embedded vector after dimensionality reduction, and the number of adjacent points when constructing the neighborhood after dimensionality reduction; the second parameter includes at least one of the following: the minimum number of vectors contained in a vector cluster, the minimum number of vectors in the neighborhood of the center of a vector cluster; the third prompt word is used to instruct the large language model to generate parameters for dimensionality reduction processing and parameters for clustering based on the length average and standard deviation of the input text; The encoding of each first text to obtain a first embedding vector of each first text includes: Encode each first text, and perform dimensionality reduction processing on the encoding result according to the first parameter to obtain a first embedding vector of each first text; The clustering of the first embedding vector based on a preset clustering algorithm to obtain at least one vector cluster includes: According to the second parameter, the first embedding vectors are clustered based on a preset clustering algorithm to obtain at least one vector cluster.

4. The method according to claim 1, characterized in that: The step of using the large language model to determine, from each second text, a text matching the first subject viewpoint as a third text representing the event to be processed includes: For each second text, input the second text and the first prompt word into the large language model to obtain a second topic view of each second text; Calculating the similarity between the first topic view and the second topic view of each second text; If the calculated similarity is greater than a preset similarity threshold, the second text is determined as the third text representing the event to be processed.

5. The method according to claim 1, characterized in that The step of using the large language model to determine, from each second text, a text matching the first subject viewpoint as a third text representing the event to be processed includes: For each second text, the second text, the first topic viewpoint and the fourth prompt word are input into the large language model to obtain the matching degree of the second text; wherein the fourth prompt word is used to indicate the matching degree between the text generated by the large language model and the topic viewpoint; if the matching degree of the second text is greater than a preset matching degree threshold, the second text is determined as the third text representing the event to be processed; and / or, The step of obtaining a plurality of texts containing event information of an event as a first text includes: Acquire multiple texts containing event information of an event; preprocess each acquired text to obtain multiple first texts; wherein the preprocessing includes at least one of the following: denoising, stop word filtering and stem extraction; and / or, The method further comprises: Based on the occurrence time of the event represented by the third text, a time series analysis algorithm is used to obtain an analysis result of the occurrence period of the events in the category to be processed.

6. The method according to claim 1, characterized in that After using the large language model to determine, from each second text, a text matching the first subject viewpoint as a third text representing the event to be processed, the method further includes: Inputting the third text and the second prompt word into the large language model to obtain the title of the text of the category to be processed; wherein the second prompt word is used to instruct the large language model to generate the title of the input text; and / or, Inputting the third text and the fifth prompt word into the large language model to obtain the cause of the event in the category to be processed; wherein the fifth prompt word is used to indicate the cause of the event represented by the input text generated by the large language model; and / or, Inputting the third text and the sixth prompt word into the large language model to obtain the risk index of the event in the category to be processed; wherein the sixth prompt word is used to instruct the large language model to generate the risk index of the event represented by the input text, and the risk index of an event includes at least one of the following: the amount involved in the event, the number of people involved in the event, and the area affected by the event; and / or, The third text and the seventh prompt word are input into the large language model to obtain the risk level of the event in the category to be processed; wherein the seventh prompt word is used to indicate the risk level of the event represented by the input text generated by the large language model, and the risk level of an event is determined based on the risk indicator of the event.

7. An event recognition device, characterized in that: The device comprises: A first text acquisition module, used to acquire multiple texts containing event information of an event as a first text; A second text determination module, used to determine, from each first text based on a preset clustering algorithm, text representing an event belonging to a category to be processed, as the second text; A topic viewpoint generation module, used for inputting each second text and the first prompt word into the large language model to obtain the first topic viewpoint of the text of the category to be processed; wherein the first prompt word is used to indicate the topic viewpoint of the input text generated by the large language model; A matching module is used to use the large language model to determine, from each second text, a text that matches the first subject viewpoint as a third text representing the event to be processed.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-6 when executing a program stored in a memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the method according to any one of claims 1 to 6.

Images