Abnormal event detection method and apparatus, electronic device, and medium

Abstract

The present disclosure provides a method for detecting abnormal events, which can be used in the financial field or other fields. The method comprises the following steps: presetting a historical time span and a time slice length; obtaining a text set of abnormal events to be detected within the historical time span; dividing the historical time span into multiple time slices; calculating a sub-abnormal value of each word in multiple words; calculating a maximum continuous sub-sequence sum of the sub-abnormal value of each word in the historical time span, obtaining an abnormal value and a maximum continuous time span of each word; sorting multiple words according to the abnormal value from large to small, selecting the first M words as center words; obtaining descriptive words similar to the distribution probability of the center words; obtaining multiple abnormal events based on the M center words and the descriptive words; and obtaining a detection result of abnormal events of the text set to be detected based on the multiple abnormal events. The present disclosure also provides an abnormal event detection device, equipment, storage medium and program product.

Description

Technical Field

[0001] This disclosure relates to the field of event detection, and can also be used in the financial field or other fields. Specifically, it relates to a method, apparatus, equipment, medium and program product for detecting abnormal events. Background Technology

[0002] Users post various text messages on social media. It is necessary to detect events with abnormal popularity from these text messages, which is helpful for cybersecurity, public opinion awareness, and emergency response.

[0003] In realizing the present invention, the inventors discovered that current methods for detecting abnormal events have complex calculation processes and high time complexity, resulting in low detection speed and efficiency. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a method, apparatus, device, medium and program product for detecting abnormal events.

[0005] According to a first aspect of this disclosure, a method for detecting anomalous events is provided, comprising: presetting a historical time span and a time slice length; acquiring a text set of anomalous events to be detected within the historical time span, wherein the text set includes multiple texts, each text including multiple words; dividing the historical time span into multiple time slices according to the time slice length; calculating a sub-anomaly value for each word among the multiple words in each of the multiple time slices; based on the sub-anomaly values, calculating the maximum consecutive sub-sum of the sub-anomaly values ​​of each word in the historical time span to obtain the anomaly value and the maximum consecutive time span of each word, wherein the maximum consecutive time span includes at least one time slice; and sorting the multiple words according to the anomaly values. Sort the words from largest to smallest and select the top M words as center words, where the center words are words used to directly describe the abnormal events, and M is an integer greater than or equal to 1. For each of the M center words, obtain descriptive words with similar probability distributions to the center words within the maximum continuous time span of the center words, where the descriptive words are words used to indirectly describe the abnormal events. Based on the M center words and descriptive words, obtain multiple abnormal events, where each abnormal event includes a center word, a descriptive word, an anomaly value, and a maximum continuous time span, and the anomaly value and maximum continuous time span of the abnormal event are the same as the anomaly value and maximum continuous time span of the center word. Based on the multiple abnormal events, obtain the detection results of abnormal events in the set of texts to be detected.

[0006] According to an embodiment of this disclosure, the step of calculating the sub-outlier value of each of the plurality of words in each of the plurality of time slices includes: calculating the actual number and expected number of occurrences of each word in each time slice; and subtracting the actual number and the expected number to obtain the sub-outlier value of the word.

[0007] According to an embodiment of this disclosure, the step of obtaining descriptive words with similar distribution probabilities to the central word within the maximum continuous time span of the central word includes: calculating the distribution probability similarity between each candidate word (excluding the central word) and the central word within the maximum continuous time span of the central word; setting a preset distribution probability similarity threshold; and selecting candidate words with distribution probability similarity greater than the distribution probability similarity threshold as descriptive words of the central word.

[0008] According to embodiments of this disclosure, the step of calculating the probability distribution similarity between each candidate word (excluding the center word) and the center word includes: calculating the probability distribution of each candidate word in each time slice within the maximum continuous time span; calculating the probability distribution of the center word in each time slice within the maximum continuous time span; calculating the probability distribution sub-similarity between each candidate word and the center word in each time slice based on the probability distribution of each candidate word and the probability distribution of the center word; and superimposing the probability distribution sub-similarity within the maximum continuous time span to obtain the probability distribution similarity between each candidate word and the center word.

[0009] According to an embodiment of this disclosure, the step of obtaining the detection result of abnormal events in the text set to be detected based on the plurality of abnormal events includes: calculating the similarity between any two abnormal events among the plurality of abnormal events; and merging the plurality of abnormal events based on the similarity calculation result to obtain the detection result of abnormal events in the text set to be detected.

[0010] According to an embodiment of this disclosure, the step of calculating the similarity between any two abnormal events among the plurality of abnormal events includes: merging the head word and descriptive word of each of the plurality of abnormal events into a word set; and calculating a first similarity of the word sets of any two abnormal events among the plurality of abnormal events and a second similarity of the maximum continuous time span.

[0011] According to an embodiment of this disclosure, the step of merging the multiple abnormal events based on the similarity calculation result includes: setting a first similarity threshold and a second similarity threshold; and merging the two abnormal events into one abnormal event when the first similarity of any two abnormal events is greater than the first similarity threshold and the second similarity is greater than the second similarity threshold.

[0012] According to an embodiment of this disclosure, the step of merging the two anomalous events into one anomalous event includes: taking the union of the head words of the two anomalous events as the merged head word; taking the union of the descriptive words of the two anomalous events as the merged descriptive word; taking the union of the maximum continuous time span of the two anomalous events as the merged maximum continuous time span; comparing the outliers of the two anomalous events and taking the larger outlier as the merged outlier; and obtaining the merged anomalous event based on the merged head word, descriptive word, maximum continuous time span, and outlier.

[0013] According to an embodiment of this disclosure, the step of obtaining a text set of anomaly events to be detected within the historical time span includes: obtaining a total text set of anomaly events to be detected within the historical time span; and extracting a text set with topic tags from the total text set as the text set of anomaly events to be detected, wherein the topic tags are tags used to describe the topic of the text.

[0014] A second aspect of this disclosure provides an anomalous event detection device, comprising: a first preset module for presetting a historical time span and a time slice length; a first acquisition module for acquiring a text set of anomalous events to be detected within the historical time span, wherein the text set includes multiple texts, and each text includes multiple words; a segmentation module for dividing the historical time span into multiple time slices according to the time slice length; a first calculation module for calculating a sub-anomaly value of each word within each of the multiple time slices; a second calculation module for calculating the maximum consecutive sub-sequence sum of the sub-anomaly values ​​of each word within the historical time span based on the sub-anomaly values, thereby obtaining the anomaly value and the maximum consecutive time span of each word, wherein the maximum consecutive time span includes at least one time slice; and a first sorting module for sorting the... The system employs a first module to sort multiple words by their outlier values ​​from largest to smallest, selecting the top M words as center words. Here, the center words are words used to directly describe the abnormal event, and M is an integer greater than or equal to 1. A second acquisition module is used to acquire, for each of the M center words, a descriptive word whose probability distribution is similar to that of the center word within its maximum continuous time span. Here, the descriptive word is a word used to indirectly describe the abnormal event. A third acquisition module is used to obtain multiple abnormal events based on the M center words and descriptive words. Each abnormal event includes a center word, a descriptive word, an outlier value, and a maximum continuous time span. The outlier value and maximum continuous time span of the abnormal event are the same as the outlier value and maximum continuous time span of the center word. Finally, a result generation module is used to obtain the detection results of the abnormal events in the text set to be detected based on the multiple abnormal events.

[0015] According to an embodiment of this disclosure, the first calculation module includes: a first calculation unit, configured to calculate the actual number and expected number of occurrences of each word in each time slice; and a second calculation unit, configured to subtract the actual number and the expected number to obtain the sub-anomaly value of the word.

[0016] According to an embodiment of this disclosure, the second acquisition module includes: a third calculation unit, configured to calculate the probability distribution similarity between each candidate word (excluding the central word) and the central word within the maximum continuous time span of the central word; a first preset unit, configured to preset a probability distribution similarity threshold; and a selection unit, configured to select candidate words with a probability distribution similarity greater than the probability distribution similarity threshold as descriptors of the central word.

[0017] According to an embodiment of this disclosure, the third calculation unit includes: a first calculation submodule, configured to calculate the probability distribution of each candidate word in each time slice within the maximum continuous time span; a second calculation submodule, configured to calculate the probability distribution of the center word in each time slice within the maximum continuous time span; a third calculation submodule, configured to calculate the probability distribution sub-similarity between each candidate word and the center word in each time slice based on the probability distribution of each candidate word and the probability distribution of the center word; and a fourth calculation submodule, configured to superimpose the probability distribution sub-similarity within the maximum continuous time span to obtain the probability distribution similarity between each candidate word and the center word.

[0018] According to an embodiment of this disclosure, the result generation module includes: a fourth calculation unit, used to calculate the similarity between any two abnormal events among the plurality of abnormal events; and a merging unit, used to merge the plurality of abnormal events based on the similarity calculation result to obtain the detection result of abnormal events in the text set to be detected.

[0019] According to an embodiment of this disclosure, the fourth calculation unit includes: a first merging submodule, used to merge the head word and descriptive word of each of the plurality of abnormal events into a word set; and a fifth calculation submodule, used to calculate the first similarity of the word sets of any two abnormal events among the plurality of abnormal events and the second similarity of the maximum continuous time span.

[0020] According to an embodiment of this disclosure, the merging unit includes: a first preset submodule, used to preset a first similarity threshold and a second similarity threshold; and a second merging submodule, used to merge the two abnormal events into one abnormal event when the first similarity of any two abnormal events among the plurality of abnormal events is greater than the first similarity threshold and the second similarity is greater than the second similarity threshold.

[0021] According to embodiments of this disclosure, the second merging submodule includes: a first merging subunit, used to take the union of the headwords of two abnormal events as the merged headword; a second merging subunit, used to take the union of the descriptive words of two abnormal events as the merged descriptive word; a third merging subunit, used to take the union of the maximum continuous time span of two abnormal events as the merged maximum continuous time span; a fourth merging subunit, used to compare the outlier values ​​of two abnormal events and take the larger outlier value as the merged outlier value; and a first determining subunit, used to obtain the merged abnormal event based on the merged headword, descriptive word, maximum continuous time span, and outlier value.

[0022] According to an embodiment of this disclosure, the first acquisition module includes: a second acquisition unit, configured to acquire a total set of texts of anomaly events to be detected within the historical time span; and an extraction unit, configured to extract a set of texts with topic tags from the total set of texts as the set of texts of anomaly events to be detected, wherein the topic tags are tags used to describe the topics of the texts.

[0023] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the methods described above.

[0024] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described above.

[0025] The fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0026] The above one or more embodiments have the following advantages or beneficial effects: The abnormal event detection method provided by the embodiments of this disclosure utilizes the maximum continuous subsequence sum and time slice technology to obtain the maximum continuous time span and central word of the abnormal event based on the maximum outlier value, and then calculates the descriptive words based on similarity. Finally, it merges repeated events based on the event descriptive words and time similarity. Without prior knowledge, it can quickly detect hot events in social text sets as well as the start and end times of the events, reducing the computational load of abnormal event detection. The calculation process is simple and the time complexity is low, thereby greatly improving the detection speed and efficiency of abnormal events. Attached Figure Description

[0027] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0028] Figure 1 The illustration schematically depicts application scenarios of methods, apparatus, devices, media, and program products for detecting abnormal events according to embodiments of the present disclosure.

[0029] Figure 2 A flowchart illustrating an abnormal event detection method according to an embodiment of the present disclosure is shown schematically;

[0030] Figure 3 A flowchart illustrating the calculation of sub-outlier values ​​for each word according to an embodiment of this disclosure is shown schematically.

[0031] Figure 4A flowchart illustrating the process of obtaining descriptive words with distribution probabilities similar to those of the center word according to an embodiment of the present disclosure is shown.

[0032] Figure 5 A flowchart illustrating the calculation of the probability similarity between each candidate word and the center word according to an embodiment of the present disclosure is shown.

[0033] Figure 6 A flowchart illustrating the calculation of the similarity between any two anomalous events among a plurality of anomalous events according to an embodiment of the present disclosure is shown.

[0034] Figure 7 A flowchart illustrating the merging of two abnormal events into one abnormal event according to an embodiment of the present disclosure is shown schematically.

[0035] Figure 8 A schematic block diagram of an abnormal event detection apparatus according to an embodiment of the present disclosure is shown; and

[0036] Figure 9 A block diagram of an electronic device suitable for implementing a method for detecting abnormal events according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0037] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0038] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0039] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0040] When using expressions such as "at least one of A, B, and C," it should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features.

[0041] Hashtags: On social media platforms, users use hashtags (#) followed by text to start topics in their posts. The text after the # is called a hashtag, also known as a tic-tac-toe tag or topic tag. Posts tagged with hashtags are more likely to become trending topics on social networks, and their core words tend to appear more frequently over a period of time. In contrast, common words are less likely to show significant deviations from expectations. Trending events are also known as anomalous events. An anomalous value for an anomalous event refers to the frequency of words representing the anomalous event exceeding the average frequency of that word in the dictionary within a given time period.

[0042] Users post various text messages on social media, making it essential to detect events with abnormal popularity from these messages. This is helpful for cybersecurity, public opinion awareness, and emergency response. Current technologies for detecting abnormal events suffer from complex computations, high time complexity, and poor readability. Furthermore, they typically assume a fixed duration for all events, making it impossible to determine the specific start and end times of abnormal events. For example, commonly used topic clustering techniques can model trending events on social networks. LDA topic modeling is a generative model that assumes each word in each text is selected from a topic with a certain probability, and then a word is selected from that topic with a certain probability. The prior probabilities of topic-text and word-topic both follow a Dirichlet distribution. Finally, keywords for each topic are obtained through parameter estimation. However, LDA technology also has significant drawbacks. First, LDA topic modeling requires two prior parameters: the prior distributions of topics and words, and the prior distributions of text and topics. Second, LDA topic modeling has high time complexity and slow convergence speed. Third, LDA topic modeling can only provide the topic of the current text set, not the time from when the topic occurred to when it ended. In summary, LDA topic modeling requires additional knowledge support, has low algorithmic efficiency, and its readability needs improvement.

[0043] In view of the above problems, embodiments of this disclosure provide a method for detecting abnormal events, including: presetting a historical time span and a time slice length; obtaining a text set of abnormal events to be detected within the historical time span, wherein the text set includes multiple texts, and each text includes multiple words; dividing the historical time span into multiple time slices according to the time slice length; calculating a sub-anomaly value for each word among the multiple words in each of the multiple time slices; based on the sub-anomaly values, calculating the maximum consecutive sub-sum of the sub-anomaly values ​​of each word in the historical time span to obtain the anomaly value and the maximum consecutive time span of each word, wherein the maximum consecutive time span includes at least one time slice; and sorting the multiple words according to the anomaly... The values ​​are sorted from largest to smallest, and the top M words are selected as center words, where the center words are words used to directly describe the abnormal event, and M is an integer greater than or equal to 1. For each of the M center words, a descriptive word with a similar probability distribution to the center word within its maximum continuous time span is obtained, where the descriptive word is a word used to indirectly describe the abnormal event. Based on the M center words and descriptive words, multiple abnormal events are obtained, where each abnormal event includes a center word, a descriptive word, an anomaly value, and a maximum continuous time span, and the anomaly value and maximum continuous time span of the abnormal event are the same as the anomaly value and maximum continuous time span of the center word. Based on the multiple abnormal events, the detection results of abnormal events in the text set to be detected are obtained. The abnormal event detection method provided by this disclosure can quickly detect hot events and the start and end times of events in a social text set without prior knowledge, reducing the computational load of abnormal event detection. The calculation process is simple and has low time complexity, thereby greatly improving the detection speed and efficiency of abnormal events.

[0044] It should be noted that the methods and apparatus specified in this disclosure can be used for detecting abnormal events in the financial field, as well as for detecting abnormal events in any field other than the financial field. The application fields of the methods and apparatus for detecting abnormal events disclosed in this disclosure are not limited.

[0045] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0046] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0047] Figure 1The illustration schematically depicts an application scenario of an anomaly detection method, apparatus, device, medium, and program product according to embodiments of the present disclosure.

[0048] like Figure 1 As shown, application scenario 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as a medium for providing a communication link between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0049] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0050] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0051] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0052] It should be noted that the abnormal event detection method provided in this disclosure embodiment can generally be executed by server 105. Correspondingly, the abnormal event detection device provided in this disclosure embodiment can generally be located in server 105. The abnormal event detection method provided in this disclosure embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the abnormal event detection device provided in this disclosure embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0053] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0054] Figure 2 A flowchart illustrating an abnormal event detection method according to an embodiment of the present disclosure is shown schematically.

[0055] like Figure 2 As shown, the abnormal event detection method of this embodiment includes operations S210 to S290.

[0056] In operation S210, the historical time span and time slice length are preset. The historical time span is the period to be analyzed; for example, a preset historical time span of 1 day would analyze abnormal events in texts posted by users within the past day. The time slice length is used to divide the historical time span into multiple time slices of equal length, facilitating the calculation of sub-anomaly values ​​within each time slice and determining the start and end times of abnormal events based on the time slice. For example, a preset time slice length of 1 hour would divide the past day into 24 time slices, with calculations performed within each time slice. It should be noted that a shorter time slice length allows for more time slices to be divided into the historical time span, resulting in more accurate identification of the core words and descriptive words of abnormal events, as well as the start and end times of the abnormal events; however, this increases the computational workload. Therefore, to balance computational accuracy and computation time, the time slice length needs to be set according to the actual situation of the historical time span; the embodiments of this disclosure do not limit this.

[0057] In operation S220, a text set of anomaly events to be detected within the historical time span is obtained, wherein the text set includes multiple texts, and each text includes multiple words. For example, the text set of anomaly events to be detected can be obtained from a website using a web crawler. Furthermore, to facilitate subsequent calculations, the text in the text set can be preprocessed, using methods such as word segmentation to convert the text into a set of words, thereby accelerating subsequent word-based calculations.

[0058] According to embodiments of this disclosure, the step of obtaining a text set of anomaly events to be detected within the historical time span includes: obtaining a total text set of anomaly events to be detected within the historical time span; and extracting a text set with topic tags from the total text set as the text set of anomaly events to be detected, wherein the topic tags are tags used to describe the topics of the text. Since text with topic tags is more likely to contain anomalies, extracting a text set with topic tags from the total text set as the text set of anomaly events to be detected can improve the detection efficiency of anomalies and avoid interference from other noise.

[0059] In embodiments of this disclosure, user consent or authorization may be obtained before acquiring the text set of anomalies to be detected within the historical time span. For example, a request to acquire text posted by the user may be sent to the user before operation S220. Operation S220 is performed if the user consents or authorizes the acquisition of the user-posted text.

[0060] In operation S230, the historical time span is divided into multiple time slices according to the time slice length. For example, if the historical time span is 24 hours and the time slice length is 1 hour, then the historical time span is divided into 24 time slices.

[0061] In operation S240, within each of the plurality of time slices, the sub-anomaly value of each of the plurality of words is calculated.

[0062] Figure 3 A flowchart illustrating the calculation of sub-outlier values ​​for each word according to an embodiment of the present disclosure is shown schematically.

[0063] like Figure 3 As shown, the calculation of sub-outlier values ​​for each word in this embodiment includes operations S310 to S320.

[0064] In operation S310, within each time slice, the actual number and expected number of occurrences of each word are calculated.

[0065] In operation S320, the difference between the actual number and the expected number is used to obtain the sub-anomaly value of the word. It should be noted that an anomaly only occurs when the actual number of occurrences of the word is strictly greater than the expected number; if it is equal to or less than the expected number, the word is considered not to have an anomaly.

[0066] For example, when the text set of the abnormal event to be detected is the text set with topic tags extracted from the total text set, it is assumed that the probability of the occurrence of words and topic tags follows a binomial distribution, as shown in equation (1), where, This refers to the number of words w that appear together with the topic tags in the i-th time slice. The probability of.

[0067] Calculate P according to formula (2) #w , where P #w This refers to the probability that word w and topic tags appear together. The expected number of times word w appears in the i-th time slice, E[w|i], is calculated according to equation (3). The difference between the actual number of times word w appears in the i-th time slice and the expected number is calculated according to equation (4), yielding the sub-outlier anomaly (w, i) of word w in the i-th time slice. This sub-outlier calculation method is simple and effective, requires less computational data, and can accelerate the overall computation speed.

[0068]

[0069]

[0070] E[w|i]=N i P #w (3)

[0071]

[0072] The parameters involved in equations (1)-(4) are shown in Table 1:

[0073]

[0074]

[0075] Table 1 Parameter Table

[0076] In operation S250, based on the sub-outliers, the maximum consecutive sub-sum of sub-outliers for each word within the historical time span is calculated, resulting in the outlier and maximum consecutive time span for each word. The maximum consecutive time span includes at least one time slice. It should be noted that the time slices included in the maximum consecutive time span must be consecutive time slices to reflect the start and end times of the anomalous event. For determining the outbreak period of word w, the maximum consecutive sub-sum of sub-outliers for word w within the historical time span is calculated, resulting in the maximum consecutive time slice for each word. Here, the outlier refers to the maximum consecutive sub-sum of sub-outliers for word w, and the size of the outlier represents the magnitude of the event's influence. Its time complexity is O(N), where N is the number of words.

[0077] For example, formula (5) for determining the maximum continuous time span Time(w, I) of word w is:

[0078]

[0079] In the formula, n is the number of time slices into which the maximum continuous time span is divided.

[0080] Taking the historical time span divided into 6 time slices as an example, the sub-outliers of word w in the 6 time slices are: -1, 5, -4, 3, 2, 1. Since the sum of 5, -4, 3, 2, 1 in this sequence is the largest consecutive sub-sum, this largest consecutive sub-sum is the outlier of the word. The time slices corresponding to the sub-outliers 5 and 1 are the start and end time slices of the word's largest consecutive time span, respectively. Accordingly, the time complexity of finding the largest consecutive sub-sum is O(N).

[0081] In operation S260, the multiple words are sorted from largest to smallest according to their outlier values, and the top M words are selected as center words. Here, the center words are words used to directly describe the abnormal event, and M is an integer greater than or equal to 1. Selecting the top M words as center words reduces the number of center words, thereby reducing the computational load of subsequent descriptive words and accelerating the overall calculation speed. Furthermore, it should be noted that center words with excessively small outlier values ​​correspond to events that are not typical enough, and not treating them as abnormal events will not affect the overall calculation accuracy.

[0082] In operation S270, for each of the M center words, a descriptive word with a probability distribution similar to that of the center word within its maximum continuous time span is obtained. This descriptive word is used to indirectly describe the abnormal event. Further obtaining descriptive words based on the center words makes the description of the abnormal event more complete. Optionally, the calculation can start from the center word with the largest outlier.

[0083] Figure 4 A flowchart illustrating the process of obtaining descriptive words with distribution probabilities similar to those of the center word according to an embodiment of the present disclosure is shown.

[0084] like Figure 4 As shown, the acquisition of descriptive words with distribution probabilities similar to those of the center word in this embodiment includes operations S410 to S430.

[0085] In operation S410, within the maximum continuous time span of the center word, the probability distribution similarity between each candidate word (excluding the center word) and the center word is calculated.

[0086] In operation S420, a preset distribution probability similarity threshold is set.

[0087] In operation S430, candidate words with a distribution probability similarity greater than the distribution probability similarity threshold are selected as descriptors for the center word. The descriptors should have a similar distribution to the center word, so similarity is used for comparison. Descriptors with too low similarity are not typical enough, and not using them as descriptors for the center word will not affect the determination of the abnormal event. Furthermore, selecting descriptors with a distribution probability similarity greater than the distribution probability similarity threshold simplifies the amount of data required for subsequent calculations and speeds up the overall calculation.

[0088] Figure 5 A flowchart illustrating the calculation of the probability similarity between each candidate word and the center word according to an embodiment of the present disclosure is shown.

[0089] like Figure 5As shown, the calculation of the probability similarity between each candidate word and the center word in this embodiment includes operations S510 to S540.

[0090] In operation S510, the distribution probability of each candidate word in each time slice within the maximum continuous time span is calculated.

[0091] In operation S520, the distribution probability of the center word in each time slice within the maximum continuous time span is calculated.

[0092] In operation S530, based on the probability distribution of each candidate word and the probability distribution of the center word, the probability distribution sub-similarity between each candidate word and the center word in each time slice is calculated.

[0093] In operation S540, the probability distribution sub-similarity is superimposed within the maximum continuous time span to obtain the probability distribution similarity between each candidate word and the center word.

[0094] For example, the probability similarity Sim(x, y) between the center word x and a candidate word y over the maximum continuous time span of the center word x can be calculated according to the following formula (6):

[0095]

[0096] In the formula, P(x time P(y) represents the probability of the head word x appearing in time slice time. time ) represents the probability that word y appears in the text containing the head word x in time slice time. end_slice and start_slice represent the start and end time slices of the maximum continuous time span of the head word x, respectively.

[0097] In operation S280, based on the M center words and descriptive words, multiple abnormal events are obtained. Each abnormal event includes a center word, a descriptive word, an outlier, and a maximum continuous time span. The outlier and maximum continuous time span of the abnormal event are the same as the outlier and maximum continuous time span of the center word. An abnormal event can be completely described using the center word, descriptive word, outlier, and maximum continuous time span.

[0098] In operation S290, based on the multiple abnormal events, the detection results of abnormal events in the set of texts to be detected are obtained.

[0099] Optionally, when there are many abnormal events after merging, the abnormal values ​​can be sorted from largest to smallest according to the abnormal values ​​of the abnormal events, and a preset number of abnormal events can be selected as the detection results of the abnormal events in the text set to be detected, so as to make the detection results more intuitive and more readable.

[0100] According to embodiments of this disclosure, the step of obtaining the detection results of abnormal events in the text set to be detected based on the plurality of abnormal events includes: calculating the similarity between any two abnormal events among the plurality of abnormal events; and merging the plurality of abnormal events based on the similarity calculation result to obtain the detection results of abnormal events in the text set to be detected. Since the calculated abnormal events may overlap, it is necessary to merge the abnormal events with similarity to reduce the amount of data, making the final detection results more intuitive and readable.

[0101] Figure 6 A flowchart illustrating the calculation of the similarity between any two anomalous events among a plurality of anomalous events is shown schematically according to an embodiment of the present disclosure.

[0102] like Figure 6 As shown, the calculation of the similarity between any two abnormal events among multiple abnormal events in this embodiment includes operations S610 to S620.

[0103] In operation S610, the central word and descriptive word of each of the multiple abnormal events are merged into a word set.

[0104] In operation S620, the first similarity of the word sets and the second similarity of the maximum continuous time span of any two abnormal events among the plurality of abnormal events are calculated.

[0105] For example, the specific formulas for calculating the first similarity α and the second similarity β are as follows:

[0106]

[0107]

[0108] In the formula, W1 and W2 represent the word sets of two events 1 and 2, respectively, and I1 and I2 represent the maximum continuous time span of the word sets of two events 1 and 2, respectively.

[0109] According to embodiments of this disclosure, the step of merging the multiple anomalous events based on the similarity calculation results includes: setting a first similarity threshold and a second similarity threshold; and merging the two anomalous events into one anomalous event when the first similarity of any two anomalous events is greater than the first similarity threshold and the second similarity is greater than the second similarity threshold. If the first similarity α of the word sets and the second similarity β of the maximum continuous time span of two events exceed the thresholds, they are considered as one event, and in this case, the two anomalous events need to be merged to reduce redundant data in the event detection results. Furthermore, calculating the similarity of the word sets and the time span separately makes the similarity calculation between anomalous events more accurate.

[0110] Figure 7 The flowchart illustrating the merging of two abnormal events into one abnormal event according to an embodiment of the present disclosure is shown schematically.

[0111] like Figure 7 As shown, in this embodiment, merging two abnormal events into one abnormal event includes operations S710 to S750.

[0112] When operating S710, the union of the central words of the two abnormal events is taken as the central word of the merged event.

[0113] When operating S720, the descriptors of two abnormal events are combined to form the merged descriptor.

[0114] When operating S730, the union of the maximum continuous time spans of the two abnormal events is taken as the maximum continuous time span after merging.

[0115] When operating S740, the outlier values ​​of two abnormal events are compared, and the larger outlier value is used as the merged outlier value.

[0116] In operation S750, based on the merged headword, descriptor, maximum continuous time span, and outlier, the merged abnormal event is obtained. Since the headword, descriptor, and time span are each taken as a union, the calculation process is simplified, and the overall computation speed is improved.

[0117] For example, when the first similarity between event 1 and event 2 is greater than the first similarity threshold and the second similarity is greater than the second similarity threshold, event 1 and event 2 can be considered as occurring at the same time. In this case, the merging process for event 1 and event 2 is as follows:

[0118] Event 1:

[0119] Central word A

[0120] Descriptive words BCDE

[0121] Start time: 2022.2.24 02:00

[0122] End time: 12:00 PM, March 15, 2022

[0123] Outlier: 1300

[0124] Event 2:

[0125] Keyword: B

[0126] Descriptor: ACDF

[0127] Start time: 01:00, February 24, 2022

[0128] End time: 8:00 AM, March 15, 2022

[0129] Outlier: 1200

[0130] When merging, the core terms are merged and their union is taken; the descriptive terms are merged and their union is taken; the largest continuous time span is taken and its union is taken, with the smaller start time and the larger end time; outliers are taken, and the merged events are obtained as follows:

[0131] event:

[0132] Keyword: BA

[0133] Descriptor: CDEF

[0134] Start time: 2022.2.24 01:00

[0135] End time: 12:00 PM, March 15, 2022

[0136] Outlier: 1300

[0137] The abnormal event detection method provided in this disclosure utilizes the maximum continuous subsequence sum and time slice techniques. It obtains the maximum continuous time span and headword of the abnormal event based on the maximum outlier, then calculates descriptive words based on similarity, and finally merges duplicate events based on the event descriptive words and time similarity. This allows for rapid detection of hot events in social text sets, as well as the start and end times of these events, without prior knowledge. This reduces the computational load of abnormal event detection and improves detection speed and efficiency. The time complexity of the abnormal event detection method provided in this disclosure is O(N), where N is the number of words. The algorithm overhead is mainly used to calculate the sum of the maximum continuous time outlier subsequence for each headword. However, the time complexity of LDA is O(K*N*T), where K is the number of topics, N is the number of words in the corpus, and T is the number of iterations. The empirical value for the number of iterations in large-scale corpora is much greater than 1000, making its complexity far higher than that of this method. Therefore, the method in this disclosure can significantly improve detection speed and efficiency. Furthermore, the method provided in this disclosure highlights the central words for easy reading, and the central words are usually named entities, which helps to understand the core participants of the event and has good readability; it can identify events with many central words and long durations and has good compatibility.

[0138] Based on the above-described method for detecting abnormal events, this disclosure also provides a device for detecting abnormal events. The following will be combined with... Figure 8 The device is described in detail.

[0139] Figure 8 A schematic block diagram of an abnormal event detection apparatus according to an embodiment of the present disclosure is shown.

[0140] like Figure 8 As shown, the abnormal event detection device 800 of this embodiment includes a first preset module 810, a first acquisition module 820, a division module 830, a first calculation module 840, a second calculation module 850, a first sorting module 860, a second acquisition module 870, a third acquisition module 880, and a result generation module 890.

[0141] The first preset module 810 is used to preset the historical time span and time slice length. In one embodiment, the first preset module 810 can be used to perform the operation S210 described above, which will not be repeated here.

[0142] The first acquisition module 820 is used to acquire a text set of anomaly events to be detected within the historical time span, wherein the text set includes multiple texts, and each text includes multiple words. In one embodiment, the first acquisition module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0143] The partitioning module 830 is used to divide the historical time span into multiple time slices according to the time slice length. In one embodiment, the partitioning module 830 can be used to perform the operation S230 described above, which will not be repeated here.

[0144] The first calculation module 840 is configured to calculate the sub-anomaly value of each of the plurality of words within each of the plurality of time slices. In one embodiment, the first calculation module 840 may be used to perform the operation S240 described above, which will not be repeated here.

[0145] The second calculation module 850 is used to calculate the maximum consecutive sub-sequence sum of the sub-outliers for each word within the historical time span, based on the sub-outliers, to obtain the outlier and maximum consecutive time span for each word, wherein the maximum consecutive time span includes at least one time slice. In one embodiment, the second calculation module 850 can be used to perform the operation S250 described above, which will not be repeated here.

[0146] The first sorting module 860 is used to sort the plurality of words according to their outlier values ​​from largest to smallest, and select the top M words as center words, wherein the center words are words used to directly describe the abnormal event, and M is an integer greater than or equal to 1. In one embodiment, the first sorting module 860 can be used to perform the operation S260 described above, which will not be repeated here.

[0147] The second acquisition module 870 is used to acquire, for each of the M center words, a descriptive word whose distribution probability is similar to that of the center word within the maximum continuous time span of the center word, wherein the descriptive word is a word used to indirectly describe the abnormal event. In one embodiment, the second acquisition module 870 can be used to perform the operation S270 described above, which will not be repeated here.

[0148] The third acquisition module 880 is used to obtain multiple abnormal events based on the M center words and descriptive words. Each abnormal event includes a center word, a descriptive word, an outlier, and a maximum continuous time span. The outlier and maximum continuous time span of the abnormal event are the outlier and maximum continuous time span of the center word. In one embodiment, the third acquisition module 880 can be used to perform the operation S280 described above, which will not be repeated here.

[0149] The result generation module 890 is used to obtain the detection results of abnormal events in the set of texts to be detected based on the plurality of abnormal events. In one embodiment, the result generation module 890 can be used to perform the operation S290 described above, which will not be repeated here.

[0150] According to an embodiment of this disclosure, the first calculation module includes: a first calculation unit, configured to calculate the actual number and expected number of occurrences of each word in each time slice; and a second calculation unit, configured to subtract the actual number and the expected number to obtain the sub-anomaly value of the word.

[0151] According to an embodiment of this disclosure, the second acquisition module includes: a third calculation unit, configured to calculate the probability distribution similarity between each candidate word (excluding the central word) and the central word within the maximum continuous time span of the central word; a first preset unit, configured to preset a probability distribution similarity threshold; and a selection unit, configured to select candidate words with a probability distribution similarity greater than the probability distribution similarity threshold as descriptors of the central word.

[0152] According to an embodiment of this disclosure, the third calculation unit includes: a first calculation submodule, configured to calculate the probability distribution of each candidate word in each time slice within the maximum continuous time span; a second calculation submodule, configured to calculate the probability distribution of the center word in each time slice within the maximum continuous time span; a third calculation submodule, configured to calculate the probability distribution sub-similarity between each candidate word and the center word in each time slice based on the probability distribution of each candidate word and the probability distribution of the center word; and a fourth calculation submodule, configured to superimpose the probability distribution sub-similarity within the maximum continuous time span to obtain the probability distribution similarity between each candidate word and the center word.

[0153] According to an embodiment of this disclosure, the result generation module includes: a fourth calculation unit, used to calculate the similarity between any two abnormal events among the plurality of abnormal events; and a merging unit, used to merge the plurality of abnormal events based on the similarity calculation result to obtain the detection result of abnormal events in the text set to be detected.

[0154] According to an embodiment of this disclosure, the fourth calculation unit includes: a first merging submodule, used to merge the head word and descriptive word of each of the plurality of abnormal events into a word set; and a fifth calculation submodule, used to calculate the first similarity of the word sets of any two abnormal events among the plurality of abnormal events and the second similarity of the maximum continuous time span.

[0155] According to an embodiment of this disclosure, the merging unit includes: a first preset submodule, used to preset a first similarity threshold and a second similarity threshold; and a second merging submodule, used to merge the two abnormal events into one abnormal event when the first similarity of any two abnormal events among the plurality of abnormal events is greater than the first similarity threshold and the second similarity is greater than the second similarity threshold.

[0156] According to embodiments of this disclosure, the second merging submodule includes: a first merging subunit, used to take the union of the headwords of two abnormal events as the merged headword; a second merging subunit, used to take the union of the descriptive words of two abnormal events as the merged descriptive word; a third merging subunit, used to take the union of the maximum continuous time span of two abnormal events as the merged maximum continuous time span; a fourth merging subunit, used to compare the outlier values ​​of two abnormal events and take the larger outlier value as the merged outlier value; and a first determining subunit, used to obtain the merged abnormal event based on the merged headword, descriptive word, maximum continuous time span, and outlier value.

[0157] According to an embodiment of this disclosure, the first acquisition module includes: a second acquisition unit, configured to acquire a total set of texts of anomaly events to be detected within the historical time span; and an extraction unit, configured to extract a set of texts with topic tags from the total set of texts as the set of texts of anomaly events to be detected, wherein the topic tags are tags used to describe the topics of the texts.

[0158] According to embodiments of this disclosure, any and multiple modules among the first preset module 810, first acquisition module 820, division module 830, first calculation module 840, second calculation module 850, first sorting module 860, second acquisition module 870, third acquisition module 880, and result generation module 890 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the first preset module 810, the first acquisition module 820, the partitioning module 830, the first calculation module 840, the second calculation module 850, the first sorting module 860, the second acquisition module 870, the third acquisition module 880, and the result generation module 890 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or any other reasonable means of integrating or packaging circuits, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations, or in a suitable combination of any of these. Alternatively, at least one of the first preset module 810, the first acquisition module 820, the partitioning module 830, the first calculation module 840, the second calculation module 850, the first sorting module 860, the second acquisition module 870, the third acquisition module 880, and the result generation module 890 can be at least partially implemented as computer program modules, which can perform corresponding functions when the computer program module is run.

[0159] Figure 9 A block diagram of an electronic device suitable for implementing a method for detecting abnormal events according to an embodiment of the present disclosure is shown schematically.

[0160] like Figure 9 As shown, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0161] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 902 and / or RAM 903. It should be noted that the programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0162] According to embodiments of this disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0163] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0164] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903 described above.

[0165] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods of the embodiments of this disclosure.

[0166] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0167] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0168] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0169] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0170] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0171] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0172] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for detecting abnormal events, characterized in that, The method includes: Preset the historical time span and time slice length; Obtain a text set of anomaly events to be detected within the historical time span, wherein the text set includes multiple texts, and each text includes multiple words; The historical time span is divided into multiple time slices according to the length of the time slice; Within each of the plurality of time slices, calculate the sub-outlier value for each of the plurality of words; Based on the sub-outliers, the maximum consecutive sub-sequence sum of the sub-outliers of each word in the historical time span is calculated to obtain the outlier and the maximum consecutive time span of each word, wherein the maximum consecutive time span includes at least one time slice. The multiple words are sorted from largest to smallest according to their outlier values, and the top M words are selected as the center words. The center words are words used to directly describe the abnormal events, and M is an integer greater than or equal to 1. For each of the M central words, obtain the descriptive word whose probability distribution is similar to that of the central word within the maximum continuous time span of the central word, wherein the descriptive word is a word used to indirectly describe the abnormal event; Based on the M center words and descriptive words, multiple anomalous events are obtained, wherein each anomalous event includes a center word, a descriptive word, an outlier, and a maximum continuous time span, and the outlier and maximum continuous time span of the anomalous event are the outlier and maximum continuous time span of the center word; and Based on the aforementioned multiple abnormal events, the detection results of abnormal events in the set of texts to be detected are obtained.

2. The method according to claim 1, characterized in that, The step of calculating the sub-outlier value of each of the plurality of words within each of the plurality of time slices includes: Within each time slice, calculate the actual and expected number of occurrences of each word; and The sub-outlier value of the word is obtained by subtracting the actual number from the expected number.

3. The method according to claim 1, characterized in that, The step of obtaining descriptive words with similar probability distributions to the central word within the maximum continuous time span of the central word includes: Within the maximum continuous time span of the central word, calculate the probability distribution similarity between each candidate word (excluding the central word) and the central word. Preset distribution probability similarity threshold; and Candidate words with a probability similarity greater than the probability similarity threshold are selected as descriptors of the center word.

4. The method according to claim 3, characterized in that, The step of calculating the probability similarity between each candidate word (excluding the center word) and the center word includes: Calculate the probability distribution of each candidate word in each time slice within the maximum continuous time span; Calculate the distribution probability of the central word in each time slice within the maximum continuous time span; Based on the probability distribution of each candidate word and the probability distribution of the center word, calculate the probability distribution sub-similarity between each candidate word and the center word in each time slice; and The probability distribution sub-similarity is superimposed over the maximum continuous time span to obtain the probability distribution similarity between each candidate word and the center word.

5. The method according to claim 1, characterized in that, The step of obtaining the detection results of abnormal events in the set of texts to be detected based on the multiple abnormal events includes: Calculate the similarity between any two abnormal events among the plurality of abnormal events; and Based on the similarity calculation results, the multiple abnormal events are merged to obtain the detection results of abnormal events in the set of texts to be detected.

6. The method according to claim 5, characterized in that, The step of calculating the similarity between any two abnormal events among the plurality of abnormal events includes: The headword and descriptive word of each of the multiple abnormal events are merged into a word set; and Calculate the first similarity of the word sets and the second similarity of the maximum continuous time span between any two abnormal events among the plurality of abnormal events.

7. The method according to claim 6, characterized in that, The step of merging the multiple abnormal events based on the similarity calculation results includes: Preset a first similarity threshold and a second similarity threshold; and When the first similarity of any two abnormal events among the plurality of abnormal events is greater than the first similarity threshold and the second similarity is greater than the second similarity threshold, the two abnormal events are merged into one abnormal event.

8. The method according to claim 7, characterized in that, The step of merging the two abnormal events into one abnormal event includes: Take the union of the central words of the two abnormal events and use it as the central word of the merged event; Take the union of the descriptors of the two abnormal events and use it as the merged descriptor; The union of the maximum continuous time spans of the two abnormal events is taken as the maximum continuous time span after merging. Compare the outliers of two abnormal events and use the larger outlier as the merged outlier; and Based on the merged headword, descriptive word, maximum continuous time span, and outlier, the merged abnormal event is obtained.

9. The method according to claim 1, characterized in that, The step of obtaining the text set of the abnormal events to be detected within the historical time span includes: Obtain the total text set of the anomaly events to be detected within the stated historical time span; and Extract a set of texts with topic tags from the overall text set, and use this set as the text set of the abnormal events to be detected, wherein the topic tags are tags used to describe the topics of the texts.

10. An abnormal event detection device, comprising: The first preset module is used to preset the historical time span and time slice length; The first acquisition module is used to acquire a text set of anomaly events to be detected within the historical time span, wherein the text set includes multiple texts, and each text includes multiple words; The segmentation module is used to divide the historical time span into multiple time slices according to the length of the time slice; The first calculation module is used to calculate the sub-anomaly value of each word among the multiple words in each of the multiple time slices; The second calculation module is used to calculate the maximum consecutive sub-sequence sum of the sub-outliers of each word in the historical time span based on the sub-outliers, so as to obtain the outlier and the maximum consecutive time span of each word, wherein the maximum consecutive time span includes at least one time slice. The first sorting module is used to sort the multiple words according to the outlier value from largest to smallest, and select the first M words as the center words, wherein the center words are words used to directly describe the abnormal event, and M is an integer greater than or equal to 1; The second acquisition module is used to acquire, for each of the M central words, a descriptive word whose distribution probability is similar to that of the central word within the maximum continuous time span of the central word, wherein the descriptive word is a word used to indirectly describe the abnormal event; The third acquisition module is used to obtain multiple abnormal events based on the M center words and descriptive words, wherein each abnormal event includes a center word, a descriptive word, an outlier, and a maximum continuous time span, and the outlier and maximum continuous time span of the abnormal event are the outlier and maximum continuous time span of the center word; and The result generation module is used to obtain the detection results of abnormal events in the set of texts to be detected based on the multiple abnormal events.

11. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images