Semantic similarity-based event association

Abstract

The present disclosure proposes a method, an apparatus, and a computer program product for event association. A current event vector for the current event may be calculated. A set of historical event vectors corresponding to the set of historical events may be obtained. A similar event set similar to the current event can be searched from the historical event set according to the current event vector and the historical event vector set. Association criteria in a natural language form may be received. A cue word may be created based on the current event, the set of similar events, and the association criteria. According to the method, an association result for the current event can be generated based on the cue word through a large language model, and the association result indicates that the current event is associated with at least one similar event in the similar event set. The invention further provides an event association system. The event association system may include an event search engine, an association decision engine, and an association operation engine.

Description

Background Art

[0001] Today, people can access and use a variety of services via the internet using devices such as desktop computers, tablets, and smartphones. In this context, a service refers to an application or function that relies on computer and network technologies. Examples of services include cloud computing, video services, search services, and conferencing services. Services are operated and managed by service providers. To ensure reliable service, service providers continuously monitor various service performance indicators to promptly identify and resolve any service issues. Summary of the Invention

[0002] This summary is provided to introduce a group of concepts that will be further described in the following detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0003] The embodiments of the present disclosure propose a method, apparatus and computer program product for event association. A current event vector of a current event can be calculated. A set of historical event vectors corresponding to a set of historical events can be obtained. Based on the current event vector and the set of historical event vectors, a set of similar events similar to the current event can be searched from the set of historical events. Association criteria in natural language form can be received. Prompt words can be created based on the current event, the set of similar events and the association criteria. An association result for the current event can be generated based on the prompt word through a large language model, and the association result indicates that the current event is associated with at least one similar event in the set of similar events. The embodiments of the present disclosure also propose an event association system. The event association system may include an event search engine, an association decision engine and an association operation engine.

[0004] It should be noted that one or more of the above aspects include features that are described in detail below and particularly pointed out in the claims. The following description and drawings set forth in detail certain illustrative features of the one or more aspects. These features are merely indicative of various ways in which the principles of the various aspects may be employed, and the present disclosure is intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The disclosed aspects will be described below with reference to the accompanying drawings, which are provided to illustrate rather than limit the disclosed aspects.

[0006] Figure 1 An exemplary process for event correlation according to an embodiment of the present disclosure is shown.

[0007] Figure 2An exemplary process for searching a similar event set similar to a current event from a historical event set according to an embodiment of the present disclosure is shown.

[0008] Figure 3 Another exemplary process for searching a similar event set similar to a current event from a historical event set according to an embodiment of the present disclosure is shown.

[0009] Figure 4 An exemplary process for generating a correlation result for a current event according to an embodiment of the present disclosure is shown.

[0010] Figure 5 An exemplary process for verifying the association result according to an embodiment of the present disclosure is shown.

[0011] Figure 6 is a flowchart of an exemplary method for event correlation according to an embodiment of the present disclosure.

[0012] Figure 7 An exemplary event correlation system according to an embodiment of the present disclosure is shown.

[0013] Figure 8 An exemplary apparatus for event correlation according to an embodiment of the present disclosure is shown.

[0014] Figure 9 Another exemplary apparatus for event correlation according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0015] The present disclosure will now be discussed with reference to several exemplary embodiments. It should be understood that the discussion of these embodiments is only for enabling those skilled in the art to better understand and thereby implement the embodiments of the present disclosure, and does not teach any limitation on the scope of the present disclosure.

[0016] Time series data generated during service operation can be monitored. In this document, time series data refers to a sequence of data recorded in chronological order. The data points in this data sequence reflect the state or degree of change of a specific phenomenon, indicator, or object over time. When the software or hardware used to run the service fails, abnormal data points may appear in the corresponding time series data. When the number and / or value of abnormal data points in the monitored time series data reaches a predetermined threshold, an alert indicating the abnormal situation can be automatically generated and notified to relevant technical personnel via email, text message, phone call, etc. In this document, alerts generated due to software or hardware failures during service operation are referred to as incidents. The specific event information contained in an event depends on pre-defined event generation rules. Accordingly, the event information included in events generated by different failures varies, but generally speaking, each event may include, for example, a title, content, and one or more attribute values. The content may include text or charts describing the event, screenshots containing abnormal data points, etc. Attribute values ​​may be the values ​​of pre-specified attribute fields. Attribute fields may include, for example, the error type and the identifier of the monitor that monitors the time series data. Technical personnel can handle incidents, for example, by analyzing them, locating their causes, and taking action to resolve them. In this article, technical personnel handling incidents are referred to as users. Services can generate a large number of incidents during operation. Handling each incident individually would require significant human and machine resources, resulting in low overall incident handling efficiency. This would make it difficult to resolve service failures in a timely manner.

[0017] Newly generated, unprocessed events may be similar to events that have already been processed or are currently being processed, for example, sharing the same root cause, indicating the same problem, etc. Herein, newly generated, unprocessed events are referred to as current events, and events that have already been processed or are currently being processed are referred to as historical events. Embodiments of the present disclosure propose determining whether a current event can be associated with at least one historical event in a set of historical events based on pre-established association criteria. The association criteria are in the form of natural language. Herein, natural language refers to the language humans use to communicate with each other. Natural language is contrasted with computer languages ​​such as source code languages, object code languages, and computer scripting languages, but it should be understood that there can be some overlap between natural and computer languages. The association criteria can specify conditions for associating two events, such as sharing the same root cause, having the same anomaly, indicating the same problem, and being resolved in the same manner. Alternatively or additionally, the association criteria can specify event information that should be examined and / or event information that should be excluded when generating the association results. If it is determined that the current event can be associated with at least one historical event, the current event can be automatically associated with that historical event. While automatically associating the current event with the historical event, the severity level of the current event can be lowered, and notifications indicating the current event can be suppressed. The technical effect of this approach is to reduce the number of events notified to the user, allowing the user to focus on handling events with high severity levels. When the historical event associated with the current event is resolved, the current event can also be resolved synchronously, thereby improving the overall processing efficiency of the event and speeding up the resolution of faults that affect service operations. Alternatively, if it is determined that the current event can be associated with at least one historical event, a suggestion can be generated to associate the current event with the historical event. The suggestion may include the reason why the current event is associated with the historical event. The suggestion can be provided to the user. The technical effect of this approach is to prompt the user that the current event can be associated with a certain historical event and the reason why the current event is associated with the historical event, so that the user can decide whether to associate the current event with the historical event based on this.

[0018] When a current event is received, a set of similar events similar to the current event can be searched from a set of historical events through semantic similarity matching, or through both time window filtering and semantic similarity matching. Time window filtering aims to identify events whose generation time is close to the generation time of the current event from the set of historical events. Semantic similarity matching aims to search for events with semantic similarity to the current event from the set of historical events. Events that are close to the generation time of the current event and / or semantically similar to the current event are more likely to be associated with the current event, and thus the set of similar events can serve as a set of candidate events that are potentially associated with the current event. A prompt word can be created based on the current event, the set of similar events, and the association criteria. Subsequently, a large language model (LLM) can be used to generate association results for the current event based on the prompt word. In this article, a large language model refers to a deep learning model trained using a large amount of training data that can understand the meaning of natural language, generate natural language text, or perform other natural language tasks. It should be understood that a large language model includes a multimodal model that can perform processing tasks in multiple modalities including natural language. The association result may indicate whether the current event is associated with at least one similar event in the set of similar events. The association result may be in the form of natural language. The large language model may utilize its semantic understanding ability, logical reasoning ability, big data support ability, language expression ability, etc. to semantically match the current event with each similar event in the set of similar events to determine whether the current event and the similar event meet the association criteria. If the large language model determines that the current event and at least one similar event in the set of similar events meet the association criteria, an association result may be generated indicating that the current event is associated with at least one similar event in the set of similar events. In this case, preferably, the association result may include the reason why the current event is associated with the similar event. If the large language model determines that the current event and all similar events in the set of similar events do not meet the association criteria, an association result may be generated indicating that the current event is not associated with any similar event in the set of similar events.

[0019] In the above process, a set of similar events similar to the current event is first searched from the historical event set through semantic similarity matching, or through a combination of time window filtering and semantic similarity matching. Subsequently, the current event is semantically matched with each similar event in the set of similar events using a large language model to determine whether the current event and the similar events meet the association criteria. Based on this, the event association according to the embodiments of the present disclosure can be referred to as event association based on semantic similarity.

[0020] Before generating a correlation result for the current event using the large language model, a search is performed on a set of historical events for a set of similar events similar to the current event, either through semantic similarity matching, or through a combination of time window filtering and semantic similarity matching. Only the searched set of similar events is then provided to the large language model. This approach reduces the number of historical events provided to the large language model and ensures that the historical events provided to the large language model are those that are more likely to be associated with the current event, thereby improving the efficiency of the large language model in generating correlation results and increasing the accuracy of the correlation results. An accurate correlation result can accurately indicate whether the current event can be associated with a certain historical event. If so, the current event can be associated with the historical event, the severity level of the current event can be lowered, and notifications indicating the current event can be suppressed. If the correlation result is incorrect, the severity level of the current event, which should not be associated with a historical event, will be lowered. Furthermore, because the user does not receive a notification indicating the current event, the current event will not be resolved in a timely manner, which will cause the related fault to persist.

[0021] Providing natural language association criteria to the large language model allows for semantically determining whether the current event can be associated with a historical event. Furthermore, natural language association criteria are flexible, applicable to a wide range of events and scenarios, and easy for users to understand and customize. The technical benefit of this approach is to increase the event association rate, reduce the number of notifications sent to users, and ensure the accuracy of the association results.

[0022] In order to further improve the accuracy of the association results, the embodiments of the present disclosure also propose that when creating prompt words, in addition to considering the current event, the set of similar events and the association criteria, the user's feedback on the historical association results previously generated by the large language model is also considered. The user can be the user who handles the current event or other users. Feedback can include positive feedback and / or negative feedback. The user's positive feedback on the association result can indicate that the user believes that the association result is correct. The user's negative feedback on the association result can indicate that the user believes that the association result is wrong. The prompt word is the input used by the large language model to generate the association result. The technical effect of additionally considering the user's feedback on the historical association results when creating the prompt word is to enable the large language model to learn from the feedback the aspects that it can refer to and the aspects that need to be improved when generating association results in the past, thereby improving its accuracy in generating association results for the current event.

[0023] Generative language models such as large language models sometimes produce meaningless or unreliable outputs, and this problem is called the hallucination problem. The embodiments of the present disclosure propose to use a large language model to verify the correctness of the association results generated by it. For example, when the association result generated by the large language model indicates that the current event is associated with at least one historical event, the association result and the event information of the current event and the historical event can be input into the large language model. The large language model can determine whether the association result is valid by performing logical reasoning. If not, the association result is considered to be wrong, that is, the current event should not be associated with the historical event. In this case, the association action for the current event will not be performed. The technical effect of this method is to avoid executing erroneous association actions and ensure that the current event is only associated with historical events that can indeed be associated.

[0024] Various embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0025] Figure 1 An exemplary process 100 for event correlation according to an embodiment of the present disclosure is shown. The process 100 may be performed by an event correlation system 110 according to an embodiment of the present disclosure.

[0026] A current event 102 may be obtained. The current event 102 may be a newly generated, yet-to-be-processed event. Preferably, the current event 102 may be an event with a higher severity level, i.e., a more serious event. The current event 102 may include current event information. Herein, the event information of the current event is referred to as current event information. The current event information may include at least one of the title, content, and attribute values ​​of the current event 102. The content may include text or charts describing the current event, screenshots containing abnormal data points, and the like. The attribute values ​​may be values ​​of pre-specified attribute fields. The attribute fields may include, for example, an error type, a monitor identifier, and the like.

[0027] The historical event collection 104 may include multiple historical events that have been processed or are being processed. Each historical event in the historical event collection 104 may have historical event information. Herein, the event information of a historical event is referred to as historical event information. The historical event information may include at least one of the title, content, attribute values, and processing results of the historical event. The processing results of a historical event may include the processing history for the event, such as the identified cause of the event, how the event was resolved, and so on.

[0028] The event correlation system 110 may include an event search engine 120. The event search engine 120 may search the historical event set 104 for a similar event set 122 that is similar to the current event 102. It should be understood that similar events in the similar event set 122 are historical events. The similar event set 122 may be searched in a variety of ways.

[0029] In one embodiment, a similar event set 122 can be searched by semantic similarity matching. Semantic similarity matching aims to search for events that are semantically similar to the current event from the historical event set. For example, the current event vector of the current event 102 can be calculated. In this article, the vector of the current event is referred to as the current event vector. A set of historical event vectors corresponding to the historical event set 104 can be obtained. In this article, the vector of the historical event is referred to as the historical event vector. Subsequently, based on the current event vector and the set of historical event vectors, a similar event set 122 that is similar to the current event 102 can be searched from the historical event set 104. For example, the similar event set 122 can be searched by performing semantic similarity matching on the current event vector and the set of historical event vectors. This will be discussed later in conjunction with Figure 2 hereinafter is to illustrate an exemplary process for searching for similar event sets through semantic similarity matching.

[0030] In another embodiment, the similar event set 122 can be searched by both time window filtering and semantic similarity matching. Time window filtering aims to identify events whose generation time is close to the generation time of the current event from the historical event set. For example, the current event vector of the current event 102 can be calculated. A historical event vector set corresponding to the historical event set 104 can be obtained. A subset of historical events whose generation time is close to the generation time of the current event 102 can be identified from the historical event set 104, and a subset of historical event vectors corresponding to the subset of historical events can be extracted from the set of historical event vectors. Subsequently, a similar event set 122 similar to the current event 102 can be searched from the subset of historical events by performing semantic similarity matching on the current event vector and the subset of historical event vectors. This will be discussed later in conjunction with Figure 3 An exemplary process for searching for similar event sets through both time window filtering and semantic similarity matching is described below.

[0031] The event correlation system 110 may also include a correlation decision engine 130. The correlation decision engine 130 may receive correlation criteria 124 in natural language. The correlation criteria 124 may specify conditions under which two events are correlated, such as having the same root cause, having the same anomaly, indicating the same problem, being resolved in the same manner, etc. It should be understood that the above conditions are merely exemplary, and any other conditions may be used to evaluate whether two events are correlated. Alternatively or additionally, the correlation criteria 124 may specify event information that should be examined and / or event information that should be excluded when generating correlation results. As an example, the correlation criteria 124 may specify that the anomaly data points and anomaly messages of each event should be reviewed when generating correlation results. As another example, the correlation criteria 124 may specify that the source address of the reporting event should be excluded when generating correlation results. The correlation criteria 124 may be default correlation criteria provided by the event correlation system 110, or custom correlation criteria provided by the user. Custom correlation criteria may be included in the user's configuration information. The association decision engine 130 can generate an association result 132 for the current event 102 based on the current event 102, the similar event set 122, and the association criteria 124. For example, the association result 132 can be generated by a large language model. The association result 132 can indicate whether the current event 102 can be associated with at least one similar event in the similar event set 122. In the case where the association result 132 indicates that the current event 102 can be associated with at least one similar event in the similar event set 122, preferably, the association result 132 can include the reason why the current event is associated with the similar event. Figure 4 The exemplary process of generating the association result for the current event is described in detail below. The association result 132 can be provided to the user so that the user can clearly know which historical event the current event can be associated with and the reason why the current event is associated with the historical event.

[0032] When the association result 132 indicates that the current event 102 is not associated with any similar event in the similar event set 122, the current event 102 can be processed according to the usual process, such as keeping the severity level of the current event 102 unchanged, sending a notification indicating the current event 102, etc.

[0033] If the correlation result 132 indicates that the current event 102 can be associated with at least one similar event in the similar event set 122, the correlation result 132 can be provided to the correlation operation engine 140 in the event correlation system 110. The correlation operation engine 140 can perform a correlation action 142 for the current event 102 based on the correlation result 132 and the operation configuration 134. The correlation action 142 can include, for example, automatically associating the current event 102 with the at least one similar event, defining the at least one similar event as a parent event of the current event 102, reducing the severity level of the current event 102, suppressing the sending of notifications indicating the current event 102, generating a suggestion to associate the current event 102 with the at least one similar event, and the like. The operation configuration 134 can be included in the user's configuration information. The operation configuration 134 can indicate the correlation action that the correlation operation engine 140 should perform upon receiving the correlation result 132.

[0034] As an example, when the operation configuration 134 indicates that the association operation is allowed to be automatically performed on the current event, the association operation engine 140 can automatically associate the current event 102 with the similar events included in the association result 132. In addition, the association operation engine 140 can define the similar event as the parent event of the current event 102, that is, the current event 102 can be a child event of the similar event. The technical effect of this operation is that when the historical event associated with the current event is resolved, the current event can also be resolved synchronously, thereby improving the overall processing efficiency of the event and speeding up the resolution of faults that affect service operation. In addition, the association operation engine 140 can reduce the severity level of the current event 102 and / or suppress the sending of notifications indicating the current event 102. The technical effect of this operation is to reduce the number of events notified to the user, so that the user can focus on handling events with high severity levels.

[0035] As another example, when the operation configuration 134 indicates that the automatic execution of the association operation for the current event is not allowed, the association operation engine 140 can generate a suggestion to associate the current event 102 with a similar event included in the association result 132. The suggestion can include the reason why the current event 102 is associated with the similar event. The suggestion can be provided to the user. The technical effect of this approach is to prompt the user that the current event can be associated with a historical event and the reason why the current event is associated with the historical event, so that the user can decide whether to associate the current event with the historical event based on this.

[0036] Generative language models such as large language models sometimes produce meaningless or unreliable outputs, which is called the hallucination problem. Preferably, when the association result 132 indicates that the current event 102 is associated with at least one historical event, the association operation engine 140 can first verify the correctness of the association result 132 through the large language model, and only when the association result 132 is correct, execute the association action 142 for the current event 102 according to the association result 132 and the operation configuration 134. Figure 5 An exemplary process for verifying the association results is described below.

[0037] It should be understood that the above Figure 1 The process for event correlation described is only exemplary. According to actual application requirements, the steps in the process for event correlation can be replaced or modified in any manner, and the process can include more or fewer steps. In addition, it should be understood that Figure 1 The event correlation system 110 shown is only an example of an event correlation system. Depending on actual application requirements, the event correlation system can have any other structure and can include more or fewer modules. In addition, each module in the event correlation system 110 can perform other operations in addition to the above operations.

[0038] Figure 2 An exemplary process 200 for searching a similar event set from a historical event set according to an embodiment of the present disclosure is shown. The process 200 may correspond to Figure 1 In the process 200, a similar event set may be searched through semantic similarity matching.

[0039] The current event 202 and the historical event set 204 may correspond to Figure 1 The current event 102 and the historical event collection 104 in .

[0040] The current event vector 222 of the current event 202 can be calculated. The original current event information 206 of the current event 202 can be obtained first. The original current event information 206 can include at least one of the title, content, and attribute value of the current event 202. The original current event information 206 can be provided to the pre-processing module 210. The pre-processing module 210 can splice the various event information into a continuous string to generate pre-processed current event information 212. When the event information contains non-text information such as images, the pre-processing module 210 can convert the non-text information into text information through techniques such as encoding. The pre-processed current event information 212 can be provided to the encoder 220. The encoder 220 can be a machine learning model capable of generating a vector or embedding based on text input. The encoder 220 can generate the current event vector 222 of the current event 202 based on the pre-processed current event information 212. The input of the encoder 220 may be length-limited. Preferably, the pre-processing module 210 can remove information that is not valuable for understanding the semantics of the current event 202, such as format information, style information, etc., from the original current event information 206. The original current event information and the pre-processed current event information can be collectively referred to as current event information. It should be understood that the original current event information and the pre-processed current event information substantially include the title, content, attribute values, etc. of the current event.

[0041] A set of historical event vectors 208 corresponding to the set of historical events 204 may be obtained. The set of historical event vectors 208 may be obtained from a database. The database may be, for example, a database included in the event correlation system or a database accessible to the event correlation system. Each historical event vector in the set of historical event vectors 208 may correspond to a historical event in the set of historical events 204. The historical event vectors may have been previously generated using a technique similar to that used to generate the current event vector 222 and stored in the database.

[0042] Based on the current event vector 222 and the historical event vector set 208, a similar event set 232 similar to the current event 202 can be searched from the historical event set 204. For example, the similarity matching module 230 can perform semantic similarity matching on the current event vector 222 and the historical event vector set 208 to search for the similar event set 122. The similarity matching module 230 can use a technique such as a nearest neighbor search to search the historical event vector set 208 for several historical event vectors closest to the current event vector 222, thereby determining several historical events that are most semantically similar to the current event 202. These several historical events can be combined into a similar event set 232.

[0043] Figure 3 Another exemplary process 300 for searching a set of similar events from a set of historical events according to an embodiment of the present disclosure is shown. The process 300 may correspond to Figure 1 In the process 300, a similar event set may be searched through both time window filtering and semantic similarity matching.

[0044] The current event 302 and the historical event set 304 may correspond to Figure 1 The current event 102 and the historical event collection 104 in .

[0045] A current event vector 322 may be calculated for the current event 302. The process of generating the current event vector 322 may be combined with Figure 2 The process for generating the current event vector 222 is similar to that described above. Original current event information 306 for the current event 302 can be obtained first. Original current event information 306 can include at least one of the title, content, and attribute values ​​of the current event 302. The original current event information 306 can be provided to a pre-processing module 310. The pre-processing module 310 can concatenate the various pieces of event information into a continuous string, thereby generating pre-processed current event information 312. If the event information includes non-text information such as images, the pre-processing module 310 can convert the non-text information into text information using techniques such as encoding. The pre-processed current event information 312 can be provided to an encoder 320. The encoder 320 can generate a current event vector 322 for the current event 302 based on the pre-processed current event information 312. The input to the encoder 320 may be length-limited. Preferably, the pre-processing module 310 can remove information that is not valuable for understanding the semantics of the current event 302, such as formatting information and style information, from the original current event information 306.

[0046] A historical event subset 332 whose generation time is close to the generation time of the current event 302 can be identified from the historical event set 304 by the time window filtering module 330. For example, historical events whose generation time is within a predetermined time interval between the generation time of the current event 302 and the generation time of the historical event set 304 can be identified, and the identified historical events can be combined into the historical event subset 332. A historical event vector subset 334 corresponding to the historical event subset 332 can be extracted from the historical event vector set corresponding to the historical event set 304. Based on the current event vector 322 and the historical event vector subset 334, a similar event set 342 similar to the current event 302 can be searched from the historical event subset 332. For example, a semantic similarity matching can be performed on the current event vector 322 and the historical event vector subset 334 by the similarity matching module 340 to search for the similar event set 342. Similarity matching module 340 can use techniques such as nearest neighbor search to search for several historical event vectors closest to current event vector 322 from historical event vector subset 334, thereby determining several historical events that are most semantically similar to current event 302. These several historical events can be combined into similar event set 342.

[0047] In process 200 or process 300, similar event set 232 or similar event set 342 can be provided to the large language model as a candidate event set with which the current event may be associated. Providing only the similar event set that is similar to the current event, searched from the historical event set, to the large language model, rather than providing the entire historical event set to the large language model, reduces the number of historical events provided to the large language model and ensures that the historical events provided to the large language model are more likely to be associated with the current event, thereby improving the efficiency of the large language model in generating association results and increasing the accuracy of the association results.

[0048] First, through time window filtering, a subset of historical events whose occurrence times are close to the occurrence time of the current event is identified from the historical event set. Then, through semantic similarity matching, events with semantic similarity to the current event are searched from the historical event subset. This can eliminate historical events whose occurrence times are far from the current time from the historical event set. Such events are less likely to be similar to the current event. The technical effect of this method is to reduce the number of historical events provided to the similarity matching module and ensure that the historical events provided to the similarity matching module are events that are more likely to be similar to the current event, thereby improving the search efficiency and quality of the similarity matching module.

[0049] It should be understood that the above Figures 2 to 3The process described for searching a set of similar events from a set of historical events for a set of similar events that is similar to the current event is merely exemplary. The steps in the process for searching a set of similar events from a set of historical events for a set of similar events that is similar to the current event can be replaced or modified in any manner, and the process can include more or fewer steps, depending on actual application requirements. Furthermore, the specific order or hierarchy of steps in process 200 and process 300 is merely exemplary, and the process for searching a set of similar events can be performed in an order different from that described.

[0050] Figure 4 An exemplary process 400 for generating a correlation result for a current event according to an embodiment of the present disclosure is shown. The process 400 may correspond to Figure 1 In the process 400, a prompt word creator 410 may create a prompt word 412 based on at least a current event 402, a set of similar events 404, and a relevance criterion 406. A large language model 420 may generate a relevance result 422 for the current event 402 based on the prompt word 412.

[0051] The current event 402 may correspond to Figure 1 Current Events 102.

[0052] Similar event set 404 can be obtained by Figure 1 Event search engine 120 in , or through Figure 2 or Figure 3 The process shown searches for a similar event set from the historical event set that is similar to the current event 402. The similar event set 404 can be used as a candidate event set that is potentially associated with the current event.

[0053] The association criteria 406 may correspond to Figure 1Correlation criteria 124 in the event correlation system. Correlation criteria 406 can be expressed in natural language. Correlation criteria 406 can specify conditions for correlating two events, such as having the same root cause, having the same anomaly, indicating the same problem, being resolved in the same manner, etc. Alternatively or additionally, correlation criteria 406 can specify event information that should be examined and / or event information that should be excluded when generating correlation results. As an example, correlation criteria 406 can specify that the abnormal data points and abnormal messages of each event should be traversed when generating correlation results. As another example, correlation criteria 406 can specify that the source address of the reporting event should be excluded when generating correlation results. Correlation criteria 406 can be default correlation criteria provided by the event correlation system, or customized correlation criteria provided by the user. Customized correlation criteria can be included in the user's configuration information. Providing correlation criteria in natural language to the large language model can determine, at a semantic level, whether the current event can be associated with a historical event. Furthermore, correlation criteria in natural language are flexible, applicable to a wide range of events and scenarios, and easy for users to understand and customize. The technical effect of this approach is to improve the correlation rate of events, reduce the number of notifications sent to users, and ensure the accuracy of the correlation results.

[0054] A prompt word creator 410 can create a prompt word 412 based on the current event 402, the similar event set 404, and the association criteria 406. Current event information of the current event 402, such as the title, content, attribute values, etc. of the current event 402, can be obtained. The current event information can be the original current event information or the pre-processed current event information. Similar event information of each similar event in the similar event set 404 can be obtained. In this document, the event information of similar events is referred to as similar event information. The similar event information of similar events includes, for example, the title, content, attribute values, processing results, etc. of the similar events. The processing results of similar events can include the processing history for the event, such as the identified causes of the event, how the event was resolved, etc. The technical effect of the similar event information of similar events including the processing results is to provide the large language model with more contextual information about the similar events, thereby promoting the large language model to generate more accurate association results. The current event information of the current event 402 , the similar event information set corresponding to the similar event set 404 , and the association criteria 406 may be combined into a prompt word 412 .

[0055] The large language model 420 can generate an association result 422 for the current event 402 based on the prompt word 412. By way of example only and not limitation, the large language model 420 can be a fourth-generation generative pre-trained transformer (GPT-4) model, a GTP-3.5Turbo model, or the like. The association result 422 can indicate whether the current event 402 is associated with at least one similar event in the similar event set 404. The association result 422 can be in the form of natural language. The large language model 420 can use its semantic understanding ability, logical reasoning ability, big data support ability, language expression ability, etc. to semantically match the current event 402 with each similar event in the similar event set 404 to determine whether the current event 402 and the similar event meet the association criteria 406.

[0056] If large language model 420 determines that current event 402 and at least one similar event in similar event set 404 meet association criteria 406, an association result 422 may be generated indicating that current event 402 is associated with at least one similar event in similar event set 404. For example, association result 422 may include an indication of the similar event associated with the current event, such as an identifier, number, title, etc. of the similar event. Preferably, association result 422 may include a reason why current event 402 is associated with the similar event. Assume that the number of the similar event is "46670799." The correlation result 422 may include the following statement: "The current event can be associated with event 46670799. The current event and event 46670799 both have the same monitor identifier (6d87d-140e-45f2-332f6) and have similar error messages indicating a 'Popssl Probe' availability problem. The error messages of both events indicate a 'request timeout' error and a 'backend authentication timeout' problem. Therefore, the current event can be associated with event 46670799". The correlation result 422 may be provided to the user so that the user can clearly know which historical event the current event can be associated with and the reason why the current event is associated with the historical event.

[0057] If large language model 420 determines that current event 402 and all similar events in set of similar events 404 do not meet association criteria 406 , association result 422 may be generated indicating that current event 402 is not associated with any similar event in set of similar events 404 .

[0058] Preferably, when creating prompt words 412, in addition to considering the current event 402, the set of similar events 404, and the association criteria 406, feedback 408 on historical association results may also be considered. Historical association results may be previously generated by the large language model 420 and may include two events previously determined by the large language model 420 to be associated. Feedback 408 on historical association results may be previously provided by a user. The user may be the user processing the current event or another user. Feedback 408 on historical association results may include positive feedback and / or negative feedback. Positive user feedback on an association result may indicate that the user believes the association result is correct. Negative user feedback on an association result may indicate that the user believes the association result is incorrect. Prompt words serve as input for the large language model to generate association results. The technical effect of additionally considering user feedback on historical association results when creating prompt words is that the large language model can learn from this feedback about aspects it could have drawn upon and areas that need improvement when generating association results in the past, thereby improving its accuracy in generating association results for the current event.

[0059] It should be understood that the above Figure 4 The process for generating the correlation result for the current event described is only exemplary. According to actual application requirements, the steps in the process for generating the correlation result for the current event can be replaced or modified in any way, and the process can include more or fewer steps.

[0060] Generative language models, such as large language models, sometimes produce meaningless or unreliable outputs, a problem known as hallucination. Embodiments of the present disclosure propose using a large language model to verify the correctness of the association results it generates. Figure 5 An exemplary process 500 for verifying the association result according to an embodiment of the present disclosure is shown. The process 500 may be performed by an association operation engine, for example, Figure 1 The process 500 may be performed for example by the association operation engine 140 in Figure 4 The execution is performed based on the association result 422 generated by the large language model 420 in the example.

[0061] Process 500 can begin at 502. At 504, it can be determined whether the correlation result indicates that the current event is correlated with at least one historical event.

[0062] If it is determined at step 504 that the association result does not indicate that the current event is associated with at least one historical event, i.e., the large language model determines that the current event is not associated with any historical event, process 500 may proceed to step 512. At 512, no association action may be performed for the current event. In this case, the current event may be handled according to the usual process, such as maintaining the severity level of the current event unchanged, sending a notification indicating the current event, etc.

[0063] If it is determined at step 504 that the association result indicates that the current event is associated with at least one historical event, process 500 may proceed to step 506. At 506, the correctness of the association result may be verified using a large language model. The large language model used to verify the correctness of the association result may be the same model as the large language model used to generate the association result. The association result and event information of the current event and the historical event may be input into the large language model. The large language model may determine whether the association result is valid by performing logical reasoning. If valid, the association result may be considered correct; if not, the association result may be considered incorrect.

[0064] If it is determined at 508 that the correlation result is correct, process 500 may proceed to step 510. At 510, a correlation action for the current event may be executed. For example, the correlation operation engine may execute a correlation action for the current event based on the correlation result and the operation configuration.

[0065] If it is determined at step 508 that the correlation result is not correct, i.e., the correlation result is erroneous, process 500 may proceed to step 512. At step 512, the correlation action for the current event may not be performed. In this case, the current event may be handled according to the usual process, such as maintaining the severity level of the current event, sending a notification indicating the current event, etc.

[0066] In process 500, if the large language model determines that the association result is invalid through logical reasoning, the association result is considered incorrect, that is, the current event should not be associated with the historical event. In this case, the association action for the current event will not be executed. The technical effect of this approach is to avoid executing incorrect association actions and ensure that the current event is only associated with historical events that can be associated.

[0067] It should be understood that the above Figure 5The process for verifying the correlation results described is merely exemplary. Depending on actual application requirements, the steps in the process for verifying the correlation results may be replaced or modified in any manner, and the process may include more or fewer steps. Furthermore, the specific order or hierarchy of the steps in process 500 is merely exemplary, and the process for verifying the correlation results may be performed in an order different from that described.

[0068] Figure 6 is a flow chart of an exemplary method 600 for event correlation according to an embodiment of the present disclosure.

[0069] At 610 , a current event vector for the current event may be calculated.

[0070] At 620 , a set of historical event vectors corresponding to the set of historical events may be obtained.

[0071] At 630 , a similar event set similar to the current event may be searched from the historical event set based on the current event vector and the historical event vector set.

[0072] At 640 , association criteria in natural language form can be received.

[0073] At 650 , a prompt word may be created based on the current event, the set of similar events, and the association criteria.

[0074] At 660 , a large language model may be used to generate an association result for the current event based on the prompt word, where the association result indicates that the current event is associated with at least one similar event in the set of similar events.

[0075] In one embodiment, the current event may include current event information. The current event information may include at least one of a title, content, and attribute values. Each similar event in the similar event set may include similar event information. The similar event information may include at least one of a title, content, attribute values, and a processing result.

[0076] In one embodiment, method 600 may further include: identifying a subset of historical events from the historical event set whose occurrence time is close to the occurrence time of the current event; and extracting a subset of historical event vectors corresponding to the subset of historical events from the historical event vector set. Searching the historical event set for a set of similar events similar to the current event may include: searching the historical event subset for the set of similar events based on the current event vector and the subset of historical event vectors.

[0077] In one embodiment, the correlation criteria may specify conditions for correlating two events, and the conditions may include at least one of the following: having the same root cause, having the same anomaly, indicating the same problem, and being resolved in the same manner.

[0078] In one embodiment, the current event and each event in the set of similar events may have one or more pieces of event information. The association criteria may specify event information that should be checked and / or event information that should be excluded when generating the association result.

[0079] In one embodiment, method 600 may further include obtaining feedback on historical association results previously generated by the large language model, the feedback including positive feedback and / or negative feedback. Creating the prompt word may include creating the prompt word based on the current event, the set of similar events, the association criteria, and the feedback.

[0080] In one embodiment, the association result may include a reason why the current event is associated with the at least one similar event.

[0081] In one embodiment, method 600 may also include: performing an association action for the current event based on the association result and operation configuration, the association action including at least one of the following: automatically associating the current event to the at least one similar event; defining the at least one similar event as the parent event of the current event; reducing the severity level of the current event; suppressing sending notifications indicating the current event; and generating a recommendation to associate the current event with the at least one similar event.

[0082] Method 600 may further include: verifying the correctness of the association result using the large language model. The executing the association action for the current event includes: in response to the association result being correct, executing the association action according to the association result and the operation configuration.

[0083] It should be understood that the method 600 may also include any other steps / processes for event correlation according to the embodiments of the present disclosure described above.

[0084] Figure 7 An exemplary event correlation system 700 according to an embodiment of the present disclosure is shown.

[0085] The event association system 700 may include: an event search engine 710, used to: calculate the current event vector of the current event, obtain a set of historical event vectors corresponding to the historical event set, and search the historical event set for a set of similar events similar to the current event based on the current event vector and the historical event vector set; and an association decision engine 720, used to: receive association criteria in natural language form, create prompt words based on the current event, the similar event set and the association criteria, and generate an association result for the current event based on the prompt words through a large language model, wherein the association result indicates that the current event is associated with at least one similar event in the similar event set.

[0086] The event association system 700 may also include: an association operation engine 730, which is used to perform association actions for the current event based on the association results and operation configuration, and the association actions include at least one of the following: automatically associating the current event to the at least one similar event; defining the at least one similar event as the parent event of the current event; reducing the severity level of the current event; suppressing the sending of notifications indicating the current event; and generating a recommendation to associate the current event with the at least one similar event.

[0087] It should be understood that the event correlation system 700 may further include any other modules configured for event correlation according to the embodiments of the present disclosure as described above.

[0088] Figure 8 An exemplary apparatus 800 for event correlation according to an embodiment of the present disclosure is shown.

[0089] The device 800 may include: a current event vector calculation module 810 for calculating the current event vector of the current event; a historical event vector set acquisition module 820 for obtaining a historical event vector set corresponding to the historical event set; a similar event set search module 830 for searching the historical event set for a similar event set similar to the current event based on the current event vector and the historical event vector set; an association standard receiving module 840 for receiving an association standard in natural language form; a prompt word creation module 850 for creating a prompt word based on the current event, the similar event set, and the association standard; and an association result generation module 860 for generating an association result for the current event based on the prompt word through a large language model, the association result indicating that the current event is associated with at least one similar event in the similar event set. In addition, the device 800 may also include any other modules configured for event association according to the embodiments of the present disclosure as described above.

[0090] Figure 9 Another exemplary apparatus 900 for event correlation according to an embodiment of the present disclosure is shown.

[0091] The apparatus 900 may include: a processor 910; and a memory 920 storing computer-executable instructions. When executed, the computer-executable instructions may cause the processor 910 to: calculate a current event vector for a current event; obtain a set of historical event vectors corresponding to a set of historical events; search the set of historical events for a set of similar events similar to the current event based on the current event vector and the set of historical event vectors; receive association criteria in natural language form; create a prompt word based on the current event, the set of similar events, and the association criteria; and generate an association result for the current event based on the prompt word using a large language model, the association result indicating that the current event is associated with at least one similar event in the set of similar events.

[0092] In one embodiment, when executed, the computer-executable instructions may further cause the processor 910 to: identify a subset of historical events from the historical event set whose occurrence time is close to the occurrence time of the current event; and extract a subset of historical event vectors corresponding to the subset of historical events from the historical event vector set. Searching the historical event set for a set of similar events similar to the current event may include searching the historical event subset for the set of similar events based on the current event vector and the subset of historical event vectors.

[0093] In one embodiment, the correlation criteria may specify conditions for correlating two events, and the conditions may include at least one of the following: having the same root cause, having the same anomaly, indicating the same problem, and being resolved in the same manner.

[0094] In one embodiment, the current event and each event in the set of similar events may have one or more pieces of event information. The association criteria may specify event information that should be checked and / or event information that should be excluded when generating the association result.

[0095] In one embodiment, when executed, the computer-executable instructions may further cause the processor 910 to obtain feedback on historical association results previously generated by the large language model, the feedback including positive feedback and / or negative feedback. Creating the prompt word may include creating the prompt word based on the current event, the set of similar events, the association criteria, and the feedback.

[0096] In one embodiment, the association result may include a reason why the current event is associated with the at least one similar event.

[0097] In one embodiment, when the computer-executable instructions are executed, they may also cause the processor 910 to: perform an association action for the current event based on the association result and the operation configuration, the association action including at least one of the following: automatically associating the current event to the at least one similar event; defining the at least one similar event as the parent event of the current event; reducing the severity level of the current event; suppressing sending notifications indicating the current event; and generating a suggestion to associate the current event with the at least one similar event.

[0098] When executed, the computer-executable instructions may further cause the processor 910 to verify the correctness of the association result using the large language model. The performing of the association action for the current event includes: in response to the association result being correct, performing the association action according to the association result and the operation configuration.

[0099] It should be understood that the processor 910 may also execute any other steps / processes of the method for event correlation according to the embodiments of the present disclosure described above.

[0100] An embodiment of the present disclosure proposes a computer program product for event association, including a computer program, which is executed by a processor to: calculate a current event vector of a current event; obtain a set of historical event vectors corresponding to a set of historical events; search the historical event set for a set of similar events similar to the current event based on the current event vector and the set of historical event vectors; receive association criteria in natural language form; create prompt words based on the current event, the set of similar events, and the association criteria; and generate an association result for the current event based on the prompt words through a large language model, the association result indicating that the current event is associated with at least one similar event in the set of similar events. In addition, the computer program can also be executed to implement any other steps / processes of the method for event association according to the embodiment of the present disclosure as described above.

[0101] Embodiments of the present disclosure may be embodied in a computer-readable medium for event association. The computer-readable medium may include instructions that, when executed, cause the processor to: calculate a current event vector for a current event; obtain a set of historical event vectors corresponding to a set of historical events; search the set of historical events for a set of similar events similar to the current event based on the current event vector and the set of historical event vectors; receive association criteria in natural language form; create prompt words based on the current event, the set of similar events, and the association criteria; and generate an association result for the current event based on the prompt words through a large language model, the association result indicating that the current event is associated with at least one similar event in the set of similar events. In addition, when executed, the instructions may also cause the processor to perform any other steps / processes of the method for event association according to the embodiments of the present disclosure as described above.

[0102] It should be understood that all operations in the methods described above are merely exemplary, and the present disclosure is not limited to any operations or the order of these operations in the methods, but should cover all other equivalent transformations under the same or similar concepts. In addition, unless otherwise specified or it is clear from the context that they are directed to a singular form, the articles "a" and "an" as used in this specification and the appended claims should generally be interpreted as meaning "one" or "one or more".

[0103] It should also be understood that all modules in the above-described device can be implemented in various ways. These modules can be implemented as hardware, software, or a combination thereof. In addition, any module in these modules can be further divided into submodules or combined together in function.

[0104] Processors have been described in conjunction with various devices and methods. These processors can be implemented using electronic hardware, computer software or any combination thereof. Whether these processors are implemented as hardware or software will depend on specific application and the overall design constraints imposed on the system. As an example, the processor provided in this disclosure, any part of the processor or any combination of processors can utilize microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic units, discrete hardware circuits and other suitable processing components configured for performing the various functions described in this disclosure to implement. The function of the processor provided in this disclosure, any part of the processor or any combination of processors can utilize the software performed by microprocessors, microcontrollers, DSPs or other suitable platforms to implement.

[0105] Software should be broadly considered to mean instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, running threads, processes, functions, etc. Software can reside in a computer-readable medium. A computer-readable medium can include, for example, a memory, which can be, for example, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic stripe), an optical disk, a smart card, a flash memory device, a random access memory (RAM), a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a register, or a removable disk. Although the memory is shown as being separated from the processor in the various aspects provided in the present disclosure, the memory can also be located inside the processor, such as a cache or a register.

[0106] The above description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein. All structural and functional equivalents of the elements of the various aspects described in this disclosure that are known or to be known to those of ordinary skill in the art are expressly incorporated herein and covered by the claims.

Claims

1. A method for event correlation, comprising: Calculate the current event vector of the current event; Obtaining a set of historical event vectors corresponding to the set of historical events; Searching, from the historical event set, for a similar event set similar to the current event based on the current event vector and the historical event vector set; receiving the association criteria in natural language form; Creating prompt words based on the current event, the set of similar events, and the association criteria; as well as By using a large language model, based on the prompt word, an association result for the current event is generated, where the association result indicates that the current event is associated with at least one similar event in the set of similar events.

2. The method according to claim 1, wherein: The current event has current event information, and the current event information includes at least one of a title, content, and attribute value, and / or Each similar event in the similar event set has similar event information, and the similar event information includes at least one of a title, content, an attribute value, and a processing result.

3. The method according to claim 1, further comprising: Identifying, from the set of historical events, a subset of historical events whose generation times are close to the generation time of the current event; as well as Extracting a subset of historical event vectors corresponding to the subset of historical events from the set of historical event vectors, and The step of searching the historical event set for a similar event set that is similar to the current event includes: According to the current event vector and the historical event vector subset, the similar event set is searched from the historical event subset.

4. The method according to claim 1, wherein The correlation criteria specify conditions for two events to be correlated, the conditions including at least one of: having the same root cause, having the same exception, indicating the same problem, and being resolved in the same manner.

5. The method according to claim 1, wherein Each event in the current event and the set of similar events has one or more items of event information, and the association criteria specify event information that should be checked and / or event information that should be excluded when generating the association result.

6. The method according to claim 1, further comprising: Obtaining feedback on historical association results previously generated by the large language model, wherein the feedback includes positive feedback and / or negative feedback, and The creation of prompt words includes: The prompt word is created based on the current event, the set of similar events, the association criteria and the feedback.

7. The method according to claim 1, wherein The association result includes a reason why the current event is associated with the at least one similar event.

8. The method according to claim 1, further comprising: Execute an associated action for the current event according to the association result and the operation configuration, where the associated action includes at least one of the following: automatically associating the current event with the at least one similar event; defining the at least one similar event as a parent event of the current event; reducing the severity level of the current incident; suppressing sending of notifications indicating the current event; as well as A suggestion is generated that relates the current event to the at least one similar event.

9. The method according to claim 8, further comprising: Verify the correctness of the association result through the large language model, and The executing of the associated action for the current event includes: In response to the association result being correct, the association action is executed according to the association result and the operation configuration.

10. An event correlation system comprising: Event search engine for: Calculate the current event vector for the current event, Obtain a set of historical event vectors corresponding to the set of historical events, Searching, from the historical event set, for a similar event set similar to the current event based on the current event vector and the historical event vector set; as well as A correlation decision engine for: receiving association criteria in natural language form, Creating prompt words based on the current event, the set of similar events, and the association criteria, and By using a large language model, based on the prompt word, an association result for the current event is generated, where the association result indicates that the current event is associated with at least one similar event in the set of similar events.

11. The event correlation system according to claim 10, further comprising: and a correlation operation engine, configured to execute a correlation action for the current event according to the correlation result and the operation configuration, wherein the correlation action includes at least one of the following: automatically associating the current event with the at least one similar event; defining the at least one similar event as a parent event of the current event; reducing the severity level of the current incident; suppressing sending of notifications indicating the current event; as well as A suggestion is generated that relates the current event to the at least one similar event.

12. An apparatus for event correlation, comprising: processor; as well as a memory storing computer-executable instructions that, when executed, cause the processor to: Calculate the current event vector for the current event, Obtain a set of historical event vectors corresponding to the set of historical events, According to the current event vector and the historical event vector set, searching the historical event set for a similar event set that is similar to the current event, receiving association criteria in natural language form, Creating prompt words based on the current event, the set of similar events, and the association criteria, and By using a large language model, based on the prompt word, an association result for the current event is generated, where the association result indicates that the current event is associated with at least one similar event in the set of similar events.

13. The device according to claim 12, wherein The computer executable instructions, when executed, further cause the processor to: Identifying, from the set of historical events, a subset of historical events whose generation times are close to the generation time of the current event; and Extracting a subset of historical event vectors corresponding to the subset of historical events from the set of historical event vectors, and The step of searching the historical event set for a similar event set that is similar to the current event includes: According to the current event vector and the historical event vector subset, the similar event set is searched from the historical event subset.

14. The device according to claim 12, wherein The correlation criteria specify conditions for two events to be correlated, the conditions including at least one of: having the same root cause, having the same exception, indicating the same problem, and being resolved in the same manner.

15. The device according to claim 12, wherein Each event in the current event and the set of similar events has one or more items of event information, and the association criteria specify event information that should be checked and / or event information that should be excluded when generating the association result.

16. The device according to claim 12, wherein The computer executable instructions, when executed, further cause the processor to: Obtaining feedback on historical association results previously generated by the large language model, wherein the feedback includes positive feedback and / or negative feedback, and The creation of prompt words includes: The prompt word is created based on the current event, the set of similar events, the association criteria and the feedback.

17. The device according to claim 12, wherein The association result includes a reason why the current event is associated with the at least one similar event.

18. The device according to claim 12, wherein The computer executable instructions, when executed, further cause the processor to: Execute an associated action for the current event according to the association result and the operation configuration, where the associated action includes at least one of the following: automatically associating the current event with the at least one similar event; defining the at least one similar event as a parent event of the current event; reducing the severity level of the current incident; suppressing sending of notifications indicating the current event; as well as A suggestion is generated that relates the current event to the at least one similar event.

19. The device according to claim 18, wherein The computer executable instructions, when executed, further cause the processor to: Verify the correctness of the association result through the large language model, and The executing of the associated action for the current event includes: In response to the association result being correct, the association action is executed according to the association result and the operation configuration.

20. A computer program product for event correlation, comprising a computer program, the computer program being executed by a processor to: Calculate the current event vector of the current event; Obtaining a set of historical event vectors corresponding to the set of historical events; Searching, from the historical event set, for a similar event set similar to the current event based on the current event vector and the historical event vector set; receiving the association criteria in natural language form; Creating prompt words based on the current event, the set of similar events, and the association criteria; as well as By using a large language model, based on the prompt word, an association result for the current event is generated, where the association result indicates that the current event is associated with at least one similar event in the set of similar events.

Images