Complex event detection method and system based on data analysis

Abstract

The present invention relates to the technical field of complex event query optimization, and in particular to a complex event detection method and system based on data analysis. The method obtains a comprehensive radiation index based on the duration of a basic event of the type to which each basic time belongs and the spatial distribution change of an insider, and uses the same to adjust the matching window to obtain an adjusted matching window; obtains an event source based on the degree of association between the comprehensive radiation index of the basic time and the basic time in its adjusted matching window; adjusts the original distance measurement between the basic event and the event source based on the difference in the degree of association between any two basic events and the same event source, and then performs complex event detection on the basic events in the overall event stream. The invention adjusts the matching window and screens the event source based on the spatial distribution change and duration of the insider of the basic event, thereby improving the matching accuracy during complex event detection.

Description

Technical Field

[0001] The present invention relates to the technical field of complex event query optimization, and in particular to a complex event detection method and system based on data analysis. Background Art

[0002] Complex event detection method is an automated detection and analysis method involving events. It is mainly used to identify complex events with specific semantics and context from large-scale data streams. This method regards system data as different types of events, analyzes the relationship between events, establishes a relationship sequence library of different events, and uses filtering, association and aggregation techniques to ultimately generate high-level events or business processes from simple events. It is usually used in real-time risk management, real-time transaction analysis, online fraud, network attacks, market trend analysis and other fields.

[0003] The processing model based on nondeterministic finite automaton (NFA) is the most popular implementation of complex event processing (CEP) technology. In complex event processing, the NFA processing model defines a matching mechanism: when matching each specific event on the event stream, a temporary matching event is generated to ensure that the temporary matching event can be used by subsequent new events for matching, thereby generating new temporary matching events and the final complete matching results. When the event complexity is large and the event stream time scale spans a large number of overlapping temporary matching events, the matching mechanism process will generate a large number of overlapping temporary matching events. Since NFA needs to maintain a large number of temporary matching states and complex state transitions during the processing process, it is easy to have a large number of repeated operations and redundant calculations, causing traditional methods to misjudge the complexity of specific event streams and classify simple events as complex events, resulting in low matching accuracy of the overall system when processing complex event detection, thus affecting the final complex event detection results. Summary of the invention

[0004] In order to solve the technical problem that the complexity of event streams and the large time scale span lead to low matching accuracy during complex event detection, the purpose of the present invention is to provide a complex event detection method and system based on data analysis. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a complex event detection method based on data analysis, the method comprising:

[0006] Obtaining the duration of each basic event in the overall event stream of the basic event database, where the overall event stream contains different types of basic events;

[0007] Selecting any basic event from the overall event stream as the target event; obtaining a comprehensive radiation index of the target event according to the duration of the basic events of the type to which the target event belongs and the spatial distribution change of the informants of the basic event; adjusting the matching window in the pattern matching process of the target event based on the comprehensive radiation index to obtain an adjusted matching window of the target event;

[0008] Filter the event sources in the overall event stream according to the comprehensive radiation index of each basic event in the overall event stream and the degree of association between the basic event and the basic event in the adjustment matching window;

[0009] According to the difference in the degree of association between any two basic events in the overall event stream and the same event source, the original distance measurement between the basic events and each event source in the pattern matching process is adjusted, and the adjusted original distance measurement is used to perform complex event detection on the basic events in the overall event stream.

[0010] Furthermore, the method for obtaining the comprehensive radiation index of the target event includes:

[0011] According to the spatial changes of the positions of the insiders at the start and end times of the preset fermentation time period of the target event, the spatial change index of the target event is obtained;

[0012] According to the duration of the target event and the spatial variation index, the local radiation index of the target event is obtained; the mean of the local radiation index of the remaining basic events of the type to which the target event belongs except the target event is normalized to obtain the comprehensive radiation index of the target event.

[0013] Furthermore, the method for obtaining the spatial variation index of the target event includes:

[0014] The start and end time of the preset fermentation time period of the target event are recorded as analysis time. For each analysis time, based on the distance between the position of each insider at the target event at the analysis time and the occurrence position of the target event, the positions of all insiders at the target event at the analysis time are arranged in order to obtain the analysis sequence of the target event at the analysis time.

[0015] A three-dimensional coordinate system is established with the longitude of the insider's position at the target event at the analysis time as the horizontal axis, the latitude of the insider's position at the analysis time as the vertical axis, and time as the vertical axis; the elements in the analysis sequence of the target event at the start time are marked in the three-dimensional coordinate system to obtain the start coordinate points of the corresponding elements, and the elements in the analysis sequence at the end time are marked in the three-dimensional coordinate system to obtain the end coordinate points of the corresponding elements; the start coordinate points and the end coordinate points in the three-dimensional coordinate system are randomly connected for different times to obtain the analysis line segments of each start coordinate point in each connection; the cumulative sum of the lengths of the analysis line segments of all the start coordinate points in each connection is used as the judgment basis for each connection;

[0016] The analysis line segment corresponding to the minimum judgment criterion is recorded as the target line segment; the angle between each target line segment and the reference plane is negatively correlated to obtain the reference value of each target line segment; the reference plane is parallel to the plane formed by the horizontal axis and the vertical axis of the three-dimensional coordinate system, and passes through the end time on the vertical axis of the three-dimensional coordinate system;

[0017] The average of the baseline values ​​of all target line segments is recorded as the spatial variation index of the target event.

[0018] Furthermore, the method for adjusting the matching window of the target event includes:

[0019] Using the event pattern detection method, event pattern matching is performed on the target event and the other basic events in the overall event flow to obtain the matching window of the target event in the matching process; using the sum of the comprehensive radiation index of the target event and the constant 1, the size of the matching window of the target event is weighted to obtain the adjusted matching window of the target event.

[0020] Furthermore, the method for screening event sources in the overall event stream includes:

[0021] The probability of occurrence of the benchmark event of the type to which each basic event belongs in the basic events of the overall event flow is recorded as the occurrence probability of each basic event; in the process of pattern matching of the target event, any basic event is selected from the adjustment matching window of the target event and recorded as the analysis event, and the occurrence probabilities of the target event and the analysis event are substituted into the joint entropy formula to obtain the local correlation index between the target event and the analysis event;

[0022] Negative correlation mapping is performed between the target event and the mean values ​​of the local correlation indexes of all basic events in the adjustment matching window of the target event to obtain a comprehensive correlation index of the target event;

[0023] According to the comprehensive correlation index and the comprehensive radiation index of the target event, the network centrality index of the target event is obtained; among the basic events of the overall event flow, the basic events greater than a preset judgment threshold are taken as event sources in the overall event flow.

[0024] Furthermore, the method for performing complex event detection on basic events in the overall event stream using the adjusted original distance metric includes:

[0025] Select any two basic events from the overall event flow as the events to be tested, and select any event source as the target source;

[0026] The absolute values ​​of the differences between the local correlation indicators of the two events to be tested and the target source are recorded as correlation difference coefficients between the two events to be tested and the target source;

[0027] Using the event pattern detection method, each event to be tested is matched with the target source to obtain the original distance measurement between each event to be tested and the target source; using the correlation difference coefficient to adjust the original distance measurement to obtain the corrected distance measurement between each event to be tested and the target source;

[0028] For each event source, according to the modified distance metric between each basic event in the overall event stream and the event source, a complex event detection is performed on the basic events in the overall event stream using an event pattern detection method.

[0029] Furthermore, the preset fermentation time period is 3 days.

[0030] Furthermore, the preset judgment threshold takes an empirical value of 0.6.

[0031] Furthermore, the event pattern detection method is a non-deterministic finite state automaton method.

[0032] In a second aspect, another embodiment of the present invention provides a complex event detection system based on data analysis, the system comprising:

[0033] A data acquisition module is used to obtain the duration of each basic event in the overall event stream of the basic event database, where the overall event stream contains different types of basic events;

[0034] A matching window adjustment module is used to select any basic event from the overall event stream and record it as a target event; obtain a comprehensive radiation index of the target event according to the duration of the basic events of the type to which the target event belongs and the spatial distribution change of the informants of the basic event; adjust the matching window in the pattern matching process of the target event based on the comprehensive radiation index to obtain an adjusted matching window of the target event;

[0035] An event source selection module, used to screen event sources in the overall event stream according to the comprehensive radiation index of each basic event in the overall event stream and the degree of association between the basic event and the basic event in the adjustment matching window;

[0036] The complex event detection optimization module is used to adjust the original distance measurement between the basic events and each event source in the pattern matching process according to the difference in the degree of association between any two basic events in the overall event flow and the same event source, and use the adjusted original distance measurement to perform complex event detection on the basic events in the overall event flow.

[0037] The present invention has the following beneficial effects:

[0038] In an embodiment of the present invention, when the event complexity is large and the event stream has a large time scale, a large number of overlapping temporary matching events will be generated in the matching mechanism process of the complex event processing method. The time span of the event is analyzed by the duration of the basic event, and the spatial span of the event is analyzed by the spatial distribution change of the informants of the basic event. The two can be combined to obtain the radiation degree of the basic event to the time and space range to different degrees, that is, the comprehensive radiation index; and the size of the matching window in the event pattern matching process is adjusted using the comprehensive radiation index to obtain the adjusted matching window, so as to accurately capture the full impact range of the basic event and reduce the influence of the factors of event complexity and large time scale span on event matching; the event source and the basic event stream are combined to obtain the comprehensive radiation index. If there is a strong correlation between the events, the degree of correlation between the basic events in the overall event flow and the event flow can be used, combined with the comprehensive radiation indicators that reflect the importance and centrality of the basic events in the overall event flow, to screen the event sources in the overall event flow; when detecting complex events, the distance between the basic events and the event sources is usually analyzed, and the screened event sources are used for subsequent analysis to further improve the accuracy of complex event extraction; then, the difference in the degree of correlation between the two basic events and the same event source is used to adjust the distance between the basic event and the event source, and perform complex event detection, thereby improving the matching accuracy of the overall system when processing complex event detection, and effectively extracting more accurate and reliable complex event streams in the overall event flow. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1A flowchart of a complex event detection method based on data analysis provided by an embodiment of the present invention;

[0041] Figure 2 A flowchart of a method for obtaining a comprehensive radiation index provided by an embodiment of the present invention;

[0042] Figure 3 A structural diagram of a complex event detection system based on data analysis provided by one embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a computer device for a complex event detection device based on data analysis provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the complex event detection method and system based on data analysis proposed by the present invention, its specific implementation method, structure, features and effects are described in detail as follows in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0045] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0046] Specific scenarios targeted by the present invention: This case targets traditional complex event detection methods. When the complexity of the event flow is high and the time span is large, a large number of repeated operations and redundant calculations are likely to occur, causing the traditional method to misjudge the complexity of the specific event flow and classify simple events as complex events, thereby affecting the final complex event detection results.

[0047] The specific scheme of the complex event detection method and system based on data analysis provided by the present invention is described in detail below with reference to the accompanying drawings.

[0048] Embodiment 1:

[0049] This invention proposes a complex event detection method based on data analysis, please refer to Figure 1 , which shows a flowchart of a complex event detection method based on data analysis provided by an embodiment of the present invention, the method comprising:

[0050] Step S1: Obtain the duration of each basic event in the overall event stream of the basic event database, where the overall event stream contains different types of basic events.

[0051] This solution takes the development of a complex event detection method suitable for smart traffic data as an example to illustrate:

[0052] Acquire all traffic data generated in the smart transportation system during the historical time period, such as all sensor data, camera input data, global positioning system (GPS) information, traffic signal status, etc., and pre-process these traffic data, including: data cleaning, parsing and structuring to ensure the quality and consistency of the data format. It should be noted that the embodiment of the present invention uses data extraction, transformation and loading (Extract-Transform-Load, ETL) tools to pre-process the traffic data generated in the smart transportation data, such as tools such as Talend and Informatica; other data pre-processing software and automated processes can be used for pre-processing in other embodiments of the present invention, such as Apache Spark computing engine and Apache Kafka system.

[0053] Feature selection technology is used to extract meaningful feature events from preprocessed traffic data and record them as basic events, such as the change pattern of traffic flow (such as the current large traffic volume on the traffic road), the interaction between vehicles (such as vehicle collisions), the congestion level of the road section (such as the large number of vehicles on the road, resulting in traffic obstruction and congestion), etc. A basic event database is constructed based on the selected basic events, and each basic event corresponds to a type, such as sudden vehicle deceleration, emergency stop, collision and road section closure, so that the subsequent complex detection algorithm can work more effectively.

[0054] It should be noted that the feature selection technology in the embodiment of the present invention selects the variance selection method, and in other embodiments, the correlation coefficient method, the mutual information method, the recursive feature elimination method, etc. can also be selected.

[0055] An event stream refers to a sequence of events that occur continuously. The overall event stream of the basic event database, that is, the overall view of all events, includes basic events of all types and sources. The duration of the time period from the start time to the end time of each basic event is recorded as the duration, and basic events of the same field and the same degree are regarded as basic events of the same type, such as vehicle collision events of the same degree.

[0056] Step S2: Select any basic event from the overall event stream and record it as the target event; obtain the comprehensive radiation index of the target event according to the duration of the basic events of the type to which the target event belongs and the spatial distribution changes of the informants of the basic events; adjust the matching window in the pattern matching process of the target event based on the comprehensive radiation index to obtain the adjusted matching window of the target event.

[0057] In complex event processing, an event refers to any observable behavior or state change that occurs in a data system (such as a sudden vehicle collision); an event pattern refers to the relationship and sequence between a set of events that together constitute a meaningful composite event (such as a series of events in which a vehicle suddenly decelerates, changes its driving trajectory, and triggers emergency braking may constitute an accident event pattern); an event stream refers to a sequence of events that occur continuously.

[0058] General complex event processing methods usually consider the temporal and spatial attributes of events in the event stream. A complex event stream usually contains multiple event sources and events at multiple time points. There may be significant time differences or spatial position differences between some events in the event stream, while some events occur almost simultaneously and at close distances, resulting in smaller time differences or spatial position differences, i.e., smaller asynchrony. For basic events with smaller asynchrony, temporary matching events generated by the system will have a large number of overlaps. When using the NFA processing model for event pattern matching, this overlap makes it impossible for NFA to accurately distinguish overlapping temporary matching events, resulting in misjudgment of the complexity of the event pattern, and mistaking some basic events that do not belong to the same event pattern as conforming to the same event pattern. Therefore, when performing matching analysis, not only the correlation between basic events in the target event stream should be considered, but also the temporal and spatial attributes of the basic events should be combined to determine their impact range, i.e., the comprehensive radiation index.

[0059] The impact of some events may be local, while some events may have a wider geographical or temporal span. The time span of the events is analyzed by the duration of the basic events, and the spatial span of the events is analyzed by the changes in the spatial distribution of the informants of the basic events. Combining the two, we can obtain that the basic events have different degrees of radiation in the time and space ranges; and the radiation degree of each basic event is quantified by the radiation degree of the basic events of the type to which the basic event belongs, that is, the comprehensive radiation index.

[0060] For the convenience of analysis, any basic event from the overall event flow is selected as the target event.

[0061] See also Figure 2 , which shows a flowchart of a method for obtaining a comprehensive radiation index provided by an embodiment of the present invention, the method comprising:

[0062] Step S211: obtaining a spatial change index of the target event according to the spatial change of the location of the insider at the start time and the end time of the preset fermentation time period of the target event.

[0063] When a vehicle runs a red light, i.e., a target event occurs, the owner posts information related to the target event online. In the embodiment of the present invention, the end time of the target event is used as the start time of the preset fermentation time period, and the people who browsed the post before the end time of the target event in the system are used as insiders at the start time of the preset fermentation time period of the target event; after the post ferments online for a period of time, i.e., the preset fermentation time period, the people who browsed the post before the end time of the preset fermentation time period of the target event in the system are used as insiders at the end time of the preset fermentation time period of the target event. The location of the target event is obtained from the intelligent transportation system; the insider is located by GPS in the software to obtain the location of the insider.

[0064] It should be noted that the duration of the preset fermentation time period is 3 days, which is an empirical value, and the implementer can set it according to the specific situation. In other embodiments of the present invention, people within a certain distance range from the location of the target event can also be regarded as insiders.

[0065] Preferably, the method for obtaining the spatial variation index includes: recording the start time and the end time of the preset fermentation time period of the target event as the analysis time, and for each analysis time, arranging the positions of all insiders of the target event at the analysis time in order based on the distance between the position of each insider of the target event at the analysis time and the occurrence position of the target event, so as to obtain the analysis sequence of the target event at the analysis time; establishing a three-dimensional coordinate system with the longitude of the position of the insider of the target event at the analysis time as the horizontal axis, the latitude of the position of the insider at the analysis time as the vertical axis, and time as the vertical axis; marking the elements in the analysis sequence of the target event at the start time in the three-dimensional coordinate system to obtain the start coordinate point of the corresponding element, and marking the start coordinate point of the analysis sequence of the target event at the end time as the vertical axis. The elements in the analysis sequence are marked in the three-dimensional coordinate system to obtain the end coordinate points of the corresponding elements; the start coordinate points and the end coordinate points in the three-dimensional coordinate system are randomly connected for different times to obtain the analysis line segments of each start coordinate point in each connection; the cumulative sum of the lengths of the analysis line segments of all the start coordinate points in each adjacent connection is used as the judgment basis for each connection; the analysis line segment corresponding to the smallest judgment basis is recorded as the target line segment; the angle between each target line segment and the reference plane is recorded as the reference angle of each target line segment; the reference plane is parallel to the plane formed by the horizontal axis and the vertical axis of the three-dimensional coordinate system, and passes through the end time on the vertical axis of the three-dimensional coordinate system; the average value of the reference angles of all target line segments is recorded as the spatial change index of the target event.

[0066] It should be noted that in order to analyze the distance factor when the target event is propagated, the positions of all insiders of the target event at the analysis time are arranged in order from small to large according to the distance between the insider's position and the location of the target event, and the analysis sequence of the target event at the analysis time is obtained. The starting coordinate point is connected to the ending coordinate point, which means that the target event may be propagated from the insider corresponding to the starting coordinate point to the insider corresponding to the ending coordinate point; each connected judgment benchmark presents the possibility that the insider corresponding to the starting coordinate point propagates to the insider corresponding to the ending coordinate point, and the connection state of the analysis line segment with the smallest judgment benchmark is selected for subsequent analysis, which can better represent that the insider corresponding to the ending coordinate point is propagated by the insider corresponding to the starting coordinate point, thereby improving the accuracy of subsequent analysis. If the angle between the target line segment of the starting coordinate point and the reference plane is smaller, the spatial position of the insider changes more, indicating that the influence range of the target event spreads farther, otherwise it means that the influence range of the event is more concentrated; therefore, it is necessary to negatively correlate the above angle to obtain the reference value. The spread range of the target event is measured by the benchmark values ​​of all the insiders corresponding to the start coordinate points and the end coordinate points, and the average of the benchmark values ​​of all target line segments of the target event in the three-dimensional coordinate system is recorded as the spatial variation index of the target event; the larger the spatial variation index, the farther the impact range of the target event spreads. In this embodiment, the distance between two coordinate points and the distance between two positions are both Euclidean distances.

[0067] As an example, if the target event has start coordinates a and b in the three-dimensional coordinate system, and end coordinates A, B, C, and D, the start coordinates and end coordinates are randomly connected multiple times. It should be noted that the start coordinates are not connected to each other, the end coordinates are not connected to each other, and all the start coordinates and end coordinates are connected. The following lists some of the connection situations: the first connection situation is: point a is connected to point A, point a is connected to point B, point b is connected to point C, and point b is connected to point D; the second connection situation is: point a is connected to point A, point a is connected to point B, point a is connected to point C, and point b is connected to point D.

[0068] Step S212: According to the duration and spatial variation index of the target event, the local radiation index of the target event is obtained; the mean of the local radiation indexes of the other basic events of the type to which the target event belongs except the target event is normalized to obtain the comprehensive radiation index of the target event.

[0069] If the duration of the target event is longer, the target event has a wider impact range; if the spatial change index is larger, the spatial position of the informant changes more during the target event, indicating that the impact range of the target event spreads farther. Therefore, the duration and spatial change index are positively correlated with the local radiation index. Since basic events of the same field and the same degree are regarded as basic events of the same type, errors are prone to occur only based on the relevant information analysis of the target event. The local radiation index of the remaining basic events of the type to which the target event belongs is used to measure the impact range of the target event. By increasing the sample, the accuracy is improved and the comprehensive radiation index of the target event is obtained. The basic events that meet the same complex event flow are determined by the comprehensive radiation index. The more similar the comprehensive radiation index is, the more likely it is that the basic events belong to the same complex event flow.

[0070] In a specific implementation of the embodiment of the present invention, the comprehensive radiation index is expressed by the formula:

[0071]

[0072] Where R is the comprehensive radiation index of the target event; I is the number of basic events of the type to which the target event belongs; T i is the duration of the i-th basic event of the type to which the target event belongs except the target event; M i is the number of target line segments corresponding to the i-th basic events of the type to which the target event belongs except the target event; The mth corresponding to the i-th basic event of the type to which the target event belongs except the target event i The angle between the target line segment and the reference plane; The mth corresponding to the i-th basic event of the type to which the target event belongs except the target event i The baseline value of the target line segment; is the spatial variation index of the i-th basic event of the type to which the target event belongs, except for the target event; Norm is the normalization function; cos is the cosine function. When the comprehensive radiation index R is larger, the duration of the target event is longer and the scope of the event is wider, indicating that the impact of the basic event on time and space is more extensive and lasting. Negative correlation mapping can also be performed using the exp(-x) function, which is not limited here.

[0073] The comprehensive radiation index takes into account both the temporal and spatial attributes of the event. When the event pattern detection method is used for event pattern matching, the comprehensive radiation index is used to adjust the size of the matching window in the matching process. When the comprehensive radiation index is larger, the matching window of the target event is expanded to accurately capture the full impact range of the basic event.

[0074] Preferably, the method of adjusting the matching window of the basic event using the comprehensive radiation index is: using the event pattern detection method, performing event pattern matching on the target event and the remaining basic events in the overall event stream, and obtaining the matching window of the target event in the matching process; using the sum of the comprehensive radiation index of the target event and the constant 1, weighting the size of the matching window of the target event, and obtaining the adjusted matching window of the target event. In an embodiment of the present invention, the event pattern detection method is a non-deterministic finite state automaton method.

[0075] It should be noted that in the process of implementing event pattern matching, a matching window can be used to control the effective matching range of the event pattern. The purpose of the matching window is to ensure that only the most recent part of the data is taken into account during matching. The matching window can be a one-dimensional window based on time (such as a sliding time window) or the number of events (such as a length window), that is, the width of the window is 1. This type of matching window only uses the sum of the comprehensive radiation index of the target event and the constant 1, and the product of the length of the matching window as the length and width of the matching window, and the width remains unchanged; the matching window can be a two-dimensional window based on space. This type of matching window uses the sum of the comprehensive radiation index of the target event and the constant 1, and the product of the length and width of the matching window as the length and width of the matching window, respectively.

[0076] It should be noted that the method for obtaining the comprehensive radiation index and the adjustment matching window of each basic event in the overall event flow is the same as the method for obtaining the comprehensive radiation index and the adjustment matching window of the target event.

[0077] Step S3: Filter the event sources in the overall event stream according to the comprehensive radiation index of each basic event in the overall event stream and the degree of correlation between the basic event and the basic event in its adjustment matching window.

[0078] In the overall event stream, the basic events of the basic event stream of a complex event are not strongly correlated in time, space or other attributes. These basic events may occur independently, that is, the events have a weak correlation, but they may be combined into a complex event in specific actual situations. Therefore, the basic events in the same complex event stream have a strong correlation with the event source. For example: when detecting a complex event such as a traffic accident, the traffic accident involves multiple basic events, such as vehicle emergency stop, collision sensor triggering and vehicle speed drop. These basic events may not have a strong correlation in time and space, but they all have a high correlation with the event source of the traffic accident. Therefore, when conducting a correlation analysis between basic events, it is necessary to use comprehensive radiation indicators to correct the degree of correlation between basic events, obtain a more accurate correlation coefficient between events, and thus select the event source.

[0079] Preferably, the method for obtaining the comprehensive correlation index of the target event is as follows: the probability of occurrence of the benchmark event of the type to which each basic event belongs in the basic event of the overall event flow is recorded as the probability of occurrence of each basic event; in the process of pattern matching the target event, any basic event is selected from the adjustment matching window of the target event as the analysis event, and the occurrence probabilities of the target event and the analysis event are substituted into the joint entropy formula to obtain the local correlation index between the target event and the analysis event; the mean of the local correlation index between the target event and all the basic events in the adjustment matching window of the target event is negatively correlated to obtain the comprehensive correlation index of the target event; according to the comprehensive correlation index and the comprehensive radiation index of the target event, the network centrality index of the target event is obtained. Among them, the method for obtaining the joint entropy is a well-known technology for those skilled in the art and will not be repeated here.

[0080] In a specific implementation of the embodiment of the present invention, taking the target event as an example, the network centrality index of the target event is expressed by the formula:

[0081]

[0082] Where H is the network centrality index of the target event; R is the comprehensive radiation index of the target event; N is the number of basic events in the adjustment matching window of the target event; G n is the local correlation index between the target event and the remaining nth basic events in the adjustment matching window except the target event; is the comprehensive correlation index of the target event; Norm is the normalization function. Realize Other methods can also be used for negative correlation mapping, which will not be given as examples here.

[0083] It should be noted that when The smaller the value is, the more certain the joint distribution of the target event and the other basic events in its adjustment matching window is, indicating that the target event has a stronger correlation with the other basic events in its adjustment matching window, implying that the target event has a higher importance and central position in the event stream, so it is necessary to The larger the comprehensive radiation index R is, the more extensive and lasting the impact of the target event on time and space is, and the more extensive the influence of the target event on the overall event flow is. Correction is made. When the network centrality index H is larger, the target event has a high correlation and extensive influence in the overall event flow. The higher the central position of the target event in the event flow, the greater the possibility that the target event is the source of the key event. In summary, the comprehensive correlation index and the comprehensive radiation index are both positively correlated with the network centrality index.

[0084] For example: in smart transportation, traffic collision accident events, as a key data source in complex data streams, have a high correlation with basic events such as vehicle emergency stops, collision sensor triggering, and vehicle speed drops, and the diffusion impact range is large. The larger the network centrality index of traffic collision accident events, the higher the network centrality index of traffic collision accident events.

[0085] The method for obtaining the local correlation index between the target event and each basic event in its adjustment matching window is the same as the method for obtaining the local correlation index between the target event and the analysis event; the method for obtaining the network centrality index of all target events in the overall event flow is the same as the method for obtaining the network centrality index of the target event.

[0086] Among the basic events in the overall event flow, the basic events greater than the preset judgment threshold are taken as event sources in the overall event flow. It should be noted that the preset judgment threshold is an empirical value of 0.6, and the implementer can set it according to the specific situation.

[0087] Step S4: According to the difference in the degree of association between any two basic events in the overall event stream and the same event source, adjust the original distance measurement between the basic events and each event source in the pattern matching process, and use the adjusted original distance measurement to perform complex event detection on the basic events in the overall event stream.

[0088] When using non-deterministic finite state automata for complex event processing, for each basic event, it is usually necessary to measure the original distance between each event source (or the location where each event occurs). This is because the definition of complex events usually involves the temporal or spatial relationship between events, and the certainty or uncertainty of these relationships may affect the final event matching and triggering.

[0089] Generally, when two basic events conform to the same complex event flow, their association with the same event source is close. The NFA matching rule is that the time distance or spatial distance between two basic events can be used as a matching metric. If two basic events are close enough in these dimensions, they are considered to be associated and can be used as part of a complex event. The original distance metric when two basic events are matched by NFA is obtained, and the original distance metric is corrected using the difference in the degree of association between the two basic events and the same event source.

[0090] Preferably, in some possible implementation modes of the embodiments of the present invention, the method for obtaining the revised distance metric includes: randomly selecting two basic events from the overall event stream as events to be tested, and randomly selecting one event source as a target source; recording the absolute value of the difference between the local correlation indicators of the two events to be tested and the target source respectively as the correlation difference coefficient between the two events to be tested and the target source; using an event pattern detection method to perform event pattern matching on each event to be tested and the target source to obtain the original distance metric between each event to be tested and the target source; adjusting the original distance metric using the correlation difference coefficient to obtain the revised distance metric between each event to be tested and the target source; for each event source, according to the revised distance metric between each basic event in the overall event stream and the event source, using the event pattern detection method to perform complex event detection on the basic events in the overall event stream, which can effectively extract a more accurate and reliable complex event stream from the overall event stream.

[0091] It should be noted that when the correlation difference coefficient between the two events to be tested and the target source is larger, the possibility that the two events to be tested belong to the same complex event flow is smaller. When performing NFA matching later, it is necessary to increase the distance metric between the two to ensure that the two are not aggregated into the same complex event flow. The method for obtaining the modified distance metric between each basic event and each event source in the overall event flow is the same as the method for obtaining the modified distance metric between the event to be tested and the target source.

[0092] When using a non-deterministic finite state automaton to perform complex event processing on basic events in the overall event flow, the event source is the event source obtained in step S3, and the original distance metric between each basic event and each event source is replaced by the modified distance metric, and other contents remain unchanged.

[0093] So far, the present invention is completed.

[0094] In summary, in the embodiment of the present invention, the comprehensive radiation index is obtained according to the duration of the basic event of the type to which each basic time belongs and the spatial distribution change of the informant, and the adjustment matching window is obtained by adjusting the matching window; the event source is obtained by the degree of association between the comprehensive radiation index of the basic time and the basic time in the adjustment matching window; the original distance measurement between the basic event and the event source is adjusted according to the difference in the degree of association between any two basic events and the same event source, and then complex event detection is performed on the basic events in the overall event flow. The spatial distribution change of the informant of the basic event of the present invention and the duration adjust the matching window and filter the event source, so as to improve the matching accuracy during complex event detection.

[0095] Embodiment 2:

[0096] This invention proposes a complex event detection system based on data analysis, please refer to Figure 3, which shows a structure diagram of a complex event detection system based on data analysis provided by an embodiment of the present invention, the system comprising:

[0097] The data acquisition module 510 is used to obtain the duration of each basic event in the overall event stream of the basic event database, where the overall event stream contains different types of basic events;

[0098] The matching window adjustment module 520 is used to select any basic event from the overall event stream as the target event; obtain the comprehensive radiation index of the target event according to the duration of the basic events of the type to which the target event belongs and the spatial distribution change of the informants of the basic event; adjust the matching window in the process of pattern matching of the target event based on the comprehensive radiation index to obtain the adjusted matching window of the target event;

[0099] An event source selection module 530 is used to screen event sources in the overall event stream according to the comprehensive radiation index of each basic event in the overall event stream and the degree of association between the basic event and the basic event in the adjustment matching window;

[0100] The complex event detection optimization module 540 is used to adjust the original distance measurement between the basic events and each event source in the pattern matching process according to the difference in the degree of association between any two basic events in the overall event stream and the same event source, and use the adjusted original distance measurement to perform complex event detection on the basic events in the overall event stream.

[0101] It should be noted that: the device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the complex event detection system based on data analysis and the complex event detection method embodiment based on data analysis provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0102] Embodiment 3:

[0103] Figure 4 A computer device schematic diagram of a complex event detection device based on data analysis provided by an embodiment of the present invention. For example, Figure 4 As shown, the computer device includes: a memory 601, a processor 602, and a computer program 603 stored in the memory 601 and running on the processor 602, wherein when the processor 602 executes the computer program 603, the computer device can execute any one of the complex event detection methods based on data analysis introduced above.

[0104] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to execute a complex event detection method based on data analysis provided by an embodiment of the present application.

[0105] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0106] It should be understood that the device provided in this embodiment is used to execute the above-mentioned complex event detection method based on data analysis, and thus can achieve the same effect as the above-mentioned implementation method.

[0107] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a device, the processing module may be used to control and manage the actions of the device. The storage module may be used to support the device to execute mutual program codes, etc.

[0108] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits disclosed in conjunction with the present application. The processor may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0109] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0110] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A complex event detection method based on data analysis, characterized in that: The method includes: Obtaining the duration of each basic event in the overall event stream of the basic event database, where the overall event stream contains different types of basic events; Selecting any basic event from the overall event stream as the target event; obtaining a comprehensive radiation index of the target event according to the duration of the basic events of the type to which the target event belongs and the spatial distribution change of the informants of the basic event; adjusting the matching window in the pattern matching process of the target event based on the comprehensive radiation index to obtain an adjusted matching window of the target event; Filter the event sources in the overall event stream according to the comprehensive radiation index of each basic event in the overall event stream and the degree of association between the basic event and the basic event in the adjustment matching window; According to the difference in the degree of association between any two basic events in the overall event stream and the same event source, the original distance measurement between the basic event and each event source in the pattern matching process is adjusted, and the basic events in the overall event stream are detected with the adjusted original distance measurement; The method for adjusting the matching window of the target event includes: Using the event pattern detection method, event pattern matching is performed on the target event and the other basic events in the overall event flow to obtain the matching window of the target event in the matching process; using the sum of the comprehensive radiation index of the target event and the constant 1, the size of the matching window of the target event is weighted to obtain the adjusted matching window of the target event.

2. A complex event detection method based on data analysis according to claim 1, characterized in that: The method for obtaining the comprehensive radiation index of the target event includes: According to the spatial changes of the positions of the insiders at the start and end times of the preset fermentation time period of the target event, the spatial change index of the target event is obtained; According to the duration of the target event and the spatial variation index, the local radiation index of the target event is obtained; the mean of the local radiation index of the remaining basic events of the type to which the target event belongs except the target event is normalized to obtain the comprehensive radiation index of the target event.

3. A complex event detection method based on data analysis according to claim 2, characterized in that: The method for obtaining the spatial variation index of the target event includes: The start and end time of the preset fermentation time period of the target event are recorded as analysis time. For each analysis time, based on the distance between the position of each insider at the target event at the analysis time and the occurrence position of the target event, the positions of all insiders at the target event at the analysis time are arranged in order to obtain the analysis sequence of the target event at the analysis time. A three-dimensional coordinate system is established with the longitude of the insider's position at the target event at the analysis time as the horizontal axis, the latitude of the insider's position at the analysis time as the vertical axis, and time as the vertical axis; the elements in the analysis sequence of the target event at the start time are marked in the three-dimensional coordinate system to obtain the start coordinate points of the corresponding elements, and the elements in the analysis sequence at the end time are marked in the three-dimensional coordinate system to obtain the end coordinate points of the corresponding elements; the start coordinate points and the end coordinate points in the three-dimensional coordinate system are randomly connected for different times to obtain the analysis line segments of each start coordinate point in each connection; the cumulative sum of the lengths of the analysis line segments of all the start coordinate points in each connection is used as the judgment basis for each connection; The analysis line segment corresponding to the minimum judgment criterion is recorded as the target line segment; the angle between each target line segment and the reference plane is negatively correlated to obtain the reference value of each target line segment; the reference plane is parallel to the plane formed by the horizontal axis and the vertical axis of the three-dimensional coordinate system, and passes through the end time on the vertical axis of the three-dimensional coordinate system; The average of the baseline values ​​of all target line segments is recorded as the spatial variation index of the target event.

4. The complex event detection method based on data analysis according to claim 1, characterized in that: The method for screening event sources in the overall event stream comprises: The probability of occurrence of the benchmark event of the type to which each basic event belongs in the basic events of the overall event flow is recorded as the occurrence probability of each basic event; in the process of pattern matching of the target event, any basic event is selected from the adjustment matching window of the target event and recorded as the analysis event, and the occurrence probabilities of the target event and the analysis event are substituted into the joint entropy formula to obtain the local correlation index between the target event and the analysis event; Negative correlation mapping is performed between the target event and the mean values ​​of the local correlation indexes of all basic events in the adjustment matching window of the target event to obtain a comprehensive correlation index of the target event; According to the comprehensive correlation index and the comprehensive radiation index of the target event, the network centrality index of the target event is obtained; among the basic events of the overall event flow, the basic events greater than a preset judgment threshold are taken as event sources in the overall event flow.

5. A complex event detection method based on data analysis according to claim 4, characterized in that: The method for performing complex event detection on basic events in the overall event flow using the adjusted original distance metric comprises: Select any two basic events from the overall event flow as the events to be tested, and select any event source as the target source; The absolute values ​​of the differences between the local correlation indicators of the two events to be tested and the target source are recorded as correlation difference coefficients between the two events to be tested and the target source; Using the event pattern detection method, each event to be tested is matched with the target source to obtain the original distance measurement between each event to be tested and the target source; using the correlation difference coefficient to adjust the original distance measurement to obtain the corrected distance measurement between each event to be tested and the target source; For each event source, according to the modified distance metric between each basic event in the overall event stream and the event source, a complex event detection is performed on the basic events in the overall event stream using an event pattern detection method.

6. The complex event detection method based on data analysis according to claim 3 is characterized in that: The preset fermentation time period is 3 days.

7. The complex event detection method based on data analysis according to claim 4, characterized in that: The preset judgment threshold takes the empirical value of 0.

6.

8. The complex event detection method based on data analysis according to claim 1, characterized in that: The event pattern detection method is a non-deterministic finite state automaton method.

9. A complex event detection system based on data analysis, characterized in that: The system includes: A data acquisition module is used to obtain the duration of each basic event in the overall event stream of the basic event database, where the overall event stream contains different types of basic events; A matching window adjustment module is used to select any basic event from the overall event stream and record it as a target event; obtain a comprehensive radiation index of the target event according to the duration of the basic events of the type to which the target event belongs and the spatial distribution change of the informants of the basic event; adjust the matching window in the pattern matching process of the target event based on the comprehensive radiation index to obtain an adjusted matching window of the target event; An event source selection module, used to screen event sources in the overall event stream according to the comprehensive radiation index of each basic event in the overall event stream and the degree of association between the basic event and the basic event in the adjustment matching window; A complex event detection optimization module is used to adjust the original distance metric between the basic events and each event source in the pattern matching process according to the difference in the degree of association between any two basic events in the overall event stream and the same event source, and use the adjusted original distance metric to perform complex event detection on the basic events in the overall event stream; The method for adjusting the matching window of the target event includes: Using the event pattern detection method, event pattern matching is performed on the target event and the other basic events in the overall event flow to obtain the matching window of the target event in the matching process; using the sum of the comprehensive radiation index of the target event and the constant 1, the size of the matching window of the target event is weighted to obtain the adjusted matching window of the target event.

Images