Real-time data stream processing method and system based on distributed computing

Abstract

The present invention belongs to the technical field of data mining and analysis processing, and specifically discloses a real-time data stream processing method and system based on distributed computing, including: obtaining the real-time and historical data streams of the server program window, using time synchronization to quickly locate the startup window matching path and historical abnormal performance to meet high real-time requirements; combining historical abnormal event statistics with real-time data stream similarity analysis, dynamically building a distributed tracking link, using a depth-first search algorithm to predict risk events, and accurately quantifying abnormal probability and deviation; configuring differentiated optimization strategies for different performance parameters to form a multi-level defense mechanism and enhance the system's adaptability. The present invention comprehensively analyzes the interaction trajectory and performance parameter set, marks the abnormal performance link relationship, and refines the abnormal event correlation analysis, thereby enhancing the system's maintainability and scalability and significantly improving data processing efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of data mining and analysis processing, and relates to a real-time data stream processing method and system based on distributed computing. Background Art

[0002] As digitalization accelerates, real-time data stream processing in server clusters has become a core technology requirement in fields such as cloud computing and the Internet of Things. Efficiently analyzing the interaction trajectory and performance parameters of program windows is crucial for improving system stability and optimizing resource scheduling.

[0003] There are also some related solutions related to data analysis and processing in the existing technology. For example, Chinese Patent Publication No. CN103136217B discloses a distributed data stream processing method and system, which divides the original data stream into a real-time data stream and a historical data stream; processes the real-time data stream and the historical data stream in parallel, and generates respective processing results; and integrates the generated processing results. This application makes real-time calculation of large amounts of data possible. The operation of real-time data streams can be processed in a distributed and parallel manner to the maximum extent, while ensuring large-scale data processing and high real-time performance, thereby improving the response speed of the system.

[0004] Chinese patent publication number CN119356879B discloses a distributed real-time data stream processing system and method. It analyzes the data traffic change sequence, dynamically identifies and classifies traffic status, and improves real-time monitoring capabilities; quantifies node resource pressure and enhances load balancing predictability; combines dynamic adjustment of task priorities to optimize resource allocation, alleviate peak pressure, improve data processing efficiency and system stability, and enhance the adaptability and reliability of large-scale data stream processing.

[0005] The above solution has the following limitations: First, it lacks a mechanism for rapidly locating both real-time and historical data streams within server program windows, resulting in inefficient analysis in high-real-time scenarios. Second, its risk prediction methods are limited, failing to combine historical anomalies with real-time data streams to build a dynamic tracking link. This makes it impossible to accurately locate risk events and their probability. Furthermore, it lacks differentiated optimization strategies and multi-layered defense mechanisms for different performance parameters within the data stream, making intelligent adaptive processing difficult. Third, its analysis of the correlation between program window operation paths is not thorough enough, and link relationships are not effectively utilized to refine abnormal event analysis, limiting the system's maintainability and scalability. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present invention proposes the following technical solution: a real-time data stream processing method based on distributed computing, comprising the following steps: Step 1, obtaining the corresponding operation paths of each program window on the server, and based on the server log, obtaining the corresponding real-time data stream and historical data stream of the startup window, the operation path refers to the execution sequence of the controllable buttons in the program window, and the data stream refers to the interaction trajectory and performance parameter set of the program window in the time series.

[0007] Step 2: Locate each matching path of the startup window in the real-time data stream, and obtain each historical abnormal event corresponding to each matching path of the startup window based on the historical data stream.

[0008] Step 3: Determine the corresponding historical abnormal performance of each matching path in the startup window and its corresponding link relationship in a time-series synchronization manner. The link relationship includes a binding relationship and a fusion relationship.

[0009] Step 4: Obtain the corresponding interaction trajectory and performance parameter set of the real-time data stream of the startup window, analyze the abnormal probability of generating the corresponding historical abnormal performance of each matching path, and dynamically build the distributed tracking link of the startup window based on the link relationship.

[0010] Step 5: Based on the distributed tracking link, predict the risk events of the real-time data stream, determine the corresponding risk attributes of the risk events of the real-time data stream, and set the optimized processing flow accordingly.

[0011] The second aspect of the present invention provides a real-time data stream processing system based on distributed computing, including: a data acquisition module, which obtains the corresponding operation paths of each program window on the server, and obtains the corresponding real-time data stream and historical data stream of the startup window based on the server log. The operation path refers to the execution sequence of the controllable buttons in the program window, and the data stream refers to the interaction trajectory and performance parameter set of the program window in the time series.

[0012] The data stream parsing module locates each matching path of the startup window in the real-time data stream, and obtains each historical abnormal event corresponding to each matching path of the startup window based on the historical data stream.

[0013] The abnormal performance determination module determines the corresponding historical abnormal performance of each matching path of the startup window and its corresponding link relationship in a time sequence synchronization manner, wherein the link relationship includes a binding relationship and a fusion relationship.

[0014] The link construction module obtains the corresponding interaction trajectory and performance parameter set of the real-time data stream of the startup window, analyzes the anomaly probability of generating the corresponding historical abnormal performance of each matching path, and dynamically builds the distributed tracking link of the startup window based on the link relationship.

[0015] The risk event prediction module predicts risk events in real-time data streams based on distributed tracking links, determines the corresponding risk attributes of risk events in real-time data streams, and sets up optimized processing procedures accordingly.

[0016] Compared with the existing technology, the beneficial effects of the present invention are as follows: (1) The present invention synchronously obtains real-time data streams and historical data streams, and quickly locates the matching path of the startup window and its historical abnormal performance based on the timing synchronization method, which helps to meet the needs of application scenarios with high real-time requirements.

[0017] (2) This invention achieves comprehensive tracking and management of program window operation paths through comprehensive analysis of interaction trajectories and performance parameter sets. By using binding and fusion relationships to mark the links of abnormal performance, the correlation analysis of abnormal events is further refined, providing more comprehensive data support for system optimization and enhancing the maintainability and scalability of the system.

[0018] (3) This invention combines historical abnormal event statistics with real-time data stream similarity analysis to dynamically construct a distributed tracking link and use a depth-first search algorithm to predict risk events. By quantifying abnormal probability and deviation, it can accurately identify potential risks and take optimization measures in advance to ensure the stable operation of the server. At the same time, differentiated optimization strategies are configured for different performance parameters to form a multi-level defense mechanism, which not only improves the system's adaptive capabilities but also reduces the cost of manual intervention, achieving efficient and reliable operation of the business. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 The figure is a flow chart of the steps for implementing the method of the present invention.

[0021] Figure 2 This is a schematic diagram of the system module connection of the present invention.

[0022] Figure 3 Schematic diagram of the distributed tracing link for the startup window of the present invention.

[0023] Reference numerals: x, y represent natural constants. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0025] Example 1

[0026] like Figure 1 As shown, the real-time data stream processing method based on distributed computing proposed in the present invention includes the following steps: Step 1, obtaining the corresponding operation paths of each program window on the server, and based on the server log, obtaining the corresponding real-time data stream and historical data stream of the startup window, wherein each program window refers to the application window with a visual interface currently opened on the server, including but not limited to a browser window and a document editing window, and the operation path refers to the execution sequence of the controllable buttons in the program window, such as clicking a button on one page to enter another page, and the data stream refers to the interaction trajectory and performance parameter set of the program window in the time series.

[0027] The performance parameter set refers to parameter sets corresponding to different types of performance, including but not limited to bandwidth-related data (such as throughput, bandwidth utilization) and delay-related data (such as round-trip delay, jitter rate) corresponding to basic transmission performance, TCP / UDP indicators (such as number of TCP connections, TCP window size) and HTTP / HTTPS indicators (such as request response time, status code distribution) corresponding to protocol layer performance, error data (such as CRC errors, TCP errors) and queue congestion data (such as buffer overflow, congestion window) corresponding to network quality performance.

[0028] Step 2: Locate each matching path of the startup window in the real-time data stream, and obtain each historical abnormal event corresponding to each matching path of the startup window based on the historical data stream.

[0029] The historical abnormal events include but are not limited to abnormal device operation events (such as uncontrolled mouse rotation, slow browser loading, and scanner paper jams), abnormal business operation process events (such as incorrect operation sequence and permission violations), and abnormal network communication status events (such as network connection interruption and network congestion).

[0030] The interaction trajectory refers to the user's habitual operating actions in the corresponding loaded page of each controllable button, including but not limited to the mouse coordinate sequence and the focus control switching order. The performance parameter set includes but is not limited to the corresponding data of basic transmission performance, protocol layer performance and network quality performance.

[0031] In a preferred embodiment, locating each matching path of the startup window in the real-time data stream includes: obtaining a set of operation paths that have been traversed by the startup window in the real-time data stream.

[0032] The operation path set is compared one by one with the corresponding operation paths of the startup window on the server.

[0033] A set of paths that completely match the traversed operation paths is extracted through a screening mechanism and defined as the matching paths of the startup window in the real-time data stream.

[0034] Step 3: Determine the corresponding historical abnormal performance of each matching path in the startup window and its corresponding link relationship in a time-series synchronization manner. The link relationship includes a binding relationship and a fusion relationship.

[0035] In a preferred embodiment, the timing synchronization method is specifically: taking the startup timestamp of the program window as the reference point, recording the time from the startup timestamp ( ) to the closing timestamp ( ) continuous time series , where n is a natural number and The end timestamp of the program window running period.

[0036] Based on the continuous time series, the interaction trajectories and performance parameter sets within the program window are time-aligned to form a structured data cube, that is, each timestamp corresponds to a structured data cube.

[0037] In a further preferred embodiment, the corresponding historical abnormal performance of each matching path of the startup window and the link relationship to which it belongs are determined in a time-series synchronization manner, including: step 3-1, dividing the performance parameter set into parameter sets corresponding to various types of performance, and defining the trajectory behavior of at least one performance containing at least one abnormal parameter in the interactive trajectory of the program window on the time series as an abnormal event, and the abnormal parameter refers to a parameter that deviates from the standard value by more than a preset deviation range.

[0038] Step 3-2: Count the number of historical abnormal events for each performance category on each matching path in the startup window, and obtain the ratio of the number of historical abnormal events to the total number of historical abnormal events, which is recorded as the historical occurrence probability of abnormal events for each performance category on each matching path in the startup window.

[0039] Step 3-3: If the historical probability of an abnormal event in a certain type of performance on a matching path of the startup window is 1, it means that the matching path of the startup window has only experienced this type of performance abnormality. Therefore, this type of performance on the matching path of the startup window is marked as historical abnormal performance, and the link relationship is a binding relationship.

[0040] Otherwise, it means that there have been at least two performance abnormalities on the matching path of the startup window. Then, multiple types of performance with a non-zero probability of abnormal events on the matching path of the startup window are obtained, and marked as the corresponding multiple types of historical abnormal performance of the matching path of the startup window, and the link relationship is a fusion relationship.

[0041] Step 3-4: According to step 3-3, the corresponding historical abnormal performance of each matching path in the startup window and the link relationship to which it belongs are obtained.

[0042] The present invention synchronously obtains real-time data streams and historical data streams, and quickly locates the matching path of the startup window and its historical abnormal performance based on a time sequence synchronization method, which helps to meet the needs of application scenarios with high real-time requirements.

[0043] Step 4: Obtain the corresponding interaction trajectory and performance parameter set of the real-time data stream of the startup window, analyze the abnormal probability of generating the corresponding historical abnormal performance of each matching path, and dynamically build the distributed tracking link of the startup window based on the link relationship.

[0044] In a preferred embodiment, the acquisition of the interaction trajectory and performance parameter set corresponding to the real-time data stream of the startup window and the analysis of the abnormal probability of generating the corresponding historical abnormal performance of each matching path are performed, and the analysis steps are as follows: the interaction trajectory and performance parameter set in the corresponding historical data stream of each matching path of the startup window and the real-time data stream are sorted in time series, and compared one by one in time series to generate behavioral interaction similarity features and network communication similarity features between the corresponding historical data stream of each matching path of the startup window and the real-time data stream.

[0045] By accumulating behavioral interaction similarity features and network communication similarity features, the similarity between the real-time data flow and the historical data flow corresponding to each matching path of the startup window is obtained.

[0046] The similarity between the real-time data stream and the historical data stream is mapped to a value in the range of 0-1 through the sigmoid() function, and the abnormal probability of the real-time data stream of the startup window generating the corresponding historical abnormal performance of each matching path is generated.

[0047] For example, if the behavioral interaction similarity feature and network communication similarity feature of the historical data stream and the real-time data stream corresponding to a matching path of the startup window are 0.5 and 2 respectively, then the similarity between the real-time data stream and the historical data stream corresponding to the matching path of the startup window is 0.5×2=1. Substituting 1 into the sigmoid() function, the abnormal probability of the real-time data stream of the startup window generating the corresponding historical abnormal performance of the matching path is obtained.

[0048] Step 4-1, for the behavioral interaction similarity features: 4-1.1, obtain the corresponding execution action set of the interaction trajectory of the startup window in the real-time data stream, and sort them in time series as: timestamp 1-action a1, timestamp 2-action a2, timestamp 3-action a3. The execution action set includes but is not limited to the mouse click position and the page browsing position.

[0049] 4-1.2. Sort the execution action set in the historical data stream corresponding to a matching path of the startup window in time sequence as follows: timestamp 1-action a12, timestamp 2-action a22, timestamp 3-action a32.

[0050] 4-1.3. Compare actions a1 and a12, a2 and a22, and a3 and a32. If both actions a1 and a12 involve browsing a page or clicking a location on a page, mark the timestamp as identical. Based on this, count the number of identical actions in the set of actions executed for the matching path in the startup window.

[0051] 4-1.4. By comparing the number of identical actions in the action set executed by each matching path of the startup window with the total number of actions, the behavioral interaction similarity characteristics of the corresponding historical data stream and the real-time data stream of each matching path of the startup window are obtained.

[0052] Step 4-2, for the similar characteristics of the network communication: 4-2.1, obtain the performance parameter set of the startup window in the real-time data stream, and sort them in time series as: timestamp 1-network bandwidth d1, timestamp 2-network bandwidth d2, timestamp 3-network bandwidth d3, such as the network bandwidths d1, d2, and d3 are 2MB / s, 1MB / s, and 4MB / s, respectively.

[0053] 4-2.2. The performance parameter set in the corresponding historical data stream of a matching path of the startup window is sorted in time series as follows: timestamp 1-network bandwidth d12, timestamp 2-network bandwidth d22, timestamp 3-network bandwidth d32. For example, the network bandwidths d12, d22, and d32 are 2 MB / s, 3 MB / s, and 2 MB / s, respectively.

[0054] 4-2.3. Compare network bandwidth d1 with network bandwidth d12, network bandwidth d2 with network bandwidth d22, and network bandwidth d3 with network bandwidth d32, respectively. Calculate the ratio of the bandwidth difference at each timestamp (0 MB / s, 2 MB / s, 2 MB / s) to the preset reference value (2 MB / s), generate a communication similarity sequence (0, 1, 1), and ultimately accumulate network communication similarity feature 2 for the historical data stream and the real-time data stream corresponding to the matching path in the startup window.

[0055] 4-2.4. According to steps 4-2.1 to 4-2.4, the network communication similarity characteristics of the corresponding historical data stream and the real-time data stream of each matching path of the startup window are obtained.

[0056] See also Figure 3 As shown, in a further preferred embodiment, the dynamic construction of the distributed tracing link of the startup window includes: comparing the abnormal probability of the corresponding historical abnormal performance of each matching path generated by the real-time data stream of the startup window with a preset abnormal probability baseline value, screening out the matching paths whose abnormal probability exceeds the preset abnormal probability baseline value, and recording them as risk paths.

[0057] The link relationship of the historical abnormal performance corresponding to the risk path of the startup window is obtained, and the performance tracking path corresponding to the risk path of the startup window is constructed based on the link relationship.

[0058] For example, if the historical abnormal performance corresponding to the risk path is a binding relationship, such as the historical abnormal performance is the basic transmission performance, then the parameter timing performance of the basic transmission performance corresponding to the risk path is set as the performance tracking path; if the historical abnormal performance corresponding to the risk path is a fusion relationship, then multiple types of performance with a non-zero probability of abnormal events on the risk path are obtained. If the multiple types of performance include basic transmission performance and protocol layer performance, then the parameter timing performance of the basic transmission performance and protocol layer performance corresponding to the risk path are recorded as the performance tracking path, and the performance levels are sorted in descending order of the historical probability of abnormal events on the basic transmission performance and protocol layer performance. For example, if the historical probability of abnormal events on the basic transmission performance is 0.6 and the historical probability of abnormal events on the protocol layer performance is 0.4, then the basic transmission performance is used as the first-level tracking link and the protocol layer performance is used as the second-level tracking link; according to this logic, the performance tracking path corresponding to the risk path of the startup window is obtained.

[0059] The parameter timing performance refers to the monitored values of the parameters corresponding to various performance types.

[0060] The corresponding abnormal number of various abnormal events generated by the risk path of the startup window on the historical abnormal performance is counted, and the ratio of the abnormal number to the total number of historical abnormal events is obtained. This ratio is recorded as the occurrence weight of various abnormal events generated by the risk path of the startup window in the real-time streaming data. The various abnormal events refer to different abnormal events.

[0061] Count all abnormal events with non-zero weights and use them as tracing nodes. Then, deploy each tracing node on the performance tracing path, and derive the distributed tracing link of the startup window.

[0062] This invention achieves comprehensive tracking and management of program window operation paths through comprehensive analysis of interaction trajectories and performance parameter sets. By using binding and fusion relationships to mark links between abnormal performance, it further refines the correlation analysis of abnormal events, providing more comprehensive data support for system optimization and enhancing the system's maintainability and scalability.

[0063] Step 5: Based on the distributed tracking link, predict the risk events of the real-time data stream, determine the corresponding risk attributes of the risk events of the real-time data stream, and set the optimized processing flow accordingly.

[0064] In a preferred embodiment, the method of predicting risk events of real-time data streams based on distributed tracing links includes defining the ratio of the deviation of a performance parameter from a standard value exceeding a preset deviation range as a deviation degree.

[0065] Obtain parameter performance of real-time data streams along the performance tracking path and identify their deviations accordingly.

[0066] Based on historical data streams, statistics are collected on the risk paths of the startup window and the deviations of various performance parameters monitored at the timestamps of various abnormal events. , is the number of the abnormal event, , is the performance type number, , is the number of each parameter, The performance parameter deviation set includes the deviation of parameters such as throughput, bandwidth utilization, round-trip delay, jitter rate, etc., and obtains the deviation of each performance parameter corresponding to the risk path of the startup window in the real-time data stream. Based on this, the risk execution rate of various abnormal events in the risk path of the startup window is calculated ,in Indicates the first Class performance corresponding to The parameter deviation and risk path are in the The timestamp of the abnormal event is detected. Class performance corresponding to The difference between the deviations of the parameters is greater. The greater the difference, the higher the deviation of the corresponding parameter in the real-time data stream, which means the higher the risk execution rate of the parameter. Indicates the reference deviation value of the preset parameter deviation, which is used to map the difference between the parameter deviation in the real-time data stream and the corresponding parameter deviation monitored by the risk path at the timestamps of various abnormal events. The larger the value of , the higher the risk execution rate.

[0067] The tracking nodes on the distributed tracking link of the startup window are traversed through the depth-first search method, and abnormal events whose risk execution rate exceeds the preset risk execution rate baseline value are screened out and marked as risk events of the real-time data stream.

[0068] Specifically, distributed tracing links are constructed based on risk paths and their associated historical abnormal performance links, which have a certain hierarchy and logical relationship. Depth-first search effectively explores along this hierarchy, preferentially visiting all nodes along a path, which conforms to the logical structure and construction of links.

[0069] In a further preferred embodiment, determining the corresponding risk attributes of the risk event of the real-time data stream includes: obtaining a set of performance parameter deviations of the risk path within the startup window in the real-time data stream from the distributed tracking link corresponding to the startup window.

[0070] Parameters in the performance parameter deviation set whose deviation is greater than or equal to the corresponding performance parameter deviation in the performance parameter deviation set monitored at the timestamp of the risk event occurrence are screened out and marked as risk attributes of the risk event.

[0071] For example, if the deviation of the bandwidth utilization in the real-time data stream within the current startup window is greater than the bandwidth utilization deviation monitored before the risk event occurs, the bandwidth utilization is marked as the risk attribute corresponding to the risk event.

[0072] In a further preferred embodiment, the setting of the optimization processing flow includes: configuring differentiated optimization strategies for different performance parameters. Specifically, when anomalies occur in delay-related performance parameters, such as when the round-trip delay exceeds a preset threshold or the jitter rate exceeds a standard range, the following optimization measures can be implemented: First, optimize and adjust the network topology structure to effectively shorten the data transmission path by reducing the number of jumps of network nodes; second, adopt QoS (Quality of Service) technology to give higher data transmission priority to services with high real-time requirements such as video conferencing and voice calls, thereby reducing the jitter rate of service data.

[0073] Based on the risk attributes determined by risk events in real-time data streams, performance parameters that match the risk attributes are identified, and corresponding optimization strategies are set for the performance parameters to build a defense mechanism to prevent the occurrence of risk events.

[0074] This invention combines historical anomaly statistics with real-time data stream similarity analysis to dynamically construct distributed tracking links. It also uses a depth-first search algorithm to predict risk events. By quantifying anomaly probabilities and deviations, it can accurately identify potential risks, proactively implement optimization measures, and ensure stable server operation. Furthermore, it configures differentiated optimization strategies for different performance parameters, forming a multi-layered defense mechanism. This not only improves the system's adaptability but also reduces the cost of manual intervention, ensuring efficient and reliable service operation.

[0075] Example 2

[0076] like Figure 2 As shown, based on the first embodiment, the second aspect of the present invention provides a real-time data stream processing system based on distributed computing, including: a data acquisition module, a data stream parsing module, an abnormal performance determination module, a link construction module and a risk event prediction module.

[0077] The data acquisition module, data flow analysis module, abnormal performance determination module, link construction module and risk event prediction module are connected in sequence.

[0078] The data acquisition module is used to obtain the corresponding operation paths of each program window on the server, and based on the server log, obtain the corresponding real-time data stream and historical data stream of the startup window. The operation path refers to the execution sequence of the controllable buttons in the program window, and the data stream refers to the interaction trajectory and performance parameter set of the program window in the time series.

[0079] The data stream parsing module is used to locate each matching path of the startup window in the real-time data stream, and obtain each historical abnormal event corresponding to each matching path of the startup window based on the historical data stream.

[0080] The abnormal performance determination module is used to determine the corresponding historical abnormal performance of each matching path of the startup window and its corresponding link relationship in a time sequence synchronization manner, wherein the link relationship includes a binding relationship and a fusion relationship.

[0081] The link construction module is used to obtain the corresponding interaction trajectory and performance parameter set of the real-time data stream of the startup window, analyze the abnormal probability of generating the corresponding historical abnormal performance of each matching path, and dynamically build the distributed tracking link of the startup window based on the link relationship.

[0082] The risk event prediction module is used to predict risk events of real-time data streams based on distributed tracking links, and determine the corresponding risk attributes of risk events of real-time data streams, and set an optimized processing flow accordingly.

[0083] It should be noted that the formulas described above, through the principle of dimensional consistency and mathematical standardization (e.g., normalization, dimensionless parameter conversion, or unified unit system), can translate physical quantities of different attributes into unitless standard values or homogeneous, superimposable parameters. This eliminates the interference of different dimensions on the operational logic, ensuring that the formulas retain the distribution characteristics of the original data while maintaining mathematical rationality and adaptability to objective laws. The above are merely exemplary embodiments of the present invention and are not intended to limit the scope of the present invention.

Claims

1. A real-time data stream processing method based on distributed computing, characterized in that: The following steps are involved: Step 1: Obtain the corresponding operation paths of each program window on the server, and obtain the real-time data stream and historical data stream of the startup window based on the server log. The operation path refers to the execution sequence of the controllable buttons in the program window, and the data stream refers to the interaction trajectory and performance parameter set of the program window in the time series; Step 2: Locate each matching path of the startup window in the real-time data stream, and obtain each historical abnormal event corresponding to each matching path of the startup window based on the historical data stream; Step 3: Determine the historical abnormal performance of each matching path in the startup window and its corresponding link relationship in a time-series synchronization manner, wherein the link relationship includes a binding relationship and a fusion relationship; The determination of the corresponding historical abnormal performance of each matching path in the startup window and its corresponding link relationship in a time-series synchronization manner includes: Step 3-1: Divide the performance parameter set into various performance parameter sets, and define the trajectory behavior of at least one abnormal performance parameter in the interactive trajectory of the program window on the time series as an abnormal event. The abnormal performance parameter refers to a performance parameter that deviates from the standard value by more than a preset deviation range; Step 3-2: Count the number of historical abnormal events for each performance category on each matching path in the startup window, and calculate the ratio of the number of historical abnormal events to the total number of historical abnormal events. This ratio is recorded as the historical probability of occurrence of abnormal events for each performance category on each matching path in the startup window. Step 3-3: If the historical probability of an abnormal event on a certain type of performance for a matching path in the startup window is 1, mark the performance of the matching path in the startup window as historical abnormal performance, and set the link relationship to a binding relationship; Otherwise, obtain the multi-category performance on the matching path of the startup window where the probability of an abnormal event occurring is not 0, mark it as the corresponding multi-category historical abnormal performance of the matching path of the startup window, and the link relationship is a fusion relationship; Step 3-4: According to step 3-3, obtain the corresponding historical abnormal performance of each matching path in the startup window and the link relationship to which it belongs; Step 4: Obtain the corresponding interaction trajectory and performance parameter set of the real-time data stream of the startup window, analyze the abnormal probability of generating the corresponding historical abnormal performance of each matching path, and dynamically build a distributed tracking link for the startup window based on the link relationship; Step 5: Based on the distributed tracking link, predict the risk events of the real-time data stream, determine the corresponding risk attributes of the risk events of the real-time data stream, and set the optimized processing flow accordingly.

2. The real-time data stream processing method based on distributed computing according to claim 1 is characterized in that: The method of locating matching paths of the start window in the real-time data stream includes: Get the set of operation paths that have been traversed in the startup window within the real-time data stream; Compare the operation path set with the corresponding operation paths of the startup window on the server one by one; A set of paths that completely match the traversed operation paths is extracted through a screening mechanism and defined as the matching paths of the startup window in the real-time data stream.

3. The real-time data stream processing method based on distributed computing according to claim 1 is characterized in that: The timing synchronization method is specifically as follows: Taking the startup timestamp of the program window as the reference point, record the continuous time series from the startup timestamp to the closing timestamp; Based on the continuous time series, the interaction trajectories and performance parameter sets within the program window are time-aligned to form a structured data cube, that is, each timestamp corresponds to a structured data cube.

4. The real-time data stream processing method based on distributed computing according to claim 1 is characterized in that: The acquisition of the corresponding interaction trajectory and performance parameter set of the real-time data stream of the startup window and the analysis of the abnormal probability of generating the corresponding historical abnormal performance of each matching path are performed in the following steps: The interaction trajectories and performance parameter sets in the historical data streams and real-time data streams corresponding to each matching path of the startup window are sorted in time series and compared one by one in time series to generate behavioral interaction similarity features and network communication similarity features between the historical data streams and real-time data streams corresponding to each matching path of the startup window; Accumulate behavioral interaction similarity features and network communication similarity features to obtain the similarity between the real-time data flow and the historical data flow corresponding to each matching path of the startup window; The similarity between the real-time data stream and the historical data stream is mapped to a value in the range of 0-1 through the sigmoid() function, and the abnormal probability of the real-time data stream of the startup window generating the corresponding historical abnormal performance of each matching path is generated.

5. The real-time data stream processing method based on distributed computing according to claim 1 is characterized in that: The dynamically constructed distributed tracing link for the startup window includes: Compare the abnormal probability of the corresponding historical abnormal performance of each matching path generated by the real-time data stream of the startup window with the preset abnormal probability baseline value, and screen out the matching paths with abnormal probabilities exceeding the preset abnormal probability baseline value, and record them as risk paths; Obtain the link relationship of the historical abnormal performance corresponding to the risk path of the startup window, and build the corresponding performance tracking path of the risk path of the startup window based on it; Count the number of abnormal events corresponding to various abnormal events generated by the risk path of the startup window in the historical abnormal performance, and obtain the ratio of the abnormal event to the total number of historical abnormal events. This ratio is recorded as the occurrence weight of various abnormal events generated by the risk path of the startup window in the real-time streaming data. The various abnormal events refer to different abnormal events. Count all abnormal events with non-zero weights and use them as tracing nodes. Then, deploy each tracing node on the performance tracing path, and derive the distributed tracing link of the startup window.

6. The real-time data stream processing method based on distributed computing according to claim 5 is characterized in that: The distributed tracing link is used to predict risk events in real-time data streams, including: The deviation is defined as the ratio of the deviation of the performance parameter from the standard value exceeding the preset deviation range; Obtain parameter performance of real-time data streams along the performance tracking path and identify their deviation accordingly; Based on historical data streams, the deviations of various performance parameters of the startup window's risk path monitored at the timestamps of various abnormal events are counted. The deviations of various performance parameters of the startup window's risk path in real-time data streams are also obtained. Based on this, the risk execution rate of the startup window's risk path in the event of various abnormal events is calculated. The tracking nodes on the distributed tracking link of the startup window are traversed through the depth-first search method, and abnormal events whose risk execution rate exceeds the preset risk execution rate baseline value are screened out and marked as risk events of the real-time data stream.

7. The real-time data stream processing method based on distributed computing according to claim 6 is characterized in that: Determining the corresponding risk attributes of risk events in the real-time data stream includes: Obtaining, from the distributed tracing link corresponding to the startup window, a set of performance parameter deviations of the risk path within the startup window in the real-time data stream; Parameters in the performance parameter deviation set whose deviation is greater than or equal to the corresponding performance parameter deviation in the performance parameter deviation set monitored at the timestamp of the risk event occurrence are screened out and marked as risk attributes of the risk event.

8. The real-time data stream processing method based on distributed computing according to claim 1 is characterized in that: The setting optimization process includes: Configure differentiated optimization strategies for different performance parameters; Based on the risk attributes determined by risk events in real-time data streams, performance parameters matching the risk attributes are identified, and corresponding optimization strategies are set for the performance parameters.

9. A real-time data stream processing system based on distributed computing, characterized in that: include, A data acquisition module acquires the corresponding operation paths of each program window on the server and, based on the server log, acquires the real-time data stream and historical data stream corresponding to the startup window. The operation path refers to the execution sequence of the controllable buttons in the program window, and the data stream refers to the interaction trajectory and performance parameter set of the program window in the time series. The data stream parsing module locates each matching path of the startup window in the real-time data stream and obtains each historical abnormal event corresponding to each matching path of the startup window based on the historical data stream; An abnormal performance determination module determines the corresponding historical abnormal performance of each matching path in the startup window and its corresponding link relationship in a time-series synchronization manner, wherein the link relationship includes a binding relationship and a fusion relationship; The determination of the corresponding historical abnormal performance of each matching path in the startup window and its corresponding link relationship in a time-series synchronization manner includes: Step 3-1: Divide the performance parameter set into various performance parameter sets, and define the trajectory behavior of at least one abnormal performance parameter in the interactive trajectory of the program window on the time series as an abnormal event. The abnormal performance parameter refers to a performance parameter that deviates from the standard value by more than a preset deviation range; Step 3-2: Count the number of historical abnormal events for each performance category on each matching path in the startup window, and calculate the ratio of the number of historical abnormal events to the total number of historical abnormal events. This ratio is recorded as the historical probability of occurrence of abnormal events for each performance category on each matching path in the startup window. Step 3-3: If the historical probability of an abnormal event on a certain type of performance for a matching path in the startup window is 1, mark the performance of the matching path in the startup window as historical abnormal performance, and set the link relationship to a binding relationship; Otherwise, obtain the multi-category performance on the matching path of the startup window where the probability of an abnormal event occurring is not 0, mark it as the corresponding multi-category historical abnormal performance of the matching path of the startup window, and the link relationship is a fusion relationship; Step 3-4: According to step 3-3, obtain the corresponding historical abnormal performance of each matching path in the startup window and the link relationship to which it belongs; The link construction module obtains the corresponding interaction trajectory and performance parameter set of the real-time data stream of the startup window, analyzes the abnormal probability of generating the corresponding historical abnormal performance of each matching path, and dynamically builds the distributed tracking link of the startup window based on the link relationship; The risk event prediction module predicts risk events in real-time data streams based on distributed tracking links, determines the corresponding risk attributes of risk events in real-time data streams, and sets up optimized processing procedures accordingly.

Images