An abnormality monitoring and root cause attribution method and system for a network car funnel

Abstract

The application provides an abnormality monitoring and root cause attribution method and system of a network car funnel, and relates to the technical field of computers.The method comprises the following steps: obtaining single-quantity time series data of each node of the network car funnel in the channel dimension, the city dimension and the city*channel combination dimension; performing double-period STL decomposition on each single-quantity time series data, sequentially stripping the day seasonal term and the week seasonal term, and extracting a target final residual corresponding to the current window from the obtained final residual sequence; calculating each conversion rate in the channel dimension, the city dimension and the city*channel combination dimension; performing standardization processing on each target final residual and conversion rate; performing abnormality detection based on the target final residual and the conversion rate after the standardization processing; performing root cause attribution analysis according to the detection information of the abnormal node and external correlation data, and generating and outputting an attribution report.The application can effectively distinguish between the periodic peak and valley of normal business and the real business abnormality, and reduce the false positive rate.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a method and system for anomaly monitoring and root cause attribution in ride-hailing service funnels. Background Technology

[0002] In the ride-hailing online service industry, the core transaction process for users typically exhibits a multi-stage funnel structure, such as inquiry → order placement → order acceptance → winning the bid → order completion / cancellation. Abnormal fluctuations in business metrics at any stage of the ride-hailing funnel, such as inquiry volume and order volume, can significantly impact the health of the business. Therefore, monitoring and attributing anomalies within the ride-hailing funnel are crucial.

[0003] In existing technologies, monitoring is performed using statistical thresholds based on a single indicator (such as the Z-Score method) or fixed thresholds. When the indicator deviates from the preset normal range, an alarm is triggered, and relevant personnel manually investigate the root cause of the problem after receiving the alarm.

[0004] However, existing technologies struggle to effectively distinguish between normal business cycles of peaks and troughs and genuine business anomalies, resulting in a high false alarm rate. Summary of the Invention

[0005] In view of this, in order to solve the technical problems existing in the prior art, the present invention provides a method and system for abnormal monitoring and root cause attribution of ride-hailing funnel.

[0006] The present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for anomaly monitoring and root cause attribution of ride-hailing funnels, including: Using the first preset duration as the granularity of the scrolling window, the order volume time series data of each node in the ride-hailing funnel is obtained in the dimensions of channel, city, and city × channel combination. The order volume time series data includes the order volume of the current window and the order volume of the same time period of the previous second preset duration. Perform a two-period STL decomposition on each single-quantity time series data, and successively remove the daily seasonal term and the weekly seasonal term. Extract the target final residual corresponding to the current window from the obtained final residual sequence. Calculate the conversion rate for each node based on the number of orders at the current window, under the channel dimension, city dimension, and city × channel combination dimension respectively; the conversion rate is the ratio of the number of orders at the downstream node to the number of orders at the upstream node. Standardize the final residuals and conversion rates for each objective; Anomaly detection is performed based on the final target residual and conversion rate after standardization. When the anomaly detection results show the presence of anomaly nodes, root cause attribution analysis is performed based on the detection information of the anomaly nodes and external related data, and an attribution report is generated and output.

[0008] Optionally, anomaly detection can be performed based on the standardized final residual and conversion rate, specifically including: Based on a pre-defined anomaly detection mechanism, an unsupervised anomaly detection algorithm is used to perform anomaly detection, resulting in a first detection result. The anomaly detection mechanism includes: performing anomaly detection layer by layer at the channel level, city level, and channel × city level for each node's final residual and conversion rate dimensions; pruning all lower-level sub-dimensions of the anomaly when an anomaly is detected at an upper level; where the target object is the final residual or conversion rate; the input of the unsupervised anomaly detection algorithm is the target object, and the output is the anomaly score of the target object; when the anomaly score is greater than a pre-defined score threshold, the corresponding target object is determined to be anomaly; the anomaly node is the node corresponding to the anomaly of the target object. Based on a preset noise reduction mechanism, the first detection result is denoised to obtain the anomaly detection result.

[0009] Optionally, the first detection result includes channel dimension sub-detection results, city dimension sub-detection results, and combined dimension sub-detection results; Based on a preset noise reduction mechanism, the first detection result is denoised to obtain the anomaly detection result, specifically including: For each sub-detection result, when the sub-detection result shows that there is a target abnormal node, for each target abnormal node, a combined judgment is made on whether the adjacent upstream node of the target abnormal node is abnormal and whether the conversion rate between the two nodes is abnormal, to obtain a combined judgment result; where the target abnormal node is the abnormal node that is not the most upstream. Based on the combined judgment results, the anomalies of the target abnormal nodes are classified as transmission effects, independent anomalies, or superimposed anomalies, and nodes classified as transmission effects are pruned.

[0010] Optionally, based on the combined judgment results, the anomalies of the target abnormal nodes are classified into transmission effects, independent anomalies, or superimposed anomalies, specifically including: When the upstream neighbor of the target abnormal node is an abnormal node, and the conversion rate between the target abnormal node and its upstream neighbor is normal, the abnormality of the target abnormal node is classified as a transmission effect. When the upstream neighbor of the target abnormal node is a non-abnormal node, and the conversion rate between the target abnormal node and its upstream neighbor is abnormal, the abnormality of the target abnormal node is classified as an independent abnormality. When the upstream neighbor of the target abnormal node is an abnormal node, and the conversion rate between the target abnormal node and its upstream neighbor is abnormal, the abnormality of the target abnormal node is classified as superimposed abnormality.

[0011] Optionally, root cause attribution analysis can be performed based on the detection information of abnormal nodes and external correlation data, specifically including: Combine detection information and external correlation data into an attribution context; The attribution context is fed into the rule engine for deterministic reasoning to obtain the deterministic reasoning result; the rule engine contains multiple attribution rules, and each attribution rule is configured with a preset confidence level; If the deterministic inference result shows that an attribution rule is matched, then output the attribution rule and its confidence level.

[0012] Optional, also includes: If the deterministic inference results show that multiple attribution rules are hit, then among the hit attribution rules, the attribution rule with the highest confidence is identified as the principal cause. Determine if there are any secondary root causes among the other attribution rules in each hit attribution rule that have a confidence difference with the primary root cause that is less than a preset difference threshold. If so, generate a multi-factor overlay label and output the primary root cause, secondary root cause, relevant confidence, and multi-factor overlay label. Otherwise, output the primary root cause and its confidence level.

[0013] Optionally, the rules engine is used to match each attribution rule one by one according to a preset priority.

[0014] Optionally, the unsupervised anomaly detection algorithm is the Isolation Forest algorithm.

[0015] Optionally, after generating and outputting the attribution report, the following may also be included: For each abnormal node, the abnormal score of the corresponding target object is determined as the abnormal score of the abnormal node; Select the target score range where the abnormal score of the abnormal node is located from multiple preset score ranges; The anomaly level corresponding to the target score range is determined as the anomaly level of the anomaly node; Execute the preset alarm action corresponding to the abnormal level for abnormal nodes.

[0016] Secondly, the present invention provides an anomaly monitoring and root cause attribution system for ride-hailing funnels, comprising: The acquisition module is used to acquire the order volume time series data of each node of the ride-hailing funnel in the channel dimension, city dimension, and city × channel combination dimension, with the first preset duration as the granularity of the scrolling window; the order volume time series data includes the order volume of the current window and the order volume of the same time period of the previous day within the second preset duration. The decomposition module is used to perform a two-period STL decomposition on each single-quantity time series data, sequentially stripping the daily seasonal term and the weekly seasonal term, and extracting the target final residual corresponding to the current window from the obtained final residual sequence; The calculation module is used to calculate the conversion rate of each node based on the number of orders in the current window under the channel dimension, city dimension, and city × channel combination dimension respectively; the conversion rate is the ratio of the number of orders of the downstream node to the number of orders of the upstream node. The processing module is used to standardize the final residuals and conversion rates of each target. The detection module is used to perform anomaly detection based on the final residual and conversion rate of the target after standardization. The generation module is used to perform root cause attribution analysis based on the detection information of the abnormal nodes and external related data when the anomaly detection results show that there are abnormal nodes, and generate and output an attribution report.

[0017] This invention employs the above technical solution to provide an anomaly monitoring and root cause attribution method for a ride-hailing funnel, comprising: acquiring time-series order volume data for each node in the ride-hailing funnel at the channel, city, and city × channel combination dimensions, using a first preset duration as the granularity of the rolling window; the time-series order volume data includes the order volume of the current window and the order volume of the same time period of each day within the preceding second preset duration; performing a two-period STL decomposition on each time-series order volume data, sequentially stripping the daily and weekly seasonal terms, and extracting the target final residual corresponding to the current window from the obtained final residual sequence; calculating each conversion rate based on the order volume of each node in the current window at the channel, city, and city × channel combination dimensions; the conversion rate is the ratio of the order volume of the downstream node to the order volume of the upstream node; standardizing each target final residual and conversion rate; performing anomaly detection based on the standardized target final residual and conversion rate; and when the anomaly detection results indicate the existence of anomaly nodes, performing root cause attribution analysis based on the detection information of the anomaly nodes and external related data, generating and outputting an attribution report. Based on this, by performing dual-period STL decomposition on each single-volume time-series data and sequentially removing the daily and weekly seasonal items, this invention effectively eliminates the inherent daily and weekly dual-periodic fluctuations in business data, significantly improves the accuracy of anomaly detection, and avoids false alarms caused by normal periodic fluctuations in business. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an anomaly monitoring and root cause attribution method for a ride-hailing funnel provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of an anomaly monitoring and root cause attribution system for a ride-hailing funnel provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] Figure 1 This is a flowchart illustrating an anomaly monitoring and root cause attribution method for a ride-hailing funnel provided in an embodiment of the present invention. Figure 1 As shown, this process includes: Step 101: Using the first preset duration as the granularity of the scrolling window, obtain the time-series data of the order volume of each node in the ride-hailing funnel under the channel dimension, city dimension, and city × channel combination dimension; the time-series data of the order volume includes the order volume of the current window and the order volume of the same time period of the previous second preset duration.

[0022] In a specific example, the core transaction chain of a ride-hailing funnel is: inquiry → order placement → order acceptance → bid winning → order completion / cancellation. Therefore, the nodes include inquiry, order placement, order acceptance, bid winning, order completion, and cancellation. This invention aims to monitor the order volumes—inquiry volume, order placement volume, order acceptance volume, bid winning volume, order completion volume, and cancellation volume—in real time. Specifically, inquiry refers to the estimated price given by the platform after a passenger selects their origin and destination. Order placement refers to the passenger clicking the order button. Order acceptance refers to the company platform accepting the order, starting to find a suitable driver, and reporting back to the upstream platform. Bid winning refers to the driver successfully accepting the order. Order completion refers to the driver completing the service. Cancellation refers to the driver not completing the service.

[0023] Channels refer to the upstream order sources or traffic cooperation platforms for ride-hailing orders, such as Didi, Gaode, and Tencent. The first preset duration can be 10 minutes, and the following explanation will use a first preset duration of 10 minutes as an example. The second preset duration is longer than the first preset duration; for example, the second preset duration can be one month.

[0024] In practical applications, this invention can synchronize order data, service log data, and configuration change data from the MySQL business database in real time via DTS (Data Transmission Service). This data is then uniformly written to the DataHub message queue as a data relay hub, and different computing engines pull and consume it from DataHub. Finally, all calculated results are written to the Hologres real-time data warehouse for unified storage and use by subsequent detection modules. It should be noted that this invention uses parallel queries to obtain service log data, configuration change data, and external environment data. When the external query times out by 5 seconds, a default value is used to ensure that the rule engine can still infer based on the core data.

[0025] Specifically, for order data, this invention uses Flink to consume order event streams from DataHub in real time. Using a first preset duration as the granularity of the rolling window, it counts the number of orders for each node within the current window according to both channel and city dimensions, and writes the order count results into the Hologres database. Subsequently, based on Hologres, it is possible to obtain time-series order data for each node in the ride-hailing funnel under the channel, city, and city × channel combination dimensions.

[0026] For service log data, this invention uses Flink to consume service log data from DataHub in real time, and after parsing and cleaning, writes it to Hologres.

[0027] For configuration change data, this invention uses Flink to consume configuration change data from DataHub in real time, and after parsing and cleaning, writes it to Hologres.

[0028] In addition, this invention also collects external environmental data, such as extreme weather, urban traffic control, and major sporting events. Specifically, it uses MaxCompute's PyODPS3 script to periodically pull external environmental data and write it to Hologres once per hour.

[0029] Step 102: Perform a two-period STL (Seasonal and Trend decomposition using LOESS) decomposition on each single time series data, and successively remove the daily seasonal term and the weekly seasonal term, and extract the target final residual corresponding to the current window from the obtained final residual sequence.

[0030] Understandably, online business data typically exhibits clear daily periodicity (24-hour intervals) and weekly periodicity (7-day intervals). To eliminate the interference of these normal periodic fluctuations on anomaly detection, this invention employs an innovative two-round cascaded seasonal-trend decomposition method.

[0031] Specifically, for any single-quantity time-series data Y(t), t represents the aggregated time point sequence with a first preset time interval. Y(t) is modeled as being composed of the superposition of four independent components: Y(t) = T(t) + S_daily(t) + S_weekly(t) + R(t) In the formula, T(t) is the trend term, reflecting the long-term trend of the data; S_daily(t) is the daily seasonal term, reflecting the periodic fluctuations caused by regular changes such as morning and evening travel peaks and off-peak periods within each day; S_weekly(t) is the weekly seasonal term, reflecting the periodic fluctuations caused by the difference in travel demand between weekdays and rest days within a week; R(t) is the final residual term, which is the random fluctuation component remaining after simultaneously removing the two regular fluctuations of daily and weekly periods. The real outliers are hidden in R(t).

[0032] It should be noted that the standard STL algorithm can only process a single period at a time. This invention uses a two-round serial execution method to achieve dual-period decomposition. The first round of decomposition is executed with the daily period as the parameter, and the second round of decomposition is executed again with the weekly period as the parameter to decompose the residual from the first round. This achieves dual-period decomposition without modifying the core STL algorithm. The specific decomposition process is as follows: (1) For the current Y(t), perform the first round of STL decomposition with a daily cycle. Since the data uses a 10-minute aggregation granularity, there are a total of 144 aggregation intervals in a day, so the daily cycle parameter is set to 144.

[0033] The first round of STL decomposition decomposes Y(t) into: Y(t) = T1(t) + S_daily(t) + R1(t) In the formula, T1(t) is the trend term obtained from the first round of decomposition, S_daily(t) is the extracted daily seasonal term, and R1(t) is the residual from the first round. The residual R1(t) = Y(t) - T1(t) - S_daily(t) has eliminated the influence of daily cycle fluctuations, but still retains weekly cycle fluctuations and possible real anomaly information.

[0034] (2) The first round residual R1(t) obtained from the first round of decomposition is used as the input sequence to perform the second round of STL decomposition. Since a week contains 7 days × 144 aggregation intervals / day = 1008 aggregation intervals, the week period parameter is set to 1008.

[0035] The second round of STL decomposition further decomposes R1(t) into: R1(t) = T2(t) + S_weekly(t) + R(t) In the formula, T2(t) is the trend term obtained from the second round of decomposition, S_weekly(t) is the extracted weekly seasonal term, and R(t) is the final residual. The final residual R(t) = R1(t) - T2(t) - S_weekly(t) has now eliminated both daily and weekly fluctuations, retaining only random noise and real business anomaly signals.

[0036] After the two rounds of STL decomposition described above, the target final residual corresponding to the current window is extracted from the final residual sequence R(t). Since the final residual has eliminated the regular fluctuations of the daily and weekly cycles, its signal-to-noise ratio is significantly higher than that of the original time series data. This can effectively avoid false alarms caused by business cycle fluctuations such as morning and evening peak hours and weekend troughs, enabling the Isolation Forest to accurately identify anomalies that are truly caused by business failures or external events.

[0037] In a specific example, taking the inquiry node under the combined dimensions of Channel 1 and City 1, data from 08:00 to 08:10 each day for seven consecutive natural days is taken, as shown in Table 1: Table 1

[0038] Step 103: Calculate the conversion rate for each node based on the number of orders in the current window, under the channel dimension, city dimension, and city × channel combination dimension respectively; the conversion rate is the ratio of the number of orders in the downstream node to the number of orders in the upstream node.

[0039] Specifically, conversion rates include order placement rate, order acceptance rate, bid success rate, order completion rate, and cancellation rate. Order placement rate = number of orders placed ÷ number of inquiries; order acceptance rate = number of accepted orders ÷ number of orders placed; bid success rate = number of successful bids ÷ number of accepted orders; order completion rate = number of completed orders ÷ number of successful bids; cancellation rate = 1 - order completion rate.

[0040] Step 104: Standardize the final residuals and conversion rates for each objective.

[0041] Specifically, this invention uses the Z-Score standardization method to standardize the final residuals and conversion rates of each objective:

[0042] In the formula, Let be the standardized feature value of the j-th index; is the original feature value of the j-th indicator; The average value of the j-th indicator; Let be the standard deviation of the j-th indicator.

[0043] It should be noted that standardization processing requires... and It is calculated based on the current and historical original feature values ​​of the corresponding indicators. For example, for the inquiry node in the combination dimension of channel 1 and city 1, the historical target final residuals corresponding to the current window over the past 28 days are obtained, and the mean and standard deviation of the current target final residuals of the inquiry node and the final residuals of each historical target are calculated.

[0044] Step 105: Perform anomaly detection based on the final target residual and conversion rate after standardization.

[0045] In this embodiment of the invention, anomaly detection is performed based on the final target residual and conversion rate after standardization, which may specifically include: (1051) Based on the preset anomaly detection mechanism, an unsupervised anomaly detection algorithm is used to perform anomaly detection to obtain the first detection result; the anomaly detection mechanism includes: performing anomaly detection of the corresponding target object layer by layer according to the channel level, city level, and channel × city level under the target final residual dimension and conversion rate dimension of each node; when an anomaly is detected at the upper layer, all lower-level sub-dimensions of the anomaly are pruned; wherein, the target object is the target final residual or conversion rate; the input of the unsupervised anomaly detection algorithm is the target object, and the output is the anomaly score of the target object; when the anomaly score is greater than the preset score threshold, the corresponding target object is determined to be abnormal; the anomaly node is the node corresponding to the target object being abnormal.

[0046] Specifically, the unsupervised anomaly detection algorithm can be the Isolation Forest algorithm. The core selection criteria are as follows: First, Isolation Forest is an unsupervised algorithm that can be trained without labeled data, making it naturally suitable for business scenarios where outlier samples are extremely scarce and traditional supervised learning is difficult to effectively model. Second, the core principle of this algorithm is that outlier data points have shorter paths in the feature space and are easier to isolate. This is highly consistent with the sparse distribution characteristics of ride-hailing funnel anomalies, which occur only a handful of times a year, and can effectively identify outliers with very little prior knowledge.

[0047] In a specific example, the key parameter configurations of the isolated forest model in this invention are as follows: The number of decision trees (n_estimators) is set to 200 to integrate a sufficient number of isolated trees to ensure the stability of the detection results.

[0048] The max_samples (subsampling size) is set to 256, following the recommended value in the original algorithm literature, to ensure the training efficiency of each tree while taking into account the diversity of the model.

[0049] The contamination (abnormality ratio prior) is set to 0.03, which means that approximately 3% of the normal data is assumed to be abnormal. This value can be manually and dynamically adjusted according to the actual needs of the business for recall and precision.

[0050] Setting max_features (feature usage ratio) to 1.0 means that all standardized feature dimensions are used when splitting nodes in each tree to fully explore the joint anomaly information between multi-dimensional indicators.

[0051] The random_state (random seed) is fixed at 42 to ensure the reproducibility of model training and inference results, facilitating problem backtracking and debugging.

[0052] The above parameter combination achieves a good balance between detection accuracy and computational efficiency, and is suitable for the data characteristics and real-time requirements of ride-hailing funnel monitoring scenarios.

[0053] The formula for calculating the anomaly score s(x,n) of the x-th target object is: s(x,n)=2^(-E[h(x)] / c(n)) In the formula, E[h(x)] is the average path length of the x-th target object in all isolated trees, and c(n) is the normalization factor, which is the expected value of the average path length calculated based on the total number of samples n. The closer s(x,n) is to 1, the more abnormal the target object is; the closer s(x,n) is to 0.5, the more normal the target object is.

[0054] In a specific example, when s(x,n) is greater than or equal to 0.7, the corresponding target object is determined to be abnormal; conversely, when s(x,n) is less than 0.7, the corresponding target object is determined to be normal. Furthermore, when s(x,n) ≥ 0.85, the abnormality level of the corresponding target object is determined to be P0; when 0.75 ≤ s(x,n) < 0.85, the abnormality level of the corresponding target object is determined to be P1; and when 0.70 ≤ s(x,n) < 0.75, the abnormality level of the corresponding target object is determined to be P2.

[0055] It should be noted that the Isolation Forest algorithm can be trained using relevant data from the past 28 days. The specific training method is existing technology and will not be elaborated here.

[0056] In the process of anomaly detection using an unsupervised anomaly detection algorithm based on a preset anomaly detection mechanism, channel-level detection is performed first. Specifically, for each channel, the unsupervised anomaly detection algorithm is used to detect the target final residual and multiple conversion rates of each node under the current channel. Considering that cancellation rate = 1 - completion rate, meaning that an abnormal completion rate necessarily corresponds to an abnormal cancellation rate, this invention uses an unsupervised anomaly detection algorithm to detect anomalies in order placement rate, order acceptance rate, bid winning rate, and completion rate. Then, the abnormal state of the cancellation rate is determined based on the abnormal state of the completion rate. Similarly, other places using the unsupervised anomaly detection algorithm to detect anomalies in conversion rates do not directly use the unsupervised anomaly detection algorithm to detect anomalies in the cancellation rate; instead, the abnormal state of the cancellation rate is determined based on the abnormal state of the completion rate.

[0057] If an anomaly is detected in a target object within the current channel, such as an abnormal order rate, all lower-level sub-dimensions of the anomaly are pruned. This means that subsequent city-level and channel × city-level checks will no longer be performed on the order rate within the current channel. Simultaneously, one channel-level alarm can be output. The channel-level alarm can include the abnormal channel and the anomaly confidence level, where the anomaly confidence level is equal to the corresponding anomaly score. Furthermore, the channel-level alarm can also include the affected cities.

[0058] Secondly, city-level detection is performed. Specifically, for each city, an unsupervised anomaly detection algorithm is used to detect the final residual of the target at each node under the current city and multiple conversion rates (order placement rate, order acceptance rate, bid winning rate, and order completion rate). If an anomaly is detected in a target object under the current city, such as an anomaly in the final residual of the target at the inquiry node under the current city, all lower-level sub-dimensions of the anomaly are pruned. That is, subsequent channel × city-level detection is no longer performed on the final residual of the target at the inquiry node under the current city. At the same time, one city-level alarm can be output. The city-level alarm can include the abnormal city and the anomaly confidence level. In addition, the city-level alarm can also include the affected channels.

[0059] Finally, channel-city level detection is performed. Specifically, for each channel-city (e.g., channel 1, city 1), an unsupervised anomaly detection algorithm is used to detect the final residuals and multiple conversion rates (order placement rate, order acceptance rate, bid success rate, and order completion rate) of each node under the current channel-city. If an anomaly is detected in a target object under the current channel-city, a channel-city level alarm can be output. The channel-city level alarm can include the abnormal channel-city and the anomaly confidence level.

[0060] It should be noted that each level is configured with an isolated forest model independently, using standardized parameters and training samples from the aggregated data of its respective level, to ensure that anomaly detection at each level is based on a model that adapts to the characteristics of its data distribution.

[0061] In the above scheme, by performing anomaly detection of the corresponding target object layer by layer according to channel level, city level, and channel × city level, when an anomaly is detected at the upper level, all lower-level sub-dimensions of the anomaly are pruned. This allows the present invention to change the spatial dimension anomaly localization from "enumerating all subdivision combinations" to "rapid convergence along the causal propagation chain". The generation of redundant alarms is cut off at the source of the causal chain, thereby compressing the spatial dimension alarm quantity of a single anomaly from a scale that is linearly related to the number of cities and channels to a constant level (i.e., only 1 alarm is output for each anomaly event). The noise reduction effect becomes more and more significant with the increase of the number of channels and cities, fundamentally solving the structural problem of repeated alarms in multiple dimension combinations of the same root cause, and greatly reducing alarm noise.

[0062] (1052) Based on the preset noise reduction mechanism, the first detection result is denoised to obtain the abnormal detection result.

[0063] Specifically, the first detection result includes channel dimension sub-detection results, city dimension sub-detection results, and combined dimension sub-detection results.

[0064] Based on a preset noise reduction mechanism, the first detection result is denoised to obtain an anomaly detection result, which may specifically include: (1) For each sub-detection result, when the sub-detection result shows that there is a target abnormal node, for each target abnormal node, a combined judgment is made on whether the adjacent upstream node of the target abnormal node is abnormal and whether the conversion rate between the two nodes is abnormal, and a combined judgment result is obtained; where the target abnormal node is the abnormal node that is not the most upstream.

[0065] (2) Based on the combined judgment results, the abnormality of the target abnormal node is classified into transmission effect, independent abnormality or superposition abnormality, and the node classified as transmission effect is pruned.

[0066] Specifically, when the upstream neighbor of the target abnormal node is an abnormal node, and the conversion rate between the target abnormal node and its upstream neighbor is normal, the abnormality of the target abnormal node is classified as a transmission effect.

[0067] When the upstream neighbor of the target abnormal node is a non-abnormal node, and the conversion rate between the target abnormal node and its upstream neighbor is abnormal, the abnormality of the target abnormal node is classified as an independent abnormality.

[0068] When the upstream neighbor of the target abnormal node is an abnormal node, and the conversion rate between the target abnormal node and its upstream neighbor is abnormal, the abnormality of the target abnormal node is classified as superimposed abnormality.

[0069] The above solution introduces conversion rate as a criterion to accurately distinguish between passive fluctuations caused by upstream anomalies flowing downstream and active anomalies caused by the current link itself, from a causal transmission perspective. The former, which does not carry new fault information, is eliminated, while the latter, representing an independent root cause, is retained, thus overcoming the limitations of traditional single-index alarms. This mechanism vertically eliminates transmission redundancy between the upstream and downstream of the funnel, further reducing alarm noise.

[0070] In a specific example, taking the Gaode channel failure scenario as an example, in the traditional solution, alarms would be triggered at 6 nodes in each of the 7 cities under the Gaode channel, generating a total of 42 alarms. After adopting the anomaly detection mechanism of this invention, since the Gaode channel anomaly is detected at the channel level and pruning is triggered, the cities under it are no longer detected, reducing the number of alarms from 42 to 6. After further superimposing the noise reduction mechanism of this invention, based on the conversion rate criterion, only the inquiry volume is identified as the independent root cause of the anomaly, while the anomalies of the other 5 indicators are passive fluctuations caused by the downstream transmission of the sharp drop in inquiry volume, and are pruned accordingly. The number of alarms is reduced from 6 to 1. It can be seen that this invention can greatly reduce alarm noise.

[0071] Step 106: When the anomaly detection results show the existence of anomaly nodes, perform root cause attribution analysis based on the detection information of the anomaly nodes and external related data, and generate and output an attribution report.

[0072] Specifically, external related data includes service log data, configuration change data, and external environment data.

[0073] When anomaly detection results indicate the presence of an abnormal node, the anomaly detection layer assembles an anomaly event package (i.e., the detection information of the abnormal node) and asynchronously transmits it to the root cause attribution layer via Kafka. The anomaly event package is the unique data contract between the anomaly detection layer and the root cause attribution layer, containing the following core information fields: Basic information: unique event ID, anomaly detection time, detection level (channel level, city level, channel × city level).

[0074] Abnormal indicator information: Abnormal indicator name, abnormal score, abnormal level (P0 / P1 / P2), target final residual, and original value.

[0075] Residual snapshots for each dimension: final residual distribution of targets for each channel, final residual distribution of targets for each city, and final residual matrix of cross-targets for each channel and city.

[0076] STL decomposition result snapshot: trend term, daily seasonal term, weekly seasonal term, target final residual.

[0077] Funnel-based end-to-end residual snapshot: the final target residual of all nodes.

[0078] Conversion Rate Snapshot: The current value of the conversion rate between 4 adjacent nodes.

[0079] Funnel pruning results: list of independent anomalous nodes, list of superimposed anomalous nodes, list of nodes with transmission effects, and pruning details.

[0080] Conversion rate anomaly detection results: anomaly scores for 4 conversion rates.

[0081] In this embodiment of the invention, root cause attribution analysis is performed based on the detection information of abnormal nodes and external correlation data, which may specifically include: (1) Combine detection information and external related data into an attribution context.

[0082] (2) The attribution context is sent to the rule engine for deterministic reasoning to obtain the deterministic reasoning result; the rule engine contains multiple attribution rules, and each attribution rule is configured with a preset confidence level.

[0083] Specifically, the rule engine is used to match each attribution rule one by one according to a preset priority. The rule engine of this invention was built by technical personnel based on prior business knowledge, and each attribution rule is defined as a structured logical expression of "condition → conclusion + confidence level".

[0084] In a specific example, this invention predefines 13 attribution rules. The conclusion section points to one of the four root cause categories (channel-side problems, R&D service-side problems, operational configuration problems, and external environmental influences). The confidence level is a value between 0 and 1, used to quantify the credibility of the rule when it is true. The 13 attribution rules are shown in Table 2: Table 2

[0085] (3) If the deterministic reasoning result shows that an attribution rule is hit, output the attribution rule and its confidence level.

[0086] If the deterministic inference result shows that multiple attribution rules are matched, then among the matched attribution rules, the attribution rule with the highest confidence is determined as the primary root cause; it is then determined whether there are any secondary root causes among the other attribution rules among the matched attribution rules whose confidence difference with the primary root cause is less than a preset difference threshold. If so, a multi-cause superposition label is generated, and the primary root cause, secondary root cause, relevant confidence, and multi-cause superposition label are output; otherwise, the primary root cause and its confidence are output.

[0087] After performing root cause attribution analysis based on the detection information of abnormal nodes and external related data, the condition values ​​of the output attribution rules are filled into the preset evidence chain template to obtain the evidence chain. The evidence chain and the attribution analysis output data are then used to generate and output the attribution report.

[0088] In this embodiment of the invention, after generating and outputting the attribution report, the method of the present invention may further include: (1) For each abnormal node, determine the abnormal score of the corresponding target object as the abnormal score of the abnormal node.

[0089] (2) Select the target score interval where the abnormal score of the abnormal node is located from multiple preset score intervals.

[0090] (3) Determine the abnormality level of the abnormal node as the abnormality level corresponding to the target score interval.

[0091] (4) Execute the preset alarm action corresponding to the abnormal level for the abnormal node.

[0092] That is, when s(x,n)≥0.85, the anomaly level of the corresponding target object is determined to be P0; when 0.75≤s(x,n)<0.85, the anomaly level of the corresponding target object is determined to be P1; and when 0.70≤s(x,n)<0.75, the anomaly level of the corresponding target object is determined to be P2.

[0093] In addition, the preset alarm actions for P0 can include push notifications via WeChat / DingTalk and telephone calls to ensure timely response from on-duty personnel. The preset alarm actions for P1 can be push notifications only via WeChat / DingTalk. The preset alarm actions for P2 can be simply marked on the dashboard without pushing immediate messages.

[0094] This invention also allows for manual review of attribution reports. If an error in the root cause classification is found in the attribution report, the attribution report can be manually corrected and written back to a preset annotation database for rule engine optimization, forming a closed loop of continuous improvement.

[0095] Based on a general inventive concept, the present invention also provides an anomaly monitoring and root cause attribution system for ride-hailing funnels, which is applied to the anomaly monitoring and root cause attribution method for ride-hailing funnels as described above. Figure 2 This is a schematic diagram of the structure of an anomaly monitoring and root cause attribution system for a ride-hailing funnel provided in an embodiment of the present invention. Figure 2 As shown, this system includes: The acquisition module 21 is used to acquire the order volume time series data of each node of the ride-hailing funnel in the channel dimension, city dimension and city × channel combination dimension with the first preset duration as the scrolling window granularity; the order volume time series data includes the order volume of the current window and the order volume of the same time period of the day within the second preset duration.

[0096] The decomposition module 22 is used to perform a two-period STL decomposition on each single-quantity time series data, sequentially stripping the daily seasonal term and the weekly seasonal term, and extracting the target final residual corresponding to the current window from the obtained final residual sequence.

[0097] The calculation module 23 is used to calculate the conversion rate of each node based on the number of orders in the current window under the channel dimension, city dimension, and city × channel combination dimension respectively; the conversion rate is the ratio of the number of orders of the downstream node to the number of orders of the upstream node.

[0098] Processing module 24 is used to standardize the final residuals and conversion rates of each target.

[0099] The detection module 25 is used to perform anomaly detection based on the final residual and conversion rate of the target after standardization.

[0100] The generation module 26 is used to perform root cause attribution analysis based on the detection information of the abnormal node and external related data when the anomaly detection results show that there is an abnormal node, and generate and output an attribution report.

[0101] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0102] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0103] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0104] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0105] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0106] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0107] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0108] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0109] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for anomaly monitoring and root cause attribution in a ride-hailing funnel, characterized in that, include: Using the first preset duration as the granularity of the scrolling window, the order volume time series data of each node in the ride-hailing funnel is obtained in the dimensions of channel, city, and city × channel combination. The order volume time series data includes the order volume of the current window and the order volume of the same time period of the previous second preset duration. Perform a two-period STL decomposition on each single-quantity time series data, and successively remove the daily seasonal term and the weekly seasonal term. Extract the target final residual corresponding to the current window from the obtained final residual sequence. Calculate the conversion rate for each node based on the number of orders at the current window, under the channel dimension, city dimension, and city × channel combination dimension respectively; the conversion rate is the ratio of the number of orders at the downstream node to the number of orders at the upstream node. Standardize the final residuals and conversion rates for each objective; Anomaly detection is performed based on the final target residual and conversion rate after standardization. When the anomaly detection results show the presence of anomaly nodes, root cause attribution analysis is performed based on the detection information of the anomaly nodes and external related data, and an attribution report is generated and output.

2. The method for anomaly monitoring and root cause attribution of the ride-hailing funnel according to claim 1, characterized in that, Anomaly detection is performed based on the standardized final residual and conversion rate, specifically including: Based on a pre-defined anomaly detection mechanism, an unsupervised anomaly detection algorithm is used to perform anomaly detection, resulting in a first detection result. The anomaly detection mechanism includes: performing anomaly detection layer by layer at the channel level, city level, and channel × city level for each node's final residual and conversion rate dimensions; pruning all lower-level sub-dimensions of the anomaly detected at the upper level; where the target object is the final residual or conversion rate; the input of the unsupervised anomaly detection algorithm is the target object, and the output is the anomaly score of the target object; when the anomaly score is greater than a pre-defined score threshold, the corresponding target object is determined to be anomaly; the anomaly node is the node corresponding to the anomaly of the target object. Based on a preset noise reduction mechanism, the first detection result is denoised to obtain the anomaly detection result.

3. The method for anomaly monitoring and root cause attribution of the ride-hailing funnel according to claim 2, characterized in that, The first detection result includes channel dimension sub-detection results, city dimension sub-detection results, and combined dimension sub-detection results; Based on a preset noise reduction mechanism, the first detection result is denoised to obtain the anomaly detection result, specifically including: For each sub-detection result, when the sub-detection result shows that there is a target abnormal node, for each target abnormal node, a combined judgment is made on whether the adjacent upstream node of the target abnormal node is abnormal and whether the conversion rate between the two nodes is abnormal, to obtain a combined judgment result; where the target abnormal node is the abnormal node that is not the most upstream. Based on the combined judgment results, the anomalies of the target abnormal nodes are classified as transmission effects, independent anomalies, or superimposed anomalies, and nodes classified as transmission effects are pruned.

4. The method for anomaly monitoring and root cause attribution of the ride-hailing funnel according to claim 3, characterized in that, Based on the combined judgment results, the anomalies of the target abnormal nodes are classified into transmission effects, independent anomalies, or superimposed anomalies, specifically including: When the upstream neighbor of the target abnormal node is an abnormal node, and the conversion rate between the target abnormal node and its upstream neighbor is normal, the abnormality of the target abnormal node is classified as a transmission effect. When the upstream neighbor of the target abnormal node is a non-abnormal node, and the conversion rate between the target abnormal node and its upstream neighbor is abnormal, the abnormality of the target abnormal node is classified as an independent abnormality. When the upstream neighbor of the target abnormal node is an abnormal node, and the conversion rate between the target abnormal node and its upstream neighbor is abnormal, the abnormality of the target abnormal node is classified as superimposed abnormality.

5. The method for anomaly monitoring and root cause attribution of the ride-hailing funnel according to claim 1, characterized in that, Root cause analysis is performed based on the detection information of abnormal nodes and external correlation data, specifically including: Combine detection information and external correlation data into an attribution context; The attribution context is fed into the rule engine for deterministic reasoning to obtain the deterministic reasoning result; the rule engine contains multiple attribution rules, and each attribution rule is configured with a preset confidence level; If the deterministic inference result shows that an attribution rule is matched, then output the attribution rule and its confidence level.

6. The method for anomaly monitoring and root cause attribution of the ride-hailing funnel according to claim 5, characterized in that, Also includes: If the deterministic inference results show that multiple attribution rules are hit, then among the hit attribution rules, the attribution rule with the highest confidence is identified as the principal cause. Determine if there are any secondary root causes among the other attribution rules in each hit attribution rule that have a confidence difference with the primary root cause that is less than a preset difference threshold. If so, generate a multi-factor overlay label and output the primary root cause, secondary root cause, relevant confidence, and multi-factor overlay label. Otherwise, output the primary root cause and its confidence level.

7. The method for anomaly monitoring and root cause attribution of the ride-hailing funnel according to claim 5, characterized in that, The rule engine is used to match each attribution rule one by one according to preset priorities.

8. The method for anomaly monitoring and root cause attribution of the ride-hailing funnel according to claim 2, characterized in that, The unsupervised anomaly detection algorithm is the Isolation Forest algorithm.

9. The method for anomaly monitoring and root cause attribution of the ride-hailing funnel according to claim 2, characterized in that, After generating and outputting the attribution report, the following is also included: For each abnormal node, the abnormal score of the corresponding target object is determined as the abnormal score of the abnormal node; Select the target score range where the abnormal score of the abnormal node is located from multiple preset score ranges; The anomaly level corresponding to the target score range is determined as the anomaly level of the anomaly node; Execute the preset alarm action corresponding to the abnormal level for abnormal nodes.

10. An anomaly monitoring and root cause attribution system for ride-hailing funnels, characterized in that, include: The acquisition module is used to acquire the order volume time series data of each node of the ride-hailing funnel in the channel dimension, city dimension, and city × channel combination dimension, with the first preset duration as the granularity of the scrolling window; the order volume time series data includes the order volume of the current window and the order volume of the same time period of the previous day within the second preset duration. The decomposition module is used to perform a two-period STL decomposition on each single-quantity time series data, sequentially stripping the daily seasonal term and the weekly seasonal term, and extracting the target final residual corresponding to the current window from the obtained final residual sequence; The calculation module is used to calculate the conversion rate of each node based on the number of orders in the current window under the channel dimension, city dimension, and city × channel combination dimension respectively; the conversion rate is the ratio of the number of orders of the downstream node to the number of orders of the upstream node. The processing module is used to standardize the final residuals and conversion rates of each target. The detection module is used to perform anomaly detection based on the final residual and conversion rate of the target after standardization. The generation module is used to perform root cause attribution analysis based on the detection information of the abnormal nodes and external related data when the anomaly detection results show that there are abnormal nodes, and generate and output an attribution report.

Images