A large-scale alarm deployment method, device and equipment

Abstract

The embodiments of this specification disclose a large-scale alarm deployment method, device and equipment. The scheme includes: determining whether an indicator in a large-scale indicator set is a non-long-tail indicator or a long-tail indicator; for the non-long-tail indicator, using a first time interval to perform task scheduling on the flow of the non-long-tail indicator, and performing alarm calculation by executing the task; for the long-tail indicator, performing aggregation processing on the flow of the long-tail indicator, using a second time interval longer than the first time interval, performing task scheduling on the corresponding aggregated data, and performing alarm calculation by executing the task.

Description

Technical Field

[0001] The present invention relates to the field of security monitoring technology, and in particular to a large-scale alarm deployment method, device and equipment. Background Art

[0002] With the development of Internet technology and the popularization of mobile terminals, most businesses can be conducted online based on mobile applications.

[0003] For some large and medium-sized applications or companies, there are massive businesses in their application ecosystems, involving massive indicators that need to be monitored in order to detect anomalies and alarms in a timely manner. The indicators mentioned here can be understood as objects or parameters in a certain dimension, such as massive merchants, massive applications, massive applets, massive commodities, etc. Since the number of indicators is very large, it may even reach tens of millions (for example, tens of millions of merchants, etc.). For traditional monitoring platforms, from data collection to alarm notification, they often use the same set of processes, which consumes huge storage and computing resources, and the alarm noise is large, making it difficult to accurately and comprehensively hit key risk points.

[0004] Based on this, a better alarm deployment solution is needed for large-scale indicators. Summary of the invention

[0005] One or more embodiments of the present specification provide a large-scale alarm deployment method, apparatus, device, and storage medium to solve the following technical problem: a better alarm deployment solution for large-scale indicators is needed.

[0006] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0007] One or more embodiments of this specification provide a large-scale alarm deployment method, including:

[0008] Determine whether the indicators in the large-scale indicator set are non-long-tail indicators or long-tail indicators;

[0009] For the non-long-tail indicator, a first time interval is used to perform task scheduling on the traffic of the non-long-tail indicator, and alarm calculation is performed by executing the task;

[0010] For the long-tail indicator, the traffic of the long-tail indicator is aggregated, a second time interval longer than the first time interval is used, task scheduling is performed on the corresponding aggregated data, and alarm calculation is performed by executing the task.

[0011] One or more embodiments of this specification provide a large-scale alarm deployment device, including:

[0012] The long-tail distinction module determines whether an indicator in a large-scale indicator set is a non-long-tail indicator or a long-tail indicator;

[0013] A non-long-tail processing module, for the non-long-tail indicator, uses a first time interval to perform task scheduling on the flow of the non-long-tail indicator, and performs alarm calculation by executing the task;

[0014] The long-tail processing module aggregates the traffic of the long-tail indicators, uses a second time interval longer than the first time interval to schedule tasks for the corresponding aggregated data, and performs alarm calculations by executing the tasks.

[0015] One or more embodiments of this specification provide a large-scale alarm deployment device, including:

[0016] at least one processor; and,

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0019] Determine whether the indicators in the large-scale indicator set are non-long-tail indicators or long-tail indicators;

[0020] For the non-long-tail indicator, a first time interval is used to perform task scheduling on the traffic of the non-long-tail indicator, and alarm calculation is performed by executing the task;

[0021] For the long-tail indicator, the traffic of the long-tail indicator is aggregated, a second time interval longer than the first time interval is used, task scheduling is performed on the corresponding aggregated data, and alarm calculation is performed by executing the task.

[0022] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to:

[0023] Determine whether the indicators in the large-scale indicator set are non-long-tail indicators or long-tail indicators;

[0024] For the non-long-tail indicator, a first time interval is used to perform task scheduling on the traffic of the non-long-tail indicator, and alarm calculation is performed by executing the task;

[0025] For the long-tail indicator, the traffic of the long-tail indicator is aggregated, a second time interval longer than the first time interval is used, task scheduling is performed on the corresponding aggregated data, and alarm calculation is performed by executing the task.

[0026] At least one of the above-mentioned technical solutions adopted in one or more embodiments of the present specification can achieve the following beneficial effects: in a large-scale indicator set, by distinguishing a small number of high-frequency traffic indicators from a large number of low-frequency traffic indicators, long-tail indicators that were previously difficult to monitor accurately are effectively identified. For the traffic of long-tail indicators (referred to as long-tail traffic), considering that the accumulated total traffic cannot be ignored, but is often very discrete and sparse in time series, a relatively long time interval is used to aggregate it and then perform alarm calculations. In this way, it can more accurately and comprehensively cover the risk points that were previously easily overlooked, and reduce the pressure on traffic data storage. For non-long-tail traffic, a relatively short time interval is used to perform alarm calculations. In this way, through the above-mentioned hierarchical processing of long-tail traffic and non-long-tail traffic, a more refined alarm deployment capability is achieved. While taking into account large-scale indicators in an all-round way, processing resources are truly used where they are needed most, which can effectively reduce business risks and monitoring costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0028] Figure 1 A flowchart of a large-scale alarm deployment method provided for one or more embodiments of this specification;

[0029] Figure 2 In a practical application scenario provided by one or more embodiments of this specification, Figure 1 A schematic diagram of an implementation scheme of the method;

[0030] Figure 3 A schematic diagram of dynamic fresh-keeping of merchant levels provided in one or more embodiments of this specification;

[0031] Figure 4 One or more embodiments of this specification provide Figure 2 A process timing diagram involved in the scheme;

[0032] Figure 5A schematic diagram of the structure of a large-scale alarm deployment device provided in one or more embodiments of this specification;

[0033] Figure 6 A schematic diagram of the structure of a large-scale alarm deployment device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0034] The embodiments of this specification provide a large-scale alarm deployment method, device, equipment and storage medium.

[0035] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0036] The background technology mentioned that for traditional monitoring platforms, from data collection to alarm notification, the same set of processes are often used. However, in actual scenarios, there are huge differences between indicators at the priority level and data level. Therefore, using traditional monitoring methods will lead to a series of specific problems such as excessive resource consumption, untimely detection of core business alarms, and high noise of non-core business alarms. Especially in the scenario of massive indicators, these problems will be more prominent, making it difficult to truly and effectively deploy alarms. Based on this, this application is committed to mainly targeting massive indicator scenarios, starting from the perspective of indicators, and trying to enable all core and non-core businesses to have monitoring and emergency response capabilities.

[0037] Based on actual business data experience and statistical theory, it is found that in the massive indicator scenario, a large number of indicators with rare demand constitute a huge number, which is one of the root causes of the above-mentioned consumption problem, long-tail traffic.

[0038] Take the indicator as an example of merchant analysis. If there are a large number of merchants on the e-commerce platform, it is necessary to monitor the traffic of a large number of merchants, so as to warn and deal with emergencies when the traffic is abnormal, and the transaction business data of the corresponding merchant can be selected as the monitoring object in the traffic. In the transaction link of the merchant, the commonly used monitoring perspective is to monitor and warn all merchants from the platform side, so that it can be observed from a global perspective whether the overall transaction link has a fault, but in actual applications, most of the traffic or large blocks of concentrated traffic are contributed by a smaller number of core merchants (for example, more than one thousand large and medium-sized customers) in the total number of merchants, and the remaining traffic is contributed by a large number of non-core customers (for example, millions or even tens of millions of small merchants). The traffic generated by these non-core merchants is very small and low-frequency, resulting in a wide and sparse distribution. In the embodiment of the present application, such traffic is called long-tail traffic, and monitoring and warning of long-tail traffic requires a lot of resources. In addition, it is difficult to find when there is a problem with long-tail traffic, so it is impossible to warn in time and carry out emergency stop loss processing.

[0039] In order to ensure that such non-core indicators corresponding to long-tail traffic can also have good monitoring and alarm capabilities, while enabling the core indicators corresponding to non-long-tail traffic to be effectively monitored, and avoiding excessive resource consumption and traffic overload, the indicators corresponding to long-tail traffic and non-long-tail traffic are distinguished based on data characteristics. In some embodiments, the huge long-tail indicators that constitute the massive indicators are optionally advanced graded. According to the classification results, the monitoring data is routed to a targeted execution strategy, and the indicators and their traffic at different levels are differentiated in terms of time intervals and aggregation granularity. The following is a detailed description based on this idea.

[0040] Figure 1 A flowchart of a large-scale alarm deployment method provided for one or more embodiments of this specification. The method can be applied to different business fields, such as e-commerce business field, electronic payment business field, social business field, game business field, official business field, etc. The process can be executed on the risk control related equipment of the platform in these fields. Certain input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.

[0041] Figure 1 The process in includes the following steps:

[0042] S102: Determine long-tail indicators and non-long-tail indicators in a large-scale indicator set.

[0043] In one or more embodiments of the present specification, the characteristics of long-tail indicators are that they are large in number and proportion in a large-scale indicator set, and their individual traffic is low-frequency, that is, a large number of low-frequency traffic indicators. Conversely, non-long-tail indicators are small in number and proportion in a large-scale indicator set, and their individual traffic is high-frequency, that is, a small number of high-frequency traffic indicators.

[0044] Taking the merchant indicator as an example, for merchants, their traffic, such as transaction business data, is composed of transactions with a time series. The long-tail indicator can be a large number of merchants with low transaction frequency, that is, long-tail merchants, and the non-long-tail indicator is a small number of merchants with high transaction frequency, that is, non-long-tail merchants.

[0045] Taking the mini program as an example, its traffic, such as click-through data, can be used to distinguish between mini programs with a large number of low-frequency clicks as long-tail mini programs and mini programs with a small number of high-frequency clicks as non-long-tail mini programs.

[0046] Taking performance parameters as an example, the traffic is the collected performance parameter performance data, which can distinguish a large number of low-frequency performance parameters (for example, the number of overloads, the number of memory overflows, etc.) and a small number of high-frequency performance parameters (for example, real-time transmission rate, CPU occupancy, etc.).

[0047] It should be noted that low-frequency traffic and high-frequency traffic are relative concepts. How high is considered high and how low is considered low should also be determined by referring to the number in the large-scale indicator set. Most or even most of the indicators with relatively low-frequency traffic in the large-scale indicator set can be classified as long-tail indicators. Only in this way can we identify the truly long-tail traffic, and then we can give full play to the advantages of this solution.

[0048] In one or more embodiments of the present specification, a periodic adjustment can be set for long-tail indicators and non-long-tail indicators. For example, a ratio is specified, and each adjustment period is automatically triggered to re-divide the long-tail indicators and non-long-tail indicators in the large-scale indicator set according to the latest situation of the traffic frequency of each indicator according to the ratio.

[0049] In one or more embodiments of the present specification, long-tail and non-long-tail indicators may be adjusted manually. For example, in actual applications, the flow of a certain indicator is relatively high-frequency, but very irregular and uneven, such as in the form of a pulse with a high peak value. In this case, in order to monitor the indicator more comprehensively, the indicator may be manually determined to be a long-tail indicator.

[0050] In one or more embodiments of the present specification, the two major categories of indicators divided above can be further subdivided in each category according to actual needs.

[0051] Taking non-long-tail indicators as an example, for example, according to the high frequency of traffic, they can be subdivided into higher-frequency traffic indicators, higher-frequency traffic indicators, etc.

[0052] As for long-tail indicators, the situation is more complicated due to the huge number of indicators, and the heterogeneity between indicators may be more prominent. Therefore, if further segmentation is to be carried out, it may not be based on or may not be mainly based on the degree of high frequency of traffic. Taking merchants as an example, for example, characteristics such as region, product type, target customers, and position in the industrial chain can also be considered. Segmentation can be based on a variety of characteristics to make the segmented categories more homogeneous.

[0053] S104: For the non-long-tail indicator, use a first time interval to perform task scheduling on the traffic of the non-long-tail indicator, and perform alarm calculation by executing the task.

[0054] In one or more embodiments of the present specification, for long-tail traffic and non-long-tail traffic, differentiated time intervals are used to perform task scheduling. The tasks here include at least alarm calculation, and may also include corresponding traffic collection and preprocessing.

[0055] The traffic of non-long-tail indicators belongs to non-long-tail traffic, which is relatively more core and important. Therefore, the scheduling cycle can be shortened and scheduling can be performed at shorter time intervals. For example, detailed traffic data at the second level can be collected and second-level scheduling can be performed. If the non-long-tail indicators are subdivided, the first time interval can also be subdivided accordingly. For example, for higher-frequency indicators of traffic, second-level scheduling is used, and for higher-frequency indicators of traffic, minute-level scheduling is used, and so on.

[0056] S106: For the long-tail indicator, aggregate the traffic of the long-tail indicator, use a second time interval longer than the first time interval, perform task scheduling on the corresponding aggregated data, and perform alarm calculation by executing the task.

[0057] The number of long-tail indicators is huge, which is not only not conducive to accurate monitoring, but also not conducive to effective storage and query. Assuming that the same standard and unified processing are adopted as non-long-tail indicators, for example, minute-level or even second-level collection and storage, there will be at least two problems: first, there is no valid data most of the time, which makes monitoring meaningless; second, the sparse individual traffic data easily leads to inefficient and unreasonable storage structure, which will bring huge pressure to storage at the level of tens of millions.

[0058] Based on this, on the one hand, compared with non-long-tail indicators, the scheduling cycle is lengthened and a longer time interval is used for scheduling, for example, at least not less than the minute level, and can be extended to the hour level or even longer as needed; on the other hand, the traffic of long-tail indicators is aggregated, for example, multiple transactions within a specified time interval (for example, a time interval not longer than the second time interval) are statistically merged or feature nonlinearly fused and aggregated into one transaction, thereby making sparse traffic more concentrated, so as to perform targeted alarm calculations, reduce the number of task scheduling times, and at the same time improve the effectiveness of each task scheduling.

[0059] If necessary, aggregation can also be applied to some non-long-tail traffic, for example, for higher-frequency indicators of traffic, such as minute-level aggregation processing. It should be noted that there is still a difference between the two. For non-long-tail traffic, assuming that aggregation processing is required, online aggregation can be used first, such as nonlinear high-dimensional feature fusion of multiple transactions based on online machine learning models, and remapping them into one transaction. For long-tail traffic, offline aggregation methods with lower cost and real-time requirements can be considered, such as aggregation processing through offline archiving, which is more in line with the characteristics of long-tail traffic with small and widely distributed traffic.

[0060] pass Figure 1 The method effectively identifies long-tail indicators that were difficult to monitor accurately in the past by distinguishing a small number of high-frequency traffic indicators from a large number of low-frequency traffic indicators in a large-scale indicator set. For the traffic of long-tail indicators (called long-tail traffic), considering that the accumulated total traffic cannot be ignored, but is often very discrete and sparse in time series, a relatively long time interval is used to aggregate it and then calculate the alarm. In this way, it can more accurately and comprehensively cover the risk points that were easily overlooked in the past, and reduce the pressure on traffic data storage. For non-long-tail traffic, a relatively short time interval is used to calculate the alarm. In this way, through the above hierarchical processing of long-tail traffic and non-long-tail traffic, a more refined alarm deployment capability is achieved. While taking into account large-scale indicators in an all-round way, processing resources are truly used on the cutting edge, which can effectively reduce business risks and monitoring costs.

[0061] based on Figure 1 This specification also provides some specific implementation plans and extension plans of the method. For a more intuitive understanding, the following description is mainly based on the merchant scenario.

[0062] In one or more embodiments of the present specification, it is mentioned above that the heterogeneity of long-tail traffic is more prominent. Therefore, in addition to high-level classification to distinguish long-tail traffic from non-long-tail traffic, it is also possible to delve deeper into the long-tail traffic and focus on further reclassification of the long-tail traffic.

[0063] For example, multiple different levels of second time intervals (e.g., minutes, hours, etc.) that are longer than the first time interval (e.g., at least longer than seconds) are determined, and for the current long-tail merchant or the current moment, the second time interval hit by it in the second time intervals at multiple different levels is determined as the dynamic time interval. The dynamic time interval is used to aggregate the transaction business data of the current long-tail merchant and / or its associated long-tail merchants (e.g., temporarily retaining some unprocessed long-tail traffic in the previous cycle, etc.), and task scheduling is performed on the corresponding aggregated data.

[0064] The dynamics mentioned here may include dynamics in the dimension of merchant conditions, for example, according to the low frequency or importance of transactions of long-tail merchants, multiple different levels of long-tail users are classified, and for long-tail merchants with lower frequency or more important, a longer second time interval is used; dynamics may also include dynamics in the dimension of moment conditions, for example, dynamically using a shorter second time interval when non-long-tail traffic is easing, dynamically using a shorter second time interval when the number of abnormal reports from long-tail merchants increases, and using a longer second time interval at business peaks, etc. The dynamic setting of the second time interval can be intelligently and adaptively performed to strike a balance between fully and effectively utilizing resources and better serving long-tail merchants.

[0065] Similarly, non-long-tail traffic can be segmented more simply. For example, when it is determined that an indicator in a large-scale indicator set is a non-long-tail indicator, at least two categories of non-long-tail merchants are determined based on different degrees of transaction high frequency, and the different degrees of transaction high frequency in the first time interval are negatively correlated. It is determined which of the at least two categories the merchant in the large-scale merchant set belongs to, so as to use the corresponding first time interval (e.g., seconds or minutes, etc.) for task scheduling.

[0066] In one or more embodiments of the present specification, when performing alarm calculations through task scheduling, for non-long-tail traffic, its homogeneity is relatively higher. In order to improve calculation efficiency, a unified rule threshold can be set without distinguishing between different merchants. Alarm calculations are performed by triggering the corresponding rule threshold detection process. This method is direct and efficient, and avoids complex calculation processes.

[0067] However, for long-tail traffic, especially sparser and less regular traffic, the applicability of a unified rule threshold will be reduced, and it is difficult to set a reliable rule threshold for direct comparison. Therefore, based on a machine learning model, intelligent algorithm detection can be used to adaptively and reliably trigger alarms through feature high-dimensional mapping and mining. Specifically, it can be determined whether the above-mentioned dynamic time interval is long enough. If so, and the industry characteristics and / or historical data of the corresponding transaction business data can be obtained (for example, using these data in advance for model training, or, for current long-tail merchants, more immediate small sample learning to fine-tune the model), then according to the industry characteristics and / or historical data, the intelligent algorithm detection process is triggered to perform alarm calculations. Otherwise, the rule threshold detection process can be triggered to perform alarm calculations.

[0068] According to the above description, one or more embodiments of this specification also provide a practical application scenario. Figure 1 A schematic diagram of an implementation scheme of the method is shown in FIG. Figure 2 shown.

[0069] In this scenario, firstly, based on merchant characteristics (e.g., transaction volume, transaction frequency, etc.), three categories of merchants are divided from the large-scale merchant collection: the most important first category of non-long-tail merchants, such as some core leading merchants with a small number of merchants but high transaction frequency; the second most important second category of non-long-tail merchants, such as some leading merchants with a small number of merchants but high transaction frequency; and long-tail merchants, such as some ordinary merchants with a large number of merchants but low transaction frequency.

[0070] For different types of merchants, we adopt refined transaction business data collection and aggregation methods. For example, for core head merchants, the monitoring data is detailed data at the second level, for head merchants, the monitoring data is data at the minute level, and for ordinary merchants, it can be data above the minute level. For the transaction business data of ordinary users, due to the small traffic and wide distribution, it can be aggregated by archiving (for example, offline archiving). For the transaction business data of head merchants, it can also be more immediate online aggregation processing.

[0071] For ordinary merchants, the corresponding time intervals are more flexible. They can be minute-level (for example, data is archived once every n* minutes), hour-level (for example, data is archived once every n* hours), other custom archiving cycles, and so on. It should be noted that since the number of ordinary merchants is very large, ordinary merchants can be classified first and then dynamically adjusted. In this case, at the same time, there may be ordinary merchants corresponding to the minute level, ordinary merchants corresponding to the hour level, and ordinary merchants corresponding to other custom levels. This multi-layer dynamic grading method for long-tail merchants can give different long-tail merchants more adequate and detailed attention in terms of both intensity and granularity, but not excessive attention, which helps to fully tap the hidden risk points in long-tail traffic.

[0072] In order to better differentiate merchants in accordance with the facts, a merchant grade dimension table is generated and maintained, and the corresponding grade is used to dynamically represent the category of the merchant, so that the current category information is fresh and reliable. Figure 3 . Figure 3 A schematic diagram of dynamic freshness preservation of merchant levels provided in one or more embodiments of this specification.

[0073] exist Figure 3 In the platform, a corresponding merchant level maintenance page can be provided, and merchant-level creation or adjustment operations can be performed on the page according to the classification results. The platform can generate and maintain a merchant level dimension table for dynamically recording and updating operation results. In addition to including the merchant level representing the corresponding category, the operation result can also include the time interval corresponding to the level for task scheduling. The time interval will change according to the change of the merchant level.

[0074] The dispatching system performs refined group scheduling based on the level, time interval and other information recorded in the merchant level dimension table. Different levels generate monitoring and detection tasks with different frequencies, and then specific alarm calculations can be performed.

[0075] Based on this, continue Figure 2 For explanation. Merchants of different levels can have different data collection and detection task scheduling cycles, and the above time interval is the length of one cycle. The data collection cycle and the detection task scheduling cycle can adapt to each other, for example, try to keep them consistent so that they can be connected more smoothly. In this way, the detection task can be scheduled according to the data collection cycle. For example, for core head merchants, the data collection cycle is at the second level, so the detection task can also be scheduled at the second level.

[0076] Data collection and detection tasks are scheduled periodically according to the time interval corresponding to the merchant level. At least for the transaction business data of ordinary users, additional aggregation processing is performed after collection according to the time interval, and the corresponding aggregated data is stored on the ground. Later, the detection can be adapted to the time interval, and the corresponding aggregation time corresponding to the aggregation processing (the data before aggregation corresponds to multiple different times, and after aggregation, it is represented by a unified time, which is called the aggregation time) is queried out, and tasks are scheduled accordingly, which can effectively reduce storage and query pressure.

[0077] When the alarm calculation is performed after the detection task is triggered, rule threshold detection and intelligent algorithm detection are supported. The former, for example, determines whether an alarm is needed through a given threshold and operator, while the latter, for example, determines whether it is abnormal through historical data and industry characteristics. As mentioned earlier, adaptive detection methods can be selected for long-tail traffic in particular.

[0078] In order to improve efficiency, alarms are actively merged during the alarm calculation process to avoid ignoring key risks due to excessive alarm noise. Specifically, it includes: detecting whether there are abnormalities in transaction business data according to the specified monitoring items, generating corresponding abnormal events if they are, and merging alarms for multiple abnormal events belonging to the same monitoring item to uniformly initiate alarm notifications to relevant emergency personnel. As a result, emergency personnel may originally receive multiple alarms with similar attributions or performances, but after merging, they can only receive one alarm.

[0079] Furthermore, in the merchant scenario, timeliness and timeliness are particularly important. Therefore, for data collection and detection during the monitoring process, one or more embodiments of this specification also provide Figure 2 The solution in the example involves a process timing diagram, such as Figure 4 shown.

[0080] Figure 4 The process in includes the following steps:

[0081] The data collection service (or module) collects, cleans, and generates corresponding time series data through structured logs that reflect transaction business data recorded by the system. If aggregation is required, it is aggregated according to the corresponding merchant level. It can then be used for monitoring or other business processes.

[0082] During the monitoring process, the time series data is stored in the time series database, and the data writing completion notification signal is sent to the alarm computing service.

[0083] The alarm computing service detects the alarm configuration and related time series data of the user.

[0084] If the alarm trigger conditions are met, the emergency process will be initiated and emergency personnel will be called out.

[0085] Of course, the monitoring process may also optionally include a maintenance process for the merchant level dimension table.

[0086] Based on the above solution, it is possible to truly realize comprehensive and accurate monitoring and alarm deployment of massive indicators. In the merchant scenario, it can especially help a large number of small merchants eliminate risks in a timely manner and improve the robustness and user experience of the entire platform.

[0087] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figure 5 , Figure 6 The apparatus and device can accordingly execute the above method and related optional solutions.

[0088] Figure 5 A schematic diagram of the structure of a large-scale alarm deployment device provided in one or more embodiments of this specification, the device comprising:

[0089] A long-tail distinguishing module 502 determines whether an indicator in a large-scale indicator set is a non-long-tail indicator or a long-tail indicator;

[0090] The non-long tail processing module 504 performs task scheduling on the flow of the non-long tail indicator at a first time interval for the non-long tail indicator, and performs alarm calculation by executing the task;

[0091] The long-tail processing module 506 aggregates the traffic of the long-tail indicator, uses a second time interval longer than the first time interval to perform task scheduling on the corresponding aggregated data, and performs alarm calculation by executing the task.

[0092] Optionally, the indicator is a merchant, the non-long-tail indicator is a non-long-tail merchant, the long-tail indicator is a long-tail merchant, and the traffic is transaction business data of the corresponding merchant.

[0093] Optionally, the long-tail processing module 506 determines a second time interval of a plurality of different levels that is longer than the first time interval;

[0094] For the current long-tail merchant or the current moment, determine the second time interval hit by the merchant in the second time intervals of the multiple different levels as the dynamic time interval;

[0095] The dynamic time interval is used to aggregate the transaction data of the current long-tail merchant and / or its associated long-tail merchants, and to perform task scheduling on the corresponding aggregated data.

[0096] Optionally, the long tail processing module 506 determines whether the dynamic time interval is long enough;

[0097] If so, and the industry characteristics and / or historical data of the corresponding transaction business data can be obtained, then according to the industry characteristics and / or historical data, the intelligent algorithm detection process is triggered to perform alarm calculation;

[0098] Otherwise, the rule threshold detection process is triggered to perform alarm calculation.

[0099] Optionally, the long-tail processing module 506 aggregates the transaction business data of the long-tail merchants through offline archiving;

[0100] The task scheduling of the traffic of the non-long-tail indicator specifically includes:

[0101] The transaction business data of the non-long-tail merchants is aggregated online to facilitate task scheduling.

[0102] Optionally, it also includes:

[0103] A level update module, before determining whether an indicator in the large-scale indicator set is a non-long-tail indicator or a long-tail indicator, generates and maintains a merchant level dimension table, dynamically updates the merchant's level and its corresponding time interval for performing the task scheduling in the merchant level dimension table, wherein the level reflects the current time interval corresponding to the corresponding merchant;

[0104] The long-tail processing module 506 uses the time interval to aggregate the transaction data of the current corresponding merchant and stores the corresponding aggregated data;

[0105] Adapting to the time interval, according to the aggregation time corresponding to the aggregation processing, querying the corresponding aggregation data, and performing task scheduling accordingly.

[0106] Optionally, the long-tail distinguishing module 502 determines at least two types of non-long-tail merchants divided according to different degrees of transaction high frequency, and the first time interval is negatively correlated with the different degrees of transaction high frequency;

[0107] Determine which of the at least two categories the merchants in the large-scale merchant set belong to, so as to use the corresponding first time interval for task scheduling.

[0108] Optionally, the non-long-tail processing module 504 or the long-tail processing module 506 detects whether the transaction business data is abnormal according to the specified monitoring items, and generates a corresponding abnormal event if so;

[0109] The alarms of the multiple abnormal events belonging to the same monitoring item are merged to uniformly initiate alarm notifications to relevant emergency personnel.

[0110] Optionally, the first time interval includes at least a time interval of seconds, and the second time interval is a time interval of minutes or more.

[0111] Figure 6 A schematic diagram of a large-scale alarm deployment device provided in one or more embodiments of this specification, the device comprising:

[0112] at least one processor; and,

[0113] a memory communicatively connected to the at least one processor; wherein,

[0114] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0115] Determine whether the indicators in the large-scale indicator set are non-long-tail indicators or long-tail indicators;

[0116] For the non-long-tail indicator, a first time interval is used to perform task scheduling on the traffic of the non-long-tail indicator, and alarm calculation is performed by executing the task;

[0117] For the long-tail indicator, the traffic of the long-tail indicator is aggregated, a second time interval longer than the first time interval is used, task scheduling is performed on the corresponding aggregated data, and alarm calculation is performed by executing the task.

[0118] Based on the same idea, one or more embodiments of this specification further provide a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:

[0119] Determine whether the indicators in the large-scale indicator set are non-long-tail indicators or long-tail indicators;

[0120] For the non-long-tail indicator, a first time interval is used to perform task scheduling on the traffic of the non-long-tail indicator, and alarm calculation is performed by executing the task;

[0121] For the long-tail indicator, the traffic of the long-tail indicator is aggregated, a second time interval longer than the first time interval is used, task scheduling is performed on the corresponding aggregated data, and alarm calculation is performed by executing the task.

[0122] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0123] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0124] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0125] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0126] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0127] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0130] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0131] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0132] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0133] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0134] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0135] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0136] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0137] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A large-scale alarm deployment method, comprising: Determine whether the indicators in the large-scale indicator set are non-long-tail indicators or long-tail indicators. Long-tail indicators refer to low-frequency indicators with a large number of traffic, while non-long-tail indicators refer to high-frequency indicators with a small number of traffic. For the non-long-tail indicator, a first time interval is used to perform task scheduling on the traffic of the non-long-tail indicator, and alarm calculation is performed by executing the task; For the long-tail indicator, the traffic of the long-tail indicator is aggregated, a second time interval longer than the first time interval is used, task scheduling is performed on the corresponding aggregated data, and alarm calculation is performed by executing the task.

2. The method as claimed in claim 1, wherein the indicator is a merchant, the non-long-tail indicator is a non-long-tail merchant, the long-tail indicator is a long-tail merchant, and the traffic is transaction business data of the corresponding merchant.

3. The method according to claim 2, wherein for the long-tail indicator, the traffic of the long-tail indicator is aggregated, and a second time interval longer than the first time interval is used to perform task scheduling on the corresponding aggregated data, specifically comprising: determining a plurality of different levels of second time intervals longer than the first time interval; For the current long-tail merchant or the current moment, determine the second time interval hit by the merchant in the second time intervals of the multiple different levels as the dynamic time interval; The dynamic time interval is used to aggregate the transaction data of the current long-tail merchant and / or its associated long-tail merchants, and to perform task scheduling on the corresponding aggregated data.

4. The method according to claim 3, wherein the performing alarm calculation specifically comprises: Determining whether the dynamic time interval is long enough; If so, and the industry characteristics and / or historical data of the corresponding transaction business data can be obtained, then according to the industry characteristics and / or historical data, the intelligent algorithm detection process is triggered to perform alarm calculation; Otherwise, the rule threshold detection process is triggered to perform alarm calculation.

5. The method according to claim 2, wherein the aggregating processing of the traffic of the long-tail indicator specifically comprises: Aggregate the transaction data of the long-tail merchants through offline archiving; The task scheduling of the traffic of the non-long-tail indicator specifically includes: The transaction business data of the non-long-tail merchants is aggregated online to facilitate task scheduling.

6. The method according to claim 2, wherein the step of determining whether an indicator in a large-scale indicator set is a non-long-tail indicator or before a long-tail indicator further comprises: Generate and maintain a merchant grade dimension table, dynamically update the merchant grade and its corresponding time interval for performing the task scheduling in the merchant grade dimension table, the grade reflects the current corresponding time interval of the corresponding merchant; The task scheduling specifically includes: Aggregate the transaction data of the current corresponding merchant at the time interval, and store the corresponding aggregated data; Adapting to the time interval, according to the aggregation time corresponding to the aggregation processing, querying the corresponding aggregation data, and performing task scheduling accordingly.

7. The method according to claim 2, wherein determining that an indicator in a large-scale indicator set is a non-long-tail indicator specifically comprises: Determine at least two categories of non-long-tail merchants divided according to different degrees of transaction high frequency, wherein the first time interval is negatively correlated with the different degrees of transaction high frequency; Determine which of the at least two categories the merchant in the large-scale indicator set belongs to, so as to use the corresponding first time interval for task scheduling.

8. The method according to claim 2, wherein the performing alarm calculation specifically comprises: According to the specified monitoring items, detect whether the transaction business data is abnormal, and if so, generate corresponding abnormal events; The alarms of the multiple abnormal events belonging to the same monitoring item are merged to uniformly initiate alarm notifications to relevant emergency personnel.

9. A large-scale alarm deployment device, comprising: The long-tail distinction module determines whether an indicator in a large-scale indicator set is a non-long-tail indicator or a long-tail indicator. Long-tail indicators refer to low-frequency indicators with a large number of traffic, while non-long-tail indicators refer to high-frequency indicators with a small number of traffic. A non-long-tail processing module, for the non-long-tail indicator, uses a first time interval to perform task scheduling on the flow of the non-long-tail indicator, and performs alarm calculation by executing the task; The long-tail processing module aggregates the traffic of the long-tail indicators, uses a second time interval longer than the first time interval to schedule tasks for the corresponding aggregated data, and performs alarm calculations by executing the tasks.

10. The device as claimed in claim 9, wherein the indicator is a merchant, the non-long-tail indicator is a non-long-tail merchant, the long-tail indicator is a long-tail merchant, and the traffic is transaction business data of the corresponding merchant.

11. The apparatus of claim 10, wherein the long-tail processing module determines a plurality of second time intervals of different levels that are longer than the first time interval; For the current long-tail merchant or the current moment, determine the second time interval hit by the merchant in the second time intervals of the multiple different levels as the dynamic time interval; The dynamic time interval is used to aggregate the transaction data of the current long-tail merchant and / or its associated long-tail merchants, and to perform task scheduling on the corresponding aggregated data.

12. The device according to claim 11, wherein the long tail processing module determines whether the dynamic time interval is long enough; If so, and the industry characteristics and / or historical data of the corresponding transaction business data can be obtained, then according to the industry characteristics and / or historical data, the intelligent algorithm detection process is triggered to perform alarm calculation; Otherwise, the rule threshold detection process is triggered to perform alarm calculation.

13. The device according to claim 10, wherein the long-tail processing module aggregates the transaction data of the long-tail merchants through offline archiving; The task scheduling of the traffic of the non-long-tail indicator specifically includes: The transaction business data of the non-long-tail merchants is aggregated online to facilitate task scheduling.

14. The apparatus of claim 10, further comprising: A level update module, before determining whether an indicator in the large-scale indicator set is a non-long-tail indicator or a long-tail indicator, generates and maintains a merchant level dimension table, dynamically updates the merchant's level and its corresponding time interval for performing the task scheduling in the merchant level dimension table, wherein the level reflects the current time interval corresponding to the corresponding merchant; The long-tail processing module aggregates the transaction data of the current corresponding merchant at the time interval, and stores the corresponding aggregated data; Adapting to the time interval, according to the aggregation time corresponding to the aggregation processing, querying the corresponding aggregation data, and performing task scheduling accordingly.

15. The device according to claim 10, wherein the non-long-tail processing module or the long-tail processing module detects whether the transaction business data is abnormal according to the specified monitoring items, and generates a corresponding abnormal event if so; The alarms of the multiple abnormal events belonging to the same monitoring item are merged to uniformly initiate alarm notifications to relevant emergency personnel.

16. A large-scale alarm deployment device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute: Determine whether the indicators in the large-scale indicator set are non-long-tail indicators or long-tail indicators. Long-tail indicators refer to low-frequency indicators with a large number of traffic, while non-long-tail indicators refer to high-frequency indicators with a small number of traffic. For the non-long-tail indicator, a first time interval is used to perform task scheduling on the traffic of the non-long-tail indicator, and alarm calculation is performed by executing the task; For the long-tail indicator, the traffic of the long-tail indicator is aggregated, a second time interval longer than the first time interval is used, task scheduling is performed on the corresponding aggregated data, and alarm calculation is performed by executing the task.

Images