Business processing method, device and computer-readable storage medium

Abstract

The present application discloses a business processing method, device, and computer-readable storage medium. The business processing method includes: obtaining business log information, extracting keywords corresponding to the business based on the log information; sorting the keywords, splicing the sorted keywords to obtain a spliced ​​field, and obtaining a hash value of the spliced ​​field; searching a knowledge base for a target hash value that matches the hash value, where the knowledge base is a repository generated based on keywords, hash values, and business anomaly information; obtaining business anomaly information corresponding to the target hash value, and outputting an alarm message based on the anomaly information. The present application improves the effectiveness of business monitoring by searching a knowledge base for a target hash value that matches the hash value, obtaining anomaly information corresponding to the target hash value, and then outputting an alarm message based on the anomaly information.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a business processing method, device, and computer-readable storage medium. Background Art

[0002] Currently, there are several main types of monitoring technologies for business data: First, monitoring business performance data and hardware data related to servers and other related services. This type of monitoring lacks analysis of actual business implications and relies on hardware metrics, making its application specific to specific scenarios. Second, using machine algorithms to learn from business data and generate customized alerts fails to meet the demand for precise push notifications. As can be seen, existing business monitoring and alerting technologies primarily focus on business performance data or hardware-related monitoring. Few technologies analyze and monitor actual business-generated data, such as business log data. Consequently, current business monitoring effectiveness is low. Summary of the Invention

[0003] The embodiments of the present application aim to solve the problem of low effectiveness of existing business monitoring by providing a business processing method, device and computer-readable storage medium.

[0004] To achieve the above objectives, the present application provides a service processing method, which includes:

[0005] Obtaining business log information, and extracting keywords corresponding to the business based on the log information;

[0006] Sorting the keywords, concatenating the sorted keywords to obtain a concatenated field, and obtaining a hash value of the concatenated field;

[0007] Searching for a target hash value that matches the hash value in a knowledge base, where the knowledge base is a repository generated based on the keyword, the hash value, and abnormal information of the business;

[0008] Obtain abnormal information of the business corresponding to the target hash value, and output alarm information according to the abnormal information.

[0009] Optionally, the step of outputting alarm information according to the abnormal information includes:

[0010] Obtain the alarm content corresponding to the abnormal information;

[0011] Obtain the alarm level, alarm method, and alarm object corresponding to the alarm content;

[0012] Outputting alarm information including the alarm content to the alarm object according to the alarm level and the alarm mode.

[0013] Optionally, before the step of obtaining the alarm level, alarm mode, and alarm object corresponding to the alarm content, the step includes:

[0014] Obtaining the alarm status corresponding to the abnormal information;

[0015] When the alarm condition is an alarm triggered when abnormal statistical information is collected, the alarm level is set to a normal alarm;

[0016] When the alarm condition is that the frequency of occurrence of the service anomaly is greater than or equal to a first preset value, setting the alarm level to a prompt alarm;

[0017] When the alarm condition is that the number of service anomalies is greater than or equal to a second preset value, the alarm level is set to a fault alarm.

[0018] Optionally, before the step of obtaining the alarm level, alarm mode, and alarm object corresponding to the abnormal information, the step includes:

[0019] Obtain the calling interface corresponding to the exception information;

[0020] Obtaining the access party corresponding to each of the calling interfaces and identification information of the access party;

[0021] The alarm object is determined according to the hash value and the identification information.

[0022] Optionally, the method further includes:

[0023] Determining whether the knowledge base stores the hash value;

[0024] When the hash value is stored in the knowledge base, storing the exception information in the knowledge base;

[0025] When the hash value is not stored in the knowledge base, creating a business link according to the keyword to obtain a target business link;

[0026] The hash value and the exception information are stored in the target service link.

[0027] Optionally, the method further includes:

[0028] Obtaining a default failure rate of the external interface of the service, and an initial acceptable failure rate and an acceptable failure rate of the access party corresponding to the external interface;

[0029] When the acceptable failure rate is greater than or equal to the default failure rate, converting the business to an online processing mode;

[0030] When the acceptable failure rate is less than or equal to the initial acceptable failure rate, the service is converted to an offline self-learning mode.

[0031] Optionally, the method further includes:

[0032] Obtaining a first performance indicator value of the external interface;

[0033] Obtaining a second performance indicator value of each of the access parties, and obtaining a sum of the second performance indicator values;

[0034] Obtaining a quotient of the sum value and the first performance indicator value;

[0035] The product of the quotient and the default failure rate of the external interface is obtained, and the product is used as the initial acceptable failure rate.

[0036] In addition, to achieve the above-mentioned purpose, the present application further provides a business processing device, which includes an acquisition module, a processing module, a search module, and an output module, wherein:

[0037] The acquisition module is used to acquire log information of the business and extract keywords corresponding to the business according to the log information;

[0038] The processing module is configured to sort the keywords, concatenate the sorted keywords to obtain a concatenated field, and obtain a hash value of the concatenated field;

[0039] The search module is configured to search a knowledge base for a target hash value that matches the hash value, wherein the knowledge base is a repository generated based on the keyword, the hash value, and the abnormal information of the business;

[0040] The output module is used to obtain abnormal information of the business corresponding to the target hash value and output alarm information according to the abnormal information.

[0041] In addition, to achieve the above-mentioned purpose, the present application also provides a business processing device on the other hand, which includes a memory, a processor, and a business processing program stored in the memory and running on the processor, and when the processor executes the business processing program, it implements the steps of the business processing method described above.

[0042] In addition, to achieve the above-mentioned purpose, the present application further provides a computer-readable storage medium on the other hand, on which a business processing program is stored, and when the business processing program is executed by a processor, the steps of the business processing method described above are implemented.

[0043] The present application proposes a business processing method, which obtains the log information of the business, extracts the keywords corresponding to the business based on the log information; sorts the keywords, splices the sorted keywords to obtain a spliced ​​field, and obtains the hash value of the spliced ​​field; searches for a target hash value that matches the hash value in a knowledge base, where the knowledge base is a repository generated based on keywords, hash values, and business anomaly information; obtains the business anomaly information corresponding to the target hash value, and outputs an alarm message based on the anomaly information. The present application improves the effectiveness of business monitoring by searching for a target hash value that matches the hash value in a knowledge base, obtaining the anomaly information corresponding to the target hash value, and then outputting an alarm message based on the anomaly information. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application;

[0045] Figure 2 This is a flowchart of the first embodiment of the business processing method of this application;

[0046] Figure 3 This is a flowchart of the second embodiment of the business processing method of this application;

[0047] Figure 4 This is a flowchart of the third embodiment of the business processing method of this application;

[0048] Figure 5 This is a schematic diagram of the architecture of the business processing system for this application;

[0049] Figure 6 This is a schematic diagram of the structure of the knowledge base of this application;

[0050] Figure 7 This is a self-learning process diagram for this application business log;

[0051] Figure 8 This is a flowchart of the dynamic switching algorithm of this application;

[0052] Figure 9 This is a module diagram of the business processing device of this application.

[0053] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0055] The main solution of the embodiment of the present application is: obtaining the log information of the business, extracting the keywords corresponding to the business based on the log information; sorting the keywords, splicing the sorted keywords to obtain a spliced ​​field, and obtaining the hash value of the spliced ​​field; searching for a target hash value matching the hash value in a knowledge base, where the knowledge base is a storage repository generated based on the keywords, the hash value, and the abnormal information of the business; obtaining the abnormal information of the business corresponding to the target hash value, and outputting alarm information based on the abnormal information.

[0056] Existing business monitoring and alarm technologies primarily focus on business performance data or hardware-related monitoring, and are limited in their analysis and monitoring of actual business-generated data, such as business log data. Consequently, existing business monitoring suffers from low effectiveness.

[0057] The present application obtains the log information of the business, extracts the keywords corresponding to the business based on the log information; sorts the keywords, splices the sorted keywords to obtain a spliced ​​field, and obtains the hash value of the spliced ​​field; searches for a target hash value that matches the hash value in a knowledge base, which is a repository generated based on keywords, hash values, and business anomaly information; obtains the business anomaly information corresponding to the target hash value, and outputs an alarm message based on the anomaly information. The present application improves the effectiveness of business monitoring by searching for a target hash value that matches the hash value in a knowledge base, obtaining the anomaly information corresponding to the target hash value, and then outputting an alarm message based on the anomaly information.

[0058] like Figure 1 As shown, Figure 1 This is a schematic diagram of the terminal device structure of the hardware operating environment involved in the embodiment of the present application.

[0059] like Figure 1 As shown, the terminal device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0060] Those skilled in the art will understand that Figure 1 The terminal device structure shown in the figure does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0061] like Figure 1 As shown, the memory 1005 as a computer-readable storage medium may include a business processing program.

[0062] exist Figure 1 In the terminal device shown, the network interface 1004 is mainly used for data communication with the backend server; the user interface 1003 is mainly used for data communication with the client (user end); the processor 1001 can be used to call the business processing program in the memory 1005 and perform the following operations:

[0063] Obtaining business log information, and extracting keywords corresponding to the business based on the log information;

[0064] Sorting the keywords, concatenating the sorted keywords to obtain a concatenated field, and obtaining a hash value of the concatenated field;

[0065] Searching for a target hash value that matches the hash value in a knowledge base, where the knowledge base is a repository generated based on the keyword, the hash value, and abnormal information of the business;

[0066] Obtain abnormal information of the business corresponding to the target hash value, and output alarm information according to the abnormal information.

[0067] refer to Figure 2 , Figure 2 This is a flowchart of the first embodiment of the business processing method of this application.

[0068] The embodiment of the present application provides a business processing method. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown here.

[0069] The service processing method of this embodiment includes the following steps:

[0070] Step S10, obtaining log information of the business, and extracting keywords corresponding to the business according to the log information;

[0071] It should be noted that the reference Figure 5 , Figure 5 The business processing system architecture is divided into three parts: offline side, online side, and support side.

[0072] Offline side: This mainly involves self-learning business logs, abstracting data models, and recording them in the knowledge base.

[0073] Online side: includes online performance monitoring and online business monitoring. Online business monitoring mainly analyzes and records real-time data from the business platform.

[0074] Support side: mainly includes: configuration module, alarm module, visualization module, etc. The configuration module is mainly used to configure key dimensions and indicators in the system; the alarm module is a module for sending alarms; the visualization module is a specific display of the business through visualization tools.

[0075] The main function of the configuration module of this system is to ensure the normal operation of its logic and business functions during system operation. Since this application is designed for business-related monitoring and alarms, and needs to meet the purpose of accurate push notifications, business-related configuration is particularly important. This application mainly divides the configuration into system configuration, business configuration, and key algorithm configuration, among which:

[0076] System configuration: This supports the normal operation of the system and is a system-level configuration. When the system is ported to a different business platform, this configuration can be modified and adapted, while its functionality and usage remain unchanged. Key configurations include: log-related I / O, service node information, and repository information.

[0077] Business configuration: Considering the complexity of business monitoring and the portability of this system, business-related configurations will also be more complex. For example, different business interfaces require different key information to be collected. When analyzing logs, the system extracts key information and then pushes subsequent alarms. The content pushed is the information that the alarm receiver is concerned about. The key business configurations extracted by this system are:

[0078] Business log related: time, keywords, service nodes, etc. Extract key information such as time, location, and events.

[0079] Interface related: key fields of upstream and downstream interfaces, key input and output parameters, etc. Extract key information of the interface layer.

[0080] Alarm related: alarm method, alarm level, alarm object, alarm threshold, alarm frequency, etc. Based on different business anomaly monitoring results and system-defined alarm levels, the system will use different alarm methods and alarm types to notify the corresponding alarm objects.

[0081] Based on the above configuration, the system can extract complete event information from the business log, associate the alarm object according to the log information, thereby determining the complete logic of the alarm push, and realizing the precise alarm push design from the aspects of alarm content and alarm push.

[0082] Configuration of key algorithms: This system mainly includes two key algorithms: self-learning algorithm and offline self-learning and real-time monitoring dynamic conversion algorithm, among which:

[0083] Self-learning algorithm: The initial window time for log reading, pre-set basic knowledge, and manual correction configuration. Algorithm configuration ensures the correct function of the self-learning algorithm and the continuous iteration of the self-learning process, thereby generating a rich knowledge base.

[0084] Offline self-learning and real-time monitoring dynamic conversion algorithm: default failure rate, interface performance indicators, time window upgrade index, etc.

[0085] In this embodiment, the log information of the business is read through the self-learning module. Since the log information read by the self-learning module is non-real-time, the self-learning module will read the business log information within the range according to the initial time window set by the system. For example, the initial time window set by the system is 10 minutes, that is, the time range window from 10 minutes ago to the current time. Then, the keywords corresponding to the business are extracted from the log information. The keywords include business-level keywords and system-level keywords. Among them, the business-level keywords are mainly the basic knowledge preset in the initial stage of the system and the keywords that are continuously enriched in self-learning; the system-level keywords refer to the system-level exceptions that occur during the operation of the business platform. It is irrelevant to or weakly related to the specific platform business, such as "NPE" and "Timeout" that often occur in the business.

[0086] Step S20, sorting the keywords, concatenating the sorted keywords to obtain a concatenated field, and obtaining a hash value of the concatenated field;

[0087] It should be noted that the reference Figure 6 , Figure 6 The schematic diagram of the structure of the knowledge base of the present application, the present application has constructed a knowledge base through self-learning of business logs. The knowledge base is a repository of a tree structure based on a link and KV information, wherein a link is a business logic link composed of keywords, the K value in the KV information is a hash value, and the V information is abnormal information. In the knowledge base, each node in the tree structure forms the entire link relationship of the abnormal keyword, and the abnormal process can be clearly recorded; the bottom node of the tree is used to store KV information, K represents the hash value generated according to the link keyword, in order to quickly query the node, and V represents abnormal information (because the original abnormal information is large, the system will extract the key information and form a record, and then store it), in order to store abnormal information or parse abnormal information (after parsing, statistics can be made, or push it). Wherein, specific abnormal key information is stored in the V information, which can be understood as a List. After obtaining the V information, abnormal information can be stored, the information inside the List can be obtained, the abnormal number can be parsed, and then the abnormal information can be pushed.

[0088] In this embodiment, reference Figure 6 The extracted keywords correspond to the nodes in the above figure. The link from keyword K1->keyword K11->keyword information KV11 represents the key link parsed from the log. The parsed KV information is stored in each leaf node.

[0089] KV information structure: KV information is the final information generated after log self-learning, so it must meet the requirements of fast search and complete key information. The K information is generated according to Formula 1:

[0090] K KVii =Hash(String.bind(sort(K i ,K ii ),&)) Formula 1)

[0091] In the formula, sort(K i ,K ii ) means that the extracted keywords are sorted in natural alphabetical order, String.bind(sort(K i ,K ii ),&) means that the sorted keywords are concatenated using the "&" symbol to obtain an ordered and fixed string. Hash means that the concatenated string is passed through the Hash algorithm to obtain a fixed and unique hash value, which is the message K value.

[0092] Step S30, searching a knowledge base for a target hash value that matches the hash value, wherein the knowledge base is a repository generated based on the keyword, the hash value, and abnormal information of the business;

[0093] In this embodiment, after extracting a keyword, the K value of the corresponding KV information is directly calculated according to Formula 1. Then, the key business information in the log is extracted to generate the V value of the KV information. The KV information in the knowledge base is directly searched for the same K value. If it does, it is stored in the knowledge base and the link resolution step is skipped. Because the KV information in the knowledge base can be directly matched based on the hash value, the time complexity is O(1), which is the most efficient.

[0094] The business processing system of this application uses Redis+DB to store the knowledge content of the knowledge base. The knowledge base is divided into two parts: one is the knowledge base structure, which is a tree structure with keywords as nodes, and the other is the KV information recorded in the log. To ensure the IO efficiency of the knowledge base, the system interacts directly with the knowledge base through Redis, where the knowledge base structure only needs to be read once during initialization, and the KV information recorded in the log is stored in Redis after self-learning is completed. The system background has asynchronous processing to write the written KV information to the DB, which is for data point persistence. If Redis is abnormal, the system interacts directly with the DB.

[0095] Step S40: Acquire abnormal information of the business corresponding to the target hash value, and output alarm information according to the abnormal information.

[0096] When a target hash value that matches the calculated hash value is found in the knowledge base, the abnormal information (i.e., V information) stored corresponding to the target hash value is obtained, and then, based on the abnormal information, an alarm message is output. In one embodiment, the alarm content corresponding to the abnormal information, as well as the alarm level, alarm method, and alarm object corresponding to the alarm content are obtained, and then, according to the alarm level and alarm method, an alarm message including the alarm content is output to the alarm object. Among them, the support side of this system is provided with an alarm module, which directly determines the response time of the alarm to the business operation and maintenance personnel, thereby affecting the efficiency of business operation and maintenance. Therefore, this application designs the alarm module from the dimensions of alarm level, alarm method, alarm formation, etc.

[0097] The warning content of this system is generated based on the V information in the KV information, and mainly includes: the warning log time range, the number of exceptions, the exception details (V information), etc.

[0098] The system's alarm levels are divided into three categories: general alarm, prompt alarm, and fault alarm. These are divided according to the urgency of the alarm. Different alarm levels are triggered in different ways:

[0099] Regular alarm: It is an alarm triggered when abnormal information is regularly counted. It occurs during the self-learning stage of the business log on the offline side. Self-learning will regularly read the log and parse relevant information when the business is normal, and then trigger regular alarms based on the results of self-learning. The regular alarm configuration of this system has two trigger thresholds: time threshold and abnormal record number threshold. For example, the time threshold is "1h", which means that regular alarm information will be pushed once every hour; the abnormal record number threshold is "10", which means that when the cumulative number of abnormal records exceeds 10, a regular alarm will be triggered. Among them, the idea of ​​designing regular alarms in this application is: when the business platform is operating normally, occasional abnormalities will not affect the business, and at the same time ensure that business operation and maintenance personnel can pay attention to abnormal information, so regular alarms need to be pushed, and the threshold is the way for this system to control personnel costs. Business operation and maintenance personnel can control the time and efficiency of paying attention to business information.

[0100] Prompt Alerts: When the system detects a high frequency of anomalies while self-studying offline business logs, it will proactively trigger a prompt alert. This alert serves as a warning, allowing business operations personnel to quickly respond and locate the issue, preventing escalation of business anomalies. The design of this application's prompt alerts is based on the principle that, to ensure rapid response, these alerts will be sent to a group of operations personnel, preventing some personnel from delaying reading the message.

[0101] Fault alarm: When the system monitors the business in real time online, if the number of exceptions exceeds the set threshold, the system will actively trigger a fault alarm. This alarm needs to provide detailed exception information to the operation and maintenance personnel to help quickly locate the problem. The fault alarm of this system is configured with two trigger thresholds: the number of exceptions and the failure rate within the log time window. The number of exceptions refers to the number of abnormal logs, and the failure rate refers to the ratio of abnormal logs to logs. Among them, the idea of ​​designing the fault alarm in this application is: when the fault alarm is triggered in real time, the business abnormality has reached a serious level, so the detailed exception information will be accurately pushed to the operation and maintenance personnel.

[0102] This system includes the following alarm methods: email, SMS, and WeChat messages. Different alarm methods have different characteristics:

[0103] Email: non-real-time, detailed content;

[0104] SMS: real-time, short content, and risk of loss;

[0105] WeChat messages: real-time, detailed content.

[0106] The design of the alarm mode is based on the association and definition of the alarm category and the urgency of the alarm. The correlation between the two sides of this system is shown in Table 1:

[0107] Table 1 Alarm category and alarm mode association

[0108] Alarm Category Supported alarm methods General Alerts mail Prompt alarm SMS and WeChat messages Fault alarm Email, WeChat messages

[0109] The system's configuration of alert objects is based on the KV information in the knowledge base. This takes into account the different anomalies that may occur in different business scenarios, and the need to distinguish the corresponding alert objects for these anomalies. This allows different anomaly alerts to be pushed to the corresponding alert objects, achieving the goal of accurate push notifications.

[0110] The system administrator will configure the alarm objects corresponding to each KV node based on the existing knowledge in the knowledge base. The configuration rules mainly include (but are not limited to): business information in K+V (such as key input parameters) -> alarm object. For example, obtain the calling interface (keyword interface) of the V information in the KV information, obtain the access party corresponding to the calling interface, and the id (appid) corresponding to each access party, where appid is the identification information of the access party, and then determine the alarm object based on the K value and appid, such as K+appid->alarm object 1, where the information of alarm object 1 mainly includes: user name, mobile phone number, email address, WeChat id and other information.

[0111] This embodiment generates monitoring data with actual business significance by analyzing the business meaning of actual business data; at the same time, it combines the business meaning to achieve accurate push notifications, so that business-related personnel can respond quickly and shorten operation and maintenance time; furthermore, the design of accurate alarm push notifications is based on the knowledge base, and the push type, push information, push object, etc. of the alarm can be configured according to the content of the knowledge base. Not only can the messages be pushed accurately, but the details of the anomalies can also be queried in real time, so as to achieve the purpose of rapid response, positioning, and problem solving, thereby improving the effectiveness of business monitoring.

[0112] Further, refer to Figure 3 , a second embodiment of the business processing method of this application is proposed.

[0113] The method further comprises:

[0114] Step S50, determining whether the knowledge base stores the hash value;

[0115] Step S60, when the hash value is stored in the knowledge base, storing the abnormal information in the knowledge base;

[0116] Step S70: when the knowledge base does not store the hash value, create a business link according to the keyword to obtain a target business link;

[0117] Step S80: Store the hash value and the exception information in the target service link.

[0118] It's important to note that business log self-learning involves continuously studying log information, extracting key information of interest to the system, and ultimately building a knowledge base. This process is broadly divided into: log reading, keyword extraction, link parsing, extracting key-value information, and writing to the knowledge base. Because business logs are learned and parsed, business-layer logs from different platforms vary. Therefore, during the initial stages of the system's self-learning, basic knowledge pre-sets and mid-term knowledge base corrections are necessary. Through continuous learning, the system ultimately generates a comprehensive knowledge base.

[0119] refer to Figure 7 , Figure 7 This is a diagram of the self-learning process for the business log of this application. In Figure 7, basic knowledge pre-setting and manual verification: Different platforms have their own specificities when printing business layer logs, for example, the business exception type thrown by the platform, the code-level path, etc. For these specific and different key information, this system calls it basic knowledge, and pre-setting basic knowledge is necessary: ​​first, this system can complete self-learning more accurately and quickly based on this knowledge; second, it can improve the scalability and portability of this system. Manual verification refers to the verification of the knowledge base to avoid errors that occur during self-learning. The system retains a manual backup processing entry for the accuracy of the knowledge base.

[0120] Reading logs: In the self-learning module, log reading is non-real-time. Business logs within the range will be read according to the initial time window set by this system. For example, this system sets the initial time window to 10 minutes, that is, the time range window from 10 minutes ago to the current time.

[0121] Extract keywords: Keywords include business-level keywords and system-level keywords. Business-level keywords primarily consist of basic knowledge pre-installed in the system and keywords that are continuously enriched through self-learning. System-level keywords refer to system-level exceptions that occur during the operation of the business platform. These are unrelated or weakly related to specific platform services. For example, frequently occurring "NPE" and "Timeout" errors may occur.

[0122] Extract KV information: After extracting keywords, a K value is generated based on the keywords. V information is extracted based on the abnormal information in the log, such as time, interface address, abnormal information, key input parameters, etc.

[0123] Link parsing: Link parsing is the process of concatenating keywords based on the semantic context of the log to obtain meaningful business logic links. The parsed links are also complete links from the root node of the knowledge base structure to the leaf nodes of specific KV information.

[0124] Writing to the knowledge base: The knowledge base is a tree-structured repository based on links and KV information. Based on the extracted KV information, it determines whether the K information already exists. If so, the KV information is directly written to the knowledge base. If not, a link is created and the KV information is written to the leaf node of the new link.

[0125] This embodiment proposes a method for learning and building a knowledge base based on logs. This method is an adaptive iterative optimization data analysis method that can self-select analysis methods according to different business logs, and generate specific KV exception information, which is stored in a custom knowledge base. This allows the abnormal information to be quickly found in the knowledge base based on the KV information, thereby shortening the time for finding abnormal information and improving the effectiveness of business monitoring.

[0126] Further, refer to Figure 4 , a third embodiment of the business processing method of this application is proposed.

[0127] The business processing method further includes:

[0128] Step S90, obtaining a default failure rate of the external interface of the service, and an initial acceptable failure rate and an acceptable failure rate of the access party corresponding to the external interface;

[0129] Step S100, when the acceptable failure rate is greater than or equal to the default failure rate, converting the service to an online processing mode;

[0130] Step S110 : When the acceptable failure rate is less than or equal to the initial acceptable failure rate, convert the service to an offline self-learning mode.

[0131] It should be noted that the self-learning of offline business logs is a process in which the system reads business logs at regular intervals according to the initial parameter settings and records the learning results into the knowledge base. In the process of self-learning, since each system will have occasional exceptions, there is a certain acceptability for daily logs, and there is no need to check and solve all exceptions in real time. In this way, the cost of operation and maintenance personnel can be reduced. Among them, the initial parameter configuration, for example: this system sets the initial offline time window to 10 minutes (that is, the log is initially read once every 10 minutes, and the log range is the record within the last 10 minutes), the initial online time window is 1 minute (that is, the abnormal failure information is initially counted every 1 minute), the default minimum number of switching algorithm records in the offline time window is 50 (that is, the minimum number of records involved in the switching algorithm calculation), and the default interface failure rate is 5% (that is, the default maximum failure rate of the interface). This embodiment involves the default interface failure rate, the initial acceptable failure rate, the failure rate, and the acceptable failure rate, where:

[0132] Default interface failure rate (i.e., the default failure rate of an interface): refers to the default acceptable failure rate of an interface set by the system. For example, 1% means that the interface system can accept 1 request failure out of 100 requests.

[0133] Initial receivable failure rate: refers to the specific receivable failure rate of a certain access point of the interface, which is calculated based on the requested TPS (performance requirements) of multiple access points. For example, if the default interface failure rate of the interface is 1%, there are three access points, and each access point has the same performance requirements, then the initial receivable failure rate of a certain access point is 33% * 1% = 0.33%.

[0134] Failure rate: refers to the failure rate calculated when the time window and failure rate are adjusted for the i-th time.

[0135] Acceptable failure rate: When calculating the failure rate for the i-th log read, the failure rate result for the i-1th read is used as the acceptable failure rate for the i-th read.

[0136] refer to Figure 8 , Figure 8 This is a flow chart of the dynamic switching algorithm of this application.

[0137] 1. Calculating the Initial Acceptable Failure Rate: The initial acceptable failure rate is a key indicator of the handover algorithm and serves as the algorithm's entry point. When the service failure rate of a particular interface's access point exceeds the initial acceptable failure rate, the system enters the "Observation Phase," continuously adjusting and calculating subsequent failure rates until it transitions to real-time online monitoring or reverts to the normal self-learning phase.

[0138] The business platform has an external interface A, and the interface performance indicator is t a ;The default failure rate of the interface is fp a ; Interface access side: AC1...ACi...ACn, respectively have performance indicators: t ac1 ...t aci ...t acn The initial acceptable failure rate of a certain access point is:

[0139]

[0140] 2. Analysis of the process of switching from offline to online

[0141] Assume that the performance indicator of interface A is 500 tps; the default failure rate of the interface is 5%; and the three access parties of the interface are AC1, AC2, and AC3, with performance indicators of 200 tps, 150 tps, and 150 tps, respectively. The initial acceptable failure rate of AC1 is 1.25% according to Formula 2.

[0142] 2.1. Read the log for the first time:

[0143] Read the log records within the offline time window (the default offline time window is 10 minutes), self-learn and record them in the knowledge base, and calculate the failure rate. Taking the access party AC1 as an example, if the failure rate is > 1.25%, adjust the acceptable failure rate to: the acceptable failure rate of the current time * 2; adjust the time window to: the current offline time window / 2; and adjust the minimum number of switching algorithm records / 2. The failure rate is calculated as follows:

[0144]

[0145] Where, count aci The org_count represents the number of requests from the access point ACi, and the minimum number of switching algorithm records within the default offline time window. Comparing the minimum number of records is done to prevent the calculated failure rate from being meaningless due to a low request volume. For example, if there are five requests in a certain time period and one fails, the failure rate is 20%. However, calculating a failure rate with such a small base number is not very meaningful from a business perspective. Therefore, there is no need to waste operation and maintenance costs on real-time monitoring of specific anomalies.

[0146] 2.2, Read the log for the i-th time:

[0147] Read the log records within the time window (i-1), self-learn, and record them in the knowledge base to calculate the failure rate. Taking the access party AC1 as an example, if the failure rate is greater than the acceptable failure rate (i-1), the acceptable failure rate is adjusted to: acceptable failure rate (i-1) * 2; the time window is adjusted to: time window (i-1) / 2; the minimum number of switching algorithm records is adjusted to: minimum number of switching algorithm records (i-1) / 2. At this time, the system will push the abnormal information to the business operation and maintenance personnel, and the alarm level will be "prompt alarm". The failure rate is calculated as follows:

[0148]

[0149] In the formula, A is condition 1, B is condition 2, and condition 1 is:

[0150] connt aci >acpt-connt(i-1)and fp aci >acpt-fp aci (i-1);

[0151] Condition 2 is:

[0152] connt aci ≤acpt-connt(i-1)or fp aci ≤acpt_fp aci (i-1);

[0153] Where, acpt_count(i-1) represents the number of records of the minimum switching algorithm after reading the log for the i-1th time and calculating the adjusted minimum number of records; acpt_fp aci (i-1) represents the acceptable failure rate for the i-1th access point. In Formula 4, the maximum acceptable failure rate allowed for a particular access point is the default interface failure rate of the corresponding interface.

[0154] 2.3. Offline self-learning conversion to online real-time business monitoring

[0155] When the i-th acceptable failure rate is greater than or equal to the default interface failure rate of the interface, it will switch to real-time monitoring online processing mode. At this time, the specific exception information will be accurately pushed to the corresponding operation and maintenance personnel in real time. The alarm level at this time is the highest level "fault alarm" until the operation and maintenance personnel resolve the exception.

[0156] 3. Analysis of the process of switching from online to offline

[0157] In the online real-time monitoring mode, the failure rate is continuously analyzed and calculated, and subsequent adjustments are made based on the failure rate. It should be noted that online real-time monitoring directly monitors the business platform, rather than reading and parsing logs.

[0158] 3.1. The first statistical failure rate

[0159] Records within the online time window (the default online time window is 1 minute) are counted to calculate the failure rate. For example, if the failure rate is less than 5% on access point AC1, the acceptable failure rate is adjusted to: interface default failure rate / 2; the time window is adjusted to: current online time window * 2; and the minimum number of handover algorithm records is adjusted to: minimum number of handover algorithm records (i) / 2. The minimum number of handover algorithm records (i) is the adjusted minimum number of handover algorithm records from the last offline side self-learning, recorded in the system as: last_off_count.

[0160]

[0161] In the formula, A is condition 1, B is condition 2, and condition 1 is:

[0162] connt aci >acpt_connt(i-1)and fp aci >acpt_fp aci (i-1);

[0163] Condition 2 is:

[0164] connt aci ≤acpt_connt(i-1)or fp aci ≤acpt-fp aci(i-1);

[0165] Where fp aci Indicates the failure rate of ACi at that time.

[0166] 3.2. Statistical failure rate for the i-th time

[0167] Count the records within the online time window (i-1) and calculate the failure rate. For example, on access point AC1, if the failure rate is less than the acceptable failure rate (i-1), adjust the acceptable failure rate to: acceptable failure rate (i-1) / 2; adjust the time window to: time window (i-1) * 2; and adjust the minimum number of handover algorithm records to: minimum number of handover algorithm records (i-1) / 2.

[0168]

[0169] In the formula, A is condition 1, B is condition 2, and condition 1 is:

[0170] connt aci >acpt-connt(i-1)and fp aci <acpt-fp aci (i-1);

[0171] Condition 2 is:

[0172] connt aci ≤acpt-connt(i-1)or fp aci ≥acpt-fp aci (i-1);

[0173] Where, org_fp aci is the initial acceptable failure rate of ACi.

[0174] 3.3. Transition from online real-time business monitoring to offline self-learning

[0175] When the acceptable failure rate for the i-th time is less than or equal to the initial acceptable failure rate of ACi, it will switch to the offline self-learning mode, and no abnormal information will be pushed to the operation and maintenance personnel.

[0176] The dynamic switching between the offline side and the online side is a reciprocal process. The system will also choose to push different levels of alarm information based on the status of the switching process to provide more obvious prompts to operation and maintenance personnel.

[0177] This embodiment proposes a dynamic switching algorithm for offline learning and online real-time monitoring of service data. This algorithm dynamically adjusts thresholds for various indicators based on multiple dimensions, such as service interface performance and upstream and downstream interface performance requirements. The algorithm then uses these thresholds to determine the dynamic switching between the offline and online sides. Furthermore, this algorithm automatically adjusts monitoring alarm levels and responds to the alerting methods and content of service operations personnel, thereby improving operational efficiency.

[0178] In addition, the present application also provides a business processing device, which includes a memory, a processor, and a business processing program stored in the memory and running on the processor, wherein the device obtains log information of the business, extracts keywords corresponding to the business based on the log information; sorts the keywords, splices the sorted keywords to obtain a spliced ​​field, and obtains a hash value of the spliced ​​field; searches for a target hash value that matches the hash value in a knowledge base, where the knowledge base is a storage repository generated based on keywords, hash values, and business anomaly information; obtains business anomaly information corresponding to the target hash value, and outputs alarm information based on the anomaly information. The present application improves the effectiveness of business monitoring by searching for a target hash value that matches the hash value in a knowledge base, obtaining anomaly information corresponding to the target hash value, and then outputting an alarm information based on the anomaly information.

[0179] Further, refer to Figure 9 , Figure 9 Schematic diagram of the modules of the business processing device of the present application. The business processing device 100 includes an acquisition module 10, a processing module 20, a search module 30 and an output module 40, wherein:

[0180] The acquisition module 10 is used to acquire log information of the business and extract keywords corresponding to the business according to the log information;

[0181] The processing module 20 is configured to sort the keywords, concatenate the sorted keywords to obtain a concatenated field, and obtain a hash value of the concatenated field;

[0182] The search module 30 is configured to search a knowledge base for a target hash value that matches the hash value, wherein the knowledge base is a repository generated based on the keyword, the hash value, and the abnormal information of the business;

[0183] The output module 40 is configured to obtain abnormal information of the service corresponding to the target hash value, and output alarm information according to the abnormal information.

[0184] Furthermore, the output module 40 includes an acquisition unit and an output unit;

[0185] The acquiring unit is configured to acquire the alarm content corresponding to the abnormal information;

[0186] The acquisition unit is further configured to acquire the alarm level, alarm mode, and alarm object corresponding to the alarm content;

[0187] The output unit is configured to output alarm information including the alarm content to the alarm object according to the alarm level and the alarm mode.

[0188] Furthermore, the acquisition unit includes an acquisition subunit and a setting subunit;

[0189] The acquisition subunit is used to obtain the alarm situation corresponding to the abnormal information;

[0190] The setting subunit is configured to set the alarm level to a normal alarm when the alarm condition is an alarm triggered by statistical abnormal information;

[0191] The setting subunit is further configured to set the alarm level to a prompt alarm when the alarm condition is that the frequency of occurrence of the service anomaly is greater than or equal to a first preset value;

[0192] The setting subunit is further configured to set the alarm level to a fault alarm when the alarm condition is that the number of service anomalies is greater than or equal to a second preset value.

[0193] Furthermore, the acquisition unit further includes a determination subunit;

[0194] The acquisition subunit is further used to obtain the calling interface corresponding to the abnormal information;

[0195] The acquisition subunit is used to obtain the access party corresponding to each of the calling interfaces and the identification information of the access party;

[0196] The determining subunit is configured to determine the alarm object according to the hash value and the identification information.

[0197] Furthermore, the business processing device 100 further includes a judgment module, a storage module and a creation module;

[0198] The judgment module is used to judge whether the knowledge base stores the hash value;

[0199] The storage module is configured to store the exception information in the knowledge base when the hash value is stored in the knowledge base;

[0200] The creation module is configured to create a business link according to the keyword to obtain a target business link when the hash value is not stored in the knowledge base;

[0201] The storage module is further configured to store the hash value and the exception information in the target service link.

[0202] Furthermore, the service processing device 100 further includes a conversion module;

[0203] The acquisition module is further configured to acquire a default failure rate of the external interface of the service, and an initial acceptable failure rate and an acceptable failure rate of the access party corresponding to the external interface;

[0204] The conversion module is configured to convert the service into an online processing mode when the acceptable failure rate is greater than or equal to the default failure rate;

[0205] The conversion module is further configured to convert the service into an offline self-learning mode when the acceptable failure rate is less than or equal to the initial acceptable failure rate.

[0206] Furthermore, the acquisition module is further configured to acquire a first performance indicator value of the external interface;

[0207] The acquisition module is further configured to acquire a second performance indicator value of each of the access parties, and acquire a sum of the second performance indicator values;

[0208] The acquisition module is further configured to obtain a quotient of the sum value and the first performance indicator value;

[0209] The acquisition module is further configured to obtain a product of the quotient value and a default failure rate of the external interface, and use the product as the initial acceptable failure rate.

[0210] The implementation of the functions of each module of the above-mentioned business processing device is similar to the process in the above-mentioned method embodiment, and will not be described in detail here.

[0211] In addition, the present application also provides a computer-readable storage medium, on which a business processing method program is stored. When the business processing method program is executed by a processor, the steps of the above business processing method are implemented.

[0212] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0213] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0214] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, TV, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0215] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A business processing method, characterized in that: The method comprises: Obtaining business log information, and extracting keywords corresponding to the business based on the log information; Sorting the keywords, concatenating the sorted keywords to obtain a concatenated field, and obtaining a hash value of the concatenated field; Searching for a target hash value that matches the hash value in a knowledge base, wherein the knowledge base is a tree-structured repository generated based on the keyword, the hash value, and the abnormal information of the business. The knowledge base is a tree-structured repository based on links and KV information, wherein a link is a business logic link composed of keywords; in the knowledge base, each node in the tree structure forms the entire link relationship of the abnormal keyword, which is used to record the abnormal process, wherein the bottom-level node of the tree is used to store KV information, K represents the hash value generated according to the link keyword, which is used to query the node, and V represents the abnormal information, which is used to store the abnormal information or parse the abnormal information; Obtaining abnormal information of the business corresponding to the target hash value, and obtaining alarm content corresponding to the abnormal information, where the alarm content is generated based on V information in the KV information; Obtain the alarm level, alarm mode, and alarm object corresponding to the alarm content. The alarm levels are divided into regular alarm, prompt alarm, and fault alarm. Different alarm levels have different triggering modes; Outputting alarm information including the alarm content to the alarm object according to the alarm level and the alarm mode.

2. The service processing method according to claim 1, wherein: Before the step of obtaining the alarm level, alarm mode, and alarm object corresponding to the alarm content, the method includes: Obtaining the alarm status corresponding to the abnormal information; When the alarm condition is an alarm triggered when abnormal statistical information is collected, the alarm level is set to a normal alarm; When the alarm condition is that the frequency of occurrence of the service anomaly is greater than or equal to a first preset value, setting the alarm level to a prompt alarm; When the alarm condition is that the number of service anomalies is greater than or equal to a second preset value, the alarm level is set to a fault alarm.

3. The service processing method according to claim 1, wherein: Before the step of obtaining the alarm level, alarm mode, and alarm object corresponding to the abnormal information, the method includes: Obtain the calling interface corresponding to the exception information; Obtaining the access party corresponding to each of the calling interfaces and identification information of the access party; The alarm object is determined according to the hash value and the identification information.

4. The business processing method according to claim 1, wherein: The method further comprises: Determining whether the knowledge base stores the hash value; When the hash value is stored in the knowledge base, storing the exception information in the knowledge base; When the hash value is not stored in the knowledge base, creating a business link according to the keyword to obtain a target business link; The hash value and the exception information are stored in the target service link.

5. The business processing method according to claim 1, wherein: The method further comprises: Obtaining a default failure rate of the external interface of the service, and an initial acceptable failure rate and an acceptable failure rate of the access party corresponding to the external interface; When the acceptable failure rate is greater than or equal to the default failure rate, converting the business to an online processing mode; When the acceptable failure rate is less than or equal to the initial acceptable failure rate, the service is converted to an offline self-learning mode.

6. The service processing method according to claim 5, wherein: The method further comprises: Obtaining a first performance indicator value of the external interface; Obtaining a second performance indicator value of each of the access parties, and obtaining a sum of the second performance indicator values; Obtaining a quotient of the sum value and the first performance indicator value; The product of the quotient and the default failure rate of the external interface is obtained, and the product is used as the initial acceptable failure rate.

7. A business processing device, characterized in that: The business processing device includes an acquisition module, a processing module, a search module and an output module, wherein: The acquisition module is used to acquire log information of the business and extract keywords corresponding to the business according to the log information; The processing module is configured to sort the keywords, concatenate the sorted keywords to obtain a concatenated field, and obtain a hash value of the concatenated field; The search module is used to search for a target hash value that matches the hash value in a knowledge base. The knowledge base is a tree-structured repository generated based on the keyword, the hash value, and the abnormal information of the business. The knowledge base is a tree-structured repository based on links and KV information, wherein a link is a business logic link composed of keywords. In the knowledge base, each node in the tree structure forms the entire link relationship of the abnormal keyword, which is used to record the abnormal process. The bottom-level node of the tree is used to store KV information, where K represents the hash value generated according to the link keyword, which is used to query the node, and V represents the abnormal information, which is used to store the abnormal information or parse the abnormal information. The output module is used to obtain the abnormal information of the business corresponding to the target hash value, and obtain the alarm content corresponding to the abnormal information, where the alarm content is generated based on the V information in the KV information; obtain the alarm level, alarm method and alarm object corresponding to the alarm content, where the alarm levels are divided into regular alarms, prompt alarms and fault alarms, and different alarm levels have different triggering methods; and output the alarm information including the alarm content to the alarm object according to the alarm level and the alarm method.

8. A business processing device, characterized in that: The device includes a memory, a processor, and a business processing program stored in the memory and running on the processor. When the processor executes the business processing program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a business processing program, which implements the steps of the method according to any one of claims 1 to 6 when executed by a processor.

Images