Cloud product monitoring data processing method and device, equipment, medium and product

Abstract

The invention provides a cloud product monitoring data processing method and device, equipment, a medium and a product, and relates to the field of cloud platform data monitoring. The method comprises the following steps: acquiring a built-in rule and a product docking configuration document of a to-be-processed cloud product; generating a rule constraint set and an index calculation formula set of the to-be-processed cloud product according to the built-in rule and the product docking configuration document; collecting a monitoring data log corresponding to the to-be-processed cloud product according to a built-in rule; according to the rule constraint set and the index calculation formula set, performing verification processing on the monitoring data log; and according to a verification result, determining an abnormal item in the to-be-processed cloud product. The technical problem that the monitoring data processing efficiency of the cloud product is low due to the fact that an existing monitoring data processing method of the cloud product cannot quickly and effectively analyze the correctness of the monitoring data reported by different products is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cloud platform data monitoring, and in particular to a cloud product monitoring data processing method and device, equipment, medium and product. BACKGROUND

[0002] In the scenario of different cloud products, cloud services, and cloud devices (hereinafter collectively referred to as products) accessing a monitoring and operation platform, different products are designed and implemented by different development teams for monitoring data reporting. Due to differences in product business use scenarios, product configurations, and deployment, etc., the monitoring data reported by different products is diverse.

[0003] Currently, monitoring testers and operation personnel often need to quickly complete a large amount of monitoring data analysis and checking, such as the following situations: batch new products are quickly accessed in parallel, a product monitoring index item is large, up to 100 or more, monitoring and operation platform reconstruction, transformation and evolution, and shortage of test human resources, one person is responsible for tens of products or even hundreds of products accessing monitoring, etc.

[0004] However, the existing cloud product monitoring data processing method cannot quickly and effectively analyze the correctness of the monitoring data reported by different products, thereby resulting in low efficiency of cloud product monitoring data processing. SUMMARY

[0005] The present application provides a cloud product monitoring data processing method, device, equipment, medium and product to solve the problem that the existing cloud product monitoring data processing method cannot quickly and effectively analyze the correctness of the monitoring data reported by different products, thereby resulting in low efficiency of cloud product monitoring data processing.

[0006] In a first aspect, the present application provides a cloud product monitoring data processing method, comprising:

[0007] obtaining built-in rules and product interfacing configuration documents of a cloud product to be processed;

[0008] generating a rule constraint set and an index calculation formula set of the cloud product to be processed according to the built-in rules and the product interfacing configuration documents;

[0009] collecting monitoring data logs corresponding to the cloud product to be processed according to the built-in rules;

[0010] performing checking processing on the monitoring data logs according to the rule constraint set and the index calculation formula set;

[0011] determining an abnormal item in the cloud product to be processed according to the checking result.

[0012] In a possible design, according to built-in rules and a product interfacing configuration document, a rule constraint set and an index calculation formula set of a to-be-processed cloud product are generated, including:

[0013] According to the product interfacing configuration document, index dimension information is extracted.

[0014] According to the index dimension information, key field extraction and matching processing are performed on the built-in rules, to obtain key fields corresponding to the index dimension information.

[0015] According to the index dimension information and the key fields, the rule constraint set and the index calculation formula set of the to-be-processed cloud product are generated.

[0016] In a possible design, the product interfacing configuration document includes multiple types of index names, product categories, theme keywords, corresponding monitoring log field names, unique judgments, and index calculation formulas.

[0017] In a possible design, according to historical data, the built-in rules include product category information and product theme keyword information.

[0018] In a possible design, according to the built-in rules, monitoring data logs corresponding to the to-be-processed cloud product are collected, including:

[0019] According to the product category information and the product theme keyword information, monitoring data collection rules are determined.

[0020] According to the monitoring data collection rules, the monitoring data logs corresponding to the to-be-processed cloud product are collected.

[0021] In a possible design, according to the rule constraint set and the index calculation formula set, the monitoring data logs are verified, including:

[0022] According to the rule constraint set and the index calculation formula set, multiple judgment rules are determined.

[0023] For each judgment rule, the monitoring data logs are verified to obtain a verification result corresponding to each judgment rule, where the verification result includes verification success and verification failure.

[0024] In a possible design, according to the verification result, an abnormal item in the to-be-processed cloud product is determined, including:

[0025] According to a judgment rule with a verification result of verification failure, the abnormal item in the to-be-processed cloud product is determined.

[0026] In a possible design, after the abnormal item in the to-be-processed cloud product is determined according to the judgment rule with the verification result of verification failure, the method further includes:

[0027] According to the abnormal item, analysis data for the cloud product is generated; wherein the analysis data comprises one of table type analysis data, image type analysis data and text type analysis data.

[0028] In a second aspect, there is provided:

[0029] An acquisition module is configured to acquire built-in rules and product interfacing configuration documents of the cloud product to be processed.

[0030] A generation module is configured to generate a rule constraint set and an index calculation formula set of the cloud product to be processed according to the built-in rules and the product interfacing configuration documents.

[0031] A collection module is configured to collect monitoring data logs corresponding to the cloud product to be processed according to the built-in rules.

[0032] A verification module is configured to perform verification processing on the monitoring data logs according to the rule constraint set and the index calculation formula set.

[0033] A determination module is configured to determine abnormal items in the cloud product to be processed according to the verification result.

[0034] In a third aspect, the present application provides a cloud product monitoring data processing device, comprising a memory and a processor.

[0035] The memory stores computer execution instructions.

[0036] The processor executes the computer execution instructions stored in the memory, so that the processor executes the cloud product monitoring data processing method according to the first aspect.

[0037] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the cloud product monitoring data processing method according to the first aspect.

[0038] In a fifth aspect, the present application provides a computer program product, which comprises a computer program, and the computer program is executed by the processor to implement the cloud product monitoring data processing method according to the first aspect.

[0039] The cloud product monitoring data processing method, device, equipment, medium and product provided by the application, the built-in rules and product docking configuration documents of a to-be-processed cloud product are acquired; the rule constraint set and the index calculation formula set of the to-be-processed cloud product are generated according to the built-in rules and the product docking configuration documents; the monitoring data logs corresponding to the to-be-processed cloud product are collected according to the built-in rules; the monitoring data logs are checked according to the rule constraint set and the index calculation formula set; and the abnormal items in the to-be-processed cloud product are determined according to the checking result. Compared with the existing cloud product monitoring data processing method, the correctness of the monitoring data reported by different products cannot be quickly and effectively analyzed, and thus the cloud product monitoring data processing efficiency is low. Based on the pre-configured built-in rules and product docking configuration documents corresponding to the cloud product, the rule constraint set and the index calculation formula set suitable for the cloud product type can be automatically generated. When the product accesses the monitoring operation and maintenance platform, the monitoring data logs and the product docking configuration documents reported by the test personnel are input into the analysis method model of the patent, and the possible problems of the monitoring data reported by the product side can be output, so that the rapid iteration repair of the correctness problem of the monitoring data is realized, and the cloud product monitoring data processing efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0041] Figure 1 A system architecture schematic diagram of a cloud product monitoring data processing method provided by the embodiment of the present application;

[0042] Figure 2 A cloud product monitoring data processing method flowchart provided by the embodiment of the present application Figure 1 ;

[0043] Figure 3 A cloud product monitoring data processing method flowchart provided by the embodiment of the present application Figure 2 ;

[0044] Figure 4 An index judgment rule group schematic diagram provided by the embodiment of the present application;

[0045] Figure 5 A cloud product monitoring data processing method flowchart provided by the embodiment of the present application Figure 3 ;

[0046] Figure 6A monitoring data log verification process schematic diagram provided by an embodiment of the present application;

[0047] Figure 7 A structure schematic diagram of a monitoring data processing device of a cloud product provided by an embodiment of the present application;

[0048] Figure 8 A structure schematic diagram of a monitoring data processing device of a cloud product provided by an embodiment of the present application;

[0049] Figure 9 A monitoring data processing method flowchart of a cloud product provided by an embodiment of the present application Figure 4 . DETAILED DESCRIPTION

[0050] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is made with reference to the accompanying drawings in which like reference numerals refer to like elements in the several figures. The following description of exemplary embodiments is not representative of all possible embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0051] In the embodiments of the present application, the same items or similar items with basically same functions and effects are distinguished by using “first”, “second”, etc. The skilled in the art can understand that “first”, “second”, etc. do not limit the quantity and execution order, and “first”, “second”, etc. do not necessarily mean different. It should be noted that in the embodiments of the present application, “exemplary” or “for example” is used to represent as an example, illustration or description. Any embodiment or design scheme described as “exemplary” or “for example” in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. In fact, using “exemplary” or “for example” is intended to present the relevant concept in a specific way. In the embodiments of the present application, “at least one” means one or more, and “multiple” means two or more than two.

[0052] It should be noted that “at the time of” in the embodiments of the present application can be at the moment when a certain condition occurs, or in a period of time after a certain condition occurs, which is not limited in the embodiments of the present application. In addition, the monitoring data processing method of the cloud product provided by the embodiments of the present application is only as an example, and the monitoring data processing method of the cloud product can also include more or less content.

[0053] With the development of information technology, in the scenario of different cloud products, cloud services, cloud devices (hereinafter collectively referred to as products) accessing the monitoring and operation platform, different products are designed and implemented by different development teams to report monitoring data. Due to the great differences in product business use scenarios, product configuration and deployment, etc., the monitoring data reported by different products is diverse.

[0054] However, with the development of business scenarios, monitoring testers and operation personnel often need to quickly complete a large amount of monitoring data analysis and checking, such as the following situations: (1) batch of new products are quickly accessed in parallel (2) a product has a large number of monitoring index items, up to 100 or more (3) the monitoring and operation platform is restructured, transformed and evolved (4) test human resources are short, one person is responsible for tens of products or even hundreds of products accessing monitoring.

[0055] Therefore, the current monitoring data processing method of cloud products mainly has the following problems:

[0056] On the one hand, there is currently a lack of solutions to verify the correctness of massive monitoring data in the scenario of multiple types of cloud products, cloud services and cloud devices accessing the monitoring and operation platform.

[0057] On the other hand, the current cloud monitoring data related to the cloud monitoring product help documents, monitoring product introductions, cloud product test methods, cloud monitoring system scheme designs, or monitoring data analysis systems based on cloud platforms in a specific field, etc.

[0058] Based on this, the embodiments of the present application provide a cloud product monitoring data processing method, device, equipment, medium and product, which can be used in the field of cloud platform data monitoring, and aims to solve the above technical problems of the prior art.

[0059] The application concept of the present application is that, in order to solve the above problems, the inventors found that the existing cloud product monitoring data processing method cannot quickly and effectively analyze the correctness of the monitoring data reported by different products during the research on the efficiency of cloud product monitoring data processing. Based on this, the inventors designed a general analysis method based on the general built-in rules of different access products, the interface configuration document of the product to be accessed, and the association relationship between monitoring items in special cases. The method is an automatic analysis method model that can adapt to multiple types of product monitoring data. When the product accesses the monitoring and operation platform, the test personnel inputs the reported monitoring data log and the product interface configuration document as input into the analysis method model, which can output the possible problems of the monitoring data reported by the product side, thereby realizing the rapid iteration and repair of the monitoring data correctness problem. Based on this, the present application proposes a cloud product monitoring data processing method to further improve the efficiency of cloud product monitoring data processing.

[0060] The technical solutions of the present application and how the technical solutions solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes can not be described again in some examples. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0061] Figure 1 A system architecture diagram of a cloud product monitoring data processing method is provided for the embodiments of the present application. In Figure 1 the above architecture includes at least one of a data acquisition device 101, a processing device 102, and a display device 103.

[0062] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the system architecture of the cloud product monitoring data processing system. In other feasible embodiments of the present application, the above architecture can include more or fewer components than the diagram, or combine certain components, or split certain components, or different component arrangements, which can be determined according to actual application scenarios, and are not limited herein. Figure 1 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0063] In the specific implementation process, the data acquisition device 101 can include an input / output interface and can also include a communication interface. The data acquisition device 101 can be connected to the processing device through the input / output interface or the communication interface to obtain a plurality of cloud product monitoring data to be processed.

[0064] The processing device 102 can collect monitoring data logs corresponding to the cloud product to be processed according to the plurality of cloud product monitoring data to be processed, perform verification processing, and determine abnormal items in the cloud product to be processed according to the verification result.

[0065] The display device 103 can also be a touch display screen or a screen of a terminal device, which is used to trigger alarm information or store synchronization data while displaying the above content, so as to realize interaction with the user.

[0066] It should be understood that the above processing device can be implemented by a processor reading and executing instructions in a memory, or by a chip circuit.

[0067] In addition, the network architecture and business scenarios described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It can be known by those skilled in the art that, with the evolution of network architecture and the appearance of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0068] The technical solutions of the present application will be described in detail below with specific examples.

[0069] Figure 2 A cloud product monitoring data processing method flow provided for the embodiments of the present application Figure 1 As shown in the method comprises: Figure 2

[0070] S201, obtain the built-in rules and product docking configuration documents of the cloud product to be processed.

[0071] Among them, the built-in rules include product category information and product topic keyword information.

[0072] For example, according to the agreement of product access to the monitoring and operation platform, the following data is preset:

[0073] Optionally, the necessary information: product category list and topic keyword, wherein the product category list is the product category information, and the topic keyword is the product topic keyword information.

[0074] Optionally, the optional information: the value constraint type and the associated constraint type of the specific index.

[0075] Among them, the product category and the topic keyword must be accurately filled in. The index value constraint and the index association constraint are optional, and if they are accurately preset, more accurate constraint rule set and calculation formula can be obtained.

[0076] Among them, the product docking configuration document includes multiple kinds of index name, product category, topic keyword, corresponding monitoring log field name, unique judgment and index calculation formula.

[0077] For example, the product docking configuration document can also be called product docking document. The product docking document generally needs to include: monitoring index item, product category flag, topic, corresponding index monitoring log field name, unique judgment, and index calculation formula.

[0078] S202, according to the built-in rules and the product docking configuration document, generate the rule constraint set and the index calculation formula set of the cloud product to be processed.

[0079] It should be noted that the general built-in rules set the common default configuration, which has a certain universality and can be directly applied to common product monitoring data testing.

[0080] S203, according to the built-in rules, collect the monitoring data logs corresponding to the cloud product to be processed.

[0081] Specifically, according to the product category information and the product topic keyword information, the monitoring data collection rules are determined. ​

[0082] Specifically, according to the monitoring data collection rule, the monitoring data log corresponding to the to-be-processed cloud product is collected.

[0083] In this embodiment, according to the preset product category List and topic keyword, the data checking time range can be set by the user, the monitoring data collection rule is determined, and the monitoring data log file is obtained.

[0084] S204, according to the rule constraint set and the index calculation formula set, the monitoring data log is checked.

[0085] S205, according to the checking result, the abnormal item in the to-be-processed cloud product is determined.

[0086] Specifically, according to the checking result, the abnormal item in the to-be-processed cloud product is determined.

[0087] It should be noted that after step S205, it further includes: generating analysis data for the cloud product according to the abnormal item.

[0088] The analysis data includes one of table type analysis data, image type analysis data and text type analysis data.

[0089] The cloud product monitoring data processing method provided in this embodiment obtains the built-in rule and product docking configuration document of the to-be-processed cloud product; generates the rule constraint set and the index calculation formula set of the to-be-processed cloud product according to the built-in rule and the product docking configuration document; collects the monitoring data log corresponding to the to-be-processed cloud product according to the built-in rule; checks the monitoring data log according to the rule constraint set and the index calculation formula set; and determines the abnormal item in the to-be-processed cloud product according to the checking result. Compared with the existing cloud product monitoring data processing method, the correctness of the monitoring data reported by different products cannot be quickly and effectively analyzed, which leads to low efficiency of cloud product monitoring data processing. Based on the pre-configured built-in rule and product docking configuration document corresponding to the cloud product, the rule constraint set and the index calculation formula set suitable for the cloud product type can be automatically generated. When the product accesses the monitoring and operation platform, the test personnel inputs the reported monitoring data log and the product docking configuration document as input into the analysis method model of the patent, and the possible problems of the monitoring data reported by the product side can be output, so that the rapid iteration repair of the correctness of the monitoring data is realized, and the efficiency of the cloud product monitoring data processing is improved.

[0090] Figure 3 A cloud product monitoring data processing method flowchart provided by the embodiment of the present application Figure 2 As shown in Figure 3 The specific implementation steps of S202 include:

[0091] S301, extracting index dimension information according to product docking configuration documents.

[0092] S302, extracting and matching key fields of built-in rules according to index dimension information to obtain key fields corresponding to index dimension information.

[0093] S303, generating rule constraint set and index calculation formula set of the to-be-processed cloud product according to index dimension information and key fields.

[0094] For example, the constraint rule setting includes:

[0095] Setting one, product category list setting: ProductTypeList[P1,...,Pn]. In practice, it is usually 1 or 2.

[0096] Setting two, product topic keyword list setting: TopicList[T1,...,Tn]. In practice, it is usually 1-4.

[0097] Setting three, monitoring data push time interval setting, which is usually 1 minute.

[0098] Setting four, monitoring data message body setting:

[0099] Among them, the format is Json format.

[0100] Among them, the field and field position are shown in Table 1:

[0101] Table 1

[0102] Field name Whether necessary Position Product category Yes Located in the first key of Json Topic Yes Located in the first key of Json Push timestamp Yes Located in the first key of Json Monitoring index Yes Can be set, default Json first key Unique mark field Yes Can be set, default Json first key

[0103] Setting five includes:

[0104] Optionally, settings one to four are met.

[0105] Optionally, the index value constraint is met. Since the index data is mostly numerical, it can be set according to Table 2:

[0106] Table 2

[0107]

[0108] Optionally, the index association constraint is met, which can be set according to Table 3:

[0109] Table 3

[0110]

[0111] For example, common index calculation formulas include: instantaneous value - default formula, instantaneous value / time, increment, increment / time, sum of several indexes, and extension, i.e., a specified complex calculation formula.

[0112] For further example, the pre-configuration and product docking document is input into the product docking configuration data analysis module to form a rule constraint set and an index calculation formula set adapted to the product, which is stored according to the index as a dimension, taking Table 4 as an example:

[0113] Table 4

[0114]

[0115] It should be noted that Tables 1-4 are only illustrative and do not affect the scope of protection of the embodiments of the present application.

[0116] In the present embodiment, based on the built-in rules and the product docking configuration document, the built-in rules are subjected to key field extraction and matching processing to obtain the key fields corresponding to the index dimension information, and the rule constraint set and the index calculation formula set of the cloud product to be processed are generated. Through systematic dimension extraction, accurate rule matching and dynamic knowledge generation, a full-link automation system covering data modeling, logical verification and knowledge reuse is constructed, which significantly improves the accuracy and processing efficiency of the monitoring data verification, realizes a significant reduction in rule configuration time and a significant reduction in system anomaly detection rate, and at the same time, through modular design, ensures the rapid iteration and version management of the verification rules, finally forms an efficient, reliable and scalable intelligent verification solution, thereby improving the monitoring data processing efficiency of the cloud product.

[0117] Figure 5 A cloud product monitoring data processing method flowchart provided by the embodiments of the present application Figure 3 As shown in Figure 5 , the specific implementation steps of the above S204 include:

[0118] It should be noted that the judgment rule group can be obtained according to Table 1, Table 2, Table 3 and Table 4.

[0119] In one possible embodiment, Figure 4 A judgment rule group provided by the embodiments of the present application is shown in Figure 4 .

[0120] Optionally, rule one: product category is correct, used to verify the accuracy of product classification.

[0121] Optionally, rule two: monitoring data reporting frequency is correct, responsible for checking whether the data acquisition period meets the preset standard.

[0122] Optionally, rule three: message body is correct.

[0123] Specifically, it includes: message body format is correct, necessary field is not missing, and specified field position is correct.

[0124] Among them, the message body format is correct for checking whether the data packet structure conforms to the protocol specification.

[0125] Among them, the necessary field is not missing for ensuring that key parameters such as timestamps, device IDs, and other must-have items are complete.

[0126] Among them, the specified field position is correct for verifying whether the position of a specific parameter in the data packet conforms to the encoding rules.

[0127] Optionally, rule four: index value constraint: pass, for checking whether the numerical index is within a reasonable threshold range.

[0128] Optionally, rule five: index association constraint: pass, for verifying the logical association between indexes (such as A index exception when B index should change synchronously).

[0129] Optionally, rule six: extension rule, a reserved custom verification interface, can dynamically add new verification rules according to business needs.

[0130] It should be noted that when the product accesses the monitoring and operation platform, the test personnel will report the monitoring data log of the product and the product interface configuration document as input, input into the model, and output the possible problems of the monitoring data reported by the product side. At the same time, the test personnel can use the extension field in the rule knowledge base to further expand the rules, combine the learned rule constraint set and specific rules to obtain more accurate analysis results.

[0131] It should be noted that the extension rule field is used to improve the extensibility of the system, and can be expanded and supplemented for specific constraint rules of specific monitoring items, which can be applied to more complex monitoring index item analysis and has universality.

[0132] It should also be noted that Figure 4 The entire process combines a tree branch structure to intuitively present a multi-level quality control system from basic attribute verification to complex index association verification

[0133] S501, according to the rule constraint set and the index calculation formula set, determine a plurality of judgment rules.

[0134] S502, for each judgment rule, the monitoring data log is verified and processed to obtain a verification result corresponding to each judgment rule.

[0135] Among them, the verification result includes verification success and verification failure.

[0136] In one possible embodiment,Figure 6 The monitoring data log verification process schematic diagram provided for the embodiments of the present application is shown in FIG. 1. Figure 6 As shown, the complete verification chain from data extraction to result integration is presented, and a linear process combined with parallel rule base architecture design is adopted:

[0137] Firstly, the first monitoring index in the table, Metric1, is analyzed.

[0138] The index contains the following information: index name, product category, topic keyword, corresponding monitoring log field name, unique judgment, index calculation formula, index value constraint type, and index association constraint type.

[0139] Secondly, according to the index name Metric1, the log data is filtered to obtain the index name, product category, necessary field, reporting time, and index value in the log.

[0140] Thirdly, the obtained log data is sequentially judged by the rule set rules to obtain the judgment result of the monitoring index.

[0141] Then, the above three processes are performed on all index records in the table.

[0142] Finally, all index items that do not pass are integrated and output.

[0143] Optionally, the data source input: the process starts from the "monitoring data log" data source, and the target index record "Metrici" is obtained through a special extraction module to establish the verification object basis.

[0144] Optionally, multi-level judgment rule engine: the extracted index record enters a serial judgment rule group, and the rule group contains N judgment modules (judgment rule 1 to judgment rule N) executed in sequence. Each rule module independently verifies the index record to generate corresponding structured judgment results (Metrici: judgment result 1 to judgment result N), realizing multi-dimensional and hierarchical index verification.

[0145] Optionally, abnormal result integration mechanism: after all judgment rules are executed, the process enters the "all index items that do not pass integration result" link. This module aggregates the items that do not pass in each judgment result to form a unified abnormal item list, providing data support for subsequent problem analysis.

[0146] Optionally, rule base and calculation engine: the "product adaptation constraint set, index calculation formula set" module is set in parallel at the top of the process, which contains two types of core elements:

[0147] Among them, the constraint set: defines the specific threshold and logical conditions of each judgment rule.

[0148] Among them, the formula set: provide mathematical model and method for index calculation.

[0149] It should be noted that the whole process clearly shows the data flow through the one-way arrow, from log extraction, through rule checking, result generation, to finally complete the exception integration, while the top rule base realizes the parameterization configuration of the verification logic, forming a complete closed-loop verification system.

[0150] It should also be noted that simple input of docking configuration document and monitoring data log can output results, effectively solving or avoiding problems such as lack of human resources to cope with large number of new product rapid access and monitoring index items, and improving work efficiency.

[0151] In this embodiment, based on product category information and product topic keyword information, the monitoring data collection rule is determined, and the monitoring data log corresponding to the to-be-processed cloud product is collected. Through the systematic application of rule constraint set and index formula, an automatic verification engine is constructed, efficient and accurate verification of monitoring data is realized, and the processing efficiency of cloud product monitoring data is improved.

[0152] In one possible embodiment, Figure 9 A cloud product monitoring data processing method flow provided by the embodiment of the application Figure 4 As Figure 9 shown, after receiving the test demand of the cloud product access to the monitoring operation and maintenance platform, the test personnel can utilize the richness of the product general built-in rule knowledge base module rule set and the learning ability of the product docking configuration data analysis module to obtain the rule constraint set and the index calculation formula set adapted to the product. Then, for each monitoring index item to be tested, the monitoring data analysis module obtains the key information, and finally the product rule constraint set and the index calculation formula set are tested through the rule interpreter one by one, and finally the unqualified items of all indexes are output.

[0153] Specifically, the flowchart shows a double-channel analysis system based on product configuration and monitoring data, which adopts a three-layer architecture design of pre-configuration-analysis-decision, and the running logic is as follows:

[0154] Optionally, the double-pre-configuration system sets two groups of independent pre-configuration parameters when the process is started:

[0155] Among them, pre-configuration 1 (rule generation layer): contains product category whitelist (mandatory), Topic keyword (mandatory), index constraint type (optional, default instantaneous value), and associated constraint type (optional, default no association).

[0156] Among them, pre-configuration 2 (data filtering layer): contains product category filtering (mandatory), Topic matching word (mandatory), and time range limitation (optional, default full amount).

[0157] Optionally, dual data source input, the system receives two types of data sources in parallel:

[0158] Among them, the structured configuration document: through the "product docking configuration data analysis" module, the output rule generates the required dimension information.

[0159] Among them, real-time monitoring log: through the "monitoring data log analysis" module, the extracted index data to be checked.

[0160] Optionally, rule and data fusion, in the "rule and data selection" core node realizes double channel fusion:

[0161] Among them, the rule channel: based on pre-configuration 1 and configuration document to generate dynamic rule set (including constraint set / formula set).

[0162] Among them, the data channel: according to pre-configuration 2 from monitoring log to filter effective index data.

[0163] Among them, channel interaction: through the bidirectional arrow to realize the accurate matching of rules and data (such as the association mechanism of feature engineering and sample set in financial risk control system).

[0164] Optionally, intelligent verification decision, the fused rule-data pair enters the "rule interpreter" to execute verification:

[0165] Among them, the multi-mode verification engine is adopted: combined with rule engine (such as Drools) and machine learning model (such as random forest).

[0166] Among them, the output structured result: generate index unpassed item list, including specific rule ID, data deviation value, timestamp and other metadata.

[0167] It should be noted that when batch products, product indexes and other access monitoring operation and maintenance platform, test and operation and maintenance personnel only need to input docking configuration document and monitoring data log, can obtain output result, effectively solve or avoid due to shortage of human resources, can not deal with the problem of batch new product rapid access, monitoring index item huge, improve work efficiency and help new product rapid verification.

[0168] In the embodiment, based on the functions of each module (configuration type, data type and decision type), and the interaction relationship between data and rules, a complete closed loop from configuration management to real-time verification is formed. It can be applied to the scene of monitoring and operation and maintenance platform access of multiple types of cloud products, and the scene of monitoring data correctness verification. When the test and operation and maintenance personnel receive the test requirements, on the one hand, the general built-in constraint set, rule set and calculation formula set can be used, and only the monitoring data log reported by the product and the product interface configuration document are input, the analysis result of whether the monitoring data reported by the product side has a problem can be obtained. On the other hand, the special rules of the specific monitoring items can be set by using the extension field in each module. The built-in rule constraint set and the specific rules are used in combination, and more accurate analysis results can be obtained.

[0169] Figure 7 A structure diagram of a cloud product monitoring data processing device provided by the embodiment of the application is shown in Figure 7 The device comprises an acquisition module 71, a generation module 72, a collection module 73, a verification module 74 and a determination module 75.

[0170] The acquisition module 71 is used to acquire the built-in rules of the cloud product to be processed and the product interface configuration document.

[0171] The generation module 72 is used to generate the rule constraint set and the index calculation formula set of the cloud product to be processed according to the built-in rules and the product interface configuration document.

[0172] The collection module 73 is used to collect the monitoring data log corresponding to the cloud product to be processed according to the built-in rules.

[0173] The verification module 74 is used to perform verification processing on the monitoring data log according to the rule constraint set and the index calculation formula set.

[0174] The determination module 75 is used to determine the abnormal items in the cloud product to be processed according to the verification result.

[0175] In a possible design, the rule constraint set and the index calculation formula set of the cloud product to be processed are generated according to the built-in rules and the product interface configuration document, and the generation comprises:

[0176] The generation module 72 is further used to extract the index dimension information according to the product interface configuration document.

[0177] The built-in rules are subjected to extraction and matching processing of the key fields according to the index dimension information, so as to obtain the key fields corresponding to the index dimension information.

[0178] The rule constraint set and the index calculation formula set of the cloud product to be processed are generated according to the index dimension information and the key fields.

[0179] In a possible design, the product docking configuration document includes multiple types of index names, product categories, subject keywords, corresponding monitoring log field names, unique judgments, and index calculation formulas.

[0180] In a possible design, the built-in rules include product category information and product subject keyword information according to historical data.

[0181] In a possible design, according to the built-in rules, the monitoring data logs corresponding to the cloud product to be processed are collected, including:

[0182] The collection module 73 is further configured to determine a monitoring data collection rule according to the product category information and the product subject keyword information.

[0183] The monitoring data logs corresponding to the cloud product to be processed are collected according to the monitoring data collection rule.

[0184] In a possible design, the monitoring data logs are verified according to the rule constraint set and the index calculation formula set, including:

[0185] The verification module 74 is further configured to determine multiple judgment rules according to the rule constraint set and the index calculation formula set.

[0186] The monitoring data logs are verified according to each judgment rule, to obtain a verification result corresponding to each judgment rule, wherein the verification result includes verification success and verification failure.

[0187] In a possible design, the abnormal items in the cloud product to be processed are determined according to the verification result, including:

[0188] The determination module 75 is further configured to determine abnormal items in the cloud product to be processed for the judgment rules with verification failure according to the verification result.

[0189] In a possible design, after the abnormal items in the cloud product to be processed are determined for the judgment rules with verification failure according to the verification result, the method further includes:

[0190] The determination module 75 is further configured to generate analysis data for the cloud product according to the abnormal items, wherein the analysis data includes one of table type analysis data, image type analysis data, and text type analysis data.

[0191] The cloud product monitoring data processing apparatus provided in this embodiment can execute the cloud product monitoring data processing method in the above-described embodiments, and has similar implementation principles and technical effects, which will not be described here again.

[0192] In the specific implementation of the cloud product monitoring data processing method, each module can be implemented as a processor, and the processor can execute computer execution instructions stored in the memory, so that the processor executes the cloud product monitoring data processing method.

[0193] Figure 8 A structural diagram of a cloud product monitoring data processing device provided by an embodiment of the present application is shown in FIG. 8. As shown in the figure, the cloud product monitoring data processing device 80 includes at least one processor 81 and a memory 82. The cloud product monitoring data processing device 80 further includes a communication component 83. The processor 81, the memory 82 and the communication component 83 are connected through a second bus 84. Figure 8

[0194] In the specific implementation process, the at least one processor 81 executes the computer execution instructions stored in the memory 82, so that the at least one processor 81 executes the cloud product monitoring data processing method executed by the cloud product monitoring data processing device side as described above.

[0195] The specific implementation process of the processor 81 can refer to the method embodiments described above, which has similar implementation principles and technical effects, and will not be described here again in this embodiment.

[0196] In the above embodiments, it should be understood that the processor can be a central processing unit (English: Central Processing Unit, CPU for short), and can also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, DSP for short), application specific integrated circuits (English: Application Specific Integrated Circuit, ASIC for short) and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like. The steps of the method disclosed in the application can be directly embodied as hardware processor execution or executed by a combination of hardware and software modules in the processor.

[0197] The memory can include a high-speed RAM memory, and can also include a non-volatile storage NVM, for example, at least one disk memory.

[0198] ​The second bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0199] The functions implemented by the cloud product monitoring data processing device and the master control device described above are introduced for the scheme provided by the embodiments of the present application. It can be understood that, in order to implement the above functions, the cloud product monitoring data processing device or the master control device contains the corresponding hardware structure and / or software module for executing each function. In combination with the units and algorithm steps of each example described in the embodiments disclosed in the embodiments of the present application, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or in the form of computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present application.

[0200] The present application also provides a computer readable storage medium, the computer readable storage medium stores computer execution instructions, when the processor executes the computer execution instructions, the computer execution instructions are used to implement the cloud product monitoring data processing method.

[0201] The above readable storage medium can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0202] An example readable storage medium is coupled to the processor such that the processor can read information from, and can write information to, the readable storage medium. Of course, the readable storage medium can be part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can be located in a cloud product monitoring data processing device or a master device as discrete components.

[0203] The application also provides a computer program product, which comprises a computer program stored in a readable storage medium, at least one processor of a cloud product monitoring data processing device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to enable the cloud product monitoring data processing device to perform the scheme provided in any of the above embodiments.

[0204] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The foregoing program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the foregoing storage medium includes ROM, RAM, magnetic disc or optical disc and various storage media that can store program codes.

[0205] So far, the technical scheme of the application has been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the application is obviously not limited to these specific embodiments, and the above embodiments are only used to illustrate the technical scheme of the application, but not to limit it; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical scheme recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical scheme deviate from the scope of the technical scheme of the embodiments of the application.

Claims

1. A cloud product monitoring data processing method, characterized in that, The method comprises the following steps: obtaining built-in rules and product connection configuration documents of a cloud product to be processed; generating a rule constraint set and an index calculation formula set of the cloud product to be processed according to the built-in rules and the product connection configuration documents; collecting monitoring data logs corresponding to the cloud product to be processed according to the built-in rules; performing verification processing on the monitoring data logs according to the rule constraint set and the index calculation formula set; determining an abnormal item in the cloud product to be processed according to a verification result.

2. The method of claim 1, wherein, The generating of the rule constraint set and the index calculation formula set of the cloud product to be processed according to the built-in rules and the product connection configuration documents comprises the following steps: extracting index dimension information according to the product connection configuration documents; extracting and matching processing of key fields are performed on the built-in rules according to the index dimension information, so as to obtain key fields corresponding to the index dimension information; generating the rule constraint set and the index calculation formula set of the cloud product to be processed according to the index dimension information and the key fields.

3. The method of claim 2, wherein, The product connection configuration documents comprise multiple types of index names, product categories, theme keywords, corresponding monitoring log field names, unique judgments and index calculation formulas.

4. The method of claim 2, wherein, The built-in rules comprise product category information and product theme keyword information.

5. The method of claim 4, wherein, The collecting of the monitoring data logs corresponding to the cloud product to be processed according to the built-in rules comprises the following steps: determining a monitoring data collection rule according to the product category information and the product theme keyword information; collecting the monitoring data logs corresponding to the cloud product to be processed according to the monitoring data collection rule.

6. The method according to any one of claims 1 to 5, characterized in that, The verification processing on the monitoring data logs according to the rule constraint set and the index calculation formula set comprises the following steps: determining multiple judgment rules according to the rule constraint set and the index calculation formula set; performing verification processing on the monitoring data logs for each judgment rule to obtain a verification result corresponding to each judgment rule; wherein the verification result comprises verification success and verification failure.

7. The method of claim 6, wherein, The determining of the abnormal item in the cloud product to be processed according to the verification result comprises the following steps: determining the abnormal item in the cloud product to be processed according to a judgment rule with a verification failure in the verification result.

8. The method of claim 7, wherein, After the determining of the abnormal item in the cloud product to be processed according to the judgment rule with the verification failure in the verification result, the method further comprises the following steps: generating analysis data for the cloud product according to the abnormal item; wherein the analysis data comprises one of table type analysis data, image type analysis data and text type analysis data.

9. A monitoring data processing apparatus of a cloud product, characterized by, The method comprises the following steps: an obtaining module, configured to obtain built-in rules and product connection configuration documents of a cloud product to be processed; a generating module, configured to generate a rule constraint set and an index calculation formula set of the cloud product to be processed according to the built-in rules and the product connection configuration documents; a collecting module, configured to collect monitoring data logs corresponding to the cloud product to be processed according to the built-in rules; a verification module, configured to perform verification processing on the monitoring data logs according to the rule constraint set and the index calculation formula set; A determining module is configured to determine an abnormal item in the cloud product to be processed according to the check result.

10. A cloud product monitoring data processing device, characterized by, The application relates to a computer readable storage medium, comprising a computer program. A memory, a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method in any one of claims 1-8.

12. A computer program product, characterised in that, The application relates to a computer program, which is executed by the processor to implement the method in any one of claims 1-8.

Images