Data processing method, device and system, computer equipment and readable storage medium

Abstract

The invention relates to the technical field of computers, in particular to a data processing method, device and system, computer equipment and a readable storage medium. The data processing method comprises the steps of obtaining to-be-processed performance data; acquiring a data grouping rule; grouping the performance data according to a data grouping rule to form a plurality of group packages; respectively rendering a plurality of performance curves of the performance data in each group package to form curve charts respectively belonging to each group package; and performing anomaly evaluation on the performance curve, and giving an anomaly identifier to the anomaly information in response to the judgment that the performance curve has the anomaly information. By adopting the method, interface display can be simplified, and data display information can be enriched.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, a data processing device, a data processing system, a computer device, and a computer-readable storage medium. Background Art

[0002] As the size of the cluster continues to expand, the resources that can be used for performance management also increase, which makes the performance report data for real-time monitoring very redundant.

[0003] However, due to the size design of the display page / curve chart, it is difficult to quickly and accurately find the performance data you need to view from the massive statistical data. At the same time, taking the monitoring of node performance data as an example, at least one curve is usually used to represent the performance change of a node. With the increase in the objects for performance statistics, the number of curves in the same curve chart becomes redundant, and it is easy for the curve chart to be complicated and difficult to identify, which makes it difficult to quickly extract effective information. Summary of the invention

[0004] Based on this, it is necessary to provide a data processing method, a data processing device, a data processing system, a computer device and a computer-readable storage medium to address the above-mentioned technical problems, which can simplify the interface display and enrich the information displayed by the data.

[0005] On the one hand, a data processing method is provided, and the data processing method includes: obtaining performance data to be processed; obtaining data grouping rules; grouping the performance data according to the data grouping rules to form a number of group packages; rendering a number of performance curves of the performance data in each group package respectively to form curve charts belonging to each group package respectively; performing an abnormality evaluation on the performance curve, and in response to determining that abnormal information exists in the performance curve, assigning an abnormality mark to the abnormal information.

[0006] In one embodiment of the present application, the data grouping rule includes at least one of a curve number threshold sub-rule and a curve clone sub-rule of a curve chart; grouping the performance data according to the data grouping rule to form a plurality of group packages includes: in response to the data grouping rule including the curve number threshold sub-rule of the curve chart, making the number of performance curves that can be rendered by the performance data divided into each group package less than or equal to the curve number threshold; wherein, differences in the number of performance curves that can be rendered by the group package are allowed; in response to the performance data belonging to a performance curve carrying a clone identifier, determining that the data grouping rule complies with the curve clone sub-rule, treating the performance data carrying the clone identifier as the performance data to be cloned, and writing the performance data to be cloned into multiple target groups; when rendering the group packages of the multiple target groups, rendering the performance data to be cloned to form performance curves as clone curves, so that the curve chart formed by the group package including the performance data to be cloned displays the clone curve.

[0007] In one embodiment of the present application, the data grouping rules include an adaptive grouping number sub-rule; obtaining the data grouping rules includes: obtaining the resolution height of the display page; wherein the display page is used to display the performance curve; obtaining the height occupied by the fixed components in the display page; evaluating the height of the performance curve and the number of columns of the performance curve; the calculation formula for the adaptive grouping number is: X=(L1-L2) / L3*n; wherein X represents the adaptive grouping number; L1 represents the resolution height of the display page; L2 represents the height occupied by the fixed components in the display page; L3 represents the height of the performance curve; n represents the number of columns of the performance curve.

[0008] In one embodiment of the present application, performing an abnormality evaluation on a performance curve includes: identifying an analysis category of the performance curve; wherein the analysis category includes at least one of a threshold statistical category, an incremental statistical category, and a discrete statistical category; in response to the analysis category of the performance curve being a threshold statistical category, traversing performance data of the performance curve, and taking performance data exceeding a first threshold as abnormal information; in response to the analysis category of the performance curve being an incremental statistical category, fitting the performance curve, and deriving the fitted performance curve, and taking performance data whose increasing rate exceeds a second threshold as abnormal information; in response to the analysis category of the performance curve being a discrete statistical category, using a normal distribution to evaluate the performance curve for abnormal information.

[0009] In one embodiment of the present application, the abnormality evaluation includes: evaluating abnormal data points in the performance curve, and / or evaluating abnormal performance curves of various preset performances; in response to evaluating the abnormal data points in the performance curve, evaluating the abnormal information of the performance curve using the normal distribution includes: calculating the data mean of the performance data contained in the performance curve; calculating the data standard deviation of the performance data in the performance curve based on the data mean; taking the sum of the data mean and the data standard deviation magnified by a first preset multiple as the first endpoint value, and taking the difference between the two as the second endpoint value, so as to form an allowable interval based on the first endpoint value and the second endpoint value; and taking the performance data in the performance curve that is not within the allowable interval as the first endpoint value. as abnormal information; in response to evaluating abnormal performance curves of various preset performances, using normal distribution to evaluate the performance curves for abnormal information includes: obtaining the average values ​​of each performance curve belonging to the current performance category respectively as the curve average values ​​of each performance curve; calculating the category curve average value and the category curve standard deviation of the current performance category based on the curve average value; taking the sum of the category curve average value and the category curve standard deviation magnified by a second preset multiple as the third endpoint value, and taking the difference between the two as the fourth endpoint value, so as to form an allowable interval based on the third endpoint value and the fourth endpoint value; taking the performance curve whose curve average value in the current performance category is not within the allowable interval as abnormal information.

[0010] In one embodiment of the present application, forming a curve chart belonging to each group of packages includes: calculating the ranking factor combination of each performance curve respectively; wherein the ranking factor includes at least one of the maximum value, minimum value, variance, mean value, median, and maximum and minimum values ​​of the performance curve; in response to obtaining a ranking instruction, identifying the target ranking factor indicated by the ranking instruction; wherein the target ranking factor is one of the ranking factor combination or the moment of the performance curve; traversing the performance curves of each group of packages, sorting them in a target order according to the target ranking factor, and forming a new plurality of curve charts of the sorted performance curves in a target order and a target number; wherein the target number is the number of performance curves in the specified new curve chart.

[0011] On the other hand, a data processing device is provided, which includes: an input module and a processing module; the input module is used to obtain performance data to be processed; the processing module is connected to the input module, and is used to implement a data processing method as in any of the above embodiments.

[0012] On the other hand, a data processing system is provided, which includes: a data server, a communication module and a data processing device as in the above embodiment; the data server is used to store performance data; the data processing device is connected to the data server through the communication module to perform data processing locally.

[0013] On the other hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: obtaining performance data to be processed; obtaining data grouping rules; grouping the performance data according to the data grouping rules to form a number of group packages; rendering a number of performance curves of the performance data in each group package respectively to form curve charts belonging to each group package; performing an abnormality evaluation on the performance curve, and in response to determining that abnormal information exists in the performance curve, assigning an abnormality identifier to the abnormal information.

[0014] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: obtaining performance data to be processed; obtaining data grouping rules; grouping the performance data according to the data grouping rules to form a number of group packages; rendering a number of performance curves of the performance data in each group package respectively to form a curve chart belonging to each group package; performing an abnormality evaluation on the performance curve, and in response to determining that abnormal information exists in the performance curve, assigning an abnormality mark to the abnormal information.

[0015] The above data processing method, data processing device, data processing system, computer equipment and computer-readable storage medium can constrain the number of performance curves in each curve chart, so as to help clarify the structure and trend of each performance curve, improve the intuitiveness of the curve chart, simplify the interface display, and also help to extract the efficiency of the information expressed by each performance curve. At the same time, it can also analyze the abnormal information and mark the abnormality to improve the information richness when the performance curve is used for data display and improve the efficiency of abnormal location based on the performance curve. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural diagram of an embodiment of a data processing system of the present application;

[0017] Figure 2 It is a structural schematic diagram of an embodiment of a data processing device of the present application;

[0018] Figure 3 It is a flowchart of an embodiment of the data processing method of the present application;

[0019] Figure 4 It is a flowchart of another embodiment of the data processing method of the present application;

[0020] Figure 5 This is a schematic diagram of the interface of an embodiment of the display page of this application;

[0021] Figure 6 This is a schematic diagram of the interface of another embodiment of the display page of the present application;

[0022] Figure 7 is a diagrammatic illustration of an embodiment of abnormal performance data annotation of the present application;

[0023] Figure 8 It is a diagrammatic illustration of an abnormal performance curve marking an embodiment of the present application;

[0024] Fig. 9 It is a flowchart of another embodiment of the data processing method of the present application;

[0025] Fig.10 It is a structural diagram of an embodiment of a computer device of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] In order to solve the technical problem in the related art that the rendering effect of curve charts is poor, which makes it difficult to extract effective information, the present application provides a data processing method, a data processing device, a data processing system, a computer device and a computer-readable storage medium. The data processing method includes obtaining performance data to be processed; obtaining data grouping rules; grouping the performance data according to the data grouping rules to form a number of group packages; rendering a number of performance curves of the performance data in each group package respectively to form curve charts belonging to each group package; performing an abnormal evaluation on the performance curve, and in response to determining that the performance curve has abnormal information, assigning an abnormal identifier to the abnormal information. The specific architecture and working principle of the present application are described in detail below.

[0028] See also Figure 1 , Figure 1 It is a structural diagram of an embodiment of a data processing system of the present application.

[0029] In this embodiment, the data processing system may include a data service end 10 , a communication module 20 and a data processing device 30 .

[0030] The data server 10 is used to store performance data.

[0031] The data processing device 30 is connected to the data service end 10 through the communication module 20 to obtain performance data from the data service end 10, and processes the performance data locally so that the performance data can be stored in the cloud such as a server. The specific data processing process is allocated to the local device, namely the data processing device 30, to reduce the computing burden of the data service end 10, which is beneficial to ensuring the performance of the data service end 10, and processing the data locally in the data processing device 30 is beneficial to improving processing efficiency, and is also beneficial to improving the efficiency of adjusting the response curve chart and the performance curve.

[0032] In an alternative embodiment, the performance data may be local to the data processing device 30, and the performance data to be processed may be obtained through bus communication or other means, that is, it may not be obtained through a network. Alternatively, the data processing device 30 and the data service end 10 may both be located on the server side, which is not limited here.

[0033] The specific structure and working principle of the data processing device are described in detail below.

[0034] See also Figure 2 , Figure 2 It is a structural diagram of an embodiment of a data processing device of the present application.

[0035] In this embodiment, the data processing device includes an input module 31 and a processing module 32 .

[0036] The input module 31 is used to communicate with the data service end of the data processing system so as to receive the performance data to be processed.

[0037] The processing module 32 is connected to the input module 31 and is used to process the performance data to be processed and render a curve chart.

[0038] Specifically, the processing module 32 can obtain performance data to be processed; obtain data grouping rules; group the performance data according to the data grouping rules to form a number of group packages; render a number of performance curves of the performance data in each group package respectively to form curve charts belonging to each group package respectively; perform an abnormality evaluation on the performance curve, and in response to determining that abnormal information exists in the performance curve, assign an abnormal identifier to the abnormal information.

[0039] It can be seen that the data processing device in this embodiment can restrict the number of performance curves in each curve chart, avoiding too many performance curves in the curve chart so that it is difficult to distinguish each performance curve, which can help to clarify the structure and trend of each performance curve, improve the intuitiveness of the curve chart, simplify the interface display, and also help to extract the efficiency of the information expressed by each performance curve. At the same time, it can also analyze the abnormal information and mark the abnormality to improve the information richness when the performance curve is used for data display and improve the efficiency of abnormal location based on the performance curve.

[0040] For the specific definition of the data processing device, please refer to the definition of the data processing method in the following text, which will not be repeated here. Each module in the above-mentioned data processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0041] See also Figure 3 , Figure 3 It is a flow chart of an embodiment of the data processing method of the present application. It should be noted that the current data processing method can be applied to distributed storage systems, server clusters, data centers, etc., and is not limited here.

[0042] S101: Obtain performance data to be processed; obtain data grouping rules.

[0043] In this embodiment, the performance data to be processed refers to data to be rendered to form a performance curve or a curve chart. The performance data to be processed may include performance data of several performance categories of several nodes. Optionally, the acquired performance data may be data with a label, or acquired in the form of a data list, which is not limited here.

[0044] As the name implies, the data grouping rule indicates the basis for grouping the performance data to be processed.

[0045] S102: Grouping the performance data according to data grouping rules to form a plurality of group packages.

[0046] In this embodiment, in response to obtaining the performance data to be processed and the data grouping rule, the performance data may be grouped according to the constraints of the data grouping rule to form a plurality of group packages. Each group package may include performance data forming a plurality of performance curves located on the same curve icon.

[0047] S103: Rendering a plurality of performance curves of performance data in each group of packages respectively, to form curve charts belonging to each group of packages respectively.

[0048] In this embodiment, rendering can be performed based on the performance data in each group of packages to obtain a curve chart in each group of packages, and each curve chart contains a plurality of performance curves.

[0049] S104: Perform an abnormality assessment on the performance curve, and in response to determining that abnormal information exists in the performance curve, assign an abnormality mark to the abnormal information.

[0050] In this embodiment, in response to obtaining the performance curve, the performance curve can be analyzed to evaluate the potential or current abnormalities of the node performance based on the performance curve. In response to determining that the performance curve has abnormal information, it can be considered that the node to which the performance curve belongs and its corresponding performance have potential abnormal risks or are in an abnormal state, and an abnormal identification can be given to the abnormal information to provide an abnormal reference for node operation and maintenance, and to facilitate rapid positioning of abnormal nodes and corresponding performance based on the curve chart.

[0051] Among them, the abnormal mark can be a display format, color, preset mark, bubble box, etc. that is different from other information, and is not limited here.

[0052] It can be seen that in this embodiment, the number of performance curves in each curve chart can be restricted to avoid too many performance curves in the curve chart so that it is difficult to distinguish each performance curve, which can help to clarify the structure and trend of each performance curve, improve the intuitiveness of the curve chart, simplify the interface display, and also help to extract the efficiency of the information expressed by each performance curve. At the same time, it can also analyze the abnormal information and mark the abnormality to improve the information richness when the performance curve is displayed for data and improve the efficiency of abnormal location.

[0053] See also Figure 4 , Figure 4 It is a flowchart of another embodiment of the data processing method of the present application.

[0054] In this embodiment, the data processing device can obtain the performance data to be processed from the database of the data service end. For example, the performance data list of the back end can be obtained through an https request. Among them, https (Hypertext Transfer Protocol Secure) is HTTP (Hypertext Transfer Protocol) with security as the goal. It ensures the security of the transmission process through transmission encryption and identity authentication on the basis of HTTP. HTTP is a request-response protocol, usually running on TCP (Transmission Control Protocol), which can specify the messages that the client may send to the server and the responses obtained.

[0055] The processing module may include an abnormal data processing module, a data preprocessing module and an interface rendering module.

[0056] When the performance data to be processed is obtained, each performance curve can be preprocessed by the data preprocessing module to provide the processed data to the data rendering module. Among them, the data preprocessing module can group the performance data, and each group of data includes a number of performance curves formed in a curve chart, so as to reduce the number of performance curves contained in each curve chart and improve the intuitiveness of the curve icon. In layman's terms, for example, it is agreed that 5 performance curves are displayed in each curve chart. If there are 20 performance curves in total, they can be divided into 4 curve charts for display, reducing the number of performance curves in each curve chart, so as to reduce the risk of affecting identification and observation when viewing the performance curves. Furthermore, the present embodiment can also support dragging of performance curves. In order to facilitate data comparison, the performance curves of the same type of curve charts after splitting can be dragged. The performance curve legend you want to move can be dragged to the target curve chart.

[0057] Specifically, the data grouping rule may include at least one of a curve quantity threshold sub-rule and a curve splitting sub-rule of the curve table.

[0058] For example, in response to the data grouping rule including a curve number threshold sub-rule of the curve table, the number of performance curves that can be rendered by the performance data divided into each group package is less than or equal to the curve number threshold. Wherein, the number of performance curves that can be rendered by the group package is allowed to be different. And / or,

[0059] In response to performance data belonging to a performance curve carrying a clone identifier, it is determined that the data grouping rule meets the curve clone sub-rule, the performance data carrying the clone identifier is used as the performance data to be cloned, and the performance data to be cloned is written into multiple target groups; when the group encapsulation of the multiple target groups is rendered into a chart, the performance data to be cloned is rendered to form a performance curve as a clone curve, so that the curve chart formed by the group encapsulation including the performance data to be cloned displays the clone curve.

[0060] In layman's terms, in this embodiment, the performance data can be packaged according to the curve number threshold of each curve chart input by the user on the interface. If the custom grouping function is enabled, the performance data can be packaged in groups according to the information of the custom grouping; if the curve clone function is enabled, the curve data after clone can be packaged into multiple groups respectively.

[0061] Furthermore, in this embodiment, data can be adaptively grouped and packaged according to the total number of performance curves to ensure that under the default configuration, performance curves / curve charts can be displayed simultaneously on the same display page, and all performance curves can be consulted without dragging the scroll bar.

[0062] Specifically, the data grouping rule includes an adaptive grouping number sub-rule. As such, the present embodiment can obtain the resolution height of the display page. The display page is used to display the performance curve. The height occupied by the fixed components in the display page can also be obtained, the height of the performance curve is evaluated, and the number of columns of the performance curve is arranged. The calculation formula for the adaptive grouping number can be shown as follows:

[0063] X=(L1-L2) / L3*nFormula 1-1

[0064] Among them, X represents the number of adaptive groups; L1 represents the resolution height of the display page; L2 represents the height occupied by fixed components in the display page; L3 represents the height of the performance curve; and n represents the number of columns of the performance curve.

[0065] Please continue reading Figure 4 In one embodiment, several ranking factors may be calculated for each performance curve, and the ranking factors may be packaged together as a ranking factor combination into a group package to provide a reference indicator for the subsequent display page custom ranking order, and to improve the efficiency of responding to the ranking instruction.

[0066] The ranking factor may include at least one of the maximum value, minimum value, variance, mean value, median, and maximum and minimum values ​​of the performance curve.

[0067] The calculation formula of the average value can be shown as follows:

[0068]

[0069] in, Indicates the average value of the performance curve; x i It represents the i-th performance data belonging to the performance curve, where i is a positive integer; n represents the number of performance data contained in the performance curve.

[0070] The calculation formula of variance can be shown as follows:

[0071]

[0072] Among them, σ 2 represents the variance of the performance curve; Indicates the average value of the performance curve; x i It represents the i-th performance data belonging to the performance curve, where i is a positive integer; n represents the number of performance data contained in the performance curve.

[0073] The median means that after the performance data included in the performance curve are sorted, if the length of the performance data is an odd number, the middle element is taken; if the length of the performance data is an even number, the average of the two middle elements is taken.

[0074] Specifically, the ranking factor combinations of each performance curve may be calculated respectively. In response to obtaining the ranking instruction, the target ranking factor indicated by the ranking instruction may be identified. The target ranking factor is one of the ranking factor combinations or the moment of the performance curve.

[0075] In this way, the performance curves of each group package can be traversed, sorted in a target order according to the target sorting factor, and the sorted performance curves are formed into a plurality of new curve charts in a target order and a target number, wherein the target number is the number of performance curves in the specified new curve chart.

[0076] like Figure 5 as well as Figure 6 As shown in the example, Figure 5 This is a schematic diagram of the interface of an embodiment of the display page of this application. Figure 6 This is an interface diagram of another embodiment of the display page of this application. Figure 5 The example shows the curve chart and display page formed according to the data grouping rules. Figure 6 An example is shown of a curve chart and a display page that are sorted according to the sorting factor indicated by the sorting instruction.

[0077] Please continue reading Figure 4 , Figure 7 as well as Figure 8 , Figure 7 is a diagrammatic illustration of an embodiment of abnormal performance data annotation of the present application, Figure 8 It is a diagrammatic illustration of an abnormal performance curve marking an embodiment of the present application.

[0078] In one embodiment, the abnormal data processing module is used to evaluate whether the performance curve is abnormal / whether there is abnormal data in the performance curve. The abnormality evaluation can be performed on the performance curve that has been drawn, or the performance data belonging to each performance curve can be evaluated for abnormality.

[0079] In this embodiment, in response to turning on the abnormal data monitoring function, abnormal data can be identified and marked for display, and analysis strategies can be designed to adapt to various performance categories, i.e., the analysis categories in this embodiment, to improve the reliability of abnormal analysis and thus improve the reliability of abnormal information.

[0080] The analysis category may include at least one of a threshold-type statistical category, an incremental-type statistical category, and a discrete statistical category.

[0081] Further, in response to determining that there is abnormal information, the performance category to which the abnormal information belongs can be obtained and the cause of the abnormal information can be analyzed. Find whether the preset cause of the performance category in the preset abnormality library includes the current cause. In response to the presence of a preset cause matching the current cause, obtain the repair method associated with the preset cause matching the current cause, and execute the repair method to achieve automatic repair. In response to the absence of a preset cause matching the current cause, write the current cause into the preset abnormality library, record the process of repairing the abnormality caused by the current cause, and write it into the preset abnormality library as the repair method associated with the current cause.

[0082] The specific working principle of this embodiment is described below with examples.

[0083] The performance categories of the adaptation threshold class statistical categories may include: one or more of: CPU (Central Processing Unit) utilization, memory utilization, CPU temperature, motherboard temperature, latency, task queue length, network port utilization, hard disk busyness, IO (Input / Output) response time, data reconstruction speed, average load, etc.

[0084] The performance categories adapted to the incremental statistical categories may include: one or more of: hard disk utilization, capacity increment, total packet loss, total error packet number, etc.

[0085] The performance categories adapted to the discrete statistical categories may include: one or more of: IOPS (Input / Output Operations PerSecond, the number of read and write operations per second), bandwidth, network port flow rate, data throughput, IO size, etc.

[0086] For example, the following table illustrates the relationship between analysis categories and performance categories:

[0087] Table 1 Correlation table between analysis categories and performance categories

[0088]

[0089]

[0090] Specifically, in this embodiment, the analysis category of the performance curve may be identified.

[0091] In response to the analysis category of the performance curve being the threshold statistical category, the performance data of the performance curve can be traversed, and the performance data exceeding the first threshold can be used as the abnormal information. In layman's terms, for a performance curve adapted to the threshold statistical category, each point of the performance curve can be traversed, and then all points exceeding the threshold range can be marked.

[0092] In response to the analysis category of the performance curve being the incremental statistical category, the performance curve can be fitted, and the fitted performance curve can be differentiated, and the performance data whose increase rate exceeds the second threshold value can be used as the abnormal information. In layman's terms, the capacity increment can usually be fitted as a straight line with a positive slope, and its slope is the speed of data increment. For such a curve, the least squares method is used for curve fitting, and then the curve is differentiated to find the points where the increase rate is greater than the threshold and mark them.

[0093] In response to the analysis category of the performance curve being the discrete statistical category, the normal distribution can be used to evaluate abnormal information of the performance curve. In layman's terms, for data transmission statistical items adapted to discrete statistical categories, the transmission rate is generally fitted as a straight line parallel to the horizontal axis, and the data points are discretely distributed above and below the straight line. For such performance curves, the normal distribution model is used in this embodiment, and data points exceeding 3 times the standard deviation are marked.

[0094] Furthermore, the abnormality evaluation includes: evaluating abnormal data points in the performance curve, and / or evaluating abnormal performance curves of various preset performances.

[0095] In response to evaluating the abnormal data point in the performance curve, a data average of the performance data included in the performance curve can be calculated. The data standard deviation of the performance data in the performance curve is calculated based on the data average. The sum of the data average and the data standard deviation amplified by a first preset multiple is used as a first endpoint value, and the difference between the two is used as a second endpoint value, so as to form an allowable interval based on the first endpoint value and the second endpoint value. The performance data in the performance curve that is not in the allowable interval is used as abnormal information.

[0096] Among them, the calculation formulas for normal distribution, mean value and standard deviation can be shown as follows:

[0097]

[0098] Where f(x) represents the probability distribution of the random variable x under normal distribution with a location parameter μ and a scale parameter σ; σ represents the standard deviation of the performance curve; Indicates the average value of the performance curve; x i It represents the i-th performance data belonging to the performance curve, where i is a positive integer; n represents the number of performance data contained in the performance curve.

[0099] In response to the abnormal performance curves of the various preset performances evaluated, the average values ​​of the performance curves belonging to the current performance category can be obtained respectively as the curve average values ​​of the performance curves. The category curve average value and the category curve standard deviation of the current performance category are calculated based on the curve average value. The sum of the category curve average value and the category curve standard deviation amplified by a second preset multiple is taken as the third endpoint value, and the difference between the two is taken as the fourth endpoint value, so as to form an allowable interval based on the third endpoint value and the fourth endpoint value; the performance curve in the current performance category whose curve average value is not within the allowable interval is taken as abnormal information.

[0100] That is to say, the normal distribution model is used for the average value of the same performance curve. If the average value of a performance curve exceeds 3 times the standard deviation, the entire curve is considered abnormal. The entire curve is marked to facilitate identification of the object of the curve and determine whether it has a fault.

[0101] Please continue reading Figure 4 In one embodiment, the interface rendering module is implemented based on visualization components, etc. It can render the performance data provided by the input module and the data processing module to the display page, and can monitor the operations from the display interface in real time, such as manual grouping of curves, curve clones, curve dragging, sorting instructions, etc., and pass the monitoring results to the data processing module for processing and then render them to the display interface again. According to the reporting results of the abnormal data processing module, the abnormal points and abnormal curves are rendered.

[0102] The following is a brief example of the complete working principle of this application. Fig. 9 , Fig. 9 It is a flowchart of another embodiment of the data processing method of the present application.

[0103] S201: Obtain performance data to be processed.

[0104] S202: Group the performance data and calculate the maximum value, minimum value, variance, mean value, median, and maximum and minimum values ​​of each performance curve.

[0105] S203: Abnormal performance data monitoring, evaluating abnormal data points and / or abnormal curves.

[0106] S204: Display interface rendering.

[0107] S205: Determine whether to obtain an operation instruction.

[0108] In this embodiment, when it is determined that an operation instruction is obtained, step S202 is executed to adjust the data analysis of the performance data or sort the performance curve based on the operation instruction; when it is determined that no operation instruction is obtained, step S205 is executed or the process ends.

[0109] In summary, this application can simplify the interface display, make the number of curves in each chart controllable, reduce the mutual influence between curves, and make the interface display more concise and beautiful. It provides usability-enhancing functions such as curve dragging, curve clones, custom grouping, and sorting, which facilitates monitoring data comparison and makes it easier to find problems and anomalies. It provides an automatic abnormal data identification function and marks possible abnormal data, which facilitates the rapid discovery of abnormal monitoring data, thereby improving the effectiveness of interface monitoring.

[0110] It should be understood that although Figure 3-Figure 4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 3-Figure 4 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0111] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps of the data processing method described above are implemented, which will not be described in detail here.

[0112] See also Fig.10 , Fig.10 It is a structural diagram of an embodiment of a computer device of the present application.

[0113] In one embodiment, the computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10 As shown in the example.

[0114] The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus.

[0115] Among them, the processor of the computer device is used to provide computing and control capabilities.

[0116] The memory of the computer device includes a non-volatile storage medium and an internal memory.

[0117] The non-volatile storage medium stores an operating system and a computer program.

[0118] The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.

[0119] The network interface of the computer device is used to communicate with an external terminal via a network connection.

[0120] When the computer program is executed by a processor, a data processing method is implemented.

[0121] The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the display page for displaying the curve chart mentioned in the above embodiment may be displayed on the display screen.

[0122] The input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0123] Those skilled in the art will understand that Fig.10 The structure shown as an example is merely a block diagram of a portion of the structure related to the present application solution, and does not constitute a limitation on the computer device to which the present application solution is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0124] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0125] S101: Obtain performance data to be processed; obtain data grouping rules.

[0126] S102: Grouping the performance data according to data grouping rules to form a plurality of group packages.

[0127] S103: Rendering a plurality of performance curves of performance data in each group of packages respectively, to form curve charts belonging to each group of packages respectively.

[0128] S104: Perform an abnormality assessment on the performance curve, and in response to determining that abnormal information exists in the performance curve, assign an abnormality mark to the abnormal information.

[0129] In one embodiment, the data grouping rule includes at least one of a curve number threshold sub-rule and a curve clone sub-rule of a curve chart; when the performance data is grouped according to the data grouping rule to form a plurality of group packages, the processor further implements the following steps when executing the computer program: in response to the data grouping rule including the curve number threshold sub-rule of the curve chart, the number of performance curves that can be rendered by the performance data divided into each group package is less than or equal to the curve number threshold; wherein, differences in the number of performance curves that can be rendered by the group package are allowed; in response to the performance data belonging to a performance curve carrying a clone identifier, it is determined that the data grouping rule meets the curve clone sub-rule, the performance data carrying the clone identifier is used as the performance data to be cloned, and the performance data to be cloned is written into multiple target groups; when the group packages of the multiple target groups are rendered as charts, the performance data to be cloned are all rendered to form performance curves as clone curves, so that the curve chart formed by the group package including the performance data to be cloned displays the clone curve.

[0130] In one embodiment, the data grouping rule includes an adaptive grouping number sub-rule; when obtaining the data grouping rule, the processor also implements the following steps when executing the computer program: obtaining the resolution height of the display page; wherein the display page is used to display the performance curve; obtaining the height occupied by the fixed components in the display page; evaluating the height of the performance curve and the number of columns of the performance curve; the calculation formula for the adaptive grouping number is: X=(L1-L2) / L3*n; wherein X represents the adaptive grouping number; L1 represents the resolution height of the display page; L2 represents the height occupied by the fixed components in the display page; L3 represents the height of the performance curve; and n represents the number of columns of the performance curve.

[0131] In one embodiment, when performing an abnormality evaluation on a performance curve, the processor also implements the following steps when executing a computer program: identifying an analysis category of the performance curve; wherein the analysis category includes at least one of a threshold statistical category, an incremental statistical category, and a discrete statistical category; in response to the analysis category of the performance curve being a threshold statistical category, traversing the performance data of the performance curve, and taking performance data exceeding a first threshold as abnormal information; in response to the analysis category of the performance curve being an incremental statistical category, fitting the performance curve, and deriving the fitted performance curve, and taking performance data whose increasing rate exceeds a second threshold as abnormal information; in response to the analysis category of the performance curve being a discrete statistical category, using a normal distribution to evaluate the performance curve for abnormal information.

[0132] In one embodiment, the abnormality evaluation includes: evaluating abnormal data points in a performance curve, and / or evaluating abnormal performance curves of various preset performances; in response to evaluating abnormal data points in the performance curve, when the performance curve is evaluated for abnormal information using a normal distribution, the processor further implements the following steps when executing the computer program: calculating a data average value of the performance data included in the performance curve; calculating a data standard deviation of the performance data in the performance curve based on the data average value; taking the sum of the data average value and the data standard deviation magnified by a first preset multiple as a first endpoint value, and taking the difference between the two as a second endpoint value, so as to form an allowable interval based on the first endpoint value and the second endpoint value; taking the performance data in the performance curve that is not in the allowable interval as Abnormal information; in response to evaluating abnormal performance curves of various preset performances, when the normal distribution is used to evaluate the abnormal information of the performance curves, the processor also implements the following steps when executing the computer program: respectively obtaining the average values ​​of each performance curve belonging to the current performance category as the curve average value of each performance curve; calculating the category curve average value and the category curve standard deviation of the current performance category based on the curve average value; taking the sum of the category curve average value and the category curve standard deviation magnified by a second preset multiple as the third endpoint value, and taking the difference between the two as the fourth endpoint value, so as to form an allowable interval based on the third endpoint value and the fourth endpoint value; taking the performance curve whose curve average value in the current performance category is not within the allowable interval as abnormal information.

[0133] In one embodiment, when forming curve charts belonging to each group of packages, the processor also implements the following steps when executing the computer program: respectively calculating the ranking factor combination of each performance curve; wherein the ranking factor includes at least one of the maximum value, minimum value, variance, mean value, median, and maximum and minimum values ​​of the performance curve; in response to obtaining a sorting instruction, identifying the target ranking factor indicated by the sorting instruction; wherein the target ranking factor is one of the ranking factor combinations or the moment of the performance curve; traversing the performance curves of each group of packages, sorting them in a target order according to the target ranking factors, and forming a new plurality of curve charts with the sorted performance curves in a target order and a target number; wherein the target number is the number of performance curves in the specified new curve chart.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0135] S101: Obtain performance data to be processed; obtain data grouping rules.

[0136] S102: Grouping the performance data according to data grouping rules to form a plurality of group packages.

[0137] S103: Rendering a plurality of performance curves of performance data in each group of packages respectively, to form curve charts belonging to each group of packages respectively.

[0138] S104: Perform an abnormality assessment on the performance curve, and in response to determining that abnormal information exists in the performance curve, assign an abnormality mark to the abnormal information.

[0139] In one embodiment, the data grouping rule includes at least one of a curve number threshold sub-rule and a curve clone sub-rule of a curve chart; when the performance data is grouped according to the data grouping rule to form a plurality of group packages, the computer program further implements the following steps when executed by the processor: in response to the data grouping rule including the curve number threshold sub-rule of the curve chart, the number of performance curves that can be rendered by the performance data divided into each group package is less than or equal to the curve number threshold; wherein, differences in the number of performance curves that can be rendered by the group package are allowed; in response to the performance data belonging to a performance curve carrying a clone identifier, it is determined that the data grouping rule meets the curve clone sub-rule, the performance data carrying the clone identifier is used as the performance data to be cloned, and the performance data to be cloned is written into multiple target groups; when the group packages of the multiple target groups are rendered as charts, the performance data to be cloned are all rendered to form performance curves as clone curves, so that the curve chart formed by the group package including the performance data to be cloned displays the clone curve.

[0140] In one embodiment, the data grouping rule includes an adaptive grouping number sub-rule; when obtaining the data grouping rule, the computer program also implements the following steps when executed by the processor: obtaining the resolution height of the display page; wherein the display page is used to display the performance curve; obtaining the height occupied by the fixed components in the display page; evaluating the height of the performance curve and the number of columns of the performance curve; the calculation formula for the adaptive grouping number is: X=(L1-L2) / L3*n; wherein X represents the adaptive grouping number; L1 represents the resolution height of the display page; L2 represents the height occupied by the fixed components in the display page; L3 represents the height of the performance curve; and n represents the number of columns of the performance curve.

[0141] In one embodiment, when the performance curve is evaluated for abnormality, the computer program also implements the following steps when executed by the processor: identifying the analysis category of the performance curve; wherein the analysis category includes at least one of a threshold statistical category, an incremental statistical category, and a discrete statistical category; in response to the analysis category of the performance curve being a threshold statistical category, traversing the performance data of the performance curve, and taking the performance data exceeding a first threshold as abnormal information; in response to the analysis category of the performance curve being an incremental statistical category, fitting the performance curve, and derivatizing the fitted performance curve, and taking the performance data whose increasing rate exceeds a second threshold as abnormal information; in response to the analysis category of the performance curve being a discrete statistical category, using a normal distribution to evaluate the performance curve for abnormal information.

[0142] In one embodiment, the abnormality evaluation includes: evaluating abnormal data points in a performance curve, and / or evaluating abnormal performance curves of various preset performances; in response to evaluating abnormal data points in the performance curve, when the performance curve is evaluated for abnormal information using a normal distribution, the computer program further implements the following steps when executed by the processor: calculating a data mean value of the performance data contained in the performance curve; calculating a data standard deviation of the performance data in the performance curve based on the data mean value; taking the sum of the data mean value and the data standard deviation magnified by a first preset multiple as a first endpoint value, and taking the difference between the two as a second endpoint value, so as to form an allowable interval based on the first endpoint value and the second endpoint value; taking the performance data in the performance curve that is not in the allowable interval as Abnormal information; in response to evaluating abnormal performance curves of various preset performances, when the performance curves are evaluated for abnormal information using normal distribution, the computer program also implements the following steps when executed by the processor: respectively obtaining the average values ​​of each performance curve belonging to the current performance category as the curve average value of each performance curve; calculating the category curve average value and the category curve standard deviation of the current performance category based on the curve average value; taking the sum of the category curve average value and the category curve standard deviation magnified by a second preset multiple as the third endpoint value, and taking the difference between the two as the fourth endpoint value, so as to form an allowable interval based on the third endpoint value and the fourth endpoint value; taking the performance curve whose curve average value in the current performance category is not within the allowable interval as abnormal information.

[0143] In one embodiment, when forming curve charts respectively belonging to each group of packages, the computer program also implements the following steps when executed by the processor: respectively calculate the ranking factor combination of each performance curve; wherein the ranking factor includes at least one of the maximum value, minimum value, variance, mean value, median, and maximum and minimum values ​​of the performance curve; in response to obtaining a sorting instruction, identify the target sorting factor indicated by the sorting instruction; wherein the target sorting factor is one of the ranking factor combinations or the moment of the performance curve; traverse the performance curves of each group of packages, sort them in a target order according to the target sorting factor, and form a new plurality of curve charts with the sorted performance curves in a target order and a target number; wherein the target number is the number of performance curves in the specified new curve chart.

[0144] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A data processing method, characterized in that: The data processing method comprises: Obtain performance data to be processed; obtain data grouping rules; Grouping the performance data according to the data grouping rule to form a plurality of group packages; Respectively rendering a plurality of performance curves of the performance data in each group of packages to form curve charts belonging to each group of packages; The performance curve is evaluated for abnormality, and in response to determining that abnormal information exists in the performance curve, an abnormal identification is assigned to the abnormal information.

2. The data processing method according to claim 1, characterized in that: The data grouping rule includes at least one of a curve quantity threshold sub-rule and a curve splitting sub-rule of the curve table; The step of grouping the performance data according to the data grouping rule to form a plurality of group packages includes: In response to the data grouping rule including a curve quantity threshold sub-rule of the curve table, the number of performance curves that can be rendered by the performance data divided into each of the group packages is less than or equal to the curve quantity threshold; wherein the groups are allowed to have differences in the number of renderable performance curves encapsulated; In response to performance data belonging to a performance curve carrying a clone identifier, it is determined that the data grouping rule meets the curve clone sub-rule, the performance data carrying the clone identifier is used as the performance data to be cloned, and the performance data to be cloned is written into multiple target groups; when the group encapsulation of the multiple target groups is rendered as a chart, the performance curve formed by the performance data to be cloned is rendered as a clone curve, so that the curve chart formed by the group encapsulation including the performance data to be cloned displays the clone curve.

3. The data processing method according to claim 1 or 2, characterized in that: The data grouping rule includes an adaptive grouping quantity sub-rule; The data grouping rules include: Obtaining a resolution height of a display page; wherein the display page is used to display the performance curve; Obtaining the height occupied by the fixed components in the display page; evaluating the height of the performance curve and the number of columns of the performance curve; The formula for calculating the number of adaptive groups is: X=(L1-L2) / L3*n Among them, X represents the number of adaptive groups; L1 represents the resolution height of the display page; L2 represents the height occupied by fixed components in the display page; L3 represents the height of the performance curve; and n represents the number of columns of the performance curve.

4. The data processing method according to claim 1, characterized in that: The abnormality evaluation of the performance curve comprises: Identify the analysis category of the performance curve; wherein the analysis category includes at least one of a threshold-based statistical category, an incremental-based statistical category, and a discrete statistical category; In response to the analysis category of the performance curve being the threshold-based statistical category, traversing the performance data of the performance curve, and taking the performance data exceeding a first threshold as the abnormal information; In response to the analysis category of the performance curve being the incremental statistical category, fitting the performance curve, and deriving the fitted performance curve, and taking the performance data whose increasing speed exceeds a second threshold as the abnormal information; In response to the analysis category of the performance curve being the discrete statistics category, abnormal information evaluation is performed on the performance curve using normal distribution.

5. The data processing method according to claim 4, characterized in that: The abnormality evaluation includes: evaluating abnormal data points in the performance curve, and / or evaluating abnormal performance curves of various preset performances; In response to evaluating the abnormal data point in the performance curve, evaluating the performance curve for abnormal information using a normal distribution includes: Calculating the data average of the performance data included in the performance curve; Calculate the data standard deviation of the performance data in the performance curve based on the data average value; The sum of the data average value and the data standard deviation amplified by a first preset multiple is used as a first endpoint value, and the difference between the two is used as a second endpoint value, so as to form an allowable interval based on the first endpoint value and the second endpoint value; and the performance data in the performance curve that is not in the allowable interval is used as abnormal information; In response to the abnormal performance curves of evaluating various preset performances, the evaluating the abnormal information of the performance curves using normal distribution includes: Respectively obtaining average values ​​of each performance curve belonging to the current performance category as curve average values ​​of each performance curve; Calculate the category curve average value and category curve standard deviation of the current performance category based on the curve average value; The sum of the category curve average value and the category curve standard deviation magnified by a second preset multiple is taken as the third endpoint value, and the difference between the two is taken as the fourth endpoint value, so as to form an allowable range based on the third endpoint value and the fourth endpoint value; the performance curve in the current performance category whose curve average value is not within the allowable range is taken as abnormal information.

6. The data processing method according to claim 1, characterized in that: The forming of the curve charts belonging to each group of packages includes: Calculate the ranking factor combination of each performance curve respectively; wherein the ranking factor includes at least one of the maximum value, minimum value, variance, mean value, median, and maximum and minimum value of the performance curve; In response to obtaining a sorting instruction, identifying a target sorting factor indicated by the sorting instruction; wherein the target sorting factor is one of the sorting factor combinations or a moment of the performance curve; The performance curves of each group package are traversed, and are sorted in a target order according to the target sorting factor, and the sorted performance curves are formed into a plurality of new curve charts in the target order and with a target number; wherein the target number is the number of performance curves in the specified new curve chart.

7. A data processing device, characterized in that: The data processing device comprises: An input module, used to obtain performance data to be processed; A processing module, connected to the input module, is used to implement the data processing method as claimed in any one of claims 1 to 6.

8. A data processing system, characterized in that: The data processing system comprises: Data server, used to store performance data; Communication module; The data processing device as described in claim 7 is connected to the data server through the communication module to perform data processing locally.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the data processing method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 6 are implemented.

Images