Data processing method, device, system, computer device and readable storage medium

Abstract

The application relates to the computer technical field, in particular to a data processing method, device and system, computer equipment and readable storage medium. The data processing method comprises the following steps: obtaining performance data to be processed; obtaining a data grouping rule; grouping the performance data according to the data grouping rule to form a plurality of group encapsulations; respectively rendering a plurality of performance curves of the performance data in each group encapsulation to form a curve chart table respectively belonging to each group encapsulation; performing abnormality evaluation on the performance curves, and giving an abnormality mark to abnormal information in response to determining that the performance curves have the abnormal information. The method can simplify interface display and is beneficial to enriching information of data display.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, a data processing apparatus, a data processing system, a computer device, and a computer-readable storage medium. Background Art

[0002] As the cluster size continues to expand, the resources available for performance management also increase, which results in very cumbersome performance report data for real-time monitoring.

[0003] However, the size of the display pages and graphs makes it difficult to quickly and accurately find the performance data you need from the massive amount of statistical data. Furthermore, for example, when monitoring node performance data, at least one curve is typically used to represent the performance changes of a node. As the number of objects for performance statistics increases, the number of curves within a single graph increases, making it difficult to identify the graphs, which in turn makes it difficult to quickly extract useful information. Summary of the Invention

[0004] Based on this, it is necessary to provide a data processing method, a data processing device, a data processing system, a computer device and a computer-readable storage medium to address the above technical problems, which can simplify the interface display and enrich the information displayed by the data.

[0005] On the one hand, a data processing method is provided, which includes: obtaining performance data to be processed; obtaining data grouping rules; grouping the performance data according to the data grouping rules to form a number of group packages; rendering a number of performance curves of the performance data in each group package respectively to form a curve chart belonging to each group package; performing an abnormality evaluation on the performance curve, and in response to determining that abnormal information exists in the performance curve, assigning an abnormality identifier to the abnormal information.

[0006] In one embodiment of the present application, the data grouping rule includes at least one of a curve number threshold sub-rule and a curve clone sub-rule of a curve chart; grouping the performance data according to the data grouping rule to form a number of group packages includes: in response to the data grouping rule including the curve number threshold sub-rule of the curve chart, the number of performance curves that can be rendered by the performance data divided into each group package is less than or equal to the curve number threshold; wherein, differences in the number of performance curves that can be rendered by the group package are allowed; in response to the performance data belonging to a performance curve carrying a clone identifier, determining that the data grouping rule meets the curve clone sub-rule, taking the performance data carrying the clone identifier as the performance data to be cloned, and writing the performance data to be cloned into multiple target groups; when rendering the group packages of multiple target groups, all the performance data to be cloned are rendered to form performance curves as clone curves, so that the curve chart formed by the group package including the performance data to be cloned displays the clone curve.

[0007] In one embodiment of the present application, the data grouping rule includes an adaptive grouping number sub-rule; obtaining the data grouping rule includes: obtaining the resolution height of the display page; wherein the display page is used to display the performance curve; obtaining the height occupied by the fixed components in the display page; evaluating the height of the performance curve and the number of columns of the performance curve; the calculation formula for the adaptive grouping number is: X = (L1-L2) / L3*n; wherein X represents the adaptive grouping number; L1 represents the resolution height of the display page; L2 represents the height occupied by the fixed components in the display page; L3 represents the height of the performance curve; n represents the number of columns of the performance curve.

[0008] In one embodiment of the present application, performing an abnormality evaluation on a performance curve includes: identifying an analysis category of the performance curve; wherein the analysis category includes at least one of a threshold statistical category, an incremental statistical category, and a discrete statistical category; in response to the analysis category of the performance curve being a threshold statistical category, traversing performance data of the performance curve, and taking performance data exceeding a first threshold as abnormal information; in response to the analysis category of the performance curve being an incremental statistical category, fitting the performance curve, and differentiating the fitted performance curve, and taking performance data whose increasing rate exceeds a second threshold as abnormal information; in response to the analysis category of the performance curve being a discrete statistical category, using a normal distribution to evaluate the performance curve for abnormal information.

[0009] In one embodiment of the present application, the abnormality evaluation includes: evaluating abnormal data points in the performance curve, and / or evaluating abnormal performance curves of various preset performances; in response to evaluating abnormal data points in the performance curve, using the normal distribution to evaluate abnormal information of the performance curve includes: calculating the data average of the performance data contained in the performance curve; calculating the data standard deviation of the performance data in the performance curve based on the data average; taking the sum of the data average and the data standard deviation magnified by a first preset multiple as the first endpoint value, and taking the difference between the two as the second endpoint value, so as to form an allowable interval based on the first endpoint value and the second endpoint value; and taking the performance data in the performance curve that is not within the allowable interval as the first endpoint value. as abnormal information; in response to evaluating abnormal performance curves of various preset performances, using normal distribution to evaluate the performance curves for abnormal information includes: respectively obtaining the average values ​​of each performance curve belonging to the current performance category as the curve average values ​​of each performance curve; calculating the category curve average value and the category curve standard deviation of the current performance category based on the curve average values; taking the sum of the category curve average value and the category curve standard deviation magnified by a second preset multiple as the third endpoint value, and taking the difference between the two as the fourth endpoint value, so as to form an allowable range based on the third endpoint value and the fourth endpoint value; taking the performance curve whose curve average value in the current performance category is not within the allowable range as abnormal information.

[0010] In one embodiment of the present application, forming a curve chart belonging to each group of packages includes: calculating the ranking factor combination of each performance curve respectively; wherein the ranking factor includes at least one of the maximum value, minimum value, variance, mean value, median, and maximum and minimum values ​​of the performance curve; in response to obtaining a ranking instruction, identifying the target ranking factor indicated by the ranking instruction; wherein the target ranking factor is one of the ranking factor combination or the moment of the performance curve; traversing the performance curves of each group of packages, sorting them in a target order according to the target ranking factor, and forming a new plurality of curve charts of the sorted performance curves in a target order and a target number; wherein the target number specifies the number of performance curves in the new curve chart.

[0011] On the other hand, a data processing device is provided, which includes: an input module and a processing module; the input module is used to obtain performance data to be processed; the processing module is connected to the input module to implement the data processing method in any of the above embodiments.

[0012] On the other hand, a data processing system is provided, which includes: a data service end, a communication module and a data processing device as in the above embodiment; the data service end is used to store performance data; the data processing device is connected to the data service end through the communication module to perform data processing locally.

[0013] On the other hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: obtaining performance data to be processed; obtaining data grouping rules; grouping the performance data according to the data grouping rules to form a plurality of group packages; rendering a plurality of performance curves of the performance data in each group package respectively to form a curve chart belonging to each group package; performing an abnormality evaluation on the performance curve, and assigning an abnormality identifier to the abnormal information in response to determining that abnormal information exists in the performance curve.

[0014] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining performance data to be processed; obtaining data grouping rules; grouping the performance data according to the data grouping rules to form a number of group packages; rendering a number of performance curves of the performance data in each group package respectively to form a curve chart belonging to each group package; performing an abnormality evaluation on the performance curve, and in response to determining that abnormal information exists in the performance curve, assigning an abnormality identifier to the abnormal information.

[0015] The aforementioned data processing method, data processing apparatus, data processing system, computer device, and computer-readable storage medium can constrain the number of performance curves within each graph, thereby clarifying the structure and trends of each performance curve, improving the intuitiveness of the graph, simplifying the interface display, and facilitating efficient extraction of the information expressed by each performance curve. Furthermore, they can analyze and identify anomalies, thereby increasing the information richness of performance curve data display and improving the efficiency of anomaly location based on performance curves. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural diagram of an embodiment of the data processing system of the present application;

[0017] Figure 2 This is a structural diagram of an embodiment of a data processing device of the present application;

[0018] Figure 3 This is a flow chart of an embodiment of the data processing method of the present application;

[0019] Figure 4 This is a flow chart of another embodiment of the data processing method of the present application;

[0020] Figure 5 This is a schematic diagram of an interface of an embodiment of a display page of this application;

[0021] Figure 6 This is a schematic diagram of the interface of another embodiment of the display page of this application;

[0022] Figure 7 is a diagrammatical representation of an embodiment of the data processing method of the present application;

[0023] Figure 8 is a diagrammatical representation of an embodiment of the data processing method of the present application;

[0024] Figure 9 is a flow chart of another embodiment of the data processing method of the present application;

[0025] Figure 10 is a structural schematic diagram of an embodiment of the computer device of the present application. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0027] In order to solve the technical problem of poor rendering effect of curve charts in the related art, which leads to difficulty in extracting effective information, the present application provides a data processing method, a data processing device, a data processing system, a computer device and a computer readable storage medium. The data processing method comprises: obtaining performance data to be processed; obtaining a data grouping rule; grouping the performance data according to the data grouping rule to form a plurality of group encapsulations; rendering a plurality of performance curves of the performance data in each group encapsulation respectively to form a curve chart respectively belonging to each group encapsulation; performing abnormal evaluation on the performance curves, and giving an abnormal identifier to abnormal information in response to determining that the performance curves have abnormal information. The specific architecture and working principle of the present application will be described in detail below.

[0028] Please refer to Figure 1 , Figure 1 is a structural schematic diagram of an embodiment of the data processing system of the present application.

[0029] In the present embodiment, the data processing system can comprise a data server 10, a communication module 20 and a data processing device 30.

[0030] The data server 10 is used to store performance data.

[0031] The data processing device 30 is connected to the data service end 10 through the communication module 20 to obtain performance data from the data service end 10 and process the performance data locally so that the performance data can be stored in the cloud such as a server. The specific data processing process is allocated to the local device, namely the data processing device 30, to reduce the computing burden of the data service end 10, which is beneficial to ensuring the performance of the data service end 10, and processing the data locally in the data processing device 30 is beneficial to improving processing efficiency, and is also beneficial to improving the efficiency of response curve charts and performance curve adjustments.

[0032] In an alternative embodiment, the performance data can be stored locally on the data processing device 30, and the performance data to be processed can be obtained through bus communication or other means, that is, it can be obtained without going through the network. Alternatively, the data processing device 30 and the data server 10 can both be located on the server side, which is not limited here.

[0033] The specific structure and working principle of the data processing device are described in detail below.

[0034] See also Figure 2 , Figure 2 It is a structural diagram of an embodiment of the data processing device of the present application.

[0035] In this embodiment, the data processing device includes an input module 31 and a processing module 32 .

[0036] The input module 31 is used to communicate with the data service end of the data processing system so as to receive the performance data to be processed.

[0037] The processing module 32 is connected to the input module 31 and is used to process the performance data to be processed and render a curve chart.

[0038] Specifically, the processing module 32 can obtain the performance data to be processed; obtain the data grouping rules; group the performance data according to the data grouping rules to form several group packages; render several performance curves of the performance data in each group package respectively to form curve charts belonging to each group package; perform an abnormality evaluation on the performance curve, and in response to determining that there is abnormal information in the performance curve, assign an abnormality identifier to the abnormal information.

[0039] As can be seen, the data processing device in this embodiment can constrain the number of performance curves within each graph, preventing the graph from containing too many performance curves, making it difficult to distinguish between them. This helps clarify the structure and trends of each performance curve, improves the intuitiveness of the graph, simplifies the interface display, and facilitates the efficient extraction of the information expressed by each performance curve. Furthermore, it can analyze and identify anomalies, thereby increasing the information richness of the performance curve data display and improving the efficiency of anomaly location based on the performance curve.

[0040] For the specific definition of the data processing device, please refer to the definition of the data processing method below and will not be repeated here. Each module in the above-mentioned data processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software so that the processor can call and execute the operations corresponding to each of the above modules.

[0041] See also Figure 3 , Figure 3 It should be noted that the current data processing method can be applied to distributed storage systems, server clusters, data centers, etc., and is not limited here.

[0042] S101: Acquire performance data to be processed; acquire data grouping rules.

[0043] In this embodiment, the performance data to be processed represents data to be rendered into a performance curve or graph. The performance data to be processed may include performance data of several performance categories for several nodes. Optionally, the acquired performance data may be tagged data or acquired in the form of a data list, which is not limited here.

[0044] As the name suggests, the data grouping rule indicates the basis for grouping the performance data to be processed.

[0045] S102: Grouping the performance data according to data grouping rules to form a plurality of group packages.

[0046] In this embodiment, in response to obtaining the performance data to be processed and the data grouping rules, the performance data can be grouped according to the constraints of the data grouping rules to form a plurality of group packages. Each group package can include performance data forming a plurality of performance curves located on the same curve icon.

[0047] S103: Rendering a plurality of performance curves of the performance data in each group of packages respectively to form curve charts belonging to each group of packages.

[0048] In this embodiment, rendering can be performed based on the performance data in each group of packages to obtain a curve chart in each group of packages, and each curve chart contains several performance curves.

[0049] S104: Perform an abnormality assessment on the performance curve, and in response to determining that abnormal information exists in the performance curve, assign an abnormality identifier to the abnormal information.

[0050] In this embodiment, in response to obtaining a performance curve, the performance curve can be analyzed to assess potential or current abnormalities in node performance based on the performance curve. In response to determining that the performance curve contains abnormal information, it can be considered that the node to which the performance curve belongs and its corresponding performance have a potential abnormal risk or are in an abnormal state. The abnormal information can be assigned an abnormality flag to provide an abnormality reference for node operation and maintenance, and facilitate rapid location of the abnormal node and corresponding performance based on the curve chart.

[0051] Among them, the abnormal indicator can be a display format, color, preset logo, bubble box, etc. that is different from other information, and is not limited here.

[0052] As can be seen, this embodiment can constrain the number of performance curves within each graph, preventing an excessive number of performance curves from being present, making it difficult to distinguish between them. This helps clarify the structure and trends of each performance curve, improves the intuitiveness of the graph, simplifies the interface display, and facilitates efficient extraction of the information expressed by each performance curve. Furthermore, it can analyze and identify anomalies, thereby increasing the information richness of the performance curve data display and improving the efficiency of anomaly location.

[0053] See also Figure 4 , Figure 4 It is a flowchart of another embodiment of the data processing method of the present application.

[0054] In this embodiment, the data processing device can obtain the performance data to be processed from the database of the data service end. For example, a list of performance data of the back end can be obtained through an https request. Among them, https (Hypertext Transfer Protocol Secure) is an HTTP (Hypertext Transfer Protocol) with security as its goal. It ensures the security of the transmission process through transmission encryption and identity authentication based on HTTP. HTTP is a request-response protocol, usually running on TCP (Transmission Control Protocol), which can specify the messages that the client may send to the server and the responses received.

[0055] The processing module can include an exception data processing module, a data preprocessing module, and an interface rendering module.

[0056] When the performance data to be processed is obtained, the data preprocessing module can preprocess each performance curve to provide the processed data to the data rendering module. The data preprocessing module can group the performance data, and each group of data includes a plurality of performance curves formed in a curve chart to reduce the number of performance curves included in each curve chart and improve the intuitiveness of the curve chart. In a simple way, for example, it is agreed that 5 performance curves are displayed in each curve chart. If there are 20 performance curves, they can be displayed in 4 curve charts to reduce the number of performance curves in each curve chart and reduce the risk of mutual influence and identification when observing the performance curves. Further, the performance curves can be dragged in this embodiment. In order to facilitate data comparison, the performance curves of the same type of curve chart after splitting can be dragged. The performance curve legend that needs to be moved can be dragged to the target curve chart.

[0057] Specifically, the data grouping rule can include at least one of a curve number threshold sub-rule of a curve chart and a curve duplication sub-rule.

[0058] For example, in response to the data grouping rule including the curve number threshold sub-rule of the curve chart, the number of performance curves that can be rendered in the performance data encapsulated in each group is less than or equal to the curve number threshold. The number of performance curves that can be rendered in the group encapsulation is allowed to be different. And / or,

[0059] In response to the performance data belonging to a performance curve carrying a duplication identifier, it is determined that the data grouping rule meets the curve duplication sub-rule, the performance data carrying the duplication identifier is taken as the performance data to be duplicated, and the performance data to be duplicated is written into a plurality of target groups. When the group encapsulation of the plurality of target groups is rendered into a chart, the performance data to be duplicated is rendered into a performance curve as a duplicate curve, and the group encapsulation including the performance data to be duplicated is displayed as a curve chart showing the duplicate curve.

[0060] In a simple way, the performance data can be encapsulated according to the curve number threshold of each curve chart input by the user on the interface in this embodiment. If the custom grouping function is enabled, the performance data can be encapsulated according to the custom grouping information. If the curve duplication function is enabled, the duplicated curve data can be encapsulated into a plurality of groups.

[0061] Furthermore, this embodiment can also adaptively group and encapsulate data according to the total number of performance curves to ensure that under the default configuration, the performance curves / curve charts can be displayed simultaneously on the same display page, and all performance curves can be viewed without dragging the scroll bar.

[0062] Specifically, the data grouping rule includes an adaptive grouping number sub-rule. Thus, this embodiment can obtain the resolution height of the display page. The display page is used to display the performance curve. The height occupied by fixed components within the display page can also be obtained to evaluate the height of the performance curve and the number of columns of the performance curve. The formula for calculating the adaptive grouping number can be shown as follows:

[0063] X=(L1-L2) / L3*nFormula 1-1

[0064] Among them, X represents the number of adaptive groups; L1 represents the resolution height of the display page; L2 represents the height occupied by fixed components in the display page; L3 represents the height of the performance curve; and n represents the number of columns of the performance curve.

[0065] Please continue reading Figure 4 In one embodiment, several ranking factors may be calculated for each performance curve, and these ranking factors may be packaged together as a ranking factor combination into a group package to provide a reference indicator for customizing the sorting order of subsequent display pages and to improve the efficiency of responding to sorting instructions.

[0066] The ranking factor may include at least one of the maximum value, minimum value, variance, average value, median, and maximum and minimum values ​​of the performance curve.

[0067] The calculation formula of the average value can be shown as follows:

[0068]

[0069] in, Indicates the average value of the performance curve; x i It represents the i-th performance data belonging to the performance curve, where i is a positive integer; n represents the number of performance data included in the performance curve.

[0070] The calculation formula of variance can be shown as follows:

[0071]

[0072] Among them, σ 2 represents the variance of the performance curve; Indicates the average value of the performance curve; x i It represents the i-th performance data belonging to the performance curve, where i is a positive integer; n represents the number of performance data included in the performance curve.

[0073] The median means that after the performance data included in the performance curve are sorted, if the length of the performance data is an odd number, the middle element is taken; if the length of the performance data is an even number, the average of the two middle elements is taken.

[0074] Specifically, the ranking factor combination of each performance curve can be calculated separately. In response to obtaining the ranking instruction, the target ranking factor indicated by the ranking instruction can be identified. The target ranking factor is one of the ranking factor combinations or the moment of the performance curve.

[0075] In this way, the performance curves of each group of packages can be traversed, sorted in a target order according to the target sorting factor, and the sorted performance curves can be formed into a new plurality of curve charts in the target order and a target number. The target number is the number of performance curves in the new curve chart.

[0076] like Figure 5 as well as Figure 6 As shown in the example, Figure 5 This is a schematic diagram of the interface of an embodiment of the display page of this application. Figure 6 This is an interface diagram of another embodiment of the display page of this application. Figure 5 The example shows the curve chart and display page formed according to the data grouping rules. Figure 6 The example shows a curve chart and display page that are sorted according to the sorting factor indicated by the sorting instruction.

[0077] Please continue reading Figure 4 、 Figure 7 as well as Figure 8 , Figure 7 This is a diagram illustrating an embodiment of abnormal performance data annotation in this application. Figure 8 It is a diagrammatic illustration of an abnormal performance curve marking an embodiment of the present application.

[0078] In one embodiment, the abnormal data processing module is used to evaluate whether the performance curve is abnormal or whether the performance curve contains abnormal data. The abnormality evaluation can be performed on the drawn performance curve or on the performance data belonging to each performance curve.

[0079] In this embodiment, in response to turning on the abnormal data monitoring function, abnormal data can be identified and marked for display, and analysis strategies can be designed to adapt to various performance categories, that is, the analysis categories in this embodiment, to improve the reliability of abnormal analysis and thus improve the reliability of abnormal information.

[0080] The analysis category may include at least one of a threshold-type statistical category, an incremental-type statistical category, and a discrete statistical category.

[0081] Furthermore, in response to determining that abnormal information exists, the performance category to which the abnormal information belongs can be obtained and the cause of the abnormal information can be analyzed. A search is performed to determine whether the preset causes for the performance category in the preset abnormality library include the current cause. If a preset cause matching the current cause exists, a repair method associated with the preset cause matching the current cause is obtained and executed to achieve automated repair. If no preset cause matching the current cause exists, the current cause is written to the preset abnormality library, and the process of repairing the abnormality caused by the current cause is recorded and written to the preset abnormality library as the repair method associated with the current cause.

[0082] The specific working principle of this embodiment is described below with examples.

[0083] The performance categories of the adaptation threshold statistical category may include: one or more of: CPU (Central Processing Unit) usage, memory usage, CPU temperature, motherboard temperature, latency, task queue length, network port utilization, hard disk busyness, IO (Input / Output) response time, data reconstruction speed, average load, etc.

[0084] The performance categories adapted to the incremental statistical category may include one or more of: hard disk utilization, capacity increment, total packet loss, total error packet number, and the like.

[0085] The performance categories adapted to the discrete statistical categories may include: one or more of: IOPS (Input / Output Operations Per Second, the number of read and write operations per second), bandwidth, network port flow rate, data throughput, IO size, and the like.

[0086] For example, the following table illustrates the relationship between analysis categories and performance categories:

[0087] Table 1 Correlation table between analysis categories and performance categories

[0088]

[0089]

[0090] Specifically, in this embodiment, the analysis category of the performance curve can be identified.

[0091] In response to the analysis category of the performance curve being the threshold-based statistical category, performance data of the performance curve may be traversed, and the performance data exceeding a first threshold may be used as the abnormal information. In layman's terms, for a performance curve adapted to the threshold-based statistical category, each point of the performance curve may be traversed, and all points exceeding the threshold range may be marked.

[0092] In response to the performance curve being analyzed as an incremental statistical category, the performance curve can be fitted, and the fitted performance curve can be differentiated, with performance data whose rate of increase exceeds a second threshold being used as the abnormal information. Generally speaking, capacity increments can typically be fitted to a straight line with a positive slope, where the slope represents the rate of data increment. For such curves, the least squares method is used for curve fitting, and the curve is then differentiated to find points where the rate of increase exceeds the threshold and mark them.

[0093] In response to the performance curve being analyzed as discrete statistics, a normal distribution can be used to assess abnormal information in the performance curve. Generally speaking, for data transmission statistics that fit the discrete statistics category, the transmission rate is typically fitted to a straight line parallel to the horizontal axis, with data points discretely distributed above and below the line. For this type of performance curve, this embodiment uses a normal distribution model, annotating data points that exceed three standard deviations.

[0094] Furthermore, the abnormality evaluation includes: evaluating abnormal data points in the performance curve, and / or evaluating abnormal performance curves of various preset performances.

[0095] In response to evaluating the abnormal data points within the performance curve, a data average of the performance data included in the performance curve may be calculated. A data standard deviation of the performance data within the performance curve may be calculated based on the data average. The sum of the data average and the data standard deviation magnified by a first preset factor is used as a first endpoint value, and the difference between the two is used as a second endpoint value, thereby forming an allowable range based on the first endpoint value and the second endpoint value. Performance data within the performance curve that is not within the allowable range is treated as abnormal information.

[0096] Among them, the calculation formulas of normal distribution, mean value and standard deviation can be shown as follows:

[0097]

[0098] Where f(x) represents the probability distribution of the random variable x under normal distribution with a location parameter μ and a scale parameter σ; σ represents the standard deviation of the performance curve; Indicates the average value of the performance curve; x i It represents the i-th performance data belonging to the performance curve, where i is a positive integer; n represents the number of performance data included in the performance curve.

[0099] In response to the evaluation of abnormal performance curves for each type of preset performance, an average value of each performance curve belonging to the current performance category can be obtained as the curve average value of each performance curve. Based on the curve average value, a category curve average value and a category curve standard deviation for the current performance category are calculated. The sum of the category curve average value and the category curve standard deviation magnified by a second preset factor is used as a third endpoint value, and the difference between the two is used as a fourth endpoint value, so as to form an allowable range based on the third endpoint value and the fourth endpoint value; and performance curves within the current performance category whose curve average value is not within the allowable range are treated as abnormal information.

[0100] In other words, a normal distribution model is applied to the average values ​​of similar performance curves. If the average value of a performance curve exceeds three standard deviations, the entire curve is considered abnormal. The entire curve is annotated to facilitate identification of the object of the curve and determine whether it has a fault.

[0101] Please continue reading Figure 4 In one embodiment, the interface rendering module is implemented based on visualization components, etc. It can render the performance data provided by the input module and the data processing module to the display page, and can monitor operations from the display interface in real time, such as manual curve grouping, curve cloning, curve dragging, sorting instructions, etc., and pass the monitoring results to the data processing module for processing before rendering it to the display interface again. Based on the results reported by the abnormal data processing module, abnormal points and abnormal curves are rendered.

[0102] The following is a brief example of the complete working principle of this application, please refer to Figure 9 , Figure 9 It is a flowchart of another embodiment of the data processing method of the present application.

[0103] S201: Acquire performance data to be processed.

[0104] S202: Group the performance data and calculate the maximum value, minimum value, variance, mean value, median, and maximum and minimum values ​​of each performance curve.

[0105] S203: Abnormal performance data monitoring, evaluating abnormal data points and / or abnormal curves.

[0106] S204: Display interface rendering.

[0107] S205: Determine whether an operation instruction is obtained.

[0108] In this embodiment, when it is determined that an operation instruction is obtained, step S202 is executed to adjust the data analysis of the performance data or sort the performance curve based on the operation instruction; when it is determined that no operation instruction is obtained, step S205 is executed or the process ends.

[0109] In summary, this application can simplify the interface display, making the number of curves in each chart controllable, reducing the mutual influence between curves, and making the interface display more concise and beautiful. It provides usability-enhancing functions such as curve dragging, curve cloning, custom grouping, and sorting, making it easier to compare monitoring data and find problems and anomalies. It provides an automatic abnormal data identification function and marks possible abnormal data, making it easy to quickly find abnormal monitoring data, thereby improving the effectiveness of interface monitoring.

[0110] It should be understood that although Figure 3-Figure 4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 3-Figure 4 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0111] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps of the data processing method described above are implemented, which will not be described in detail here.

[0112] See also Figure 10 , Figure 10 It is a structural diagram of an embodiment of the computer device of the present application.

[0113] In one embodiment, the computer device may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown in the example.

[0114] The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus.

[0115] The processor of the computer device is used to provide computing and control capabilities.

[0116] The memory of the computer device includes a non-volatile storage medium and an internal memory.

[0117] The non-volatile storage medium stores an operating system and a computer program.

[0118] The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.

[0119] The network interface of the computer device is used to communicate with an external terminal via a network connection.

[0120] When the computer program is executed by a processor, a data processing method is implemented.

[0121] The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the display page for displaying the curve chart mentioned in the above embodiment may be displayed on the display screen.

[0122] The input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0123] Those skilled in the art will understand that Figure 10 The structure shown as an example is merely a block diagram of a portion of the structure related to the present application solution, and does not constitute a limitation on the computer device to which the present application solution is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0124] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0125] S101: Acquire performance data to be processed; acquire data grouping rules.

[0126] S102: Grouping the performance data according to data grouping rules to form a plurality of group packages.

[0127] S103: Rendering a plurality of performance curves of the performance data in each group of packages respectively to form curve charts belonging to each group of packages.

[0128] S104: Perform an abnormality assessment on the performance curve, and in response to determining that abnormal information exists in the performance curve, assign an abnormality identifier to the abnormal information.

[0129] In an embodiment, the data grouping rule comprises at least one of a threshold number of curves of a graph sub-rule and a curve splitting sub-rule; when grouping the performance data according to the data grouping rule to form a plurality of group packages, the processor executing the computer program further implements the following steps: in response to the data grouping rule comprising the threshold number of curves of a graph sub-rule, the performance data divided into the group packages is allowed to render a number of performance curves less than or equal to the threshold number of curves; wherein the number of performance curves allowed to be rendered by the group packages is allowed to be different; in response to the performance data belonging to a performance curve carrying a splitting identifier, it is determined that the data grouping rule meets the curve splitting sub-rule, the performance data carrying the splitting identifier is taken as to-be-split performance data, and the to-be-split performance data is written into a plurality of target groups; when rendering a graph on the group packages of the plurality of target groups, the to-be-split performance data is rendered to form a performance curve as a split curve, and the group packages comprising the to-be-split performance data form a graph displaying the split curve.

[0130] In an embodiment, the data grouping rule comprises an adaptive grouping number sub-rule; when obtaining the data grouping rule, the processor executing the computer program further implements the following steps: obtaining a resolution height of a display page; wherein the display page is used to display a performance curve; obtaining a height occupied by a fixed component in the display page; evaluating a height of the performance curve and a number of arranged columns of the performance curve; and a calculation formula of the adaptive grouping number is X=(L1-L2) / L3*n; wherein X represents the adaptive grouping number; L1 represents the resolution height of the display page; L2 represents the height occupied by the fixed component in the display page; L3 represents the height of the performance curve; and n represents the number of arranged columns of the performance curve.

[0131] In an embodiment, when performing abnormal evaluation on the performance curve, the processor executing the computer program further implements the following steps: identifying an analysis category of the performance curve; wherein the analysis category comprises at least one of a threshold category, an increment category and a discrete category; in response to the analysis category of the performance curve being the threshold category, traversing performance data of the performance curve, and taking performance data exceeding a first threshold as abnormal information; in response to the analysis category of the performance curve being the increment category, performing fitting processing on the performance curve, and performing derivation on the fitted performance curve, and taking performance data with an increase speed exceeding a second threshold as abnormal information; and in response to the analysis category of the performance curve being the discrete category, performing abnormal information evaluation on the performance curve by using normal distribution.

[0132] In one embodiment, the abnormality evaluation includes: evaluating abnormal data points in the performance curve, and / or evaluating abnormal performance curves of various preset performances; in response to evaluating abnormal data points in the performance curve, when the performance curve is evaluated for abnormal information using a normal distribution, the processor further implements the following steps when executing the computer program: calculating the data average of the performance data contained in the performance curve; calculating the data standard deviation of the performance data in the performance curve based on the data average; taking the sum of the data average and the data standard deviation magnified by a first preset multiple as the first endpoint value, and taking the difference between the two as the second endpoint value, so as to form an allowable range based on the first endpoint value and the second endpoint value; taking the performance data in the performance curve that is not in the allowable range as Abnormal information; in response to evaluating abnormal performance curves of various preset performances, when the performance curves are evaluated for abnormal information using normal distribution, the processor also implements the following steps when executing the computer program: respectively obtaining the average values ​​of each performance curve belonging to the current performance category as the curve average value of each performance curve; calculating the category curve average value and the category curve standard deviation of the current performance category based on the curve average value; taking the sum of the category curve average value and the category curve standard deviation magnified by a second preset multiple as the third endpoint value, and taking the difference between the two as the fourth endpoint value, so as to form an allowable range based on the third endpoint value and the fourth endpoint value; taking the performance curve in the current performance category whose curve average value is not within the allowable range as abnormal information.

[0133] In one embodiment, when forming curve charts belonging to each group of packages, the processor also implements the following steps when executing the computer program: respectively calculating the ranking factor combination of each performance curve; wherein the ranking factor includes at least one of the maximum value, minimum value, variance, mean value, median, and maximum and minimum values ​​of the performance curve; in response to obtaining a ranking instruction, identifying the target ranking factor indicated by the ranking instruction; wherein the target ranking factor is one of the ranking factor combination or the moment of the performance curve; traversing the performance curves of each group of packages, sorting them in a target order according to the target ranking factor, and forming a new plurality of curve charts of the sorted performance curves in a target order and a target number; wherein the target number specifies the number of performance curves in the new curve chart.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0135] S101: Acquire performance data to be processed; acquire data grouping rules.

[0136] S102: Grouping the performance data according to data grouping rules to form a plurality of group packages.

[0137] S103: Rendering a plurality of performance curves of the performance data in each group of packages respectively to form curve charts belonging to each group of packages.

[0138] S104: Perform an abnormality assessment on the performance curve, and in response to determining that abnormal information exists in the performance curve, assign an abnormality identifier to the abnormal information.

[0139] In one embodiment, the data grouping rule includes at least one of a curve number threshold sub-rule and a curve clone sub-rule of a curve chart; when the performance data is grouped according to the data grouping rule to form a plurality of group packages, the computer program further implements the following steps when executed by the processor: in response to the data grouping rule including the curve number threshold sub-rule of the curve chart, the number of performance curves that can be rendered by the performance data divided into each group package is less than or equal to the curve number threshold; wherein, differences in the number of performance curves that can be rendered by the group package are allowed; in response to the performance data belonging to a performance curve carrying a clone identifier, it is determined that the data grouping rule meets the curve clone sub-rule, the performance data carrying the clone identifier is used as the performance data to be cloned, and the performance data to be cloned is written into multiple target groups; when the group packages of the multiple target groups are rendered as charts, the performance data to be cloned are all rendered to form performance curves as clone curves, so that the curve chart formed by the group package including the performance data to be cloned displays the clone curve.

[0140] In one embodiment, the data grouping rule includes an adaptive grouping number sub-rule; when obtaining the data grouping rule, the computer program further implements the following steps when executed by the processor: obtaining the resolution height of the display page; wherein the display page is used to display the performance curve; obtaining the height occupied by the fixed components in the display page; evaluating the height of the performance curve and the number of columns of the performance curve; the calculation formula for the adaptive grouping number is: X = (L1-L2) / L3*n; wherein X represents the adaptive grouping number; L1 represents the resolution height of the display page; L2 represents the height occupied by the fixed components in the display page; L3 represents the height of the performance curve; and n represents the number of columns of the performance curve.

[0141] In one embodiment, when performing an abnormality evaluation on a performance curve, the computer program further implements the following steps when executed by the processor: identifying the analysis category of the performance curve; wherein the analysis category includes at least one of a threshold statistical category, an incremental statistical category, and a discrete statistical category; in response to the analysis category of the performance curve being a threshold statistical category, traversing the performance data of the performance curve, and taking the performance data exceeding a first threshold as abnormal information; in response to the analysis category of the performance curve being an incremental statistical category, fitting the performance curve, and derivatizing the fitted performance curve, and taking the performance data whose increasing rate exceeds a second threshold as abnormal information; in response to the analysis category of the performance curve being a discrete statistical category, using a normal distribution to evaluate the performance curve for abnormal information.

[0142] In one embodiment, the abnormality evaluation includes: evaluating abnormal data points in a performance curve, and / or evaluating abnormal performance curves of various preset performances; in response to evaluating abnormal data points in the performance curve, when the performance curve is evaluated for abnormal information using a normal distribution, the computer program, when executed by the processor, further implements the following steps: calculating a data average value of the performance data contained in the performance curve; calculating a data standard deviation of the performance data in the performance curve based on the data average value; taking the sum of the data average value and the data standard deviation amplified by a first preset multiple as a first endpoint value, and taking the difference between the two as a second endpoint value, so as to form an allowable interval based on the first endpoint value and the second endpoint value; taking the performance data in the performance curve that is not in the allowable interval as Abnormal information; in response to evaluating abnormal performance curves of various preset performances, when the performance curves are evaluated for abnormal information using normal distribution, the computer program, when executed by the processor, also implements the following steps: respectively obtaining the average values ​​of each performance curve belonging to the current performance category as the curve average values ​​of each performance curve; calculating the category curve average value and the category curve standard deviation of the current performance category based on the curve average values; taking the sum of the category curve average value and the category curve standard deviation magnified by a second preset multiple as the third endpoint value, and taking the difference between the two as the fourth endpoint value, so as to form an allowable range based on the third endpoint value and the fourth endpoint value; taking the performance curve whose curve average value in the current performance category is not within the allowable range as abnormal information.

[0143] In one embodiment, when forming curve charts belonging to each group of packages, the computer program further implements the following steps when executed by the processor: respectively calculating the ranking factor combination of each performance curve; wherein the ranking factor includes at least one of the maximum value, minimum value, variance, mean value, median, and maximum and minimum values ​​of the performance curve; in response to obtaining a ranking instruction, identifying the target ranking factor indicated by the ranking instruction; wherein the target ranking factor is one of the ranking factor combination or the moment of the performance curve; traversing the performance curves of each group of packages, sorting them in a target order according to the target ranking factor, and forming a new plurality of curve charts of the sorted performance curves in a target order and a target number; wherein the target number specifies the number of performance curves in the new curve chart.

[0144] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0145] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make numerous variations and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: The data processing method includes: Obtain performance data to be processed; obtain data grouping rules; Grouping the performance data according to the data grouping rule to form a plurality of group packages; Rendering a plurality of performance curves of the performance data in each group of packages respectively to form curve charts belonging to each group of packages; performing an abnormality assessment on the performance curve, and in response to determining that abnormal information exists in the performance curve, assigning an abnormality identifier to the abnormal information; The abnormality evaluation of the performance curve includes: Identifying an analysis category of the performance curve; wherein the analysis category includes at least one of a threshold-based statistical category, an incremental statistical category, and a discrete statistical category; In response to the analysis category of the performance curve being the threshold-based statistical category, traversing the performance data of the performance curve, and taking the performance data exceeding a first threshold as the abnormal information; In response to the analysis category of the performance curve being the incremental statistical category, fitting the performance curve, and deriving the fitted performance curve, and taking performance data whose increasing speed exceeds a second threshold as the abnormal information; In response to the analysis category of the performance curve being the discrete statistics category, an abnormality assessment is performed on the performance curve using a normal distribution, the abnormality assessment including assessing abnormal data points within the performance curve and assessing abnormal performance curves of various preset performances: In response to evaluating an abnormal data point in the performance curve, evaluating the performance curve for abnormal information using a normal distribution includes: Calculating a data average of the performance data included in the performance curve; Calculating the data standard deviation of the performance data in the performance curve based on the data average; The sum of the data average value and the data standard deviation amplified by a first preset multiple is used as a first endpoint value, and the difference between the two is used as a second endpoint value, so as to form an allowable range based on the first endpoint value and the second endpoint value; and the performance data in the performance curve that is not within the allowable range is regarded as abnormal information; In response to evaluating abnormal performance curves of various preset performances, evaluating abnormal information of the performance curves using normal distribution includes: respectively obtaining an average value of each performance curve belonging to the current performance category as a curve average value of each performance curve; Calculating a category curve average and a category curve standard deviation of the current performance category based on the curve average; The sum of the category curve average value and the category curve standard deviation magnified by a second preset multiple is taken as the third endpoint value, and the difference between the two is taken as the fourth endpoint value, so as to form an allowable range based on the third endpoint value and the fourth endpoint value; the performance curve in the current performance category whose curve average value is not within the allowable range is taken as abnormal information.

2. The data processing method according to claim 1, wherein: The data grouping rule includes at least one of a curve quantity threshold sub-rule and a curve split sub-rule of the curve table; The step of grouping the performance data according to the data grouping rule to form a plurality of group packages includes: In response to the data grouping rule including a curve number threshold sub-rule of the curve table, the number of performance curves that can be rendered by the performance data divided into each of the group packages is less than or equal to the curve number threshold; wherein the groups are allowed to have different numbers of renderable performance curves; In response to the performance data belonging to a performance curve carrying a clone identifier, it is determined that the data grouping rule meets the curve clone sub-rule, and the performance data carrying the clone identifier is used as the performance data to be cloned, and the performance data to be cloned is written into multiple target groups; when the group encapsulation of the multiple target groups is rendered into a chart, the performance curve formed by the performance data to be cloned is rendered as a clone curve, so that the curve chart formed by the group encapsulation including the performance data to be cloned displays the clone curve.

3. The data processing method according to claim 1 or 2, characterized in that: The data grouping rule includes an adaptive grouping quantity sub-rule; The data grouping rules include: Obtaining a resolution height of a display page; wherein the display page is used to display the performance curve; Obtaining the height occupied by the fixed components in the display page; evaluating the height of the performance curve and the number of columns of the performance curve; The formula for calculating the number of adaptive groups is: X=(L1-L2) / L3*n Among them, X represents the number of adaptive groups; L1 represents the resolution height of the display page; L2 represents the height occupied by fixed components in the display page; L3 represents the height of the performance curve; and n represents the number of columns of the performance curve.

4. The data processing method according to claim 1, wherein: The forming of the curve charts belonging to each group of packages includes: Calculate the ranking factor combination of each performance curve respectively; wherein the ranking factor includes at least one of the maximum value, minimum value, variance, mean value, median, and maximum and minimum value of the performance curve; In response to obtaining a ranking instruction, identifying a target ranking factor indicated by the ranking instruction; wherein the target ranking factor is one of the ranking factor combinations or a moment of the performance curve; Traversing the performance curves of each group package, sorting them in a target order according to the target sorting factor, and forming a plurality of new curve charts with the sorted performance curves in the target order and a target number; wherein the target number is the number of performance curves in the specified new curve chart.

5. A data processing device for implementing the data processing method according to any one of claims 1 to 4, characterized in that: The data processing device includes: An input module, used to obtain performance data to be processed; A processing module, connected to the input module, is used to implement the data processing method according to any one of claims 1 to 4.

6. A data processing system, characterized in that: The data processing system comprises: Data server, used to store performance data; Communication module; The data processing device as claimed in claim 5 is connected to the data server through the communication module to perform data processing locally.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the data processing method according to any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 4 are implemented.

Images