Application system performance evaluation method, device, electronic device and storage medium

Abstract

The present invention discloses an application system performance evaluation method, device, electronic device and storage medium, wherein the method comprises: obtaining application system monitoring data; performing data cleaning and normalization processing on the application system monitoring data to obtain normalized data; performing vector folding processing on the normalized data according to a preset time slice to obtain image feature data; inputting the image feature data into a performance evaluation model for evaluation processing to obtain a prediction result. The present invention processes the image feature data through the performance evaluation model, thereby improving the efficiency and accuracy of the performance evaluation of the application system, and can be widely used in the field of artificial intelligence technology.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an application system performance evaluation method, device, electronic device and storage medium, device, electronic device and storage medium. Background Art

[0002] In recent years, science and technology have developed by leaps and bounds, and various application systems have emerged. However, there has been a lack of a scientific and effective comprehensive performance evaluation method for the performance evaluation of application systems, and it is impossible to objectively evaluate the functions and performance of various application systems. The relevant technologies are mainly based on statistical algorithms, rule-based algorithms, or the subjective experience of operation and maintenance personnel for evaluation, and there are problems such as inaccurate evaluation results. In summary, the technical problems existing in the relevant technologies need to be solved urgently. Summary of the invention

[0003] In view of this, embodiments of the present invention provide an application system performance evaluation method, device, electronic device and storage medium, device, electronic device and storage medium to improve the accuracy of application system performance evaluation.

[0004] In one aspect, the present invention provides an application system performance evaluation method, comprising:

[0005] Obtain application system monitoring data;

[0006] Performing data cleaning and normalization processing on the application system monitoring data to obtain normalized data;

[0007] Performing vector folding processing on the normalized data according to a preset time slice to obtain image feature data;

[0008] The image feature data is input into the performance evaluation model for evaluation processing to obtain a prediction result.

[0009] Optionally, obtaining application system monitoring data includes:

[0010] Use scripts to find and connect to the application system's interface and obtain the application system's monitoring data;

[0011] When the script fails to find the interface of the application system, the database background data is obtained through a limited account to obtain the application system monitoring data.

[0012] Optionally, the performing data cleaning and normalization processing on the application system monitoring data to obtain normalized data includes:

[0013] Performing quantile data statistical processing on the application system monitoring data to obtain statistical data;

[0014] Performing abnormality processing and missing processing on the statistical data to obtain cleaned data;

[0015] The cleaned data is standardized to obtain normalized data.

[0016] Optionally, performing vector folding processing on the normalized data according to the time slice to obtain image feature data includes:

[0017] Vectorizing the normalized data according to a preset time slice, and folding the vectorized data to obtain single-dimensional graph data;

[0018] Select multiple indicators based on system categories and determine the graph dimensions;

[0019] The single-dimensional image data is overlaid according to the image dimension to obtain image feature data.

[0020] Optionally, before inputting the image feature data into the performance evaluation model for evaluation processing, the method further includes pre-training the performance evaluation model, specifically including:

[0021] Acquire historical monitoring data, perform label generation processing on the historical monitoring data, and obtain training data;

[0022] Inputting the training data into the performance evaluation model to obtain an evaluation result;

[0023] Calculating the evaluation result and the label of the training data according to the mean square loss function to determine the loss value of the training;

[0024] The parameters of the performance evaluation model are updated according to the loss value.

[0025] Optionally, the performing label generation processing on the historical monitoring data to obtain training data includes:

[0026] Inputting the historical monitoring data into a clustering model for clustering processing to obtain label data;

[0027] The label data is cleaned to obtain training data.

[0028] Optionally, the method further comprises:

[0029] determining an inefficient system based on the prediction results;

[0030] Performing cost calculation on the inefficient system according to the public cloud price to obtain resource cost;

[0031] Inefficient systems are shut down and reduced in capacity based on the resource costs.

[0032] On the other hand, an embodiment of the present invention further provides an application system performance evaluation device, comprising:

[0033] The first module is used to obtain application system monitoring data;

[0034] The second module is used to perform data cleaning and normalization processing on the application system monitoring data to obtain normalized data;

[0035] The third module is used to perform vector folding processing on the normalized data according to a preset time slice to obtain image feature data;

[0036] The fourth module is used to input the image feature data into the performance evaluation model for evaluation processing to obtain a prediction result.

[0037] On the other hand, an embodiment of the present invention further discloses an electronic device, including a processor and a memory;

[0038] The memory is used to store programs;

[0039] The processor executes the program to implement the method described above.

[0040] On the other hand, an embodiment of the present invention further discloses a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0041] On the other hand, an embodiment of the present invention further discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0042] Compared with the prior art, the present invention adopts the above technical scheme and has the following technical effects: the present invention processes the monitoring data to obtain image feature data, which is input into the performance evaluation model for evaluation processing, and can analyze the performance of the current application system through the performance evaluation model to improve resource utilization efficiency; in addition, the present invention folds the monitoring data into image feature data to simulate the performance evaluation model, so that the performance evaluation model can automatically learn features and process multi-dimensional data, thereby improving the accuracy and generalization ability of the evaluation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 is a flow chart of an application system performance evaluation method provided by an embodiment of the present invention;

[0045] Figure 2 It is a schematic diagram of a convolutional neural network structure provided by an embodiment of the present invention;

[0046] Figure 3 is an algorithm flow chart provided by an embodiment of the present invention;

[0047] Figure 4 It is a system architecture diagram provided by an embodiment of the present invention;

[0048] Figure 5 is a schematic diagram of a system query provided by an embodiment of the present invention;

[0049] Figure 6 is a schematic diagram of an evaluation result provided by an embodiment of the present invention;

[0050] Figure 7 It is a schematic diagram of an agency work order provided by an embodiment of the present invention;

[0051] Figure 8 It is a structural diagram of an application system effectiveness evaluation device provided by an embodiment of the present invention;

[0052] Fig. 9 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0053] Fig.10 It is a structural schematic diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0055] First, several nouns involved in the present invention are analyzed:

[0056] Intelligent automated operation and maintenance (ai operations, Aiops) is a general term for technologies that apply artificial intelligence to traditional operation and maintenance operations, using machine learning, big data and other technologies to achieve automated and intelligent operation and maintenance.

[0057] Convolutional neural networks (CNN) are a type of feedforward neural networks that include convolutional calculations and have a deep structure. They are one of the representative algorithms of deep learning.

[0058] Python is a computer programming language that provides efficient high-level data structures and simple and effective object-oriented programming. Python syntax and dynamic typing, as well as the nature of an interpreted language, make it a programming language for writing scripts and rapidly developing applications on most platforms. With the continuous update of versions and the addition of new language features, it is gradually being used for the development of independent and large-scale projects.

[0059] Page View (PV), each time a user visits a website, it is recorded. Multiple visits by users to the same page are cumulative. Specifically, it refers to the page views or clicks of the website.

[0060] Unique Visitors (UV): A computer client that visits a website is a visitor. Visitors are distinguished by IP address. Repeated visits within a period of time are also considered a UV.

[0061] Transactions PerSecond (TPS) refers to the number of transactions completed per unit time. A transaction can be any operation that needs to be performed, such as inserting, updating, or deleting a database, sending and receiving network requests, etc. TPS is usually used to evaluate the throughput of a system, that is, the amount of work that the system can handle per unit time. A high TPS value indicates that the system has high processing power and concurrency performance.

[0062] Queries Per Second (QPS) refers to the number of queries processed per unit time. It is mainly used to evaluate the performance of a database or other storage system, especially its ability to process read operations (queries). A high QPS value indicates that the system can respond to query requests quickly and has a high reading efficiency.

[0063] In the related technologies, many issues regarding the efficiency of application resource utilization are evaluated based on statistical algorithms, rule-based algorithms, or subjective experience of operation and maintenance personnel. In recent years, artificial intelligence machine learning technology has developed by leaps and bounds, and its practical applications have become more and more extensive. The present invention combines the technical concept of aiops to collect and analyze the operating data, business access data, database access data, system architecture data, and other data of the system's host, and establishes a model to evaluate the effectiveness of the system, so as to facilitate the provision of business system shutdown and transfer processing suggestions, improve resource utilization efficiency, and standardize system use. The present invention uses a CNN neural network machine learning algorithm to evaluate system performance. First, it connects to the operating system monitoring system, database monitoring system, network monitoring system, and gateway system to obtain the required indicator data, and then establishes a system evaluation model to evaluate the latest data every day, and initiates an inefficient system disposal process for systems with low scores.

[0064] In view of this, an application system performance evaluation method is provided in an embodiment of the present invention. The application system performance evaluation method in the embodiment of the present invention can be applied to a terminal, can be applied to a server, or can be software running in a terminal or a server. The terminal can be a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0065] Reference Figure 1 The embodiment of the present invention provides an application system performance evaluation method, comprising:

[0066] S101, obtaining application system monitoring data;

[0067] S102, performing data cleaning and normalization processing on the application system monitoring data to obtain normalized data;

[0068] S103, performing vector folding processing on the normalized data according to a preset time slice to obtain image feature data;

[0069] S104: Input the image feature data into the performance evaluation model for evaluation processing to obtain a prediction result.

[0070] In an embodiment of the present invention, the application system monitoring data is first obtained, specifically by connecting the operating system monitoring system, the database monitoring system, the network monitoring system and the gateway system, etc., to obtain the required monitoring data, which is used to input the performance evaluation model for performance evaluation, including application access data, database monitoring data, network traffic data, server monitoring data and unified gateway data. Then, these application system monitoring data are cleaned and normalized, invalid data such as error data, missing data, etc. are cleaned, and standardized to obtain normalized data. Then, the normalized data is vector folded according to a preset time slice, wherein the preset time slice can be set to 30 seconds, which can be set according to actual conditions. That is, every 30 seconds is used as a time slice to select data from the normalized data to obtain a vector, and then the vector is folded to obtain image feature data, which is multidimensional data. Finally, the image feature data is input into the performance evaluation model for evaluation processing to obtain evaluation data. The performance evaluation model can adopt a convolutional neural network model, and supervised modeling is performed by the convolutional neural network method, and finally the specific system score of 1-100 is obtained by outputting the performance evaluation model.

[0071] It should be noted that, in various specific embodiments of the present invention, when it comes to the need to perform relevant processing based on the target object's information, the target object's behavioral data, the target object's historical data, the target object's location information, and other data related to the target object's identity or characteristics, the target object's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present invention needs to obtain sensitive information of the target object, it will obtain the target object's separate permission or consent through a pop-up window or jump to a confirmation page, and after clearly obtaining the target object's separate permission or consent, it will obtain the necessary target object-related data for the embodiment of the present invention to enable the normal operation.

[0072] As an optional implementation, in the above step S101, obtaining application system monitoring data includes:

[0073] Use scripts to find and connect to the application system's interface and obtain the application system's monitoring data;

[0074] When the script fails to find the interface of the application system, the database background data is obtained through a limited account to obtain the application system monitoring data.

[0075] In an embodiment of the present invention, the application system monitoring data is obtained by docking the interface of the application system through a Python script. The main sources of data include application access data, database monitoring data, network traffic data, server monitoring data and unified gateway data. Among them, indicators such as central processing unit (CPU), memory, disk usage, domain page visits (PV), number of independent access users (UV), traffic, transmission control protocol (TCP), number of transactions per second (TPS) and number of queries per second (QPS) of the database can be used as system monitoring data. When the application system has no interface, the corresponding monitoring data cannot be obtained through the script. The database background data can be directly obtained through a read-only restricted permission account to obtain the application system monitoring data. By obtaining the application system monitoring data, the embodiment of the present invention can analyze the performance of the application system from multiple dimensions, thereby improving the comprehensiveness of the application system performance evaluation.

[0076] As an optional implementation, in the above step S102, the data cleaning and normalization processing of the application system monitoring data to obtain normalized data includes:

[0077] Performing quantile data statistical processing on the application system monitoring data to obtain statistical data;

[0078] Performing abnormality processing and missing processing on the statistical data to obtain cleaned data;

[0079] The cleaned data is standardized to obtain normalized data.

[0080] In an embodiment of the present invention, the acquired application system monitoring data is subjected to quantile data statistical processing, specifically by statistically analyzing the 95% quantile data to balance the actual operation status and the peak status data. It is conceivable that the present invention can also set the 90% quantile value according to the actual situation. Among them, the 95% quantile data refers to the number that exceeds 95% of the application system monitoring data, which is a statistical concept. For the application system, it is very useful to accurately count its traffic, and it can take out some accidentally obtained abnormal values. By selecting the 95th percentile in the application system monitoring data as the highest value used in statistics, the data exceeding 5% will be discarded. In this way, the instantaneous burrs (spikes) can be removed, making the statistical average more meaningful. In the embodiment of the present invention, the statistical data can be obtained by using the nearest ordinal method, the linear interpolation method between the nearest ordinals, the weighted percentile method and other algorithms for statistics. Then the statistical data is cleaned, specifically by removing the abnormal processing and missing data to obtain the cleaned data. Finally, the normalized data is obtained by normalizing the cleaned data. Specifically, each cleaned data can be normalized and converted into a value in the range of [0, 1] to obtain the normalized data. The embodiment of the present invention can remove abnormal data by performing data cleaning and normalization on the application system monitoring data, thereby improving the accuracy and efficiency of the application system performance evaluation.

[0081] As an optional implementation, in the above step S103, the vector folding process is performed on the normalized data according to the time slice to obtain the image feature data, including:

[0082] Vectorizing the normalized data according to a preset time slice, and folding the vectorized data to obtain single-dimensional graph data;

[0083] Select multiple indicators based on system categories and determine the graph dimensions;

[0084] The single-dimensional image data is overlaid according to the image dimension to obtain image feature data.

[0085] In an embodiment of the present invention, the normalized data after cleaning and normalization is subjected to vector folding processing according to a preset time slice, specifically by simulating the normalized data construction as image dimension data. In a feasible embodiment, the following processing is performed for each application system monitoring data collected: first, the preset time slice is 30 seconds, 24 hours * 60 minutes * 1 time slice / every 30 seconds, one day is 2880 time slices, and one time slice takes one normalized data. Every 30 seconds is a time slice, and a feature map is constructed with a 7-day time slice, and a 1x20160 vector is obtained. This vector is folded into a 224*224 map (the excess part is filled with 0), that is, a single-dimensional map data. Then, according to the system category, n indicators are selected as the dimension of the image, and n*244*244 image features are generated, and the class graphic features spliced ​​into n*224*224 like an image are obtained, thereby obtaining image feature data. Wherein, n is a positive integer greater than 2. Next, the image feature data can be input into the performance evaluation model obtained by modeling with the convolutional neural network model for identification and evaluation. The embodiment of the present invention compares the monitoring data to the pixels of the image, takes the nearest data and folds it into a single-dimensional image of the system, and N indicators constitute an N-dimensional vector graph, and then performs supervised modeling in the form of a CNN convolutional neural network, making full use of the CNN convolutional neural network for system performance evaluation, which can automatically learn features, process multi-dimensional data, model nonlinear relationships, and has the advantages of parameter sharing and scalability, thereby improving the accuracy and generalization ability of the evaluation model.

[0086] As a further optional implementation, before inputting the image feature data into the performance evaluation model for evaluation processing, the method further includes pre-training the performance evaluation model, specifically including:

[0087] Acquire historical monitoring data, perform label generation processing on the historical monitoring data, and obtain training data;

[0088] Inputting the training data into the performance evaluation model to obtain an evaluation result;

[0089] Calculating the evaluation result and the label of the training data according to the mean square loss function to determine the loss value of the training;

[0090] The parameters of the performance evaluation model are updated according to the loss value.

[0091] In the embodiment of the present invention, historical monitoring data is obtained and label processed to generate training data. The training data is then input into the performance evaluation model to obtain the evaluation result. Figure 2In the embodiment of the present invention, a convolutional neural network model is used as the performance evaluation model, and a mean square error loss function is used as the loss function. The evaluation result and the label of the training data are calculated according to the mean square loss function to determine the loss value of the training, so that the parameters of the performance evaluation model are updated according to the loss value to obtain the trained performance evaluation model. Specifically, after the data in the training data is input into the performance evaluation model, the evaluation result output by the model can be obtained, and the accuracy of the model prediction can be evaluated according to the evaluation result and the aforementioned label, so as to update the parameters of the model. For the performance evaluation model, the accuracy of the model evaluation result can be measured by the mean square loss function. The loss function is defined on a single training data and is used to measure the prediction error of a training data. Specifically, the loss value of the training data is determined by the label of the single training data and the prediction result of the model for the training data. In actual training, a training data set has a lot of training data, so a cost function is generally used to measure the overall error of the training data set. The cost function is defined on the entire training data set and is used to calculate the average value of the prediction error of all training data, which can better measure the prediction effect of the model. For general machine learning models, the aforementioned cost function, plus a regularization term that measures the complexity of the model, can be used as the objective function for training. Based on this objective function, the loss value of the entire training data set can be calculated. Based on the training loss value, the back propagation algorithm is used to update the parameters of the model, and a trained performance evaluation model can be obtained after several rounds of iteration. The specific number of iterations can be set in advance, or the training is considered complete when the test set meets the accuracy requirements.

[0092] As a further optional implementation, the performing label generation processing on the historical monitoring data to obtain training data includes:

[0093] Inputting the historical monitoring data into a clustering model for clustering processing to obtain label data;

[0094] The label data is cleaned to obtain training data.

[0095] In an embodiment of the present invention, because there is less accumulation of label data, the following method can be used to generate label data, thereby obtaining training data. Specifically, historical monitoring data is obtained by extracting data from the historical system from the monitoring data, and the K-Means clustering model can be used for clustering. It is divided into the incubation period, growth period, maturity period, and withdrawal period. Data features are obtained from the historical monitoring data at each stage, so as to quickly generate benchmark data with different scores, avoid the tedious manual labeling process, and the indicator data and scores can constitute the sample data required for training. The embodiment of the present invention can also update the model based on the updated data in the early morning of each day, which can reduce the working time of manual labeling. Finally, the training data is obtained by cleaning the sample data.

[0096] As an optional implementation, the method further includes:

[0097] determining an inefficient system based on the prediction results;

[0098] Performing cost calculation on the inefficient system according to the public cloud price to obtain resource cost;

[0099] Inefficient systems are shut down and reduced in capacity based on the resource costs.

[0100] In an embodiment of the present invention, by collecting monitoring data and evaluating the monitoring data based on an evaluation model, a prediction result is obtained. According to the prediction result, inefficient systems can be found, and the resource cost of each system can be calculated according to the public cloud price. Inefficient systems can be shut down and scaled down in time to improve the overall resource utilization efficiency. Specifically, in an embodiment of the present invention, the monitoring data of each system can be input into the performance evaluation model for calculation every morning, and the output result is a system prediction result of 1-100 points, and the system triggers emails and to-dos for systems with a score below 60, and the application manager handles the feedback. It should be noted that as the cloud migration work progresses, the embodiment of the present invention can also expand the performance evaluation model to the cloud platform at the same time, and build a system evaluation model suitable for the group's cloud migration system based on the cloud platform data, so as to improve the overall resource utilization efficiency on the cloud.

[0101] Reference Figure 3, the process of the embodiment of the present invention specifically includes: monitoring and collecting data from the application system, performing data cleaning processing on the collected monitoring data, specifically processing the abnormal and missing data of the collected monitoring data, and standardizing it into normalized data. Then the normalized data is input into the K-Means model for clustering to obtain the label data. The label data is then cleaned, and after being confirmed as valid data, it is input into the trained performance evaluation model for evaluation. Among them, the training process of the model is: converting the features of the sample data into image feature data, and then establishing the performance evaluation model according to the CNN convolutional neural network algorithm, by training the performance evaluation model, adjusting the weights and deviations, and finally deriving the trained performance evaluation model. The embodiment of the present invention scores the data by inputting it into the performance evaluation model, and displays the corresponding scoring results. The embodiment of the present invention also synchronizes new resource data every day, cleans the new resource data, and then inputs it into the performance evaluation model for evaluation.

[0102] Reference Figure 4 The system framework of the embodiment of the present invention may include platform data, performance evaluation and process feedback. Specifically, it obtains the resources and monitoring data of each system from the Wing Loong platform and the configuration management library (CMDB), synchronizes them to the performance evaluation system, evaluates the data through the performance evaluation model, and displays the system score and resource utilization through charts and other means. It enters the low-efficiency processing process, combines the IT service management (ITSM) low-efficiency process with the continuous low-efficiency system, promotes shutdown, transformation and reduction of capacity, and synchronizes the system information to the Wing Loong platform and the configuration management library to improve the overall resource utilization efficiency.

[0103] In a feasible embodiment, the embodiment of the present invention generates sample data based on the historical monitoring data of the inefficient system. CNN modeling is performed by analogy with images to obtain an efficiency evaluation model to apply to more systems, achieve sufficient standardization, and adapt to the daily operation and maintenance scenarios of all application systems. The embodiment of the present invention collects and monitors the host CPU, memory, TCP, number of connections, database TPS, QPS, PV / UV of the domain name corresponding to the application in the gateway system (if there is no domain name, the TCP 95% percentile value of the host corresponding to the system is calculated) and other monitoring data through the development system; then each indicator is cleaned and the latest 20160 indicator data are taken; then each indicator is normalized and converted into a value within the range of [0,1]. Then each indicator is vectorized and folded into a 244*244 graph; finally, n indicators are selected according to the system category as the dimension of the image to generate n*244*244 image features. The CNN convolutional neural network is used to predict the image features and obtain a score of 1-100 as the evaluation result. For scores below 60, the system triggers emails and to-do items to be handled and feedback is provided by the application manager. Figure 5The evaluation results of each application system are displayed through the backend system, and the results can be queried by entering the corresponding business type, guarantee level, date and other data. Figure 6 , the embodiment of the present invention can also display monthly rental costs, monthly rating trends, project basic information, operating conditions of the host components and other data in a graphical manner. Figure 7 , the embodiment of the present invention triggers the inefficient system background to-do work order process to the application person in charge for disposal.

[0104] Reference Figure 8 The embodiment of the present invention further provides an application system performance evaluation device, comprising:

[0105] The first module 801 is used to obtain application system monitoring data;

[0106] The second module 802 is used to perform data cleaning and normalization processing on the application system monitoring data to obtain normalized data;

[0107] The third module 803 is used to perform vector folding processing on the normalized data according to a preset time slice to obtain image feature data;

[0108] The fourth module 804 is used to input the image feature data into the performance evaluation model for evaluation processing to obtain a prediction result.

[0109] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0110] and Fig. 9 Corresponding to the method, an embodiment of the present invention further provides an electronic device, including a processor 901 and a memory 902; the memory is used to store a program; the processor executes the program to implement the method as described above.

[0111] and Fig.10 Corresponding to the method, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium 1001 stores a program 1002, and the program is executed by a processor to implement the method described above.

[0112] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 1 The method shown.

[0113] In summary, the embodiments of the present invention have the following advantages:

[0114] 1) In the embodiment of the present invention, the monitoring index data is compared to the pixels of an image, and the most recent data is folded into a single-dimensional image of the system. N indicators constitute an N-dimensional vector graph, and then supervised modeling is performed in the form of a CNN convolutional neural network. The performance of the application system is evaluated through a machine learning method, thereby improving the accuracy and efficiency of the evaluation.

[0115] 2) The embodiment of the present invention folds the indicator data into n-dimensional vector data and uses the CNN convolutional neural network to evaluate the system performance. It can automatically learn features, process multi-dimensional data, model nonlinear relationships, and has the advantages of parameter sharing and scalability, thereby improving the accuracy and generalization ability of the evaluation model.

[0116] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0117] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0118] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0119] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0120] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0121] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0122] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0123] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0124] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for evaluating the performance of an application system, characterized in that: The method comprises: Obtain application system monitoring data; Performing data cleaning and normalization processing on the application system monitoring data to obtain normalized data; Performing vector folding processing on the normalized data according to a preset time slice to obtain image feature data; Inputting the image feature data into the performance evaluation model for evaluation processing to obtain a prediction result; The step of performing vector folding processing on the normalized data according to a preset time slice to obtain image feature data includes: The normalized data is vectorized according to a preset time slice, and the vectorized data is folded to obtain single-dimensional graph data; the single-dimensional graph data is used to represent graph data of a single indicator; Select multiple indicators according to the application system category to determine the graph dimension; the graph dimension is the number of indicators selected by the application system; Performing superposition processing on the single-dimensional image data according to the image dimension to obtain image feature data; The obtaining of application system monitoring data includes: Use scripts to find and connect to the application system's interface and obtain the application system's monitoring data; When the script fails to find the interface of the application system, the database background data is obtained through a restricted account to obtain application system monitoring data, which includes central processing unit, memory, disk usage, domain name page visits, number of independent access users, traffic, transmission control protocol, number of database transactions per second and number of queries per second.

2. The method according to claim 1, characterized in that The data cleaning and normalization processing of the application system monitoring data to obtain normalized data includes: Performing quantile data statistical processing on the application system monitoring data to obtain statistical data; Performing abnormal and missing processing on the statistical data to obtain cleaned data; The cleaned data is standardized to obtain normalized data.

3. The method according to claim 1, characterized in that: Before inputting the image feature data into the performance evaluation model for evaluation processing, the method further includes pre-training the performance evaluation model, specifically including: Acquire historical monitoring data, perform label generation processing on the historical monitoring data, and obtain training data; Inputting the training data into the performance evaluation model to obtain an evaluation result; Calculating the evaluation result and the label of the training data according to the mean square loss function to determine the loss value of the training; The parameters of the performance evaluation model are updated according to the loss value.

4. The method according to claim 3, characterized in that The label generation process is performed on the historical monitoring data to obtain training data, including: Inputting the historical monitoring data into a clustering model for clustering processing to obtain label data; The label data is cleaned to obtain training data.

5. The method according to claim 1, characterized in that The method further comprises: determining inefficient systems based on the prediction results; Performing cost calculation on the inefficient system according to the public cloud price to obtain resource cost; Inefficient systems are shut down and reduced in capacity based on the resource costs.

6. An application system performance evaluation device, characterized in that: The device comprises: The first module is used to obtain application system monitoring data; The second module is used to perform data cleaning and normalization processing on the application system monitoring data to obtain normalized data; The third module is used to perform vector folding processing on the normalized data according to a preset time slice to obtain image feature data; The fourth module is used to input the image feature data into the performance evaluation model for evaluation processing to obtain a prediction result; The third module is used to perform vector folding processing on the normalized data according to a preset time slice to obtain image feature data, including: The normalized data is vectorized according to a preset time slice, and the vectorized data is folded to obtain single-dimensional graph data; the single-dimensional graph data is used to represent graph data of a single indicator; Select multiple indicators according to the application system category to determine the graph dimension; the graph dimension is the number of indicators selected by the application system; Performing superposition processing on the single-dimensional image data according to the image dimension to obtain image feature data; The first module is used to obtain application system monitoring data, including: Use scripts to find and connect to the application system's interface and obtain the application system's monitoring data; When the script fails to find the interface of the application system, the database background data is obtained through a restricted account to obtain application system monitoring data, which includes central processing unit, memory, disk usage, domain name page visits, number of independent access users, traffic, transmission control protocol, number of database transactions per second and number of queries per second.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images