Performance tuning method and system for linux system

Abstract

The invention discloses a performance tuning method and system for a linux system. The method comprises the following steps: acquiring system information and generating a corresponding text file; extracting a key index from the text file, updating an index threshold in real time, and comparing the key index with the updated index threshold to judge whether an abnormal key index exists or not; if the abnormal key index exists, the system performance bottleneck and the tuning suggestion are matched according to the abnormal key index, and then the abnormal key index and the tuning suggestion are displayed; and waiting and obtaining the tuning file, and performing tuning operation according to the content of the tuning file. According to the method, the performance bottleneck can be automatically identified, the optimization strategy can be intelligently recommended, the tuning action can be efficiently executed, and the method has great significance in improving the stability, the response speed and the overall performance of the Linux system under high load.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a performance tuning method and system for a Linux system. Background Art

[0002] System performance, as a key indicator to measure the operating system's execution efficiency, stability and responsiveness, is directly related to user experience and overall system performance. In the Linux system environment, in the face of high-load scenarios, performance bottlenecks such as excessive CPU resource consumption, lack of memory resources, disk I / O congestion and network failures are becoming increasingly prominent. These problems not only seriously weaken the stability of the system, but also lead to increased response delays, directly hindering the smooth operation and high performance of the system, and posing a potential threat to business continuity.

[0003] In view of the above challenges, accurate and efficient performance tuning of Linux servers has become a technical problem that needs to be solved urgently. At present, the performance tuning process generally covers key steps such as benchmarking and goal setting; stress testing and real-time monitoring; bottleneck identification, optimization strategy implementation and effect verification. However, this process not only has a high technical threshold, requiring operators to have deep operating system knowledge and rich tuning experience, but also in actual applications, facing complex and changing system environments, manual positioning of performance bottlenecks is often time-consuming and labor-intensive, inefficient, and difficult to meet the needs of rapid response and continuous optimization.

[0004] The Chinese patent application with publication number CN112612692A discloses a performance tuning method for a Linux system, which selects a reference tuning Linux operating system for comparison with the Linux operating system to be tuned, installs the two systems on the same server, uses the same performance testing tool to test the two systems respectively, and selects the test items whose test scores of the Linux operating system to be tuned are lower than those of the reference tuning Linux operating system, then excludes irrelevant items from the low-scoring test items, and finally optimizes the remaining low-scoring test items after excluding irrelevant items, thereby avoiding the need to perform performance testing and tuning on all software packages of the Linux operating system to be tuned in sequence, thereby being able to quickly discover performance defects and optimize them, solving the technical problem that the prior art cannot quickly find software packages that affect the performance of the Linux system, and improving the efficiency of Linux system performance testing and performance optimization. This solution has the following defects:

[0005] (1) Two operating systems need to be installed at the same time for comparative testing and optimization. This process is rather cumbersome and places high demands on the operator.

[0006] (2) The actual application scenarios are relatively limited and it is difficult to apply to various production scenarios.

[0007] (3) Tuning is mainly performed at the system level, and the testing and tuning methods are biased towards the bottom layer, which is of limited help in solving tuning problems in actual applications. Summary of the invention

[0008] The technical problem to be solved by the present invention is as follows: In view of the above-mentioned problems in the prior art, a performance tuning method and system for a Linux system are provided, which can automatically identify performance bottlenecks, intelligently recommend optimization strategies, and efficiently execute tuning actions, which is of great significance for improving the stability, response speed and overall performance of the Linux system under high load.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0010] A performance tuning method for a Linux system includes the following steps:

[0011] Get system information and generate corresponding text files;

[0012] Extracting key indicators from the text file, reading indicator thresholds, and then comparing the key indicators with corresponding indicator thresholds to determine whether there are abnormal key indicators;

[0013] If there are abnormal key indicators, match the system performance bottlenecks and tuning suggestions based on the abnormal key indicators, and then display the abnormal key indicators and tuning suggestions;

[0014] Wait for and obtain the tuning file, and perform tuning operations based on the contents of the tuning file.

[0015] Furthermore, when obtaining system information, it includes:

[0016] According to user configuration, obtain the corresponding system configuration information;

[0017] Use the specified system indicator collection command to collect the corresponding system indicator information;

[0018] Use the perf tool to collect performance event information and statistics to capture hot functions in the system.

[0019] Furthermore, when extracting key indicators from the text file, indicators reflecting software and hardware performance determined based on tuning experience are extracted from the text file as key indicators based on one or more of the collection type, indicator name, row and column information, and device information, and the key indicators and their related data are stored in a key indicator data structure based on a pre-built tuning knowledge base.

[0020] Furthermore, the key indicators include system bottom-level key indicators and system upper-level key indicators, each system bottom-level key indicator corresponds to one or more system upper-level key indicators, and when comparing the key indicators with the corresponding indicator thresholds, it specifically includes: comparing the current system bottom-level key indicators with the corresponding indicator thresholds and judging whether the current system bottom-level key indicators are abnormal based on the comparison results; if yes, selecting the corresponding system upper-level key indicators and comparing them with the indicator thresholds corresponding to the selected system upper-level key indicators and judging whether the selected system upper-level key indicators are abnormal based on the comparison results.

[0021] Furthermore, when matching system performance bottlenecks and tuning suggestions based on abnormal key indicators, it includes:

[0022] If the number of abnormal key indicators that exist at the same time is greater than the preset value, traverse all abnormal key indicators, select abnormal key indicators with associated relationships according to the aforementioned logical judgment statements, and then progressively screen the remaining abnormal key indicators that have a progressive relationship with the selected abnormal key indicators, and finally generate an abnormal key indicator combination, match the abnormal key indicator combination indicator with the abnormal key indicator and bottleneck description mapping relationship corresponding to the combination in the preset bottleneck diagnosis logic, derive the corresponding system bottleneck description, and match the system bottleneck description with the preset tuning strategy library to obtain the corresponding suggestion index, and finally obtain specific tuning suggestions based on the suggestion index.

[0023] Furthermore, before displaying abnormal key indicators and tuning suggestions, it also includes:

[0024] The key indicators of the exception are stored in the exception data structure. If the system performance bottleneck and tuning suggestions are matched according to the key indicators of the exception, the display mark in the exception data structure is updated to the specified value to display the key indicators of the exception.

[0025] Furthermore, after waiting for and obtaining the tuning file, the process further includes the steps of performing an obfuscation operation and / or an encryption operation on the tuning file, wherein the encryption operation specifically uses a symmetric key to encrypt the tuning file and generate a message authentication code to ensure the integrity and authenticity of the data during transmission, wherein the obfuscation operation step includes:

[0026] Randomly select n1 preset replacement rules to replace the strings that meet the conditions in the tuning file to hide the actual tuning parameters;

[0027] Randomly select n2 strings to be inserted and embed them into random positions of existing parameters to improve obfuscation;

[0028] Randomly select n3 parameters from the preset obfuscation parameter library and insert them into random positions in the tuning file;

[0029] Output the obfuscated file content and change it to the specified extension.

[0030] Furthermore, the steps of performing predefined tuning are also included, including:

[0031] Match the current application scenario with the preset scenario information and the corresponding configuration file. If the match is successful, send a confirmation request to the user.

[0032] Wait for and obtain the user's confirmation information, write the parameters in the configuration file to the system environment, and restart to make the configuration file take effect.

[0033] Furthermore, the method further includes the steps of performing real-time dynamic tuning, specifically including:

[0034] Get the tuning information in the configuration file and back up the system environment. Then run the Bayesian optimization algorithm based on the tuning information to get the optimal configuration parameter combination. Finally, restore the system environment.

[0035] The present invention also provides a performance tuning system for a Linux system, comprising a microprocessor and a computer-readable storage medium connected to each other, wherein the microprocessor is programmed or configured to execute any one of the performance tuning methods for the Linux system.

[0036] Compared with the prior art, the advantages of the present invention are:

[0037] The present invention can comprehensively collect detailed system configuration information and real-time system indicator information, covering the operation status of resources such as CPU, memory, disk, network, system log, hot data collection, etc., to provide users with an overall overview of the system; after processing and analyzing the system indicator data, the system bottleneck is analyzed, and according to the currently identified bottleneck situation, targeted tuning suggestions and operation instructions are output, providing users with valuable tuning references in actual application scenarios, and helping to optimize system performance; at the same time, tuning parameters are provided for diversified application scenarios, and the tuning parameters are issued, and the best configuration combination of the tuning scenario is obtained, providing efficient and flexible tuning solutions for various scenarios; in addition, it supports encryption and obfuscation of tuning files, which can completely hide the original tuning file content, effectively prevent the leakage of tuning means, and greatly simplify the tuning operation process while ensuring security. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is an overall flow chart of an embodiment of the present invention.

[0039] Figure 2 The figure is a schematic diagram showing bottleneck information in an embodiment of the present invention.

[0040] Figure 3A schematic diagram showing optimization suggestions in an embodiment of the present invention.

[0041] Figure 4 This is a diagram of the encryption tuning system architecture in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0043] Before introducing the specific embodiments of the present invention, related concepts or terms are first explained.

[0044] Application scenarios: Application scenarios refer to the specific situations in which operating systems are actually used in different fields and environments. Different application scenarios have different requirements for the performance, functions, and security of operating systems. Common application scenarios include big data, databases, storage, and networks.

[0045] Performance tuning: refers to the process of configuring, adjusting and optimizing the operating system to improve its performance indicators such as processing speed, responsiveness, resource utilization and system stability. It is necessary to comprehensively consider multiple factors and significantly improve the performance and efficiency of the operating system through reasonable resource allocation, scheduling strategy optimization, memory management optimization, file system management optimization and other measures to meet the needs of users and systems for higher performance.

[0046] Intelligent tuning: refers to the use of artificial intelligence technology to tune the operating system performance, automatically adjust and optimize the settings of the operating system and the software and hardware running on it to achieve the optimal working state.

[0047] Embodiment 1

[0048] In order to effectively solve the many challenges faced in the system tuning process, and to cope with the high requirements for operating system performance in current actual scenarios: the tuning process and problem location are highly complex, the tuning is difficult, the technical level of operators is high, and the traditional manual tuning method is inefficient. This embodiment proposes a lightweight, cross-platform, and multi-scenario Linux system performance tuning method, such as Figure 1 As shown, the following steps are included:

[0049] One-click collection of all indicators: obtain system information and generate corresponding text files;

[0050] Automatic bottleneck analysis and expert tuning guidance: extract key indicators from the text file, read the indicator threshold, and then compare the key indicators with the corresponding indicator threshold to determine whether there are abnormal key indicators; if there are abnormal key indicators, match the system performance bottleneck and tuning suggestions based on the abnormal key indicators, and then display the abnormal key indicators and tuning suggestions;

[0051] Predefined tuning: Match the current application scenario with the preset scenario information and the corresponding configuration file. If the match is successful, send a confirmation request to the user; wait for and obtain the user's confirmation information, write the parameters in the configuration file to the system environment, and restart to make the configuration file effective;

[0052] Real-time dynamic tuning: obtain the tuning information in the configuration file and back up the system environment, then run the Bayesian optimization algorithm based on the tuning information to obtain the optimal configuration parameter combination, and finally restore the system environment;

[0053] Encrypted tuning: Wait for and obtain the tuning file, obfuscate or encrypt the tuning file and then output it, so that the Linux system can perform tuning operations according to the content of the tuning file.

[0054] In the above steps, the one-click collection of all indicators, automatic bottleneck analysis and expert tuning guidance provide general scenario tuning functions, while predefined tuning, real-time dynamic tuning and encryption tuning all provide functions independent of the general scenario tuning function. The above steps can collect all system information, automatically identify bottlenecks and provide expert tuning guidance for the current system operation status, significantly improving the efficiency of problem diagnosis and resolution; and can provide efficient and flexible tuning solutions for a variety of application scenarios, supporting one-click parameter distribution and recovery; in addition, the intelligent optimization algorithm is integrated into the scene tuning process, and the intelligent optimization algorithm (Bayesian optimization algorithm) can be run according to the configuration file provided by the user for precise analysis, providing users with the best parameter configuration in the actual application scenario, thereby providing an effective performance optimization reference; at the same time, it supports the encryption and obfuscation functions of the tuning file, and supports the one-click distribution of the encrypted / obfuscated file content. In the actual tuning scenario, it can effectively prevent the leakage of the tuning means, greatly simplify the tuning operation process while ensuring security, so as to improve the efficiency and security of the work.

[0055] The following is a detailed description of each step.

[0056] In this embodiment, the one-click collection of all indicators is to comprehensively collect detailed system configuration information and real-time system indicator information, aiming to efficiently and comprehensively collect important information of the system. This function not only includes basic information such as CPU, memory, disk, network, etc., but also includes advanced information such as logs and comprehensive data, to assist users in comprehensively understanding the system. In the existing tuning work, usually only a single aspect is focused on, such as kernel tuning, JVM tuning, glibc tuning, etc., and its technical means are complex, the operation process is numerous, and there are high limitations, with the disadvantages of long time consumption and low efficiency, and it is difficult to perform comprehensive tuning of various components of the operating system in a short time in the actual application scenario. For example, in terms of system indicator collection, the existing means are difficult to collect indicators of various aspects of the system at the same time in a short time, such as within 1 minute. This embodiment supports user-defined configuration collection information, integrates different interfaces in the system, covers the collection of various indicators of the operating system, and provides a comprehensive system monitoring solution, thereby realizing real-time feedback for technical personnel to conduct comprehensive analysis and tuning.

[0057] Specifically, in this embodiment, the one-click collection of all indicators includes the following steps:

[0058] Step 1: Configure collection parameters: To collect all indicators in the Linux system, the user first configures the information collection parameters in the specified path, such as selecting which system information to collect (CPU, memory, disk, network, etc.), the number of collections, the duration of a single collection, etc., and customizes the collected information.

[0059] Step 2: Get system information, including:

[0060] According to the user configuration, obtain the corresponding system configuration information, including host name, system name, kernel version, CPU model / architecture / number of cores, total memory size / free space, gcc / glibc / jdk version information, network disk device information, etc., to provide an overall overview of the system;

[0061] Use the specified system indicator collection command to collect the corresponding system indicator information; use the perf tool to collect performance event information and statistical data to capture hot functions in the system;

[0062] After the collection is completed, the corresponding txt files (such as CPUinfo.txt, diskinfo.txt, etc.) are generated. Finally, the txt data files are cleaned, filtered and summarized, and a visual report is generated in html format.

[0063] Step 2 of this embodiment not only supports the aggregation of basic information such as the host, system, and kernel version, but also comprehensively captures common performance and bottlenecks in the Linux system by integrating multiple monitoring instructions, such as top, numactl, pidstat, sar, free, vmstat, ethtool, etc. At the same time, it supports perf data collection. By enabling this function, hot functions in the system are captured, and functions with high system call proportions are obtained, and are intuitively displayed through cold and hot flame graphs, helping users quickly identify performance bottlenecks and optimize them.

[0064] The automatic bottleneck analysis and expert tuning guidance stage of this embodiment is divided into four core modules: accurate indicator data collection, intelligent abnormal indicator identification, expert-level tuning strategy recommendations, and intuitive abnormality display and guidance.

[0065] The precise indicator data collection module is dedicated to efficiently extracting key indicators from multiple customized text (txt) sources, and integrating these indicators and their related data into a carefully designed data structure based on a pre-built tuning knowledge base. This structure not only covers the indicator's unique identifier, name, and real-time value, but also includes commands for data retrieval and detailed explanations of the indicator. More importantly, each indicator value is accompanied by detailed metadata, such as the specific value, row and column position in the data source, and associated device information, laying the foundation for subsequent intelligent analysis and display.

[0066] When obtaining system information in step 2, the commands for collecting different software and hardware indicators are summarized and output to different txt files. In each txt file, different commands are classified and identified using main commands and slave commands according to the different collected information. Key indicators are indicators that can reflect the performance of software and hardware, such as network bandwidth, CPU frequency, etc. (mainly derived from system tuning experience). When extracting key indicators, the indicators reflecting the performance of software and hardware determined based on tuning experience are extracted from the text file according to the collection type, indicator name, row and column information, device information, etc. as key indicators, and are read accurately to ensure the efficiency and accuracy of key indicator extraction.

[0067] The intelligent abnormal indicator identification module first encodes the preset threshold strategy into the configuration file. Then, the system reads these thresholds and makes a precise comparison with the indicator values ​​collected in real time. Once any indicator that exceeds the normal range is detected, the system will immediately mark it and store it in the abnormal data structure, or include it in the highlighted data structure for quick identification. This highlight structure not only retains the basic information, but also introduces an additional Boolean control variable to flexibly determine whether the indicator is highlighted in the final display stage.

[0068] The expert-level tuning strategy recommendation module integrates advanced bottleneck diagnosis logic, which can conduct comprehensive analysis on the identified abnormal indicators and comprehensively consider the potential correlation between multiple abnormal factors, so as to accurately depict the overall picture of the system bottleneck. Based on this comprehensive evaluation, the system can automatically search the tuning strategy library, match the most appropriate recommendation index for each bottleneck situation, and organize these indexes into a list form. Finally, the system extracts specific tuning suggestions and detailed operation guides from the configuration file based on the index list, providing users with a scientific and feasible optimization path.

[0069] In this embodiment, the key indicators include the system bottom layer key indicators and the system upper layer key indicators. Each system bottom layer key indicator corresponds to one or more system upper layer key indicators. When diagnosing bottlenecks, the key indicators of the system bottom layer are first judged (for example, network interface utilization). Specifically, the current system bottom layer key indicators are compared with the corresponding indicator thresholds and the current system bottom layer key indicators are judged based on the comparison results. Whether the abnormal indicator is an abnormal indicator, the upper layer indicator (for example, the total number of error packets received per second) is judged. Specifically, the corresponding system upper layer key indicators are selected and compared with the indicator thresholds corresponding to the selected system upper layer key indicators, and whether the selected system upper layer key indicators are abnormal is judged based on the comparison results. If several abnormal indicators exist at the same time, the corresponding bottleneck and tuning suggestions are obtained according to the bottleneck diagnosis logic. The bottleneck diagnosis logic comes from actual tuning experience and is embedded in the program code.

[0070] The intuitive abnormality display and guidance module presents abnormal indicators and tuning suggestions to users in an intuitive and easy-to-understand way. The display content not only covers the specific information of the abnormal indicators (such as name, value, meaning and source), but also lists the tuning suggestions, operation steps and the logic behind them in detail. In addition, the system also introduces highlighting and hovering explanation functions to visually enhance abnormal data and highlight the indicator name. Users only need to hover to get detailed explanations, which greatly improves the convenience and efficiency of the tuning process.

[0071] In terms of system bottleneck analysis, the thresholds of existing automated tuning methods are usually static and difficult to adjust according to actual conditions, or they only rely on a single indicator to judge the system bottleneck. The cause-effect relationship is relatively direct, and there is a lack of comprehensive judgment of complex system indicators, which makes it difficult to apply to complex scenario tuning. Some existing inventions even require optimized systems or mature tuning solutions as reference comparisons, and lack the ability to tune scenarios that have never been encountered. In addition, tuning methods based on algorithms such as machine learning and neural networks require a lot of training and time-consuming system data collection, which has high technical and time costs.

[0072] When performing bottleneck analysis, the present invention incorporates a large amount of manual tuning experience into the code, and specifically analyzes different loads and bottlenecks according to different indicators, without the need for existing tuning solutions as a comparative reference, and is applicable to a wider range of scenarios; secondly, it supports threshold adjustment in the configuration file, and different indicator thresholds can be adjusted according to different scenarios to adapt to complex and changeable scenarios; in addition, for the analysis of system bottlenecks, the present invention comprehensively considers the bottlenecks that appear in different software and hardware indicators, and makes a systematic judgment, which is more suitable for tuning in complex scenarios; and the present invention does not require a large number of training processes, and in actual scenarios it also greatly reduces the time consumption for collecting system data, and is more suitable for application scenarios with strict time constraints.

[0073] For example, when monitoring the system disk performance, if it is detected that iowait (IO wait time ratio) and util (disk I / O utilization rate) are both high, further analysis is performed. If wkB / s (bytes written per second) is also high at this time, it means that the system business is mainly focused on disk activities, and frequent disk writes may become a system bottleneck. Based on this, the system will recommend "adjusting the dirty data refresh configuration" to speed up disk writes and improve overall system performance. The present invention cleverly integrates such complex logical relationships into the code, covering a variety of indicators, bottleneck information and tuning strategies for different software and hardware environments, and constructs a complex logical network to flexibly respond to performance tuning requirements in various scenarios.

[0074] In actual tuning scenarios, we will focus on different system indicator data according to different business types, including real-time CPU occupancy, real-time NUMA status, real-time network traffic status, real-time disk I / O read and write status, kernel parameter statistics, system resource configuration, system real-time logs and other indicator data. When there are abnormalities in these indicators, we will further focus on the corresponding system indicators.

[0075] For example, when the disk IO usage (util) is high, the corresponding tuning suggestions are made and the disk performance indicators are further monitored; if it is detected that the CPU is waiting for IO operations (iowait) for a long time, it indicates that disk reading and writing may be the reason affecting system performance, and corresponding tuning suggestions are given, and further judgment is made on which aspect of reading / writing affects system performance; if there is a large amount of disk writing (wkB / s) at this time, and the disk reading speed is not high, it indicates that frequent disk writing may become a system bottleneck. Through in-depth analysis of system performance indicators, the bottleneck can be accurately located, thereby providing users with intuitive tuning guidance and implementing optimization measures efficiently.

[0076] These indicators are widely present in the collected system information. When performing system tuning, key indicators are selected based on the significance of different indicators and the relationship between them, either complementing each other or from rough to specific. However, system tuning is a complex process. The above is just a simple example. In fact, the logical relationship between various indicators during bottleneck analysis is intricate, and there are cross-cutting and reuse situations. Based on previous expert tuning experience, the present invention embeds these logical judgment statements that describe the correlation and progressive relationship between different indicators into the code logic, thereby giving effective tuning suggestions based on the system bottlenecks reflected by different indicators.

[0077] Therefore, the automatic bottleneck analysis and expert tuning guidance in this embodiment specifically include the following steps:

[0078] Step 3: Get key indicator data. Based on existing tuning experience, get key indicators and data from different collected txt files, and store these indicators and related data into the designed key indicator data structure based on the pre-built tuning knowledge base, including indicator identification name, indicator name, indicator value, acquisition command, indicator meaning, and several key-value pairs. The indicator value also includes the specific value of the indicator, row index, column index, device and other information for subsequent abnormal indicator identification and display.

[0079] Step 4: Abnormal indicator identification. Based on existing tuning experience and survey data, the thresholds of key indicators are organized and written into the configuration file. While obtaining the key indicator values, the configuration file is read to obtain the corresponding thresholds, and the obtained key indicator values ​​are compared with the thresholds. When they exceed the normal range of the threshold limit, they are stored in the abnormal data structure or the highlight data structure. The highlight data structure is basically the same as the previous key indicator data structure, and is used for highlighting abnormal indicators in the report later, making the display of abnormal indicators more intuitive. In addition to the content contained in the key indicator data storage structure, the abnormal data structure also includes a Boolean type variable: whether to display the indicator, which is used to control whether the indicator is displayed in the abnormal indicator display part. Because bottleneck analysis in actual scenarios is very complex, when multiple abnormal indicators exist at the same time, it is judged that the current system has certain bottlenecks. Therefore, only when it is comprehensively judged that a bottleneck exists, the variable will be set to True, and then the corresponding abnormal indicator will be displayed, providing users with a clear tuning reference. Therefore, before displaying the abnormal key indicators and tuning suggestions, this embodiment also includes: storing the abnormal key indicators in the abnormal data structure, and if the system performance bottleneck and tuning suggestions are matched according to the abnormal key indicators, updating the display mark in the abnormal data structure to the specified value to display the abnormal key indicators.

[0080] Step 5: Tuning suggestions and guidance. This part makes a comprehensive judgment on the abnormal indicator data obtained according to the bottleneck judgment rules. The rules are formed based on previous tuning experience and tuning materials. The rules are integrated into the code, and the abnormal indicators are combined through logical judgment statements that describe the causal relationship, correlation and progressive relationship between different indicators. When multiple abnormal indicators in the abnormal indicator combination exist at the same time, the corresponding system bottleneck description is obtained according to the mapping relationship between these abnormal indicators and the system bottleneck, and the corresponding suggestion index is obtained according to the bottleneck description, and then the suggestion index is stored in the list. Finally, the configuration file is read according to the suggestion index list to obtain the corresponding suggestions and operation instructions. Therefore, in this embodiment, when matching system performance bottlenecks and tuning suggestions based on abnormal key indicators, it includes: if the number of abnormal key indicators that exist at the same time is greater than the preset value, traverse all abnormal key indicators, and according to the aforementioned logical judgment statement, select abnormal key indicators with associated relationships, and then progressively screen the remaining abnormal key indicators that have a progressive relationship with the selected abnormal key indicators, and finally generate an abnormal key indicator combination, match the abnormal key indicator combination indicator with the abnormal key indicator and bottleneck description mapping relationship corresponding to the combination in the preset bottleneck diagnosis logic, derive the corresponding system bottleneck description, and match the system bottleneck description with the preset tuning strategy library to obtain the corresponding suggestion index, and finally obtain specific tuning suggestions according to the suggestion index.

[0081] Step 6: Display of abnormal indicators and suggestions. This part comprehensively displays the abnormal indicators and tuning suggestions obtained in steps 4 and 5, providing guidance for scene tuning, such as Figure 2 , Figure 3 shown. Figure 2 A partial schematic diagram showing bottleneck information, including specific indicators, values, meanings and sources. Figure 3 This is a diagram of tuning suggestions and operation instructions, including specific suggestions, operation instructions, and tuning instructions. Tuning suggestions are targeted suggestions for the current system bottlenecks. Operation instructions are practical operation steps, and additional instructions are provided for the operation instructions so that users can understand the tuning steps and reasons more clearly. In addition, the abnormal display part also includes highlighting of abnormal data. According to the highlighted data obtained in step 3, the corresponding indicator values ​​are highlighted in the corresponding collection information display part, and the key indicators are highlighted and the mouse hover explanation function is added to provide users with references and facilitate the tuning process.

[0082] The predefined tuning of this embodiment is a set of predefined parameter configurations applied when the system service is started. When the system performance bottleneck is difficult to locate through dynamic monitoring and tuning and the system scenario resources are limited, the predefined tuning function can be used to improve the application scenario performance with minimal resource consumption.

[0083] In this embodiment, the configuration files corresponding to the preset scenario information include system optimization configuration solutions for five common application scenarios based on Kylin operating system, including MySQL database, DM database, Kafka middleware, http cache scenario Traffic Server, and http performance scenario SPECweb 2009. The configuration files are all general performance parameters integrated by system tuning engineers based on the background of deployment and actual application scenario cases on Kylin operating system, combined with the characteristics of application services and the security and reliability characteristics of Kylin operating system.

[0084] In this embodiment, the scenario optimization configuration scheme of the configuration file includes the optimization configuration of the system general parameter configuration file sysctl.conf and each application configuration file. When the user applies the predefined tuning, the tool prompts the user to further check the configuration scheme to adapt to the current environment, and prompts the user to restart to make the configuration scheme effective. The complete predefined tuning function design aims to lower the threshold for ordinary users to perform professional tuning.

[0085] The sub-functions of predefined tuning in this embodiment include the function of issuing scene parameter configuration files, the recovery function, and the scene list function. The issuing function includes configuration inquiry to the user, configuration issuing, and configuration backup. Configuration issuing makes one-click changes to the scene application configuration file and the system configuration file sysctl.conf. The configuration parameters are identified with extuner-related identifiers to distinguish them from the parameters issued by this function and the original parameters of the environment, so as to achieve the purpose of parameter backup and restoration. The scene list function shows the five major application scenarios supported by the current predefined tuning function. All return information of the function will be displayed on the terminal in the form of standard output and recorded in the log file.

[0086] Therefore, the steps of pre-defining the optimization in this embodiment specifically include:

[0087] Step 7: Read the scene name and the directory where its application configuration file is located from the command line. Check whether the scene name is supported, check whether the directory where the application configuration file is located exists, and whether the directory contains the corresponding configuration file.

[0088] Step 8: Further confirm the delivery to the user. The predefined tuning configuration scheme provided by the tool may have uncertainties due to version differences. The user needs to further confirm whether to deliver the configuration. If the configuration is not delivered, this function is terminated.

[0089] Step 9: When you run the predefined tuning for the scene for the first time, the configuration file of the scene application will be backed up. The backup file is stored in the corresponding scene directory in the extuner / profile / bak directory for subsequent restore functions. If you run the predefined tuning for the scene multiple times, the data will not be backed up again.

[0090] Step 10: Send the application configuration part of the expert tuning configuration plan to the system environment. If the scenario is MySQL, you need to put the parameter values ​​under each section in the expert configuration plan into the corresponding section of my.cnf. If the section does not exist in my.cnf, create a new section and put the parameter values. If the scenario is DM, Kafka, TrafficServer, SPECweb2009, you can add the expert tuning parameters directly to the end of the file. When the application service reads in sequence, it will automatically overwrite the previously configured parameter values. If the expert tuning parameters are written for the first time, they are directly appended to the end of the configuration file, and the start and end lines of the added parameters are marked with extuner comment lines. If the expert tuning parameters are written multiple times, that is, the extuner-related parameters already exist in the file, the content in the extuner comment mark line will be replaced.

[0091] Step 11: Send the system configuration part of the expert tuning configuration plan to the system environment. Append the expert tuning parameters directly to the end of the sysctl.conf file. If the expert tuning parameters are written for the first time, append them directly to the end of the configuration file and use extuner comment lines to mark the start and end lines of the added parameters. If the expert tuning parameters are written multiple times, that is, the extuner related parameters already exist in the file, the content in the extuner comment line will be replaced. Then run the sysctl-p command to make the system configuration parameters effective.

[0092] Step 12: The standard output prompts the user to restart the application service to take effect the current application configuration plan.

[0093] In this embodiment, real-time dynamic tuning is mainly used to perform real-time dynamic tuning in any scenario configured by the user to infer the optimal configuration parameter combination. Using the Bayesian optimization algorithm in the dynamic inference of the optimal configuration parameters can converge faster in the direction of optimizing system performance indicators, so that the optimal parameter configuration combination can be inferred within the number of tuning times or tuning duration set by the user, so that application performance can be improved.

[0094] The configuration file of real-time dynamic tuning supports configuration of parameters such as number of iterations, tuning duration, commands for obtaining and issuing parameters to be adjusted, parameter type, parameter adjustment range, parameter step, benchmark script, benchmark performance indicator and its optimization direction, and expected performance improvement ratio. It supports setting one performance indicator and multiple parameters to be adjusted. The real-time dynamic tuning function supports finding the optimal performance configuration combination within the current number of iterations, or obtaining the optimal performance configuration combination within the specified duration. At the same time, if the expected performance improvement ratio is achieved in advance during the tuning project, the tuning can be terminated in advance.

[0095] The real-time dynamic tuning function is divided into the configuration file module, the environment configuration module, the optimizer module, and the output module. The configuration file module includes the legitimacy check and read backup of the configuration file provided by the user, including the parameters to be adjusted, benchmarks, tuning indicators and other related information, to ensure that the relevant information required by the Bayesian optimization algorithm can be correctly read. The environment configuration module includes the steps of backing up the initial system environment, issuing new parameters in the system environment, and running the tuning script to obtain performance indicators, etc., which are used to prepare the dynamic tuning data required by the optimization algorithm. The optimizer module is responsible for obtaining new parameters using the Bayesian algorithm, handing them over to the environment configuration module for issuing parameters and returning new performance indicator data. The Bayesian optimization algorithm uses the Bayesian theorem. At each iteration, the historical tuning performance information recorded previously is used to infer the next performance optimization point, so as to find the optimal value of the performance indicator value faster. The output module is responsible for outputting the tuning results to the standard output and recording them in the tuning result file.

[0096] Therefore, the steps of real-time dynamic tuning in this embodiment include:

[0097] Step 13: Read the relevant tuning information in the user configuration file. This includes the number of tuning iterations, tuning duration, commands for obtaining and issuing parameters to be adjusted, parameter type, parameter adjustment range, parameter step, benchmark script, benchmark performance indicators and optimization direction, expected performance improvement ratio, etc. Check the integrity of the configuration file provided by the user. If there are unreasonable parameter values, the user will be prompted to modify them.

[0098] Step 14: System environment backup. Back up the initial values ​​of the parameters to be adjusted for system restoration after tuning.

[0099] Step 15: Run the benchmark and record the initial environment performance baseline value. Also record the duration of the benchmark run to determine the overall tuning duration.

[0100] Step 16: Run the Bayesian optimization algorithm to obtain the optimal parameter value. If the user sets the tuning duration, it must first be converted into the number of tuning times. The benchmark running time accounts for the majority of the single tuning time, and other steps take less time and can be ignored. Therefore, the tuning duration divided by the single tuning benchmark duration can be regarded as the required number of tuning times. Within the limited number of tuning times, according to the Bayesian theorem, the next optimal parameter point is selected from the known tuning records, and more accurate optimal parameter points are obtained through continuous tuning.

[0101] Step 17: Restore the system environment. Restore the environment using the data backed up in step 14.

[0102] Step 18: Output the tuning results to the current directory, the tuning result file.

[0103] In this embodiment, encryption tuning realizes deep obfuscation and encryption processing of tuning files, and one-click delivery of obfuscated and encrypted tuning file contents. The overall architecture design is as follows: Figure 4 shown.

[0104] In the parameter obfuscation stage, the present invention integrates three obfuscation strategies of replacement, insertion and addition to construct a multi-level confusion matrix. The replacement strategy dynamically replaces the qualified strings in the file by randomly selecting n1 preset replacement rules, effectively hiding the real tuning parameters. The insertion strategy randomly selects n2 strings to be inserted and embeds them into random positions of existing parameters, further improving the degree of obfuscation. The new addition strategy randomly selects n3 parameters from the preset obfuscation parameter library, accurately inserts them into random positions in the file, and ensures that the original function of the tuning file is not interfered with. This series of randomized obfuscation operations not only greatly enhances the randomness and unpredictability of the obfuscated parameters, but also successfully hides the original tuning strategy in the complex obfuscation parameters, effectively preventing the leakage of the tuning strategy. Subsequently, the system will output the obfuscated file content and give it a specific extension for subsequent restoration operations.

[0105] In the deobfuscation stage, the present invention first uses intelligent recognition technology to accurately distinguish between ordinary files and obfuscated files. Subsequently, the obfuscated file is operated, and the system uses a reverse algorithm to gradually restore it to the original tuning file. This process includes deleting all obfuscation parameters, removing inserted strings, and restoring replaced strings to ensure that the content of the deobfuscated file is completely consistent with the original file and can be directly executed without any modification. Finally, the system executes the deobfuscated file content, implements the distribution of tuning parameters and other tuning operations in the current system environment, and ensures the smoothness and efficiency of the tuning process.

[0106] In terms of encryption, the present invention adopts symmetric key encryption technology, and the key is cleverly embedded in the code and is completely invisible to the user. This encryption mechanism is not only used for data encryption and decryption, but also generates a message authentication code to ensure the integrity and authenticity of the data during transmission, effectively preventing data tampering and leakage. After encryption is completed, the real file content is completely hidden and saved with a specific extension, greatly improving the security of the data. The decryption process uses the same key to perform reverse operations, restore the encrypted file to the original content, and store it in the cache, and then execute the file content, ensuring the seamless flow and efficient use of data.

[0107] In the existing system tuning methods, it is common to directly issue and adjust system parameters, and there is a lack of research on the secure transmission of tuning files (including tuning parameters and methods). In actual tuning scenarios, R&D personnel cannot directly operate the tuning system, so the secure transmission of tuning files and the efficient issuance of tuning means are very important. In the present invention, the obfuscation operation is to confuse the parameters of the plaintext file, and the encryption operation is to directly encrypt the file content into ciphertext. The dual method completely hides the file content and enhances the security of the content during file transmission. In addition, the present invention supports separate obfuscation or encryption, and supports one-click restoration and execution of encrypted and obfuscated files, which ensures the security of the file while taking into account the efficiency of the tuning process.

[0108] Therefore, the specific steps of encryption tuning in this embodiment include:

[0109] Step 19: The obfuscation process is to perform obfuscation operations on the parameter modification statements starting with "sysctl-w" in the tuning file provided by the user: including three operations: replacement, insertion, and addition. The replacement operation is to randomly replace the string, randomly select n1 rules from the established replacement rules, and then replace the strings that meet the rules in the file to hide the actual tuning parameters; the insertion operation is to randomly select n2 strings to be inserted and insert them at random positions in the existing parameters; the addition operation is to randomly select n3 parameters from the established obfuscation parameters and insert them at random positions in the file without destroying the existing functions of the tuning file and affecting the execution of other statements. After the obfuscation operation, the file content is output and modified to the specified extension to distinguish the obfuscated file for restoration. The obfuscation operation can hide the original tuning parameters in the obfuscated parameters to prevent the leakage of tuning methods.

[0110] Step 20: The deobfuscation process first determines the file type to distinguish between ordinary files and obfuscated files; then, the obfuscated file is operated to restore it to the initial tuning file, including deleting obfuscation parameters, deleting inserted strings, and restoring replacement strings. After deobfuscation, its content is exactly the same as the original file content and can be executed directly; finally, the deobfuscated file content is executed, the tuning parameters are issued on the current system, and other tuning operations are performed.

[0111] Step 21: The file encryption and decryption process uses symmetric key encryption. The key is hidden in the code and invisible to the user. It is used to encrypt and decrypt data, and generate a message authentication code (MAC) to ensure data integrity and prevent data from being tampered with during transmission. After encryption, the real file content is completely hidden and saved with a specified extension.

[0112] Step 22: The decryption process uses the same key to decrypt, restore the encrypted file to the original file content, store it in the cache, and execute the file content, so that the original tuning file content is directly effective in the current system. In the actual tuning scenario, the R&D personnel can use the encryption function to hide the tuning file content, and then use the present invention to send the encrypted file in the application environment with one click, which greatly simplifies the tuning operation process while ensuring security.

[0113] Embodiment 2

[0114] This embodiment provides a performance tuning system for a Linux system, including a microprocessor and a computer-readable storage medium connected to each other, wherein the microprocessor is programmed or configured to execute the performance tuning method for the Linux system described in the first embodiment.

[0115] In summary, the present invention proposes a lightweight, cross-platform, multi-scenario Linux system performance tuning method and system, which provides basic functions including full-quantity index collection, automatic bottleneck analysis and expert tuning guidance, and supports additional functions such as predefined tuning, real-time dynamic tuning, and encryption tuning. It can assist users to comprehensively and deeply analyze system performance bottlenecks and provide guidance and suggestions; based on expert experience and combined with Bayesian optimization algorithm, it provides users with the best parameter configuration in actual application scenarios; it provides encryption / obfuscation and one-click distribution of encrypted / obfuscated file contents, effectively preventing the leakage of tuning means, simplifying the operation process, and ensuring the efficiency and security of work.

[0116] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A performance tuning method for a Linux system, characterized in that: The following steps are involved: Get system information and generate corresponding text files; Extracting key indicators from the text file, reading indicator thresholds, and then comparing the key indicators with corresponding indicator thresholds to determine whether there are abnormal key indicators; If there are abnormal key indicators, match the system performance bottlenecks and tuning suggestions based on the abnormal key indicators, and then display the abnormal key indicators and tuning suggestions; Wait for and obtain the tuning file, and perform tuning operations based on the contents of the tuning file.

2. The performance tuning method of the Linux system according to claim 1, characterized in that: When obtaining system information, including: According to user configuration, obtain the corresponding system configuration information; Use the specified system indicator collection command to collect the corresponding system indicator information; Use the perf tool to collect performance event information and statistics to capture hot functions in the system.

3. The performance tuning method of the Linux system according to claim 1, characterized in that: When extracting key indicators from the text file, specifically, indicators reflecting software and hardware performance determined based on tuning experience are extracted from the text file as key indicators based on one or more of the collection type, indicator name, row and column information, and device information, and the key indicators and their related data are stored in a key indicator data structure based on a pre-built tuning knowledge base.

4. The performance tuning method of the Linux system according to claim 1, characterized in that: The key indicators include system bottom-level key indicators and system upper-level key indicators. Each system bottom-level key indicator corresponds to one or more system upper-level key indicators. When comparing the key indicators with the corresponding indicator thresholds, it specifically includes: comparing the current system bottom-level key indicators with the corresponding indicator thresholds and judging whether the current system bottom-level key indicators are abnormal based on the comparison results; if yes, selecting the corresponding system upper-level key indicators to compare with the indicator thresholds corresponding to the selected system upper-level key indicators and judging whether the selected system upper-level key indicators are abnormal based on the comparison results.

5. The performance tuning method of the Linux system according to claim 1, characterized in that: When matching system performance bottlenecks and tuning suggestions based on abnormal key indicators, it includes: If the number of abnormal key indicators that exist at the same time is greater than the preset value, traverse all abnormal key indicators, select abnormal key indicators with associated relationships according to the aforementioned logical judgment statements, and then progressively screen the remaining abnormal key indicators that have a progressive relationship with the selected abnormal key indicators, and finally generate an abnormal key indicator combination, match the abnormal key indicator combination indicator with the abnormal key indicator and bottleneck description mapping relationship corresponding to the combination in the preset bottleneck diagnosis logic, derive the corresponding system bottleneck description, and match the system bottleneck description with the preset tuning strategy library to obtain the corresponding suggestion index, and finally obtain specific tuning suggestions based on the suggestion index.

6. The performance tuning method of the Linux system according to claim 1, characterized in that: Before displaying abnormal key indicators and tuning suggestions, it also includes: The key indicators of the exception are stored in the exception data structure. If the system performance bottleneck and tuning suggestions are matched according to the key indicators of the exception, the display mark in the exception data structure is updated to the specified value to display the key indicators of the exception.

7. The performance tuning method of the Linux system according to claim 1, characterized in that: After waiting for and obtaining the tuning file, the process further includes performing an obfuscation operation and / or encryption operation on the tuning file, wherein the encryption operation specifically uses a symmetric key to encrypt the tuning file and generate a message authentication code to ensure the integrity and authenticity of the data during transmission. The obfuscation operation step includes: Randomly select n1 preset replacement rules to replace the strings that meet the conditions in the tuning file to hide the actual tuning parameters; Randomly select n2 strings to be inserted and embed them into random positions of existing parameters to improve obfuscation; Randomly select n3 parameters from the preset obfuscation parameter library and insert them into random positions in the tuning file; Output the obfuscated file content and change it to the specified extension.

8. The performance tuning method of the Linux system according to claim 1, characterized in that: It also includes steps for performing predefined tuning, including: Match the current application scenario with the preset scenario information and the corresponding configuration file. If the match is successful, send a confirmation request to the user. Wait for and obtain the user's confirmation information, write the parameters in the configuration file to the system environment, and restart to make the configuration file take effect.

9. The performance tuning method of the Linux system according to claim 1, characterized in that: It also includes the steps of real-time dynamic tuning, including: Get the tuning information in the configuration file and back up the system environment. Then run the Bayesian optimization algorithm based on the tuning information to get the optimal configuration parameter combination. Finally, restore the system environment.

10. A performance tuning system for a Linux system, characterized in that: The invention comprises a microprocessor and a computer-readable storage medium connected to each other, wherein the microprocessor is programmed or configured to execute the performance tuning method of the Linux system according to any one of claims 1 to 9.

Images