A server performance optimization method based on big data analysis

By collecting and analyzing multi-dimensional server data in real time, dynamically adjusting the amount of data read, identifying and optimizing performance bottlenecks, the problem of data read and processing imbalance and poor module coordination in traditional server optimization methods has been solved, achieving efficient server performance optimization.

CN121166366BActive Publication Date: 2026-04-10QINGDAO DONGSHUNCHIXIN TECHNOLOGY CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
QINGDAO DONGSHUNCHIXIN TECHNOLOGY CO LTD
Filing Date
2025-09-22
Publication Date
2026-04-10

AI Technical Summary

Technical Problem

Traditional server performance optimization methods cannot dynamically adjust the amount of data read according to real-time demand, resulting in an imbalance between data reading and processing rhythm, affecting the timeliness of performance bottleneck identification. Furthermore, the fragmented division of functional modules and the delay in data flow reduce the efficiency and accuracy of server performance optimization.

Method used

By collecting multi-dimensional operational data from the server in real time, storing it in a cache queue according to priority, and statistically analyzing queue status parameters in real time, the data read volume is dynamically calculated, performance bottleneck types are identified, initial optimization strategies are matched, and final optimization strategies are formed to achieve dynamic adjustment of the data read volume.

Benefits of technology

It improves the collaborative efficiency of server performance optimization, ensures a balance between data reading and processing, accurately identifies performance bottlenecks, optimizes resource utilization, and enhances overall performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121166366B_ABST
    Figure CN121166366B_ABST
Patent Text Reader

Abstract

The application relates to the technical field of server performance optimization, in particular to a server performance optimization method based on big data analysis, which comprises the following steps: collecting multi-dimensional running data of a server in real time, storing the running data of the server in a cache queue according to priorities, and statistically processing queue state parameters in real time; collecting real-time resource states of a data processing server and server optimization stage data, calculating a data reading amount, determining a current batch reading amount according to a calculation result, identifying a server performance bottleneck type, and outputting a bottleneck identification result; matching an initial optimization strategy according to the bottleneck identification result, obtaining an initial optimization strategy for real-time running data of the server, and obtaining a final optimization strategy; and through the above mode, the data reading amount is dynamically adjusted, so that the collaborative efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of server performance optimization, in particular to a server performance optimization method based on big data analysis. BACKGROUND

[0002] With the rapid development of Internet business, servers need to handle massive and dynamically changing business requests, and traditional server performance optimization methods have significant defects in data processing efficiency and resource adaptability.

[0003] On the one hand, in the prior art, when reading data from the data aggregation module cache queue, a fixed batch reading strategy is usually used, which cannot be dynamically adjusted according to the real-time optimization requirements of the server, the data backlog of the cache queue and the hardware processing capacity, resulting in an imbalance between data reading and processing rhythm. When the cache queue data is seriously accumulated, fixed small batch reading will prolong the data processing cycle and affect the timeliness of performance bottleneck identification; when the cache queue data is small and the server is in a high-load optimization state, fixed large batch reading will occupy too many CPU and memory resources, and aggravate the server performance burden. On the other hand, the functional modules of the traditional optimization scheme are relatively scattered, and there is a lack of close cooperation between the data aggregation, data reading and strategy generation and execution modules, and the data flow is delayed, further reducing the efficiency and accuracy of server performance optimization.

[0004] Therefore, it is necessary to propose a server performance optimization method capable of dynamically adjusting the data reading amount to improve the cooperation efficiency. SUMMARY

[0005] The application aims to provide a server performance optimization method based on big data analysis, which aims to dynamically adjust the data reading amount to improve the cooperation efficiency.

[0006] To achieve the above-mentioned purpose, the application adopts a server performance optimization method based on big data analysis, which comprises the following steps:

[0007] Real-time collection of multi-dimensional server running data, storage of server running data in a cache queue according to priority, and real-time statistics of queue state parameters;

[0008] Collecting real-time resource state and server optimization stage data of the data processing server, calculating the data reading amount, determining the current batch reading amount according to the calculation result, identifying the type of server performance bottleneck, and outputting the bottleneck identification result;

[0009] According to the bottleneck identification result, an initial optimization strategy is matched to obtain the initial optimization strategy of the real-time running data of the server, and a final optimization strategy is obtained.

[0010] In the step of collecting multi-dimensional running data of the server in real time, storing the running data of the server according to priorities by using a cache queue, and statistically counting queue state parameters in real time:

[0011] Deploying a lightweight collection agent component at each node of the server, collecting multi-dimensional running data of the server in real time, and encapsulating the multi-dimensional running data;

[0012] Filtering effective data from the multi-dimensional running data;

[0013] Storing the effective data into the cache queue, setting data storage priorities according to dual dimensions of data collection time stamps and data importance, and orderly storing the data.

[0014] In the step of filtering effective data from the multi-dimensional running data:

[0015] Removing invalid data from the multi-dimensional running data according to preset rules and retaining effective data; wherein the preset rules include eliminating null data, filtering the same time stamp data collected repeatedly, and filtering effective data meeting data format requirements.

[0016] After the step of storing the effective data into the cache queue, setting data storage priorities according to dual dimensions of data collection time stamps and data importance, and orderly storing the data:

[0017] Statistically counting current data total amount, data average generation rate, and queue remaining storage space of the cache queue in real time.

[0018] In the step of collecting real-time resource state and server optimization phase data of the data processing server, calculating data reading amount, determining current batch reading amount according to the calculation result, performing server performance bottleneck type identification, and outputting bottleneck identification result:

[0019] Collecting resource state data of the data processing server in real time, and obtaining the optimization phase in which the server is located; wherein the optimization phase includes a bottleneck identification phase, a strategy execution phase, and an effect evaluation phase;

[0020] Reading cache queue state parameters, and setting corresponding weight coefficients according to different optimization phases;

[0021] According to the weight coefficients, sequentially calculating pressure conditions of the cache queue and resource bearing capacity of the data processing server, and combining preset basic reading amount to calculate current adaptive batch reading amount;

[0022] Reading data from the cache queue according to the batch reading amount, and storing the data into a distributed database;

[0023] Analyze the correlation between each performance indicator, and combine historical data trend prediction to identify the type of server performance bottleneck, and generate a bottleneck identification result.

[0024] In the step of reading the cache queue state parameter and setting the corresponding weight coefficient according to different optimization stages:

[0025] If the optimization stage is the bottleneck identification stage, the data timeliness is emphasized, and the weight coefficient is inclined to the data acquisition efficiency;

[0026] If the optimization stage is the strategy execution stage, the resource saving is emphasized, and the weight coefficient is inclined to the server resource occupation;

[0027] If the optimization stage is the effect evaluation stage, the data timeliness and resource saving are balanced, and the weight coefficient is evenly distributed.

[0028] After the step of reading data from the cache queue according to the batch reading amount and storing the data to the distributed database:

[0029] Perform data cleaning, data integration, and data conversion preprocessing operations.

[0030] In the step of analyzing the correlation between each performance indicator, and combining historical data trend prediction to identify the type of server performance bottleneck, and generating a bottleneck identification result:

[0031] The types include hardware resource bottleneck: insufficient CPU resources, insufficient memory; software performance bottleneck: low application code efficiency, insufficient database query optimization; business load bottleneck: sudden increase in user access leading to insufficient request processing capacity.

[0032] In the step of matching the initial optimization strategy according to the bottleneck identification result, obtaining the server real-time running data optimization initial optimization strategy, and obtaining the final optimization strategy:

[0033] Obtain the bottleneck identification result, and filter out the initial optimization strategy matching the current bottleneck;

[0034] Real-time acquisition of server running data, adjustment and optimization of initial strategy parameters, and formation of final optimization strategy.

[0035] Before the step of obtaining the bottleneck identification result, and filtering out the initial optimization strategy matching the current bottleneck:

[0036] Build an optimization strategy knowledge base to store pre-set optimization strategies for different performance bottleneck types and historical optimization case data.

[0037] The application discloses a server performance optimization method based on big data analysis. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only represent some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0039] Figure 1 is a step flow chart of the server performance optimization method based on big data analysis of the present application.

[0040] Figure 2 is a step flow chart of S100 of the present application.

[0041] Figure 3 is a step flow chart of S200 of the present application.

[0042] Figure 4 is a step flow chart of S300 of the present application.

[0043] Figure 5 is a structure principle diagram of the server performance optimization system based on big data analysis of the present application.

[0044] Figure 6 is a structure principle diagram of the electronic device of the present application.

[0045] 401-server running data aggregation module, 402-data reading amount dynamic adjustment module, 403-optimization strategy generation and execution module. DETAILED DESCRIPTION

[0046] The exemplary embodiments will be described in detail herein with reference to the drawings. Unless otherwise specified, the same or similar components are denoted by the same reference numerals, and the same or similar components in different drawings are denoted by the same reference numerals. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application.

[0047] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in this application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0048] It is to be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, a first information can be termed a second information, and similarly, a second information can also be termed a first information, without departing from the scope of the present application. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining" or "in response to a determination".

[0049] Referring to Figures 1-4 The present application provides a server performance optimization method based on big data analysis, comprising the following steps:

[0050] S100: Real-time collection of server multi-dimensional running data, storage of server running data by priority using a cache queue, and real-time statistics of queue state parameters.

[0051] In the present embodiment, real-time collection of server multi-dimensional running data, storage of server running data by priority using a cache queue, and real-time statistics of queue state parameters. The specific process is as follows:

[0052] S101: Deploying a lightweight collection agent component at each node of the server, real-time collection of server multi-dimensional running data, and encapsulation of the multi-dimensional running data;

[0053] S102: Removing invalid data in the multi-dimensional running data according to a preset rule, and retaining valid data; wherein the preset rule includes eliminating null data, filtering the same timestamp data collected repeatedly, and screening valid data meeting the data format requirements;

[0054] S103: Storing the valid data into the cache queue, setting data storage priority according to the data collection timestamp and data importance in a dual dimension, and performing ordered storage of the data;

[0055] S104: Real-time statistics of the current data total amount, data average generation rate, and queue remaining storage space of the cache queue.

[0056] In the above process, a lightweight collection agent component is deployed on each node of the server, which supports multi-thread concurrent collection mode and can adapt to server nodes of different architectures. The component collects multi-dimensional running data of the server in real time, including hardware performance data (CPU usage, memory occupancy, disk IO rate, network bandwidth utilization, etc.), software running data (application response time, database query time, operating system process resource occupancy, log error information, etc.), and business load data (user access volume per unit time, request concurrency, data transmission volume, business transaction success rate, etc.).

[0057] The collection agent component uniformly encapsulates the collected various data in a preset JSON format to ensure the data structure specification, and then transmits the encapsulated data through a communication module supporting TCP / IP and HTTP dual protocols. The received data is preliminarily processed according to preset rules (such as excluding null data, filtering duplicate collection of the same timestamp data, and screening valid data meeting the data format requirements) to remove invalid data and retain valid data.

[0058] The filtered valid data is transmitted into a cache queue, which adopts a priority queue structure to set data storage priorities according to data collection timestamps (the newer the timestamp, the higher the priority) and data importance (hardware performance data > software running data > business load data), thereby completing ordered storage of the data.

[0059] The running state of the cache queue is continuously monitored to real-time calculate and record the current total amount of data in the queue, the average generation rate of data (the number of newly added data per unit time), and the remaining storage space of the queue (the maximum number of data that can be accommodated), which can provide a basis for dynamic calculation of the data reading amount.

[0060] S200: Collect real-time resource status and server optimization phase data of the data processing server, calculate the data reading amount, determine the current batch reading amount according to the calculation result, identify the server performance bottleneck type, and output the bottleneck identification result.

[0061] In this embodiment, the real-time resource status and server optimization phase data of the data processing server are collected, the data reading amount is calculated, the current batch reading amount is determined according to the calculation result, the server performance bottleneck type is identified, and the bottleneck identification result is output. The specific process is as follows:

[0062] S201: Collect resource status data of the data processing server in real time, and obtain the optimization phase of the server; wherein the optimization phase includes a bottleneck identification phase, a strategy execution phase, and an effect evaluation phase;

[0063] S202: Read the cache queue state parameters, set the corresponding weight coefficients according to different optimization stages; if the optimization stage is the bottleneck identification stage, focus on data timeliness, and the weight coefficient is inclined to data acquisition efficiency; if the optimization stage is the strategy execution stage, focus on resource saving, and the weight coefficient is inclined to server resource occupation; if the optimization stage is the effect evaluation stage, balance data timeliness and resource saving, and the weight coefficient is evenly distributed;

[0064] S203: According to the weight coefficient, the pressure condition of the cache queue and the resource carrying capacity of the data processing server are calculated in turn, and the current adaptive batch reading quantity is calculated combined with the preset basic reading quantity;

[0065] S204: Read data from the cache queue according to the batch reading quantity, and store the data to the distributed database;

[0066] S205: Perform preprocessing operations such as data cleaning, data integration, and data conversion;

[0067] S206: Analyze the correlation between performance indicators, and combine historical data trend prediction to identify the type of server performance bottleneck, and generate bottleneck identification results; the types include hardware resource bottleneck: insufficient CPU resources, insufficient memory; software performance bottleneck: low application code efficiency, insufficient database query optimization; business load bottleneck: sudden increase in user access leading to insufficient request processing capacity.

[0068] In the above process, the resource state data of the data processing server is collected in real time, including CPU usage and memory occupancy, and the optimization stage of the server (divided into bottleneck identification stage, strategy execution stage, and effect evaluation stage) is obtained, and these data are integrated.

[0069] Combined with the cache queue state parameters (total queue data, average data generation rate, and remaining storage space) obtained from step S100, the corresponding weight coefficients are set according to different optimization stages, wherein the bottleneck identification stage focuses on data timeliness, and the weight coefficient is inclined to data acquisition efficiency; the strategy execution stage focuses on resource saving, and the weight coefficient is inclined to server resource occupation; the effect evaluation stage balances the two, and the weight coefficient is evenly distributed.

[0070] Based on the set weight coefficient, the pressure condition of the cache queue (judged by the ratio of total queue data to total queue capacity, the higher the ratio, the greater the queue pressure) is calculated first, and then the resource carrying capacity of the data processing server (judged by the comprehensive proportion of CPU usage and memory occupancy, the higher the proportion, the more resource is nervous) is calculated, combined with the preset basic reading quantity, the current adaptive batch reading quantity is calculated by comprehensively considering the above factors, to ensure that the reading quantity neither leads to data backlog of the cache queue, nor excessively occupies the data processing server resources.

[0071] According to the batch reading quantity determined by calculation, data is read from the cache queue, stored to the distributed database, and then the preprocessing operations of data cleaning (abnormal data is removed and repaired, and the abnormal data is identified by combining statistical analysis and clustering analysis), data integration (the association table of different dimension data is established to ensure data integrity and consistency), and data conversion (the performance index data of different magnitudes is converted to a unified magnitude to facilitate subsequent analysis) are performed, and then a performance analysis model is constructed. Through analyzing the correlation between performance indicators and combining historical data trend prediction, the type of server performance bottleneck (including hardware resource bottleneck such as insufficient CPU resources, insufficient memory, etc.; software performance bottleneck such as low application code efficiency, insufficient database query optimization; business load bottleneck such as insufficient request processing capacity caused by sudden increase of user access volume, etc.) is accurately identified, and a bottleneck identification result is generated.

[0072] According to the change of the cache queue pressure and the fluctuation of the data processing server resource state, the batch reading quantity is dynamically adjusted, and the performance analysis model combines the complete data after preprocessing to ensure the accuracy of the bottleneck identification result, and provides a reliable basis for generating subsequent optimization strategies.

[0073] S300: According to the bottleneck identification result, an initial optimization strategy is matched, server real-time running data is obtained to optimize the initial optimization strategy, and a final optimization strategy is obtained.

[0074] In this embodiment, according to the bottleneck identification result, an initial optimization strategy is matched, server real-time running data is obtained to optimize the initial optimization strategy, and a final optimization strategy is obtained. The specific process is as follows:

[0075] S301: An optimization strategy knowledge base is constructed, and preset optimization strategies and historical optimization case data for different performance bottleneck types are stored;

[0076] S302: The bottleneck identification result is obtained, and an initial optimization strategy matching the current bottleneck is screened out;

[0077] S303: Real-time server running data is obtained, the parameters of the initial strategy are adjusted and optimized, and a final optimization strategy is formed.

[0078] In the above process, an optimization strategy knowledge base is constructed, and preset optimization strategies and historical optimization case data for different performance bottleneck types are stored; wherein the preset optimization strategies are, for example, for a CPU resource insufficient bottleneck, the preset strategies include dynamically adjusting the CPU resource allocation ratio, closing non-critical processes to release resources, etc.; for a database query bottleneck, the preset strategies include optimizing query statements, establishing query indexes, and increasing database cache space, etc.

[0079] The bottleneck recognition result output in step S200 is received, and according to the specific type (hardware, software, and business load bottleneck) of the bottleneck and the sub-scene (for example, CPU bottleneck and memory bottleneck in the hardware bottleneck), the initial optimization strategy with the highest matching degree with the current bottleneck is screened from the optimization strategy knowledge base.

[0080] Real-time running data of the server is acquired, including real-time values of bottleneck-related indexes (for example, CPU occupation rate and CPU idle time proportion of each process when there is a CPU bottleneck, database connection number and query statement execution time when there is a database bottleneck), other associated performance index data, and overall usage of server resources.

[0081] The acquired real-time running data of the server is substituted into the initial optimization strategy, and the parameters (for example, resource allocation proportion, process closing priority, and field selection for index establishment) of the initial strategy are adjusted and optimized by using a genetic algorithm. The adjusted strategy is simulated and evaluated, the optimal parameter combination of the strategy is screened, and the final optimization strategy is formed.

[0082] Through real-time synchronization of the bottleneck information in the current optimization process, the initial strategy, the final strategy after optimization, and the corresponding parameters, the content of the knowledge base is enriched. Meanwhile, the strategy optimization component continuously checks the adaptability of real-time running data and strategy parameters, so as to ensure that the final optimization strategy can accurately solve the server performance bottleneck.

[0083] Corresponding to the foregoing embodiments of the server performance optimization method based on big data analysis, the present application also provides embodiments of a server performance optimization system based on big data analysis.

[0084] Figure 5 is a server performance optimization system block diagram based on big data analysis according to an exemplary embodiment. Referring to Figure 5 The system can include a server running data aggregation module 401, a data read amount dynamic adjustment module 402, and an optimization strategy generation and execution module 403.

[0085] The server running data aggregation module 401 is configured to collect multi-dimensional running data of the server in real time, store the running data of the server according to priority by using a cache queue, and statistically calculate queue state parameters in real time.

[0086] The data read amount dynamic adjustment module 402 is configured to collect real-time resource states of a data processing server and server optimization stage data, calculate a data read amount, determine a current batch read amount according to a calculation result, identify a server performance bottleneck type, and output a bottleneck recognition result.

[0087] The optimization strategy generation and execution module 403 is configured to match an initial optimization strategy according to the bottleneck identification result, optimize the initial optimization strategy by using the real-time server running data, and obtain a final optimization strategy.

[0088] In the embodiment, the server running data aggregation module 401 collects real-time multi-dimensional server running data, stores the server running data in a cache queue according to priority, and calculates queue state parameters in real time. The data reading amount dynamic adjustment module 402 collects real-time resource states of a data processing server and server optimization stage data, calculates a data reading amount, determines a current batch reading amount according to the calculation result, identifies a server performance bottleneck type, and outputs a bottleneck identification result. The optimization strategy generation and execution module 403 matches an initial optimization strategy according to the bottleneck identification result, optimizes the initial optimization strategy by using the real-time server running data, and obtains a final optimization strategy. In this way, the data reading amount is dynamically adjusted, and the collaboration efficiency is improved.

[0089] As to the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be described in detail here.

[0090] For the system embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The device embodiment described above is only schematic, and the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to actual needs, some or all of the modules can be selected to achieve the purpose of the scheme of the present application. Those skilled in the art can understand and implement it without creative labor.

[0091] Correspondingly, the present application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the server performance optimization method based on big data analysis as described above. As Figure 6 As shown in the figure, a hardware structure diagram of a server performance optimization system based on big data analysis provided by an embodiment of the present application is in any device with data processing capability. In addition to the processor, the memory and the network interface shown in the figure, the device in the embodiment is usually provided with other hardware according to the actual function of the device with data processing capability, which will not be described here. Figure 6 As shown in the figure, a hardware structure diagram of a server performance optimization system based on big data analysis provided by an embodiment of the present application is in any device with data processing capability. In addition to the processor, the memory and the network interface shown in the figure, the device in the embodiment is usually provided with other hardware according to the actual function of the device with data processing capability, which will not be described here.

[0092] Correspondingly, the application further provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to implement the server performance optimization method based on big data analysis. The computer readable storage medium can be an internal storage unit of any device with data processing capability, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit of any device with data processing capability and the external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the device with data processing capability, and can also be used to temporarily store data that has been output or will be output.

[0093] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the application embrace any and all variations of the present application that fall within the scope of the general inventive concept as defined by the appended claims and their equivalents. Accordingly, the application is not limited to only those embodiments that have been specifically described herein.

[0094] It will be understood that the application is not limited to the precise structures herein described and illustrated in the drawings, and that various modifications and changes can be made without departing from the scope of the application.

Claims

1. A method for server performance optimization based on big data analysis, characterized in that, Comprise the following steps: Real-time acquisition of server multi-dimensional running data, using a cache queue to store server running data by priority, and real-time statistics of queue state parameters; The specific process is: Deploy lightweight acquisition agent components on each node of the server, real-time acquisition of multi-dimensional running data of the server, and encapsulation of multi-dimensional running data; Screening effective data from multi-dimensional running data; According to the preset rule, remove invalid data from multi-dimensional running data and retain valid data; The preset rule includes eliminating null data, filtering the same timestamp data collected repeatedly, and screening valid data meeting the data format requirements; Store the effective data into the cache queue, set the data storage priority according to the dual dimensions of data acquisition timestamp and data importance, and store the data in order; Real-time statistics of the current data total amount, data average generation rate and queue remaining storage space; Real-time resource state and server optimization stage data of the data processing server are collected, the data read amount is calculated, the current batch read amount is determined according to the calculation result, and the server performance bottleneck type is identified, and the bottleneck identification result is output; The specific process is: Real-time acquisition of resource state data of the data processing server, and acquisition of the optimization stage of the server; The optimization stage includes bottleneck identification stage, strategy execution stage and effect evaluation stage; Read the cache queue state parameters, and set the corresponding weight coefficient according to different optimization stages; According to the weight coefficient, the pressure of the cache queue and the resource carrying capacity of the data processing server are calculated in turn, and the current adaptive batch read amount is calculated combined with the preset basic read amount; According to the batch read amount, data is read from the cache queue and stored in the distributed database; Analyze the correlation between performance indicators, and identify the type of server performance bottleneck by combining historical data trend prediction, and generate bottleneck identification result; According to the bottleneck identification result, match the initial optimization strategy, obtain the server real-time running data optimization initial optimization strategy, and obtain the final optimization strategy.

2. The big data analytics based server performance optimization method of claim 1, wherein, In the step of reading the cache queue state parameters, and setting the corresponding weight coefficient according to different optimization stages: If the optimization stage is bottleneck identification stage, the data timeliness is focused, and the weight coefficient is inclined to data acquisition efficiency; If the optimization stage is strategy execution stage, the resource saving is focused, and the weight coefficient is inclined to server resource occupation; If the optimization stage is effect evaluation stage, balance data timeliness and resource saving, and the weight coefficient is evenly distributed.

3. The big data analytics based server performance optimization method as claimed in claim 1, wherein, After the step of reading data from the cache queue according to the batch read amount and storing the data into the distributed database: Perform data cleaning, data integration and data conversion preprocessing operations.

4. The big data analytics based server performance optimization method of claim 1, wherein, In the step of analyzing the correlation between performance indicators, and identifying the type of server performance bottleneck by combining historical data trend prediction, and generating bottleneck identification result: The types include hardware resource bottleneck: CPU resource shortage, memory shortage; Software performance bottleneck: low application code efficiency, insufficient database query optimization; Business load bottleneck: user access surge leading to insufficient request processing capacity.

5. The big data analytics based server performance optimization method of claim 1, wherein, In the step of matching the initial optimization strategy according to the bottleneck identification result, obtaining the real-time running data of the server to optimize the initial optimization strategy, and obtaining the final optimization strategy: Obtain the bottleneck identification result, and screen out the initial optimization strategy matched with the current bottleneck. Obtain the real-time running data of the server, adjust and optimize the parameters of the initial strategy, and form the final optimization strategy.

6. The big data analytics based server performance optimization method of claim 5, wherein, Before the step of obtaining the bottleneck identification result and screening out the initial optimization strategy matched with the current bottleneck: Construct an optimization strategy knowledge base to store preset optimization strategies and historical optimization case data for different performance bottleneck types.

Citation Information

Patent Citations

  • Server optimization method, system and device and storage medium

    CN113535407A