A JVM parameter self-adaptive recommendation method and device, electronic equipment and medium

Abstract

The application relates to the technical field of Java virtual machine performance optimization, in particular to a JVM parameter self-adaptive recommendation method and device, electronic equipment and a medium. The method comprises the following steps: collecting multi-dimensional runtime data through a monitoring agent and constructing a state feature vector; obtaining a candidate JVM parameter adjustment action according to the state feature vector and an offline reinforcement learning model, wherein the model comprises an action space, a reward function and a safety constraint condition; simulating and deducing the candidate JVM parameter adjustment action in an isolated environment, evaluating the influence of the candidate JVM parameter adjustment action on performance and safety constraints and generating a confidence score; and generating a recommendation report comprising a parameter adjustment suggestion, a rollback scheme and an expected performance change amount according to the confidence score. The application realizes safe and automatic tuning without online trial and error, and solves the core problem that the prior art cannot simultaneously consider safety and self-adaptation.

Description

Technical Field

[0001] This invention relates to the field of Java Virtual Machine performance optimization technology, and more specifically, to a method, apparatus, electronic device, and medium for adaptive recommendation of JVM parameters. Background Technology

[0002] In today's software development field, which heavily relies on the Java technology stack, optimizing the Java Virtual Machine (JVM) is crucial for improving system responsiveness, reducing resource consumption, and ensuring service stability. A properly configured JVM can significantly improve application throughput, reduce latency, and ensure high availability; conversely, a poorly configured JVM can lead to frequent garbage collection (GC), soaring response latency, and even service interruptions.

[0003] For a long time, JVM parameter tuning has heavily relied on expert experience and has been regarded as a "black art" requiring deep knowledge and extensive trial and error. With increasingly complex application architectures and expanding business scale, traditional tuning methods face severe challenges in both efficiency and effectiveness, driving the field towards automation and intelligence. Existing technical solutions can be mainly divided into the following two categories: 1. Manual tuning mode based on expert experience and rules This is the most mainstream practice in current production environments. At its core, engineers use monitoring tools (such as JConsole, VisualVM, and JProfiler) to collect runtime metrics such as GC logs, memory usage, and thread status. Through manual analysis and experience, they identify performance bottlenecks and adjust parameters according to known best practices.

[0004] Typical process and methods: Engineers first monitor key metrics, such as recording GC logs using -XX:+PrintGCDetails, and analyzing Full GC frequency, pause times, etc. Then, based on the analysis results, they apply a series of preset rules for adjustments, such as: increasing heap memory for high-concurrency services (-Xms, -Xmx) and enabling the G1 collector (-XX:+UseG1GC) to control the maximum pause time (-XX:MaxGCPauseMillis); for specific large object load scenarios, fine-tuning the ratio of the young generation to the old generation (-XX:NewRatio) or the object promotion threshold (-XX:MaxTenuringThreshold) to optimize GC behavior.

[0005] Solution Evaluation: This method heavily relies on engineers' personal experience, making the optimization process time-consuming and labor-intensive, and difficult to standardize into reusable knowledge. More importantly, it is inherently static and reactive—once the parameters are set, they cannot be adaptively adjusted according to dynamic changes in the application's actual load. When business traffic characteristics change or the application is iterated and updated, the originally optimized configuration may become invalid, requiring a restart of the cycle of manual analysis and optimization, which is inefficient and lagging.

[0006] 2. Machine learning auto-tuning scheme based on historical data analysis To overcome the limitations of manual tuning, the industry has begun to explore automated tuning using data-driven methods. Among them, the existing technology most similar to this invention is a scheme based on machine learning to analyze garbage collection (GC) logs to predict optimal JVM parameters.

[0007] Core Principles and Steps: This scheme attempts to establish a mapping model from GC log characteristics to ideal JVM parameters. Its main steps include: Data Acquisition and Feature Engineering: Collect detailed GC logs during application runtime and extract key features such as heap memory usage, GC pause time, and object allocation rate.

[0008] Model training: Use processed GC log data to train machine learning models (such as Gaussian process regressors GPR, random forests, etc.) to learn the complex relationship between performance metrics (such as throughput, latency) and specific JVM parameters (such as the young generation size parameter -XX:G1MaxNewSizePercent).

[0009] Parameter prediction and application: The trained model can predict a set of JVM parameter values ​​that may improve performance based on real-time or recent GC log characteristics, which can be used by operations and maintenance personnel for reference or automatic application.

[0010] Solution Evaluation: This solution represents a significant advancement in JVM tuning, moving from manual to automated methods. Through data-driven approaches, it can uncover complex parameter combinations that are difficult for the human mind to perceive intuitively, and has demonstrated the potential to improve application throughput by up to 20% in testing. However, this solution has a fundamental limitation and risk: it is essentially an "online trial and error" or "online application after offline simulation" model. The parameters recommended by the model are "optimal" predictions trained on historical data, but their actual effectiveness and safety when directly applied to a production environment have not been verified under real business loads. The model may recommend unsafe parameters under certain boundary conditions (such as sudden traffic or abnormal data) that could lead to memory overflows, GC pause times exceeding thresholds, or even application crashes, thus putting the production environment at risk.

[0011] Based on comparative analysis, existing technologies can be mainly divided into two categories: manual tuning based on expert experience, which relies on human experience and static rules, and the tuning process depends on manual monitoring and verification cycles. It lacks adaptability and is inefficient. Although the safety risks are relatively controllable, human error still exists. Automatic tuning based on machine learning, which relies on historical data to drive the model, has limited adaptability and cannot effectively cope with unknown states. Moreover, the model may recommend unsafe parameters, leading to system failures, which is considered "risky automation." To address the above shortcomings, the target solution of this invention uses offline reinforcement learning and dynamic safety constraints as the core driving forces. It adopts an offline training, model inference, and safety verification followed by a recommended tuning process, enabling the model to continuously evolve to adapt to dynamic changes. At the same time, it avoids high-risk actions in advance through safety constraint modeling and simulation verification mechanisms, achieving a unity of safety, efficiency, and adaptability. Although the system design and implementation are more complex, it fundamentally solves the core problem of "inability to balance safety and adaptability" in existing technologies.

[0012] In summary, existing technologies, including the most advanced machine learning-based automated tuning solutions, have failed to effectively solve the core challenge of "achieving adaptive and intelligent recommendation of JVM parameters while ensuring absolute safety and eliminating the need for online trial and error." The former is inefficient and reliant on manual labor, while the latter, although improving automation, lacks reliable safety safeguards and is therefore unreliable for use in production environments where stability is paramount. Summary of the Invention

[0013] The technical problem to be solved by the present invention is to provide a JVM parameter adaptive recommendation method, device, electronic device and medium, which aims to solve at least one of the above-mentioned technical problems.

[0014] Firstly, the technical solution of the present invention to solve the above-mentioned technical problems is as follows: a JVM parameter adaptive recommendation method, the method comprising: S1, through a monitoring agent deployed on the target JVM, non-intrusively collects multi-dimensional runtime data of the target JVM in the current cycle, and performs feature extraction and normalization processing on the multi-dimensional runtime data to construct a state feature vector; S2, based on the state feature vector and the pre-built offline reinforcement learning model, obtain candidate JVM parameter adjustment actions. The offline reinforcement learning model includes the JVM parameter adjustment action space, reward function and security constraints. S3 simulates and extrapolates candidate JVM parameter tuning actions in an isolated environment, evaluates the impact of candidate JVM parameter tuning actions on the target JVM performance indicators and security constraints, and generates corresponding confidence scores. S4 generates a JVM parameter recommendation report based on the confidence score.

[0015] The beneficial effects of this invention are as follows: Compared with the prior art, this invention solves the problems of low efficiency and lack of adaptability in traditional manual tuning by deploying a monitoring agent to collect multi-dimensional runtime data and constructing state feature vectors; by introducing an offline reinforcement learning model containing action space, reward function and safety constraints to generate candidate JVM parameter adjustment actions, it fundamentally avoids the security risks caused by online trial and error to the production environment, and solves the defects of existing machine learning tuning solutions that are difficult to deploy in practice due to lack of security guarantees; by conducting simulation and generating confidence scores in an isolated environment, it achieves the pre-verification of parameter adjustment effects and the quantification of risks, and by combining the confidence scores to generate JVM parameter recommendation reports, it ensures the safety, reliability and executability of tuning decisions, thereby achieving adaptive, safe and intelligent recommendation of JVM parameters without the need for online trial and error.

[0016] Based on the above technical solution, the present invention can be further improved as follows.

[0017] Furthermore, the aforementioned multi-dimensional runtime data includes multi-dimensional runtime data collected at multiple time points. Feature extraction and normalization are performed on the multi-dimensional runtime data to construct a state feature vector, including: For each time point, based on the multi-dimensional runtime data of that time point and its neighboring time points, the temporal and statistical characteristics of that time point are calculated. The temporal characteristics of each time point include the trend of indicator change, periodic fluctuations and abrupt change points, and the statistical characteristics of each time point include the mean, variance, percentiles and peak values. The temporal and statistical features extracted at the same time are combined into a feature vector for that time. For each dimension of the feature vector, the value of that dimension at different times is normalized so that the numerical distribution of that dimension is mapped to a preset uniform numerical range, thus obtaining the normalized feature vector corresponding to each time. Based on the normalized feature vectors at each time point, a time series dataset containing state-action-reward triples is constructed as the state feature vector.

[0018] Furthermore, based on the normalized feature vectors at each time step, a time-series dataset containing state-action-reward triples is constructed as the state feature vector, including: For time t, the normalized feature vector of time t is used as the current state St, the current parameter configuration of the target JVM at the same time t is obtained as the current action At, the performance index data collected from time t to the next time is obtained, the reward value Rt is calculated according to the preset reward function, and the current state St, the current action At, and the reward value Rt are associated to construct the triplet of time t. The triples at multiple time points are arranged in chronological order to form a time series dataset, which is used as the state feature vector for training the offline reinforcement learning model.

[0019] Furthermore, the aforementioned offline reinforcement learning model is constructed based on the following method: Construct a JVM parameter tuning action space, which includes the adjustable JVM parameter types and the range of JVM parameter values. JVM parameter types include one or more of the following: heap memory size, garbage collector type and parameters, ratio of young generation to old generation, GC pause time target, and number of concurrent GC threads. Construct a reward function that is positively correlated with one or more performance metrics, such as throughput, latency, and resource utilization, and negatively correlated with the degree of violation of security constraints. Construct safety constraints, which may include one or more of the following: memory limit threshold, GC pause time threshold, and CPU utilization threshold. Collect multi-dimensional runtime data of the target JVM within a historical period to construct a time-series dataset for that historical period. The safety constraints are encoded as penalty terms during the training process. Using time-series datasets from historical periods, the offline reinforcement learning model is trained with the goal of minimizing the penalty terms.

[0020] Furthermore, the above-mentioned candidate JVM parameter tuning actions are simulated and extrapolated in an isolated environment to evaluate the impact of the candidate JVM parameter tuning actions on the target JVM performance indicators and security constraints, and to generate corresponding confidence scores, including: Choose a simulation verification environment, which can be either a containerized isolated environment or a lightweight heuristic simulation environment; Apply candidate JVM parameter tuning actions to the target JVM instance launched in the simulation verification environment; In a simulated verification environment, replay the target JVM's real historical traffic or standard load test cases, and collect performance and security constraint data during the simulated operation. Based on the collected performance metrics and security constraint metrics, evaluate the performance changes in throughput, latency, and resource utilization of candidate JVM parameter adjustments compared to the baseline configuration, as well as whether they violate preset security constraints. Obtain the degree of determinism when the offline reinforcement learning model outputs candidate JVM parameter adjustment actions; obtain the result stability of candidate JVM parameter adjustment actions during simulation verification; obtain the state feature vector in the current period that serves as the input to the offline reinforcement learning model; The confidence score of the candidate JVM parameter adjustment action is calculated based on one or more of the following factors: degree of certainty, result stability, matching degree between the current period's state feature vector and the time series dataset of historical periods, adjustment magnitude of the candidate JVM parameter adjustment action compared to the current configuration, and the assessed performance change and security constraint violation.

[0021] Furthermore, the above-mentioned JVM parameter recommendation report, generated based on the confidence score, includes: The confidence score is compared with a preset confidence threshold. If the confidence score is greater than or equal to the threshold, the candidate JVM parameter adjustment action is marked as a high-confidence recommendation, and a JVM parameter recommendation report is generated.

[0022] Furthermore, the above JVM parameter recommendation report includes: Parameter tuning suggestions: including the specific adjustment values ​​of each parameter in the candidate JVM parameter tuning actions, the comparison of parameters before and after the adjustment, and the operation commands or scripts for parameter tuning; Rollback plan: including the currently configured backup information, rollback operation commands or scripts, and rollback trigger conditions; Expected performance changes: This includes one or more of the following: changes in throughput, changes in latency, and changes in resource utilization, obtained from evaluating candidate JVM parameter tuning actions.

[0023] Secondly, to solve the above-mentioned technical problems, the present invention also provides a JVM parameter adaptive recommendation device, the device comprising: The acquisition and processing module is used to non-intrusively acquire multi-dimensional runtime data of the target JVM in the current cycle through a monitoring agent deployed on the target JVM, and to perform feature extraction and normalization processing on the multi-dimensional runtime data to construct a state feature vector. The candidate JVM parameter tuning action determination module is used to obtain candidate JVM parameter tuning actions based on the state feature vector and the pre-built offline reinforcement learning model. The offline reinforcement learning model includes the JVM parameter tuning action space, reward function and security constraints. The confidence score generation module is used to simulate and extrapolate candidate JVM parameter tuning actions in an isolated environment, evaluate the impact of candidate JVM parameter tuning actions on the target JVM performance indicators and security constraints, and generate corresponding confidence scores. The recommendation module generates a JVM parameter recommendation report based on the confidence score.

[0024] Thirdly, in order to solve the above-mentioned technical problems, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the JVM parameter adaptive recommendation method of the present application.

[0025] Fourthly, in order to solve the above-mentioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the JVM parameter adaptive recommendation method of the present application.

[0026] Additional aspects and advantages of this application will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of this application. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below.

[0028] Figure 1 This is a flowchart illustrating a JVM parameter adaptive recommendation method according to an embodiment of the present invention. Figure 2 A system architecture diagram provided for one embodiment of the present invention; Figure 3 This is a schematic diagram of the data flow of a JVM parameter adaptive recommendation method provided in one embodiment of the present invention; Figure 4 This is a flowchart illustrating another JVM parameter adaptive recommendation method provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a JVM parameter adaptive recommendation device provided in one embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0029] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0030] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0031] The data acquisition process involved in this invention follows the principles of legality, legitimacy, and necessity. Based on obtaining the explicit authorization and consent of the user, only the minimum necessary information required to achieve the purpose is collected, and data security protection obligations are fulfilled in accordance with the law.

[0032] The solution provided in this invention can be applied to any application scenario that requires target detection.

[0033] This invention provides a possible implementation, such as... Figure 1 The diagram illustrates a flowchart of a JVM parameter adaptive recommendation method. This method can be executed by any electronic device, such as a terminal device, or jointly by a terminal device and a server. For ease of description, the method provided in this embodiment will be described below using a terminal device as the execution subject. Figure 1 The flowchart shown indicates that the method may include the following steps: S1, through a monitoring agent deployed on the target JVM, non-intrusively collects multi-dimensional runtime data of the target JVM in the current cycle, and performs feature extraction and normalization processing on the multi-dimensional runtime data to construct a state feature vector; S2, based on the state feature vector and the pre-built offline reinforcement learning model, obtain candidate JVM parameter adjustment actions. The offline reinforcement learning model includes the JVM parameter adjustment action space, reward function and security constraints. S3 simulates and extrapolates candidate JVM parameter tuning actions in an isolated environment, evaluates the impact of candidate JVM parameter tuning actions on the target JVM performance indicators and security constraints, and generates corresponding confidence scores. S4 generates a JVM parameter recommendation report based on the confidence score.

[0034] Compared to existing technologies, this invention addresses the problems of low efficiency and lack of adaptability in traditional manual tuning by deploying a monitoring agent to collect multi-dimensional runtime data and constructing state feature vectors. It fundamentally avoids the security risks to the production environment caused by online trial and error by introducing an offline reinforcement learning model that includes an action space, reward function, and security constraints to generate candidate JVM parameter adjustment actions, thus resolving the shortcomings of existing machine learning tuning solutions that are difficult to deploy in practice due to a lack of security guarantees. Furthermore, by conducting simulations in an isolated environment and generating confidence scores, the invention achieves pre-verifiable effects and quantifiable risks of parameter adjustment. Combining these confidence scores with the generation of a JVM parameter recommendation report ensures the safety, reliability, and executability of tuning decisions, thereby achieving adaptive, safe, and intelligent recommendation of JVM parameters without the need for online trial and error.

[0035] The following specific embodiments will further illustrate the solution of the present invention. Based on the foregoing technical background analysis, the existing technical solutions mainly have the following specific drawbacks, which directly restrict the security, efficiency and intelligence level of JVM parameter tuning.

[0036] In view of the aforementioned existing technologies, especially the core shortcomings of the closest solutions such as "online security risks" and "static inefficiency," the purpose of this invention is to provide a novel solution to achieve the following specific tasks: 1. Achieve safe and automated optimization without online trial and error. This invention aims to fundamentally change the existing "trial and error" model of automation solutions, which must bear risks in a production environment. By constructing a decision-making closed loop that performs simulations and verifications in a simulated environment with strict safety constraints, it ensures that any parameter adjustment suggestions have passed safety assessments from the outset, thereby fundamentally solving the problem of safety verification of optimization actions in a production environment and achieving reliable automation where "decision-making is safe".

[0037] 2. Establish an intelligent decision-making mechanism that integrates security constraints and performance objectives. This invention aims to address the problem of existing machine learning methods that only optimize performance while neglecting safety boundaries. By encoding stability metrics such as memory, latency, and resources as hard constraints or penalties in the training and inference process of reinforcement learning models, the agent must adhere to pre-defined safety rules while exploring optimal performance, thereby solving the problem of the lack of safety assurance mechanisms during the optimization process.

[0038] 3. Provides adaptive tuning capabilities for sustainable evolution. This invention aims to overcome the limitations of static tuning and the limited adaptability of traditional machine learning models. By designing a closed-loop system that includes continuous data acquisition, online model learning, and periodic retraining, the tuning strategy can dynamically evolve in response to application iterations, changes in business volume, and updates to the technology stack, thereby solving the problem of maintaining the effectiveness and advancement of the tuning strategy over the long term.

[0039] Therefore, this invention provides a JVM parameter adaptive recommendation system and method based on offline reinforcement learning and security constraints. Its core lies in constructing a closed-loop system of "data acquisition - offline training - simulation verification - secure deployment," which automatically discovers the optimal or near-optimal JVM parameter combinations in a data-driven manner, while ensuring through multi-layered security mechanisms that the entire process does not require risky trial and error in a production environment.

[0040] In this embodiment, a JVM parameter adaptive recommendation method may include the following steps: S1, through a monitoring agent deployed on the target JVM, non-intrusively collects multi-dimensional runtime data of the target JVM in the current cycle, and performs feature extraction and normalization processing on the multi-dimensional runtime data to construct a state feature vector; Specifically, a monitoring agent refers to a software component deployed on the target Java Virtual Machine (JVM) that non-intrusively collects runtime data of the target JVM through bytecode enhancement technology, meaning it can collect data without modifying the target application's source code. The current period refers to the time window comprised of the current moment and several adjacent historical moments. Multi-dimensional runtime data refers to JVM runtime data collected at multiple moments, including GC log data, memory profile data, thread stack data, CPU instruction data, I / O wait data, and performance metrics data. Feature extraction refers to extracting key features that characterize the JVM's runtime state from the raw multi-dimensional runtime data. Normalization refers to mapping feature values ​​of different dimensions to a unified numerical range to eliminate the impact of dimensional differences on model training. The state feature vector refers to the feature vector representation used as input to the reinforcement learning model.

[0041] like Figure 2 As shown, Figure 2 This is a system architecture diagram provided according to an embodiment of the present invention. The system architecture includes five layers: an external access layer, a core business layer, a support layer, a configuration and integration layer, and an infrastructure layer. The external access layer includes a Java Agent layer and an API gateway layer. The Java Agent layer non-intrusively collects runtime data of the target JVM through bytecode enhancement technology and supports low-performance data collection and dynamic instrumentation. The API gateway layer provides RESTful API interfaces and supports external system integration functions such as parameter recommendation request processing, training task management, and result feedback collection.

[0042] The core business layer includes Parameter Recommendation Service, Offline Reinforcement Learning Service, Simulation Service, and Data Management Service. The support layer includes a tool layer (containing Confidence Calculator, JvmParameterValidator, Reward Calculator, and MetricAnalyzer) and a data model layer (containing DTOs, Entities, and VOs). The configuration and integration layer includes a configuration layer and an external integration layer, interfacing with Docker engines, TensorFlow / PyTorch, monitoring systems (Prometheus / Grafana), configuration centers (Apollo / Nacos), etc. The infrastructure layer includes databases (MySQL, H2), file storage, containerization support (Docker, Kubernetes), and message queues. Figure 2 The system architecture shown provides complete system support for the implementation of the method of the present invention.

[0043] In an optional embodiment of the present invention, the multi-dimensional runtime data includes multi-dimensional runtime data collected at multiple times. Therefore, the feature extraction and normalization processing of the multi-dimensional runtime data in step S1 above, to construct a state feature vector, specifically includes the following steps S11 to S13: S11. For each time point, the temporal and statistical characteristics of that time point are calculated based on the multi-dimensional runtime data of that time point and its neighboring time points. The temporal characteristics of each time point include the changing trend of the indicators, periodic fluctuations and abrupt change points, and the statistical characteristics of each time point include the mean, variance, percentiles and peak values.

[0044] S12, combine the temporal features and statistical features extracted at the same time to form the feature vector of that time. For each dimension of the feature vector, normalize the value of the feature at different times so that the numerical distribution of the feature is mapped to a preset uniform numerical range, and obtain the normalized feature vector corresponding to each time.

[0045] S13. Based on the normalized feature vectors at each time step, construct a time series dataset containing state-action-reward triples as the state feature vector.

[0046] Optionally, in S10 above, one way to normalize the values ​​of each dimension of the feature vector at different times, so that the numerical distribution of the feature dimension is mapped to a preset uniform numerical range, is as follows: Method 1: Normalization based on global statistics (offline normalization) During the offline training phase, the time series dataset for all historical periods is first scanned. For each dimension of the feature vector, the distribution of that dimension's values ​​across all time points is calculated, and the maximum value (max) of that dimension is obtained. i and minimum value min i Then, for each feature dimension, the min-max normalization method is used to obtain the original value x of that feature dimension at each time t. i(t) Mapped to a preset uniform numerical range [0,1], the normalized eigenvalues ​​x i'(t) The calculation formula is: x i'(t) = (x i(t) - min i ) / (max i - min i ); Where, x i(t) Let be the original value of the i-th feature at time t, min i The minimum value of the i-th feature across all time points, max i Let x be the maximum value of the i-th feature across all time points. i'(t) Let be the normalized feature value of the i-th feature at time t, with a value range of [0,1].

[0047] If the preset uniform numerical range is [-1, 1], then the normalized eigenvalue x i'(t) The calculation formula is: x i'(t) = 2×(x i(t) - min i ) / (max i - min i -1; When max i equal to min i If the feature dimension is constant at all times, then the normalized value of the feature dimension at all times is uniformly set to the preset default value (e.g., 0 or 0.5).

[0048] Method 2: Normalization based on sliding window (online normalization) During the online inference phase, since it is impossible to obtain the global maximum and minimum values ​​of all historical data in advance, a normalization method based on a sliding window is adopted. Specifically, for the current time t, a sliding window (where L is the preset window length) is maintained, containing the current time t and the previous L-1 times. For each dimension of the feature vector, the maximum value (max) of that dimension within the current sliding window is calculated. i_window(t) and minimum value min i_window(t) The min-max normalization method is used to obtain the original value x of this feature at the current time t. i(t) Mapped to a preset uniform numerical range [0,1], the normalized eigenvalues ​​x i'(t) The calculation formula is: x i'(t) = (x i(t) - min i_window(t) ) / (max i_window(t) - min i_window(t) ); Where, x i(t) Let be the original value of the i-th feature at time t, min i_window(t) Let max be the minimum value of the i-th feature within the current sliding window. i_window(t) Let x be the maximum value of the i-th feature within the current sliding window. i'(t) Let be the normalized eigenvalue of the i-th feature at time t. When max i_window(t) equal to min i_window(t) When, x i'(t) Set to the default value (e.g., 0.5).

[0049] For time t, the eigenvalues ​​x of each dimension after the above normalization process are... 1'(t) , x 2'(t) , ..., x n'(t) The normalized feature vector V(t) at that moment is formed by combining the features in a preset order: V(t) = [x 1'(t) , x 2'(t) , ..., x n'(t) ]; Where n is the total dimension of the feature vectors, x i'(t) Let V(t) be the normalized feature value of the i-th feature at time t, and let V(t) be the normalized feature vector at time t, which is used for subsequent state feature vector construction and model input.

[0050] By employing either Method 1 or Method 2 of the above-mentioned normalization method, this invention can map feature values ​​with different dimensions and numerical ranges to a unified numerical interval, eliminating the differences in numerical ranges between features caused by different physical meanings and units. This makes different features comparable and improves the training stability and convergence speed of offline reinforcement learning models. Method 1 is suitable for batch normalization processing during the offline training phase, while Method 2 is suitable for real-time normalization processing during the online inference phase. In practical applications, a suitable normalization method can be selected based on the specific application scenario and data characteristics, or multiple normalization methods can be combined.

[0051] In an optional embodiment of the present invention, the above-mentioned S13 (i.e., constructing a time series dataset containing state-action-reward triples as state feature vectors based on the normalized feature vectors at each time point) specifically includes the following S131 to S135: S131, for time t, the normalized eigenvector of time t is taken as the current state St.

[0052] S132, obtain the current parameter configuration of the target JVM at the same time t as the current action At.

[0053] S133: Obtain the performance index data collected from time t to the next time, and calculate the reward value Rt according to the preset reward function.

[0054] S134. Associate the current state St, the current action At, and the reward value Rt to construct the triple (St, At, Rt) at time t.

[0055] S135 arranges the triples from multiple time points in chronological order to form a time series dataset, which serves as the state feature vector for training the offline reinforcement learning model.

[0056] As an example, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the data flow of a JVM parameter adaptive recommendation method provided in one embodiment of the present invention. Figure 3 It details the flow of data between the aforementioned steps and modules. Specifically, Figure 3It includes six stages: The first stage is the data acquisition layer, which uses GC log collector, memory profile collector, thread stack collector, CPU instruction collector, IO wait collector, and performance indicator collector to monitor the frequency and duration of GC events in real time, monitor the utilization rate and object allocation rate of each area of ​​heap memory, analyze thread state distribution and lock contention hotspots, analyze CPU utilization and cache hit rate, monitor file IO and network IO wait time, and collect application-level throughput and latency performance indicators. The above collectors together constitute the core functional modules of the monitoring agent in S1. The second stage is feature engineering and state construction. The feature extractor normalizes the multidimensional monitoring data and extracts time-series and statistical features. The state-action-reward dataset construction module associates JVM parameter configuration and performance indicators, and adds security constraint labels and anomaly markers. This stage corresponds to the feature extraction and triplet construction process from S11 to S13. The third stage is offline reinforcement learning training, which uses the CQL (Conservative Q-Learning) algorithm to learn the optimal policy on the offline dataset. A conservative penalty term is added to the loss function to prevent the risk of overestimating out-of-distribution actions, and the memory limit and GC pause threshold are encoded as constraints. A penalty term is added to the loss function to prevent the violation of constraints. This stage corresponds to the training process of the offline reinforcement learning model in S2. The fourth stage is simulation and confidence assessment, which verifies parameters using a Docker containerized simulation environment or a lightweight heuristic simulation environment. The reliability of the recommendation results is quantified by combining the model output, feature coverage, and parameter conservatism using a confidence calculator. This stage corresponds to the simulation and confidence score generation process in S3. The fifth stage is intelligent recommendation and automated operation and maintenance, which generates detailed optimization plan reports and supports one-click deployment and rollback. This stage corresponds to the recommendation report generation process in S4. The sixth stage is system management and monitoring, including configuration management, performance monitoring and alerting, and knowledge base accumulation. Figure 3 The data flow shown fully demonstrates the closed-loop data flow of this invention from data acquisition to final deployment.

[0057] S2, based on the state feature vector and the pre-built offline reinforcement learning model, obtain candidate JVM parameter adjustment actions. The offline reinforcement learning model includes the JVM parameter adjustment action space, reward function and security constraints. Specifically, an offline reinforcement learning model refers to a reinforcement learning model trained solely on historical runtime data. Its training process does not require online interaction with the production environment, thus avoiding the security risks associated with online exploration. Candidate JVM parameter tuning actions refer to a set of JVM parameter tuning suggestions output by the model based on the input state feature vector, including specific adjustment values ​​for each parameter. The JVM parameter tuning action space refers to the set of all possible parameter tuning actions that the model can output. The reward function is a quantization function used to evaluate the quality of an action. Safety constraints refer to safety boundaries that the model's output actions must not exceed.

[0058] In an optional embodiment of the present invention, the above-mentioned offline reinforcement learning model is pre-built based on the following method, specifically including the following steps S21 to S25: S21. Construct the JVM parameter tuning action space. The JVM parameter tuning action space includes the adjustable JVM parameter types and the corresponding value ranges of each parameter. The JVM parameter types include one or more of the following: heap memory size (e.g., -Xms, -Xmx), garbage collector type and parameters (e.g., -XX:+UseG1GC), young generation to old generation ratio (e.g., -XX:NewRatio), GC pause time target (e.g., -XX:MaxGCPauseMillis), and number of concurrent GC threads (e.g., -XX:ConcGCThreads).

[0059] S22, Construct a reward function. This reward function is positively correlated with one or more performance metrics, including throughput, latency, and resource utilization, and negatively correlated with the degree of violation of security constraints. For example, the reward function R can be expressed as: R equals the throughput improvement coefficient minus the latency growth coefficient, and then minus the security constraint violation penalty term. Here, the throughput improvement coefficient measures the degree of improvement in application processing capacity, the latency growth coefficient measures the degree of deterioration in response time, and the security constraint violation penalty term is used to penalize actions that violate preset security boundaries.

[0060] S23. Construct safety constraints, which may include one or more of the following: memory limit threshold, GC pause time threshold, and CPU utilization threshold. For example, set the GC pause time to not exceed 200 milliseconds, the heap memory utilization to not exceed 85%, and the CPU utilization to not exceed 90%.

[0061] S24: Collect multi-dimensional runtime data of the target JVM during the historical period, and construct the time series dataset of the historical period according to the method in S1 above.

[0062] S25, the safety constraints are encoded as penalty terms during training. Using a historical time-series dataset, an offline reinforcement learning model is trained with the goal of minimizing the empirical loss with the penalty term. Specifically, when the model's output action causes the simulated predicted GC pause time to exceed a preset threshold or triggers a memory overflow, a very large negative penalty value is assigned to the loss function, thereby enabling the model to automatically learn to avoid generating actions that violate safety constraints during training.

[0063] S3 simulates and extrapolates candidate JVM parameter tuning actions in an isolated environment, evaluates the impact of candidate JVM parameter tuning actions on the target JVM performance indicators and security constraints, and generates corresponding confidence scores. Specifically, an isolated environment refers to an independent operating environment isolated from the production environment, including containerized isolated environments or lightweight heuristic simulation environments. Simulation involves applying candidate parameters to adjust actions and running load tests in an isolated environment to observe its actual effects. Performance metrics include throughput, latency, and resource utilization. Confidence scores are numerical values ​​used to quantify the reliability of recommendation results.

[0064] In an optional embodiment of the present invention, S3 specifically includes the following S31 to S37: S31. Select a simulation verification environment, which can be either a containerized isolated environment or a lightweight heuristic simulation environment. When high-fidelity verification is required, select the Docker containerized isolated environment, create an image consistent with the application's production environment, and start the container; when rapid verification is required or the Docker environment is unavailable, select the lightweight simulation environment based on historical data statistics and heuristic rules.

[0065] S32 applies the candidate JVM parameter tuning action to the target JVM instance launched in the simulation verification environment.

[0066] S33 replays the target JVM's real historical traffic or standard load test cases in a simulated verification environment, and collects performance index data and security constraint index data during the simulated operation.

[0067] S34. Based on the collected performance metric data and security constraint metric data, evaluate the performance changes in throughput, latency, and resource utilization of the candidate JVM parameter tuning actions compared to the baseline configuration, and whether they violate preset security constraints. The baseline configuration refers to the original parameter configuration of the target JVM before the candidate JVM parameter tuning actions were applied.

[0068] S35, obtain the degree of determinism when the offline reinforcement learning model outputs candidate JVM parameter adjustment actions; obtain the result stability of candidate JVM parameter adjustment actions during the simulation verification process; obtain the state feature vector in the current period that serves as the input to the offline reinforcement learning model.

[0069] The degree of certainty is used to measure the confidence level of the offline reinforcement learning model when it outputs candidate JVM parameter adjustment actions. Specifically, when the offline reinforcement learning model uses a policy network based on probability distribution, the probability value corresponding to the model's output candidate JVM parameter adjustment actions is extracted, and this probability value is used as the degree of certainty D. model When the offline reinforcement learning model uses a Q-value network, the relative difference between the Q-value of a candidate JVM parameter adjustment action and the Q-value of other candidate actions is normalized and used as the degree of determinism D. model Degree of certainty D model The value range is [0,1]. The larger the value, the more certain the model's output for that action is.

[0070] The stability of the results is used to measure the consistency of the candidate JVM parameter tuning actions during the simulation verification process. Specifically, in the simulation verification environment, multiple independent simulations are performed on the candidate JVM parameter tuning actions. Each simulation uses the same real historical traffic or standard load test cases, and key performance indicators (including one or more of throughput, latency, and GC pause time) are collected from each simulation. The variance or standard deviation of the key performance indicators in multiple simulations is calculated, and the stability S is calculated based on the variance or standard deviation. sim When variance is used, the stability of the results is S. sim The calculation formula is: S sim = 1-variance normalized Among them, variance normalized S represents the normalized variance. A smaller variance indicates more stable simulation results. sim The closer the value is to 1; the larger the variance, the greater the fluctuation in the simulation results. sim The closer the value is to 0.

[0071] S36. Calculate the confidence score of the candidate JVM parameter adjustment action based on one or more of the following factors: degree of certainty, result stability, matching degree between the current period's state feature vector and the historical time series dataset, adjustment magnitude of the candidate JVM parameter adjustment action compared to the current configuration, and the performance change and security constraint violation assessed in S34.

[0072] The matching degree measures the extent to which the state feature vector within the current period covers the time series dataset of historical periods. Specifically, the matching degree M is calculated using one or more of the following methods. history : Method 1 (Distance-based matching degree): Calculate the Mahalanobis distance or Euclidean distance between the current period's state feature vector and each historical state feature vector in the historical time series dataset. Then, normalize the calculated minimum or average distance and convert it into a matching degree M. history The smaller the distance, the higher the matching degree.

[0073] Method 2 (Matching Degree Based on Distribution Density): Model the distribution of state feature vectors in the historical time series dataset (e.g., using kernel density estimation or Gaussian mixture model), calculate the probability density value of the state feature vector in the current period within the distribution model, and use this probability density value, after normalization, as the matching degree M. history A higher probability density value indicates a better coverage of the current state in the historical data distribution and a higher degree of matching.

[0074] Method 3 (Matching Degree Based on Reconstruction Error): Train an autoencoder model using a historical time-series dataset. Input the state feature vector of the current period into the trained autoencoder model, calculate the reconstruction error, and convert the normalized reconstruction error into a matching degree M. history The smaller the reconstruction error, the smaller the deviation between the current state and the historical data distribution, and the higher the matching degree.

[0075] Specifically, the level of conservatism measures the magnitude of the adjustment made by the candidate JVM parameter tuning action compared to the current configuration. For each adjustable parameter in the candidate JVM parameter tuning action, the adjustment magnitude is calculated, where the formula for calculating the adjustment magnitude is: adjustment （i） = |value candidate（i） -value current（i） | / range （i） ; Where, value candidate（i） The value is the adjustment value of the i-th parameter in the candidate JVM parameter adjustment action. current（i） The range represents the current value of the i-th parameter in the current configuration. （i） This represents the preset value range width for the i-th parameter. The adjustment ranges of each parameter are weighted and summed, or the maximum value is taken, to obtain the overall adjustment range M. adjust Conservatism level C conservative With the overall adjustment range M adjust Negative correlation. The calculation formula is: C conservative = 1 - M adjust_normalized ; Among them, M adjust_normalized This represents the normalized overall adjustment range. A smaller overall adjustment range indicates a more conservative parameter adjustment. (C) conservative The closer the value of C is to 1, the more aggressive the parameter adjustment is. conservative The closer the value is to 0.

[0076] S37. Compare the calculated confidence score with the preset confidence threshold. If the confidence score is greater than or equal to the threshold, mark the candidate JVM parameter adjustment action as a high-confidence recommendation and execute S4. If the confidence score is less than the threshold, mark the candidate JVM parameter adjustment action as a low-confidence recommendation and generate a manual review suggestion.

[0077] In this scheme, a weighted fusion approach can be used to adjust the degree of certainty D. model Result stability S sim Match degree M history Conservatism level C conservative And the performance change ΔPerf and safety constraint violation Penalty obtained from the evaluation in S34. violation The data is then fused to calculate the confidence score for each candidate JVM parameter tuning action. The formula for calculating the confidence score is as follows: Confidence=w1·D model + w2·S sim + w3·M history + w4·C conservative + w5·ΔPerf -w6·Penalty violation ; Where w1, w2, w3, w4, w5, and w6 are the preset weight coefficients for each item, satisfying that each weight coefficient is greater than or equal to 0, and w1 + w2 + w3 + w4 + w5 = 1; ΔPerf is the normalized performance change, which takes a positive value when throughput increases and latency decreases; Penalty violation Penalty is a penalty for violating safety constraints. When the evaluation result in S34 indicates no violation of any safety constraints, Penalty... violation = 0; Penalty = 0 when the evaluation result is a violation of any safety constraint. violation Taking a preset positive value (e.g., a value of 1) significantly reduces the confidence score.

[0078] In another optional embodiment of the present invention, when only a portion of the above factors are selected for confidence calculation, a simplified fusion formula can be used. For example, when only four factors—certainty, outcome stability, matching degree, and conservatism—are used, the confidence score is calculated as: Confidence = w1·D model + w2·S sim + w3·M history +w4·C conservative When only two factors, certainty and fit, are used, the confidence score is calculated as: Confidence = w1·D model + w2·S sim + w3·M history .

[0079] Through the above steps S31 to S37, this invention integrates information from multiple dimensions, including model output characteristics, simulation verification results, historical data distribution characteristics, parameter adjustment range, actual performance changes, and security constraint violations, to quantitatively evaluate the reliability of candidate JVM parameter adjustment actions, providing a scientific quantitative basis for subsequent recommendation decisions.

[0080] S4 generates a JVM parameter recommendation report based on the confidence score.

[0081] Specifically, the JVM parameter recommendation report is an output report that includes parameter tuning suggestions, rollback plans, and expected performance changes.

[0082] In an optional embodiment of the present invention, the JVM parameter recommendation report includes the following: Parameter tuning suggestions: This includes the specific adjustment values ​​for each parameter in the candidate JVM parameter tuning actions, a comparison of the parameters before and after the adjustment, and the operation commands or scripts for parameter tuning. For example, adjusting the -Xmx parameter from 4g to 6g, and providing the specific commands for modifying the startup script.

[0083] Rollback scheme: This includes backup information of the current configuration, rollback operation commands or scripts, and rollback trigger conditions. For example, back up the original startup script before applying new parameters, provide a rollback script for one-click restoration to the original configuration, and set up automatic rollback when the application fails to restart or performance metrics deteriorate beyond a threshold.

[0084] Expected performance changes: This includes one or more of the following: changes in throughput, changes in latency, and changes in resource utilization, obtained from evaluating candidate JVM parameter tuning actions.

[0085] To better illustrate and understand the principle of the method provided by this invention, the following description uses an optional specific embodiment to illustrate the solution of this invention. It should be noted that the specific implementation of each step in this specific embodiment should not be construed as a limitation of the solution of this invention. Other implementations that can be conceived by those skilled in the art based on the principle of the solution provided by this invention should also be considered within the scope of protection of this invention.

[0086] This optional embodiment uses the JVM parameter tuning scenario of "order service" on an e-commerce platform before a major promotion as an example to provide a detailed description of the method provided by the present invention. Figure 4 This is a flowchart illustrating another JVM parameter adaptive recommendation method provided in an embodiment of the present invention. The method specifically includes the following three stages and a total of ten steps.

[0087] Phase 1: Data Acquisition and Model Training This stage corresponds to Figure 1 The training processes for the offline reinforcement learning model in S1 and S2 are used to collect historical runtime data and train the offline reinforcement learning model. It should be noted that S1 to S10 below correspond to... Figure 4 S1 to S10 in the previous text Figure 1 S1 to S4 in the text have different meanings.

[0088] S1: Collect historical runtime data via Java Agent Specifically, a Java Agent monitoring proxy deployed on the target JVM non-intrusively collects multi-dimensional runtime data of the target JVM over its historical runtime cycles. These historical runtime cycles can include normal runtime periods and runtime data from previous major promotional events. The collected multi-dimensional runtime data includes GC log data, memory profile data, thread stack data, CPU instruction data, I / O wait data, and performance metrics data.

[0089] For example, for the "order service" of a certain e-commerce platform, the system collects runtime data of the service over the past month during normal times and previous major promotional periods through the deployed Java Agent, including metrics such as the number of orders per second, heap memory usage, average G1 GC pause time, and 90th percentile thread pool queue length.

[0090] S2: Constructing the State-Action-Reward Dataset Specifically, feature extraction and normalization are performed on the collected multi-dimensional runtime data to construct state feature vectors, and a time series dataset containing state-action-reward triples is constructed based on the state feature vectors.

[0091] The construction of the state feature vector includes: for each time moment, calculating the temporal and statistical features of that time moment based on the multi-dimensional runtime data of that time moment and its neighboring time moments; combining the temporal and statistical features extracted for the same time moment into the feature vector of that time moment; and normalizing the value of each dimension of the feature vector at different times to obtain the normalized feature vector corresponding to each time moment as the state feature vector.

[0092] The construction of the state-action-reward triplet includes: for time t, using the normalized feature vector of time t as the current state St, obtaining the current parameter configuration of the target JVM at the same time t as the current action At, acquiring performance index data collected from time t to the next time, calculating the reward value Rt according to a preset reward function, and associating the current state St, current action At, and reward value Rt to construct the triplet (St, At, Rt) for time t. The triples from multiple time points are arranged in chronological order to form a time-series dataset.

[0093] For example, a data engineer defines the state characteristics as [number of orders per second, heap memory utilization, average pause time of G1 GC, and 90th percentile thread pool queue length], and the action space is the key parameters to be tuned as {'-Xmx': ['2g', '4g', '8g'], '-XX:MaxGCPauseMillis': [100, 150, 200]}, and the reward function R is expressed as the throughput improvement coefficient minus the latency growth coefficient.

[0094] S3: Train the offline RL model using safety constraints as penalties. Specifically, an offline reinforcement learning model is constructed, including a JVM parameter tuning action space, a reward function, and safety constraints. The safety constraints are encoded as penalty terms during the training process. Using a constructed historical time-series dataset, the offline reinforcement learning model is trained with the goal of minimizing the empirical loss with penalty terms.

[0095] Safety constraints include one or more of the following: memory limit threshold, GC pause time threshold, and CPU utilization threshold. For example, a safety constraint rule could be set such that if the simulated GC pause time exceeds 200 milliseconds or a memory overflow occurs, a severe penalty is imposed.

[0096] After training, an offline reinforcement learning model (i.e., a policy model) with "high performance and low risk" rules internalized is obtained.

[0097] Phase Two: Online Recommendation and Simulation Validation This stage corresponds to Figure 1The process of obtaining candidate actions using the model in S2 and S3 is used to generate personalized and safe parameter tuning suggestions for the target application and evaluate their reliability through simulation verification.

[0098] S4: Receive the request and obtain the current state S of the target application. current Specifically, operations and maintenance personnel trigger optimization recommendations through the interface. After receiving the recommendation request, the system collects multi-dimensional runtime data of the target JVM within the current cycle through a monitoring agent deployed on the target JVM, and constructs the current state feature vector S according to the feature extraction and normalization methods described above. current .

[0099] For example, a week before a major sales event, operations and maintenance personnel can trigger optimization recommendations through the interface, and the system can obtain the service status S under the current load test. current .

[0100] S5: Strategy model based on S current Recommended Action A candidate Specifically, the acquired current state feature vector S current The input is fed into the trained offline reinforcement learning model (i.e., the policy model), and the model outputs a set of candidate JVM parameter tuning actions A. candidate .

[0101] For example, S current Input a policy model, and the model outputs a set of recommended actions A. candidate This includes: adjusting the -Xmx parameter from 4g to 6g, setting -XX:MaxGCPauseMillis to 180 milliseconds, and setting -XX:ConcGCThreads to 4.

[0102] S6: Simulate the execution of A in a simulation environment candidate Effect Specifically, select a simulation verification environment and perform candidate JVM parameter adjustment action A. candidate It is applied to the target JVM instance launched in the simulation verification environment, and replays the real historical traffic or standard load test cases of the target JVM in the simulation verification environment, and collects performance index data and security constraint index data during the simulation operation.

[0103] Simulation and verification environments include containerized isolated environments (such as Docker containers) or lightweight heuristic simulation environments. When high-fidelity verification is required, a Docker containerized isolated environment should be selected. An image consistent with the application's production environment should be created and the container started, with the JVM parameters modified to A. candidate We then replayed the actual traffic for stress testing.

[0104] For example, the system automatically starts a Docker container with the same image as the production environment, modifies the JVM parameters to {'-Xmx':'6g', '-XX:MaxGCPauseMillis':180, '-XX:ConcGCThreads':4}, and replays a segment of real traffic for load testing.

[0105] S7: Evaluate the simulation results and calculate the overall confidence level. Specifically, based on the collected performance and security constraint data, the performance changes of candidate JVM parameter adjustments compared to the baseline configuration in terms of throughput, latency, and resource utilization are evaluated, as well as whether preset security constraints are violated. The determinism of the offline reinforcement learning model outputting candidate JVM parameter adjustments, the stability of the results of the candidate JVM parameter adjustments during simulation verification, and the state feature vector in the current period used as model input are obtained. A confidence score is calculated based on one or more of the above factors.

[0106] For example, the simulation showed a 22% increase in application throughput, with GC pauses stabilizing below 190 milliseconds and smooth operation. The system assessment indicated a high degree of match between the current state and historical data, high model decision certainty, and minimal fluctuations in simulation results, ultimately awarding a high confidence score of 0.88.

[0107] Phase 3: Delivery and Feedback This stage corresponds to Figure 1 The S4 process and subsequent closed-loop feedback process are used to generate recommendation reports, deploy applications, and collect feedback data.

[0108] S8: Generate recommendation reports with confidence levels and rollback plans Specifically, a JVM parameter recommendation report is generated based on the confidence score calculated by S7. The JVM parameter recommendation report includes: parameter tuning suggestions, rollback plans, and expected performance changes.

[0109] The parameter tuning suggestions include the specific adjustment values ​​of each parameter in the candidate JVM parameter tuning actions, the comparison of parameters before and after the adjustment, and the operation commands or scripts for parameter tuning; the rollback scheme includes the backup information of the current configuration, the rollback operation commands or scripts, and the rollback triggering conditions; the expected performance change includes one or more of the following: the change in throughput, the change in latency, and the change in resource utilization obtained from evaluating the candidate JVM parameter tuning actions.

[0110] For example, the system generates a recommendation report that clearly indicates an expected improvement of 22% with a confidence level of 0.88, and includes specific operation commands such as adjusting -Xmx from 4g to 6g and a script to roll back to the original configuration.

[0111] S9: User approval and one-click deployment Specifically, operations and maintenance personnel review the recommendation reports. If the confidence score is greater than or equal to the preset confidence threshold, the candidate JVM parameter adjustment action is marked as a high-confidence recommendation, and users can execute the deployment with one click during off-peak hours. If the confidence score is less than the threshold, the candidate JVM parameter adjustment action is marked as a low-confidence recommendation, and a manual review suggestion is generated for operations and maintenance personnel to review and decide whether to deploy.

[0112] For example, after the operations and maintenance personnel review the recommendation report, they can perform deployment with one click during off-peak hours, and the system will automatically adjust the JVM parameters from 4g to 6g.

[0113] S10: Monitor the actual effect and feed the data back to S2 ( Figure 4 The database of S2 in the middle Specifically, after deployment, the system continuously monitors actual operating metrics, verifies the optimization effect, and feeds back the actual effect data to the corresponding database for subsequent incremental learning or periodic retraining of the model.

[0114] For example, after deployment, the system monitors actual operating metrics and confirms that the throughput has increased by 20% and the GC pause is as expected. Then, the successful (S_current, A_candidate, actual revenue) data pair is fed back to the training database for the model's subsequent incremental learning, making it more adaptable to such high-stakes promotional scenarios.

[0115] Through the complete process of the above three stages, this optional embodiment realizes a complete closed loop from historical data collection and model training, online recommendation and simulation verification to delivery and feedback, fully embodying the core technical concept of "data collection-offline training-simulation verification-secure deployment" of this invention, and achieving the technical effect of eliminating online trial and error and secure automated optimization.

[0116] Based on and Figure 1 Based on the same principle as the method shown, this embodiment of the invention also provides a JVM parameter adaptive recommendation device 20, such as... Figure 5 As shown, the JVM parameter adaptive recommendation device 20 may include a data acquisition and processing module 210, a candidate JVM parameter adjustment action determination module 220, a confidence score generation module 230, and a recommendation module 240, wherein: The acquisition and processing module 210 is used to non-intrusively acquire multi-dimensional runtime data of the target JVM in the current cycle through a monitoring agent deployed on the target JVM, and to perform feature extraction and normalization processing on the multi-dimensional runtime data to construct a state feature vector. The candidate JVM parameter adjustment action determination module 220 is used to obtain candidate JVM parameter adjustment actions based on the state feature vector and the pre-built offline reinforcement learning model. The offline reinforcement learning model includes the JVM parameter adjustment action space, reward function and security constraints. The confidence score generation module 230 is used to simulate and extrapolate candidate JVM parameter adjustment actions in an isolated environment, evaluate the impact of candidate JVM parameter adjustment actions on the target JVM performance indicators and security constraints, and generate corresponding confidence scores. Recommendation module 240 is used to generate a JVM parameter recommendation report based on the confidence score.

[0117] Optionally, the aforementioned multi-dimensional runtime data includes multi-dimensional runtime data collected at multiple time points. When the acquisition and processing module 210 performs feature extraction and normalization on the multi-dimensional runtime data to construct a state feature vector, it is specifically used for: For each time point, based on the multi-dimensional runtime data of that time point and its neighboring time points, the temporal and statistical characteristics of that time point are calculated. The temporal characteristics of each time point include the trend of indicator change, periodic fluctuations and abrupt change points, and the statistical characteristics of each time point include the mean, variance, percentiles and peak values. The temporal and statistical features extracted at the same time are combined into a feature vector for that time. For each dimension of the feature vector, the value of that dimension at different times is normalized so that the numerical distribution of that dimension is mapped to a preset uniform numerical range, thus obtaining the normalized feature vector corresponding to each time. Based on the normalized feature vectors at each time point, a time series dataset containing state-action-reward triples is constructed as the state feature vector.

[0118] Optionally, when the acquisition and processing module 210 constructs a time-series dataset containing state-action-reward triples as state feature vectors based on the normalized feature vectors at each time point, it is specifically used for: For time t, the normalized feature vector of time t is used as the current state St, the current parameter configuration of the target JVM at the same time t is obtained as the current action At, the performance index data collected from time t to the next time is obtained, the reward value Rt is calculated according to the preset reward function, and the current state St, the current action At, and the reward value Rt are associated to construct the triplet of time t. The triples at multiple time points are arranged in chronological order to form a time series dataset, which is used as the state feature vector for training the offline reinforcement learning model.

[0119] Optionally, the above offline reinforcement learning model is constructed based on the following method: Construct a JVM parameter tuning action space, which includes the adjustable JVM parameter types and the range of JVM parameter values. JVM parameter types include one or more of the following: heap memory size, garbage collector type and parameters, ratio of young generation to old generation, GC pause time target, and number of concurrent GC threads. Construct a reward function that is positively correlated with one or more performance metrics, such as throughput, latency, and resource utilization, and negatively correlated with the degree of violation of security constraints. Construct safety constraints, which may include one or more of the following: memory limit threshold, GC pause time threshold, and CPU utilization threshold. Collect multi-dimensional runtime data of the target JVM within a historical period to construct a time-series dataset for that historical period. The safety constraints are encoded as penalty terms during the training process. Using time-series datasets from historical periods, the offline reinforcement learning model is trained with the goal of minimizing the penalty terms.

[0120] Optionally, the confidence score generation module 230, when simulating candidate JVM parameter adjustment actions in an isolated environment, evaluating the impact of candidate JVM parameter adjustment actions on the target JVM performance indicators and security constraints, and generating corresponding confidence scores, is specifically used for: Choose a simulation verification environment, which can be either a containerized isolated environment or a lightweight heuristic simulation environment; Apply candidate JVM parameter tuning actions to the target JVM instance launched in the simulation verification environment; In a simulated verification environment, replay the target JVM's real historical traffic or standard load test cases, and collect performance and security constraint data during the simulated operation. Based on the collected performance metrics and security constraint metrics, evaluate the performance changes in throughput, latency, and resource utilization of candidate JVM parameter adjustments compared to the baseline configuration, as well as whether they violate preset security constraints. Obtain the degree of determinism when the offline reinforcement learning model outputs candidate JVM parameter adjustment actions; obtain the result stability of candidate JVM parameter adjustment actions during simulation verification; obtain the state feature vector in the current period that serves as the input to the offline reinforcement learning model; The confidence score of the candidate JVM parameter adjustment action is calculated based on one or more of the following factors: degree of certainty, result stability, matching degree between the current period's state feature vector and the time series dataset of historical periods, adjustment magnitude of the candidate JVM parameter adjustment action compared to the current configuration, and the assessed performance change and security constraint violation.

[0121] Optionally, when generating the JVM parameter recommendation report based on the confidence score, the recommendation module 240 is specifically used for: The confidence score is compared with a preset confidence threshold. If the confidence score is greater than or equal to the threshold, the candidate JVM parameter adjustment action is marked as a high-confidence recommendation, and a JVM parameter recommendation report is generated.

[0122] Optionally, the above JVM parameter recommendation report includes: Parameter tuning suggestions: including the specific adjustment values ​​of each parameter in the candidate JVM parameter tuning actions, the comparison of parameters before and after the adjustment, and the operation commands or scripts for parameter tuning; Rollback plan: including the currently configured backup information, rollback operation commands or scripts, and rollback trigger conditions; Expected performance changes: This includes one or more of the following: changes in throughput, changes in latency, and changes in resource utilization, obtained from evaluating candidate JVM parameter tuning actions.

[0123] The JVM parameter adaptive recommendation device of this invention can execute the JVM parameter adaptive recommendation method provided in this invention. The implementation principle is similar. The actions performed by each module and unit in the JVM parameter adaptive recommendation device in each embodiment of this invention correspond to the steps in the JVM parameter adaptive recommendation method in each embodiment of this invention. For detailed functional descriptions of each module of the JVM parameter adaptive recommendation device, please refer to the descriptions in the corresponding JVM parameter adaptive recommendation methods shown above, which will not be repeated here.

[0124] The aforementioned JVM parameter adaptive recommendation device can be a computer program (including program code) running on a computer device, such as an application software; the device can be used to execute the corresponding steps in the method provided in the embodiments of the present invention.

[0125] In some embodiments, the JVM parameter adaptive recommendation device provided in this invention can be implemented using a combination of hardware and software. As an example, the JVM parameter adaptive recommendation device provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the JVM parameter adaptive recommendation method provided in this invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0126] In other embodiments, the JVM parameter adaptive recommendation device provided in this invention can be implemented in software. Figure 5 An adaptive JVM parameter recommendation device stored in memory is shown. It can be software in the form of programs and plug-ins, and includes a series of modules, including a data acquisition and processing module 210, a candidate JVM parameter adjustment action determination module 220, a confidence score generation module 230, and a recommendation module 240, for implementing the adaptive JVM parameter recommendation method provided in the embodiments of the present invention.

[0127] The modules described in the embodiments of the present invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0128] Based on the same principles as the methods shown in the embodiments of the present invention, the embodiments of the present invention also provide an electronic device, which may include, but is not limited to: a processor and a memory; the memory for storing computer programs; and the processor for executing the methods shown in any embodiment of the present invention by invoking the computer programs.

[0129] In one alternative embodiment, an electronic device is provided, such as Figure 6 As shown, Figure 6The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0130] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0131] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0132] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0133] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0134] Among these, electronic devices can also be terminal devices. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0135] This invention provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned method embodiments.

[0136] According to another aspect of the present invention, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various embodiments described above.

[0137] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0138] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0139] The computer-readable storage medium provided in this invention can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0140] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0141] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A JVM parameter adaptive recommendation method, characterized in that, Includes the following steps: S1, through a monitoring agent deployed on the target JVM, non-intrusive multi-dimensional runtime data of the target JVM in the current cycle is collected, and feature extraction and normalization processing are performed on the multi-dimensional runtime data to construct a state feature vector; S2, Based on the state feature vector and the pre-built offline reinforcement learning model, candidate JVM parameter adjustment actions are obtained. The offline reinforcement learning model includes the JVM parameter adjustment action space, reward function, and security constraints. S3, simulate the candidate JVM parameter adjustment actions in an isolated environment, evaluate the impact of the candidate JVM parameter adjustment actions on the target JVM performance indicators and security constraints, and generate corresponding confidence scores; S4. Generate a JVM parameter recommendation report based on the confidence score.

2. The method according to claim 1, characterized in that, The multi-dimensional runtime data includes multi-dimensional runtime data collected at multiple time points. Feature extraction and normalization are performed on the multi-dimensional runtime data to construct a state feature vector, including: For each time point, based on the multi-dimensional runtime data of that time point and its neighboring time points, the temporal and statistical characteristics of that time point are calculated. The temporal characteristics of each time point include the trend of indicator change, periodic fluctuations and abrupt change points, and the statistical characteristics of each time point include the mean, variance, percentiles and peak values. The temporal and statistical features extracted at the same time are combined into a feature vector for that time. For each dimension of the feature vector, the value of that dimension at different times is normalized so that the numerical distribution of that dimension is mapped to a preset uniform numerical range, thus obtaining the normalized feature vector corresponding to each time. Based on the normalized feature vectors at each time point, a time series dataset containing state-action-reward triples is constructed as the state feature vector.

3. The method according to claim 2, characterized in that, The step of constructing a time-series dataset containing state-action-reward triples as the state feature vector based on the normalized feature vectors at each time point includes: For time t, the normalized feature vector of time t is used as the current state St, the current parameter configuration of the target JVM at the same time t is obtained as the current action At, the performance index data collected from time t to the next time is obtained, the reward value Rt is calculated according to the preset reward function, and the current state St, the current action At, and the reward value Rt are associated to construct the triplet of time t. The triples at multiple time points are arranged in chronological order to form a time series dataset, which serves as the state feature vector for training the offline reinforcement learning model.

4. The method according to any one of claims 1 to 3, characterized in that, The offline reinforcement learning model is constructed based on the following method: Construct a JVM parameter tuning action space, which includes adjustable JVM parameter types and JVM parameter value ranges. The JVM parameter types include one or more of the following: heap memory size, garbage collector type and parameters, young generation to old generation ratio, GC pause time target, and number of concurrent GC threads. Construct a reward function that is positively correlated with one or more performance metrics, such as throughput, latency, and resource utilization, and negatively correlated with the degree of violation of security constraints. Construct security constraints, which include one or more of the following: memory upper limit threshold, GC pause time threshold, and CPU utilization threshold; Collect multi-dimensional runtime data of the target JVM within a historical period to construct a time-series dataset for that historical period. The security constraints are encoded as penalty terms during the training process. Using the time series dataset of the historical period, the offline reinforcement learning model is trained with the goal of minimizing the penalty terms.

5. The method according to claim 4, characterized in that, The process of simulating the candidate JVM parameter adjustment actions in an isolated environment, evaluating the impact of the candidate JVM parameter adjustment actions on the target JVM performance indicators and security constraints, and generating corresponding confidence scores includes: Select a simulation verification environment, which may include a containerized isolated environment or a lightweight heuristic simulation environment; The candidate JVM parameter adjustment actions are applied to the target JVM instance launched in the simulation verification environment; In the simulated verification environment, replay the target JVM's real historical traffic or standard load test cases, and collect performance index data and security constraint index data during the simulated operation. Based on the collected performance index data and security constraint index data, evaluate the performance changes of the candidate JVM parameter adjustment actions compared to the baseline configuration in terms of throughput, latency, and resource utilization, and whether they violate the preset security constraints. Obtain the degree of determinism when the offline reinforcement learning model outputs the candidate JVM parameter adjustment action; obtain the result stability of the candidate JVM parameter adjustment action during the simulation verification process; obtain the state feature vector in the current period that serves as the input to the offline reinforcement learning model; The confidence score of the candidate JVM parameter adjustment action is calculated based on one or more of the following factors: the degree of certainty, the stability of the result, the matching degree between the state feature vector in the current period and the time series dataset of the historical period, the adjustment magnitude of the candidate JVM parameter adjustment action compared to the current configuration, and the performance change and security constraint violation obtained from the evaluation.

6. The method according to any one of claims 1 to 3, characterized in that, The step of generating a JVM parameter recommendation report based on the confidence score includes: The confidence score is compared with a preset confidence threshold. If the confidence score is greater than or equal to the threshold, the candidate JVM parameter adjustment action is marked as a high-confidence recommendation, and a JVM parameter recommendation report is generated.

7. The method according to claim 6, characterized in that, The JVM parameter recommendation report includes: Parameter adjustment suggestions: including the specific adjustment values ​​of each parameter in the candidate JVM parameter adjustment actions, the comparison of parameters before and after adjustment, and the operation commands or scripts for parameter adjustment; Rollback plan: including the currently configured backup information, rollback operation commands or scripts, and rollback trigger conditions; Expected performance change: This includes one or more of the following: changes in throughput, changes in latency, and changes in resource utilization, obtained from evaluating the candidate JVM parameter tuning actions.

8. A JVM parameter adaptive recommendation device, characterized in that, include: The acquisition and processing module is used to non-intrusively acquire multi-dimensional runtime data of the target JVM in the current cycle through a monitoring agent deployed on the target JVM, and to perform feature extraction and normalization processing on the multi-dimensional runtime data to construct a state feature vector. The candidate JVM parameter adjustment action determination module is used to obtain candidate JVM parameter adjustment actions based on the state feature vector and the pre-built offline reinforcement learning model. The offline reinforcement learning model includes a JVM parameter adjustment action space, a reward function, and security constraints. The confidence score generation module is used to simulate and extrapolate the candidate JVM parameter adjustment actions in an isolated environment, evaluate the impact of the candidate JVM parameter adjustment actions on the target JVM performance indicators and security constraints, and generate corresponding confidence scores. The recommendation module is used to generate a JVM parameter recommendation report based on the confidence score.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

Images