A terminal software comprehensive analysis method and system based on artificial intelligence

Abstract

This application relates to a method and system for comprehensive analysis of terminal software based on artificial intelligence. This method obtains hardware configuration information of terminal devices and performance indicator data of terminal software, extracts features, and then performs cluster analysis based on historical samples. It then uses a reinforcement learning tuning model to generate software parameter configuration recommendations. It then conducts security assessments using white-box constraint rules and black-box performance prediction models. Ultimately, it generates a software optimization plan and monitors performance changes to optimize the model. This method can mine the inherent structure and potential patterns of the acquired data, implement intelligent tuning under different hardware environments, and dynamically optimize both security and performance, thereby improving the operational efficiency and stability of terminal software.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a terminal software comprehensive analysis method and system based on artificial intelligence. Background Art

[0002] In today's digital age, the widespread use of terminal software has become an integral part of people's lives and work. However, performance issues exposed during its operation are becoming increasingly prominent, posing numerous challenges to user experience and effective use of software.

[0003] Traditional software analysis methods often rely on fixed rules and experience, and are difficult to adapt to the complex and changing terminal device hardware configuration and software operating environment.

[0004] Traditional software performance analysis methods have numerous limitations. Traditionally, the various metrics generated during software operation have been analyzed using a combination of simple statistical analysis and manual judgment. This approach is not only inefficient but also makes it difficult to uncover underlying operational patterns and potential issues from massive amounts of data.

[0005] When performing performance analysis on terminal software, existing technologies lack the ability to intelligently tune software parameter configurations and are unable to effectively tap the potential for software performance optimization. Summary of the Invention

[0006] Based on this, it is necessary to provide an artificial intelligence-based terminal software comprehensive analysis method and system to address the above technical problems.

[0007] In a first aspect, the present application provides a terminal software comprehensive analysis method based on artificial intelligence, comprising:

[0008] Obtaining hardware configuration information of the terminal device and performance indicator data of the terminal software during its operation on the terminal device;

[0009] Perform feature extraction processing on performance indicator data to obtain performance characteristics;

[0010] Add the performance characteristics and hardware configuration information as current sample features to the historical sample feature set to update the historical sample feature set; and perform cluster analysis on all sample features in the historical sample feature set to obtain cluster analysis results for the current sample features;

[0011] Input the current sample characteristics and cluster analysis results into the reinforcement learning tuning model and output software parameter configuration suggestions;

[0012] Conduct security assessments on software parameter configuration recommendations based on white-box constraint rules and black-box performance prediction models, obtain assessment results, and generate software optimization solutions based on the assessment results;

[0013] After configuring the terminal software parameters according to the software optimization plan, monitor the performance change data of the terminal software during its operation on the terminal device, and optimize the reinforcement learning tuning model based on the performance change data;

[0014] The training process of the reinforcement learning tuning model includes:

[0015] Generate an initial sample set by coarse-grained exploration of the software configuration parameter space using a genetic algorithm;

[0016] Construct an Actor-Critic network based on the deep deterministic policy gradient algorithm and pre-train the Actor-Critic network using the initial sample set;

[0017] Obtain software performance feedback data during training and update the network parameters of the Actor-Critic network based on the software performance feedback data and the preset reward function;

[0018] The initial sample set includes states, actions, and rewards. The states include the generated hardware configuration information and generated performance characteristics obtained by the genetic algorithm, as well as the generated clustering analysis results. The generated clustering analysis results are the results of clustering analysis of the generated sample characteristics formed by the generated hardware configuration information and the generated performance characteristics within the historical sample characteristics. The actions are the software parameter configuration recommendations generated by the genetic algorithm. The rewards are the reward values ​​calculated according to the preset evaluation rules.

[0019] The constraints of the white-box constraint rules are:

[0020] ;

[0021] in, It is the upper limit of the thread pool capacity of the terminal software. is the number of CPU cores of the terminal device, It is the upper limit of the JVM heap memory of the terminal software. is the total physical memory of the terminal device, Reserve value for preset non-heap memory, 、 and is the default value.

[0022] In a second aspect, the present application further provides an artificial intelligence-based terminal software comprehensive analysis system for implementing the method described in the first aspect, the system comprising:

[0023] The data acquisition module is used to obtain the hardware configuration information of the terminal device and the performance index data of the terminal software during its operation on the terminal device;

[0024] Feature extraction module, used to perform feature extraction processing on performance indicator data to obtain performance characteristics;

[0025] The cluster analysis module is used to use the performance characteristics and hardware configuration information as the current sample characteristics, add the current sample characteristics to the historical sample characteristic set to update the historical sample characteristic set; and perform cluster analysis on all sample characteristics in the historical sample characteristic set to obtain the cluster analysis results of the current sample characteristics;

[0026] The tuning analysis module is used to input the current sample characteristics and cluster analysis results into the reinforcement learning tuning model and output software parameter configuration suggestions;

[0027] The optimization solution generation module is used to perform security assessments on software parameter configuration suggestions based on white-box constraint rules and black-box performance prediction models, obtain assessment results, and generate software optimization solutions based on the assessment results;

[0028] The feedback optimization module is used to monitor the performance change data of the terminal software during its operation on the terminal device after configuring the parameters of the terminal software according to the software optimization plan, and optimize the reinforcement learning tuning model based on the performance change data.

[0029] In a third aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements an artificial intelligence-based terminal software comprehensive analysis method as in the first aspect.

[0030] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an artificial intelligence-based terminal software comprehensive analysis method as in the first aspect.

[0031] The aforementioned AI-based terminal software comprehensive analysis method and system obtains terminal device hardware configuration information and terminal software performance metrics, extracts features, and then performs cluster analysis based on historical samples. It then utilizes a reinforcement learning tuning model to generate software parameter configuration recommendations. It then conducts security assessments using white-box constraint rules and black-box performance prediction models, ultimately generating software optimization solutions and monitoring performance changes to optimize the model. This method can exploit the inherent structure and potential patterns of the acquired data, enabling intelligent tuning under different hardware environments. It simultaneously balances security and dynamic performance optimization, improving the operational efficiency and stability of terminal software. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 A schematic diagram of a process flow of a terminal software comprehensive analysis method based on artificial intelligence provided by the present invention;

[0034] Figure 2 Schematic diagram of a process flow of a reinforcement learning tuning model training process in an optional embodiment of the present invention;

[0035] Figure 3 This is a structural diagram of an artificial intelligence-based terminal software comprehensive analysis system provided by the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0037] refer to Figure 1 , which presents a flow chart of a terminal software comprehensive analysis method based on artificial intelligence provided by the present application, the method comprising the following steps:

[0038] S10: Obtaining hardware configuration information of the terminal device and performance indicator data of the terminal software during operation on the terminal device.

[0039] Specifically, to obtain hardware configuration information for terminal devices, a scan can be initiated using a system's built-in detection tool or a specially developed information collection module adapted for the terminal system. Terminal devices can include smartphones, tablets, laptops, and other devices. Hardware configuration information can include key hardware parameters such as the central processing unit (CPU) model, number of cores, and clock frequency; RAM capacity and frequency; disk type, capacity, and read / write speeds; and graphics card (GPU) model and memory size.

[0040] When obtaining performance indicator data during the operation of terminal software, you can use the system's built-in tools, such as Windows' Task Manager and MacOS's Activity Monitor. By switching tabs, you can view key indicators such as CPU, memory, disk, etc.; you can also use the monitoring module that comes with professional software. The monitoring module can present the graphics performance, memory usage, and operation response of the software during operation; you can also use third-party monitoring software to obtain software performance indicators; you can also use Python's time module, C++ call system API and other means to directly monitor the dynamic changes of performance data such as operation response time, CPU usage, and memory usage at the code level.

[0041] S20: Perform feature extraction processing on the performance indicator data to obtain performance features.

[0042] Specifically, data mining algorithms can be used for feature extraction. For example, continuously collected performance indicator data is first smoothed to remove abnormal peaks or valleys that may be caused by momentary system fluctuations. Methods such as sliding averages are then used to make the data curve more stable and reliable. Principal component analysis (PCA) technology is then used to extract key components from a multitude of complex performance indicators. For example, in high-dimensional data such as CPU usage, memory usage, and disk I / O, the principal component that best represents the software's operating load is identified as the core performance feature. For performance data with time series characteristics, such as changes in game frame rate over time, spectral analysis methods such as Fourier transforms are used to extract frequency features and determine the periodic fluctuation patterns during software operation, which will more concisely and accurately reflect the essence of software operation.

[0043] S30: adding the performance characteristics and hardware configuration information as current sample characteristics to the historical sample characteristic set to update the historical sample characteristic set; and performing cluster analysis on all sample characteristics in the historical sample characteristic set to obtain a cluster analysis result of the current sample characteristics.

[0044] Specifically, a sample management system can be built. When new current sample features are acquired, they are immediately integrated into the historical sample feature set according to a predetermined data structure format. The historical sample feature set can be stored in a database with high read / write speed and large capacity, such as a distributed NoSQL database, to facilitate easy access and updating. During the cluster analysis phase, an appropriate clustering algorithm is selected, such as the density-based DBSCAN algorithm or the classic K-Means algorithm. Based on the joint feature space of performance characteristics and hardware configuration, similar samples are clustered together. For example, terminals with similar hardware configurations and similar software performance characteristics (such as CPU utilization and memory usage patterns within the same range) are grouped together. This clustering allows the current sample to be quickly located relative to the historical sample system, providing a classification reference for subsequent model predictions and enabling the model to make more accurate judgments based on the common patterns of similar samples.

[0045] S40: Input the current sample features and cluster analysis results into the reinforcement learning tuning model and output software parameter configuration recommendations.

[0046] Specifically, the reinforcement learning tuning model is based on a reinforcement learning algorithm. It interacts with terminal devices and the software runtime environment. Based on the environmental state information reflected by current sample features and cluster analysis results, it learns which software parameter configuration actions to take under different conditions to maximize rewards (such as improved software performance and user experience). Model construction involves defining the state space (represented by current sample features and cluster analysis results), the action space (adjustable software parameter configuration options), the reward function (setting the reward value based on changes in software performance indicators), and selecting an appropriate reinforcement learning algorithm (such as Q-learning or deep deterministic policy gradient algorithm) for training.

[0047] The current sample features and cluster analysis results are fed into a pre-trained reinforcement learning tuning model. Based on its learned knowledge and strategies, the model outputs software parameter configuration recommendations tailored to the current terminal device and software operating status. For example, adjustments can be made to the software's memory allocation strategy, thread scheduling method, graphics rendering parameters, and network communication parameters. By optimizing these parameters, software performance and stability can be improved.

[0048] S50: Perform a security assessment on the software parameter configuration suggestions based on the white-box constraint rules and the black-box performance prediction model, obtain the assessment results, and generate a software optimization plan based on the assessment results.

[0049] Specifically, this step includes two parts: security assessment and optimization solution generation. The security assessment includes white-box constraint rule checking and black-box performance prediction model evaluation.

[0050] (1) White box constraint rule checking:

[0051] A series of white-box constraints can be developed based on software design principles, programming specifications, and safety standards. These rules include constraints on the range of software parameter values, the relationships between parameters, and the software's logical structure. For example, database software may stipulate that the primary key of a data table must be unique and non-null, that foreign keys must reference existing primary keys, and that conditional expressions in query statements must conform to logical operation rules.

[0052] A dedicated rule-checking tool or module can be developed to perform a check on each software parameter configuration suggestion. Based on the defined constraint rules, the checking tool can verify each parameter value in the parameter configuration to determine whether it meets the corresponding constraint conditions. If the parameter value does not meet the constraint conditions, the parameter configuration is considered to pose a security risk and needs to be corrected or discarded.

[0053] The results of the white-box constraint rule check are fed back to the subsequent optimization solution generation phase. If the check results indicate that the software parameter configuration suggestion poses a security risk, the risky parameters need to be adjusted or optimized when generating the software optimization solution to ensure the security and stability of the software.

[0054] (2) Black box performance prediction model evaluation:

[0055] Use historical sample data to build a black-box performance prediction model. This model can be constructed using machine learning algorithms (such as linear regression, support vector machines, and neural networks) or statistical analysis methods (such as time series analysis and regression analysis). The model input is the software parameter configuration, and the output is the predicted value of the software's performance indicators (such as response time, throughput, and resource utilization).

[0056] The performance prediction model is trained using the software parameter configurations and corresponding performance metrics from historical sample data as the training set. By adjusting the model's parameters and structure, the model is able to accurately predict software performance metrics. After training, a portion of untrained data is used as the test set to verify the model's predictive performance. Metrics such as the model's prediction error and accuracy are calculated to assess the model's reliability and effectiveness.

[0057] Input the software parameter configuration recommendations into the trained black-box performance prediction model to obtain predicted performance indicators for the software under these parameters. The performance effect of the software parameter configuration recommendations is evaluated by comparing the predicted values ​​with the optimization targets. If the predicted performance indicators do not meet the optimization targets or there is a significant risk of performance degradation, the recommended parameter configuration is considered infeasible and requires adjustment or optimization.

[0058] (3) Software optimization solution generation:

[0059] Comprehensively evaluate software parameter configuration recommendations based on the results of white-box constraint rule checks and black-box performance prediction model evaluation. If a parameter configuration recommendation satisfies both white-box constraint rules and achieves the optimization goal without causing performance degradation, the recommendation is considered feasible and can be used as part of the software optimization solution. If a parameter configuration recommendation poses a security risk or substandard performance, adjustments and optimizations will be required based on the evaluation results.

[0060] Based on the evaluation results, a software optimization plan is developed. This plan may include the software parameters that need to be adjusted, their new values, the reasons for the adjustments, and the expected effects. Furthermore, auxiliary optimization measures, such as code optimization, algorithm improvements, and resource allocation optimization, may be provided to further improve software performance and security.

[0061] Before implementing a software optimization solution, you can use a simulation environment or test data to verify the solution and evaluate its effectiveness and feasibility. If the verification results show that the optimization solution can achieve the expected results, it can be applied to the actual software system. If the verification results are not ideal, the solution needs to be adjusted and optimized until it meets the requirements.

[0062] S60: After configuring the parameters of the terminal software according to the software optimization solution, monitor the performance change data of the terminal software during its operation on the terminal device, and optimize the reinforcement learning tuning model according to the performance change data.

[0063] Specifically, after implementing the software optimization plan, continuously monitor the performance changes of the terminal software on the terminal device, including previously collected performance indicator data (such as CPU usage and memory usage). By comparing the performance data before and after optimization, the actual effect of the software optimization plan can be evaluated to determine whether the software performance has truly improved and calculate the extent of the performance improvement.

[0064] Performance change data is used as feedback to optimize the reinforcement learning tuning model. Based on performance improvements or declines, the model's reward function, policy parameters, and other factors are adjusted to enable the model to learn more accurate and effective software parameter configuration strategies. For example, if software performance significantly improves, the corresponding reward value can be increased to strengthen the model's learning and memory of the optimization solution. If performance falls short of expectations, the cause can be analyzed and the model's policy adjusted to avoid making incorrect decisions in similar situations. By continuously optimizing the reinforcement learning tuning model based on actual performance feedback, it can better adapt to different terminal devices and software operating environments, improving the effectiveness and accuracy of software parameter configuration optimization.

[0065] This AI-based comprehensive terminal software analysis method obtains terminal device hardware configuration information and terminal software performance metrics, extracts features, and then performs cluster analysis based on historical samples. It then utilizes a reinforcement learning tuning model to generate software parameter configuration recommendations. It then conducts security assessments using white-box constraint rules and black-box performance prediction models. Ultimately, it generates software optimization plans and monitors performance changes to optimize the model. This method can exploit the inherent structure and underlying patterns of the acquired data, enabling intelligent tuning under different hardware environments. It simultaneously balances security and dynamic performance optimization, improving the operational efficiency and stability of terminal software.

[0066] refer to Figure 2 In an optional embodiment, the training process of the reinforcement learning tuning model includes the following steps:

[0067] S41: Use genetic algorithm to conduct coarse-grained exploration of software configuration parameter space and generate initial sample set.

[0068] Specifically, a genetic algorithm is a search algorithm that simulates the biological evolution process, gradually optimizing the quality of candidate solutions through operations such as selection, crossover, and mutation. In this embodiment, a genetic algorithm is used to perform coarse-grained exploration of the software configuration parameter space, generating diverse software parameter configuration suggestions in a relatively short period of time, providing initial samples for subsequent reinforcement learning model training.

[0069] (1) For the generation of state:

[0070] (11) Generation of generative hardware configuration information:

[0071] Genetic algorithms are used to generate a range of possible hardware configuration information, including CPU model, memory capacity, hard disk capacity, graphics card model, etc. For example, CPU models of different performance levels can be generated, such as high-end, mid-end, and low-end models, as well as memory and hard disk combinations of different capacities.

[0072] In this step, the genetic algorithm is applied as follows: a set of hardware configuration information is randomly generated, each containing parameters such as CPU model, memory capacity, hard drive capacity, and graphics card model. Based on a preset fitness function, hardware configuration information with good performance is selected as the parent generation. A crossover operation is performed on the parent generation to generate new hardware configuration information. Mutation is performed on the newly generated hardware configuration information to increase the diversity of the population. The performance of the hardware configuration is evaluated using a fitness function based on performance indicators of the hardware configuration information.

[0073] Specific examples include: generating CPU models of different performance levels, such as high-end (Intel i9), mid-range (Intel i5), and low-end (Intel i3); generating memory configurations of different capacities, such as 8GB, 16GB, and 32GB; generating hard drive configurations of different capacities, such as 256GB, 512GB, and 1TB; and generating graphics card models of different performance levels, such as high-end (NVIDIA RTX 3080), mid-range (NVIDIA GTX 1660), and low-end (NVIDIA GT 1030).

[0074] (12) Generation of generative performance characteristics:

[0075] During the genetic algorithm's exploration process, the software's performance under different hardware environments is simulated based on the generated hardware configuration information, generating corresponding performance characteristic data. These performance characteristics may include CPU usage, memory usage, disk I / O read / write speed, network bandwidth utilization, etc.

[0076] In this step, the performance simulation method involves building a software performance model based on historical data and performance indicators. Based on the generated hardware configuration information, the parameters in the performance model are adjusted to simulate the software's performance under different hardware environments. The simulated performance indicators, such as CPU utilization, memory usage, disk I / O read / write speed, and network bandwidth utilization, are calculated.

[0077] Specific examples include: simulating the software's CPU usage under different loads based on the CPU model and number of cores in the hardware configuration information. simulating the software's memory usage under different operating states based on the memory capacity in the hardware configuration information. simulating the software's disk I / O read / write speed under different operations based on the hard drive type and capacity in the hardware configuration information. simulating the software's bandwidth utilization under different network environments based on the network adapter performance in the hardware configuration information.

[0078] (13) Obtaining generative cluster analysis results:

[0079] The generated hardware configuration information and performance characteristics are combined into generative sample features. These generative sample features are then clustered with features from the historical sample feature set. Cluster analysis reveals the cluster categories to which the generative sample features belong, i.e., the generative cluster analysis results. This result reflects the distribution of the generated samples in the historical sample space, helping the reinforcement learning model better understand the similarities and differences between samples.

[0080] In this step, the cluster analysis method is as follows: The generated hardware configuration information and performance characteristics are combined into generative sample features. Cluster analysis is performed using the K-Means algorithm. The number of clusters, K, is determined using the elbow method. K sample features are randomly selected from the historical sample feature set as initial cluster centers. The distance from the sample features to the cluster centers is calculated using the improved weighted cosine distance formula. Based on the clustering results, the cluster centers of each cluster are recalculated. Each cluster is labeled with a preset cluster label according to the preset criteria.

[0081] A specific example is: Using the elbow method, we determine the number of clusters, K, to be 5. We randomly select five historical sample features as initial cluster centers. We use the improved weighted cosine distance formula to calculate the distance from the generated sample features to each cluster center. Based on the clustering results, we recalculate the cluster center for each cluster. Based on pre-set criteria, we label each cluster with a pre-set cluster label, such as "high load operation mode," "medium load operation mode," or "low load operation mode."

[0082] (2) For the generation of actions:

[0083] Genetic algorithms are used to generate software parameter configuration recommendations as actions in the initial sample set. The genetic algorithm randomly generates an initial population and then gradually optimizes the software parameter configuration through operations such as selection, crossover, and mutation, generating a series of configurations with different performance outcomes. These configurations can include different parameter value combinations, such as different memory allocation strategies, thread scheduling methods, and graphics rendering parameters.

[0084] In this step, the genetic algorithm is applied to randomly generate a set of software parameter configuration suggestions. Each suggestion includes parameters such as memory allocation strategy, thread scheduling method, and graphics rendering parameters. Based on a preset fitness function, the software parameter configuration suggestions with better performance are selected as parents. A crossover operation is performed on the parents to generate new software parameter configuration suggestions. These newly generated software parameter configuration suggestions are mutated to increase the diversity of the population. The fitness function evaluates the performance of the software parameter configuration based on software performance metrics such as response time, throughput, and resource utilization.

[0085] Specific examples include: generating different memory allocation strategies, such as fixed allocation, dynamic allocation, and segmented allocation; generating different thread scheduling methods, such as round-robin scheduling, priority scheduling, and fair scheduling; and generating different graphics rendering parameters, such as resolution, frame rate, and anti-aliasing.

[0086] (3) Reward calculation: The generated software parameter configuration suggestions are evaluated according to the preset evaluation rules, and the corresponding reward value is calculated. The preset evaluation rules can be based on software performance indicators such as response time, throughput, resource utilization, etc., and can also take into account factors such as software stability and compatibility. For example, it can be set that configuration solutions with response times below a certain threshold receive higher reward values, while configuration solutions with response times exceeding the threshold receive lower reward values.

[0087] S42: Construct an Actor-Critic network based on the deep deterministic policy gradient algorithm and pre-train the Actor-Critic network using the initial sample set.

[0088] Specifically, the Actor-Critic network is a reinforcement learning algorithm that combines the policy gradient method and the value function method. By simultaneously learning the policy and value networks, it can not only directly optimize the policy like the policy gradient method, but also use the value function to reduce the variance of the gradient estimate. In this embodiment, the specific structure of the Actor-Critic network is designed as follows:

[0089] (1) Actor network (strategy network):

[0090] Input layer: Receives the current state as input. The state includes hardware configuration information and performance characteristics generated by the genetic algorithm. The number of neurons in the input layer corresponds to the dimensions of the state characteristics. For example, if the state characteristics include 10 dimensions of hardware configuration information, 5 dimensions of performance characteristics, and 1 dimension of cluster analysis results, for a total of 16 dimensions, the input layer has 16 neurons.

[0091] Hidden layers: A fully connected layer structure is used. One or more hidden layers can be set. The number of neurons in each hidden layer can be adjusted based on the actual situation. For example, two hidden layers can be set, each with 64 neurons. The activation function of the hidden layer is usually the ReLU function to introduce nonlinearity and improve the network's expressiveness.

[0092] Output layer: Outputs the probability distribution of actions, that is, the probability of each action being selected in the current state. The number of neurons in the output layer corresponds to the dimensionality of the action space. For example, if the action space contains three adjustable software parameters, each with five possible values, the output layer will have 3 × 5 = 15 neurons. The activation function of the output layer uses the Softmax function to normalize the output values ​​to a probability distribution, ensuring that the sum of the probabilities of all actions is 1.

[0093] (2) Critic Network (Value Network):

[0094] Input layer: It also receives the current state as input. The number of neurons in the input layer is the same as the dimension of the state feature, for example, 15 neurons.

[0095] Hidden layer: Similar to the Actor Network, this layer uses a fully connected layer structure. One or more hidden layers can be set, and the number of neurons in each layer can be adjusted based on the actual situation. For example, two hidden layers can be set, each with 64 neurons, and the activation function is the ReLU function.

[0096] Output layer: Outputs an estimated value for the current state, that is, the expected value of the long-term cumulative reward that can be obtained by following the current strategy in the current state. The output layer has only one neuron and outputs a scalar value representing the value of the state.

[0097] The model formula of the above Actor network is , Indicates the output action, i.e. software parameter configuration suggestion, Represents the Actor network, Indicates the current sample characteristics and cluster analysis results, Represents the parameters of the Actor network. The model formula of the above Critic network is , Indicates the output value, represents the Critic network, Indicates the current sample characteristics and cluster analysis results, Represents the parameters of the Critic network.

[0098] The training process of the Actor-Critic network is as follows: the Actor network and the Critic network learn through alternating optimization. The specific training method is as follows:

[0099] Actor Network Training: The goal of the actor network is to maximize the expected long-term reward. This involves adjusting policy parameters so that the action chosen in the current state results in a higher long-term cumulative reward. During training, the policy gradient is calculated based on the value estimate provided by the critic network, and then the actor network parameters are updated using gradient ascent. Specifically, the actor network first outputs a probability distribution of actions based on the current state. An action is then selected based on this probability distribution and executed to obtain a reward. Next, the current state and reward are input into the critic network to obtain a value estimate for the state. Based on the value estimate and the actual reward, an advantage function is calculated. The advantage function measures the relative performance of the current action relative to the average action. Finally, the gradient of the actor network is calculated based on the advantage function and the policy gradient formula, and the network parameters are updated.

[0100] Critic Network Training: The goal of the Critic Network is to minimize the error between the estimated state value and the actual reward received. Specifically, by adjusting the value function parameters, the network's output state value estimate is closer to the actual long-term cumulative reward. During training, the target value (TargetValue) is calculated based on the current state and the actual reward received. The target value can be estimated using the temporal difference (TD) method, where the value of the current state is equal to the current reward plus the estimated value of the next state multiplied by a discount factor. Next, the error between the state value estimate output by the Critic Network and the target value is calculated, using the mean squared error (MSE) as the loss function. Finally, the parameters of the Critic Network are updated based on the loss function and gradient descent.

[0101] The goal of the Actor network is to maximize the long-term reward, and its update formula is:

[0102] ;

[0103] in, Indicates cumulative rewards, represents the cumulative reward function Actor network parameters The gradient of , which is used to update the parameters of the Actor network , to maximize the cumulative reward . represents the number of samples, and Respectively represent The state and action of each sample.

[0104] The goal of the Critic network is to minimize the square error of the reward value, and its update formula is:

[0105] ;

[0106] in, , Indicates reward, represents the discount factor, Represents the parameters of the target network.

[0107] S43: Obtain software performance feedback data during the training process, and update the network parameters of the Actor-Critic network based on the software performance feedback data and a preset reward function.

[0108] Specifically, (1) the process of obtaining software performance feedback data is as follows:

[0109] After implementing the software optimization plan, run the terminal software in real time and collect performance feedback data using performance monitoring tools. This data includes performance indicators such as software response time, throughput, resource utilization, and error rate, as well as software operating status information (such as CPU temperature and graphics card load). Monitoring data can be obtained through software APIs, system logs, network traffic analysis, and other methods, and stored in a database or file.

[0110] Collected software performance feedback data is preprocessed to remove outliers, fill in missing values, and store it in a unified format. Data can be stored in databases (such as MySQL and PostgreSQL) or distributed storage systems (such as Hadoop HDFS and Apache Cassandra) to facilitate subsequent data analysis and model updates. Additionally, auxiliary information such as timestamps, software version numbers, and device identifiers can be added to the data to facilitate query and analysis.

[0111] (2) The reward function design and calculation process is as follows:

[0112] Design an appropriate reward function based on the software's optimization objectives and performance metrics. Reward functions can be single-objective or multi-objective. Single-objective reward functions typically prioritize a single performance metric, such as the inverse of response time. Multi-objective reward functions can combine multiple performance metrics using methods such as weighted summation or the TOPSIS method.

[0113] Based on real-time software performance feedback, the reward value for each time step or state-action pair is calculated. For example, if software performance data is collected once per time step, the corresponding reward value is calculated based on the performance metric at that time step. The reward value can be used as an immediate reward or accumulated as a long-term reward signal to guide optimization decisions in the reinforcement learning tuning model.

[0114] (3) The network parameter update process is:

[0115] To improve training stability, a target network approach can be used. A target network is a replica of the actor and critic networks, with its parameters periodically copied from the main networks but updated less frequently. Calculating target values ​​through the target network avoids instability during network updates. For example, the target value of the critic network is calculated by the target critic network, and the target actor network is used to generate the target action.

[0116] The loss functions for the actor and critic networks are calculated based on the current state, action, and reward, as well as the target value predicted by the target network. For the critic network, the loss function is the mean squared error loss function, which measures the difference between the network's predicted reward and the target value. For the actor network, the loss function is based on policy gradients, using the advantage function provided by the critic network to update the policy parameters.

[0117] Using the backpropagation algorithm, the parameters of the actor and critic networks are adjusted based on the calculated loss function. Optimization algorithms (such as gradient descent) and learning rates are used to control the step size and direction of parameter updates. For example, for the critic network, the gradient of the mean squared error loss function is used to update the network weights and biases; for the actor network, the policy gradient and the gradient of the advantage function are used to update the policy parameters.

[0118] Using a batch-by-batch update approach, a batch of samples is randomly sampled from the training data, the loss function is calculated, and the parameters are updated. Optimization algorithms such as mini-batch stochastic gradient descent (SGD) or Adam can be used to accelerate learning and convergence. For example, the batch size can be set to 64 or 128, and the network parameters are updated once per batch. Furthermore, a learning rate decay strategy can be used to gradually reduce the learning rate as training progresses to improve model convergence accuracy.

[0119] The detailed derivation process of the above Actor network is as follows:

[0120] (1) Calculate the gradient of each sample: ;

[0121] (2) Calculate the gradient of the Actor network: ;

[0122] (3) Update the parameters of the Actor network: .in, is the learning rate;

[0123] The detailed derivation process of the above Critic network is as follows:

[0124] (1) Calculate the target value : ;

[0125] (2) Calculate the loss function: ;

[0126] (3) Update the parameters of the Critic network: ;in, is the learning rate.

[0127] The aforementioned reinforcement learning tuning model combines the genetic algorithm and the deep deterministic policy gradient (DDPG) algorithm. Compared with standard reinforcement learning algorithms such as A3C, PPO, and DDPG, it has the following key advantages:

[0128] (1) Advantages compared to standard A3C:

[0129] Higher sample utilization efficiency: Although the A3C algorithm improves computational efficiency through multi-threaded asynchronous updates, its sample utilization efficiency is relatively low because each thread running independently may lead to data duplication and redundancy. The current model, on the other hand, uses a genetic algorithm to generate the initial sample set and a deep deterministic policy gradient algorithm for training, making better use of sample data and improving sample utilization efficiency.

[0130] Greater adaptability: The A3C algorithm may have relied too heavily on historical data during training, making it less adaptable to new environments. The current model, through cluster analysis and reinforcement learning optimization, can better adapt to different hardware configurations and software operating environments, generating software parameter configuration recommendations that better meet actual needs.

[0131] More rational reward function design: The reward function of the A3C algorithm is typically simple and fails to fully consider the multi-dimensional indicators of software performance. The current model's reward function comprehensively considers multiple factors such as response time, resource utilization, and abnormal event counts. This allows for a more comprehensive assessment of the performance of software parameter configurations and guides the model to generate higher-quality configuration recommendations.

[0132] (2) Advantages compared with PPO:

[0133] Improved training stability: Although the PPO algorithm improves training stability by limiting the policy update range through a clipping mechanism, it can still reduce training efficiency when handling complex tasks due to overly conservative policy updates. The current model combines the genetic algorithm and the DDPG algorithm, ensuring training stability while allowing for more flexible policy adjustments and improving training efficiency.

[0134] Wider applicability: The PPO algorithm is primarily suitable for tasks in continuous action spaces and performs poorly for tasks in discrete action spaces. The current model uses a genetic algorithm to generate software parameter configuration recommendations, which can handle both discrete and continuous parameter optimization problems and has a wider range of applicability.

[0135] More flexible reward function design: The reward function design of the PPO algorithm is relatively fixed, making it difficult to flexibly adjust according to specific task requirements. The current model's reward function can be customized based on multi-dimensional indicators of software performance, which is more in line with actual optimization needs.

[0136] (3) Advantages compared to DDPG:

[0137] Enhanced exploration capabilities: The DDPG algorithm can get stuck in local optimal solutions during exploration, resulting in insufficient exploration capabilities. The current model uses a genetic algorithm to generate an initial sample set, enabling a more extensive exploration of the parameter space, avoiding local optimal solutions and improving the model's exploration capabilities.

[0138] Higher sample utilization efficiency: The DDPG algorithm has relatively low sample utilization efficiency and requires a large amount of sample data for training. The current model uses a genetic algorithm to generate an initial sample set and uses the DDPG algorithm for training, which can make better use of sample data and improve sample utilization efficiency.

[0139] Greater adaptability: When handling complex tasks, the DDPG algorithm may experience performance degradation due to environmental changes. The current model, through cluster analysis and reinforcement learning optimization, can better adapt to different hardware configurations and software operating environments, generating software parameter configuration recommendations that better meet actual needs.

[0140] Therefore, the above-mentioned reinforcement learning tuning model has significant advantages in sample utilization efficiency, adaptability, reward function design, etc., and can more effectively optimize the performance of terminal software.

[0141] In an optional embodiment, the reward function of the reinforcement learning tuning model during training is expressed as:

[0142] ;

[0143] in, is the software response time before adjustment, is the adjusted software response time, is the adjusted resource utilization, is the preset target resource utilization, is the adjusted abnormal event count, is the preset abnormal event counting threshold, 、 and is the weight coefficient; It is an abnormal event indicator function, which takes 1 when the condition is met, otherwise it takes 0.

[0144] Specifically, the expression of the reward function includes three parts: response time factor, resource utilization factor and abnormal event count factor.

[0145] The response time factor is , which reflects the impact of the relative change of software response time before and after adjustment on the reward value. is the software response time before adjustment, is the adjusted software response time. By calculating the ratio of the difference in response time before and after adjustment to the response time before adjustment, the relative improvement in response time can be obtained. After multiplying by the weight coefficient α, the contribution of this part to the reward value depends on the improvement in response time. If the response time after adjustment is shorter than before adjustment, that is, ,but A positive number contributes positively to the reward value, encouraging software optimization solutions to effectively reduce response time and improve software performance and user experience after adjusting parameters.

[0146] The resource utilization factor is , which reflects the impact of the relationship between the adjusted resource utilization and the preset target resource utilization on the reward value. is the adjusted resource utilization, is the preset target resource utilization. The tanh function (hyperbolic tangent function) The value of is mapped to the range of [-1,1], making the change of the reward value smoother and more stable. After multiplying by the weight coefficient β, the contribution of this part to the reward value depends on the relative size of the adjusted resource utilization and the target resource utilization. If the adjusted resource utilization is close to the target resource utilization, that is, Close to 1, then Close to 0, the impact on the reward value is small; if the adjusted resource utilization is lower than the target resource utilization, that is, Less than 1, then If it is a negative number, it has a negative contribution to the reward value, indicating that further optimization is needed to improve resource utilization; on the contrary, if the adjusted resource utilization is higher than the target resource utilization, then A positive number contributes positively to the reward value, encouraging software optimization solutions to make resource utilization closer to the preset target after adjusting parameters.

[0147] The abnormal event counting factor is , which reflects the impact of the relationship between the adjusted abnormal event count and the preset abnormal event count threshold on the reward value. is the adjusted abnormal event count, It is the preset abnormal event counting threshold. is the abnormal event indicator function, when When , it takes 1, otherwise it takes 0. Multiply by the weight coefficient After that, the contribution of this part to the reward value depends on whether the adjusted abnormal event count exceeds the preset threshold. If the adjusted abnormal event count exceeds the threshold, that is, ,but If it is 1, this part has a negative contribution to the reward value, suggesting that further optimization is needed to reduce the abnormal event count; on the contrary, if the adjusted abnormal event count does not exceed the threshold, then If it is set to 0, this part has no effect on the reward value.

[0148] The weight coefficients α, β, and γ are used to balance the relative importance of response time, resource utilization, and abnormal event count in the reward function. Based on the actual needs and goals of software optimization, the values ​​of these weight coefficients can be adjusted to emphasize the importance of a particular factor or balance the influence of various factors. For example, if the primary goal of software optimization is to improve response speed, the value of α can be increased; if the optimization focus is on controlling resource utilization, the value of β can be increased; if higher requirements are placed on software stability, the value of γ can be increased. By properly setting the weight coefficients, the reward function can better reflect the actual needs and goals of software optimization, thereby guiding the reinforcement learning tuning model to generate software parameter configuration recommendations that better meet actual needs.

[0149] In an optional embodiment, the constraints of the white box constraint rule are:

[0150] ;

[0151] in, It is the upper limit of the thread pool capacity of the terminal software. is the number of CPU cores of the terminal device, It is the upper limit of the JVM heap memory of the terminal software. is the total physical memory of the terminal device, Reserve value for preset non-heap memory, 、 and is the default value.

[0152] Specifically, the constraints of the white box constraint rule include two parts: thread pool capacity upper limit constraint and JVM heap memory upper limit constraint.

[0153] The thread pool capacity upper limit is , this constraint specifies the upper limit of the thread pool capacity of the terminal software Must not exceed the number of CPU cores of the terminal device The value after linear transformation. and This is a preset linear scaling factor used to adjust the thread pool capacity cap based on the number of CPU cores. By limiting the thread pool capacity cap, you can prevent software from creating too many threads, which could lead to excessive CPU resource usage and affect software performance and system stability. Setting the thread pool capacity cap relative to the number of CPU cores ensures that the number of threads matches the CPU's processing power, avoiding thread switching overhead and resource contention caused by excessive threads.

[0154] The JVM heap memory limit is , which specifies the upper limit of the JVM heap memory of the terminal software Must not exceed the total amount of physical memory used by the terminal device After linear transformation and subtracting the preset non-heap memory reservation value The value after . It is a preset linear transformation coefficient used to adjust the JVM heap memory limit according to the total amount of physical memory. By limiting the JVM heap memory limit, you can avoid the software occupying too much physical memory, resulting in insufficient system memory, affecting the operation of other applications and system stability. Ensure that the operating system and other non-Java applications have sufficient memory to avoid system crashes or performance degradation caused by insufficient memory. The JVM heap memory limit is set relative to the total physical memory, ensuring that JVM heap memory usage matches the system's memory resources and preventing excessive memory usage.

[0155] Default value 、 and Used to define the linear transformation coefficients in the white box constraint rules. Set according to the hardware configuration of the terminal device and the operating requirements of the software. The setting method is as follows:

[0156] (1) Determine the preset value :

[0157] Count the number of CPU cores on the terminal device using built-in system tools or third-party hardware detection tools. For example, in Windows, you can view the number of CPU cores using Task Manager; in Linux, you can use the lscpu command to obtain the number of CPU cores. Determine an appropriate range for the thread pool capacity based on the software design and performance requirements. The upper limit of the thread pool capacity should ensure that the software can fully utilize CPU resources without wasting resources or degrading performance due to excessive threads.

[0158] Design a series of experiments, run the terminal software under different CPU core numbers and thread pool capacity configurations, and record the software's performance indicators, such as response time, throughput, CPU utilization, etc. By comparing the performance indicators under different configurations, analyze the relationship between thread pool capacity and the number of CPU cores. For example, you can draw a relationship graph between thread pool capacity and CPU utilization to observe the changing trend of CPU utilization under different thread pool capacities. Based on the experimental data, fit the linear relationship between thread pool capacity and the number of CPU cores. For example, through linear regression analysis, obtain the linear equation between the upper limit of thread pool capacity and the number of CPU cores , thereby determining the preset value .

[0159] (2) Determine the preset value :

[0160] Select a baseline number of CPU cores, such as Run the terminal software under this configuration and record the performance indicators of the software. Under the baseline configuration, gradually adjust the thread pool capacity and record the performance indicators under different thread pool capacities. For example, you can gradually increase the thread pool capacity from 1 to 10 and record the response time and throughput under each capacity. Based on the experimental data, determine the reasonable offset of the thread pool capacity under the baseline configuration. For example, if When the thread pool capacity is 5, the performance is the best, then you can Set to 1, so .

[0161] (3) Determine the preset value :

[0162] Count the total physical memory used by the terminal device. This can be obtained using built-in system tools or third-party hardware detection tools. For example, in Windows, you can view the total physical memory using the Task Manager; in Linux, you can use the free -m command to obtain the total physical memory. Based on the software design and performance requirements, determine a reasonable upper limit for the JVM heap memory. The JVM heap memory limit should ensure that the software can run efficiently but not cause system resource constraints due to excessive memory usage.

[0163] Design a series of experiments to run the terminal software under different total physical memory and JVM heap memory upper limit configurations, and record the software's performance indicators, such as response time, throughput, and memory usage.

[0164] By comparing performance metrics under different configurations, you can analyze the relationship between the JVM heap memory limit and the total physical memory. For example, you can plot the relationship between the JVM heap memory limit and memory utilization to observe how memory utilization changes under different JVM heap memory limits.

[0165] Based on the experimental data, the linear relationship between the JVM heap memory limit and the total amount of physical memory is fitted. For example, through linear regression analysis, the linear equation between the JVM heap memory limit and the total amount of physical memory is obtained. , thereby determining the preset value .

[0166] (4) Comprehensive consideration and optimization:

[0167] Consider the hardware configuration differences of different terminal devices and ensure the preset values 、 and Able to adapt to the operating environment of different devices. For example, for devices with a large number of CPU cores, the For devices with larger physical memory, the value can be increased appropriately. According to the actual performance requirements of the software, adjust the preset value to optimize the software's performance. For example, if the software has high requirements for response time, you can increase The value can be used to increase the thread pool capacity and improve concurrent processing capabilities; if the software has high requirements for memory usage, it can be appropriately reduced. The value can lower the JVM heap memory limit and reduce memory usage.

[0168] During the software operation, the performance indicators of the terminal equipment are continuously monitored, such as CPU usage, memory usage, response time, etc. The preset values ​​are dynamically adjusted according to the monitoring results. 、 and , in order to adapt to changes in the software operating environment. For example, if you find that the CPU usage is too high when the software is running on certain devices, you can appropriately reduce To reduce the thread pool capacity; if you find that the memory usage of the software is too high when running on certain devices, you can appropriately reduce By adjusting the value of to lower the JVM heap memory limit, we can form a feedback loop through continuous monitoring and dynamic adjustment to continuously optimize the preset value and ensure that the software maintains good performance and stability on different terminal devices.

[0169] In an optional embodiment, cluster analysis uses the K-Means algorithm. Cluster analysis is performed on all sample features in the historical sample feature set to obtain the cluster analysis results of the current sample features, including the following steps:

[0170] S1: Use the elbow method to determine the number of clusters K; where K is smaller than the number of sample features in the historical sample feature set;

[0171] S2: Randomly select K sample features from the historical sample feature set as the initial clustering centers;

[0172] S3: Calculate each sample feature in the historical sample feature set according to the following improved weighted cosine distance formula To each cluster center distance :

[0173] ;

[0174] in, is the number of characteristic dimensions of the sample features, is the sample feature No. eigenvalues, is the cluster center No. eigenvalues, For the The weight of the eigenvalues;

[0175] S4: Sample features Assign to the cluster with the nearest cluster center;

[0176] S5: Recalculate each cluster according to the following formula The average value of all sample features in each feature dimension is used as the The cluster center The eigenvalues ​​at the corresponding feature dimensions :

[0177] ;

[0178] in, It is clustering The number of sample features in ;

[0179] S6: Repeat S3 to S5 until the position change of the cluster center between the two iterations is less than the preset threshold or the maximum number of iterations is reached, and the final clustering result is obtained;

[0180] S7: According to the preset criteria, the characteristics of each cluster in the final clustering result are analyzed, and according to the characteristics of each cluster, a preset cluster label is marked for each cluster; the cluster label marked on the cluster to which the current sample characteristics belong is used as the cluster analysis result; wherein different cluster labels represent different software operation modes.

[0181] Specifically, for step S1: The core principle of the elbow method is to determine the value of K by calculating the sum of squared clustering errors (SSE) under different cluster numbers K. SSE measures the sum of the squares of the distances from each sample point to the center of the cluster to which it belongs. As the value of K gradually increases, the SSE will continue to decrease. In actual operation, when plotting the SSE as a function of the K value, the curve usually presents an elbow-like shape. At the inflection point of the curve, that is, the point where the downward trend of the SSE changes from steep to gentle, the corresponding K value is the more appropriate number of clusters. This is because before the inflection point, increasing the number of clusters can significantly reduce the error, while after the inflection point, continuing to increase the number of clusters has little effect on improving the error and increases the computational complexity.

[0182] In step S2: when randomly sampling sample features from the historical sample feature set, a pseudo-random number generator can be used to determine the selected index position. For example, in a programming language, a related random function library can be used to generate K different random indices based on the length of the sample feature set. Then, based on these indices, the corresponding sample features are extracted from the sample feature set as the initial cluster centers.

[0183] For step S3: the distance calculation formula used is the improved weighted cosine distance formula Here The number of feature dimensions representing the sample characteristics, is the sample feature No. eigenvalues, is the cluster center No. eigenvalues, For the The weight of the feature value; for example, in the terminal software analysis, if the characteristics of the three dimensions of CPU usage, memory usage, and disk read and write speed are considered, then For sample characteristics and cluster centers , extract the eigenvalues ​​in each dimension respectively and And determine the weight of each dimension, and then calculate according to the formula to get the distance between the two.

[0184] For step S4: traverse each sample feature in the historical sample feature set, and assign it to the cluster corresponding to the cluster center with the smallest distance according to the distance calculated by S3. In actual programming implementation, you can compare the distances of each sample feature to each cluster center, record the cluster index corresponding to the smallest distance, and then add the sample feature to the corresponding cluster set. For example, for a new sample feature, calculate its distances to the five cluster centers as 2.1, 1.5, 3.2, 2.8, and 1.9, then it will be assigned to the cluster with a distance of 1.5, thus completing a preliminary clustering division of the sample.

[0185] For step S5: follow the formula Update each cluster The cluster center The eigenvalues ​​of each feature dimension . For each cluster , first count the number of sample features it contains Then, for each feature dimension , the characteristic values ​​of all sample features in the cluster on this dimension Sum and divide by the number of samples , get the new cluster center The characteristic value in this dimension. For example, if a cluster has 10 sample features and the characteristic values ​​in the CPU usage dimension are 0.3, 0.4, 0.2, 0.35, 0.42, 0.38, 0.25, 0.33, 0.41, and 0.36, respectively, then the characteristic value of the new cluster center in the CPU usage dimension is the sum of these values ​​divided by 10. This calculation updates the cluster center to make it more representative of the sample feature distribution of the cluster.

[0186] For step S6: the stopping condition includes that the position change of the cluster center between the two iterations is less than a preset threshold or the maximum number of iterations is reached. During each iteration, the distance from the sample feature to the new cluster center is recalculated, and the samples are redistributed and the cluster center is updated. Through continuous iteration, the clustering results are gradually stabilized. For example, the preset threshold is set to 0.001 and the maximum number of iterations is set to 100. During the iteration process, the difference between the two cluster centers in each dimension is calculated. If the difference in all dimensions is less than 0.001, or the number of iterations reaches 100, the iteration is stopped to obtain the final clustering result.

[0187] For step S7: analyze the characteristics of each cluster in the final clustering result according to the preset standard. The preset standard can be set based on some key indicator ranges or pattern characteristics during the software operation process. For example, if the sample characteristics in a cluster generally show that the CPU usage is between 50%-70%, the memory occupancy is between 40%-60%, and the disk read and write speeds are relatively stable, then the cluster can be labeled as a cluster label of "medium load operation mode". The cluster label of the cluster to which the current sample feature belongs is used as its cluster analysis result. Different cluster labels represent different software operation modes. These labels will provide important classification references for subsequent software tuning, helping to more accurately identify the operating characteristics of the software in different environments.

[0188] In an optional embodiment, the weight of each eigenvalue in the improved weighted cosine distance formula is determined by the importance of each eigenvalue. The calculation formula for the importance of each eigenvalue is:

[0189] ;

[0190] in, For the The importance of the eigenvalues, is the performance variance of all samples, is a fixed eigenvalue The conditional variance after taking the value, is the eigenvalue The mean performance change before and after adjustment, is the standard deviation of performance variation.

[0191] Specifically, the performance variance Var(S) of all samples represents the degree of fluctuation of the performance indicators of all samples. Performance indicators can be software response time, throughput, resource utilization, etc. By calculating the performance variance of all samples, we can understand the distribution and fluctuation range of performance indicators in all samples. Assume that the performance indicator value of all samples is , then the performance variance Var(S) of all samples can be calculated by the formula Calculation. Among them, is the mean performance index of all samples, and n is the number of samples.

[0192] Fixed eigenvalues Conditional variance after taking the value Indicated in the eigenvalue The degree of fluctuation of the sample on the performance indicator when taking a specific value. By calculating the conditional variance, we can understand the eigenvalue The degree of influence on the fluctuation of performance indicators. Assuming the characteristic value Get a specific value When , the corresponding sample performance index value is , then the fixed eigenvalue The value is Conditional variance It can be calculated by the following formula:

[0193] in, is a fixed eigenvalue The value is The mean of the sample performance index when is the eigenvalue The value is The number of samples at time .

[0194] The calculation formula for eigenvalue importance is:

[0195] ;

[0196] Among them, the score The performance variance of all samples With fixed eigenvalues Conditional variance after taking the value The difference between the two divided by the performance variance of all samples , which represents the eigenvalue The relative degree of explanation of the fluctuation of performance indicators can be used to compare the importance of different feature values. Represents eigenvalues The relative importance of performance changes before and after adjustment can be used to measure the feature value Sensitivity to performance indicators.

[0197] According to the calculated eigenvalue importance , the weight of each eigenvalue in the improved weighted cosine distance formula can be determined . Weight It can be calculated by the following formula:

[0198] ;

[0199] in, is the total number of eigenvalues, For the The importance of the eigenvalues.

[0200] By calculating the importance of eigenvalues, we can assign appropriate weights to each eigenvalue in the improved weighted cosine distance formula. Eigenvalues ​​with higher importance receive greater weight in the distance calculation, more accurately reflecting the similarities and differences between sample features. This helps improve the accuracy of cluster analysis, making clustering results more consistent with actual software operation modes, and thus providing a more reliable basis for optimizing software parameter configuration.

[0201] In an optional embodiment, performing feature extraction processing on the performance indicator data to obtain performance features includes the following steps:

[0202] Extract time series features of performance indicator data;

[0203] Generate correlation features based on performance indicator data and hardware configuration information; the correlation features include resource utilization features and hardware performance adaptation features;

[0204] Time series features and correlation features are used as performance features.

[0205] Specifically, specialized time series analysis techniques can be used to extract time series features from performance indicator data. First, perform a stationarity test on the time-varying data series of performance indicator data continuously collected during software operation, such as CPU usage and memory utilization. Methods such as the Augmented Dickey-Fuller Test can be used to determine whether the data possesses stable statistical properties. If the data is not stationary, transformations such as differencing may be necessary to make it stationary. Next, analysis using methods such as the autocorrelation function (ACF) and partial autocorrelation function (PACF) can be used to identify patterns such as periodicity and seasonality in the data. For example, ACF and PACF plots may reveal periodic peaks in CPU usage at specific time periods each hour. This pattern can be extracted as a time series feature for subsequent analysis to reflect changes in the software's operating characteristics over time.

[0206] When generating correlation features, performance indicator data and hardware configuration information are combined. For resource utilization features, the average utilization, peak utilization, and utilization fluctuations of different hardware resources (such as CPU, memory, disk I / O, and network bandwidth) are calculated during software execution. For example, if the average CPU utilization during a software run is 40%, but peaks at 80% during certain operations, and the standard deviation of utilization is 10%, these data constitute the CPU resource utilization feature. For hardware performance adaptation features, the hardware's adaptability to the software is assessed by comparing the software's runtime performance indicators with the hardware's theoretical performance ceiling. For example, if a software's memory utilization consistently approaches 80% of the hardware's memory capacity during runtime, and the hardware is configured with relatively low memory specifications, this indicates a possible hardware performance adaptation issue, and this information is considered a hardware performance adaptation feature. These correlation features can more comprehensively reflect the interaction between software and hardware.

[0207] Finally, the extracted time series features and generated correlation features are combined to form performance features. These performance features capture the dynamic changes in software execution over time and its correlation with the hardware environment. Compared to raw performance metrics, they are more refined and representative. They serve as key data for subsequent cluster analysis, model input, and other steps, enabling the entire analysis process to be based on more valuable information, thereby improving the accuracy and reliability of terminal software operation status prediction and optimization.

[0208] In an optional embodiment, extracting time series features of performance indicator data includes the following steps:

[0209] Record the performance indicator data as time series data;

[0210] Calculate the time series data in the past The moving average of time points; is the default value;

[0211] Calculate the performance indicator number in the past based on the moving average and time series data Standard deviation of each time point;

[0212] Use linear regression to fit time series data in the past The data of each time point is used to obtain the slope value of the time series data;

[0213] The moving average, standard deviation, and slope values ​​are used as time series features.

[0214] Specifically, time series data refers to a set of data points arranged in chronological order, with each data point corresponding to a timestamp and a performance indicator value. For example, performance indicator data such as CPU usage, memory usage, and disk I / O read / write speed can be sampled at different time points to form time series data.

[0215] Moving average is a time series analysis method used to smooth data, reduce the impact of short-term fluctuations, and highlight long-term trends. The moving average over the past Q time points means adding the data values ​​of the consecutive Q time points in the time series data and then dividing by Q to get an average value. Assume that the time series data is ,in Indicates the The performance index value of the past Q time points, n is the length of the time series data. For each time point t (Q≤t≤n), the moving average of the past Q time points is It can be calculated by the following formula:

[0216] ;

[0217] The standard deviation is an indicator of the degree of dispersion of data, which indicates the deviation between the data value and the average value. The standard deviation of the past Q time points refers to the degree of deviation between the data values ​​of the time series data at Q consecutive time points and the corresponding moving average. For each time point t (Q≤t≤n), the standard deviation of the past Q time points is It can be calculated by the following formula:

[0218] ;

[0219] Linear regression is a statistical analysis method used to establish a linear relationship model between two variables. In this embodiment, the linear regression method is used to fit the data of the time series data at the past Q time points to obtain the changing trend of the time series data, that is, the slope value. Assume that the time series data is , the corresponding time point is For each time point t (Q≤t≤n), select the data of the past Q time points Perform linear regression fitting, the linear regression model can be expressed as .in, is the slope value, is the intercept. The slope value can be solved by the least square method and other methods. and intercept Slope value Indicates the changing trend of time series data at the past Q time points. If , indicating that the data is on an upward trend; if , indicating that the data is on a downward trend; if , indicating that the data remains basically unchanged.

[0220] Finally, the calculated moving average, standard deviation, and slope value are collectively used as time series features. These features can describe the time series characteristics of performance indicator data from different perspectives. The moving average reflects the central trend of the data, the standard deviation reflects the degree of dispersion of the data, and the slope value shows the direction and rate of change of the data. Time series features that include the above characteristics can provide multi-dimensional information for subsequent software operation status analysis, helping to more accurately identify patterns and potential problems in the software operation process. For example, when analyzing the CPU resource usage of software, these time series features can help determine whether CPU utilization is gradually increasing, leading to a possible performance bottleneck, or fluctuating within a stable range and resource utilization is relatively reasonable.

[0221] The aforementioned AI-based comprehensive analysis method for terminal software first extracts features from performance indicator data, including time series features and correlation features, to comprehensively capture the dynamic changes in software performance and its correlation with hardware configuration. It then uses an improved K-Means clustering algorithm to cluster the sample features, accurately identifying the software's operating mode by reasonably determining the number of clusters, calculating weighted cosine distances, and labeling clusters. It then utilizes a reinforcement learning tuning model, combined with a genetic algorithm and a deep deterministic policy gradient algorithm, to intelligently optimize software parameter configurations, while ensuring the security and rationality of parameter configurations through white-box constraint rules. Through these technical means, software performance can be effectively improved, resource consumption can be reduced, system stability and reliability can be enhanced, and users can be provided with a better software experience. Specifically, it can significantly improve software response speed and throughput, reduce software resource utilization, and reduce the occurrence of abnormal events, thereby improving the overall software performance and user experience.

[0222] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0223] Based on the same inventive concept, embodiments of the present application also provide a system for implementing the aforementioned artificial intelligence-based terminal software comprehensive analysis method. The implementation solution provided by this system is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the artificial intelligence-based terminal software comprehensive analysis system provided below can be found in the limitations of the artificial intelligence-based terminal software comprehensive analysis method described above and will not be repeated here.

[0224] In an exemplary embodiment, Figure 3 As shown, a terminal software comprehensive analysis system 20 based on artificial intelligence is provided, including:

[0225] The data acquisition module 21 is used to acquire the hardware configuration information of the terminal device and the performance index data of the terminal software during its operation on the terminal device.

[0226] The feature extraction module 22 is used to perform feature extraction processing on the performance indicator data to obtain performance features.

[0227] The cluster analysis module 23 is used to use the performance characteristics and hardware configuration information as the current sample characteristics, add the current sample characteristics to the historical sample characteristic set to update the historical sample characteristic set; and perform cluster analysis on all sample characteristics in the historical sample characteristic set to obtain the cluster analysis results of the current sample characteristics.

[0228] The tuning analysis module 24 is used to input the current sample characteristics and cluster analysis results into the reinforcement learning tuning model and output software parameter configuration suggestions.

[0229] The optimization solution generation module 25 is used to perform a security assessment on the software parameter configuration suggestion based on the white box constraint rules and the black box performance prediction model, obtain the assessment results, and generate a software optimization solution according to the assessment results.

[0230] The feedback optimization module 26 is used to monitor the performance change data of the terminal software during its operation on the terminal device after configuring the parameters of the terminal software according to the software optimization solution, and optimize the reinforcement learning tuning model according to the performance change data.

[0231] Optionally, the tuning analysis module 24 includes a tuning training submodule 241. The tuning training submodule 241 includes:

[0232] The initial sample generating unit 2411 is configured to perform a coarse-grained exploration of the software configuration parameter space through a genetic algorithm to generate an initial sample set.

[0233] The network construction and training unit 2412 is used to construct an Actor-Critic network based on a deep deterministic policy gradient algorithm and pre-train the Actor-Critic network using an initial sample set.

[0234] The network updating unit 2413 is used to obtain software performance feedback data during the training process and update the network parameters of the Actor-Critic network according to the software performance feedback data and a preset reward function.

[0235] Among them, the initial sample set includes state, action and reward; the state includes: generative hardware configuration information and generative performance characteristics obtained by the genetic algorithm, as well as the generative clustering analysis results; the generative clustering analysis results are the results obtained by clustering analysis of the generative sample characteristics formed by the generative hardware configuration information and generative performance characteristics in the historical sample characteristics; the action is the software parameter configuration recommendation generated by the genetic algorithm; the reward is the reward value calculated according to the preset evaluation rules.

[0236] Optionally, the reward function in the network updating unit 2413 is expressed as:

[0237] ;

[0238] in, is the software response time before adjustment, is the adjusted software response time, is the adjusted resource utilization, is the preset target resource utilization, is the adjusted abnormal event count, is the preset abnormal event counting threshold, 、 and is the weight coefficient; It is an abnormal event indicator function, which takes 1 when the condition is met, otherwise it takes 0.

[0239] Optionally, the optimization solution generation module 25 includes a white box constraint unit 251. The constraint conditions of the white box constraint unit 251 are:

[0240] ;

[0241] in, It is the upper limit of the thread pool capacity of the terminal software. is the number of CPU cores of the terminal device, It is the upper limit of the JVM heap memory of the terminal software. is the total physical memory of the terminal device, Reserve value for preset non-heap memory, 、 and is the default value.

[0242] Optionally, the cluster analysis module 23 is specifically configured to perform the following steps:

[0243] S1: Use the elbow method to determine the number of clusters K; where K is smaller than the number of sample features in the historical sample feature set;

[0244] S2: Randomly select K sample features from the historical sample feature set as the initial clustering centers;

[0245] S3: Calculate each sample feature in the historical sample feature set according to the following improved weighted cosine distance formula To each cluster center distance :

[0246] ;

[0247] in, is the number of characteristic dimensions of the sample features, is the sample feature No. eigenvalues, is the cluster center No. eigenvalues, For the The weight of the eigenvalues;

[0248] S4: Sample features Assign to the cluster with the nearest cluster center;

[0249] S5: Recalculate each cluster according to the following formula The average value of all sample features in each feature dimension is used as the The cluster center The eigenvalues ​​at the corresponding feature dimensions :

[0250] ;

[0251] in, It is clustering The number of sample features in ;

[0252] S6: Repeat S3 to S5 until the position change of the cluster center between the two iterations is less than the preset threshold or the maximum number of iterations is reached, and the final clustering result is obtained;

[0253] S7: According to the preset criteria, the characteristics of each cluster in the final clustering result are analyzed, and according to the characteristics of each cluster, a preset cluster label is marked for each cluster; the cluster label marked on the cluster to which the current sample characteristics belong is used as the cluster analysis result; wherein different cluster labels represent different software operation modes.

[0254] Optionally, the cluster analysis module 23 is further configured to determine the weight of each eigenvalue in the improved weighted cosine distance formula according to the importance of each eigenvalue. The calculation formula for the importance of each eigenvalue is:

[0255] ;

[0256] in, For the The importance of the eigenvalues, is the performance variance of all samples, is a fixed eigenvalue The conditional variance after taking the value, is the eigenvalue The mean performance change before and after adjustment, is the standard deviation of performance variation.

[0257] Optionally, the feature extraction module 22 includes:

[0258] The time series feature extraction unit 221 is used to extract the time series features of the performance indicator data.

[0259] The correlation feature extraction unit 222 is used to generate correlation features based on the performance indicator data and hardware configuration information; wherein the correlation features include resource utilization features and hardware performance adaptation features.

[0260] The performance feature generating unit 223 is configured to use the time series features and the correlation features as performance features.

[0261] Optionally, the time series feature extraction unit 221 includes:

[0262] The data sorting subunit 2211 is configured to record the performance indicator data as time series data.

[0263] The moving average calculation subunit 2212 is used to calculate the time series data in the past The moving average of time points; is the default value.

[0264] The standard deviation calculation subunit 2213 is used to calculate the performance index number in the past based on the moving average value and the time series data. The standard deviation of each time point.

[0265] The slope value calculation subunit 2214 is used to use the linear regression method to fit the time series data in the past The slope value of the time series data is obtained by analyzing the data at each time point.

[0266] The data integration subunit 2215 is configured to use the moving average, the standard deviation, and the slope value as the time series features.

[0267] An embodiment of the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0268] An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0269] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0270] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. A terminal software comprehensive analysis method based on artificial intelligence, characterized in that: The method comprises: Obtaining hardware configuration information of the terminal device and performance indicator data of the terminal software during operation on the terminal device; Performing feature extraction processing on the performance indicator data to obtain performance features; Adding the performance characteristics and the hardware configuration information as current sample characteristics to a historical sample characteristic set to update the historical sample characteristic set; and performing cluster analysis on all sample characteristics in the historical sample characteristic set to obtain a cluster analysis result of the current sample characteristics; Inputting the current sample features and the cluster analysis results into a reinforcement learning tuning model, and outputting software parameter configuration suggestions; Performing a security assessment on the software parameter configuration suggestion based on white-box constraint rules and a black-box performance prediction model to obtain an assessment result, and generating a software optimization solution based on the assessment result; After configuring the parameters of the terminal software according to the software optimization solution, monitoring performance change data of the terminal software during operation on the terminal device, and optimizing the reinforcement learning tuning model according to the performance change data; The training process of the reinforcement learning tuning model includes: Generate an initial sample set by coarse-grained exploration of the software configuration parameter space using a genetic algorithm; Constructing an Actor-Critic network based on a deep deterministic policy gradient algorithm, and pre-training the Actor-Critic network using the initial sample set; Acquire software performance feedback data during training, and update the network parameters of the Actor-Critic network based on the software performance feedback data and a preset reward function; The initial sample set includes states, actions, and rewards; the states include: generative hardware configuration information and generative performance characteristics obtained by the genetic algorithm, as well as generative cluster analysis results; the generative cluster analysis results are the results of cluster analysis of generative sample characteristics formed by the generative hardware configuration information and the generative performance characteristics within the historical sample characteristics; the actions are software parameter configuration recommendations generated by the genetic algorithm; and the rewards are reward values ​​calculated according to preset evaluation rules. The constraints of the white box constraint rules are: ； in, is the upper limit of the thread pool capacity of the terminal software, is the number of CPU cores of the terminal device, is the upper limit of the JVM heap memory of the terminal software, is the total amount of physical memory of the terminal device, Reserve value for preset non-heap memory, 、 and is the default value.

2. The method according to claim 1, characterized in that The expression of the reward function is: ； in, is the software response time before adjustment, is the adjusted software response time, is the adjusted resource utilization, is the preset target resource utilization, is the adjusted abnormal event count, is the preset abnormal event counting threshold, 、 and is the weight coefficient; It is an abnormal event indicator function, which takes 1 when the condition is met, otherwise it takes 0.

3. The method according to claim 1, characterized in that The cluster analysis adopts K-Means algorithm; The cluster analysis is performed on all sample features in the historical sample feature set to obtain the cluster analysis result of the current sample feature, including: S1: Determine the number of clusters K using the elbow method; wherein K is smaller than the number of sample features in the historical sample feature set; S2: Randomly select K sample features from the historical sample feature set as initial cluster centers; S3: Calculate each sample feature in the historical sample feature set according to the following improved weighted cosine distance formula To each cluster center distance : ； in, is the number of feature dimensions of the sample features, is the sample feature No. eigenvalues, is the cluster center No. eigenvalues, For the The weight of the eigenvalues; S4: Sample features Assign to the cluster with the nearest cluster center; S5: Recalculate each cluster according to the following formula The average value of all the sample features in each feature dimension is used as the The cluster center The eigenvalues ​​at the corresponding feature dimensions : ； in, It is clustering The number of sample features in ; S6: Repeat S3 to S5 until the position change of the cluster center between the two iterations is less than the preset threshold or the maximum number of iterations is reached, and the final clustering result is obtained; S7: Analyze the characteristics of each cluster in the final clustering result according to a preset standard, and mark each cluster with a preset cluster label according to the characteristics of each cluster; use the cluster label marked on the cluster to which the current sample characteristics belong as the cluster analysis result; wherein different cluster labels represent different software operation modes.

4. The method according to claim 3, characterized in that The weight of each eigenvalue in the improved weighted cosine distance formula is determined by the importance of each eigenvalue; The calculation formula for the importance of each of the eigenvalues ​​is: ； in, For the The importance of the eigenvalues, is the performance variance of all samples, is a fixed eigenvalue The conditional variance after taking the value, is the eigenvalue The mean performance change before and after adjustment, is the standard deviation of performance variation.

5. The method according to any one of claims 1 to 4, characterized in that The performing feature extraction processing on the performance indicator data to obtain performance features includes: Extracting time series features of the performance indicator data; Generate correlation features based on the performance indicator data and the hardware configuration information; wherein the correlation features include resource utilization features and hardware performance adaptation features; The time series features and the correlation features are used as the performance features.

6. The method according to claim 5, characterized in that The extracting of the time series features of the performance indicator data includes: Recording the performance indicator data as time series data; Calculate the time series data in the past The moving average of time points; is the default value; Calculate the performance index number in the past based on the moving average and the time series data Standard deviation of each time point; Use linear regression to fit the time series data in the past The data of each time point are used to obtain the slope value of the time series data; The moving average, the standard deviation, and the slope value are used as the time series features.

7. An artificial intelligence-based terminal software comprehensive analysis system, used to implement the method according to any one of claims 1 to 6, characterized in that: The system comprises: A data acquisition module, used to obtain hardware configuration information of the terminal device and performance indicator data of the terminal software during operation on the terminal device; A feature extraction module is used to perform feature extraction processing on the performance indicator data to obtain performance characteristics; a cluster analysis module, configured to use the performance characteristics and the hardware configuration information as current sample characteristics, add the current sample characteristics to a historical sample characteristic set to update the historical sample characteristic set, and perform cluster analysis on all sample characteristics in the historical sample characteristic set to obtain a cluster analysis result of the current sample characteristics; A tuning analysis module, configured to input the current sample features and the cluster analysis results into a reinforcement learning tuning model and output software parameter configuration suggestions; an optimization solution generation module, configured to perform a security assessment on the software parameter configuration suggestion based on white-box constraint rules and a black-box performance prediction model, obtain an assessment result, and generate a software optimization solution based on the assessment result; A feedback optimization module is used to monitor the performance change data of the terminal software during its operation on the terminal device after configuring the parameters of the terminal software according to the software optimization solution, and to optimize the reinforcement learning tuning model according to the performance change data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

Images