Software system performance prediction method based on active learning

Abstract

The invention relates to the technical field of software system performance prediction, in particular to a software system performance prediction method based on active learning, and the method comprises the steps: obtaining and preprocessing multi-dimensional performance data, selecting a to-be-labeled sample through an active learning strategy, and constructing a labeled sample pool. Comparing the user feedback data in the labeling sample pool with the labeling result, and rejecting consistent samples to obtain a difficult case labeling sample pool; and constructing a training set and a test set according to the samples in the difficult case labeling sample pool, and carrying out model training and evaluation. And if the model evaluation index reaches a preset threshold value, deploying the performance prediction reference model, and generating a final performance prediction model. Otherwise, carrying out labeled sample selection, model training and model evaluation again until the threshold requirement is met; and constructing user analogue simulation data, and inputting the user analogue simulation data into the performance prediction model to obtain a system performance prediction result. Finally, the performance prediction efficiency of the software system is remarkably improved while the prediction accuracy is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of software system performance prediction, and in particular to a software system performance prediction method based on active learning. Background Art

[0002] With the rapid development of computer technology and the Internet, game software has become an important part of the entertainment industry, especially for multiplayer online games, where players usually pay more attention to the performance of the game. The smoothness, response speed and server stability of the game have become key factors affecting the player's gaming experience. Performance prediction during the testing phase can effectively optimize system performance, improve game quality, identify potential bottlenecks in advance, and ensure stable operation of the game after it goes online. Through accurate prediction of game software system performance, developers can discover and solve potential problems before going online, optimize resource allocation, and prepare in advance to deal with peak load solutions, avoid server overload, and reduce delays and freezes in game scene images, thereby improving player satisfaction and retention, and reducing risks and maintenance costs after going online.

[0003] Performance prediction methods based on traditional machine learning usually rely on a large amount of labeled data, but when the game system is highly complex, the process of obtaining data labels is extremely time-consuming and costly. In addition, with the unpredictability of player behavior and the increasing dynamics of the game system, models that rely solely on historical static data for training are usually difficult to cope with dynamic changes in the external environment, such as a surge in players caused by advertising. This results in large errors in system performance prediction after the game software is launched, increasing the difficulty of system optimization.

[0004] To solve this problem, existing technologies attempt to combine simulation technology to generate simulated data to reduce reliance on labeled data. However, existing simulation methods are usually difficult to generate high-quality simulated data that matches real user behavior, resulting in the inability to effectively verify game performance before going online. This bottleneck of data annotation and simulation seriously affects the accuracy and timeliness of system performance prediction, and will result in the inability to discover performance bottlenecks after going online in advance, affecting players' gaming experience and product market performance. There is an urgent need to develop a performance prediction method that can significantly improve efficiency while ensuring prediction accuracy to cope with dynamically changing system requirements.

[0005] Therefore, a software system performance prediction method based on active learning is proposed. Summary of the invention

[0006] The object of the present invention is to provide a software system performance prediction method based on active learning, which is mainly used in the development and testing stage of online multiplayer game software. The method first obtains multidimensional performance data, preprocesses the multidimensional performance data, generates standard performance data, and obtains sample prediction results and confidence distribution through an initial model. The samples to be labeled are selected through an active learning strategy to construct a labeled sample pool. According to user feedback data, reference data is obtained; data whose reference data is consistent with the corresponding labeling results in the labeled sample pool is removed from the labeled sample pool to obtain a difficult labeled sample pool; and samples are selected from the difficult labeled sample pool to obtain a training set and a test set. The base learner is trained using the training set to generate a performance prediction benchmark model, and the model is evaluated through the test set. If the model evaluation index reaches a preset threshold, the model is deployed as a performance prediction model, otherwise the labeled sample selection and model training are performed again. By constructing user simulation data and inputting it into the performance prediction model, the performance of the system after it goes online can be predicted. The method can effectively improve the accuracy and efficiency of software system performance prediction through active learning and simulation technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A software system performance prediction method based on active learning, comprising:

[0009] Acquiring multi-dimensional performance data of the online multi-player game software during the testing phase; the multi-dimensional performance data includes user feedback data;

[0010] Constructing a performance prediction model according to the multidimensional performance data includes the following steps:

[0011] S1: preprocessing the multidimensional performance data to obtain standard performance data; inputting the standard performance data into an initial model to obtain sample prediction results and confidence distribution;

[0012] S2: Select samples to be labeled through active learning strategy to form a sample set to be labeled;

[0013] S3: constructing a labeled sample pool according to the sample set to be labeled;

[0014] S4: Obtain the user feedback data corresponding to the samples in the annotated sample pool to obtain reference data; if the reference data is consistent with the corresponding annotation results in the annotated sample pool, remove the samples corresponding to the reference data from the annotated sample pool to obtain a difficult annotated sample pool;

[0015] S5: Select samples from the difficult example labeled sample pool to obtain a training set and a test set; S6: Input the training set into the base learner for training to obtain a performance prediction benchmark model; input the test set into the performance prediction benchmark model for model prediction to obtain a model evaluation index;

[0016] S7: If the model evaluation index reaches the preset threshold, deploy the performance prediction benchmark model to obtain the performance prediction model; otherwise, repeat S2 to S7 until the model evaluation index reaches the preset threshold;

[0017] Construct user simulation data, input the user simulation data into the performance prediction model, and obtain performance prediction results.

[0018] Furthermore, the multi-dimensional performance data also includes: system performance data, user operation data and server performance data.

[0019] Furthermore, the initial model is specifically:

[0020] The initial model is a model trained based on historical multi-dimensional performance data, and is used to predict the standard performance data and output the sample prediction result and the confidence distribution;

[0021] The initial model is one of the following models: a random forest model, a support vector machine model, a convolutional neural network model, and a long short-term memory network model.

[0022] Furthermore, selecting the sample to be labeled by the active learning strategy includes:

[0023] Through uncertainty sampling, samples whose confidence intervals of the sample prediction results fall within a preset interval are selected from unlabeled samples to obtain fuzzy samples; according to the distance of the fuzzy samples in the feature space, the fuzzy samples whose distances to the labeled samples are greater than a preset distance threshold are selected, and the samples to be labeled are determined by a staged sampling method.

[0024] Furthermore, constructing the labeled sample pool according to the sample set to be labeled includes: extracting all the samples to be labeled in the sample set to be labeled, removing the user feedback data in the samples to be labeled, and obtaining standard samples to be labeled; sending all the standard samples to be labeled to the expert client, and receiving the labeling results sent by the expert client; integrating and storing the labeling results and the labeled sample set to obtain the labeled sample pool.

[0025] Furthermore, if the reference data is consistent with the corresponding annotation result in the annotated sample pool, after removing the sample corresponding to the reference data from the annotated sample pool, the method further includes the following steps:

[0026] Filtering out samples whose reference data are consistent with the corresponding annotation results from the annotated sample pool to generate a discarded sample set;

[0027] The excluded sample set is stored in an unlabeled data pool, and the samples in the unlabeled data pool are input into the trained performance prediction benchmark model to generate prediction labels, and the prediction labels are used as pseudo-labeling results to form pseudo-labeling samples; according to the confidence of the pseudo-labeling samples, the confidence weight of the pseudo-labeling results is dynamically adjusted to generate a high-confidence pseudo-labeling sample set;

[0028] An adaptive sample weight allocation mechanism is introduced to merge the samples in the high-confidence pseudo-annotated sample set with the samples in the difficult-example annotated sample pool to construct a second training set;

[0029] Inputting the second training set into a semi-supervised learning model, and iteratively training the pseudo-labeled samples through multiple rounds of self-correction mechanisms to obtain a second performance prediction model;

[0030] After each round of iterative training, using the test set to perform performance evaluation on the second performance prediction model to obtain a second model evaluation index;

[0031] If the second model evaluation index exceeds the model evaluation index of the same training round, the second performance prediction model is used to replace the performance prediction benchmark model; if the second model evaluation index does not exceed the model evaluation index of the same training round, the performance prediction benchmark model is maintained.

[0032] Furthermore, selecting samples from the difficult example labeled sample pool to obtain the training set and the test set includes: classifying the samples in the difficult example labeled sample pool by scene to obtain multiple scene classification subsets; and using stratified sampling to extract samples from each of the scene classification subsets at a preset ratio according to the size of the scene classification subsets and the sample distribution to obtain the training set and the test set.

[0033] Furthermore, the base learner is one of the following models: a logistic regression model, a gradient boosting decision tree model, and an extreme gradient boosting model.

[0034] Furthermore, constructing the user simulation data includes:

[0035] The advertising data, social media popularity data, reservation data, historical player data and player data of similar games of the online multiplayer game software are obtained in real time, and user simulation data within a specified time period is constructed in combination with the online time of the online multiplayer game software; the user simulation data is used to predict user feedback data, system performance data, user operation data and server performance data at various time points after the online multiplayer game software is launched.

[0036] Further, constructing the user simulation data within the specified time period includes:

[0037] Dynamically predict the user attention and traffic trend of the online multiplayer game software in the specified time period by acquiring the advertising data, the social media popularity data and the reservation data of the online multiplayer game software and combining the machine learning model; generate traffic prediction data according to the user attention and the traffic trend;

[0038] A multi-agent system is introduced, in which each agent in the multi-agent system represents a different type of player; each agent combines reinforcement learning and behavior modeling technology to adaptively simulate the behavior patterns, online time and operation frequency of the different types of players, and simulates the operation behaviors of the different types of players based on the historical player data, the data of players of the same game and the traffic prediction data;

[0039] Construct time series data and a time series data set by using the historical player data and the player data of the same game, wherein each data point in the time series data set represents the change of player behavior or system performance in different time periods; train a time series prediction model based on the time series data set, wherein the time series prediction model is used to simulate the traffic fluctuation of the user group after the online multiplayer game software is launched; the intelligent agent adaptively adjusts its behavior according to environmental changes; and dynamically predicts the distribution of the number of users and the complexity of player operations during peak periods through the time series data;

[0040] Combined with the simulation optimization mechanism of reinforcement learning, the intelligent agent maximizes game participation and operation frequency through continuous interaction with the virtual environment, and generates user behavior data within the specified time period; the simulation optimization mechanism simulates high-load scenarios in different time periods by evaluating the impact of internal and external game environmental factors on user behavior;

[0041] The user simulation data is generated by the multi-agent system and the time series prediction model, including the user feedback data, the system performance data, the user operation data and the server performance data at each time point within a specified time period.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The present invention effectively improves the training efficiency and prediction accuracy of the model by introducing active learning strategies and difficult example screening mechanisms. Eliminating samples whose annotation results are consistent with reference data can reduce the interference of repeated information on the model, concentrate training resources on more challenging and error-prone samples, and help improve the model's learning ability for complex scenarios. By removing samples whose annotations are consistent between users and experts from the annotated sample pool, unnecessary computing overhead can be reduced, the model can be prevented from repeatedly processing known information, thereby shortening the training time and improving the training efficiency of the model. At the same time, this elimination mechanism allows the model to pay more attention to difficult samples, thereby optimizing the model's learning process, making it more focused on difficult samples, and improving the model's predictive ability in complex and changing game environments.

[0044] 2. The present invention introduces a pseudo-labeling strategy and an adaptive sample weight allocation mechanism under the framework of active learning, making full use of samples that are consistent with the reference data and the labeling results but not involved in the training to improve the generalization ability of the model. In the active learning process, the samples in the excluded sample set are stored in the unlabeled data pool, and the prediction labels are generated for them through the pseudo-labeling strategy, which can avoid data waste and provide more sample learning opportunities, so that the semi-supervised model also has learning opportunities for the data in the excluded sample set and the unlabeled data. The dynamic adjustment of the confidence of the pseudo-labeled samples ensures that the model focuses on high-confidence samples during the training process, thereby further improving the training efficiency and the accuracy of model prediction. The adaptive sample weight allocation mechanism enhances the learning stability of the model for pseudo-labeled data by assigning weights to different samples, and optimizes the model performance through a multi-round self-correction mechanism. Finally, after each round of iterative training, performance evaluation is performed to ensure that the model is replaced only when the performance is improved, thereby ensuring the stability and continuous optimization of the model. This method reduces the labeling cost, improves the prediction accuracy of the model in complex dynamic scenes, and enhances the robustness of the system.

[0045] 3. The present invention can accurately simulate the number of users and behavior patterns in different time periods before the game goes online by constructing user simulation data. Real-time acquisition of advertising data, social media popularity data, reservation data, historical player data and similar game player data, the diversity and real-time nature of these data sources improve the accuracy of simulation data, and fully consider the dynamic impact of external environment data. Compared with static data, by comprehensively analyzing various types of real-time dynamic data, the user traffic trend after the game goes online can be more fully predicted. This process provides developers with dynamic predictions of user behavior and load changes, making system performance predictions more reliable. At the same time, combined with machine learning models and multi-agent systems, it is possible to dynamically predict the user traffic and behavior patterns of online multiplayer game software within a specific time period. With the help of reinforcement learning and behavior modeling technology, it is possible not only to simulate the operating behaviors of different types of players, but also to adaptively adjust the player behavior patterns according to environmental changes, thereby providing more realistic and diverse user simulation scenarios. The introduction of this simulation optimization mechanism helps to further improve the accuracy of system performance prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flow chart of a software system performance prediction method based on active learning provided by an embodiment of the present invention;

[0047] Figure 2 A flowchart of steps based on active learning provided by an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of sample annotation ratio changes during active learning provided by an embodiment of the present invention;

[0049] Figure 4 A comparison chart of the actual number of users and the predicted number of users of an online multiplayer game based on an active learning model provided by an embodiment of the present invention;

[0050] Figure 5 A comparison chart of the actual server load and the predicted server load of an online multiplayer game based on an active learning model provided by an embodiment of the present invention;

[0051] Figure 6 A comparison chart of the actual number of online multiplayer game users and the predicted number of users based on a traditional prediction model provided by an embodiment of the present invention;

[0052] Figure 7 A comparison chart of the actual server load and the predicted server load of an online multiplayer game based on a traditional prediction model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Embodiment 1:

[0055] Company A plans to launch a massively multiplayer online role-playing game, which is currently in the testing phase. In order to ensure that the game can provide players with a stable operating experience and effectively handle high-concurrency server requests after it goes online, Company A applied a software system performance prediction method based on active learning during the game testing phase.

[0056] See also Figures 1 to 7 The present invention provides a software system performance prediction method based on active learning, and the technical solution is as follows:

[0057] See also Figure 1 , a software system performance prediction method based on active learning, comprising:

[0058] Acquiring multi-dimensional performance data of the online multi-player game software during the testing phase; the multi-dimensional performance data includes user feedback data;

[0059] Furthermore, the multi-dimensional performance data also includes: system performance data, user operation data and server performance data. The system performance data, user operation data and server performance data are all comprehensive test data of the online multiplayer game software based on different scenarios and different users during the testing phase.

[0060] The present invention can comprehensively analyze and predict the performance of online multiplayer game software from multiple dimensions. By integrating different types of test data, the present invention can not only capture the impact of user operations on system performance, but also monitor the server load and performance bottlenecks in real time. This combination of multidimensional data effectively improves the model's ability to handle complex scenarios and can lay the foundation for accurately predicting system performance under different conditions.

[0061] Construct a performance prediction model based on the multi-dimensional performance data, see Figure 2 , including the following steps:

[0062] See also Figure 2 S1 in S1: preprocess the multidimensional performance data to obtain standard performance data; input the standard performance data into the initial model to obtain sample prediction results and confidence distribution;

[0063] Furthermore, the initial model is specifically:

[0064] The initial model is a model trained based on historical multi-dimensional performance data, and is used to predict the standard performance data and output the sample prediction result and the confidence distribution;

[0065] The initial model is one of the following models: a random forest model, a support vector machine model, a convolutional neural network model, and a long short-term memory network model.

[0066] Specifically, the initial model is a binary classification model, which predicts the standard performance data and outputs the sample prediction results, where the sample prediction results consist of normal performance and abnormal performance.

[0067] The main function of the initial model is to make a preliminary prediction of the preprocessed multidimensional performance data, generate the prediction results of the samples and their confidence distribution. These prediction results are not only used to screen the samples to be labeled, but also provide support for uncertainty sampling in active learning. In this embodiment, the selection of the initial model is flexible, and models such as random forests, support vector machines, convolutional neural networks, or long short-term memory networks can be used according to different data types to ensure that the model can fully learn the complex relationships in the multidimensional performance data. In addition, the initial model lays the foundation for subsequent difficult sample screening, pseudo-labeled data generation and other steps by learning the potential structure in the multidimensional performance data, which can help reduce the workload of labeling and optimize the efficiency and accuracy of model training by screening out samples with high uncertainty.

[0068] See also Figure 2 S2, S2: select the samples to be labeled through active learning strategy to form the sample set to be labeled;

[0069] Furthermore, selecting the sample to be labeled by the active learning strategy includes:

[0070] Through uncertainty sampling, samples whose confidence intervals of the sample prediction results fall within a preset interval are selected from unlabeled samples to obtain fuzzy samples; according to the distance of the fuzzy samples in the feature space, the fuzzy samples whose distances to the labeled samples are greater than a preset distance threshold are selected, and the samples to be labeled are determined by a staged sampling method.

[0071] In this embodiment, the confidence interval is [0.4, 0.6], which reflects that the confidence of the model in predicting these samples is in a fuzzy state, neither uncertain nor extremely uncertain. These samples are usually the most valuable for improving the learning of the model. The preset distance threshold is 0.3.

[0072] Table 1. Sample data table in the unlabeled sample pool

[0073]

[0074] All unlabeled samples constitute the unlabeled sample pool. Table 1 shows some sample data in the unlabeled sample pool. Sample screening is performed by combining confidence and the minimum distance from the labeled samples. First, the prediction confidence of samples C and D falls between 0.4 and 0.6, which meets the selection criteria for uncertainty sampling. Secondly, by analyzing their minimum distance from the labeled samples, it can be seen that: the distance of sample D is 0.9, and the distance of sample C is 0.8, both of which are greater than the preset distance threshold of 0.3, and maintain a large difference from the labeled sample set.

[0075] Therefore, considering the confidence and distance of the samples, samples C and D meet the sampling conditions and are subsequently labeled. This selection strategy can ensure that the newly labeled samples are sufficiently representative and diverse, thereby effectively improving the generalization ability of the model.

[0076] Through uncertainty sampling, samples whose confidence intervals fall within a preset interval are selected to ensure that the model focuses on labeling those ambiguous samples with unclear prediction results, thereby giving priority to solving the most uncertain parts of the model prediction and reducing unnecessary labeling work. Secondly, combined with the distance metric in the feature space, samples that are significantly different from the labeled samples are further screened out, which ensures the diversity and representativeness of the newly labeled samples and avoids problems such as data redundancy or excessive sample similarity. The introduction of the phased sampling method ensures that the selection process of the samples to be labeled is orderly and efficient, so that each iteration can maximize the generalization ability and prediction accuracy of the model while minimizing the labeling cost. In the end, the method not only reduces the labeling cost, but also improves the robustness and adaptability of the model in complex performance prediction scenarios.

[0077] See also Figure 2 S3 in which: S3: constructing a labeled sample pool according to the sample set to be labeled;

[0078] Furthermore, constructing the labeled sample pool according to the sample set to be labeled includes: extracting all the samples to be labeled in the sample set to be labeled, removing the user feedback data in the samples to be labeled, and obtaining standard samples to be labeled; sending all the standard samples to be labeled to the expert client, and receiving the labeling results sent by the expert client; integrating and storing the labeling results and the labeled sample set to obtain the labeled sample pool.

[0079] In this embodiment, the user feedback data can be regarded as the user's annotation of the system performance data, the user operation data and the server performance data in the multidimensional performance data. User feedback data can be regarded as a weak annotation, which reflects the performance status of the system through the user's behavior and experience during actual use. Although this annotation is not obtained through manual annotation by experts, it can provide indirect information about system performance. Therefore, before annotating the multidimensional performance data, it needs to be deleted to avoid interference. The expert client is used to annotate the standard samples to be annotated, ensuring the professionalism and accuracy of the annotation results. Finally, the expert annotation results are integrated with the original annotated sample set and stored in the annotated sample pool, which further enriches the diversity and representativeness of the training data and provides more comprehensive sample support for model training.

[0080] See also Figure 2 S4 in S4: obtaining the user feedback data corresponding to the samples in the labeled sample pool to obtain reference data; if the reference data is consistent with the corresponding labeling result in the labeled sample pool, removing the samples corresponding to the reference data from the labeled sample pool to obtain a difficult labeled sample pool;

[0081] Specifically, in this embodiment, by comparing the user feedback data in the labeled sample pool with the labeling results, the consistent samples are eliminated to obtain a difficult labeled sample pool. By comparing the user feedback data in the labeled sample pool with the labeling results, it is possible to effectively identify which samples have labeling results that are consistent with the user feedback, and remove these consistent samples from the labeled sample pool. This process not only reduces the repeated selection of labeled and simple samples and improves the training efficiency of labeled samples, but also helps to screen out samples whose labeling results are inconsistent with user feedback, that is, difficult samples with high labeling uncertainty. Further labeling and processing of these difficult samples will help improve the model's predictive ability and generalization performance in complex scenarios.

[0082] If the user feedback data is empty, it is considered that the reference data is inconsistent with the corresponding annotation results in the annotation sample pool.

[0083] See also Figure 2 S5 in S5: Select samples from the difficult example labeled sample pool to obtain a training set and a test set; refer to Figure 2 S6 in which: inputting the training set into a base learner for training to obtain a performance prediction benchmark model; inputting the test set into the performance prediction benchmark model for model prediction to obtain a model evaluation index;

[0084] See also Figure 2S7, S7: if the model evaluation index reaches the preset threshold, deploy the performance prediction benchmark model to obtain the performance prediction model; otherwise, repeat steps S2 to S7 until the model evaluation index reaches the preset threshold;

[0085] The data annotation process based on active learning is shown in Figure 3 , Figure 3 The figure shows the process of changing the ratio of labeled data to unlabeled data during multiple labeling processes. Each horizontal bar represents standard performance data. This dataset is divided into two parts: labeled data (gray part on the left) and unlabeled data (white part on the right). The figure shows that in the active learning process, as the number of training rounds increases, the relative proportion of labeled data gradually increases, and the proportion of unlabeled data gradually decreases.

[0086] In this embodiment, Figure 3 The K value in is 15, that is, after 15 rounds of training, the model evaluation index (prediction accuracy) reaches the preset threshold (95%).

[0087] Specifically, in the subsequent process of repeating step S2 to step S7, M samples to be labeled have been selected for expert labeling in the first round, and M is 60 in this embodiment; the repeated steps are specifically as follows:

[0088] Continue to select M samples to be labeled through the active learning strategy to form a sample set to be labeled; construct a labeled sample pool based on the sample set to be labeled, extract all the samples to be labeled in the labeled sample pool (except for samples that have been labeled in previous rounds), remove the user feedback data in the samples to be labeled, and obtain standard samples to be labeled; send all the standard samples to be labeled to the expert client, and receive the labeling results sent by the expert client; integrate the labeling results and the labeled sample set and store them in the labeled sample pool. Obtain the user feedback data corresponding to the samples in the labeled sample pool to obtain reference data; if the reference data is consistent with the corresponding labeling results in the labeled sample pool, remove the samples corresponding to the reference data from the labeled sample pool to obtain a difficult labeled sample pool; divide the newly labeled data in the difficult labeled sample pool and add them to the training set and the test set respectively, use the incremental data newly added to the training set to perform incremental training of the model to obtain a performance prediction benchmark model; then use the updated test set to test the model to obtain model evaluation indicators;

[0089] If the model evaluation index reaches the preset threshold, the performance prediction benchmark model is deployed to obtain the performance prediction model; if the model evaluation index does not reach the preset threshold, the above-mentioned repeated steps are continued.

[0090] Construct user simulation data, input the user simulation data into the performance prediction model, and obtain performance prediction results.

[0091] Specifically, the method further includes: optimizing and adjusting the online multiplayer game software according to the performance prediction result.

[0092] Furthermore, if the reference data is consistent with the corresponding annotation result in the annotated sample pool, after removing the sample corresponding to the reference data from the annotated sample pool, the method further includes the following steps:

[0093] Filtering out samples whose reference data are consistent with the corresponding annotation results from the annotated sample pool to generate a discarded sample set;

[0094] The initial standard performance data includes all samples. After being labeled by experts, all samples are divided into labeled samples and unlabeled samples. The unlabeled data pool includes all unlabeled samples.

[0095] The excluded sample set is stored in an unlabeled data pool, and the samples in the unlabeled data pool are input into the trained performance prediction benchmark model to generate prediction labels, and the prediction labels are used as pseudo-labeling results to form pseudo-labeling samples; according to the confidence of the pseudo-labeling samples, the confidence weight of the pseudo-labeling results is dynamically adjusted to generate a high-confidence pseudo-labeling sample set;

[0096] An adaptive sample weight allocation mechanism is introduced to merge the samples in the high-confidence pseudo-annotated sample set with the samples in the difficult-example annotated sample pool to construct a second training set;

[0097] Inputting the second training set into a semi-supervised learning model, and iteratively training the pseudo-labeled samples through multiple rounds of self-correction mechanisms to obtain a second performance prediction model;

[0098] After each round of iterative training, using the test set to perform performance evaluation on the second performance prediction model to obtain a second model evaluation index;

[0099] If the second model evaluation index exceeds the model evaluation index of the same training round, the second performance prediction model is used to replace the performance prediction benchmark model; if the second model evaluation index does not exceed the model evaluation index of the same training round, the performance prediction benchmark model is maintained.

[0100] Specifically, the semi-supervised learning model can be: a generative adversarial network model, a graph convolutional network model, an autoencoder model or a semi-supervised support vector machine model.

[0101] In order to dynamically adjust the confidence weights of pseudo-annotated samples, this embodiment operates through the following process: according to the confidence of the model for each pseudo-annotated sample, samples with different confidence levels are assigned different weights, thereby ensuring that pseudo-annotated samples with high confidence levels have a greater impact on the model, while samples with low confidence levels have a smaller impact. The following is a specific adjustment example:

[0102] Each pseudo-annotated sample has a confidence score i , whose value is the probability value of the model's prediction result for the sample, and the confidence level is between 0 and 1; a confidence threshold θ is set. When the confidence level is higher than the confidence threshold θ, the sample is considered to be a high confidence sample, otherwise it is considered to be a low confidence sample; for samples higher than the threshold θ, a higher weight is given. For example, for confidence level i For samples >θ, the weights are calculated using the following formula:

[0103] ω i =1+(confidence i -θ);

[0104] For samples below the threshold, a lower weight is given, and the weight is calculated by the following formula:

[0105] ω i =confidence i / θ;

[0106] Example calculation: The confidence of sample A is 0.95, the confidence of sample B is 0.6, and the threshold confidence threshold θ = 0.7;

[0107] For sample A: A =1+(confidence i -θ)=1+(0.95-0.7)=1+0.25=1.25;

[0108] For sample B: B =0.6 / 0.7=0.857;

[0109] After the weight adjustment, sample A contributes more to the model due to its high confidence, while sample B has a smaller weight and contributes relatively less to the model. This dynamic adjustment ensures that the model focuses on learning high-confidence samples, thereby improving the overall generalization ability of the model. During model training, the loss function value is adjusted by multiplying each sample by the corresponding weight. For example, when using the weighted cross entropy loss function, the formula is as follows:

[0110]

[0111] Among them, y is the true label, is the prediction result, ω i is the weight of the ith sample, is the weighted cross entropy loss function.

[0112] By removing samples whose reference data is consistent with the annotation results from the annotated sample pool and reintroducing them into the unlabeled data pool for pseudo-annotation processing, the workload of expert annotation can be reduced. At the same time, the existing data can be fully and effectively utilized to improve training efficiency. The introduction of high-confidence pseudo-annotated samples and the combination of an adaptive sample weight allocation mechanism can help balance the impact of pseudo-annotated samples and difficult-to-annotate samples, thereby enhancing the generalization ability of the model. In the semi-supervised learning process, through multiple rounds of self-correction mechanisms, the potential of unlabeled data can be fully tapped while reducing the cost of expert annotation. In addition, the system performs performance evaluation after each round of iterative training, selects the optimal model by comparing the performance of different models, and ensures that the performance prediction model finally deployed has the best prediction accuracy and stability.

[0113] Furthermore, selecting samples from the difficult example labeled sample pool to obtain the training set and the test set includes: classifying the samples in the difficult example labeled sample pool by scene, as shown in Table 2, to obtain multiple scene classification subsets; according to the size of the scene classification subsets and the sample distribution, using stratified sampling to extract samples from each of the scene classification subsets at a preset ratio to obtain the training set and the test set.

[0114] Table 2. Sample stratified sampling table based on different scenarios

[0115]

[0116] This method of obtaining training and test sets can ensure that data in different scenarios are fully representative and covered by training. This method of classification by scenario and stratified sampling can effectively avoid the problem of unbalanced sample selection, making the data in the training and test sets more diverse and representative, thereby improving the generalization ability and prediction accuracy of the performance prediction model in different game scenarios. In addition, stratified sampling can ensure that data in scarce or complex scenarios are not ignored, reducing volatility in the evaluation process and making the evaluation results more stable and reliable.

[0117] In this embodiment, the preset ratio includes a preset training set sampling ratio and a preset training set sampling ratio. The specific ratio can be adjusted according to actual conditions. The sum of the preset training set sampling ratio and the preset training set sampling ratio does not exceed 1.

[0118] Furthermore, the base learner is one of the following models: a logistic regression model, a gradient boosting decision tree model, an extreme gradient boosting model (XGBoost model).

[0119] By using logistic regression model, gradient boosting decision tree model and extreme gradient boosting model as base learners, a variety of optional base learners are provided, which improves the flexibility and adaptability of the model.

[0120] Furthermore, constructing the user simulation data includes:

[0121] The advertising data, social media popularity data, reservation data, historical player data and player data of similar games of the online multiplayer game software are obtained in real time, and the user simulation data within the specified time period is constructed in combination with the online time of the online multiplayer game software; the user simulation data is used to predict the user feedback data, system performance data, user operation data and server performance data at various time points after the online multiplayer game software is launched. In this embodiment, the specified time period is within seven days from the online time of the game.

[0122] By acquiring real-time advertising data, social media popularity data, reservation data, historical player data, and player data of similar games, combined with the game launch time, and constructing user simulation data, it is possible to more accurately simulate user behavior and quantity changes after the game is launched. Due to the dynamic changes in the external environment (such as advertising strategies, fluctuations in social media discussion heat, and the release of similar competitive games), this fusion of multi-dimensional data makes the simulation data closer to the actual scene, and the prediction results are more reliable. It can effectively support the accurate estimation of the number of users and behavior patterns at different time points, thereby helping the game development team to identify potential system pressures and user behavior changes in advance. In this way, game operators can better optimize server resource allocation, improve user experience, and reduce the risk of performance problems and insufficient resources that may occur after the game is launched.

[0123] Further, constructing the user simulation data within the specified time period includes:

[0124] Dynamically predict the user attention and traffic trend of the online multiplayer game software in the specified time period by acquiring the advertising data, the social media popularity data and the reservation data of the online multiplayer game software and combining the machine learning model; generate traffic prediction data according to the user attention and the traffic trend;

[0125] A multi-agent system is introduced, in which each agent in the multi-agent system represents a different type of player; each agent combines reinforcement learning and behavior modeling technology to adaptively simulate the behavior patterns, online time and operation frequency of the different types of players, and simulates the operation behaviors of the different types of players based on the historical player data, the data of players of the same game and the traffic prediction data;

[0126] Construct time series data and a time series data set by using the historical player data and the player data of the same game, wherein each data point in the time series data set represents the change of player behavior or system performance in different time periods; train a time series prediction model based on the time series data set, wherein the time series prediction model is used to simulate the traffic fluctuation of the user group after the online multiplayer game software is launched; the intelligent agent adaptively adjusts its behavior according to environmental changes; and dynamically predicts the distribution of the number of users and the complexity of player operations during peak periods through the time series data;

[0127] Combined with the simulation optimization mechanism of reinforcement learning, the intelligent agent maximizes game participation and operation frequency through continuous interaction with the virtual environment, and generates user behavior data within the specified time period; the simulation optimization mechanism simulates high-load scenarios in different time periods by evaluating the impact of internal and external game environmental factors on user behavior;

[0128] The user simulation data is generated by the multi-agent system and the time series prediction model, including the user feedback data, the system performance data, the user operation data and the server performance data at each time point within a specified time period.

[0129] Through the dynamic prediction of multi-dimensional data fusion and machine learning models, the present invention can estimate the user's attention and traffic trend in advance, and then generate traffic prediction data. The introduction of a multi-agent system combined with reinforcement learning and behavior modeling technology enables the behavior patterns of different types of players to be simulated, and adaptive user behavior prediction and dynamic adjustment of online time and operation frequency can be achieved. Through the time series prediction model, user traffic fluctuations and peak operation complexity after the game is launched can be predicted. This simulation optimization mechanism enables the agent to maximize the simulation of actual player behavior, thereby predicting system performance, server load and user behavior patterns in advance. Through the comprehensive evaluation of virtual and real environments, the generated user simulation data is more realistic and forward-looking.

[0130] In order to verify the effectiveness of the software system performance prediction method based on active learning proposed in this invention, a set of comparative experiments was designed, which included multiple traditional machine learning-based performance prediction models, including unsupervised, semi-supervised and supervised learning algorithms. In the comparative experiments, different numbers of labeled samples were designed for each algorithm, and they were compared in detail with the active learning-based method proposed in this invention in different dimensions.

[0131] Table 3. Comparison data of different prediction models

[0132]

[0133] Referring to Table 3, through the active learning strategy, the method proposed in the present invention only needs 25% of the labeled samples, the labeling time and training time are reduced to 30 hours and 40 hours respectively, and the accuracy rate reaches 93.3%. Compared with supervised learning, the method of the present invention achieves basically the same prediction accuracy with less labeling and training resources.

[0134] Another set of comparative experiments is designed below to verify the effectiveness of the software system performance testing method based on the active learning model proposed in the present invention in predicting the number of online multiplayer game users and server load compared to the traditional prediction model. The active learning model of the present invention and the traditional machine learning prediction model are used to predict the number of online multiplayer game users and server load in a specific time period, and the prediction results are compared with the actual data. The user simulation data and the corresponding performance prediction results are collated and analyzed, and the number of users and server loads in different time periods are summarized in days as the statistical unit, and the prediction effects of the two models are further analyzed.

[0135] See also Figure 4 , Figure 4 This is a comparison chart of the prediction results of the number of online multiplayer game users based on the active learning model proposed by the present invention and the actual number of users. This figure shows the prediction of the number of users by the active learning model at different time points, combined with Figure 6 For comparison, Figure 6 This is a comparison chart of the prediction results of the traditional prediction model for the number of online multiplayer game users and the actual number of users. The results show that the active learning model can better track the changing trend of the number of users, especially during peak periods, when the prediction results are closer to the actual number of users with a smaller error.

[0136] See also Figure 5 , Figure 5 This is a comparison chart of the server load prediction results based on the active learning model proposed in the present invention and the actual server load. Figure 7 For comparison, Figure 7 The figure below is a comparison of the prediction results of the traditional prediction model for server load and the actual server load. The traditional model has a large error in load prediction, especially when the load is high. The model fails to accurately capture the increase in server load, resulting in a large deviation between the prediction result and the actual situation. Therefore, in server load prediction, the active learning model also performs better than the traditional model, especially when the load is peak. The model can effectively predict the upward trend of the server load, reduce the prediction deviation at the load peak, and ensure the stable operation of the system.

[0137] comprehensive Figure 4 , Figure 5 , Figure 6 and Figure 7It can be clearly observed that the method based on the active learning model proposed in the present invention has accuracy and adaptability in predicting the number of users and server load, which better reflects the game user behavior and system load changes, and improves the stability and operation efficiency of the system.

[0138] Experimental results show that the active learning model of the present invention can more accurately capture changes in user behavior and system load trends, thereby improving the prediction accuracy and reliability of online multiplayer game systems.

[0139] The present invention effectively solves the problem of prediction accuracy caused by insufficient or unbalanced labeled samples in the performance prediction process of traditional software systems by introducing active learning strategies. By preprocessing the multidimensional performance data of online multiplayer game software in the test phase, combined with the prediction results and confidence distribution of the initial model, the samples to be labeled can be accurately screened, which effectively reduces unnecessary labeling work and reduces the labeling cost. At the same time, by gradually building a labeled sample pool and eliminating samples consistent with user feedback data, a difficult labeled sample pool is generated, so that the model can focus on the learning of difficult samples and improve the generalization ability of the model. Further, through the training and testing of the base learner, the performance prediction model is continuously optimized to ensure that the model can be predicted stably and accurately after reaching the preset threshold. In addition, by constructing user simulation data and inputting it into the performance prediction model, the user behavior and server load of online multiplayer games can be simulated more comprehensively and dynamically, thereby improving the prediction accuracy and stability of the system. The software system performance prediction method based on active learning proposed by the present invention significantly improves the learning efficiency and prediction effect of the model, and is suitable for large-scale, dynamically changing online multiplayer game environments.

[0140] Embodiment 2:

[0141] Company B is a large game software development company. It is conducting the final stress test for its upcoming massively multiplayer online game software. This game is released to players around the world and is expected to attract a large number of players from all continents, countries and regions in the first few days after its release. As players are widely distributed across multiple time zones, especially during peak hours, the server will face extremely high concurrent requests from different network environments around the world. In order to ensure a smooth gaming experience during peak hours, Company B plans to conduct a simulation test, focusing on the performance of the system under ultra-high concurrent player operations.

[0142] Company B applied a software system performance prediction method based on active learning, including:

[0143] Acquiring multi-dimensional performance data of the online multi-player game software during the testing phase; the multi-dimensional performance data includes user feedback data;

[0144] Constructing a performance prediction model according to the multidimensional performance data includes the following steps:

[0145] S1: preprocessing the multidimensional performance data to obtain standard performance data; inputting the standard performance data into an initial model to obtain sample prediction results and confidence distribution;

[0146] S2: Select samples to be labeled through active learning strategy to form a sample set to be labeled;

[0147] S3: constructing a labeled sample pool according to the sample set to be labeled;

[0148] S4: Obtain the user feedback data corresponding to the samples in the annotated sample pool to obtain reference data; if the reference data is consistent with the corresponding annotation results in the annotated sample pool, remove the samples corresponding to the reference data from the annotated sample pool to obtain a difficult annotated sample pool;

[0149] S5: Select samples from the difficult example labeled sample pool to obtain a training set and a test set; S6: Input the training set into the base learner for training to obtain a performance prediction benchmark model; input the test set into the performance prediction benchmark model for model prediction to obtain a model evaluation index;

[0150] S7: If the model evaluation index reaches the preset threshold, deploy the performance prediction benchmark model to obtain the performance prediction model; otherwise, repeat S2 to S7 until the model evaluation index reaches the preset threshold;

[0151] Construct user simulation data, input the user simulation data into the performance prediction model, and obtain performance prediction results.

[0152] Specifically, the method further includes: optimizing and adjusting the online multiplayer game software according to the performance prediction results. Based on the prediction results, the developer team optimizes the configuration of the game server, adjusts the game's matching algorithm and content loading strategy, and ensures that the game can still provide a stable low-latency experience during peak hours, thereby improving player satisfaction and game retention rate.

[0153] Furthermore, the multi-dimensional performance data also includes: system performance data, user operation data and server performance data.

[0154] Furthermore, the initial model is specifically:

[0155] The initial model is a model trained based on historical multidimensional performance data, which is used to predict the standard performance data and output the sample prediction results and the confidence distribution; the historical multidimensional performance data has exactly the same dimensions as the multidimensional performance data, but the historical multidimensional performance data comes from the recorded data of the system during its past operation, including historical system performance data, historical user operation data, historical server performance data and historical user feedback data.

[0156] The initial model is one of the following models: a random forest model, a support vector machine model, a convolutional neural network model, and a long short-term memory network model.

[0157] In this embodiment, the sample prediction result (prediction label) is performance reaching the standard (1) or performance not reaching the standard (0); the confidence distribution is the confidence of the prediction result (eg, the probability of performance reaching the standard is 90%).

[0158] Furthermore, selecting the sample to be labeled by the active learning strategy includes:

[0159] Through uncertainty sampling, samples whose confidence intervals of the sample prediction results fall within a preset interval are selected from unlabeled samples to obtain fuzzy samples; according to the distance of the fuzzy samples in the feature space, the fuzzy samples whose distances to the labeled samples are greater than a preset distance threshold are selected, and the samples to be labeled are determined by a staged sampling method.

[0160] Furthermore, constructing the labeled sample pool according to the sample set to be labeled includes: extracting all the samples to be labeled in the sample set to be labeled, removing the user feedback data in the samples to be labeled, and obtaining standard samples to be labeled; sending all the standard samples to be labeled to the expert client, and receiving the labeling results sent by the expert client; integrating and storing the labeling results and the labeled sample set to obtain the labeled sample pool.

[0161] Furthermore, if the reference data is consistent with the corresponding annotation result in the annotated sample pool, after removing the sample corresponding to the reference data from the annotated sample pool, the method further includes the following steps:

[0162] Filtering out samples whose reference data are consistent with the corresponding annotation results from the annotated sample pool to generate a discarded sample set;

[0163] The excluded sample set is stored in an unlabeled data pool, and the samples in the unlabeled data pool are input into the trained performance prediction benchmark model to generate prediction labels, and the prediction labels are used as pseudo-labeling results to form pseudo-labeling samples; according to the confidence of the pseudo-labeling samples, the confidence weight of the pseudo-labeling results is dynamically adjusted to generate a high-confidence pseudo-labeling sample set;

[0164] An adaptive sample weight allocation mechanism is introduced to merge the samples in the high-confidence pseudo-annotated sample set with the samples in the difficult-example annotated sample pool to construct a second training set;

[0165] Inputting the second training set into a semi-supervised learning model, and iteratively training the pseudo-labeled samples through multiple rounds of self-correction mechanisms to obtain a second performance prediction model;

[0166] After each round of iterative training, using the test set to perform performance evaluation on the second performance prediction model to obtain a second model evaluation index;

[0167] If the second model evaluation index exceeds the model evaluation index of the same training round, the second performance prediction model is used to replace the performance prediction benchmark model; if the second model evaluation index does not exceed the model evaluation index of the same training round, the performance prediction benchmark model is maintained.

[0168] Furthermore, selecting samples from the difficult example labeled sample pool to obtain the training set and the test set includes: classifying the samples in the difficult example labeled sample pool by scene to obtain multiple scene classification subsets; and using stratified sampling to extract samples from each of the scene classification subsets at a preset ratio according to the size of the scene classification subsets and the sample distribution to obtain the training set and the test set.

[0169] Furthermore, the base learner is one of the following models: a logistic regression model, a gradient boosting decision tree model, and an extreme gradient boosting model.

[0170] Furthermore, constructing the user simulation data includes:

[0171] The advertising data, social media popularity data, reservation data, historical player data and player data of similar games of the online multiplayer game software are obtained in real time, and user simulation data within a specified time period is constructed in combination with the online time of the online multiplayer game software; the user simulation data is used to predict user feedback data, system performance data, user operation data and server performance data at various time points after the online multiplayer game software is launched.

[0172] Further, constructing the user simulation data within the specified time period includes:

[0173] Dynamically predict the user attention and traffic trend of the online multiplayer game software in the specified time period by acquiring the advertising data, the social media popularity data and the reservation data of the online multiplayer game software and combining the machine learning model; generate traffic prediction data according to the user attention and the traffic trend;

[0174] A multi-agent system is introduced, in which each agent in the multi-agent system represents a different type of player; each agent combines reinforcement learning and behavior modeling technology to adaptively simulate the behavior patterns, online time and operation frequency of the different types of players, and simulates the operation behaviors of the different types of players based on the historical player data, the data of players of the same game and the traffic prediction data;

[0175] Construct time series data and a time series data set by using the historical player data and the player data of the same game, wherein each data point in the time series data set represents the change of player behavior or system performance in different time periods; train a time series prediction model based on the time series data set, wherein the time series prediction model is used to simulate the traffic fluctuation of the user group after the online multiplayer game software is launched; the intelligent agent adaptively adjusts its behavior according to environmental changes; and dynamically predicts the distribution of the number of users and the complexity of player operations during peak periods through the time series data;

[0176] Combined with the simulation optimization mechanism of reinforcement learning, the intelligent agent maximizes game participation and operation frequency through continuous interaction with the virtual environment, and generates user behavior data within the specified time period; the simulation optimization mechanism simulates high-load scenarios in different time periods by evaluating the impact of internal and external game environmental factors on user behavior;

[0177] The user simulation data is generated by the multi-agent system and the time series prediction model, including the user feedback data, the system performance data, the user operation data and the server performance data at each time point within a specified time period.

[0178] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A software system performance prediction method based on active learning, characterized in that: include: Obtain multi-dimensional performance data of online multiplayer game software during the testing phase; The multi-dimensional performance data includes user feedback data; Constructing a performance prediction model according to the multidimensional performance data includes the following steps: S1: preprocessing the multidimensional performance data to obtain standard performance data; inputting the standard performance data into an initial model to obtain sample prediction results and confidence distribution; S2: Select samples to be labeled through active learning strategy to form a sample set to be labeled; S3: constructing a labeled sample pool according to the sample set to be labeled; S4: Obtain the user feedback data corresponding to the samples in the annotated sample pool to obtain reference data; if the reference data is consistent with the corresponding annotation results in the annotated sample pool, remove the samples corresponding to the reference data from the annotated sample pool to obtain a difficult annotated sample pool; S5: Select samples from the difficult example labeled sample pool to obtain a training set and a test set; S6: Input the training set into the base learner for training to obtain a performance prediction benchmark model; input the test set into the performance prediction benchmark model for model prediction to obtain a model evaluation index; S7: If the model evaluation index reaches the preset threshold, deploy the performance prediction benchmark model to obtain the performance prediction model; otherwise, repeat S2 to S7 until the model evaluation index reaches the preset threshold; Construct user simulation data, input the user simulation data into the performance prediction model, and obtain performance prediction results.

2. The software system performance prediction method based on active learning according to claim 1, characterized in that: The multi-dimensional performance data also includes: system performance data, user operation data and server performance data.

3. The software system performance prediction method based on active learning according to claim 1, characterized in that: The initial model is specifically: The initial model is a model trained based on historical multi-dimensional performance data, and is used to predict the standard performance data and output the sample prediction result and the confidence distribution; The initial model is one of the following models: a random forest model, a support vector machine model, a convolutional neural network model, and a long short-term memory network model.

4. The software system performance prediction method based on active learning according to claim 1, characterized in that: Selecting the sample to be labeled by the active learning strategy includes: Through uncertainty sampling, samples whose confidence intervals of the sample prediction results fall within a preset interval are selected from unlabeled samples to obtain fuzzy samples; according to the distance of the fuzzy samples in the feature space, the fuzzy samples whose distances to the labeled samples are greater than a preset distance threshold are selected, and the samples to be labeled are determined by a staged sampling method.

5. The software system performance prediction method based on active learning according to claim 1, characterized in that: Constructing the labeled sample pool according to the sample set to be labeled includes: extracting all the samples to be labeled in the sample set to be labeled, removing the user feedback data in the samples to be labeled, and obtaining standard samples to be labeled; sending all the standard samples to be labeled to an expert client, and receiving the labeling results sent by the expert client; integrating and storing the labeling results and the labeled sample set to obtain the labeled sample pool.

6. The software system performance prediction method based on active learning according to claim 1, characterized in that: If the reference data is consistent with the corresponding annotation result in the annotated sample pool, after removing the sample corresponding to the reference data from the annotated sample pool, the method further includes the following steps: Filtering out samples whose reference data are consistent with the corresponding annotation results from the annotated sample pool to generate a discarded sample set; The excluded sample set is stored in an unlabeled data pool, and the samples in the unlabeled data pool are input into the trained performance prediction benchmark model to generate prediction labels, and the prediction labels are used as pseudo-labeling results to form pseudo-labeling samples; according to the confidence of the pseudo-labeling samples, the confidence weight of the pseudo-labeling results is dynamically adjusted to generate a high-confidence pseudo-labeling sample set; An adaptive sample weight allocation mechanism is introduced to merge the samples in the high-confidence pseudo-annotated sample set with the samples in the difficult-example annotated sample pool to construct a second training set; Inputting the second training set into a semi-supervised learning model, and iteratively training the pseudo-labeled samples through multiple rounds of self-correction mechanisms to obtain a second performance prediction model; After each round of iterative training, using the test set to perform performance evaluation on the second performance prediction model to obtain a second model evaluation index; If the second model evaluation index exceeds the model evaluation index of the same training round, the second performance prediction model is used to replace the performance prediction benchmark model; if the second model evaluation index does not exceed the model evaluation index of the same training round, the performance prediction benchmark model is maintained.

7. The software system performance prediction method based on active learning according to claim 1, characterized in that: Selecting samples from the difficult example labeled sample pool to obtain the training set and the test set includes: classifying the samples in the difficult example labeled sample pool by scene to obtain multiple scene classification subsets; according to the size of the scene classification subsets and the sample distribution, using stratified sampling to extract samples from each of the scene classification subsets at a preset ratio to obtain the training set and the test set.

8. The software system performance prediction method based on active learning according to claim 1, characterized in that: The base learner is one of the following models: a logistic regression model, a gradient boosting decision tree model, and an extreme gradient boosting model.

9. The software system performance prediction method based on active learning according to claim 1, characterized in that: Constructing the user simulation data includes: The advertising data, social media popularity data, reservation data, historical player data and player data of similar games of the online multiplayer game software are obtained in real time, and user simulation data within a specified time period is constructed in combination with the online time of the online multiplayer game software; the user simulation data is used to predict user feedback data, system performance data, user operation data and server performance data at various time points after the online multiplayer game software is launched.

10. A software system performance prediction method based on active learning according to claim 9, characterized in that: Constructing the user simulation data within the specified time period includes: Dynamically predict the user attention and traffic trend of the online multiplayer game software in the specified time period by acquiring the advertising data, the social media popularity data and the reservation data of the online multiplayer game software and combining the machine learning model; generate traffic prediction data according to the user attention and the traffic trend; A multi-agent system is introduced, in which each agent in the multi-agent system represents a different type of player; each agent combines reinforcement learning and behavior modeling technology to adaptively simulate the behavior patterns, online time and operation frequency of the different types of players, and simulates the operation behaviors of the different types of players based on the historical player data, the data of players of the same game and the traffic prediction data; Construct time series data and a time series data set by using the historical player data and the player data of the same game, wherein each data point in the time series data set represents the change of player behavior or system performance in different time periods; train a time series prediction model based on the time series data set, wherein the time series prediction model is used to simulate the traffic fluctuation of the user group after the online multiplayer game software is launched; the intelligent agent adaptively adjusts its behavior according to environmental changes; and dynamically predicts the distribution of the number of users and the complexity of player operations during peak periods through the time series data; Combined with the simulation optimization mechanism of reinforcement learning, the intelligent agent maximizes game participation and operation frequency through continuous interaction with the virtual environment, and generates user behavior data within the specified time period; the simulation optimization mechanism simulates high-load scenarios in different time periods by evaluating the impact of internal and external game environmental factors on user behavior; The user simulation data is generated by the multi-agent system and the time series prediction model, including the user feedback data, the system performance data, the user operation data and the server performance data at each time point within a specified time period.

Images