Method for Processing Thread Tasks, Storage Medium, Electronic Device and Program Product
By dynamically adjusting the thread pool parameters and based on the prediction model of client hardware resource data and thread task categories, the resource utilization imbalance caused by the use of static parameters of the thread pool is solved, and more efficient resource utilization and flexible adaptability are achieved.
Patent Information
- Application Number
- CN202510388874.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-06-27
- Estimated Expiration
- 2045-03-31
AI Technical Summary
Related technologies use thread pools configured with static parameters to process thread tasks, resulting in unbalanced resource utilization.
By obtaining the client's hardware resource data, identifying the target category of the pending thread task, determining the target parameter prediction model from multiple parameter prediction models based on the target category, and inputting the hardware resource data into the model, predicting the target parameters required by the thread pool when processing the pending thread task, adjusting the thread pool parameters and processing.
The system's resource utilization rate is optimized, resource overload or waste is avoided, and it has strong adaptability and flexibility, and can dynamically adjust thread pool parameters according to different hardware configurations, business scenarios and task characteristics.
Smart Images

Figure CN119883664B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technologies, and in particular, to a method for processing thread tasks, a storage medium, an electronic device, and a program product. Background Art
[0002] A thread pool is a technology for managing and reusing a group of worker threads, which can reduce the overhead caused by frequent creation and destruction of threads and improve the response speed of an application. By using a thread pool, a client can effectively manage the execution of concurrent tasks, optimize resource utilization, and also help the client control the maximum number of concurrent threads of the application to avoid resource exhaustion.
[0003] Currently, in the process of processing thread tasks through a thread pool in related technologies, the parameters of the thread pool used are static. However, using such a thread pool to process thread tasks will result in an unbalanced resource utilization rate during the task processing. Summary of the Invention
[0004] The present disclosure provides a method for processing thread tasks, a storage medium, an electronic device, and a program product. Its main purpose is to solve the problem that using a thread pool configured with static parameters in related technologies to process thread tasks will result in an unbalanced resource utilization rate during the task processing.
[0005] In a first aspect, the present application provides a method for processing thread tasks, including:
[0006] In response to a user triggering a thread task to be processed, obtain the current hardware resource data of the client, where the hardware resource data includes the hardware configuration data and resource usage data of the client;
[0007] Identify the target category corresponding to the thread task to be processed;
[0008] Based on the target category, determine the target parameter prediction model corresponding to the thread task to be processed from multiple parameter prediction models, and input the hardware resource data into the target parameter prediction model, and predict the target parameters that need to be configured for the thread pool in the client when processing the thread task to be processed through the target parameter prediction model;
[0009] Adjust the parameters of the thread pool to the target parameters, and process the thread task to be processed based on the adjusted thread pool.
[0010] In a second aspect, the present application provides a device for processing thread tasks, including:
[0011] An acquisition module, configured to acquire the current hardware resource data of the client in response to a user triggering a to-be-processed thread task, where the hardware resource data includes the hardware configuration data and resource usage data of the client;
[0012] An identification module, configured to identify the target category corresponding to the to-be-processed thread task;
[0013] A prediction module, configured to determine the target parameter prediction model corresponding to the to-be-processed thread task from multiple parameter prediction models based on the target category, input the hardware resource data into the target parameter prediction model, and predict the target parameters that need to be configured for the thread pool in the client before processing the to-be-processed thread task through the target parameter prediction model;
[0014] A processing module, configured to adjust the parameters of the thread pool to the target parameters and process the to-be-processed thread task based on the adjusted thread pool.
[0015] In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method of the first aspect is implemented.
[0016] In a fourth aspect, the present application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, and when the processor executes the computer program, the method of the first aspect is implemented.
[0017] In a fifth aspect, the present application provides a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, the method of the first aspect is implemented.
[0018] The present disclosure provides a method for processing thread tasks, a storage medium, an electronic device, and a program product. The method includes: in response to a user triggering a thread task to be processed, obtaining the current hardware resource data of the client, where the hardware resource data includes the hardware configuration data and resource usage data of the client; identifying the target category corresponding to the thread task to be processed; determining the target parameter prediction model corresponding to the thread task to be processed from multiple parameter prediction models based on the target category, and inputting the hardware resource data into the target parameter prediction model, and predicting, through the target parameter prediction model, the target parameters that need to be configured for the thread pool in the client when processing the thread task to be processed; adjusting the parameters of the thread pool to the target parameters, and processing the thread task to be processed based on the adjusted thread pool. Compared with the related art, the present application can predict the parameters of the thread pool using the target parameter prediction model based on the target category corresponding to the thread task to be processed and the hardware resource data of the client, comprehensively considering the hardware configuration of the environment in which the client software is deployed and the resource utilization situation before task execution, and then recommending the optimal thread pool parameters, that is, the predicted target parameters. Processing the thread task to be processed using the thread pool configured with the target parameters can optimize the resource utilization rate of the system and avoid resource overload or waste. In addition, the present application can predict different target parameters for different thread tasks to be processed, that is, the present application using dynamically adjusted thread pool parameters can make the present application have strong adaptability and flexibility.
[0019] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.
[0021] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0022] Figure 1 It shows a schematic flowchart of a method for processing a thread task provided by an embodiment of the present application;
[0023] Figure 2 It shows a schematic flowchart of another method for processing a thread task provided by an embodiment of the present application;
[0024] Figure 3Shows a schematic diagram of an example provided by an embodiment of the present application;
[0025] Figure 4 Shows a schematic diagram of an example provided by an embodiment of the present application;
[0026] Figure 5 Shows a schematic diagram of an example provided by an embodiment of the present application;
[0027] Figure 6 Shows a schematic diagram of an example provided by an embodiment of the present application;
[0028] Figure 7 Shows a schematic diagram of an example provided by an embodiment of the present application;
[0029] Figure 8 Shows a schematic diagram of the structure of a processing device for a thread task provided by an embodiment of the present application. Detailed implementation manners
[0030] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.
[0031] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.
[0032] In today's software development field, multi-threaded programming has become a key technology to improve program performance and responsiveness. As an important concept in multi-threaded processing, the thread pool is playing an indispensable role. In traditional single-threaded programs, the code is executed sequentially. However, with the development of computer hardware and the increasing complexity of software functions, the single-threaded processing method often fails to meet our requirements for program performance, and multi-threaded programming emerges as the times require. There are risks out of control and high resource overheads in multi-threaded programming. Each time a thread is created and destroyed, it consumes certain system resources, including memory allocation, initialization, and other operations. If threads are created and destroyed frequently, these overheads will become very significant, resulting in a decrease in the overall efficiency of the program. In addition, creating threads without limit may lead to the exhaustion of system resources because each thread needs to occupy a certain amount of memory and other system resources. Moreover, too many threads may also increase the complexity of thread scheduling, which in turn affects the performance and stability of the program. The thread pool is a mechanism designed to solve the above multi-threaded programming problems. The thread pool is a thread management technology implemented using the pooling technology concept, mainly to reuse threads, conveniently manage threads and tasks, and decouple the creation of threads from the execution of tasks. We can create a thread pool to reuse the created threads to reduce the resource consumption caused by frequent creation and destruction of threads. However, the operating mechanism of the thread pool is not easy to understand, and it is highly correlated with the task type. There is no mature experience in the industry to effectively set thread pool parameters.
[0033] Currently, due to the limitations of the parameter setting and effective method of thread pool parameter changes, a dynamic thread pool framework (DynamicTp) has emerged. It allows applications to dynamically adjust thread pool parameters through the configuration center during runtime, which can take effect in real time without restarting the application, and can implement monitoring and alarm functions. By creating a thread pool through configuration or code, the thread pool status can be monitored and alarms can be triggered, which is adapted to a variety of office platforms. It can dynamically adjust thread pool parameters through the configuration center, supporting multiple alarm dimensions, such as configuration change notifications, activity alarms, capacity threshold alarms, etc., to meet the needs of complex business scenarios.
[0034] To address the technical problem that using a thread pool configured with static parameters in related technologies for thread task processing may lead to uneven resource utilization during the task processing. This embodiment provides a method for processing thread tasks, as Figure 1 shown, the method includes the following steps:
[0035] Step 101, in response to a user triggering a thread task to be processed, obtain the current hardware resource data of the client.
[0036] Among them, the hardware resource data includes the hardware configuration data and resource usage data of the client.
[0037] In the embodiments of the present application, a user-triggered thread task can be started by a user's operation (such as clicking a button, submitting a form, etc.) to start one or more background threads to execute specific tasks, for example, file upload, data processing, network request, etc.
[0038] In some examples, the hardware configuration data includes, but is not limited to: Central Processing Unit (CPU) model, number of cores, memory size, hard disk capacity and type, graphics card information, etc.; correspondingly, the resource usage data generally refers to CPU utilization, memory usage, disk I / O, network traffic, etc. The resource usage data can help understand the real-time load status of the client.
[0039] Step 102: Identify the target category corresponding to the thread task to be processed.
[0040] For this embodiment, the target category corresponding to the thread task to be processed can be identified through a clustering model. The clustering model is an unsupervised learning method used to group the objects in a dataset into several clusters, so that the objects within the same cluster are as similar as possible to each other, while the objects between different clusters are as different as possible.
[0041] In the embodiments of the present application, the clustering model calculates the proportion of the use of hardware resources such as CPU and memory by different individual tasks based on historical key data, and performs clustering processing on the task types according to these proportions, and determines the clustering model of a certain type of task based on the average proportion. Through the clustering model, the target category corresponding to the thread task to be processed can be determined.
[0042] Step 103: Determine the target parameter prediction model corresponding to the thread task to be processed from multiple parameter prediction models based on the target category, and input the hardware resource data into the target parameter prediction model, and predict the target parameters that need to be configured in the thread pool in the client when processing the thread task to be processed through the target parameter prediction model.
[0043] In some examples, each predetermined category corresponds to a parameter prediction model. The multiple parameter prediction models in the embodiments of the present application include the parameter prediction models corresponding to each predetermined category, that is, after determining the target category corresponding to the thread task to be processed in the embodiments of the present application, it is necessary to determine the target parameter prediction model corresponding to the target category. Through the target parameter prediction model, the target parameters that need to be configured in the thread pool in the client when processing the thread task to be processed can be predicted more accurately.
[0044] For this embodiment, a thread pool is a design pattern that executes tasks by pre-creating a set of reusable threads, thereby avoiding the overhead caused by frequent creation and destruction of threads. This pattern can significantly improve the performance of an application, especially in cases where a large number of short-lived tasks need to be processed.
[0045] Optionally, the parameters of the thread pool may include, but are not limited to: core pool size, core pool size, keep alive time, work queue, rejected execution handler, and so on.
[0046] Step 104: Adjust the parameters of the thread pool to target parameters, and process the thread tasks to be processed based on the adjusted thread pool.
[0047] In the embodiment of the present application, adjusting the parameters of the thread pool to target parameters and processing the thread tasks to be processed based on the adjusted thread pool can enable the embodiment of the present application to comprehensively consider the hardware configuration of the software deployment environment and the resource utilization rate before task execution to recommend the best thread pool parameters, optimizing the resource utilization rate of the system and avoiding resource overload or waste; it can also enable the embodiment of the present application to predict the best thread pool parameters according to different hardware configurations, business scenarios, and task characteristics, and dynamically adjust the thread pool parameters, with strong adaptability and flexibility.
[0048] Compared with the related technology, this embodiment can predict the parameters of the thread pool using the target parameter prediction model based on the target category corresponding to the thread tasks to be processed and the hardware resource data of the client, comprehensively considering the hardware configuration of the software deployment environment of the client and the resource utilization situation before task execution, and then recommend the best thread pool parameters, that is, the predicted target parameters. Processing the thread tasks to be processed using the thread pool configured with the target parameters can optimize the resource utilization rate of the system and avoid resource overload or waste. In addition, the present application can predict different target parameters for different thread tasks to be processed, that is, the present application using dynamically adjusted thread pool parameters can make the present application have strong adaptability and flexibility.
[0049] Further, as a refinement and extension of the above embodiment, in order to specifically illustrate the training process of multiple parameter prediction models, the following methods can be adopted, but are not limited to, such as Figure 2 As shown, the method includes:
[0050] Step 201: Obtain historical key data stored in a predetermined storage location.
[0051] Among them, historical key data is the key data of multiple clients obtained through a distributed system architecture during the process of processing historical thread tasks. The historical key data includes task execution data, hardware configuration data, and resource usage data.
[0052] In the embodiments of the present application, as Figure 3 shown, distributed software systems are widely deployed in multiple customer environments. A large amount of data related to task execution will be generated during the operation of these systems, such as task duration, task attributes, resource utilization rates (CPU, memory, network, Graphics Processing Unit (GPU), disk utilization) at a certain sampling interval during task operation, hardware configuration (CPU, GPU, memory, hard disk, network bandwidth), thread pool parameters (core thread number, maximum thread number, blocking queue), etc. Through the cloud-edge collaboration architecture, this data scattered in each customer environment is reported to the cloud. The software system collects key data during task operation in real time through a lightweight data collection module. After local preprocessing of these data, they are uploaded to the centralized storage system in the cloud through a secure communication protocol. The cloud uses a distributed storage architecture (such as Alibaba Cloud OSS) to store the reported data with high availability and security. At the same time, a data processing framework (such as Apache Spark) is used to clean, standardize, and extract features from the massive data to prepare for subsequent model training. Big data analysis technology is used to deeply mine and preprocess the massive reported data to extract key features and provide high-quality data support for model training.
[0053] It should be noted that, as Figure 3 shown, the cloud-edge collaboration-driven multi-modal data collection architecture (i.e., the distributed system architecture in the embodiments of the present application) constructs a hierarchical data collection system. Based on the local preprocessing and cloud collaboration mechanism of edge nodes, lightweight data collection agents are deployed on the edge side to sample heterogeneous data such as task duration, resource utilization rates (CPU / GPU / memory, etc.), hardware configuration, and thread pool parameters; on the cloud side (server side), the data uploaded by multiple clients is processed, stored, and model training is performed. It can support the secure fusion of cross-terminal data and joint optimization of models, improving the model generalization ability; reducing the cloud load through edge computing and reducing the data upload bandwidth consumption.
[0054] Specifically, as Figure 4As shown in the figure, the client can include a task management module, a hardware monitoring module, and a resource usage monitoring module. Among them, the task management module designs a thread pool based on a dynamic thread pool framework, is responsible for concurrently executing tasks issued for different functions, and records information such as the type, task volume, task start time, task creation time, task end time, task status, and thread pool parameters of each executed task, that is, the task execution data in the embodiments of the present application can be obtained. The hardware monitoring module is responsible for acquiring and recording the hardware metrics of the deployed environment, including the number of CPU cores, memory capacity, storage device information, graphics card, and network bandwidth, etc., that is, the hardware configuration data in the embodiments of the present application can be obtained; the resource usage monitoring module is responsible for periodically collecting the resource usage conditions at certain time intervals before, during, and after the task runs, including CPU utilization, memory utilization, disk utilization, etc., that is, the resource usage data in the embodiments of the present application can be obtained.
[0055] As an optional method, as Figure 4 shown in the figure, the embodiments of the present application may further include a data collection module, which is responsible for integrating the key data of the task management module, the hardware monitoring module, and the resource usage monitoring module. After these data are locally preprocessed, they are periodically uploaded to a centralized storage system in the cloud through a secure communication protocol.
[0056] Step 202: Preprocess the historical key data to obtain sample data for training multiple parameter prediction models.
[0057] Optionally, step 202 may specifically include: determining the time series data corresponding to each piece of historical key data in the historical key data; inputting the time series data into a preset detection model, determining the reconstruction error of the time series data in the preset detection model, and in the case where it is determined that the reconstruction error is greater than a predetermined reconstruction error threshold, determining the corresponding time series data as abnormal time series data; removing the abnormal historical key data corresponding to the abnormal time series data in the historical key data, and determining the remaining historical key data as sample data.
[0058] Among them, the array in the time series data includes the time point and task metric value corresponding to each piece of historical key data.
[0059] In the embodiments of the present application, each piece of sample data has a unique ID and a timestamp, and then each part is used as a sub-object. Specifically, the time series data uses an array structure to record the time point and the corresponding metric value.
[0060] In some examples, preprocessing the historical key data may include removing outliers (such as invalid data with CPU usage exceeding 100%) and filling in missing values (such as linear interpolation or data filling based on data of similar tasks).
[0061] Specifically, asFigure 5 As shown, preprocessing historical key data can use a Long Short-Term Memory Autoencoder (LSTM-Autoencoder) to detect abnormal resource usage patterns. This detection is a deep learning-based anomaly detection method that combines the advantages of a Long Short-Term Memory network (LSTM) and an Autoencoder to identify abnormal patterns in time series data. The LSTM-Autoencoder is trained using resource usage data in the normal state to enable it to learn the time series characteristics of the normal mode. After the model training is completed, new resource usage data is input into the model. If the reconstruction error of the model is significantly higher than that of the normal data, then this data point is considered abnormal. By detecting the time series data of resource metrics in the sample data, invalid samples are eliminated, and samples with an error task status are also eliminated, obtaining the sample data in the embodiments of the present application. Using the obtained sample data for subsequent model training can improve the accuracy of model training.
[0062] It should be noted that, as Figure 5 shown, the abnormal resource pattern detection and sample purification mechanism based on LSTM-Autoencoder detects anomalies in the time series data of resource utilization rates during task execution by designing a bidirectional LSTM-Autoencoder time series reconstruction model. If the reconstruction error of the model is significantly higher than that of the normal data, then this data point is considered abnormal. By detecting the time series data of resource metrics in the sample data, invalid samples are eliminated. This enables the embodiments of the present application to be improved in anomaly detection through joint spatio-temporal feature modeling (LSTM captures long time series dependencies, and Autoencoder learns latent patterns).
[0063] Step 203: Extract features from the sample data to obtain the feature information corresponding to the sample data.
[0064] Optionally, step 203 may specifically include: determining at least one feature extraction type corresponding to the sample data; and extracting task execution features, hardware configuration features, and resource usage features from the sample data according to the at least one feature extraction type.
[0065] In the embodiments of the present application, at least one feature extraction type may include numerical type, continuous type, time series type, etc. Specifically, the process of feature extraction may include: directly retaining and normalizing (such as Z-Score) the numerical features (CPU core count, memory capacity, etc.) in the hardware configuration data; segmentally encoding the network bandwidth and hard disk type (SSD / HDD); performing one-hot encoding on the task type; extracting statistical features (mean, standard deviation, maximum value, minimum value, quantile, rising / falling trend (linear fitting slope) and time series features (sliding window trend)) for the resource utilization rate; calculating the task duration time (end time - creation time) and performing logarithmic transformation on the task duration (to alleviate the long-tailed distribution); directly retaining the numerical features of the execution quantity and performing standardization; standardizing or normalizing the numerical parameters of the thread pool (core thread count, queue length, etc.) and performing categorical encoding (Label Encoding) on the queue type.
[0066] Step 204: Analyze the resource usage of multiple clients based on the feature information, and perform clustering processing on the sample data through a preset clustering model based on the resource usage.
[0067] Optionally, step 204 may specifically include: analyzing based on the task execution features, hardware configuration features, and resource usage features to obtain the processor usage data, memory usage data, and hard disk usage data of multiple clients; respectively performing normalization processing on the processor usage data, memory usage data, and hard disk usage data to obtain the processor normalization weight corresponding to the processor usage data, the memory normalization weight corresponding to the memory usage data, and the hard disk normalization weight corresponding to the hard disk usage data; performing clustering processing on the sample data in the preset clustering model according to the predetermined categories based on the processor normalization weight, the memory normalization weight, and the hard disk normalization weight.
[0068] In the embodiments of the present application, it is necessary to calculate the usage amounts of specific types of tasks for hardware resources such as CPU, memory, and hard disk, namely the processor usage data, memory usage data, and hard disk usage data in the embodiments of the present application, based on the task execution features, hardware configuration features, and resource usage features; calculate the normalization weights of each hardware resource used during task operation (weight = the integral area of the resource / the total integral area). Then, cluster different types of tasks according to the weight values of the hardware resources (such as the improved K-means++ algorithm), and divide the task types as a whole into several large categories (such as "CPU-intensive (category A)", "memory-hard disk balanced type (category B)", "high CPU-low memory burst type (category C)", or "stable GPU-network sensitive type (category D)"), which are the predetermined categories in the embodiments of the present application.
[0069] It should be noted that the task classification and clustering algorithm driven by multi-dimensional resource weights designs a heterogeneous resource weight adaptive allocation mechanism, calculates the normalized weights of each task type for resources such as CPU, memory, and hard disk (weight = integral area of the resource / total integral area), and constructs a multi-objective weighted clustering model (such as the improved K-means++ algorithm) based on this, dividing tasks into categories with highly cohesive hardware usage patterns (such as "CPU-intensive (category A)", "memory-hard disk balanced (category B)", "high CPU-low memory burst (category C)", or "stable GPU-network sensitive (category D)"). This enables the embodiments of the present application to achieve a strong association between task classification and underlying hardware characteristics through multi-resource coupling weight analysis. Compared with traditional single-index classification methods, it can improve the similarity of task resource patterns within categories and achieve improved classification accuracy; support dynamic expansion of resource dimensions (such as adding GPU metrics), and the classification model does not need to be reconstructed, only the weight calculation logic needs to be adjusted.
[0070] Optionally, when analyzing based on task execution characteristics, hardware configuration characteristics, and resource usage characteristics to obtain processor usage data, memory usage data, and hard disk usage data of multiple clients, it may specifically include: determining, based on task execution characteristics, hardware configuration characteristics, and resource usage characteristics, the first normal distribution curve corresponding to the processor used during the execution of thread tasks, the second normal distribution curve corresponding to the memory used, and the third normal distribution curve corresponding to the hard disk used by multiple clients; obtaining processor usage data based on the first normal distribution curve, obtaining memory usage data based on the second normal distribution curve, and obtaining hard disk usage data based on the third normal distribution curve.
[0071] For this embodiment, the process of analyzing based on task execution characteristics, hardware configuration characteristics, and resource usage characteristics to obtain processor usage data, memory usage data, and hard disk usage data respectively may include: selecting, for each task type involved, the task data with a task volume of 1 for execution, and combining the numerical points of the resource usage rate recorded at a certain time interval during the execution of a single task type ( ), and representing that the usage of each resource by a single task conforms to a normal distribution through the following formula (1), as shown in Figure 6 . Fitting the normal distribution curve of the corresponding resource usage of the current task type with the recorded data, and calculating the integral area within 3 standard deviations of the mean as the usage amount of the corresponding resource, that is, the usage amounts corresponding to the processor usage data, memory usage data, and hard disk usage data in the embodiments of the present application respectively. The specific formula (1) is as follows:
[0072] (Formula 1)
[0073] In Formula 1, represents the mean value of the resource usage (calculated by Formula 2 shown below), represents the standard deviation of the resource usage (calculated by Formula 3 shown below),
[0074] The formula for calculating the mean value of the resource usage is shown as Formula 2 below:
[0075] (Formula 2)
[0076] The formula for calculating the standard deviation of the resource usage is shown as Formula 3 below:
[0077] (Formula 3)
[0078] The formula for calculating the resource usage is shown as Formula 4 below:
[0079] (Formula 4)
[0080] It should be noted that, as Figure 6 shown in the fine-grained resource usage quantization method based on normal distribution fitting, by fitting the time series data of resource utilization rates (such as CPU, memory, hard disk, etc.) periodically collected during the execution of a single task, constructing a normal distribution curve, and calculating the integral area within the mean ± 3σ interval (covering 99.7% probability density) as the quantization index of resource consumption. This enables the embodiments of the present application to break through the traditional resource evaluation mode based on the mean or peak value, accurately depict the dynamic occupancy characteristics of tasks for hardware resources through integral quantization; combined with the 3σ interval truncation, effectively exclude occasional noise interference (such as instantaneous CPU spikes), and can reduce the resource evaluation error.
[0081] Step 205: Perform model training based on the clustering result, resource usage situation, and preset resource usage threshold to obtain multiple parameter prediction models composed of parameter prediction models corresponding to each category in the predetermined category.
[0082] Optionally, step 205 may specifically include: performing model training on the sample data of each category according to the processor resource usage threshold, memory resource usage threshold, and hard disk resource usage threshold of each category in the predetermined category to obtain multiple parameter prediction models composed of parameter prediction models corresponding to each category.
[0083] In some examples, the sample data can be classified according to the classification results in the task classification module. The random forest algorithm can be used to determine the order of nodes by combining the usage weights of each resource for each type of task in the task classification results, and then train the model. Reinforcement learning can be used to train the model in combination with resource usage thresholds. Specifically, the threshold settings are as follows: The CPU utilization rate of a healthy system should be maintained between 30% and 70%. When the CPU utilization rate continuously exceeds 70%, performance problems may start to occur. The memory usage rate should generally be maintained between 50% and 80%. The memory usage rate within this range indicates that the system has enough space to handle the current tasks and also reserves some resources to cope with sudden high-load situations. The disk utilization rate should be maintained within a reasonable range, usually the disk usage rate does not exceed 80%. The LightGBM multi-output regression algorithm can also be used to train the optimal thread parameter prediction models for the total task time of the corresponding task categories to be short and the resource usage to be fully and safely utilized.
[0084] Furthermore, the method of this embodiment further includes: identifying a target category through a preset clustering model. If it is determined that the target category does not belong to the predetermined category, obtain the predetermined parameters corresponding to the thread pool; adjust the parameters of the thread pool to the predetermined parameters, and process the to-be-processed thread tasks based on the adjusted thread pool.
[0085] In the embodiment of the present application, the predetermined parameters can be the default parameters pre-configured for the thread pool, or the parameters used by the thread pool in the previous task processing. The specific values of the predetermined parameters are not limited in the embodiment of the present application.
[0086] Optionally, before executing step 103, the method of this embodiment further includes: based on the hardware resource data, determining whether there is a resource overload situation on the client; correspondingly, step 103 specifically further includes: if it is determined that there is no resource overload situation on the client, input the hardware resource data into the target parameter prediction model, and predict the target parameters through the target parameter prediction model.
[0087] Optionally, the method of this embodiment further includes: if it is determined that there is a resource overload situation on the client, adjust the parameters of the thread pool to the predetermined parameters, and process the to-be-processed thread tasks based on the adjusted thread pool.
[0088] In some examples, determining whether there is a resource overload on the client may specifically include the determination of the following parameters but is not limited to these: 1. CPU usage rate; Normal situation: The CPU usage rate should be maintained within a reasonable range. For most applications, exceeding 80% for a long time may indicate a problem. Overload symptom: If the CPU usage rate continuously approaches or reaches 100%, it indicates that the system may be experiencing high load, resulting in increased response time and even freezing. 2. Memory usage; Normal situation: The physical memory of the system should be sufficient to support all currently running applications, and the usage of the swap file is low. Overload symptom: If the memory usage rate is too high or the swap space is frequently used, this may be a manifestation of insufficient memory, resulting in performance degradation. 3. Disk I / O; Normal situation: Disk read and write operations should be completed within an acceptable latency. Overload symptom: Long high disk I / O waiting times, frequent disk errors, or disk fullness, etc., may all be signs of resource tension, and so on.
[0089] Optionally, before performing step 104, the method of this embodiment further includes: performing a security check on the target parameter; correspondingly, step 104 may specifically include: when it is determined that the target parameter meets the predetermined security check condition, adjusting the configuration parameters of the predetermined thread pool framework to the target parameter through the application programming interface of the predetermined thread pool framework in the thread pool, and processing the to-be-processed thread task based on the adjusted thread pool.
[0090] In some examples, as Figure 4 shown, the client in the embodiment of the present application may further include a task management module. Specifically, the task management module integrates a dynamic thread pool framework (DynamicTp). By adding dependencies and then configuring the basic parameters of DynamicTp (including the basic parameters of the thread pool, such as the core thread number, maximum thread number, queue type, and capacity, etc.) in the configuration file, defining threads through annotations or programming methods in the code. When the optimal thread pool parameters are predicted (such as corePoolSize = 8, maxPoolSize = 20, queueCapacity = 200), the parameters can be actively updated through the application programming interface (API) provided by DynamicTp. This adjustment mechanism can achieve dynamic adjustment of thread pool parameters in a non-invasive manner.
[0091] It should be noted that the real-time regulation system for thread pool parameters based on dynamic feedback integrates the DynamicTp enhanced dynamic thread pool framework to implement the following closed-loop control mechanism, including: the dynamic perception layer monitors the thread pool status (queue accumulation rate, task rejection rate) in real time through lightweight probes; the hot update mechanism realizes the adjustment of thread pool parameters without service interruption through bytecode injection technology; this enables the embodiments of the present application to support dynamic parameter adaptation in high-concurrency scenarios and improves the system throughput.
[0092] Optionally, the method of this embodiment further includes: when it is determined that the target parameter does not meet the predetermined security verification condition, adjusting the configuration parameters of the predetermined thread pool framework to predetermined parameters, and processing the to-be-processed thread tasks based on the adjusted thread pool.
[0093] Optionally, the method of this embodiment further includes: collecting key data during the client's processing of the to-be-processed thread tasks; sending the key data to the server at a predetermined sending cycle based on the distributed system architecture.
[0094] Among them, the key data includes task execution data, hardware configuration data, and resource usage data, and the server is used to uniformly store the key data at a predetermined storage location.
[0095] In the embodiments of the present application, the data acquisition module of the client in the embodiments of the present application is responsible for integrating the key data of the task management module, the hardware monitoring module, and the resource usage monitoring module. After local preprocessing, these data are regularly uploaded to the centralized storage system in the cloud through a secure communication protocol.
[0096] Exemplarily, as Figure 7 shown, the client in the embodiments of the present application can adaptively adjust the thread pool parameters according to real-time data, and can dynamically adjust the thread pool parameters, which involves steps such as task triggering, task category, resource evaluation, model prediction, parameter adjustment, and task execution. When a user triggers a certain batch task, after obtaining the corresponding task category through the classification model, according to the corresponding task category combined with the current hardware resources and the current resource utilization rate, the thread pool parameters corresponding to the task category are obtained through the thread pool optimal parameter prediction model, and the thread pool parameters of the dynamic task are adjusted according to the output optimal thread pool parameters, and then the task is executed.
[0097] It should be noted that the setting of the optimal parameters of the thread pool is greatly affected by the type of tasks and the hardware configuration. At present, the related technologies mainly classify tasks into CPU-intensive and IO-intensive types and give a rough calculation formula for the number of core threads. However, this method is not applicable to the actual software scenarios. The classification of tasks cannot be generally considered as either CPU-intensive or IO-intensive. Moreover, the parameters of the thread pool include not only the number of core threads but also the maximum number of threads, the blocking queue, the survival time, etc. There are many influencing factors and the parameter setting is complex. There is no mature experience in the industry to effectively set the parameters of the thread pool. The embodiments of the present application aim to solve the above problems, propose a method to classify the task types more realistically, and determine the optimal thread pool parameters on the basis of task classification to ensure the safe and efficient utilization of the thread pool.
[0098] In some examples, the embodiments of the present application propose a complete set of intelligent thread pool parameter dynamic regulation methods and systems, mainly including six modules: a task management module, a hardware monitoring module, a resource usage monitoring module, a data collection module, a task classification module, and a thread pool optimal parameter prediction module, and describe how these six modules work. The patent covers methods for task classification, data collection, data processing, specific steps of model training, dynamic adjustment mechanisms, and the architecture of the entire system. The embodiments of the present application can improve the accuracy of thread pool parameter prediction. Specifically, the original thread pool parameter prediction method uses the same thread pool parameter prediction model for various task types. However, thread pool parameters are greatly affected by the nature of tasks, which will interfere with model training. The task classification module classifies tasks before training the prediction model, fits the usage rate based on the normal distribution, calculates the usage amount of each task type for each resource as weights respectively, clusters the weights to obtain task categories, and according to the task categories, divides the collected historical task data into different training sets for model training, which can further improve the accuracy of the thread pool optimal parameter prediction model; it can also improve resource utilization optimization. Specifically, the embodiments of the present application comprehensively consider the hardware configuration of the software deployment environment and the resource utilization rate before task execution to recommend the optimal thread pool parameters, and the resource utilization rate of the system is optimized, avoiding resource overload or waste; it can also enhance adaptability. Specifically, the present application can predict the optimal thread pool parameters according to different hardware configurations, business scenarios, and task characteristics, and dynamically adjust the thread pool parameters, with strong adaptability and flexibility; it can also automatically predict the optimal thread pool parameters through a machine learning model to improve system performance; data collection is based on a cloud-edge collaboration architecture. Through the data collection module of the software system deployed on each client terminal, combined with the monitoring module, key data during the task running process is collected in real time. After local preprocessing, these data are uploaded to the centralized storage system in the cloud through a secure communication protocol. For model training, it can not only mine the value of historical data but also improve the generalization ability of the model.
[0099] Compared with the related technologies, this embodiment can predict the parameters of the thread pool based on the target category corresponding to the thread task to be processed and the target parameter prediction model using the hardware resource data of the client, comprehensively considering the hardware configuration of the software deployment environment of the client and the resource utilization situation before task execution, and then recommend the optimal thread pool parameters, that is, the predicted target parameters. Using the thread pool configured with the target parameters to process the thread task to be processed can optimize the resource utilization rate of the system, avoiding resource overload or waste. In addition, for different thread tasks to be processed, this application can predict different target parameters, that is, using the dynamically adjusted thread pool parameters in this application can make this application have strong adaptability and flexibility.
[0100] An embodiment of the present application further provides a processing device for thread tasks, such as Figure 8 shown. The device includes: an acquisition module 31, an identification module 32, a prediction module 33, and a processing module 34.
[0101] The acquisition module 31 is configured to, in response to a user triggering a thread task to be processed, acquire the current hardware resource data of the client, where the hardware resource data includes the hardware configuration data and resource usage data of the client;
[0102] The identification module 32 is configured to identify the target category corresponding to the thread task to be processed;
[0103] The prediction module 33 is configured to determine the target parameter prediction model corresponding to the thread task to be processed from multiple parameter prediction models based on the target category, input the hardware resource data into the target parameter prediction model, and predict the target parameters that need to be configured in the thread pool in the client before processing the thread task to be processed through the target parameter prediction model;
[0104] The processing module 34 is configured to adjust the parameters of the thread pool to the target parameters and process the thread task to be processed based on the adjusted thread pool.
[0105] In some examples of this embodiment, the prediction module 33 is specifically configured to acquire historical key data stored in a predetermined storage location, where the historical key data is key data of multiple clients in the process of processing historical thread tasks obtained through a distributed system architecture, and the historical key data includes task execution data, hardware configuration data, and resource usage data; preprocess the historical key data to obtain sample data for training multiple parameter prediction models; extract features from the sample data to obtain feature information corresponding to the sample data; analyze the resource usage of multiple clients based on the feature information, and perform clustering processing on the sample data through a preset clustering model based on the resource usage; perform model training based on the clustering result, resource usage, and preset resource usage threshold to obtain parameter prediction models corresponding to each category in a predetermined category, forming multiple parameter prediction models.
[0106] In some examples of this embodiment, the prediction module 33 is further specifically configured to determine the time series data corresponding to each piece of historical key data in the historical key data, where the array in the time series data includes the time point corresponding to each piece of historical key data and the task metric value; input the time series data into a preset detection model, determine the reconstruction error of the time series data in the preset detection model, and in the case where it is determined that the reconstruction error is greater than a predetermined reconstruction error threshold, determine the corresponding time series data as abnormal time series data; remove the abnormal historical key data corresponding to the abnormal time series data in the historical key data, and determine the remaining historical key data as sample data.
[0107] In some examples of this embodiment, the prediction module 33 is further specifically configured to determine at least one feature extraction type corresponding to the sample data; according to the at least one feature extraction type, extract task execution features, hardware configuration features, and resource usage features from the sample data.
[0108] In some examples of this embodiment, the prediction module 33 is further specifically configured to analyze based on the task execution features, hardware configuration features, and resource usage features to obtain processor usage data, memory usage data, and hard disk usage data of multiple clients; perform normalization processing on the processor usage data, memory usage data, and hard disk usage data respectively to obtain a processor normalization weight corresponding to the processor usage data, a memory normalization weight corresponding to the memory usage data, and a hard disk normalization weight corresponding to the hard disk usage data; perform clustering processing on the sample data according to a predetermined category in a preset clustering model based on the processor normalization weight, the memory normalization weight, and the hard disk normalization weight.
[0109] In some examples of this embodiment, the prediction module 33 is further specifically configured to determine a first normal distribution curve corresponding to the processor used during the execution thread task, a second normal distribution curve corresponding to the memory used, and a third normal distribution curve corresponding to the hard disk used by multiple clients based on the task execution features, hardware configuration features, and resource usage features; obtain the processor usage data according to the first normal distribution curve, obtain the memory usage data according to the second normal distribution curve, and obtain the hard disk usage data according to the third normal distribution curve.
[0110] In some examples of this embodiment, the prediction module 33 is further specifically configured to perform model training on the sample data of each category according to the processor resource usage threshold, memory resource usage threshold, and hard disk resource usage threshold of each category in the predetermined category to obtain a parameter prediction model corresponding to each category, and form multiple parameter prediction models.
[0111] In some examples of this embodiment, the recognition module 32 is specifically configured to recognize the target category through a preset clustering model; correspondingly, the recognition module 32 is further configured to, if it is determined that the target category does not belong to the predetermined category, obtain the predetermined parameters corresponding to the thread pool; adjust the parameters of the thread pool to the predetermined parameters, and process the to-be-processed thread tasks based on the adjusted thread pool.
[0112] In some examples of this embodiment, the prediction module 33 is further configured to determine whether there is a resource overload situation on the client based on the hardware resource data; correspondingly, the prediction module 33 is specifically further configured to, if it is determined that there is no resource overload situation on the client, input the hardware resource data into the target parameter prediction model, and predict the target parameter through the target parameter prediction model.
[0113] In some examples of this embodiment, the processing module 34 is further configured to, if it is determined that there is a resource overload situation on the client, adjust the parameters of the thread pool to the predetermined parameters, and process the to-be-processed thread tasks based on the adjusted thread pool.
[0114] In some examples of this embodiment, the processing module 34 is further configured to perform a security check on the target parameter; correspondingly, the processing module 34 is specifically further configured to, in the case where it is determined that the target parameter meets the predetermined security check conditions, adjust the configuration parameters of the predetermined thread pool framework to the target parameter through the application programming interface of the predetermined thread pool framework in the thread pool, and process the to-be-processed thread tasks based on the adjusted thread pool; in the case where it is determined that the target parameter does not meet the predetermined security check conditions, adjust the configuration parameters of the predetermined thread pool framework to the predetermined parameters, and process the to-be-processed thread tasks based on the adjusted thread pool.
[0115] In some examples of this embodiment, the processing module 34 is further configured to collect key data during the client's processing of the to-be-processed thread tasks; send the key data to the server according to a predetermined sending period based on the distributed system architecture, and the server is used to uniformly store the key data at a predetermined storage location.
[0116] It should be noted that for other corresponding descriptions of each functional unit involved in the thread task processing device provided in this embodiment, reference can be made to Figure 1 the corresponding description in, which will not be elaborated here.
[0117] Based on the method as shown in Figure 1 above, correspondingly, this embodiment further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method as shown in Figure 1 above is implemented.
[0118] Based on the method as shown in Figure 1The method described above, correspondingly, this embodiment also provides a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, it implements the method as described above Figure 1 shown.
[0119] Based on such an understanding, the technical solution of this application can be embodied in the form of a software product, and this software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of this application.
[0120] Based on the method as described above Figure 1 shown, and Figure 8 the virtual device embodiment shown, in order to achieve the above object, this embodiment of the application also provides an electronic device, such as a personal computer, a server, and this device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the method as described above Figure 1 shown.
[0121] In some embodiments, the above-mentioned physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, sensors, an audio circuit, a WI-FI module, and so on. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc., and optionally the user interface may further include a USB interface, a card reader interface, etc. The network interface may include a standard wired interface, a wireless interface (such as a WI-FI interface), etc. in some embodiments.
[0122] Those skilled in the art can understand that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and it may include more or fewer components, or combine certain components, or have different component arrangements.
[0123] The storage medium may further include an operating system and a network communication module. The operating system is a program for managing the hardware and software resources of the above-mentioned physical device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to implement communication between components inside the storage medium, and communication between other hardware and software in the information processing physical device.
[0124] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or can also be implemented by hardware. By applying the solution of this embodiment, compared with the related art, this embodiment can use the target parameter prediction model to predict the parameters of the thread pool based on the target category corresponding to the thread task to be processed and the hardware resource data of the client, comprehensively considering the hardware configuration of the environment in which the client software is deployed and the resource utilization before task execution, and then recommend the best thread pool parameters, that is, the predicted target parameters. Using the thread pool configured with the target parameters to process the thread task to be processed can optimize the resource utilization rate of the system and avoid the situation of resource overload or waste. In addition, the present application can predict different target parameters for different thread tasks to be processed, that is, the present application using dynamically adjusted thread pool parameters can make the present application have strong adaptability and flexibility.
[0125] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.
[0126] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A method for processing thread tasks, characterized in that: include: In response to a user triggering a pending thread task, obtaining current hardware resource data of the client, the hardware resource data including hardware configuration data and resource usage data of the client; Identify the target category corresponding to the pending thread task; Determine a target parameter prediction model corresponding to the pending thread task from a plurality of parameter prediction models based on the target category, input the hardware resource data into the target parameter prediction model, and predict the target parameters that the thread pool in the client needs to configure when processing the pending thread task through the target parameter prediction model; Adjusting the parameters of the thread pool to the target parameters, and processing the to-be-processed thread tasks based on the adjusted thread pool; The training process of the multiple parameter prediction models includes: Acquire historical key data stored in a predetermined storage location, wherein the historical key data is key data obtained by a plurality of clients through a distributed system architecture in a process of processing historical thread tasks, and the historical key data includes task execution data, hardware configuration data, and resource usage data; Preprocessing the historical key data to obtain sample data for training the multiple parameter prediction models; Extracting features from the sample data to obtain feature information corresponding to the sample data; Analyze the resource usage of the multiple clients based on the feature information, and perform clustering processing on the sample data through a preset clustering model based on the resource usage; wherein the clustering processing is clustering different types of tasks according to the weight value of the hardware resources, and the weight value is the normalized weight used by each hardware resource when the task is running; Model training is performed based on the clustering results, the resource usage and the preset resource usage threshold, and the parameter prediction model corresponding to each category in the predetermined category is obtained to form the multiple parameter prediction models.
2. The method according to claim 1, characterized in that The preprocessing of the historical key data to obtain sample data for training the multiple parameter prediction models includes: Determine the time series data corresponding to each piece of the historical key data, wherein the array in the time series data includes the time point and the task indicator value corresponding to each piece of the historical key data; Inputting the time series data into a preset detection model, determining a reconstruction error of the time series data in the preset detection model, and determining the corresponding time series data as abnormal time series data when it is determined that the reconstruction error is greater than a predetermined reconstruction error threshold; The abnormal historical key data corresponding to the abnormal time series data in the historical key data are removed, and the remaining historical key data are determined as the sample data.
3. The method according to claim 1, characterized in that The extracting features from the sample data to obtain feature information corresponding to the sample data includes: Determining at least one feature extraction type corresponding to the sample data; According to the at least one feature extraction type, task execution features, hardware configuration features and resource usage features are extracted from the sample data.
4. The method according to claim 3, characterized in that The analyzing the resource usage of the multiple clients based on the feature information, and clustering the sample data using a preset clustering model based on the resource usage, includes: Analyze based on the task execution characteristics, the hardware configuration characteristics and the resource usage characteristics to obtain processor usage data, memory usage data and hard disk usage data of the multiple clients; Normalizing the processor usage data, the memory usage data, and the hard disk usage data respectively to obtain a processor normalization weight corresponding to the processor usage data, a memory normalization weight corresponding to the memory usage data, and a hard disk normalization weight corresponding to the hard disk usage data; The sample data is clustered according to the predetermined category in the preset clustering model based on the processor normalization weight, the memory normalization weight and the hard disk normalization weight.
5. The method according to claim 4, characterized in that The analyzing based on the task execution characteristics, the hardware configuration characteristics and the resource usage characteristics to obtain the processor usage data, the memory usage data and the hard disk usage data of the multiple clients includes: Based on the task execution characteristics, the hardware configuration characteristics and the resource usage characteristics, determine a first normal distribution curve corresponding to the processor usage, a second normal distribution curve corresponding to the memory usage and a third normal distribution curve corresponding to the hard disk usage of the multiple clients in the process of executing the thread tasks; The processor usage data is obtained according to the first normal distribution curve, the memory usage data is obtained according to the second normal distribution curve, and the hard disk usage data is obtained according to the third normal distribution curve.
6. The method according to claim 4, characterized in that The model training is performed according to the clustering result, the resource usage and the preset resource usage threshold to obtain the multiple parameter prediction models, including: Model training is performed on sample data of each category according to the processor resource usage threshold, memory resource usage threshold and hard disk resource usage threshold of each category in the predetermined category, and a parameter prediction model corresponding to each category is obtained to form the multiple parameter prediction models.
7. The method according to claim 1, characterized in that The identifying the target category corresponding to the to-be-processed thread task includes: Identifying the target category by using a preset clustering model; After identifying the target category corresponding to the to-be-processed thread task, the method further includes: If it is determined that the target category does not belong to the predetermined category, obtaining predetermined parameters corresponding to the thread pool; The parameters of the thread pool are adjusted to predetermined parameters, and the to-be-processed thread tasks are processed based on the adjusted thread pool.
8. The method according to claim 1, characterized in that: Before inputting the hardware resource data into the target parameter prediction model and predicting, by the target parameter prediction model, the target parameters that the thread pool in the client needs to configure before processing the thread task to be processed, the method further includes: Based on the hardware resource data, determining whether the client has a resource overload condition; The step of inputting the hardware resource data into the target parameter prediction model and predicting the target parameters that the thread pool in the client needs to configure before processing the thread task to be processed by the target parameter prediction model includes: If it is determined that the client does not have a resource overload situation, the hardware resource data is input into the target parameter prediction model, and the target parameter is predicted by the target parameter prediction model.
9. The method according to claim 8, characterized in that The method further comprises: If it is determined that the client has a resource overload situation, the parameters of the thread pool are adjusted to predetermined parameters, and the to-be-processed thread tasks are processed based on the adjusted thread pool.
10. The method according to claim 8, characterized in that Before adjusting the parameters of the thread pool to the target parameters and processing the to-be-processed thread tasks based on the adjusted thread pool, the method further includes: Performing safety verification on the target parameters; The step of adjusting the parameters of the thread pool to the target parameters and processing the to-be-processed thread tasks based on the adjusted thread pool includes: When it is determined that the target parameter meets the predetermined safety verification condition, the configuration parameters of the predetermined thread pool framework are adjusted to the target parameter through the application programming interface of the predetermined thread pool framework in the thread pool, and the to-be-processed thread task is processed based on the adjusted thread pool; When it is determined that the target parameter does not meet the predetermined safety verification condition, the configuration parameters of the predetermined thread pool framework are adjusted to the predetermined parameters, and the to-be-processed thread tasks are processed based on the adjusted thread pool.
11. The method according to claim 1, characterized in that: The method further comprises: Collecting key data of the client in the process of processing the pending thread task; Based on the distributed system architecture, the key data is sent to the server according to a predetermined sending cycle, and the server is used to uniformly store the key data in a predetermined storage location.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.
13. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 11 is implemented.
14. A computer program product having a computer program stored thereon, characterized in that: When the computer program product is executed by a processor, the method according to any one of claims 1 to 11 is implemented.
Citation Information
Patent Citations
Thread pool scheduling method, system and device and readable storage medium
CN115016916A