Data processing method and device, computer device and storage medium

Abstract

The application relates to a data processing method and device, computer equipment and a storage medium. It relates to the technical field of artificial intelligence. The method comprises the following steps: determining a current prediction value of a performance index of a message queue service according to a current parameter value of a configuration parameter of the message queue service by using a performance prediction model; wherein the configuration parameter comprises at least two of a message monitoring thread number, a message processing thread number, a queue depth, interval layering information and a sleep time; the performance index comprises a message packet loss number and / or a message processing time consumption; and if it is determined that the current prediction value does not meet the performance index requirement, the current parameter value of the configuration parameter is adjusted. The method can accurately determine the adjustment time of the configuration parameter and adjust the parameter value. Therefore, the accuracy of the parameter value of the configuration parameter of the message queue service is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a data processing method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the development of computer technology, the asynchronous communication mode through the message service application program interface has gradually become popular, for example, the asynchronous communication between two application programs or between devices in a distributed system is realized through the message queue service by using the message service application program interface.

[0003] Currently, when asynchronous communication is performed through the message queue service, the parameter value of the configuration parameter of the message queue service needs to be set. Whether the parameter value setting is reasonable directly affects the performance indicators of the message queue service. Therefore, how to ensure the accuracy of the parameter value of the configuration parameter of the message queue service is a problem that needs to be solved at present. SUMMARY

[0004] Therefore, it is necessary to provide a data processing method and device, computer equipment, computer readable storage medium and computer program product capable of ensuring the accuracy of the parameter value of the configuration parameter of the message queue service in view of the above technical problems.

[0005] In a first aspect, the present application provides a data processing method. The method comprises:

[0006] determining, by a performance prediction model, a current predicted value of a performance indicator of the message queue service according to a current parameter value of a configuration parameter of the message queue service; wherein the configuration parameter comprises at least two of a message listening thread number, a message processing thread number, a queue depth, interval layering information and a sleep time; and the performance indicator comprises a message packet loss number and / or a message processing time consumption;

[0007] adjusting the current parameter value of the configuration parameter if it is determined that the current predicted value does not meet the performance indicator requirement.

[0008] In one of the embodiments, determining that the current predicted value does not meet the performance indicator requirement comprises:

[0009] determining that the current predicted value does not meet the performance indicator requirement if the current predicted value does not fall within a reference performance indicator interval.

[0010] In one of the embodiments, adjusting the current parameter value of the configuration parameter comprises:

[0011] determining a to-be-adjusted parameter and an adjustment strategy of the to-be-adjusted parameter from the configuration parameter according to the type of the performance indicator that does not meet the performance indicator requirement;

[0012] According to the adjustment strategy, the current parameter value of the to-be-adjusted parameter is adjusted.

[0013] In one of the embodiments, after adjusting the current parameter value of the configuration parameter, the method further comprises:

[0014] According to the adjusted current parameter value, the operation of re-executing the determination of the current predicted value of the performance indicator of the message queue service according to the current parameter value of the configuration parameter of the message queue service by the performance prediction model is returned until the current predicted value of the performance indicator corresponding to the current parameter value meets the performance indicator requirement.

[0015] In one of the embodiments, the performance prediction model is constructed based on a multiple linear regression algorithm.

[0016] In one of the embodiments, the method further comprises:

[0017] constructing at least one multiple linear regression model; the multiple linear regression model represents a linear relationship between the performance indicator and at least two configuration parameters;

[0018] solving model parameters of each multiple linear regression model according to sample parameter values of the configuration parameters and sample label values of the performance indicator corresponding to the sample parameter values, to obtain trained each multiple linear regression model;

[0019] determining the performance prediction model from the trained each multiple linear regression model.

[0020] In one of the embodiments, if the number of the constructed multiple linear regression models is at least two, at least one of the number, the type and the parameter weight of the configuration parameters contained in different multiple linear regression models is different.

[0021] In one of the embodiments, determining the performance prediction model from the trained each multiple linear regression model comprises:

[0022] testing the trained each multiple linear regression model by the test parameter value of the configuration parameter to obtain a test predicted value of the performance indicator corresponding to the test parameter value;

[0023] determining a correlation coefficient of the trained each multiple linear regression model according to a test label value of the performance indicator corresponding to the test parameter value and the test predicted value;

[0024] determining the performance prediction model from the trained each multiple linear regression model according to the correlation coefficient of the trained each multiple linear regression model.

[0025] In a second aspect, the present application further provides a data processing device. The device comprises:

[0026] The prediction module is used to determine the current predicted values ​​of the performance metrics of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service using a performance prediction model. The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance metrics include: number of message packet losses and / or message processing time.

[0027] The adjustment module is used to adjust the current parameter value of the configuration parameters if it is determined that the current predicted value does not meet the performance index requirements.

[0028] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0029] The performance prediction model determines the current predicted values ​​of the message queue service's performance metrics based on the current values ​​of the configuration parameters of the message queue service. The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance metrics include: number of message packet losses and / or message processing time.

[0030] If it is determined that the current predicted value does not meet the performance requirements, the current parameter values ​​of the configuration parameters will be adjusted.

[0031] Fourthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0032] The performance prediction model determines the current predicted values ​​of the message queue service's performance metrics based on the current values ​​of the configuration parameters of the message queue service. The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance metrics include: number of message packet losses and / or message processing time.

[0033] If it is determined that the current predicted value does not meet the performance requirements, the current parameter values ​​of the configuration parameters will be adjusted.

[0034] Fifthly, this application also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0035] The performance prediction model determines the current predicted values ​​of the message queue service's performance metrics based on the current values ​​of the configuration parameters of the message queue service. The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance metrics include: number of message packet losses and / or message processing time.

[0036] If it is determined that the current predicted value does not meet the performance requirements, the current parameter values ​​of the configuration parameters will be adjusted.

[0037] The aforementioned data processing method, apparatus, computer equipment, and storage medium employ a performance prediction model. Based on the current parameter values ​​of multi-dimensional configuration parameters of the message queue service, they determine the current predicted values ​​of the message queue service's performance indicators. When the current predicted values ​​do not meet the performance indicator requirements, the current parameter values ​​of the configuration parameters are adjusted. The configuration parameters involved in this embodiment include multiple dimensions such as the number of message listening threads, the number of message processing threads, queue depth, interval layering information, and sleep time. This allows for precise determination of the timing for adjusting the configuration parameters and ensures the accuracy of the parameter values ​​of the message queue service's configuration parameters, thereby guaranteeing that the message queue service can operate with optimal performance indicators. Attached Figure Description

[0038] Figure 1 This is a diagram illustrating the application environment of a data processing method in one embodiment.

[0039] Figure 2 This is a flowchart illustrating a data processing method in one embodiment;

[0040] Figure 3 This is a flowchart illustrating the data processing method in another embodiment;

[0041] Figure 4 This is a flowchart illustrating the data processing method in another embodiment;

[0042] Figure 5 This is a flowchart illustrating the data processing method in another embodiment;

[0043] Figure 6 This is a flowchart illustrating the data processing method in another embodiment;

[0044] Figure 7 This is a structural block diagram of a data processing device in one embodiment;

[0045] Figure 8 This is a structural block diagram of the data processing apparatus in another embodiment;

[0046] Figure 9This is a structural block diagram of the data processing apparatus in another embodiment;

[0047] Figure 10 This is a structural block diagram of the data processing apparatus in another embodiment;

[0048] Figure 11 This is a structural block diagram of the data processing apparatus in another embodiment;

[0049] Figure 12 This is a structural block diagram of the data processing apparatus in another embodiment;

[0050] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] The data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, in one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows. Figure 1 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to data processing. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method.

[0053] In one embodiment, such as Figure 2 As shown, a data processing method is provided that can be used in scenarios where configuration parameters of a message queue service are adjusted. This method is applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0054] S201, using a performance prediction model, determines the current predicted value of the performance metrics of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service.

[0055] The performance prediction model can be a pre-trained model capable of predicting the performance metrics of the message queue service based on the parameter values ​​of the service's configuration parameters. Optionally, this performance prediction model can be constructed based on a multiple linear regression algorithm.

[0056] Message Queue (MQ) service can be a service that enables asynchronous communication between two applications or between devices in a distributed system through the Java Message Service (JMS) application programming interface service, encompassing all data structures and delivery processes.

[0057] Optionally, the message queue service can operate as follows: The message queue service has two thread pools: a message listening thread pool and a message processing thread pool. The message listening thread pool listens for and retrieves messages from the message queue server, then sends them to the message processing thread pool. The message processing thread pool checks if the current core thread count is greater than the relevant configuration parameters for message processing threads, such as the core thread count. If so, the message queue service places the received messages into the message queue; otherwise, it directly processes the received messages using the message processing thread.

[0058] The configuration parameters for a message queue service can be the parameters required for the message queue to perform the above-mentioned tasks. For example, these can include, but are not limited to, at least two of the following: the number of message listening threads, the number of message processing threads, queue depth, interval hierarchical information, and sleep time.

[0059] The number of message listening threads can be a parameter configured internally within the message listening thread pool of the message queue service. For example, it can include the core number of message listening threads and the maximum number of message listening threads, with the core number typically less than or equal to the maximum number of message listening threads. The core number of message listening threads refers to the maximum number of core message listening threads set in the message listening thread pool; the maximum number of message listening threads refers to the total number of all message listening threads set in the message listening thread pool.

[0060] The number of message processing threads can be a parameter configured internally within the message processing thread pool of the message queue service. For example, it can include the core number of message processing threads and the maximum number of message processing threads, with the core number typically less than or equal to the maximum number of message processing threads. The core number of message processing threads refers to the maximum number of core message processing threads set in the message processing thread pool; the maximum number of message processing threads refers to the total number of all message processing threads set in the message processing thread pool.

[0061] Queue depth can be the depth value of the message queue in the message queue service that stores messages to be processed.

[0062] Interval layering information can be related to the interval layering processing of the message queue. For example, it can include the number of interval layers and the interval range. The number of interval layers can represent the number of intervals into which the message queue is divided, and the interval range can represent the depth range of the message queue corresponding to each interval after division.

[0063] The sleep time can be the duration for which the message listening service is paused when the message processing efficiency of the message queue service is lower than the message listening efficiency. Optionally, the sleep time in this embodiment can be a fixed parameter value or it can include multiple parameter values. For example, different sleep times can be set for different intervals of the message queue, and different sleep times can be selected according to the interval occupied by the messages to be processed in the message queue.

[0064] Performance metrics for a message queue service can be parameters used to measure the system's performance during data processing. These can include, but are not limited to, message packet loss count and / or message processing time. Message packet loss typically occurs because the message processing efficiency of the message queue service is lower than its message listening efficiency. The message packet loss count refers to the number of lost packets that occur during message processing. Message processing time refers to the total time it takes for the message queue service to complete the current message processing task.

[0065] Optionally, in this embodiment, the current parameter values ​​of the configuration parameters of the message queue service can be obtained, and the obtained current parameter values ​​can be input into the performance prediction model. The model can then parse and obtain the current predicted values ​​of the performance indicators of the message queue service based on the input current parameter values.

[0066] Specifically, the performance prediction model in this embodiment can be represented by the following formula (1),

[0067] y = b0 + b1x1 + ... + b k x k (1)

[0068] In the formula, y represents the current predicted value of the performance metric of the message queue service, and x1 to x... k This represents the current parameter value of the message queue service configuration parameters, from b0 to b... k This represents the model parameters to be solved during the training of the performance prediction model.

[0069] By substituting the current parameter values ​​of the configuration parameters of the message queue service into the performance prediction model, the y value in formula (1) can be calculated, which is the current predicted value of the performance index of the message queue service.

[0070] S202, If it is determined that the current predicted value does not meet the performance index requirements, the current parameter value of the configuration parameter shall be adjusted.

[0071] The performance metrics can be pre-set criteria used to measure whether the message queue service's performance meets the requirements. For example, the performance metric could be a pre-set threshold, and the performance of the message queue service is determined by whether the current predicted value of its performance metric is less than this threshold.

[0072] For example, the performance metric requirement can be a pre-set threshold. If the current predicted value is less than the threshold, it is determined that the current predicted value meets the performance metric requirement. If the current predicted value is greater than the threshold, it is determined that the current predicted value does not meet the performance metric requirement.

[0073] Alternatively, the performance requirement can be a pre-set reference performance range. The performance of the message queue service can be measured by whether the predicted performance value falls within this range.

[0074] Specifically, if the current predicted value does not fall within the reference performance index range, that is, if the current predicted value is less than the minimum interval limit or greater than the maximum interval limit, then the current predicted value is determined to not meet the performance index requirements.

[0075] By setting a reference performance index as a range, values ​​below or above this range are considered not to meet the performance index requirements. This increases the constraints on the performance index requirements, making the current parameter values ​​of the configuration parameters adjusted based on these constraints more accurate.

[0076] Optionally, if the current predicted value does not meet the performance requirements, this embodiment can adjust the current parameter value of the configuration parameters in many ways, and no limitation is made here. One possible approach is to have professional personnel adjust the configuration parameters based on their experience. Another possible approach is to adjust the configuration parameters according to pre-set adjustment rules. For example, if the number of message packet loss is high, the sleep time of each interval after queue layering can be increased; if the message processing efficiency is low, the number of message processing threads can be increased or the number of message listening threads can be decreased, etc.

[0077] In the above embodiments, the current parameter values ​​of the message queue service's configuration parameters are substituted into the performance prediction model to determine the current predicted value of the message queue service's performance indicators. When the current predicted value does not meet the performance indicator requirements, the current parameter values ​​of the configuration parameters are adjusted. Since the configuration parameters include at least two of the following: the number of message listening threads, the number of message processing threads, queue depth, interval layering information, and sleep time, the timing for adjusting the configuration parameters can be accurately determined, and adjustments can be made to configuration parameters across multiple dimensions. This ensures the accuracy of the parameter values ​​of the message queue service's configuration parameters, thereby ensuring that the message queue service operates with optimal performance indicators.

[0078] Furthermore, after adjusting the current parameter values ​​of the configuration parameters, the operation of determining the current predicted value of the performance index of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service can be re-executed according to the adjusted current parameter values, until the current predicted value of the performance index corresponding to the current parameter value meets the performance index requirements.

[0079] Specifically, after adjusting the current parameter values ​​of the configuration parameters, the adjusted values ​​are re-inputted into the performance prediction model to determine the current predicted value of the performance metric corresponding to the current parameter value. It is then determined whether the current predicted value meets the preset performance metric requirements. If not, the current parameter values ​​are adjusted again, and the process of determining the current predicted value of the performance metric for the message queue service based on the current parameter values ​​of the configuration parameters is repeated, until the current predicted value of the performance metric corresponding to the adjusted parameter value meets the performance metric requirements. If it does, then no further adjustment of the current parameter values ​​of the configuration parameters for the message queue service is necessary.

[0080] To ensure the message queue service operates at optimal performance metrics, when the current predicted value fails to meet performance requirements, the current parameter values ​​of the configuration parameters need to be adjusted. In one embodiment, such as... Figure 3 As shown, adjusting the current parameter values ​​of the configuration parameters includes:

[0081] S301 determines the parameters to be adjusted and the adjustment strategy for the parameters to be adjusted from the configuration parameters based on the types of performance indicators that do not meet the performance requirements.

[0082] In this embodiment, the performance metrics include two categories: message packet loss count and message processing time. The parameters to be adjusted can be at least one of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time.

[0083] Optionally, there are many ways to determine the parameters to be adjusted based on the types of performance metrics that do not meet the performance requirements, and there is no limitation on this. For example, there could be a pre-set correspondence between message packet loss count and the parameters to be adjusted, or a correspondence between message processing time and the parameters to be adjusted. For instance, there might be a correspondence between message packet loss count and sleep time, interval layering information, and queue depth; or a correspondence between message processing time and the number of message listening threads, and between the number of message processing threads and sleep time. In this case, the parameters to be adjusted can be determined based on the pre-set correspondence and the types of performance metrics that do not meet the performance requirements.

[0084] Optionally, the adjustment strategy for the parameter to be adjusted can be a pre-set strategy of increasing or decreasing the value of different configuration parameters corresponding to various situations where performance requirements are not met. Alternatively, the adjustment strategy can be set to allow professional technicians to adjust the parameter to be adjusted based on their experience.

[0085] S302, adjust the current parameter value of the parameter to be adjusted according to the adjustment strategy.

[0086] In this embodiment, the current value of the parameter to be adjusted can be adjusted according to the determined parameter to be adjusted and the determined adjustment strategy. For example, if the determined adjustment strategy is to adjust by professional technicians, the corresponding way to adjust the current value of the parameter to be adjusted is to adjust the configuration parameter based on the experience of the professional technicians.

[0087] In this embodiment, based on the performance indicators that do not meet the performance requirements, the parameters to be adjusted and their adjustment strategies are determined. The current values ​​of the parameters to be adjusted are then adjusted according to the determined adjustment strategies. Since different configuration parameters are adjusted for different types of performance indicators that do not meet the requirements, this embodiment improves the diversity and accuracy of configuration parameter adjustment methods. Furthermore, the adjustment strategies for each parameter in the configuration parameters are pre-set, so after the parameters to be adjusted are determined, they can be directly adjusted according to the pre-set adjustment strategies. In other words, this embodiment also improves the efficiency of adjusting configuration parameters.

[0088] Furthermore, in the scenario described above where the performance prediction model is used to determine the current predicted values ​​of the message queue service's performance metrics, this performance prediction model is constructed based on a multiple linear regression algorithm. In one embodiment, such as... Figure 4 As shown, the above data processing method also includes:

[0089] S401, Construct at least one multiple linear regression model; the multiple linear regression model characterizes the linear relationship between the performance index and at least two configuration parameters.

[0090] Among them, the multiple linear regression model is a regression analysis model performed when there is a linear relationship between multiple independent variables and the dependent variable. It can characterize the linear relationship between performance indicators and at least two configuration parameters. Specifically, in this embodiment, the multiple independent variables in the multiple linear regression model can be at least two configuration parameters, and the multiple dependent variables in the multiple linear regression model can be multiple performance indicators, such as message packet loss, message processing time, or both message packet loss and message processing time.

[0091] In this embodiment, the formula for the at least one constructed multiple linear regression model may be, for example, the above formula (1).

[0092] It is understandable that if at least two multiple linear regression models are constructed, then at least one of the following will differ: the number, type, and weights of the configuration parameters. For example, different multiple linear regression models may differ in one or more of the following: the number, type, and weights of the configuration parameters. In other words, any two different multiple linear regression models will have different independent variable x. k The quantities are different, that is, the values ​​of k in the above formula (1) are different, even if the independent variable x it contains is different. k The quantities are the same, but the independent variable x is the same. k The types of configuration parameters represented in different multilinear regression models may also differ; even if the included independent variable x k The number of parameters is the same, and the types of configuration parameters they represent are also the same. However, the parameter weights of the same type of configuration parameter may be different in different multilinear regression models.

[0093] S402, based on the sample parameter values ​​of the configuration parameters and the sample label values ​​of the performance indicators corresponding to the sample parameter values, solve the model parameters of each multiple linear regression model to obtain the trained multiple linear regression models.

[0094] The sample parameter values ​​for the configuration parameters are the historical configuration parameter values ​​of the message queue service. The sample label values ​​for the performance metrics corresponding to the sample parameter values ​​are the actual values ​​of the performance metrics under the determined sample parameter values. In this embodiment, for each constructed multivariate linear regression model, the sample parameter values ​​of each set of configuration parameters are used as independent variables x1 to x... k Substituting these values ​​into formula (1) above, we obtain the sample predicted values ​​y for each group. We then calculate the error sum ∑e (e.g., the sum of squared errors) between the sample predicted values ​​y for each group and the sample label values ​​y' of the performance metrics of the message queue service. Finally, we solve for the model parameters b0 to b when the error sum is minimized. kThe trained multiple linear regression model is obtained. The trained multiple linear regression model should be in the format of formula (1) above, and b0 to b k The value has been determined.

[0095] S403 determines the performance prediction model from the various trained multiple linear regression models.

[0096] Optionally, if the number of trained multiple linear regression models is one, then the trained multiple linear regression model is determined to be a performance prediction model.

[0097] When multiple multiple linear regression models are constructed, to ensure that the trained models can accurately predict the performance metrics of the message queue service, this embodiment selects the best-performing multiple linear regression model from among the trained models as the performance prediction model. For example, multiple sets of test parameter values ​​for the configuration parameters of the message queue service are input into the trained multiple linear regression model to obtain the test predicted value y of the performance metric, and the true value y' of the performance metric of the message queue service corresponding to the test parameter value is obtained. The error value of the multiple linear regression model is set as e = y' - y, and the magnitude of e is used to determine the accuracy of the prediction result. The smaller the value of e, the more accurate the prediction result of the multiple linear regression model. The larger the value of e, the less accurate the prediction result of the multiple linear regression model. In this embodiment, the multiple linear regression model with the smallest error value e can be selected as the performance prediction model.

[0098] In this embodiment, for the constructed multiple multiple linear regression models, each model is trained based on the true values ​​of the performance indicators of the message queue service and the current parameter values ​​of the configuration parameters of the message queue service. This results in multiple trained multiple multiple linear regression models. Then, using the error values ​​of each model, the model with the most accurate prediction result is determined as the performance prediction model. This ensures that the performance prediction model can more accurately predict the values ​​of various performance indicators of the message queue service, thus improving the accuracy of the performance indicator prediction model.

[0099] Furthermore, to improve the accuracy of performance prediction model determination, the performance prediction model can also be determined based on the principle of correlation coefficient. In one embodiment, such as... Figure 5 As shown, the performance prediction models determined from the various trained multiple linear regression models include:

[0100] S501 tests each trained multiple linear regression model by configuring the test parameter values ​​and obtains the test prediction values ​​of the performance indicators corresponding to the test parameter values.

[0101] Specifically, for each trained multiple linear regression model, the test parameter values ​​of each configuration parameter in the message queue service are input into the trained multiple linear regression model to obtain the test prediction value y of the performance index corresponding to each test parameter value output by the multiple linear regression model.

[0102] S502, based on the test label value and test prediction value of the performance index corresponding to the test parameter value, determine the correlation coefficient of each trained multiple linear regression model.

[0103] The correlation coefficient of each trained multiple linear regression model is used to measure the strength of the correlation between two variables, and the coefficient ranges from 0 to 1. That is, the closer the correlation coefficient is to 1, the more accurate the prediction result of the multiple linear regression model; conversely, the closer the correlation coefficient is to 0, the less accurate the prediction result of the multiple linear regression model.

[0104] In this embodiment, the correlation coefficients of each trained multiple linear regression model are determined based on the test parameter values ​​and the test label values ​​and test prediction values ​​of their corresponding performance indicators. The specific determination method is shown in the following formula (2).

[0105]

[0106] In the formula, x i This represents the test parameter value of the configuration parameters. Indicates with x i The reference value for the corresponding configuration parameter, y i The test label value represents the performance index corresponding to the test parameter value. Indicates with y i The corresponding performance metric label value.

[0107] Specifically, the predicted values ​​of the performance indicators corresponding to the test parameter values ​​output by each multiple linear regression model, the test label values ​​of the performance indicators of each multiple linear regression model, and the current and reference values ​​of each configuration parameter corresponding to each test parameter value are input into the above formula (2) to obtain the correlation coefficients of each multiple linear regression model.

[0108] S503 determines the performance prediction model from among the various trained multiple linear regression models based on their correlation coefficients.

[0109] Specifically, the correlation coefficients of the various multiple linear regression models are compared to obtain the comparison results. Based on the comparison results, the multiple linear regression model with the largest correlation coefficient is determined as the performance prediction model.

[0110] In this embodiment, the trained multiple linear regression models are tested using the test parameter values ​​of the configured parameters to obtain the test predicted values ​​of the performance indicators corresponding to the test parameter values. The correlation coefficients of each trained multiple linear regression model are determined based on the test label values ​​and test predicted values ​​of the performance indicators corresponding to the test parameter values. The performance prediction model is then determined by comparing the magnitudes of the correlation coefficients of each multiple linear regression model. Utilizing the principle of correlation coefficients to determine the final performance prediction model from multiple trained linear regression models improves the accuracy and reliability of the performance prediction model determination.

[0111] To facilitate understanding by those skilled in the art, the data processing methods provided in this disclosure will be described in detail, such as... Figure 6 As shown, the method may include:

[0112] S601, construct at least one multiple linear regression model, wherein each multiple linear regression model represents the linear relationship between a performance index and at least two configuration parameters.

[0113] S602, based on the sample parameter values ​​of the configuration parameters and the sample label values ​​of the performance indicators corresponding to the sample parameter values, solve the model parameters of each multiple linear regression model to obtain the trained multiple linear regression models.

[0114] S603 tests each trained multiple linear regression model by configuring the test parameter values, obtains the test prediction values ​​of the performance indicators corresponding to the test parameter values, and determines the correlation coefficients of each trained multiple linear regression model based on the test label values ​​and test prediction values ​​of the performance indicators corresponding to the test parameter values.

[0115] S604: Based on the correlation coefficients of the various trained multiple linear regression models, determine the performance prediction model from among the various trained multiple linear regression models.

[0116] S605 determines the current predicted value of the performance metrics of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service.

[0117] The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance metrics include: number of message packet losses and / or message processing time.

[0118] S606, determine whether the current predicted value meets the performance index requirements. If yes, proceed to S607; otherwise, proceed to S608.

[0119] S607, end the adjustment of configuration parameters for the message queue service.

[0120] S608 adjusts the current predicted values ​​of the performance metrics for the message queue service.

[0121] Specifically, after adjusting the current predicted value of the performance metric of the message queue service, the operation of S605 is re-executed based on the adjusted real-time setting parameters until the current predicted value of the performance metric corresponding to the current parameter value meets the performance metric requirements.

[0122] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0123] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0124] In one embodiment, such as Figure 7 As shown, a data processing device 1 is provided, including: a prediction module 10 and an adjustment module 11, wherein:

[0125] The prediction module 10 is used to determine the current predicted value of the performance indicators of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service through the performance prediction model. The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance indicators include: number of message packet loss and / or message processing time.

[0126] The adjustment module 11 is used to adjust the current parameter value of the configuration parameters if it is determined that the current predicted value does not meet the performance index requirements.

[0127] In one embodiment, such as Figure 8 As shown above, Figure 7The data processing device 1 shown also includes a first determination module 12, which is used to determine that the current predicted value does not meet the performance index requirements if the current predicted value does not fall within the reference performance index range.

[0128] In one embodiment, such as Figure 9 As shown, the adjustment module 11 includes a strategy determination unit 111 and an adjustment unit 112. Wherein:

[0129] The strategy determination unit 111 is used to determine the parameters to be adjusted and the adjustment strategy of the parameters to be adjusted from the configuration parameters according to the types of performance indicators that do not meet the performance index requirements.

[0130] The adjustment unit 112 is used to adjust the current parameter value of the parameter to be adjusted according to the adjustment strategy.

[0131] In one embodiment, such as Figure 10 As shown above, Figure 7 The data processing device 1 shown also includes a re-prediction module 13.

[0132] The re-prediction module 13 is used to re-execute the operation of determining the current predicted value of the performance index of the message queue service based on the current parameter value of the configuration parameters of the message queue service through the performance prediction model, until the current predicted value of the performance index corresponding to the current parameter value meets the performance index requirements.

[0133] It should be noted that the above performance prediction model is constructed based on the multiple linear regression algorithm.

[0134] In one embodiment, such as Figure 11 As shown above, Figure 7 The data processing device 1 shown also includes a construction module 14, a solution module 15, and a second determination module 16. Wherein:

[0135] Module 14 is used to construct at least one multiple linear regression model; the multiple linear regression model represents the linear relationship between performance indicators and at least two configuration parameters.

[0136] The solver module 15 is used to solve the model parameters of each multiple linear regression model based on the sample parameter values ​​of the configuration parameters and the sample label values ​​of the performance indicators corresponding to the sample parameter values, so as to obtain the trained multiple linear regression models.

[0137] The second determination module 16 is used to determine the performance prediction model from the various trained multiple linear regression models.

[0138] It should be noted that if at least two multiple linear regression models are constructed, then at least one of the following will be different: the number of configuration parameters, the types of parameters, and the parameter weights contained in the different multiple linear regression models.

[0139] In one embodiment, such as Figure 12 As shown, the second determining module 16 includes: a testing unit 161, a correlation coefficient determining unit 162, and a model determining unit 163. Wherein:

[0140] Test unit 161 tests each trained multiple linear regression model by configuring the test parameter values ​​and obtains the test prediction values ​​of the performance indicators corresponding to the test parameter values.

[0141] The correlation coefficient determination unit 162 determines the correlation coefficient of each trained multiple linear regression model based on the test label value and test prediction value of the performance index corresponding to the test parameter value.

[0142] Model determination unit 163 determines the performance prediction model from among the various trained multiple linear regression models based on the correlation coefficients of each model.

[0143] The application login device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so they will not be described again here.

[0144] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0145] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 13As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a data processing method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0146] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0147] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0148] The performance prediction model determines the current predicted values ​​of the message queue service's performance metrics based on the current values ​​of the configuration parameters of the message queue service. The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance metrics include: number of message packet losses and / or message processing time.

[0149] If it is determined that the current predicted value does not meet the performance requirements, the current parameter values ​​of the configuration parameters will be adjusted.

[0150] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0151] If the current predicted value does not fall within the reference performance index range, then the current predicted value is determined to not meet the performance index requirements.

[0152] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0153] Based on the types of performance indicators that do not meet the performance requirements, determine the parameters to be adjusted from the configuration parameters, as well as the adjustment strategy for the parameters to be adjusted.

[0154] According to the adjustment strategy, the current parameter value of the parameter to be adjusted is adjusted.

[0155] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0156] Based on the adjusted current parameter values, return and re-execute the operation of determining the current predicted value of the performance index of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service, until the current predicted value of the performance index corresponding to the current parameter value meets the performance index requirements.

[0157] In one embodiment, when the processor executes the computer program, it also performs the following steps: the performance prediction model is constructed based on a multiple linear regression algorithm.

[0158] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0159] Construct at least one multiple linear regression model; the multiple linear regression model characterizes the linear relationship between performance indicators and at least two configuration parameters;

[0160] Based on the sample parameter values ​​of the configuration parameters and the sample label values ​​of the performance indicators corresponding to the sample parameter values, the model parameters of each multiple linear regression model are solved to obtain the trained multiple linear regression models.

[0161] The performance prediction model is determined from the various trained multiple linear regression models.

[0162] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0163] If at least two multiple linear regression models are constructed, then at least one of the following must be different: the number of configuration parameters, the types of parameters, and the parameter weights contained in the different multiple linear regression models.

[0164] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0165] By configuring the test parameter values, the trained multiple linear regression models are tested to obtain the test prediction values ​​of the performance indicators corresponding to the test parameter values.

[0166] Based on the test label values ​​and test prediction values ​​of the performance indicators corresponding to the test parameter values, determine the correlation coefficients of each trained multiple linear regression model;

[0167] Based on the correlation coefficients of the various trained multiple linear regression models, the performance prediction model is determined from among them.

[0168] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0169] The performance prediction model determines the current predicted values ​​of the message queue service's performance metrics based on the current values ​​of the configuration parameters of the message queue service. The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance metrics include: number of message packet losses and / or message processing time.

[0170] If it is determined that the current predicted value does not meet the performance requirements, the current parameter values ​​of the configuration parameters will be adjusted.

[0171] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0172] If the current predicted value does not fall within the reference performance index range, then the current predicted value is determined to not meet the performance index requirements.

[0173] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0174] Based on the types of performance indicators that do not meet the performance requirements, determine the parameters to be adjusted from the configuration parameters, as well as the adjustment strategy for the parameters to be adjusted.

[0175] According to the adjustment strategy, the current parameter value of the parameter to be adjusted is adjusted.

[0176] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0177] Based on the adjusted current parameter values, return and re-execute the operation of determining the current predicted value of the performance index of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service, until the current predicted value of the performance index corresponding to the current parameter value meets the performance index requirements.

[0178] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: the performance prediction model is constructed based on a multiple linear regression algorithm.

[0179] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0180] Construct at least one multiple linear regression model; the multiple linear regression model characterizes the linear relationship between performance indicators and at least two configuration parameters;

[0181] Based on the sample parameter values ​​of the configuration parameters and the sample label values ​​of the performance indicators corresponding to the sample parameter values, the model parameters of each multiple linear regression model are solved to obtain the trained multiple linear regression models.

[0182] The performance prediction model is determined from the various trained multiple linear regression models.

[0183] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0184] If at least two multiple linear regression models are constructed, then at least one of the following must be different: the number of configuration parameters, the types of parameters, and the parameter weights contained in the different multiple linear regression models.

[0185] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0186] By configuring the test parameter values, the trained multiple linear regression models are tested to obtain the test prediction values ​​of the performance indicators corresponding to the test parameter values.

[0187] Based on the test label values ​​and test prediction values ​​of the performance indicators corresponding to the test parameter values, determine the correlation coefficients of each trained multiple linear regression model;

[0188] Based on the correlation coefficients of the various trained multiple linear regression models, the performance prediction model is determined from among them.

[0189] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0190] The performance prediction model determines the current predicted values ​​of the message queue service's performance metrics based on the current values ​​of the configuration parameters of the message queue service. The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance metrics include: number of message packet losses and / or message processing time.

[0191] If it is determined that the current predicted value does not meet the performance requirements, the current parameter values ​​of the configuration parameters will be adjusted.

[0192] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0193] If the current predicted value does not fall within the reference performance index range, then the current predicted value is determined to not meet the performance index requirements.

[0194] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0195] Based on the types of performance indicators that do not meet the performance requirements, determine the parameters to be adjusted from the configuration parameters, as well as the adjustment strategy for the parameters to be adjusted.

[0196] According to the adjustment strategy, the current parameter value of the parameter to be adjusted is adjusted.

[0197] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0198] Based on the adjusted current parameter values, return and re-execute the operation of determining the current predicted value of the performance index of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service, until the current predicted value of the performance index corresponding to the current parameter value meets the performance index requirements.

[0199] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: the performance prediction model is constructed based on a multiple linear regression algorithm.

[0200] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0201] Construct at least one multiple linear regression model; the multiple linear regression model characterizes the linear relationship between performance indicators and at least two configuration parameters;

[0202] Based on the sample parameter values ​​of the configuration parameters and the sample label values ​​of the performance indicators corresponding to the sample parameter values, the model parameters of each multiple linear regression model are solved to obtain the trained multiple linear regression models.

[0203] The performance prediction model is determined from the various trained multiple linear regression models.

[0204] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0205] If at least two multiple linear regression models are constructed, then at least one of the following must be different: the number of configuration parameters, the types of parameters, and the parameter weights contained in the different multiple linear regression models.

[0206] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0207] By configuring the test parameter values, the trained multiple linear regression models are tested to obtain the test prediction values ​​of the performance indicators corresponding to the test parameter values.

[0208] Based on the test label values ​​and test prediction values ​​of the performance indicators corresponding to the test parameter values, determine the correlation coefficients of each trained multiple linear regression model;

[0209] Based on the correlation coefficients of the various trained multiple linear regression models, the performance prediction model is determined from among them.

[0210] It should be noted that the configuration parameters (including but not limited to the number of message listening threads, the number of message processing threads, queue depth, interval layering information, and sleep time) and performance indicators (including the number of message packet losses and / or message processing time) involved in this application are all information and data that have been fully authorized by all parties.

[0211] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0212] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0213] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: The performance prediction model determines the current predicted values ​​of the performance metrics of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service. The configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time. The performance metrics include: number of lost messages and / or message processing time. If it is determined that the current predicted value does not meet the performance requirements, the current parameter value of the configuration parameter will be adjusted.

2. The method according to claim 1, characterized in that, The determination that the current predicted value does not meet the performance index requirements includes: If the current predicted value does not fall within the reference performance index range, then the current predicted value is determined to not meet the performance index requirements.

3. The method according to claim 1, characterized in that, Adjusting the current parameter values ​​of the configuration parameters includes: Based on the types of performance indicators that do not meet the performance requirements, determine the parameters to be adjusted from the configuration parameters, and the adjustment strategy for the parameters to be adjusted. According to the adjustment strategy, the current parameter value of the parameter to be adjusted is adjusted.

4. The method according to claim 1, characterized in that, After adjusting the current parameter value of the configuration parameter, the method further includes: Based on the adjusted current parameter value, the process is re-executed by using the performance prediction model to determine the current predicted value of the performance metric of the message queue service based on the current parameter value of the configuration parameters of the message queue service, until the current predicted value of the performance metric corresponding to the current parameter value meets the performance metric requirements.

5. The method according to any one of claims 1 to 4, characterized in that, The performance prediction model is constructed based on the multiple linear regression algorithm.

6. The method according to claim 5, characterized in that, The method further includes: Construct at least one multiple linear regression model; the multiple linear regression model characterizes the linear relationship between the performance index and at least two of the configuration parameters; Based on the sample parameter values ​​of the configuration parameters and the sample label values ​​of the performance indicators corresponding to the sample parameter values, the model parameters of each multiple linear regression model are solved to obtain each trained multiple linear regression model. The performance prediction model is determined from the various trained multiple linear regression models.

7. The method according to claim 6, characterized in that, If at least two multiple linear regression models are constructed, then at least one of the following must be different: the number of configuration parameters, the types of parameters, and the parameter weights contained in the different multiple linear regression models.

8. The method according to claim 6, characterized in that, The performance prediction model is determined from the various trained multiple linear regression models, including: By using the test parameter values ​​of the configured parameters, the trained multiple linear regression models are tested to obtain the test prediction values ​​of the performance indicators corresponding to the test parameter values. Based on the test label value and test prediction value of the performance index corresponding to the test parameter value, determine the correlation coefficient of each trained multiple linear regression model; Based on the correlation coefficients of the various trained multiple linear regression models, the performance prediction model is determined from among the various trained multiple linear regression models.

9. A data processing apparatus, characterized in that, The device includes: The prediction module is used to determine the current predicted value of the performance indicators of the message queue service based on the current parameter values ​​of the configuration parameters of the message queue service using a performance prediction model; wherein the configuration parameters include at least two of the following: number of message listening threads, number of message processing threads, queue depth, interval layering information, and sleep time; the performance indicators include: number of message packet losses and / or message processing time. An adjustment module is used to adjust the current parameter value of the configuration parameter if it is determined that the current predicted value does not meet the performance index requirements.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images