Data processing method and device, equipment, storage medium and program product

Abstract

The invention provides a data processing method and device, equipment, a storage medium and a program product, relates to the technical field of computers, and is used for improving the rationality of resource allocation. The method is applied to a first server, a first application is deployed in the first server, and the first application is the same as a second application deployed in a second server; the method comprises the steps that first flow data is played back through a first application according to a preset playback rate, the first resource consumption of the first application is obtained, the first flow data comprises a request of a second application and response data fed back for the request, and the preset playback rate is related to the predicted service access amount; and determining the resource recommendation amount of the second application according to the first resource consumption amount and the redundant resource amount required by the second application.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, equipment, storage medium and program product. Background Art

[0002] With the rapid development of Internet business, the number of applications has also increased. Usually, applications are deployed in clusters or servers, and the resources in the clusters or servers support the normal operation of the applications. When an application is allocated fewer resources, there is a risk of not being able to operate normally. Therefore, in order to ensure the stability of application operation, application operators tend to allocate more resources to applications, but the actual amount of resources consumed is far less than the allocated amount of resources. Obviously, this unreasonable resource allocation method will lead to low resource utilization in clusters or servers and serious resource waste. Summary of the invention

[0003] The present application provides a data processing method, apparatus, device, storage medium and program product for improving the rationality of resource allocation.

[0004] In a first aspect, an embodiment of the present application provides a data processing method, which is applied to a first server, in which a first application is deployed, and the first application is the same as a second application deployed in a second server, and the method comprises: replaying the first traffic data at a preset playback rate by the first application to obtain a first resource consumption of the first application, the first traffic data comprising a request of the second application and response data fed back to the request, and the preset playback rate is related to the expected business access volume; determining the recommended resource amount for the second application based on the first resource consumption and the amount of redundant resources required for the second application.

[0005] In the embodiment of the present application, since the first traffic data is the traffic data of the second application during the service processing, the first resource consumption of the first application is obtained by replaying the first traffic data at a preset playback rate in the first application. Since the preset playback rate is related to the expected service access volume, the resource consumption can also reflect the estimated resource consumption of the second application under the expected service access volume. Therefore, it is more reasonable to determine the recommended resource amount of the second application based on the first resource consumption, that is, to improve the rationality of resource allocation.

[0006] Moreover, since there is still a certain error between the resource consumption obtained by the playback evaluation and the actual resource consumption of the second application, the present application also takes into account the redundant resources required by the second application when determining the recommended amount of resources for the second application, thereby avoiding insufficient resource allocation, that is, avoiding the situation of under- or over-allocation of resources, ensuring the rationality of resource allocation, making the application's resource utilization rate higher, and improving the rationality of resource allocation.

[0007] In a second aspect, an embodiment of the present application provides a data processing device, in which a first application is deployed, and the first application is the same as a second application deployed in a second server; the device includes: a playback module, used to play back the first traffic data through the first application at a preset playback rate to obtain a first resource consumption of the first application, the first traffic data including a request of the second application and response data for feedback on the request, and the preset playback rate is related to the expected business access volume; a determination module, used to determine the recommended resource amount of the second application based on the first resource consumption and the amount of redundant resources required by the second application.

[0008] In a third aspect, an embodiment of the present application provides a data processing device, comprising: at least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the method described in the first aspect by executing the instructions stored in the memory.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes the method described in the first aspect.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed on a computer, enables the method described in the first aspect to be implemented.

[0011] Regarding the beneficial effects of the second to fifth aspects, reference can be made to the contents described in the first aspect above, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of an application scenario of a data processing method provided in an embodiment of the present application;

[0013] Figure 2 A schematic diagram of a system architecture of a data processing method provided in an embodiment of the present application;

[0014] Figure 3A flowchart of a data processing method provided in an embodiment of the present application;

[0015] Figure 4 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0016] Figure 5 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0018] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0019] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0020] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0021] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0022] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0023] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

[0024] In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology related to the embodiments of the present application is first introduced here.

[0025] The current resource allocation method is that when the operation and maintenance personnel monitor that the resource allocation of a certain application is unreasonable, they adjust the resource allocation of the application based on the resource consumption data of the application and work experience. Or the historical resource consumption data of the application is counted, and the recommended resource amount of the application is inferred based on the historical resource consumption data. It can be seen that the above two resource allocation methods are both a kind of speculation based on historical resource consumption data, and the recommended resource amount inferred based on the historical resource consumption data can only reflect the resource consumption under the historical business access volume. In view of the current rapidly changing business environment, the recommended resource amount obtained by the above method cannot well reflect the actual resource consumption under the real-time changing business access volume. For example, in some time periods, the business access volume of the application will increase or decrease suddenly. If the recommended resource amount is determined for the application based on the above resource allocation method, it is easy to cause insufficient or excessive resource allocation for the application.

[0026] In view of this, an embodiment of the present application provides a data processing method, which is applied to a first server, in which a first application is deployed, and the first application is the same as the second application in the second server; the method obtains the first resource consumption of the first application by replaying the first traffic data recorded in the second application at a preset playback rate in the first application, and since the preset playback rate is related to the expected business access volume, the first resource consumption can reflect the resource consumption of the second application under the expected business access volume. Further, considering that in an actual business environment, there is a certain error between the actual resource consumption of the second application and the first resource consumption obtained by replaying the first application, therefore, the embodiment of the present application determines the resource recommendation of the second application based on the first resource consumption and the redundant resource amount required by the second application. That is to say, in the resource allocation method provided by the embodiment of the present application, both the resource consumption of the second application under the expected business access volume is truly simulated, and the redundant resource amount required by the second application is taken into account, so that the determined resource recommendation of the second application is more reasonable, that is, the rationality of resource allocation is improved.

[0027] In order to better understand the embodiments of the present application, the following describes the application scenarios applicable to the embodiments of the present application. Figure 1 , is a schematic diagram of an application scenario of a data processing method provided in an embodiment of the present application. Figure 1 As shown, the scenario diagram includes a first server 110 and a second server 120. The first server 110 and the second server 120 can perform wired or wireless communication.

[0028] The first server 110 and the second server 120 are both servers that can deploy applications. Specifically, the first application can be deployed in the first server 110, and the second application can be deployed in the second server 120, and the first application is the same as the second application. The second server is a server running in a real business environment, and the second application performs business processing operations in the second server, and specifically, the second application can receive business access data from the outside in real time and process the business access data.

[0029] In order to ensure that the first application in the first server 110 can truly simulate the resource consumption of the second application, the same operating environment can be configured for the first application and the second application. Exemplarily, a verification environment can be built in the first server 110, and the first application can be deployed in the verification environment. A production environment can be built in the second server 120, and the second application can be deployed in the production environment. Among them, the verification environment is the same as the production environment, and the same here can refer to the same software and hardware as the production environment deployed in the verification environment. The production environment can also be called the release environment, which refers to an environment that formally provides services to the outside world, or an environment accessed by real users, while the verification environment does not provide services to the outside world. Simply put, the production environment has real traffic access data, and the traffic access data in the verification environment comes from the first traffic data of the second application.

[0030] The first server 110 and the second server 120 can be any devices in the two clusters, such as the first server 110 is any server in the first cluster, and the second server 120 is any server in the second cluster, wherein the cluster can refer to a cluster in a container cloud platform, such as Kubernetes, and the cluster can also refer to a virtual machine cluster, which is not limited in the embodiments of the present application.

[0031] That is to say, the data processing method provided in the embodiment of the present application can be applied to scenarios such as container cloud platforms, virtual machine clusters, and physical machines. In the scenario of physical machines, it is only necessary to ensure that the verification environment and the production environment are deployed in two different devices, for example, Figure 1 A first server 110 and a second server 120 are shown.

[0032] Exemplarily, the second server 120 may record the traffic of the second application's request and the response data fed back to the request within a preset time period, obtain the first traffic data, and send the first traffic data to the first server 110. The first server 110 may replay the first traffic data at a preset playback rate through the first application to obtain the first resource consumption of the first application. Furthermore, the first server 110 may determine the recommended amount of resources for the second application based on the first resource consumption and the amount of redundant resources required for the second application. The first server 110 may send the recommended amount of resources for the second application to the second server 120, and the second server 120 may adjust the resources of the second application based on the recommended amount of resources for the second application. Among them, the specific manner in which the first server determines the recommended amount of resources for the second application based on the first resource consumption and the amount of redundant resources required for the second application will be described in detail below.

[0033] It should be noted that the data processing method provided in the present application can be applied to the scenario of resource allocation. In this scenario, there are many cases of increased resource consumption. Therefore, the embodiment of the present application uses the case of increased resource consumption as an example, but the data processing method provided in the present application is not limited to the case of resource growth, and can also be applied to other similar scenarios. The embodiment of the present application does not limit this.

[0034] Please refer to Figure 2 , is a flow chart of a data processing method provided in an embodiment of the present application. Figure 2 The steps shown are explained. Figure 2 The embodiment shown is applicable to Figure 1 In the application scenario shown. Figure 2 The first server involved may be, for example, Figure 1 The first server 110 shown, Figure 2 The first application involved may be, for example, Figure 1 A first application deployed in the first server 110, Figure 2 The second server involved may be, for example, Figure 1 The second server 120 shown, Figure 2 The second application involved may be, for example, Figure 1 The second application deployed in the second server 120.

[0035] S201, replaying first traffic data at a preset playback rate through a first application to obtain a first resource consumption of the first application, the first traffic data including a request of a second application and response data to the request feedback, and the preset playback rate is related to the expected business access volume.

[0036] The first traffic data may be obtained by the first server from the second server. For example, the second server records the request of the second application and the response data fed back to the request to obtain the first traffic data, and sends the first traffic data to the first server.

[0037] The preset playback rate can be determined based on the expected business access volume, specifically based on the query rate per second (QPS) of the expected business access volume in a unit time. The expected business access volume refers to the number (number of times) of users accessing the second application in a future time period. The expected business access volume can be obtained by the second application user or the second application operator by predicting the access volume of the second application in the future time period. The future time period can be one day, one week, two weeks, etc., and the embodiment of the present application does not limit this. The following is a detailed description of the method for determining the preset playback rate based on the expected business access volume.

[0038] When replaying the first traffic data through the first application, in order to more realistically simulate the resource consumption of the second application under the expected business access volume, the first server needs to adjust the playback rate of the first traffic data so that the adjusted playback rate can reflect the resource consumption of the second application under the expected business access volume. For example, the first traffic data indicates that the query rate per second per unit time during the recording period is 1000, and the expected business access volume indicates that the query rate per second per unit time is 2000. When replaying the first traffic data, the query rate per second per unit time of the first traffic data during the playback period needs to be adjusted to 2000. Adjusting the query rate per second per unit time can be specifically achieved by adjusting the playback period of the first traffic data. Continuing with the above example, if the recording duration of the first traffic data is 10 minutes, and its query rate per second per unit time is 1000, the query rate per second per unit time during the playback period needs to be set to 2000, then the first traffic data can be played back at a 2x rate, that is, the playback of the first traffic data is completed within 5 minutes.

[0039] In a possible implementation, in order to improve the efficiency of the first server in determining the preset playback rate, the first server may pre-store a correspondence between the expected service access volume and the preset playback rate. For example, if the expected service access volume is 4000, the preset playback rate is set to 4, so that the first server plays back the first traffic data at 4 times the rate. It should be noted that, since there is an error between the expected service access volume and the actual service access volume, the expected service access volume in the embodiment of the present application refers to the maximum expected service access volume within the range of variation of the expected service access volume.

[0040] The first server plays back the first traffic data through the first application at a preset playback rate, and obtains the first resource consumption of the first application during the playback. Among them, playback can specifically refer to the process in which the first application performs the same business processing operation as the second application on the first traffic data. The first resource consumption can refer to the maximum resource consumption, average resource consumption or minimum resource consumption per unit time of the first application in the process of playing back the first traffic data, which can be set according to actual needs, and the embodiment of the present application does not limit this. The resources involved in the embodiment of the present application may include computing resources, memory resources and other resources, and the computing resources may specifically be, for example, central processing unit (CPU) resources.

[0041] Since the first traffic data is the actual traffic of the second application during business processing, the first resource consumption can also reflect the resource consumption of the second application that is relatively close to the actual situation under the expected business access volume.

[0042] S202: Determine a recommended amount of resources for the second application according to the first resource consumption and the amount of redundant resources required by the second application.

[0043] Since there is still a certain error between the first resource consumption obtained by replaying the expected business access volume and the actual resource consumption, or when a second application fails, a large amount of resources need to be consumed, it is necessary to increase a certain amount of redundant resources based on the first resource consumption. The redundant resource amount required for the second application can be pre-configured in the first server, or calculated by the first server based on a pre-configured redundant index. The following describes the method in which the first server calculates the redundant resource amount required for the second application based on a pre-configured redundant index.

[0044] The first server may determine the resource consumption change rate according to the preset playback rate, the first resource consumption, and the fifth resource consumption, wherein the resource consumption change rate is used to represent the resource consumption of the first application in replaying the first traffic data per unit time, relative to the resource consumption of the second application in recording the first traffic data per unit time. The fifth resource consumption is the maximum resource consumption of the second application in the period of recording the first traffic data per unit time. Alternatively, the fifth resource consumption may also be the average resource consumption or the minimum resource consumption of the second application in the period of recording the first traffic data per unit time.

[0045] It should be understood that when the first resource consumption is the maximum resource consumption when the first application plays back the first traffic data, the fifth resource consumption corresponds to the maximum resource consumption of the second application during the recording of the first traffic data. The embodiment of the present application is described with the first resource consumption being the maximum resource consumption when the first application plays back the first traffic data.

[0046] For different businesses, the resources required to process business access volume are different. For example, when the business access volume doubles (the traffic doubles), some applications consume less resources when processing the traffic data after the traffic doubles. However, some applications consume more resources when processing the traffic data after the traffic doubles. For example, the business characteristics of some applications require a large amount of memory resources. Therefore, when calculating the rate of change of resource consumption, the factor of traffic change also needs to be considered. Among them, the traffic change is used to represent the traffic of the first application playing back the first traffic data in unit time, relative to the change in the traffic of the second application recording the first traffic data in unit time. The traffic in unit time can also be specifically reflected in the parameter of query rate per second QPS.

[0047] When the expected business access volume shows an increasing trend, the traffic change should also increase, that is, the traffic change is positively correlated with the expected business access volume. However, since the expected business access volume is predicted by business personnel or staff based on work experience, the accuracy of the expected business access volume is low. Therefore, in order to reduce the error impact of the expected business access volume on the traffic change, the embodiment of the present application considers using a logarithmic function to represent the relationship between the traffic change and the preset playback rate, wherein the preset playback rate is related to the expected business access volume. The logarithmic function has a slower growth trend than the exponential function. Therefore, even if the accuracy of the expected business access volume is low, the error of the traffic change calculated based on the logarithmic function is also small. Specifically, the first server can determine the traffic change based on the preset playback rate and the pre-stored first formula.

[0048] Exemplarily, the first formula is as follows.

[0049] T=log2 R

[0050] Wherein, T represents the flow change, and R represents the preset playback rate.

[0051] Taking the preset playback rate as 4 times the rate as an example, T=log24=2.

[0052] The first server may determine a difference between the first resource consumption and the fifth resource consumption, and determine a resource consumption change rate according to the difference between the first resource consumption and the fifth resource consumption and a traffic change amount.

[0053] Exemplarily, a calculation formula for the resource consumption change rate is as follows:

[0054] Resource consumption change rate = (first resource consumption - fifth resource consumption) / flow change / fifth resource consumption + 1

[0055] Based on the above formula, it can be seen that the resource consumption change rate can represent the rate of change of resource consumption during playback within a unit time relative to the recording period within a unit time under the influence of the resource growth and traffic change during the playback period within a unit time relative to the recording period within a unit time.

[0056] Due to the different computing capabilities of each application, the required redundant resources are different. For example, taking CPU resources as an example, the CPU computing intensity of each application is different, resulting in different resource consumption of different applications under the same QPS. Under the same QPS, the greater the rate of change of resource consumption, the higher the computing intensity is considered. Therefore, the redundant resources required for each application can be determined based on a fixed redundancy index and the degree of computing resource dependence of each application. The redundancy index is the redundancy quota uniformly configured by the first server for each application. The redundancy index can be set according to actual needs, and the embodiment of the present application does not limit this. The computing resource dependence of each application can be characterized as the rate of change of resource consumption. Therefore, the redundant resources required for the second application can be calculated based on the rate of change of resource consumption and the preset redundancy index. Specifically, the first server can determine the amount of redundant resources required for the second application based on the first resource consumption, the rate of change of resource consumption and the pre-stored redundancy index. Exemplarily, a calculation formula for determining the amount of redundant resources required for the second application is as follows.

[0057] Redundant resource amount = first resource consumption amount * resource consumption change rate * redundancy index - first resource consumption amount

[0058] Furthermore, the first server may determine the recommended amount of resources for the second application based on the first resource consumption and the amount of redundant resources required by the second application. Specifically, the first server may determine the sum of the first resource consumption and the amount of redundant resources as the recommended amount of resources for the second application. Exemplarily, a calculation formula for the recommended amount of resources for the second application is as follows.

[0059] Recommended resource amount for the second application = first resource consumption + redundant resource amount required by the second application

[0060] Based on the aforementioned calculation formula for the amount of redundant resources, a modified calculation formula for the recommended amount of resources for the second application is as follows.

[0061] Recommended resource amount for the second application = first resource consumption amount * resource consumption change rate * redundancy index

[0062] In actual applications, the resource recommendation amount for the second application calculated by the first server may not be an integer, but the resource adjustment amount is small and has little significance relative to the application. Therefore, when the resource recommendation amount for the second application is not an integer, it is rounded up based on a preset multiple. The preset multiple can be 10, 100, etc., and can be set according to actual needs. The embodiments of the present application do not limit this.

[0063] In order to more clearly understand the specific implementation method of determining the recommended amount of resources required for the second application, the following uses CPU resources as an example to illustrate how the first server determines the recommended amount of resources required for the second application.

[0064] Assume that the first traffic data indicates all request and response data of the second application in the past 7 days, and max(cpu usage)=100 milli-cores in the production environment in the past 7 days, that is, the fifth resource consumption is 100. The preset playback rate is 4, then the max(cpu usage) collected by traffic playback at 4 times the traffic is 150 milli-cores, that is, the first resource consumption is 150. The preset redundancy index is 3.

[0065] Based on the above data, the first server can calculate the resource consumption change rate as (150-100) / 2 / 100+1=1.25, so the first server can calculate the recommended amount of resources required for the second application as 150*1.25*3=562.5. If the preset multiple is 100, the first server rounds up to the multiple of 100, and 562.5 is rounded up to 600, so the first server determines 600 as the recommended amount of resources for the second application.

[0066] In a possible implementation, the first server is also pre-configured with a third resource consumption, and the third resource consumption is the recommended minimum resource consumption required for the operation of the second application, or the third resource consumption may also refer to the minimum resource consumption required for the operation of all applications in the second server. In this implementation, the first server may determine the second resource consumption based on the first resource consumption and the redundant resources required by the second application. When the first server determines that the second resource consumption is less than the third resource consumption, the third resource consumption is determined as the recommended resource for the second application; when the first server determines that the second resource consumption is greater than the third resource consumption, the second resource consumption is determined as the recommended resource for the second application. Among them, the specific calculation method of the second resource consumption can correspond to the calculation method of the first server directly determining the recommended resource for the second application based on the first resource consumption and the redundant resources mentioned above, which will not be repeated here.

[0067] It should be understood that when the second resource consumption is the same as the third resource consumption, the first server may determine either the second resource consumption or the third resource consumption as the recommended resource amount for the second application.

[0068] Since the third resource consumption is the minimum resource consumption required for the second application to run, when the second resource consumption is less than the third resource consumption, the first server will use the third resource consumption to determine the recommended resource amount for the second application to ensure that the second application can run normally.

[0069] Exemplarily, taking the resource as CPU resource, the second resource consumption as 400, and the third resource consumption as 500, the first server determines that the second resource consumption is less than the third resource consumption, and then determines the third resource consumption of 500 as the recommended resource amount for the second application.

[0070] In a possible implementation, the first server is also pre-configured with a fourth resource consumption, which is the recommended resource consumption when the resource utilization of the second server reaches the target resource utilization. For example, the user wants to increase the resource utilization of the second server from 20% to 30%. The way to increase the resource utilization can be to deploy more applications in the second server, or to scale down the applications in the second server, which can be determined based on actual conditions. At this time, the fourth resource consumption is the amount of resources that the second application needs to consume when the resource utilization is 30%.

[0071] The fourth resource consumption can be calculated based on the current resource allocation of the second application and the target shrinkage ratio, where the target shrinkage ratio refers to the shrinkage ratio required to increase the resource utilization to the target resource utilization. A method for calculating the fourth resource consumption is as follows:

[0072] Fourth, resource consumption = current resource allocation * target reduction ratio

[0073] In this implementation, the first server may determine the largest one among the second resource consumption, the third resource consumption, and the fourth resource consumption as the recommended resource amount for the second application. That is, when the first server determines the third resource consumption as the recommended resource amount for the second application, the fourth resource consumption should be less than the third resource consumption; when the first server determines the second resource consumption as the recommended resource amount for the second application, the fourth resource consumption should be less than the second resource consumption. When the first server determines the fourth resource consumption as the recommended resource amount for the second application, the fourth resource consumption should be greater than the second resource consumption and the third resource consumption.

[0074] For example, taking the resource as CPU resource, the second resource consumption as 700, the third resource consumption as 500 and the fourth resource consumption as 600, the first server may determine that 700>600>500, therefore, the first server may determine the second resource consumption as 700 as the recommended resource amount required for the second application.

[0075] After the first server determines the recommended amount of resources required by the second application, it can send the recommended amount of resources for the second application to the second server. Correspondingly, the second server receives the recommended amount of resources for the second application from the first server and adjusts the resources of the second application based on the recommended amount of resources.

[0076] In a possible implementation, after the first server determines the recommended amount of resources required by the second application, the recommended amount of resources required by the second application may be displayed on a corresponding display interface so that operation and maintenance personnel can view the recommended amount of resources required by the second application.

[0077] Combine the following Figure 3 The system structure diagram of a data processing method shown in FIG. 1 is used to illustrate an example of a data processing method applicable to an embodiment of the present application. Figure 3 The first server 310 shown is, for example, Figure 1 The first server 110 shown, Figure 3 The second server 320 shown is, for example, Figure 1 A second server 120 is shown.

[0078] like Figure 3 As shown, a first application 311 and a third application 312 are deployed in the verification environment of the first server 310, as well as a business database 315 for storing the business data of the first application 311 and a business database 316 for storing the business data of the third application 312. A monitoring system 313 is also deployed in the first server 310, and the monitoring system 313 is used to monitor the resource consumption of the first application 311 and the third application 312 in the verification environment. The monitoring system 313 is, for example, promethus. The monitoring system 313 can implement the function of obtaining the first resource consumption in the previous step S201. A resource recommendation system 314 is also deployed in the first server, and the resource recommendation system 314 is used to determine the resource recommendation amount of the second application 321 and the fourth application 322 in the second server 320 according to the resource consumption in the monitoring system 313. Accordingly, the resource recommendation system 314 can correspondingly implement the function of the previous step S202.

[0079] The second application 321 and the fourth application 322 are deployed in the production environment of the second server 320, as well as a business database 326 for storing the business data of the second application 321 and a business database 327 for storing the business data of the fourth application 322. The second server 320 is also deployed with a traffic recording tool 323 and a traffic recording management system 325. The traffic recording tool 323 is used to record the traffic data of the second application 321 and the fourth application 322 under the control of the traffic recording management system 325, and store the recorded traffic data in the traffic recording database 328. Among them, the traffic recording tool 323 can be any open source traffic recording tool, such as repeater. The traffic recording tool 323 and the traffic recording management system 325 can correspondingly implement the function of the second server recording and obtaining the first traffic data in the above text. The second server 320 also includes a monitoring system 324 for monitoring the resource consumption of the second application 321 and the fourth application 322 in the production environment. The monitoring system 324 is, for example, promethus. Accordingly, the monitoring system 324 can implement the function of obtaining the fifth resource consumption in the above text.

[0080] The first application 311 is the same as the second application 321, and the third application 312 is the same as the fourth application 322. It should be noted that: Figure 3 The example is that the second application 321 and the fourth application 322 are deployed in the production environment, but in actual applications, one or more applications can be deployed in the production environment. The embodiment of the present application does not limit the number of applications. Accordingly, the same applications as those in the production environment should be deployed in the verification environment. The monitoring system 313 in the first server 310 and the monitoring system 324 in the second server 320 can be the same or different, and the embodiment of the present application does not limit this.

[0081] Exemplarily, the data processing method provided in the embodiment of the present application is illustrated by taking the first application 311 and the second application 321 as examples.

[0082] The traffic recording management system 325 sends traffic recording indication information to the traffic recording tool 323. The traffic recording indication information may be automatically generated in the traffic recording management system 325 at intervals of a preset time period, or after the user inputs recording information based on business needs, the traffic recording management system 325 generates the traffic recording indication information based on the recording information. The business needs may be, for example, to indicate that the resources of the second application 321 should be readjusted, or when the monitoring system 324 detects that the query rate per second in the second application 321 has exceeded the maximum value within a preset period, the traffic recording is triggered. The traffic recording indication information may indicate the recording object and the recording time period. For example, the traffic recording indication information may indicate that the traffic of the second application is recorded within a preset time period.

[0083] Based on the traffic recording indication information, the traffic recording tool 323 starts the traffic recording of the second application within the preset time period, obtains the first traffic data, and stores the first traffic data in the traffic recording database 328. After the recording is completed, the second server 320 sends the first traffic data to the first server 310, so that the resource recommendation system 314 plays back the first traffic data through the first application at a preset playback rate. After the playback is completed, the resource recommendation system 314 obtains the first resource consumption of the first application 311 during the playback process from the monitoring system 313, and determines the resource recommendation amount of the second application based on the first resource consumption and the redundant resource amount required by the pre-stored second application. Among them, the specific method for the first server 110 to determine the resource recommendation amount of the second application based on the first resource consumption and the redundant resource amount required by the pre-stored second application can be referred to the content described above, and will not be repeated here.

[0084] In a possible implementation, in order to ensure data consistency between the first application 311 and the second application 321, when the traffic recording tool 323 does not start traffic recording, the data in the business database 326 needs to be synchronized to the business database 315 in real time. When the traffic recording tool 313 starts traffic recording of the second application 321, it is necessary to stop the synchronization of business data between the business database 326 and the business database 315, and send an indication message to the first server 310 to indicate that the second application 326 has started traffic recording. After receiving the indication message, the first server 310 adjusts the business database 315 to a read-only state to avoid changes in the data in the business database 315.

[0085] Since the first application 321 needs to replay the first traffic data, all request and response data in the first traffic data need to be re-executed in the first application 311 for playback. Therefore, stopping the data synchronization between the business database 326 and the business database 315 when starting recording can ensure that when the first application 311 subsequently replays based on the first traffic data, it can execute business processing logic consistent with the second application 321, so as to ensure the accuracy of the resource consumption of the first application 311 simulating the second application 321.

[0086] Based on the same inventive concept, an embodiment of the present application provides a data processing device, which is used to implement any of the above data processing methods, for example Figure 2 The data processing method shown, and the device can also realize the function of the first server mentioned above.

[0087] Please refer to Figure 4 , is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Figure 4As shown, the data processing device 400 includes a playback module 401 and a determination module 402. A first application may be deployed in the data processing device 400, and the first application is the same as a second application deployed in the second server.

[0088] Exemplarily, a playback module 401 is used to play back first traffic data through a first application at a preset playback rate to obtain a first resource consumption of the first application, where the first traffic data includes a request from a second application and response data for feedback on the request, and the preset playback rate is related to the expected business access volume; a determination module 402 is used to determine the recommended resource amount for the second application based on the first resource consumption and the amount of redundant resources required for the second application.

[0089] In one possible implementation, determination module 402 is specifically used to: determine the second resource consumption based on the first resource consumption and the redundant resource amount required by the second application; if the second resource consumption is less than the third resource consumption, determine the third resource consumption as the recommended resource amount for the second application, and the third resource consumption is the recommended minimum resource consumption required for the operation of the second application; if the second resource consumption is greater than the third resource consumption, determine the second resource consumption as the recommended resource amount for the second application.

[0090] In one possible implementation, when the recommended resource amount for the second application is the third resource consumption, the fourth resource consumption is less than the third resource consumption, and the fourth resource consumption is the resource consumption when the recommended resource utilization of the second server reaches the target resource utilization; when the recommended resource amount for the second application is the second resource consumption, the second resource consumption is greater than the fourth resource consumption.

[0091] In a possible implementation, the determination module 402 is further configured to: if the fourth resource consumption is greater than the second resource consumption and the third resource consumption, determine the fourth resource consumption as the recommended resource amount for the second application.

[0092] In one possible implementation, the determination module 402 is also used to: determine the resource consumption change rate based on a preset playback rate, a first resource consumption, and a fifth resource consumption, the resource consumption change rate being used to represent the rate of change of the resource consumption of the first application in replaying the first traffic data per unit time relative to the resource consumption of the second application in recording the first traffic data per unit time; determine the amount of redundant resources required for the second application based on the resource consumption change rate.

[0093] In one possible implementation, the determination module 402 is specifically used to: determine a traffic change amount based on a preset playback rate, where the traffic change amount is used to represent the change in traffic of the first application in playing back the first traffic data per unit time relative to the change in traffic of the second application in recording the first traffic data per unit time; determine the resource consumption change rate based on the difference between the first resource consumption and the fifth resource consumption, and the traffic change amount.

[0094] Based on the same inventive concept, an embodiment of the present application provides a data processing device, which is used to implement any of the above data processing methods, for example Figure 2 The data processing method shown, and the device can also realize the function of the first server mentioned above.

[0095] Please refer to Figure 5 , is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Figure 5 As shown, the data processing device 500 includes: at least one processor 501 , and a memory 502 communicatively connected to the at least one processor 501 .

[0096] The processor 501 may be a general-purpose processor or a dedicated processor. The processor 501 may include, for example, a baseband processor or a central processing unit. The baseband processor may be used to process communication protocols and communication data. The central processing unit may be used to control the data processing device 500, execute software programs and / or process data. Different processors may be independent devices or may be arranged in one or more processing circuits, for example, integrated in one or more dedicated integrated circuits.

[0097] In one embodiment, the memory 502 stores instructions that can be executed by at least one processor 501. The at least one processor 501 implements the functions of the first server as described above by executing the instructions stored in the memory 502. Accordingly, the steps performed by the first server as described above can also be implemented.

[0098] In this embodiment, the data processing device 500 can also implement the functions of the previous text data processing apparatus 400 , and at least one processor 501 in the data processing device 500 can also implement the functions of the previous text playback module 401 and the determination module 402 .

[0099] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes any of the data processing methods described above, for example, Figure 2 The data processing method shown.

[0100] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed on a computer, enables any of the above-mentioned data processing methods, for example, Figure 2 The data processing method shown is implemented.

[0101] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0102] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0103] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0105] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method, characterized in that: Applied to a first server, a first application is deployed in the first server, and the first application is the same as a second application deployed in a second server, the method comprising: Replaying the first traffic data at a preset playback rate by the first application to obtain a first resource consumption of the first application, wherein the first traffic data includes a request of the second application and response data fed back to the request, and the preset playback rate is related to an expected service access volume; A recommended amount of resources for the second application is determined according to the first resource consumption and the amount of redundant resources required by the second application.

2. The method according to claim 1, characterized in that The determining the recommended amount of resources for the second application according to the first resource consumption and the amount of redundant resources required by the second application includes: determining a second resource consumption according to the first resource consumption and the redundant resource amount required by the second application; If the second resource consumption is less than the third resource consumption, determining the third resource consumption as the recommended resource consumption of the second application, the third resource consumption being the recommended minimum resource consumption required for the second application to run; If the second resource consumption is greater than the third resource consumption, the second resource consumption is determined as the recommended resource amount for the second application.

3. The method according to claim 2, characterized in that In a case where the recommended resource amount of the second application is the third resource consumption, the fourth resource consumption is less than the third resource consumption, and the fourth resource consumption is the resource consumption when the resource utilization rate of the second server recommended reaches the target resource utilization rate; When the recommended resource amount of the second application is the second resource consumption amount, the second resource consumption amount is greater than the fourth resource consumption amount.

4. The method according to claim 3, characterized in that The method further comprises: If the fourth resource consumption is greater than the second resource consumption and the third resource consumption, the fourth resource consumption is determined as the recommended resource amount of the second application.

5. The method according to claim 1, characterized in that The method further comprises: Determine a resource consumption change rate according to the preset playback rate, the first resource consumption, and the fifth resource consumption, wherein the resource consumption change rate is used to represent a change rate of the resource consumption of the first application in playing back the first traffic data within a unit time relative to the resource consumption of the second application in recording the first traffic data within a unit time; The amount of redundant resources required by the second application is determined according to the resource consumption change rate.

6. The method according to claim 5, characterized in that The determining the resource consumption change rate according to the preset playback rate, the first resource consumption, and the fifth resource consumption includes: Determine a flow rate change according to the preset playback rate, where the flow rate change is used to represent a change in the flow rate of the first application playing back the first flow data in a unit time relative to the flow rate of the second application recording the first flow data in a unit time; The resource consumption change rate is determined according to a difference between the first resource consumption and the fifth resource consumption, and the flow change.

7. A data processing device, characterized in that: The data processing device has a first application deployed therein, the first application being the same as a second application deployed in a second server; the device comprising: a playback module, configured to play back the first traffic data at a preset playback rate through the first application to obtain a first resource consumption of the first application, wherein the first traffic data includes a request of the second application and response data fed back to the request, and the preset playback rate is related to an expected service access volume; A determination module is used to determine a recommended amount of resources for the second application based on the first resource consumption and the amount of redundant resources required by the second application.

8. A data processing device, characterized in that: include: at least one processor, and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the method according to any one of claims 1 to 6 by executing the instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises computer instructions, which, when executed on a computer, enable the method according to any one of claims 1 to 6 to be implemented.

Images